Repository navigation
vello_gpu: Fix methods for bilinear and bicubic image sampling - #1963
Conversation
8ae8442 to
690d88d
Compare
690d88d to
93e02c4
Compare
| @@ -1,5 +1,6 @@ | |||
| { | |||
| "timeouts": { | |||
| "pageLoad": 1200000, | |||
There was a problem hiding this comment.
Had to add this to address another timeout failure resulting from our newly added tests: https://github.com/linebender/vello/actions/runs/36677557384/job/109765766086
f50d0ce to
4ee3473
Compare
There was a problem hiding this comment.
The border in the new snapshot is expected. The test uses bilinear sampling with repeat. This means that for example, for the top-right part, as we approach the right border of the yellow pixel, it should slowly start to fade into the red of the top-left pixel, hence why it becomes orange.
4ee3473 to
0fa8fe9
Compare
0fa8fe9 to
76aed20
Compare
217da5b to
d129a87
Compare
| let x = vec2<i32>( | ||
| external_axis_index(base_coords.x - 1.0, image_extend_modes.x, image_size.x), | ||
| external_axis_index(base_coords.x, image_extend_modes.x, image_size.x), | ||
| ); | ||
| let y = vec2<i32>( | ||
| external_axis_index(base_coords.y - 1.0, image_extend_modes.y, image_size.y), | ||
| external_axis_index(base_coords.y, image_extend_modes.y, image_size.y), | ||
| ); |
There was a problem hiding this comment.
Did you try a dedicated fast path for Pad/Pad? I'm wondering if some of the 73 → 66 FPS comes from the switch in extend_mode now running once per tap (4× for bilinear, 8× for bicubic) instead of twice per pixel. If the mobile compiler flattens it, Pad images would also pay for the repeat/reflect divisions. A single if all(image_extend_modes == vec2(EXTEND_PAD)) that just clamps the tap indices would sidestep that without changing the output, and it could also be where the hardware-sampling path from #1964 goes later.
There was a problem hiding this comment.
Yeah seems to help a bit! I'll add this for now then, but unsure if it's worth the additional complexity, especially if we are going to lean in fully on the native sampling path.
However, not sure I fully understand the second comment, I don't think the native sampling path can go in there, because it has other simplifications as well, so it will probably have to stay a separate path.
| let x = vec4<i32>( | ||
| external_axis_index(base_coords.x - 2.0, image_extend_modes.x, image_size.x), | ||
| external_axis_index(base_coords.x - 1.0, image_extend_modes.x, image_size.x), | ||
| external_axis_index(base_coords.x, image_extend_modes.x, image_size.x), | ||
| external_axis_index(base_coords.x + 1.0, image_extend_modes.x, image_size.x), | ||
| ); | ||
| let y = vec4<i32>( | ||
| external_axis_index(base_coords.y - 2.0, image_extend_modes.y, image_size.y), | ||
| external_axis_index(base_coords.y - 1.0, image_extend_modes.y, image_size.y), | ||
| external_axis_index(base_coords.y, image_extend_modes.y, image_size.y), | ||
| external_axis_index(base_coords.y + 1.0, image_extend_modes.y, image_size.y), | ||
| ); |
There was a problem hiding this comment.
Did you try extending only the first tap on each axis and stepping to the others? The taps are consecutive integers, so after one euclid_mod each neighbour is just +1 with a single conditional wrap. That would cut the divisions from 8 to 2 per bicubic pixel and from 4 to 2 for bilinear. It should be bit-identical too, since everything stays exact integer math. The only catch is tiny images, where one wrap isn't enough (repeat with width < 3, reflect with width 1), but those could fall back to the per-tap path.
There was a problem hiding this comment.
It might work, but I think special-casing smaller images adds to much complexity to the shader, so I will keep it as is for now if it's fine with you!
e56f0be to
93c6347
Compare
This PR makes the following changes:
This does unfortunately regress performance: Rendering a full-sized image on my Android phone goes from around 73FPS to 66FPS. See the below videos.
Before:
IMG_1336.MOV
After:
IMG_1337.MOV
But the previous approach was just fundamentally wrong. However, the good news is that with #1964, I will introduce a fast path that uses GPU-native bilinear sampling for images that are sampled from the whole external texture, which will not only undo this slowdown, but in fact make image rendering even faster compared to current main, when using external textures and sampling the whole texture! See that PR for more information. Since rendering whole images is the most common operation (except for glyph caching, but this is experimental right now, anyway. And nearest-neighbor sampling is less affected than bilinear sampling), in my opinion this is a trade-off worth taking.
Using the image atlas will unfortunately stay slower for now, even with #1964, but I think that's something we have to accept for now. Once we revisit glyph caching, we can figure out how to best fix this. We could also improve this in the future by restricting the possible image sampling modes for atlas images (for example, only allowing extend mode
Pad). But the problem is simply that when we sample from an arbitrary subregion of an atlas, we have to emulate correct extension ourselves, which is much slower than letting the hardware do it, so this should be avoided.