pick a SAM device the GPU can actually run in objects_3d - #483
Open
salmanmkc wants to merge 8 commits into
Open
Conversation
SlimSAM's attention buffer needs 384 MB at fp16, but a GPU reporting the WebGPU default 128 MB storage buffer binding limit cannot bind it, and onnxruntime already requests the adapter maximum. Probe the adapter for shader-f16 and the binding limit before choosing a backend, then try webgpu fp16, webgpu fp32 and cpu in turn so the same model still loads. Also stop caching a rejected load promise, which previously made every later Detect press fail until the page was reloaded.
Contributor
Author
|
@dli7319 does this fix the GPU errors in the console for you? |
Collaborator
Contributor
Author
Routing those devices to wasm made things worse: onnxruntime's wasm backend runs synchronously on the main thread and stalled the render loop for over a minute, which reads as a frozen app on a headset. The oversized binding only produces recoverable WebGPU validation warnings, and detection still completes on the GPU, so warn about the limit and carry on instead of switching backend. Shrinking the input to fit the limit is not an option either: the exported SlimSAM graph fixes pixel_values at 1024x1024.
Collaborator
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



on a GPU that reports the WebGPU default
maxStorageBufferBindingSizeof 128 MB, Detect spams uncaught validation errors during "fitting boxes…" and takes a very long time to produce anything:that 402653184 is SlimSAM's attention buffer: 4096 tokens over 12 heads at fp16 is exactly 384 MB, 3x over the cap. onnxruntime already requests the adapter maximum, so it can't be raised — the device genuinely won't go higher. machines with a big limit (mine reports 4 GB) never see it, which is why it reproduces for some people and not others.
the device pick was
'gpu' in navigator && navigator.gpu ? 'webgpu' : 'wasm'with fp16 whenever webgpu was chosen, and the comment claimed transformers.js downgrades transparently. it doesn't whendeviceis passed explicitly — it throws, and separately fp16 needs theshader-f16feature that plenty of mobile/XR GPUs don't expose.so probe the adapter and check both things before choosing, then try candidates in order:
webgpu/fp16→webgpu/fp32→wasm/fp32. same model every time, so masks don't change quality depending on the device — only where it runs. cpu stays last so a driver failure the capability check didn't predict still lands somewhere that works.also stops caching a rejected
_samPromise.getSam()hands back the cached promise, so one failed load used to make every later Detect press fail until reload.simulating the 128 MB limit locally: before, uncaught WebGPU errors and no clean completion; after, it routes to cpu and fits boxes with an empty console. an unmodified adapter is unaffected and still runs webgpu/fp16.
?mask=segmenteris untouched — still an explicit choice, never automatic.same one-liner exists in the objects3d addon on #417, fixed separately there.