Skip to content

[QNN EP] Add UDO reference sample scripts - #757

Merged
qti-chuteng merged 8 commits into
mainfrom
dev/huaychou/udo-sample
Sep 22, 2026
Merged

qti-chuteng merged 8 commits into
mainfrom
dev/huaychou/udo-sample

Conversation

@huaychou

@huaychou huaychou commented Aug 24, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Add a user-facing MyAdd QNN user-defined operation (UDO) reference sample at qcom/samples/qnn_udo_myadd/. The sample demonstrates the complete QNN EP custom-op workflow outside the unit-test binary:

  • Generate fp32 (CPU) and QDQ (HTP) ONNX models, plus deterministic ONNX test data.
  • Build Linux x86_64 CPU and HTP op packages using the canonical MyAdd op-package sources shared with the unit tests. Generated output is isolated under the sample's ignored artifacts/ directory.
  • Run the models through QNN EP using standalone C++ and Python plugin-EP examples.
  • Deploy and run the HTP UDO on Android arm64, including the required ARM registration library and DSP skel arrangement.
  • Generate an external EPContext model and run it without the custom-op schema environment variable.
  • Register the custom ONNX domain through ORT_QNN_CUSTOM_OP_DOMAINS; no manual Ort::CustomOpDomain setup or schema companion library is required.
  • Link the QNN EP UDO documentation to the sample.

Motivation and context

The MyAdd UDO coverage previously lived primarily in unit-test fixtures. This sample provides a copy-runnable developer reference for building, registering, validating, and deploying a QNN UDO across CPU, HTP simulation, and Android HTP execution.

Validation

  • Linux x86_64 CPU: built the op package; C++ and Python samples passed with maximum absolute error 0.00e+00.
  • Linux x86_64 HTP simulator: built the op package; C++ and Python samples passed with maximum absolute error 0.0118 (within the 0.0314 QDQ tolerance).
  • Linux x86_64 gtests: QnnCPUBackendTests.UDO_Op_MyAdd_AutoDomainOnly and QnnHTPBackendTests.UDO_Op_MyAdd passed.
  • Android arm64 HTP: inference passed (Models: 1, Succeeded: 1, Failed: 0).
  • Android arm64 EPContext: external context generation and inference passed without ORT_QNN_CUSTOM_OP_DOMAINS.

@huaychou
huaychou force-pushed the dev/huaychou/udo-sample branch 3 times, most recently from df4d91d to 62d7573 Compare August 26, 2026 07:05
@huaychou
huaychou force-pushed the dev/huaychou/udo-sample branch 4 times, most recently from 751404f to 67b19ec Compare September 10, 2026 08:59
Add a self-contained, copy-runnable MyAdd user-defined operation (UDO)
sample for QNN EP outside the gtest suite. The sample covers model and
test-data generation, CPU/HTP op-package builds, and C++ and Python
inference paths, including on-device usage.

The samples exercise automatic custom-op domain registration through
ORT_QNN_CUSTOM_OP_DOMAINS, so no manual Ort::CustomOpDomain setup or
schema companion library is required.
@yuhuchua-qti

Copy link
Copy Markdown
Collaborator

The PR description references register_myadd_schema.cc, but this file is not part of the PR.
The updated README/sample flow also appears to rely on ORT_QNN_CUSTOM_OP_DOMAINS instead of a schema companion library.
Could you clarify this discrepancy?
Thanks.

@huaychou

Copy link
Copy Markdown
Collaborator Author

The PR description references register_myadd_schema.cc, but this file is not part of the PR. The updated README/sample flow also appears to rely on ORT_QNN_CUSTOM_OP_DOMAINS instead of a schema companion library. Could you clarify this discrepancy? Thanks.

After #770, the manual UDO custom-op domain registration is no longer required, so register_myadd_schema.cc has been removed.
Update the PR description according to the latest changes and validation results.

Comment thread qcom/samples/qnn_udo_myadd/README.md Outdated
Comment thread qcom/samples/qnn_udo_myadd/README.md Outdated
Comment thread qcom/samples/qnn_udo_myadd/README.md Outdated
Comment thread qcom/samples/qnn_udo_myadd/README.md
Comment thread qcom/samples/qnn_udo_myadd/run_udo_sample.cc
Comment thread qcom/samples/qnn_udo_myadd/README.md Outdated
Comment thread qcom/samples/qnn_udo_myadd/README.md Outdated
Comment thread qcom/samples/qnn_udo_myadd/README.md Outdated
Comment thread qcom/samples/qnn_udo_myadd/gen_myadd_test_data.py Outdated
Comment thread qcom/samples/qnn_udo_myadd/README.md
@qti-chuteng
qti-chuteng merged commit 21a718d into main Sep 22, 2026
61 checks passed
@qti-chuteng
qti-chuteng deleted the dev/huaychou/udo-sample branch September 22, 2026 06:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants