SynthRT

SynthRT is a package-oriented runtime for singing voice synthesis. Packages describe the contributions they provide, while category implementations and plugins interpret those declarations and perform the actual work.

Warning

SynthRT is pre-release software. DS Spec 2.4 and the C++ API are still evolving together, so breaking changes currently have no deprecation period.

Repository Contents

• synthrt provides the host runtime. It reads package manifests, resolves dependencies and contribution references, discovers interpreters, and manages package loading.
• dsinfer provides the DiffSinger-facing inference contracts and implementations. Its installed CMake target remains dsinfer::dsinfer.

Runtime Model

Packages and Contributions

• A package is currently an installed directory containing a desc.json manifest.
• A package is identified by its package ID and four-part normalized version.
• The root contributions object groups declarations by contribution category.
• Built-in categories currently include inference and singer.
• Applications may register additional categories before loading their first package.
• A contribution may use an external declaration file or exist entirely as an entry in desc.json, depending on its category.

Contribution locators do not contain package versions. Dependency resolution selects package versions before contribution references are bound.

vendor/sample:inference/acoustic
vendor/sample:singer/main
:singer/main

• vendor/sample:inference/acoustic refers to acoustic in the already-resolved direct dependency vendor/sample.
• :singer/main refers to main in the current package.
• A locator names one contribution. Package dependency requirements use the separate id and version fields in desc.json.

Package Opening

SynthUnit exposes two package opening modes:

1. DataOnly reads package data without discovering or loading plugins and without publishing the package into runtime state.
2. Load performs the full load transaction:
   1. Probe package declarations and discover matching plugins without loading them.
   2. Acquire the required runtime resources.
   3. Validate that every contribution is ready.
   4. Commit the complete package graph atomically.

If any step before Commit fails, the package is not made visible as loaded. The committed package and its resolved plugin choices remain stable for the lifetime of that load.

Plugin Roles

• SingerProviderPlugin creates singer providers.
• InferenceInterpreterPlugin creates interpreters for a requested (interface, level, variant) contract.
• InferenceDriverPlugin creates model execution drivers selected by backend metadata, such as the onnx backend supplied by onnxdriver.
• One plugin bundle may advertise and create more than one supported contract.
• Each plugin category has its own ordered search path sequence.

Plugin bundles contain a shared library and a neighboring plugin.json. Build and installation paths follow this layout:

plugins/
└── dsinfer/
    ├── inferencedrivers/
    │   └── onnx/
    └── inferenceinterpreters/
        ├── acoustic/
        ├── duration/
        ├── pitch/
        ├── variance/
        └── vocoder/

Current Status

The package-loading foundation currently implements:

• DataOnly manifest inspection
• internal Probe and plugin discovery
• dependency selection and contribution locator binding
• Acquire and Ready validation
• atomic Commit visibility
• shared package identities and strong-reference-based release

The following work is still in progress:

• closing the runtime execution and import-binding lifecycle
• migrating the remaining dsinfer utilities and command-line paths to the new typed APIs
• stabilizing the interpreter, singer provider, and inference driver implementations
• implementing .dspk installation separately from directory-based package loading

See Project status for the maintained task list.

Build Requirements

• CMake 3.19 or later
• A C++17-capable compiler
• Ninja or another CMake-supported build tool
• Git, with which vcpkg and its overlays are checked out for the manifest-managed dependencies

Dependencies

The vcpkg manifest supplies the project's library and build dependencies:

• qmsetup
• stdcorelib
• stdcorelib.plugin
• stduuid
• BLAKE3
• sparsepp
• bit7z

Additional dependencies are required for particular targets:

• ONNX Runtime is required to build the ONNX inference driver.
• Boost.Test is required when SYNTHRT_BUILD_TESTS is enabled.

Prepare Dependencies

The dependencies are installed with vcpkg-workspace.cmake, downloaded into scripts/. The script checks out vcpkg and the overlays that scripts/vcpkg-sources.json specifies into deps/, and installs the dependencies there. Run the following commands from the repository root.

curl -fsSL -o scripts/vcpkg-workspace.cmake https://raw.githubusercontent.com/stdware/dev-scripts/main/scripts/vcpkg-workspace.cmake
cmake -P scripts/vcpkg-workspace.cmake install -- --x-feature=tests

On Windows, the script is downloaded in PowerShell or in cmd with

powershell -Command "Invoke-WebRequest -OutFile scripts/vcpkg-workspace.cmake https://raw.githubusercontent.com/stdware/dev-scripts/main/scripts/vcpkg-workspace.cmake"

Omit -- --x-feature=tests if automated tests are not required. cmake -P scripts/vcpkg-workspace.cmake help lists the other commands. --dry-run prints the commands without running them. scripts/vcpkg-sources.local.json takes precedence over scripts/vcpkg-sources.json, for example to use a local overlay.

Prepare ONNX Runtime

Choose one ONNX Runtime distribution and run the corresponding command from the repository root.

1. CPU build:

   cmake -E chdir third-party cmake -P ../scripts/setup-onnxruntime.cmake

2. CUDA 11 build:

   cmake -E chdir third-party cmake -Dep=cuda11 -P ../scripts/setup-onnxruntime.cmake

3. CUDA 12 build:

   cmake -E chdir third-party cmake -Dep=cuda12 -P ../scripts/setup-onnxruntime.cmake

Without a prepared ONNX Runtime distribution, CMake omits the ONNX utility and driver targets while the remaining libraries may still be built.

Build From Source

The following example uses a single-configuration Ninja build:

cmake -S . -B build/Release -G Ninja \
    -DCMAKE_INSTALL_PREFIX=<install-dir> \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_TOOLCHAIN_FILE=deps/vcpkg/scripts/buildsystems/vcpkg.cmake \
    -DSYNTHRT_BUILD_TESTS=ON

cmake --build build/Release --target all
cmake --install build/Release

Consume the Libraries with CMake

Host Runtime Only

find_package(synthrt CONFIG REQUIRED)

add_executable(example main.cpp)
target_link_libraries(example PRIVATE synthrt::synthrt)

DiffSinger Inference Support

find_package(dsinfer CONFIG REQUIRED)

add_executable(example main.cpp)
target_link_libraries(example PRIVATE dsinfer::dsinfer)

Point CMAKE_PREFIX_PATH at the installation prefix when CMake cannot find the packages automatically. The dsinfer library file is named synthrt-dsinfer, while its public CMake target is dsinfer::dsinfer.

Documentation

• DS package specification 2.4
• dsinfer level 1 revised contracts
• Development conventions

License

Apache License 2.0. See LICENSE.