Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 19 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,10 @@ canonical multi-architecture version tag.
CMake options (full table in `docs/getting-started.md`):
- `DAQIRI_ENGINE` — space-separated list of optional engines to compile. Valid values: `dpdk` (raw Ethernet) and `ibverbs` (RDMA/RoCE). Linux sockets (UDP/TCP) are always built in, so there is no `socket` value. Default is `"dpdk ibverbs"`.
- `DAQIRI_BUILD_PYTHON` — builds `pybind11` bindings from `python/`.
- `DAQIRI_BUILD_EXAMPLES` — builds the benchmark executables (default `ON`).
- `DAQIRI_BUILD_EXAMPLES` — builds the benchmark executables (default `ON`). The
hardware-free `daqiri_config_validate` tool is always built and installed.
- `BUILD_TESTING` — builds and registers the hardware-free C++ tests under
`tests/cpp/` with CTest (default `ON`). Set it to `OFF` to omit test targets.
- `DAQIRI_BUILD_APPLICATIONS` — builds the end-to-end example applications under `applications/` (default `OFF`; requires TensorRT, e.g. the `BASE_IMAGE=torch` container). Currently builds `applications/resnet50_inference/` (DAQIRI → TensorRT ResNet inference).
- `DAQIRI_ENABLE_OTEL_METRICS` — enables OpenTelemetry metrics instrumentation (default `OFF`).
- `DAQIRI_REORDER_GPU_PROFILE` — enable CUDA event timing in the DPDK reorder kernels (off by default).
Expand All @@ -45,7 +48,20 @@ python3 -m venv .venv
.venv/bin/python -m pytest
```

The default suite collects only `tests/portable/`. Future build-backed C++ tests live under `tests/cpp/`; Python-binding tests live under `tests/bindings/` and require a container built with `DAQIRI_BUILD_PYTHON=ON`. Platform tests live under `tests/platform/` and are selected by CI/CD jobs running on provisioned GPU/NIC systems; they are never part of the default pytest collection. The project container already includes the current test packages; use the container-specific dependency command in `tests/README.md` when `tests/requirements.txt` changes.
Validate checked-in configurations through the production parser and common semantic checks
without initializing hardware:

```bash
python3 scripts/check_daqiri_configs.py --validator build/tools/daqiri_config_validate
```

The default pytest suite collects only `tests/portable/`. Build-backed C++ tests live under
`tests/cpp/` and run through CTest; Python-binding tests live under `tests/bindings/` and
require a container built with `DAQIRI_BUILD_PYTHON=ON`. Platform tests live under
`tests/platform/` and are selected by CI/CD jobs running on provisioned GPU/NIC systems;
they are never part of the default pytest collection. The project container already includes
the current test packages; use the container-specific dependency command in
`tests/README.md` when `tests/requirements.txt` changes.

Integration and performance verification is done via the benchmark executables in `examples/`, driven by YAML configs. Build outputs (`examples/CMakeLists.txt:59-71`):

Expand Down Expand Up @@ -112,7 +128,7 @@ clang-format -style=file -i -fallback-style=none <files>
### Engine abstraction
`src/engine.h` defines `daqiri::Engine` — an (almost) ABC with ~50 virtual methods covering init, RX/TX burst dequeue/enqueue, header-fill helpers, buffer free, socket connection helpers, runtime TCP/UDP `setsockopt` passthrough, and RDMA connection setup. Engines live in `src/engines/<name>/` (`dpdk/`, `rdma/`, `socket/`, `ibverbs/`). `DAQIRI_ENGINE` selects the optional `dpdk` and `ibverbs` engines at CMake configure time; the `socket` engine is always built. The user-facing value `ibverbs` builds **two** internal engines that both use libibverbs: `rdma` (`src/engines/rdma/`, `DAQIRI_ENGINE_RDMA`, RoCE/InfiniBand for socket `roce://`) and `ibverbs` (`src/engines/ibverbs/`, `DAQIRI_ENGINE_IBVERBS`, the pure-DevX MPRQ raw-Ethernet engine). Each engine produces its own static library (`daqiri_dpdk`, `daqiri_rdma`, `daqiri_socket`, `daqiri_ibverbs`) linked into `daqiri_common`, and each adds a `DAQIRI_ENGINE_<NAME>=1` compile definition.

`EngineType` (`include/daqiri/types.h`) is resolved from `(stream_type, engine)`: `raw` defaults to `EngineType::IBVERBS` when that engine is built (falling back to `EngineType::DPDK` in DPDK-only builds); `raw` + `engine: "dpdk"` explicitly selects `EngineType::DPDK`; `socket` + a `roce://` endpoint (or `engine: "ibverbs"`) selects `EngineType::RDMA`. The stream-aware `config_engine_from_string(str, stream_type)` overload encodes the `ibverbs`→`{IBVERBS for raw, RDMA for socket}` split. `EngineFactory` (in `engine.h`) is a singleton that instantiates the active engine. `daqiri_init(...)` resolves which engine to use from the `NetworkConfig` and then delegates everything through the `Engine` vtable. There is only ever **one** active `Engine` per process.
`EngineType` (`include/daqiri/types.h`) is resolved from `(stream_type, engine)`: `raw` defaults to `EngineType::IBVERBS` when that engine is built (falling back to `EngineType::DPDK` in DPDK-only builds); `raw` + `engine: "dpdk"` explicitly selects `EngineType::DPDK`; `socket` + a `roce://` endpoint (or `engine: "ibverbs"`) selects `EngineType::RDMA`. The stream-aware `config_engine_from_string(str, stream_type)` overload encodes the `ibverbs`→`{IBVERBS for raw, RDMA for socket}` split. `EngineFactory` (in `engine.h`) is a singleton that instantiates the active engine. `daqiri_init(...)` resolves which engine to use from the `NetworkConfig`, runs the shared hardware-independent semantic validation, and only then delegates initialization through the `Engine` vtable. The standalone `daqiri_config_validate` tool calls the same parser and shared checks without creating an engine. There is only ever **one** active `Engine` per process.

The always-built socket engine implements Linux UDP/TCP streams directly. Applications that need kernel socket tuning call `socket_setsockopt(conn_id, level, optname, optval, optlen)` after resolving a TCP/UDP connection ID; DAQIRI passes the numeric Linux constants through without maintaining a symbolic option map. `socket_setsockopt` is not supported for `roce://` connections, which delegate to the RDMA/ibverbs path.

Expand Down
6 changes: 6 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ project(daqiri VERSION ${DAQIRI_PROJECT_VERSION} LANGUAGES C CXX CUDA)

include(GNUInstallDirs)
include(CMakePackageConfigHelpers)
include(CTest)
find_package(CUDAToolkit REQUIRED)

set(DAQIRI_ABI_VERSION "2" CACHE STRING "DAQIRI shared library ABI version")
Expand All @@ -55,6 +56,11 @@ option(DAQIRI_ENABLE_OTEL_METRICS "Enable OpenTelemetry metrics instrumentation"
set(DAQIRI_ENGINE "dpdk ibverbs" CACHE STRING "Optional engine implementations to build: dpdk, ibverbs (Linux sockets are always built in)")

add_subdirectory(src)
add_subdirectory(tools)

if(BUILD_TESTING)
add_subdirectory(tests/cpp)
endif()

if(TARGET yaml-cpp)
install(
Expand Down
8 changes: 7 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,12 +81,18 @@ python3 -m venv .venv

The default invocation collects only `tests/portable/`. Tests importing the compiled
`daqiri` module belong under `tests/bindings/` and require a container built with
`DAQIRI_BUILD_PYTHON=ON`. Future C++ tests belong under `tests/cpp/`. Tests requiring
`DAQIRI_BUILD_PYTHON=ON`. Build-backed C++ tests belong under `tests/cpp/` and run
through CTest. Tests requiring
a GPU, NIC, hugepages, privileged access, or a particular host topology belong under
`tests/platform/` and are selected by dedicated CI/CD jobs on compatible provisioned
runners. See `tests/README.md` for the container dependency command, supported
invocations, and marker policy.

Build `daqiri_config_validate` in the required project container before running
`scripts/check_pr.sh`. The check script validates representative checked-in configurations
through the production C++ parser and hardware-independent semantic checks. Set
`DAQIRI_CONFIG_VALIDATOR` when the executable is not at `build/tools/daqiri_config_validate`.

#### Pull Requests

Developer workflow for code contributions is as follows:
Expand Down
2 changes: 2 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -321,6 +321,8 @@ RUN cmake -S . -B build \
-DDAQIRI_ENABLE_S3=${DAQIRI_ENABLE_S3} \
-DDAQIRI_ENGINE="${DAQIRI_ENGINE}" \
&& cmake --build build -j "$(nproc)" \
&& python3 scripts/check_daqiri_configs.py \
--validator build/tools/daqiri_config_validate \
&& cmake --install build

# ==============================
Expand Down
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,9 @@ DAQIRI provides direct NIC hardware access in userspace, bypassing the Linux ker

## Features

- **Hardware-free configuration validation** — The installed
`daqiri_config_validate` tool uses DAQIRI's production parser and common semantic checks
without allocating packet memory, initializing an engine, or accessing a NIC.
- **High Throughput** — Sustained line rate with proper hardware and tuning.
- **Low Latency** — Direct access to NIC ring buffers; most latency is PCIe transit only.
- **Explicit hugepage allocation** — `kind: huge` always means hugetlb-backed memory for
Expand Down
14 changes: 14 additions & 0 deletions docs/api-reference/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,20 @@ want to interoperate with existing configuration code.

See `examples/daqiri_bench_*.yaml` for complete working examples.

## Validate without hardware initialization

`daqiri_config_validate` parses one or more YAML files and applies the same common semantic
checks used before `daqiri_init()`. It does not allocate packet memory, initialize CUDA or a
network engine, or access a NIC:

```bash
daqiri_config_validate config.yaml another-config.yaml
```

The command exits with status `0` when every file is valid, `1` when any file is invalid, and
`2` when no file was provided. It is built and installed even when
`DAQIRI_BUILD_EXAMPLES=OFF`.

OpenTelemetry metrics do not add YAML fields. Metrics-enabled builds use the
same interface, queue, and flow names from the active configuration as metric
labels, and applications are still responsible for configuring the OpenTelemetry
Expand Down
7 changes: 7 additions & 0 deletions docs/api-reference/cpp.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,13 @@ daqiri::NetworkConfig config;
auto status = daqiri::daqiri_init(config);
```

The YAML overloads use the production parser, and every initialization overload applies the shared
common semantic checks before allocating packet memory or initializing a transport engine. A
failed check returns a non-success status before hardware resources are touched. Application
startup performs this validation automatically; use the installed `daqiri_config_validate`
command only when a hardware-free preflight is useful. See
[Validate without hardware initialization](configuration.md#validate-without-hardware-initialization).

After `daqiri_init()` returns `Status::SUCCESS`, all memory regions are allocated, NIC
queues are configured, and worker threads are running.

Expand Down
7 changes: 7 additions & 0 deletions docs/api-reference/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,13 @@ queues, memory regions, flow steering rules, flow isolation,
hardware flow transform actions, header-data split, and optional reorder plans. After initialization,
the language API operates on that topology and, where supported, can extend it explicitly at runtime.

The YAML `daqiri_init` entry points use the production parser, and every initialization path
applies the shared common semantic checks before allocating memory or initializing a transport
engine. Application startup remains the normal validation path. For CI, batch checks, or
development without target hardware, the installed `daqiri_config_validate` command runs the same
parser and common checks without creating an engine; see the
[configuration reference](configuration.md#validate-without-hardware-initialization).

The APIs do **not** discover queues, memory, or flow steering rules on their
own. The startup configuration remains the source of truth for stream-type,
engine, endpoint selection, and immutable static flows. Applications may then
Expand Down
6 changes: 6 additions & 0 deletions docs/api-reference/python.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,12 @@ or a config-like object that provides `as_dict()`.
- `daqiri.NetworkConfig` instance
- config-like object with a `value` attribute or `as_dict()` method

The YAML and dictionary forms use the production parser, and every form is checked for common
semantic errors before packet memory or transport resources are initialized. This happens
automatically during normal application startup. For a hardware-free preflight outside the Python
process, use the installed `daqiri_config_validate` command described in the
[configuration reference](configuration.md#validate-without-hardware-initialization).

```python
import daqiri

Expand Down
13 changes: 13 additions & 0 deletions docs/concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,19 @@ sockets), see
[Choosing an example config](tutorials/configuration-walkthrough.md#choosing-an-example-config)
in the configuration walkthrough.

### Configuration validation

Application startup is the authority for configuration acceptance. The YAML `daqiri_init()` entry
points use the production parser, and every initialization path applies hardware-independent
common semantic checks before it allocates packet memory or initializes the selected engine.
Engine- and device-specific capability checks still occur during initialization because they
depend on the compiled engines and available hardware.

The installed `daqiri_config_validate` command is an optional hardware-free preflight for CI,
batch validation, or development away from the target system. It runs the same parser and common
checks as initialization, but it does not claim that a particular NIC can program every requested
feature. See the [Configuration YAML Reference](api-reference/configuration.md#validate-without-hardware-initialization).

??? example "Support and testing"

The DAQIRI library integration testing infrastructure is under active
Expand Down
3 changes: 2 additions & 1 deletion docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -213,7 +213,8 @@ DAQIRI's shared-library ABI version is tracked separately through
|--------|---------|-------------|
| `DAQIRI_ENGINE` | `"dpdk ibverbs"` | Space-separated list of optional engine implementations to compile in. Valid values: `dpdk` (Raw Ethernet) and `ibverbs`. `ibverbs` builds two libibverbs-based engines: RDMA/RoCE (for `stream_type: "socket"` with `roce://` endpoints) and the default Mellanox/mlx5 Multi-Packet (striding) Receive Queue engine for `stream_type: "raw"`. Set `engine: "dpdk"` on a raw stream to select the compiled DPDK implementation instead. Linux UDP/TCP sockets are always built in, so there is no `socket` value. |
| `DAQIRI_BUILD_PYTHON` | `OFF` | Build pybind11 Python bindings. |
| `DAQIRI_BUILD_EXAMPLES` | `ON` | Build benchmark executables. |
| `DAQIRI_BUILD_EXAMPLES` | `ON` | Build benchmark executables. The `daqiri_config_validate` tool is always built and installed, including when this option is `OFF`. |
| `BUILD_TESTING` | `ON` | Build and register the hardware-free C++ tests under `tests/cpp/` with CTest. Set this to `OFF` to omit test targets from a production-only build. |
| `DAQIRI_ENABLE_GDS` | `OFF` | Enable cuFile-backed burst file writes from CUDA device memory. Host-memory writes use POSIX APIs without GDS. |
| `DAQIRI_ENABLE_OTEL_METRICS` | `OFF` | Enable OpenTelemetry C++ metrics instrumentation. When enabled, OpenTelemetry C++ API package metadata must be available to CMake. |
| `DAQIRI_ENABLE_S3` | `OFF` | Enable AWS SDK-backed asynchronous raw packet writes to S3. |
Expand Down
6 changes: 6 additions & 0 deletions docs/tutorials/configuration-walkthrough.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,12 @@ If you don't have any NIC at all, the `*_sw_loopback*` variants of the Raw Ether

(`DAQIRI_ENGINE` at the CMake layer selects which optional engine implementations to compile in. `dpdk` enables the default raw engine, while `ibverbs` enables both the pure-DevX raw engine and `roce://` endpoints. Linux UDP/TCP sockets are always built in. The default build is `dpdk ibverbs`.)

The checked-in examples show how the pieces fit together, and `daqiri_init()` validates the chosen
configuration automatically when the application starts. You do not need a separate validation
step for normal use. The installed `daqiri_config_validate` command is available when CI or an
offline workflow needs to run the same parser and common semantic checks without initializing
hardware; see [Validate without hardware initialization](../api-reference/configuration.md#validate-without-hardware-initialization).

For a shorter selection guide, start with the [Benchmarking overview](../benchmarks/index.md). With a stream type in mind, read down the questions below and stop at the first one that matches what you're trying to do. Each section names the YAML, the binary that consumes it, and any platform-specific notes.

??? question "1. I want to measure baseline throughput"
Expand Down
Loading
Loading