C# Protocol Implementation Guide

Comprehensive reference for implementing protocol parsers in NetworkInspector. Based on the production patterns in FrameProtocol.cs, EthernetProtocol.cs, IPv4Protocol.cs, IPv6Protocol.cs, UdpProtocol.cs, and VlanProtocol.cs. Target: .NET 10+, handwritten header decoding, lazy field materialization architecture.

Architecture Overview
Threading & Execution Model
Class Structure & Naming Conventions
Source Generator Attributes
Field Attributes Reference
Constants — What to Define and What Not
Cross-Protocol References
The Parse Method
Lazy Field Materialization
Protocol Dispatch Tables
- PDU Transport → Signal Message
Heuristic Protocol Tables
Dispatch Cache Optimization
Index Groups & PacketIndex
- Field Alias Groups (Any-Match Names)
- Primary / Details Field Layout
- Mandatory Eager Rules
String Handling & Display Text
Error Handling
Binary Header Parsing
FieldValue Factories
Protocol Registration
Source Generator Output
Current Dispatch Table Map
Checklist for New Protocols
Fragment Reassembly
Testing A Protocol That Keeps State

1. Architecture Overview

NetworkInspector uses a layered protocol stack where each protocol is a self-contained parser that:

Registers its fields, dispatch tables, and settings at build time via a Source Generator.
Parses packet data at runtime, producing a tree of lazily-materialized fields.
Dispatches to sub-protocols via typed dispatch tables (e.g., EtherType → IPv4).

The field tree is the central data structure — filters, UI, and exporters all operate on it. Protocols produce fields; everything else consumes them.

Packet bytes
    │
    ▼
  FrameProtocol ──── frame.link_type table ────► EthernetProtocol
                                                       │
                                                 eth.type table
                                                   ┌───┼───┐
                                                   ▼   ▼   ▼
                                                IPv4  IPv6  VLAN
                                                  │    │
                                            ip.proto table
                                              ┌───┼───┐
                                              ▼   ▼   ▼
                                            UDP  TCP  ICMP
                                              │    │
                                        udp.port  tcp.port table
                                           │         │
                              ┌────────────┼─────┐   ▼
                              ▼            ▼     ▼  HTTP, TLS, …
                            DNS      PDU Transport  …
                                           │
                                  pdu_transport.id
                                           ▼
                                    Signal Message

Field tree structure (sibling layout, not nested):

root
├── frame: Frame 1, 128 bytes, Ethernet
│   ├── frame.id: 1
│   └── ...
├── eth: Ethernet II, Src: AA:BB:CC:DD:EE:FF, Dst: 11:22:33:44:55:66
│   ├── eth.dst: 11:22:33:44:55:66
│   ├── eth.src: AA:BB:CC:DD:EE:FF
│   └── eth.type: 0x0800 (IPv4)
├── ip: Internet Protocol Version 4, Src: 192.168.1.1, Dst: 10.0.0.1
│   ├── ip.version: 4
│   ├── ...
│   ├── ip.src: 192.168.1.1
│   └── ip.dst: 10.0.0.1
├── udp: User Datagram Protocol, Src Port: 12345, Dst Port: 53
│   ├── udp.srcport: 12345
│   ├── udp.dstport: 53
│   └── ...
├── pdu_transport: PDU Transport                 // sibling of udp when bound on udp.port
│   └── pdu_transport.pdu: PDU: BenchPdu (ID: 32)
│       ├── pdu_transport.id / length / name
│       └── pdu_transport.payload                // only when no sub-protocol matches
└── fixture_message: Fixture PDU                 // sibling of pdu_transport (dispatch on parentField)
    ├── fixture_message.EngineRpm
    └── fixture_message.Thr

Sub-protocols are siblings of their parent protocol, not children. Dispatch always happens on parentField, not inside a protocol's container field. When the frame protocol consumes less than the capture buffer, the tail is packet.unparsed_data under the packet container (§8). Ethernet padding stays on eth.

2. Threading & Execution Model

Parse Execution

The first parse of each packet is single-threaded and ordered: one thread walks the packets of a capture in ascending id order, and that thread alone drives every protocol on the stack. Once a packet has been parsed, however, it may be parsed again from any thread at any time — the UI, filter evaluation and export all do this, concurrently and while the first-parse thread is still working through later packets. So a Stack is owned by one first-parse thread, not by one thread outright. This means:

No locks, no synchronization primitives in protocol code.
Protocol instance fields that are written during parsing need the ingest gate and effect-store pattern described below; everything else must be set up during the build phase (OnStartCustom()) and treated as read-only afterwards, or kept in [ThreadStatic] fields. Most protocols are stateless and need neither.
The in MutField parameter is a ref struct — it cannot escape the call stack.

Random Access Parsing

Packets must be parseable multiple times (random access from UI, filter evaluation, export). The same packet data may be fed to Parse() again at any time. This means:

No mutable per-packet state stored on the protocol instance between parse calls. All packet-specific state lives in the field tree (MutField / FieldValue). Protocols that genuinely need cross-packet state (connection trackers, reassembly and fragment buffers) are the documented exception and must follow the ingest gate plus effect-store pattern described under First parse vs. re-parse below — otherwise a concurrent re-parse races on that state.
Caches and dispatch tables are per-stack state, initialized once in OnStartCustom() and immutable during parsing.
A LazyPopulator may be called zero or one time per packet — never assume it runs.
Lazy fields store their raw header bytes in FieldValue so re-parsing is self-contained.

First parse vs. re-parse (concurrent subsequent parses)

The only entry points are the two Packet.TryParse overloads. ParseOptions carries the field-tree mode, cache, index, replay-record flag, first protocol, and observers. ParseContext still does not carry a separate parse-intent enum. Each stateful protocol decides for itself whether a call is a first parse or a re-parse.

Mechanism — ingest gate. Every stateful protocol owns one private readonly IngestGate _Gate = new(); (NetworkInspector.Core.Protocols). Its watermark is the highest packet id whose first parse completed; _Gate.IsReplay(id) is id.Value <= watermark (one volatile read, safe from any thread). The pattern at the top of Parse:

bool isReplay = _Gate.IsReplay(packet.Id);
IngestScope scope = isReplay
    ? default                      // inert scope: a replay never raises the watermark
    : _Gate.Enter(packet.Id);      // throws InvalidOperationException for an id that is already replayed
try { return _ParseBody(/* ... */, isReplay); }
finally { scope.Dispose(); }       // the outermost scope raises the watermark

A protocol that has no cleanup in the finally can write if (_Gate.IsReplay(id)) { return _Replay(...); } followed by using IngestScope scope = _Gate.Enter(id);.

id > watermark → first parse: mutate cross-packet state (trackers, reassembly and fragment buffers) and record a compact effect for the packet.
id <= watermark → re-parse: read the recorded effect only. Never mutate shared state. A missing effect degrades to a stateless path — it never falls back to mutation.

Only the outermost scope raises the watermark, because a nested parse of the same packet (a tunnel parses a protocol twice) must still take the first-parse path. The raise happens in IngestScope.Dispose, so even error and exception exits close the door: a later re-parse of that id can never re-enter the first-parse path. Record every effect inside the scope, before it ends: the watermark write is the release that publishes the effects to replaying threads. The nesting counter is a plain int because only the ordered first-parse thread touches it; an Interlocked maximum was rejected because there is no second writer to race with.

Authors implement IProtocol.Parse. Stack.CallProtocol stamps ParseContext.SelfProtocolId and then invokes the bound ParseDelegate. A raw protocol.Parse(...) or a cached ParseDelegate is a valid entry; it simply does not go through CallProtocol. Effect keys do not depend on that stamp.

Mechanism — layer key. Effects are keyed by (PacketId, packed buffer location). MutField.TryGetEffectLayerKey(data, out int key) packs the buffer index (0 = Frame.Data, 1… = buffers stored by MutField.TryBindBuffer) into bits 31–24 and the byte offset of the Parse data slice into bits 23–0. On false the slice is not a packet buffer. The argument must be the slice passed into Parse, not a heap copy. First parse and reparse of the same frame see the same key because the bytes and additional buffers are stable. Do not key on remaining length (data.Length) or any walk ordinal. Effects live in EffectStore<TEffect>: one packed row per packet that actually ran the protocol, binary search on replay, nested layers chained when the tail entry already belongs to the same packet id. Without the packed location, the inner layer of a tunnel (or a defragmented datagram in an additional buffer) would collide with an outer layer and replay the wrong values. Reassembled, decoded, unmasked, and inflated payloads must be attached with MutField.TryBindBuffer before the nested Parse so the inner slice is a packet buffer, not a heap copy (§22.4).

Stack.ProtocolCount is how many protocols are registered. There is no packet walk ordinal. ProtocolId is assigned at RegisterProtocol time.

Protocols must not retain Packet references. Cross-packet protocol state is keyed by PacketId and layer key where needed.

Effect stores live on the protocol instances and therefore share the Stack lifetime — a stack swap creates fresh protocols with empty stores.

Contract for callers. First parses must be ordered, single-threaded, and use dense packet ids 0, 1, 2, …. Packet.TryParse returns false and ParseFailure.ParseIdGap, and does not throw, on a jump (for example id 5 after id 0). Re-parses of already-parsed ids may run on any number of threads at any time, including while later packets are being parsed for the first time. The session guarantees the first half structurally: the source loop parses under _ParseLock with ids from a monotonic allocator starting at 0, while TryGetPacket re-parses lock-free via _TryReparseFrame. Packet.TryParse with ParseOptions.Index is safe on a re-parse: TryBeginPacket returns false and the index is not mutated.

Stateful protocols still use their own ingest gate (_Gate.IsReplay(id), id <= protocol watermark → replay) because they are not invoked for every packet. The dense check lives on the Stack, which sees every TryParse call.

Audit of cross-packet mutable state (every protocol that survives state across packets):

Protocol	Cross-packet state	Measure
`TcpProtocol`	`_ConnectionTracker`, `_ReassemblyEngine`	`IngestGate` + `EffectStore<TcpLayerEffect>` (analysis facts, dispatch mode, reassembled PDU bytes). Re-parse without effect → stateless raw-port dispatch.
`UdpProtocol`	`_StreamTracker`	`IngestGate` + `EffectStore<StreamEffect>`. Re-parse without effect → no `udp.stream` field. The `udp.heuristic` fallback is stateless: a replay asks the table again and reaches the same parser.
`IPv4Protocol`	`_Defragmenter` (`FragmentReassembler<DatagramFragmentKey>`, preset `IPv4`) fragment buffers	`IngestGate` + `EffectStore<DefragLayerEffect>` holding the reassembled datagram (only for the completing fragment). Re-parse without effect → fragment fields, no reassembly. See §22.
`IPv6Protocol`	`_Defragmenter` (`FragmentReassembler<IPv6DatagramFragmentKey>`, preset `IPv6`) fragment buffers	Same as IPv4.
`SomeIpProtocol`	the acceptance decision of a unit that is not on a bound port (decrypted hand-off, content-based detection, learned ports); the segment sessions of SOME/IP-TP (`FragmentReassembler<SomeIpTpKey>` behind `SomeIpTpSessions`); the ports learned from service discovery; the pending requests of method timing (`SomeIpTimingTracker`, no lock: only the ordered first parse touches it)	`IngestGate` + `EffectStore<SomeIpMessageEffect>` (one record per key: the unit decision, and for a segmented message its `FragmentOutcome`; a message at the key of its unit shares the record; a response also stores its timing pair there). A unit on a bound port decides from the published ports alone and records nothing. Re-parse without effect → the decision is evaluated again from immutable state, a segment shows its TP word and a named `someip.tp.error.dropped`. The segment list is lazy and reads only the published effect.
`EthernetProtocol`, `IPv4Protocol`, `IPv6Protocol`	address caches	`[ThreadStatic]` — thread-local, safe as-is.
`SignalMessageProtocol`, `CanProtocol`, the `SomeIpCompiledModel` and `SomeIpSettings` of `SomeIpProtocol`, all `SparseCache` / `DelegateCache`	dispatch and display lookups	Written during registration / `OnStart` only, immutable during parsing — safe as-is.

All other protocols hold no cross-packet mutable state.

Lifecycle

Construction     → new UdpProtocol()              Default state, no IDs yet
Registration     → RegisterFields(builder, id)    Source Generator wires fields, tables, settings,
                                                  loads setting backing fields, then calls
                                                  RegisterFieldsCustom(builder, protocolId)
Stack Build      → Stack.Build()                  All registrations frozen, IDs finalized,
                                                  When*Registered callbacks fired
OnStart          → OnStartCustom(stack)           Build dispatch caches, resolve cross-protocol
                                                  fields against the frozen stack
Parse (×N)       → Parse(parentField, data, stack) Hot path — called once per packet per protocol
OnShutdown       → OnShutdownCustom(stack)        Cleanup (rarely needed)

Two custom hooks — when to use which:

RegisterFieldsCustom(IStackBuilder builder, ProtocolId protocolId) — runs during registration. Use it for anything that depends on settings or registers entries into the stack (loading config files, registering parsers in dispatch tables derived from config, declaring additional fields). Settings are already loaded into backing fields when this hook runs. Use builder.WhenFieldRegistered(...) / WhenProtocolTableRegistered(...) to defer references to entities that may be registered by another protocol later (the callbacks fire immediately if the entity already exists, otherwise during Build).
OnStartCustom(Stack stack) — runs after the stack is built and frozen. Use it for work that requires the immutable stack: pre-allocating lazy populators, building dispatch caches via stack.BuildU64DelegateCache(...), resolving cross-protocol field IDs, caching parse delegates.

The stack itself is immutable after Build(). There is no runtime API to register new fields, parsers, or settings — config-driven registration must run in RegisterFieldsCustom.

3. Class Structure & Naming Conventions

File Layout

One protocol per file. The file contains the protocol class and its binary header struct (if any). Large protocols may split sub-field modules into separate files (e.g., TcpOptions.cs, TcpAnalysis.cs).

Class Declaration

[Protocol("udp", "User Datagram Protocol", Description = "UDP (RFC 768)")]
[RegisterAtTable(IPv4Protocol.IpProtoTableName, IpProtoKey)]
public sealed partial class UdpProtocol : IProtocol

sealed — not for inheritance beyond the generator partial.
partial — required for Source Generator integration.
IProtocol — implement Parse. Prefer parentField.CallProtocol for child dispatch.

Member Ordering

1. Structural constants           (HeaderSize, MinPayloadSize, ...)
2. Table Key Constants            (public const ulong)
3. Table Name Constants           (public const string)
4. Index Group Constants          (private const string, only if used >1×)
5. Protocol-specific constants    (extension header types, magic bytes, ...)
6. Fields                         (FieldId members with [*Field] attributes)
7. Dispatch Tables                (ProtocolTableId with [ProtocolTableU64] / [UsesTable])
8. Runtime Settings               ([BoolSetting], [StringSetting], ...)
9. Cross-protocol field refs      (FieldId resolved in OnStartCustom)
10. Dispatch caches               (sparse/dense arrays, pre-allocated populator)
11. RegisterFieldsCustom / OnStartCustom / OnShutdownCustom
12. Lazy Populator method(s)
13. Parse method
14. Dispatch helper method(s)
15. Utility methods

Naming Conventions

Element	Convention	Example
Protocol class	`{Name}Protocol`	`UdpProtocol`, `VlanProtocol`
Header struct	`{Name}Header`	`UdpHeader`, `IPv4Header`
Table key constants	`public const ulong {Description}Key`	`IpProtoKey`, `EtherTypeKey`
Table key (multi)	`public const ulong {Description}Key{Variant}`	`EtherTypeKey8021Q`, `EtherTypeKeyQinQ`
Table name constants	`public const string {Desc}TableName`	`PortTableName`, `EtherTypeTableName`
Index group constants	`private const string {Name}IndexGroup`	`UdpIndexGroup`, `EthIndexGroup`
FieldId fields	`private FieldId _{PascalName}FieldId`	`_SrcPortFieldId`, `_DstFieldId`
ProtocolTableId fields	`private ProtocolTableId _{Name}TableId`	`_PortTableId`, `_EtherTypeTableId`
Populator field	`private LazyPopulator _Populator`	`_Populator`
Dispatch cache	`private ... _{Name}Cache`	`_EtherTypeSparseCache`, `_IpProtoInstanceCache`
Populator method	`Populate{Name}Fields`	`PopulateUdpFields`, `PopulateIPv4Fields`
Dispatch method	`Dispatch{TableName}`	`DispatchEtherType`, `DispatchIpProtocol`

XML Documentation

Every protocol class must have a <summary> with a field tree example:

/// <summary>
/// User Datagram Protocol (RFC 768) parser with checksum validation.
/// <para>Field tree structure:</para>
/// <code>
/// udp: User Datagram Protocol, Src Port: 12345, Dst Port: 53
/// ├── udp.srcport: 12345
/// ├── udp.dstport: 53
/// ├── udp.length: 30
/// ├── udp.checksum: 0xabcd
/// ├── udp.checksum.status: [Good] / [Bad]  [optional]
/// └── udp.payload: (22 bytes)              [optional]
/// </code>
/// </summary>

4. Source Generator Attributes

Protocol Definition

Attribute	Target	Purpose
`[Protocol("name", "UI Name")]`	Class	Defines protocol identity, generates `ProtocolName` constant
`[RegisterAtTable(table, key)]`	Class	Registers at a u64 dispatch table
`[RegisterAtStringTable(table, key)]`	Class	Registers at a string dispatch table
`[RegisterAtBoolTable(table, key)]`	Class	Registers at a bool dispatch table
`[RegisterAtAnyTable(table)]`	Class	Registers as catch-all in a table

Table Declaration & Reference

Attribute	Target	Purpose
`[ProtocolTableU64(name, uiName)]`	`ProtocolTableId` field	Declares a new u64 dispatch table
`[UsesTable(tableName)]`	`ProtocolTableId` field	References another protocol's table (deferred resolution)

Runtime Settings

Attribute	Target	Purpose
`[BoolSetting(name, uiName, group)]`	`bool` field	Runtime toggle (e.g., checksum verification)
`[StringSetting(name, uiName, group)]`	`string` field	Runtime text setting
`[F64Setting(name, uiName, group)]`	`double` field	Runtime numeric setting with Min/Max

Settings are loaded automatically in OnStart() from the stack's setting registry. The Default property sets the initial value.

5. Field Attributes Reference

Every field in the protocol tree is declared with an attribute on a FieldId field:

Attribute	`FieldType`	Typical use	Example
`[U64Field]`	`U64`	Ports, lengths, counters, enums	`[U64Field("udp.srcport", "Source Port")]`
`[I64Field]`	`I64`	Signed numeric fields	`[I64Field("tcp.window_scale", "Window Scale")]`
`[F64Field]`	`F64`	Floating-point fields	`[F64Field("frame.time_delta", "Delta Time")]`
`[BoolField]`	`Bool`	Single-bit flags	`[BoolField("ip.flags.df", "Don't Fragment")]`
`[MacField]`	`MacAddress`	48-bit MAC addresses	`[MacField("eth.dst", "Destination")]`
`[IPv4Field]`	`IPv4Address`	IPv4 addresses	`[IPv4Field("ip.src", "Source Address")]`
`[IPv6Field]`	`IPv6Address`	IPv6 addresses	`[IPv6Field("ipv6.src", "Source")]`
`[BytesField]`	`Bytes`	Raw byte data, payloads	`[BytesField("udp.payload", "Payload")]`
`[StringField]`	`Str`	Text fields, status messages	`[StringField("udp.checksum.status", "Status")]`
`[TimestampField]`	`Timestamp`	Nanosecond-precision timestamps	`[TimestampField("frame.time", "Arrival Time")]`
`[NoneField]`	`None`	Grouping node (no intrinsic value)	`[NoneField("frame", "Frame")]`

Common properties on all field attributes:

Property	Type	Purpose
`IndexGroup`	`string`	Required. Groups fields for index bitmap optimization (see §13)
`Description`	`string?`	Optional long description for tooltips

Protocol Container Field

Every protocol must declare a container field — typically a BytesField that stores the header bytes for lazy re-parsing:

[BytesField("udp", "UDP", IndexGroup = UdpIndexGroup)]
private FieldId _ProtocolFieldId;

This field serves as the root node for all protocol-specific sub-fields in the tree.

6. Constants — What to Define and What Not

Define Named Constants For:

Category	Visibility	Example
Table names	`public const string`	`public const string PortTableName = "udp.port";`
Table key values	`public const ulong`	`public const ulong IpProtoKey = 17;`
Index groups (used >1×)	`private const string`	`private const string UdpIndexGroup = "udp";`
Structural sizes	`private const int`	`private const int HeaderSize = 8;`
Protocol-specific magic numbers	`private const`	`private const byte ExpectedVersion = 4;`

Do NOT Define Constants For:

Category	Use instead	Rationale
Field names	Inline string in attribute	Used exactly once, no cross-reference
Protocol names	Auto-generated `ProtocolName`	Source Generator provides it
UI display names	Inline string in attribute	Used exactly once
Index groups (used only once)	Inline string literal	No benefit to extracting

Rationale: Table names and key values are cross-protocol contracts — they must stay in sync across files. Field names are private, used in exactly one attribute, and extracting them into constants adds indirection without benefit.

7. Cross-Protocol References

Table Ownership

The protocol that creates a dispatch table owns its name constant:

// EthernetProtocol OWNS the table → defines the name
public const string EtherTypeTableName = "eth.type";

[ProtocolTableU64(EtherTypeTableName, "EtherType")]
private ProtocolTableId _EtherTypeTableId;

Registering at Another Protocol's Table

Protocols that dispatch through another protocol's table use the owner's constant:

// IPv4 registers at Ethernet's EtherType table
[RegisterAtTable(EthernetProtocol.EtherTypeTableName, EtherTypeKey)]
public sealed partial class IPv4Protocol : IProtocol
{
    public const ulong EtherTypeKey = 0x0800;
}

// VLAN registers at two keys
[RegisterAtTable(EthernetProtocol.EtherTypeTableName, EtherTypeKey8021Q)]
[RegisterAtTable(EthernetProtocol.EtherTypeTableName, EtherTypeKeyQinQ)]
public sealed partial class VlanProtocol : IProtocol
{
    public const ulong EtherTypeKey8021Q = 0x8100;
    public const ulong EtherTypeKeyQinQ = 0x88A8;
}

Reusing Another Protocol's Table

When a protocol needs to dispatch using a table it didn't create (e.g., IPv6 dispatches using IPv4's ip.proto table), use [UsesTable]:

// IPv6 REUSES IPv4's dispatch table (deferred resolution)
[UsesTable(IPv4Protocol.IpProtoTableName)]
private ProtocolTableId _IpProtoTableId;

The Source Generator calls builder.WhenProtocolTableRegistered(...) which resolves the table ID after all protocols have registered. This works regardless of registration order.

Reading Fields From Other Protocols

When a downstream protocol needs values from an upstream protocol (e.g., UDP reading IP addresses for checksum validation), resolve field IDs in OnStartCustom():

private FieldId _IpSrcFieldId;
private FieldId _IpDstFieldId;

partial void OnStartCustom(Stack stack)
{
    _IpSrcFieldId = stack.GetFieldId("ip.src") ?? default;
    _IpDstFieldId = stack.GetFieldId("ip.dst") ?? default;
}

At parse time, read eagerly-appended fields from the packet tree:

if (_IpSrcFieldId.IsValid
    && packet.TryGetFieldValue(_IpSrcFieldId, out FieldValue ipv4Src, materialize: false)
    && ipv4Src.Type == FieldType.IPv4Address)
{
    // Use ipv4Src — materialize: false keeps sibling lazy containers untouched
}

Important: This only works if the upstream protocol eagerly appends the field (not deferred inside a lazy populator). See §9.5 for the eager-append pattern. Pass materialize: false on the parse hot path; use materialize: true only when the caller intentionally needs lazy field trees (exporters, assertions, full dumps).

Thread-Local Address Caches

The sibling-walk approach above is a safe fallback, but it is not the primary pattern for hot-path cross-protocol data such as IP addresses for checksum validation. The production pattern is a per-thread, per-packet cache stored in a [ThreadStatic] field on the providing protocol class. UdpPortCache and TcpPortCache (the source and destination port of the datagram or segment being dispatched, guarded by the packet id and withdrawn when the dispatch is done) are the same pattern for application protocols that are reached without a dispatch key (the heuristic tables, framed TCP messages, the single-owner port cache of TCP) and have to know the port they were reached on. PlaintextHandoff is the same pattern for a protocol that decrypts a transport: it publishes the transport and the connection port before it hands the plaintext on, and the receiving protocol compares the port with the ports its configuration lists for that transport; the decrypting protocol withdraws it in a finally. The decrypting protocol also owns the dispatch table of its transport (tls.port, dtls.port, keyed by the server port), where SOME/IP binds the ports its configuration lists; when no such table exists at SOME/IP's registration, which is the case in the shipped stack because its TLS and DTLS parsers do not decrypt, SOME/IP reports one warning per non-empty list.

Why thread-local instead of sibling walk?

The parse thread owns the stack exclusively (§2). A [ThreadStatic] field read or write is a single load/store with no heap allocation, no lock, and no field-tree traversal. It is the correct choice for data that is:

Produced unconditionally by an upstream protocol for every packet, and
Consumed by one or more immediate downstream protocols on the same thread.

Pattern — provider side (e.g., IPv4Protocol):

#region Thread-Local Address Cache

/// <summary>
/// Per-thread cache for the current packet's IPv4 src/dst addresses.
/// Written by <see cref="Parse"/> before dispatching; consumed by TCP, UDP.
/// Null is the correct default for <see langword="[ThreadStatic]"/>.
/// </summary>
[ThreadStatic]
private static (int PacketId, IPv4Address Src, IPv4Address Dst)? _ThreadCache;

[MethodImpl(MethodImplOptions.AggressiveInlining)]
internal static void SetCachedAddresses(PacketId packetId, IPv4Address src, IPv4Address dst)
    => _ThreadCache = (packetId.Value, src, dst);

[MethodImpl(MethodImplOptions.AggressiveInlining)]
internal static bool TryGetCachedAddresses(PacketId packetId, out IPv4Address src, out IPv4Address dst)
{
    (int PacketId, IPv4Address Src, IPv4Address Dst)? c = _ThreadCache;
    if (c.HasValue && c.Value.PacketId == packetId.Value)
    {
        src = c.Value.Src;
        dst = c.Value.Dst;
        return true;
    }
    src = default;
    dst = default;
    return false;
}

#endregion

Key design decisions:

The cache stores (PacketId, Src, Dst) — the PacketId guards against stale entries left from the previous packet on the same thread.
null is the correct initial value for [ThreadStatic] fields (no explicit initialization needed; null means "no data cached yet on this thread").
The methods are internal static so only sibling protocols in the same assembly can read the cache — they are not part of the public API.

Call site in Parse() — write before dispatch:

// Cache src/dst on the parse thread so downstream protocols can read them
// without a field-tree walk. Must be called BEFORE dispatching to the next protocol.
SetCachedAddresses(parentField.Packet.Id, src, dst);

// Now dispatch — TCP/UDP will read the cache during their Parse() call
ParseResult result = DispatchIpProtocol(in parentField, proto, payload, in context);

Pattern — consumer side (e.g., UdpProtocol):

When multiple parent protocols share the same dispatch table and one of them is a tunnel (e.g., IPv6-in-IPv4), both caches may hold valid entries for the same PacketId. Use context.Dispatch.CallerProtocolId to decide which cache to read first — this prevents selecting the outer tunnel endpoints instead of the inner transport endpoints:

PacketId packetId = parentField.Packet.Id;

// CallerProtocolId tells us which IP version dispatched us in this invocation.
// In tunnel scenarios (e.g., 6in4) both caches hold valid data for the same
// PacketId — check the correct one first.
bool callerIsIpv4 = context.Dispatch.HasDispatch && context.Dispatch.CallerProtocolId == _Ipv4ProtocolId;
bool callerIsIpv6 = context.Dispatch.HasDispatch && context.Dispatch.CallerProtocolId == _Ipv6ProtocolId;

if (!callerIsIpv6 && IPv4Protocol.TryGetCachedAddresses(packetId, out IPv4Address src4, out IPv4Address dst4))
{
    // Use IPv4 addresses
}
else if (!callerIsIpv4 && IPv6Protocol.TryGetCachedAddresses(packetId, out IPv6Address src6, out IPv6Address dst6))
{
    // Use IPv6 addresses
}
else
{
    // Fallback: walk previous siblings for edge cases (custom stacks, unusual encapsulations)
    TryFindPreviousIpAddressesFallback(in parentField, out IPv4Address fbSrc4, out IPv4Address fbDst4, in context);
}

Summary — when to use thread-local caches vs. sibling walk:

Scenario	Approach
Hot path, same-assembly protocols, unconditionally produced per packet	`[ThreadStatic]` cache (zero allocation, O(1))
Edge cases, custom stacks, unusual encapsulations	Sibling walk via `IpAddressExtractor` fallback
Cross-assembly or external consumers	Eagerly-appended field + `packet.TryGetFieldValue(..., materialize: false)` (§7 Reading Fields)

8. The Parse Method

Handwritten dissectors implement IProtocol.Parse. Stack.CallProtocol stamps ParseContext.SelfProtocolId and then invokes the bound parse method. Prefer parentField.CallProtocol so invalid ids return ParseError.

// NEVER: bind a copied local function as ParseDelegate when you need SelfProtocolId
// ALWAYS: parentField.CallProtocol(id, data, in context) or TryCallNextProtocol*

Signature

public ParseResult Parse(in MutField parentField, ReadOnlyMemory<byte> data, in ParseContext context)

parentField — the parent field to append to (sibling dispatch target).
data — the raw packet bytes for this protocol layer.
context — the parse context providing the protocol stack (dispatch tables, field registry) and dispatch state.
Returns ParseResult — consumed byte count on success, or a ParseError.

Mandatory Sequence

Step 1: Validate minimum data size
Step 2: Record protocol + index group presence
Step 3: Parse header eagerly (local variables from ReadOnlySpan<byte>)
Step 4: Validate header integrity (version, length bounds, ...)
Step 5: Compute payload bounds
Step 6: Record optional index groups conditionally
Step 7: Build summary LazyString
Step 8: Append lazy container field
Step 9: Eagerly append cross-protocol fields (if needed by downstream)
Step 10: Dispatch to next protocol (on parentField!)
Step 11: Append trailing fields (padding, trailer)
Step 12: Return consumed bytes

Complete Example (UDP)

public ParseResult Parse(in MutField parentField, ReadOnlyMemory<byte> data, in ParseContext context)
{
    // ── Step 1: Validate minimum data size ──────────────────────────
    if (data.Length < UdpHeader.HeaderSize)
    {
        return ParseError.InsufficientDataWithInfo(ProtocolName, UdpHeader.HeaderSize, (ulong)data.Length);
    }

    // ── Step 2: Record presence ─────────────────────────────────────
    context.RecordProtocolPresence(_ProtocolId);
    context.RecordGroupPresence(_UdpGroupId);

    // ── Step 3: Parse header eagerly ────────────────────────────────
    ReadOnlySpan<byte> span = data.Span;
    if (!UdpHeader.TryParse(span, out UdpHeader header))
    {
        return ParseError.InsufficientDataWithInfo(ProtocolName, UdpHeader.HeaderSize, (ulong)data.Length);
    }

    ushort srcPort = header.SrcPort;
    ushort dstPort = header.DstPort;
    ushort length = header.Length;

    // ── Step 5: Compute payload bounds ──────────────────────────────
    int payloadLen = Math.Max(0, Math.Min(length - UdpHeader.HeaderSize, data.Length - UdpHeader.HeaderSize));
    ReadOnlyMemory<byte> payloadData = payloadLen > 0
        ? data.Slice(UdpHeader.HeaderSize, payloadLen)
        : ReadOnlyMemory<byte>.Empty;

    // ── Step 6: Record optional index groups ────────────────────────
    bool checksumVerified = _VerifyChecksum && header.Checksum != 0;
    if (checksumVerified)
    {
        context.RecordGroupPresence(_UdpChecksumStatusGroupId);
    }
    if (payloadLen > 0)
    {
        context.RecordGroupPresence(_UdpPayloadGroupId);
    }

    // ── Step 7: Build summary ───────────────────────────────────────
    // FormatLazy boxes the value tuple once; the static lambda captures nothing, so no closure is allocated
    LazyString summary = LazyString.FormatLazy(
        (srcPort, dstPort),
        static s => string.Create(
            CultureInfo.InvariantCulture,
            $"User Datagram Protocol, Src Port: {s.srcPort}, Dst Port: {s.dstPort}"));

    // ── Step 8: Set packet info (transport-layer display) ───────────
    parentField.SetPacketInfo(LazyString.FormatLazy(
        (srcPort, dstPort),
        static s => string.Create(CultureInfo.InvariantCulture, $"Src Port: {s.srcPort}, Dst Port: {s.dstPort}")));

    // ── Step 9: Append lazy container ───────────────────────────────
    // Store the full UDP datagram so the populator can re-parse without captured state
    FieldValue containerValue = FieldValue.NewBytes(data)
        .WithCustomRepresentation(new LazyString("8 bytes"));
    parentField.AppendLazyWithCustomText(_ProtocolFieldId, containerValue, summary, _Populator);

    // ── Step 10: Dispatch to next protocol (on parentField!) ────────
    if (payloadLen > 0)
    {
        ushort lowPort = Math.Min(srcPort, dstPort);
        ushort highPort = Math.Max(srcPort, dstPort);

        ParseResult result = parentField.TryCallNextProtocolU64(
            _PortTableId, lowPort, payloadData, in context);
        if (result.TryPropagateError(out ParseResult error))
        {
            return error;
        }

        if (!result.TryGetConsumed(out _) && lowPort != highPort)
        {
            ParseResult highResult = parentField.TryCallNextProtocolU64(
                _PortTableId, highPort, payloadData, in context);
            if (highResult.TryPropagateError(out ParseResult highError))
            {
                return highError;
            }
        }
    }

    // ── Step 12: Return consumed bytes ──────────────────────────────
    return Math.Min(length, data.Length);
}

Critical Rules

Dispatch on parentField — sub-protocols are siblings, not children.
Append lazy container before dispatch — sub-protocols may read parent fields.
All context.RecordGroupPresence() calls happen eagerly in Parse() — never inside a lazy populator.
Guard against negative payload lengths with Math.Max(0, ...).
Propagate dispatch errors — call TryPropagateError after every TryCallNextProtocol*() call and return the out result when it is true. After a false TryPropagateError, a false TryGetConsumed is exactly NotDispatched (try another key or stop). Ok(0) returns true from TryGetConsumed with consumed == 0 and must not be treated as a miss.

Leftover bytes (`packet.unparsed_data`)

When the frame protocol returns success with consumed < frame.Length, PacketProtocol appends packet.unparsed_data (Bytes) under the packet container and records the optional index group packet.unparsed. Inner protocols do not insert a leftover data PDU. Ethernet uses the child’s consumed count to append padding/trailer and then returns data.Length, so a typical Ethernet/SLL stack never produces packet.unparsed_data.

Author rules:

Return the bytes this PDU actually used, not always data.Length, when the PDU length is known and shorter than the buffer.
Do not wrap every TryCallNextProtocol* with leftover handling. Link-layer padding and trailers stay with the frame protocol (EthernetProtocol._AppendPaddingAndTrailer).
Tails inside a fully consumed frame (for example UDP payload after a signal message that returned RequiredByteLength) are not packet.unparsed_data — they are not at the end of the capture buffer. Store only the bytes the PDU needs; the rest of that payload is not shown as a sibling protocol.

DataProtocol (Name = "data") remains a normal dissector for registered payloads (HTTP content-type, WebSocket, …). It is not the leftover fallback.

9. Lazy Field Materialization

9.1 Why Lazy?

Lazy materialization provides a ~3× throughput improvement on the parse hot path. Instead of building the complete field tree for every packet, Parse() records only the minimum information (index presence, dispatch keys) and defers the expensive field tree construction to a LazyPopulator that runs only when someone reads the fields (UI, filter, exporter).

9.2 The Pre-Allocated Populator Pattern

Inline lambda closures per packet are prohibited. Every lazy populator must be a pre-allocated delegate created once in OnStartCustom():

// The delegate field — lives on the protocol instance
private LazyPopulator _Populator = null!;

partial void OnStartCustom(Stack stack)
{
    // Allocate once — captures only 'this' (the singleton protocol instance)
    _ = PopulateUdpFields(in container);
}

This eliminates the per-packet closure allocation that a lambda would generate.

9.3 How Data Flows to the Populator

The populator re-parses header data from the FieldValue stored in the container field. At Parse() time, store the raw header bytes (or the entire datagram) as a BytesField:

// In Parse(): Store header bytes in the container's value
FieldValue containerValue = FieldValue.NewBytes(data[..headerLen])
    .WithCustomRepresentation(new LazyString("20 bytes"));
parentField.AppendLazyWithCustomText(_ProtocolFieldId, containerValue, summary, _Populator);

In the populator method, re-read and re-parse from the stored bytes:

private ParseResult PopulateIPv4Fields(in LazyField container)
{
    // Re-read the header bytes stored in the container's field value
    ReadOnlyMemory<byte> headerBytes = container.Value.Data.AsBytes();
    if (!IPv4Header.TryParse(headerBytes.Span, out IPv4Header header, out _))
    {
        return ParseError.InvalidData(ProtocolName, "Failed to parse IPv4 header");
    }

    // Append all child fields from the re-parsed header
    container.Append(_VersionFieldId, FieldValue.NewU64(header.Version));
    // ... more fields ...

    return 0;  // Success (return value ignored for populators)
}

9.4 LazyPopulator Delegate

public delegate ParseResult LazyPopulator(in LazyField parentField);

Called exactly once on first access to the container's children.
Must NOT call TryCallNextProtocol*() — dispatch belongs in Parse().
Must NOT call context.RecordGroupPresence() — index recording belongs in Parse().
Returns ParseResult — errors propagate to the caller that triggered materialization.

9.5 Eager Append for Cross-Protocol Fields

Some fields must be accessible by downstream protocols without triggering lazy materialization. For example, IPv4 eagerly appends ip.src and ip.dst so that UDP/TCP can read them for checksum validation:

// In Parse(): Append lazy container ...
MutField protoField = parentField.AppendLazyWithCustomText(
    _ProtocolFieldId, headerValue, summary, _Populator);

// ... then eagerly append specific fields AS CHILDREN of the lazy container
protoField.Append(_SrcFieldId, FieldValue.NewIPv4(src), in context);
protoField.Append(_DstFieldId, FieldValue.NewIPv4(dst), in context);

These eager fields coexist with the lazy populator. When the container is later materialized, the populator appends the remaining fields — the eagerly-appended fields remain in place.

Rule of thumb: Eagerly append only fields that downstream protocols need for their own parsing (addresses for checksum, protocol numbers for sub-dispatch). Everything else stays inside the lazy populator.

9.6 When NOT to Use Lazy Fields

Lazy materialization is not worth the complexity when:

The protocol is a leaf (no sub-dispatch) with very few fields (e.g., VLAN with 4 fields). ArpProtocol is the documented exception: a packet carries 12 to 14 of its 33 registered fields but it stays eager, because it never dispatches, its size is bounded by the header (at most 1028 bytes), and its packet info text reads most of the fields anyway, so a populator would parse the same bytes twice.
The fields are needed eagerly by every packet anyway (e.g., addresses for checksums).
The populator would be trivial (just appending 2-3 fields).

In these cases, use the direct Append() / AppendWithCustomText() methods on parentField without a lazy populator. The protocol container is still created but populated immediately.

9.7 Summary vs. Populator: Captured Data

Concern	Technique	What it captures	Allocation
Summary text	`LazyString.FormatLazy(tuple, static …)`	Small set of display values (ports, addresses)	One state box per packet
Populator	Pre-allocated `_Populator` delegate	Only `this` — re-parses from stored bytes	Zero per packet
Container value	`FieldValue.NewBytes(headerSlice)`	Raw header bytes (`ReadOnlyMemory<byte>`)	Zero (slice of existing buffer)

The summary state box is the only per-packet allocation. Minimize what it captures — prefer small value types (ushort, byte, IPv4Address) over full header structs.

Ethernet example — captures only the two 6-byte MAC addresses and the EtherType:

MacAddress src = MacAddress.FromBytes(span[6..12]);
MacAddress dst = MacAddress.FromBytes(span[..6]);
ushort typeOrLen = BinaryPrimitives.ReadUInt16BigEndian(span[12..14]);
LazyString summary = LazyString.FormatLazy(
    (src, dst, isEthernetII: typeOrLen >= MinEtherType),
    static s => string.Create(
        CultureInfo.InvariantCulture,
        $"{(s.isEthernetII ? "Ethernet II" : "IEEE 802.3")}, Src: {s.src}, Dst: {s.dst}"));

9.8 Pre-Computed Values Pattern (Parse → Populator)

Sometimes the lazy populator needs a derived value (e.g., a pre-computed checksum pseudo-header sum) that would require an expensive operation to re-derive inside the populator. The pattern is:

Eagerly compute and append an internal helper field during Parse() — just before the lazy container.
Read it back inside the populator via a short sibling walk.

This keeps the populator allocation-free and avoids re-computing or capturing state.

Step 1 — Eager append of internal helper field in Parse():

// Compute the pseudo-header checksum sum eagerly so the populator can read it
// without needing to walk the entire field tree or capture IP addresses.
ulong pseudoSum = InternetChecksum.ComputeIPv4PseudoHeaderSum(
    src.RawValue, dst.RawValue, ProtocolNumber, length);
parentField.Append(_PseudoHeaderSumFieldId, FieldValue.NewU64(pseudoSum), in context);

// Append the lazy container immediately after — the helper field is now 1 sibling back.
FieldValue containerValue = FieldValue.NewBytes(data).WithCustomRepresentation(summary);
parentField.AppendLazyWithCustomText(_ProtocolFieldId, containerValue, summary, _Populator);

Step 2 — Read back in the populator via MutField navigation:

private ParseResult PopulateMyProtocolFields(in MutField container)
{
    // Walk previous siblings with MutField APIs (no AsField conversion required).
    // Use materialize: false so sibling lazy containers are not populated.
    MutField current = container;
    while (current.TryGetPrev(out current))
    {
        // Read FieldId / Value of eagerly-appended helper fields only.
    }

    // Append remaining fields ...
    container.Append(_MyFieldId, FieldValue.NewU64(value));
    return 0;
}

Tree navigation API (Field and MutField are parallel; MutField returns MutField):

Method	Materialize arg?	Description
`IsValid`	—	Whether the cursor/reference is usable
`IsRoot`	—	Whether this is the packet root
`TryGetParent(out …)`	no	Parent field; false on root
`TryGetPrev(out …)` / `TryGetNext(out …)`	no	Previous / next sibling
`TryGetFirstChild(out …, bool materialize)`	required	First child; `false` skips lazy population
`TryGetLastChild(out …, bool materialize)`	required	Last child; `false` skips lazy population
`HasChildren(bool materialize)`	required	Whether direct children exist
`ChildCount(bool materialize)`	required	Number of direct children
`Children(bool materialize)`	required	Direct children enumerator
`Descendants(bool materialize)`	required	DFS pre-order descendants
`mutField.AsField()`	—	Read-only `Field` snapshot (optional)

Rules:

Read accessors and tree navigation are equivalent on Field and MutField.

Only MutField may mutate the tree (Append/Prepend/InsertAfter, lazy registration, value/custom-text setters, packet-info helpers, protocol dispatch).

The populator must not trigger lazy materialization of sibling containers — pass materialize: false and only read FieldId / Value of eagerly-appended fields.

The walk length must be bounded (e.g., maxWalk = 3) — helper fields are always placed immediately before the container, so longer walks indicate a bug.

context.RecordGroupPresence() must NOT be called inside the populator — all presence recording belongs in Parse().

There are no default values for materialize — callers must choose explicitly.

10. Protocol Dispatch Tables

10.1 Table Types

Table Type	Attribute	Key Type	Typical Use
U64	`[ProtocolTableU64]`	`ulong`	EtherType, IP protocol, port numbers
String	`[ProtocolTableString]`	`string`	Content types, URI schemes
Bytes	`[ProtocolTableBytes]`	`BytesKey`	Magic byte sequences
Bool	`[ProtocolTableBool]`	`bool`	Present/absent detection
Any	`[ProtocolTableAny]`	—	Catch-all fallback

U64 tables are the most common. Other types exist for application-layer protocols.

10.2 Declaring a Dispatch Table

The protocol that owns the table declares it with [ProtocolTableU64]:

public const string PortTableName = "udp.port";

[ProtocolTableU64(PortTableName, "UDP Port")]
private ProtocolTableId _PortTableId;

10.3 Dispatch Methods

Available on MutField:

// Most common — u64 key dispatch
parentField.TryCallNextProtocolU64(tableId, key, data, in context)

// String key dispatch
parentField.TryCallNextProtocolString(tableId, key, data, in context)

// Bytes key dispatch
parentField.TryCallNextProtocolBytes(tableId, key, data, in context)

// Bool key dispatch
parentField.TryCallNextProtocolBool(tableId, key, data, in context)

// Catch-all dispatch (any registered protocol gets called)
parentField.TryCallNextProtocolAny(tableId, data, in context)

// Direct protocol call by ID
parentField.CallProtocol(protocolId, data, in context)

All return ParseResult. Consume with exactly two methods:

TryPropagateError: true ⇔ Error — return the out result immediately.
TryGetConsumed: true ⇔ Ok (including Ok(0)); false after a prior false TryPropagateError ⇔ NotDispatched, and consumed is then 0.

10.4 Multi-Protocol Keys

When multiple protocols register at the same key, the stack creates a packet.choice container labelled "Choice: <table>: <key>" (e.g. "Choice: frame.link_type: 227"). All matching protocols are called as children of that container. The maximum consumed bytes is returned. Individual alternative errors are not fatal — only errors from the overall dispatch fail the parse.

10.5 Multi-Attempt Dispatch (Bidirectional Port Matching)

Transport protocols (UDP, TCP) use bidirectional port matching — try the lower port first, then the higher port if no match:

ushort lowPort = Math.Min(srcPort, dstPort);
ushort highPort = Math.Max(srcPort, dstPort);

ParseResult result = parentField.TryCallNextProtocolU64(_PortTableId, lowPort, payload, in context);
if (result.TryPropagateError(out ParseResult error)) { return error; }

if (!result.TryGetConsumed(out _) && lowPort != highPort)
{
    ParseResult highResult = parentField.TryCallNextProtocolU64(_PortTableId, highPort, payload, in context);
    if (highResult.TryPropagateError(out ParseResult highError)) { return highError; }
}

10.6 Identifying the Caller and Self from the Context

Every protocol receives two identity properties on context:

Property	Type	Value
`context.SelfProtocolId`	`ProtocolId`	The ID of this protocol — set by `Stack.CallProtocol` via `ParseContext.WithSelfProtocol`. A raw `Parse` / cached `ParseDelegate` does not stamp it.
`context.Dispatch.CallerProtocolId`	`ProtocolId`	The ID of the parent protocol that triggered the table dispatch. Only meaningful when `context.Dispatch.HasDispatch == true`.

When is CallerProtocolId useful?

Multiple parent protocols can share the same dispatch table. For example, both IPv4Protocol and IPv6Protocol register sub-protocols in the ip.proto table (TCP, UDP, ICMPv4/v6, etc.). TCP's Parse() needs to distinguish whether it was dispatched from IPv4 or IPv6 in order to build the correct pseudo-header for checksum validation:

public ParseResult Parse(in MutField parentField, ReadOnlyMemory<byte> data, in ParseContext context)
{
    // ...
    ProtocolId caller = context.Dispatch.CallerProtocolId;
    if (caller == _Ipv4ProtocolId)
    {
        // compute IPv4 pseudo-header
    }
    else if (caller == _Ipv6ProtocolId)
    {
        // compute IPv6 pseudo-header
    }
    else
    {
        // Fallback: walk previous siblings via IpAddressExtractor
    }
}

Important caveats:

CallerProtocolId is only valid when context.Dispatch.HasDispatch == true. When a protocol is invoked directly via CallProtocol() or as the root packet protocol, HasDispatch is false and CallerProtocolId is undefined.
context.SelfProtocolId equals default(ProtocolId) (Value=0, IsValid=true) for default(ParseContext) and inside lazy populators — since ParseContext is a readonly ref struct, it cannot be captured in closures. Do not use SelfProtocolId inside lazy populator lambdas.
Both IDs are set automatically — protocols never need to pass their own ID explicitly.

10.7 Reading the Dispatch Key from Context

Beyond the caller identity, context.Dispatch also exposes the key that matched this protocol invocation. This lets a protocol select internal configuration (e.g., a PDU definition by CAN ID) without re-parsing the header or walking the field tree.

Member	Returns	Description
`context.Dispatch.HasDispatch`	`bool`	`true` when invoked via a table lookup
`context.Dispatch.Kind`	`DispatchKeyKind`	Key type: `U64`, `String`, `Bytes`, `Bool`, `Any`, `None`
`context.Dispatch.TableId`	`ProtocolTableId`	Which dispatch table triggered this call
`context.Dispatch.TryGetU64(out ulong key)`	`bool`	Reads the key when `Kind == U64`
`context.Dispatch.TryGetString(out string? key)`	`bool`	Reads the key when `Kind == String`
`context.Dispatch.TryGetBytes(out BytesKey key)`	`bool`	Reads the key when `Kind == Bytes`
`context.Dispatch.TryGetBool(out bool key)`	`bool`	Reads the key when `Kind == Bool`

Example — config-driven signal messages (SignalMessageRegistration):

There is no meta loader protocol. SignalMessageRegistration.Register(builder) reads JSON (signal_message.config_file), compiles each message, and registers one SignalMessageProtocol instance per successful message. Each instance is wired into parent tables (can.id, lin.id, flexray.id, pdu_transport.id, udp.port, …) via dispatch_bindings. Parse needs no dispatch-key lookup — the table already selected the correct protocol instance.

See schema: NetworkInspector.Protocols/Schemas/signal-message-config.schema.json; compact examples: Schemas/examples/signal-message/ (can_and_pdu_bindings.json, multiplexed.json) and, with the settings that reference them, Schemas/examples/profile/. JSON may be loaded from a file setting (signal_message.config_file, may be empty) or from a caller-owned stream (SignalMessageRegistration.TryLoadConfig / Register(builder, stream)). Stream/object registration adds messages on top of the settings/profile file. After RegisterStandardProtocols, Register(builder, stream) registers only the extra messages (settings are already present). Load, compile, and registration warnings are returned to the caller — they are not discarded.

Best effort: Signal Message shows as much as it can. Invalid or colliding signals (and invalid mux selectors / mux groups) are skipped with a warning; siblings in the same message still register. A message is skipped only when its protocol identity (name / ui_name) or declared byte_length is unusable. Name collisions are detected and never throw. A faulty signal does not take down the rest of that message.

Compile vs runtime

JSON / compile	Runtime
`byte_length`	Compile-only floor: must be `>= RequiredByteLength` of the remaining signals after per-signal skips (max exclusive end byte over static + mux + mux-group signals). Not stored on `CompiledSignalMessage`.
Signal `name` / `ui_name`	JSON values are the registered field name / UI name. The parser does not prefix `{message}.`. Optional protocol suffixes `.raw`, `.enum`, and mux `.value` are appended to that name. Invalid or duplicate names skip that signal, as does a `value_names` object with two keys that denote the same number (for example `"1"` and `"01"`). Config providers (e.g. the FrameBuilder test bridge) may write `{message}.{signal}` as the JSON `name`.
CustomText cache	Signal bit length `<= 12` precomputes `{ui_name}: {phys:F}[ {unit}] ({raw})[ [{enum}]]` for every raw value at registration (4096 strings at 12 bits). Wider signals format on materialize.
`byte_order`	Required (`little_endian` / `big_endian`). Null/whitespace skips that signal with a warning; later signals and messages still compile.
`mux_signal` / `mux_groups`	Invalid mux selector skips mux and groups, static signals remain. `mux_groups` without `mux_signal` are ignored with a warning. Duplicate or out-of-range `mux_value` skips that group. Selector bit length `<= 8` builds a dense `muxValue → SignalInfo[]` table (256 slots at 8 bits). Wider selectors keep a linear group scan.

Hot-path contract for message Parse:

One length check: data.Length >= RequiredByteLength.
Record protocol/group presence; append lazy message container (AppendLazyWithCustomText + FieldValue.NewBytes(data[..RequiredByteLength])
- container populator).
Return RequiredByteLength — not data.Length. Inner leftover is not turned into a sibling data protocol; frame-level tails become packet.unparsed_data (§8).
Inside the container populator: unchecked ulong extract per signal (SignalMessageBits.ExtractRawUnchecked; raw is always unsigned). Byte-aligned little-endian 8/16/32/64-bit signals use a single load (BinaryPrimitives); unaligned 64-bit LE uses two ulongs (low 8 bytes + 9th byte), never UInt128 and never ulong << 64.
Append each signal under the container: FieldValue = physical F64 (raw × factor + offset). FieldInfo.FieldType is F64 for the parent signal, U64 for optional .raw, String for optional .enum. CustomText is {ui_name}: {phys:F}[ {unit}] ({raw})[ [{enum}]] (invariant culture). For bit length <= 12 that string is looked up from a table built at registration; wider signals call _BuildCustomText. Enum name lookup on materialize runs only when CustomText is not cached and signal.Enums.Kind != None, or when a .enum child is registered.

10.8 Accessing the Stack and Index from Context

Member	Type	Usage
`context.Stack`	`Stack?`	The protocol stack for this parse invocation. `null` only for `default(ParseContext)`.
`context.HasStack`	`bool`	`false` only for the default/empty context.
`context.HasIndex`	`bool`	`true` when a `PacketIndex` is attached (i.e., this parse is being indexed). Use to skip expensive index-only work during non-indexed reads.

context.Stack is needed only in rare cases — for example, building a new ParseContext(container.Packet.Stack) is not needed inside a lazy populator. LazyField.Append*() methods do not require a ParseContext:

private ParseResult PopulateMyFields(in LazyField container)
{
    container.Append(_MyFieldId, FieldValue.NewU64(value));
    return 0;
}

10.9 PDU Transport → Signal Message

Two hops. PDU Transport does not parse signals. Signal Message does not parse the PDU-Transport header. Each protocol binds to the table above it.

UDP  ──[udp.port ∈ pdu_transport.udp_dispatch_ports]──►  PDU Transport
                                                          │
                              [pdu_transport.id = dispatch_bindings.key]──►  Signal Message

Empty settings/profile is valid: PDU Transport still parses concatenated [ID][Length][Payload] tuples, but it never auto-binds on UDP (udp_dispatch_ports default empty), and Signal Message registers zero message protocols. Configure both hops or you get a raw pdu_transport.payload (or no PDU Transport at all).

UDP looks up min(src,dst) first, then max only if the first lookup did not consume. A well-known low port (for example 53) therefore wins over a listed high destination. Listing a port is parser selection on udp.port, not a socket bind.

Host code equivalent (tests / custom stacks) is SettingsManager.PreloadValue before RegisterStandardProtocols:

sm.PreloadValue(
    "pdu_transport.udp_dispatch_ports",
    SettingValue.U64Array([47290UL, 47291UL]));

Settings (profile group files)

RegisterStandardProtocols registers these settings. Persist them in the profile directory as group files ({group}.json). Paths in *_config_file are resolved relative to the settings storage path (the profile directory).

Group JSON and referenced *_config_file JSON are capped by SettingsManager.MaxConfigFileBytes (default 1 GiB = 1073741824 bytes) and SettingsManager.MaxJsonDepth (default 1024). Set both only in the SettingsManager / SettingsManagerFactory.Create constructor; they are not profile settings. Values <= 0 disable that check (CLI --settings-max-config-file-bytes / --settings-max-json-depth). Additional stream config (JsonConfigStream / Register(..., Stream)) is not size- or depth-capped by SettingsManager.

pdu_transport.json (settings group pdu_transport):

{
  "pdu_transport.udp_dispatch_ports": [47290, 47291],
  "pdu_transport.id_field_size": 4,
  "pdu_transport.length_field_size": 4,
  "pdu_transport.config_file": "pdu-names.json"
}

Setting	Default	Role
`pdu_transport.udp_dispatch_ports`	`[]`	JSON array of UDP ports (1–65535) that select PDU Transport. Empty = UDP never calls this parser. Elements outside 1–65535 are skipped with `SettingsLoadWarningKind.OutOfRange`; in-range ports still bind. Value must be a JSON array, not a single number.
`pdu_transport.id_field_size`	`4`	On-wire PDU ID width: `1`, `2`, or `4` (big-endian). Other values clamp to `4` with a warning.
`pdu_transport.length_field_size`	`4`	On-wire length width: `1`, `2`, or `4` (big-endian). Must match the capture.
`pdu_transport.config_file`	`""`	Optional JSON of PDU display names only. Empty = IDs still parse; `pdu_transport.name` is omitted. Does not register Signal Messages.

signal_message.json (settings group signal_message):

{
  "signal_message.config_file": "signal-messages.json",
  "signal_message.show_raw": false,
  "signal_message.show_enum": false,
  "signal_message.max_enum_values": 4096
}

Setting	Default	Role
`signal_message.config_file`	`""`	JSON of message protocols + `dispatch_bindings`. Empty = no Signal Message dissectors.
`signal_message.show_raw`	`false`	Append `{signal}.raw` (U64) under each signal.
`signal_message.show_enum`	`false`	Append `{signal}.enum` when a `value_names` hit occurs.
`signal_message.max_enum_values`	`4096`	Cap on `value_names` entries per signal. Range 1 or more; `0` is refused (reported as `OutOfRange`) and the default stays.

External JSON (not the group files)

pdu-names.json — schema Schemas/pdu-transport-config.schema.json. Names only:

{
  "pdus": [
    { "id": 32, "name": "BenchPdu" }
  ]
}

id must equal the on-wire PDU ID (same integer the Signal Message binding uses as key). A shipped example with comments is Schemas/examples/pdu-transport/pdu_names.json; the matching settings group files (pdu_transport.json, signal_message.json) are in Schemas/examples/profile/, with the referenced files in a configs/ folder of the profile.

signal-messages.json — schema Schemas/signal-message-config.schema.json. This is the dispatch configuration:

{
  "messages": [
    {
      "name": "fixture_message",
      "ui_name": "Fixture PDU",
      "byte_length": 4,
      "dispatch_bindings": [
        { "table": "pdu_transport.id", "key": 32 }
      ],
      "signals": [
        {
          "name": "fixture_message.EngineRpm",
          "ui_name": "Engine RPM",
          "start_bit": 0,
          "bit_length": 16,
          "byte_order": "little_endian",
          "factor": 0.25,
          "offset": 100.0
        }
      ]
    }
  ]
}

The binding table must be pdu_transport.id (not udp.port) when the payload sits inside PDU Transport. key is the PDU ID. The same message may list several bindings (CAN + PDU Transport, etc.).

Additional JSON by hand

Settings/profile files may be empty. Extra documents merge on top:

PduTransportRegistration.Register(builder, stream, warnings) — extra PDU names (same ID overwrites the file).
SignalMessageRegistration.Register(builder, stream) — extra messages. After RegisterStandardProtocols this does not reload the file; it only adds the stream.

Do not call PduTransportRegistration.Register(builder, stream) after RegisterStandardProtocols: that would register a second pdu_transport protocol. Put extra names in the file, or use the stream overload instead of the standard PDU registration.

Field tree (sibling dispatch)

PDU Transport and Signal Message both dispatch on parentField, same as Ethernet / IPv6 / UDP.

… ipv4 …
├── udp
│   ├── udp.srcport / udp.dstport / …
├── pdu_transport
│   └── pdu_transport.pdu
│       ├── pdu_transport.id
│       ├── pdu_transport.length
│       ├── pdu_transport.name          // from pdu-names.json when the ID hits
│       └── pdu_transport.payload       // only if no Signal Message consumed the payload
└── fixture_message                     // lazy container; sibling of pdu_transport
    ├── fixture_message.EngineRpm       // physical F64; children of the message, not of pdu
    └── fixture_message.Thr

Signals are never children of pdu_transport or pdu_transport.pdu. Optional {signal}.raw / {signal}.enum hang under the signal. Mux-group signals hang under the mux field.

Misconfiguration (silent miss, not a crash)

Symptom	Cause
No `pdu_transport` in the tree	`pdu_transport.udp_dispatch_ports` is empty, no listed port matches, the value is not a JSON array (TypeMismatch keeps `[]`), leftover `pdu_transport.udp_dispatch_port` is ignored, or UDP already consumed `min(src,dst)` (for example source `53`).
`pdu_transport.payload` instead of the message	`dispatch_bindings.table` is not `pdu_transport.id`, or `key` ≠ on-wire PDU ID, or `id_field_size` / `length_field_size` do not match the capture (wrong ID read).
Message protocol missing from the stack	`signal_message.config_file` empty or JSON failed to load (inspect `RegisterStandardProtocols` warnings).
`pdu_transport.name` missing	`pdu_transport.config_file` empty or ID not listed. Parsing and dispatch still work.

11. Heuristic Protocol Tables

11.1 Purpose

When no dispatch table key matches (e.g., unknown port), heuristic tables provide a content-based fallback. Each registered heuristic parser inspects the payload bytes and returns true if it recognizes the data.

11.2 IHeuristicParser Interface

public interface IHeuristicParser
{
    bool Test(ReadOnlyMemory<byte> data);
    ProtocolId ProtocolId { get; }
    string Name { get; }
    string UiName { get; }
    string? Description => null;
}

11.3 Declaring a Heuristic Table

// In StackBuilder registration:
HeuristicProtocolTableId heuristicTableId =
    builder.RegisterHeuristicProtocolTable(protocolId, "tcp.heuristic", "TCP Heuristic");
builder.RegisterHeuristicParser(heuristicTableId, new HttpHeuristicParser());
builder.RegisterHeuristicParser(heuristicTableId, new TlsHeuristicParser());

11.4 Dispatch with Heuristic Fallback

// Try table-based dispatch first
ParseResult result = parentField.TryCallNextProtocolU64(_PortTableId, port, payload, in context);
if (result.TryPropagateError(out ParseResult error)) { return error; }

// If no table match, try heuristic fallback. Only a false TryGetConsumed after
// a false TryPropagateError is a miss — Ok(0) means a protocol ran and consumed
// zero bytes; do not fall through.
if (!result.TryGetConsumed(out _))
{
    result = parentField.TryCallHeuristicProtocol(_HeuristicTableId, payload, in context);
    if (result.TryPropagateError(out ParseResult heuristicError)) { return heuristicError; }
}

TryCallHeuristicProtocol() scans registered heuristic parsers and calls the first matching protocol. The order of heuristic parsers determines priority.

11.4a UDP heuristic table

UDP owns the heuristic table udp.heuristic (UdpProtocol.HeuristicTableName), registered empty by RegisterStandardProtocols. UdpProtocol finds it by name in OnStartCustom (like TCP does with tcp.heuristic) and calls it only after no port parser consumed the payload: first udp.port with the lower port, then with the higher port, then the heuristic table. The decision is stateless (UDP keeps no connection state), so a re-parse takes the same path; a parser that keeps its own cross-packet state follows §2 itself.

A heuristic parser gets no dispatch key. It reads the datagram's ports from UdpPortCache.TryGet(packetId, out src, out dst), which UdpProtocol publishes on the calling thread before it dispatches and withdraws when the datagram is done; the packet-id guard makes a stale entry of another packet harmless. A parser reached from TCP reads the same from TcpPortCache; on a replay TCP publishes the ports again, so a decision that depends on them is the same as on the first parse.

11.5 Performance Consideration

Heuristic parsing inspects payload bytes — it is more expensive than key-based dispatch. Use it only as a fallback after table-based dispatch fails:

1. Try port/key-based dispatch (O(1) with cache)
2. If no match → Try heuristic dispatch (O(n) with n=number of heuristic parsers)
3. If still no match → Leave payload undissected

12. Dispatch Cache Optimization

The generic TryCallNextProtocolU64() path involves table lookup, multi-protocol handling, and virtual dispatch. For performance-critical protocols, build a dispatch cache in OnStartCustom() to bypass the generic path entirely.

12.1 Dense Cache (Small domains: u8 → 256 entries)

For key domains that fit in a small array (8-bit IP protocol field = 256 entries):

// ~2 kB array — fits in L2 cache
private ParseDelegate?[] _IpProtoDelegateCache = [];

partial void OnStartCustom(Stack stack)
{
    _IpProtoDelegateCache = stack.BuildU64DelegateCache(_IpProtoTableId, 256);
}

Dispatch is a single array lookup — O(1) with zero indirection:

private ParseResult DispatchIpProtocol(
    in MutField parentField, byte protocol, ReadOnlyMemory<byte> payload, in ParseContext context)
{
    ParseDelegate? fastParse = _IpProtoDelegateCache.Length > 0 ? _IpProtoDelegateCache[protocol] : null;
    return fastParse is not null
        ? fastParse(in parentField, payload, in context)              // Direct delegate call
        : parentField.TryCallNextProtocolU64(_IpProtoTableId, protocol, payload, in context);  // Fallback
}

Non-null entry: single protocol registered → direct delegate call.
Null entry: zero or multiple protocols → fall back to full table dispatch.

12.2 Sparse Cache (Large domains with few entries: u16 → 4–6 entries)

For key domains where only a handful of values are registered (EtherType = 65 536 possible values, but only 4–6 active):

// Tiny array — all entries fit in L1 D-cache
private (ulong Key, ParseDelegate Parse)[] _EtherTypeSparseDelegateCache = [];

partial void OnStartCustom(Stack stack)
{
    _EtherTypeSparseDelegateCache = stack.BuildU64SparseDelegateCache(_EtherTypeTableId);
}

Dispatch is a linear scan — faster than dictionary hash for 4–6 entries:

private ParseResult DispatchEtherType(
    in MutField parentField, ulong etherType, ReadOnlyMemory<byte> payload, in ParseContext context)
{
    foreach ((ulong key, ParseDelegate parse) in _EtherTypeSparseDelegateCache)
    {
        if (key == etherType)
        {
            return parse(in parentField, payload, in context);        // Direct delegate call
        }
    }

    return parentField.TryCallNextProtocolU64(_EtherTypeTableId, etherType, payload, in context);
}

12.3 Choosing a Cache Strategy

Domain size	Registered entries	Strategy	Method
≤ 256 (u8)	Any	Dense array	`BuildU64DelegateCache(table, 256)`
≤ 65 536 (u16)	1–10	Sparse array	`BuildU64SparseDelegateCache(table)`
Large / dynamic	Many	No cache	`TryCallNextProtocolU64()` directly

13. Index Groups & PacketIndex

13.1 How the PacketIndex Works

The PacketIndex is a cross-packet structure that tracks which fields are present in which packets using Roaring Bitmaps. This enables fast filtering: "show me only packets that contain field tcp.flags.syn" becomes a bitmap intersection.

Without index groups, each of the ~84+ protocol fields would need its own bitmap. With groups, fields that always appear together share one bitmap — reducing memory and lookup overhead by ~10×.

13.2 Index Group Rules

Always-present fields share the protocol's main group. If the protocol is present, these fields are guaranteed to exist.

private const string UdpIndexGroup = "udp";

[U64Field("udp.srcport", "Source Port", IndexGroup = UdpIndexGroup)]
[U64Field("udp.dstport", "Destination Port", IndexGroup = UdpIndexGroup)]
[U64Field("udp.length", "Length", IndexGroup = UdpIndexGroup)]
[U64Field("udp.checksum", "Checksum", IndexGroup = UdpIndexGroup)]

Optional / conditional fields each get their own group. They may or may not appear depending on packet content.

[BytesField("udp.payload", "Payload", IndexGroup = "udp.payload")]
[StringField("udp.checksum.status", "Checksum Status", IndexGroup = "udp.checksum.status")]

Co-occurring optional fields may share a group if they always appear together.

// Both always appear together → share one group
private const string FcsIndexGroup = "eth.fcs";
[U64Field("eth.fcs", "FCS", IndexGroup = FcsIndexGroup)]
[StringField("eth.fcs.status", "FCS Status", IndexGroup = FcsIndexGroup)]

Mutually exclusive fields each need their own group.

// Ethernet II vs IEEE 802.3 — only one appears per packet
[U64Field("eth.type", "Type", IndexGroup = "eth.type")]
[U64Field("eth.len", "Length", IndexGroup = "eth.len")]

13.3 Recording Presence in Parse()

All context.RecordGroupPresence() calls must happen eagerly in Parse() — never inside a lazy populator. The index is populated during the initial parse pass; lazy populators run much later (if at all).

public ParseResult Parse(in MutField parentField, ReadOnlyMemory<byte> data, in ParseContext context)
{
    // Always record
    context.RecordProtocolPresence(_ProtocolId);
    context.RecordGroupPresence(_UdpGroupId);

    // ... parse header ...

    // Conditionally record optional groups
    if (_VerifyChecksum && checksum != 0)
    {
        context.RecordGroupPresence(_UdpChecksumStatusGroupId);
    }
    if (payloadLen > 0)
    {
        context.RecordGroupPresence(_UdpPayloadGroupId);
    }
    // ...
}

13.4 Generated Index Group IDs

The Source Generator creates one IndexGroupId field per unique IndexGroup value:

// Auto-generated from IndexGroup strings in field attributes:
private IndexGroupId _UdpGroupId;              // "udp"
private IndexGroupId _UdpPayloadGroupId;       // "udp.payload"
private IndexGroupId _UdpChecksumStatusGroupId;// "udp.checksum.status"

// Auto-generated in RegisterFields():
_UdpGroupId = builder.GetOrCreateIndexGroup("udp");
_UdpPayloadGroupId = builder.GetOrCreateIndexGroup("udp.payload");
_UdpChecksumStatusGroupId = builder.GetOrCreateIndexGroup("udp.checksum.status");

13.5 Impact

Approach	Bitmaps for standard stack
Per-field bitmaps	~84
With index groups	~8–10

13a. Field Alias Groups (Any-Match Names)

Field alias groups are metadata-only registries that expose any-match semantic names (e.g., eth.addr, ip.addr, ipv6.addr, udp.port, tcp.port) without adding any extra field nodes to the parse tree. They replace the older pattern of physically appending a duplicate field (eth.addr appended as a child of both eth.dst and eth.src) which doubled allocations and confused enumeration.

13a.1 Core Rules (Mandatory)

No physical duplicate field is registered or appended for any-match names. Do not declare [MacField("eth.addr") private FieldId _AddrFieldId; — this is the breaking change locked by Step 4 of the alias plan.
Alias names are resolved separately, but never share a name with a protocol or a field. stack.GetFieldId("eth.addr") returns null by design. Always use stack.GetFieldAliasGroupId("eth.addr") to resolve alias names. The separate lookup keeps indexing and per-packet field-lookup paths on canonical fields only. The names of protocols, fields and aliases form one conflict namespace: the first registration keeps a name, and a later RegisterProtocol, RegisterField or RegisterFieldAliasGroup with the same name throws a DuplicateNameRegistrationException whose HolderKind is RegistrationHolderKind.Protocol, .Field or .FieldAliasGroup. The one exception is the container field of a protocol, which may carry the protocol's own name (protocol someip, field someip).
Alias members may have mixed FieldType values. The registry stores member FieldIds only; consumers query each member by its own type.

13a.2 Manual Registration Pattern

Alias groups are registered manually inside RegisterFieldsCustom (the partial that the generator invokes after all attribute fields are registered, so member FieldIds are populated):

[Protocol("eth", "Ethernet II")]
public sealed partial class EthernetProtocol : IProtocol
{
    [MacField("eth.dst", "Destination", IndexGroup = EthIndexGroup)]
    private FieldId _DstFieldId;

    [MacField("eth.src", "Source", IndexGroup = EthIndexGroup)]
    private FieldId _SrcFieldId;

    // Holds the alias group ID returned from manual registration.
    private FieldAliasGroupId _AddrAliasGroupId;

    partial void RegisterFieldsCustom(IStackBuilder builder, ProtocolId protocolId)
    {
        // Members must already be registered by the time this is called.
        _AddrAliasGroupId = builder.RegisterFieldAliasGroup(
            protocolId, "eth.addr", null, [_DstFieldId, _SrcFieldId]);
    }
}

The same pattern applies to ip.addr (→ { ip.src, ip.dst }), ipv6.addr, udp.port (→ { udp.srcport, udp.dstport }), tcp.port, and any future any-match name. Note that the alias name udp.port is independent of the protocol-table name udp.port (UDP demux) — protocol tables keep their own registry, while protocol, field and alias names are one conflict namespace (see 13a.1).

13a.3 Consumer Pattern (Enumeration via Alias)

FieldAliasGroupId? aliasId = stack.GetFieldAliasGroupId("eth.addr");
if (aliasId is { } id)
{
    FieldAliasGroupInfo info = stack.GetFieldAliasGroup(id)!;
    foreach (FieldId memberId in info.Members.Span)
    {
        FieldLookupCookie cookie = FieldLookupCookie.Start;
        // materialize: true — consumer enumeration must see lazy member fields too
        while (packet.TryGetNextField(memberId, ref cookie, out Field memberField, materialize: true))
        {
            ref readonly FieldValue value = ref memberField.Value; // process value per member type
        }
    }
}

13a.4 Ethernet I/G + L/G Bits via CustomText

The legacy eth.dst.ig, eth.dst.lg, eth.src.ig, eth.src.lg bool fields are removed. The I/G and L/G semantics are now exposed as CustomText on eth.dst and eth.src via EthernetProtocol.FormatMacAddressBits:

I/G	L/G	CustomText
false	false	`Unicast, Globally Unique`
false	true	`Unicast, Locally Administered`
true	false	`Multicast, Globally Unique`
true	true	`Multicast, Locally Administered`

Tests assert this via AssertDisplayText(stack, packet, "eth.dst", "…").

13b. Eager / Lazy Field Layout

Every in-scope protocol (eth, ip, ipv6, udp, tcp, tls, dtls) splits its work between eager fields appended during Parse() (small, summary-critical, or required by downstream protocols) and lazy fields materialised by a single per-protocol populator when the protocol container is first accessed. All decoded fields are direct children of the protocol container — there is no nested *.hdr intermediate container.

13b.1 Required Layout

Protocol	Eager (Parse)	Lazy populator children (direct under protocol container)
`eth`	—	`eth.dst`, `eth.src` (+ ig/lg CustomText), `eth.type` / `eth.len`, padding, …
`ip`	`ip.src`, `ip.dst` (required by transport protocols)	version, hlen, dscp/ecn, flags, ttl, proto, checksum, options
`ipv6`	`ipv6.src`, `ipv6.dst` (required by transport protocols)	version, tclass, flow, payload length, next header, hop limit, extension headers
`udp`	—	`udp.srcport`, `udp.dstport`, checksum (+status), length, payload
`tcp`	—	ports, checksum, seq/ack, hdr_len, flags (+ sub-flags), window, urgent ptr, options, len, payload
`tls`	—	per record: `tls.record` (content_type, version, length) followed by handshake/alert as siblings
`dtls`	—	per record: `dtls.record` (content_type, version, epoch, seq, length) followed by handshake as siblings; the handshake header is decoded only for epoch 0, and a connection ID record (type 25) has no `length` field and ends the walk
`someip`	the message tree: container, the ten header fields, `msgtype` children, names, diagnostics, the service node and the method node (filter names from the description; exact groups) and the raw `someip.payload` of every payload no other parser took (the packet index must see every group at parse time, and the fields are cheap); for a discovery message the eager scan of its payload	`someip_sd` (flags, entries, options, their diagnostics), expanded from the stored payload bytes; the scan and the population are two visitors of one walk, so the groups the scan records equal the fields the population appends. the method node of a configured member (typed parameters, array elements, TLV members, unions, bits, diagnostics), decoded from the stored payload bytes by the populator of its member direction; the group of the member direction is a superset (13c.3)
signal message (`SignalMessageProtocol`)	—	static signals (physical F64); mux container + `.value` + matching mux-group signals; optional `.raw` / `.enum`

13b.2 Implementation Sketch

// One populator delegate, captured once in OnStartCustom — zero per-packet allocation.
private LazyPopulator _Populator = null!;

partial void OnStartCustom(Stack stack) => _Populator = PopulateFoo;

private ParseResult PopulateFoo(in LazyField container)
{
    // Append all decoded fields directly under the protocol container.
    // For ip/ipv6: src/dst stay in Parse (eager); everything else goes here.
    container.Append(_VersionFieldId, FieldValue.NewU64(version));
    container.Append(_HlenFieldId, FieldValue.NewU64(hlen));
    // … remaining fields …
    return 0;
}

13c. Mandatory Eager Rules

The presence index is the single reliable filtering layer — there is no value-cache fallback. A filter that asks "does this packet contain field tls.sni / dns.opt / http.host?" must be answerable from the index alone, without materialising the lazy descriptive field tree. The five rules below are binding for every current and future protocol; they make that guarantee hold by construction. They are enforced by EagerIndexGroupRegistrationTests and the eager-dispatch tests.

13c.1 All sub-protocol dispatch is eager

All dispatch to sub-protocols must occur in Parse() — never inside a lazy populator. A populator builds descriptive fields for one protocol; it must not decide or invoke the next protocol. Dispatch governs which protocols are present in a packet, and presence must be known the moment the packet is finalised, not deferred until (or if) a container is later expanded.

13c.2 The index is finalised when the packet is finalised

A lazy populator must never mutate the index, record presence, or dispatch. It only builds the field tree. This is enforced by the type system: a populator receives a LazyField / MutField cursor with no ParseContext, so it has no API surface to touch the index. Everything index-affecting happens during the eager Parse() pass.

13c.3 Every emittable group is recorded eagerly — no false positives, no false negatives

For each packet, the index must record a group if and only if materialisation would emit at least one field of that group. Two failure modes are equally forbidden:

False negative — a lazily-materialised field whose group was not recorded in Parse(). The filter would wrongly exclude a matching packet.
False positive — a group recorded in Parse() for which no field is ever emitted. The filter would wrongly include a non-matching packet.

Unconditional groups are recorded directly. Conditional and content-dependent groups must be decided in Parse(), even when the corresponding fields are deferred to the populator. This frequently means Parse() must perform an eager payload scan that duplicates the populator's parsing logic, with each eager guard mirroring its populator's emission guard exactly. Duplicated evaluation is the accepted price of the lazy model. For example, TLS records all twelve tls.* groups by walking records/handshakes/extensions in Parse() with the same length and bounds checks the populator uses; DNS walks the question and resource-record sections to record dns.opt, dns.ds, dns.rrsig, dns.nsec, dns.dnskey, etc.

A field that is declared but never emitted (dead field) must not have its group recorded — doing so would be a false positive.

One documented deviation: the SOME/IP payload groups are supersets. A SOME/IP message of a configured member records the index group of its member direction (someip.payload.<service>.v<major>.<member>[.req|.resp]) although the parameters are decoded only when the payload container is read. Deciding exactly which parameters exist would mean running the whole decoder in Parse(), which is the cost the lazy container exists to avoid. Two cases leave a recorded group without a field: a TLV member that the message does not carry, and a payload cut off before its parameters. There is no false negative (a message that holds a parameter is always in the group), and the false positives are verified on the tree by the filter, exactly as a filter verifies every candidate. The generic payload fields share the group someip.payload, which is recorded for every message that has a payload container. The index tests of SOME/IP assert exactness for fully decodable messages and assert the superset explicitly for the two cases above. The service node and the method node are the exception to the exception: both are appended eagerly on every path of a message whose header resolves them, and their groups (someip.payload.<service>.v<major> and someip.payload.<service>.v<major>.<member>) are recorded exactly when the nodes are appended, so a filter on a service or a method is answered by the index without materializing any payload. The decoded values hang lazily below the method node, and the group names always start with someip.payload., whatever field prefix the user sets.

13c.4 Prefer eager fields; defer only with enough candidates

Append fields eagerly by default. Introduce a lazy populator only when a protocol has ≥ 2 (ideally ≥ 3) lazy-field candidates; for a single deferrable field the populator's per-access overhead is not worth it — append eagerly and omit the populator. (Recording the group eagerly per §13c.3 is required either way.)

13c.5 Follow the approved eager/lazy cut

The authoritative eager/lazy split for the in-scope protocols is the table in §13b.1 and is binding:

ip / ipv6 keep only *.src and *.dst eager, because transport demux reads them during its own Parse(); every other field is lazy.
tcp.flags (and all other TCP fields) are lazy.
eth, udp, tls, dtls are fully lazy — no eager descriptive fields — while still recording their presence and applicable groups eagerly per §13c.3.

Any new protocol must document its cut as a row consistent with §13b.1 before implementation.

14. String Handling & Display Text

14.1 Priority Order

String allocation is the primary source of GC pressure in protocol parsing. Follow this strict priority order:

Priority	Pattern	Allocation	When to use
1	`DisplayTables.Get*(value)`	Zero	Value from precomputed static table
2	Literal string `"constant text"`	Zero	Well-known constant text
3	`$"text {var}"` in lazy populator	Deferred	Dynamic text inside a populator (runs only on access)
4	`LazyString.FormatLazy(tuple, static …)`	One state box per packet	Summary text, packet info
5	`$"text {var}"` in Parse()	One string per packet	Avoid — only if no other option

14.2 Precomputed Display Tables

DisplayTables in Helpers/DisplayTables.cs provides zero-allocation lookups for fixed domains:

// Dense array — entire u8/u16 domain precomputed
string dscpText = DisplayTables.GetDscpDisplayText(dscp);         // 64 entries, 6-bit
string ecnText = DisplayTables.GetEcnDisplayText(ecn);            // 4 entries, 2-bit
string protoText = DisplayTables.GetIpProtocolDisplayText(proto);  // 256 entries, 8-bit
string etherText = DisplayTables.GetEtherTypeDisplayText(type);    // 65536 entries, 16-bit
string pcpText = DisplayTables.GetVlanPriorityDisplayText(pcp);    // 8 entries, 3-bit

// Pre-formatted hex — avoids string.Format / interpolation
string hexU8 = DisplayTables.FormatHexU8(value);                   // 256 entries: "0x00".."0xff"
string hexU16 = DisplayTables.FormatHexU16(value);                 // 65536 entries: "0x0000".."0xffff"

// Precomputed structural text
string hdrLen = DisplayTables.GetHeaderLengthDisplayText(len);     // "20 bytes", "40 bytes", ...

Adding new tables: For any new domain field, add a table in DisplayTables.cs:

Full-domain arrays (u8 = 256, u16 = 65536) for key identifier domains.
Partial arrays for structural fields (header lengths 0–60).
Named entries for well-known values; fill remaining with numeric string representation.

14.3 Deferred text with LazyString.FormatLazy

There are three forms of display text, picked by when the text is needed:

Need	Form	Cost
A fixed literal	`new LazyString("8 bytes")`	Nothing: no deferral needed
Text built from runtime values, read rarely (summary, packet info)	`LazyString.FormatLazy(tuple, static s => string.Create(…))`	One state box at parse time, one string on first read
Text the field tree stores right away (eager custom text)	`string.Create(CultureInfo.InvariantCulture, $"…")`	One string at parse time

// Deferred: the value tuple is boxed once; the lambda is static, so no closure object exists.
LazyString summary = LazyString.FormatLazy(
    (srcPort, dstPort),
    static s => string.Create(
        CultureInfo.InvariantCulture,
        $"User Datagram Protocol, Src Port: {s.srcPort}, Dst Port: {s.dstPort}"));

// Eager: when the container stores the text immediately anyway.
container.AppendWithCustomText(ids.Mss, FieldValue.None,
    string.Create(CultureInfo.InvariantCulture, $"Maximum Segment Size: {value} bytes"));

Why the lambda is static: a lambda that captures a local allocates a closure object on every call. A static lambda captures nothing; everything it needs travels in the state argument. Why the state is a tuple: the state is boxed exactly once into the deferred text, whatever its size, and the tuple element names give the lambda readable access. Why string.Create(CultureInfo.InvariantCulture, …): it writes the interpolation straight into the result string (no scratch buffer, safe to call from inside another deferred text) and it never depends on the thread culture, so a de-DE machine shows the same decimal point as an invariant one.

Use FormatLazy for:

Protocol summary text (displayed in the packet list / tree root).
Packet info text (displayed in the info column).
Any text containing runtime values that is computed per packet.

Never pass a prebuilt lazy text as the last argument of AppendWithCustomText (or its prepend/insert/lazy siblings): that text is built in the caller and cannot be skipped on a skip-mode parse. Pass the fragments instead (m.AppendWithCustomText(id, value, "Maximum Segment Size: ", value, " bytes")). Analyzer NIGEN015 reports the mistake.

When a hex fragment must print exactly eight uppercase digits, use new UpperHex8(value) (an interpolation argument and a custom-text fragment alike); DisplayTables.FormatHexU32 is lowercase and carries a 0x prefix.

14.4 LazyString

NetworkInspector.Core.Fields.LazyString defers string evaluation until first access:

// Static text — zero allocation
new LazyString("8 bytes")

// Deferred — evaluated once on first access, then cached in place via Interlocked.CompareExchange
LazyString.FormatLazy(len, static l => string.Create(CultureInfo.InvariantCulture, $"Options: ({l} bytes)"))

// Absent text
LazyString.Empty

The cache lives in the storage location that was read. LazyString is a readonly struct, so evaluating a copy caches into the copy; read through the stored field, array element, or the FieldValue that holds it.

14.5 AppendWithCustomText vs. Append

Append(fieldId, value) — uses the default formatting for the field type.
AppendWithCustomText(fieldId, value, displayText) — overrides the display text.

Use AppendWithCustomText when you need a human-readable label that differs from the raw value representation (e.g., "TCP (6)" instead of just "6").

A custom text is the whole display line. The default line of a field (Field.ToString(), MutField.ToString()) is the custom text alone when one is set, even an empty one; a protocol that wants the label on the line must put it into the text ("Maximum Segment Size: ", then the value), because a custom text without a label is shown as a bare line in the Text export and in Field.ToString(). A field without a custom text shows its UI name for a container (value None) and UI name: value for every other field. A custom representation (§ 14.6) replaces only the value part: UI name: representation. A representation on a value of type None is not part of the line (the field has no value part); give such a field a custom text instead.

Custom text is final at append. Field observers and value caches receive a field when it is appended; they never see a later change. Changing the text afterwards (SetCustomText, AppendCustomText, ClearCustomText) is therefore discouraged: a tree-free export would show the old text while the field tree shows the new one. After the packet is sealed the three methods throw InvalidOperationException — a sealed packet is read by other threads, and the text store is not synchronized (a lazy populator that decodes a payload when a reader asks for it runs after the seal, so it is covered too). A change before the seal still works, but observers do not see it. When the text depends on data decoded later (an element count), append a deferred text over a small state object and complete the object before the parse returns:

ArrayText text = new(element);                                   // sealed class: Count property, ToString() = the text
MutField node = parent.AppendWithCustomText(id, FieldValue.None, text);   // formatted with ToString() on first read
// … decode the elements …
text.Count = count;                                              // final before the populator returns

When the text depends on a value that is cheap to read first (a union selector), read it before the append and append the final text. Both patterns are used by the SOME/IP payload decoder.

14.6 WithCustomRepresentation

For container fields where the FieldValue stores raw bytes but the UI should display a summary:

FieldValue headerValue = FieldValue.NewBytes(data[..20])
    .WithCustomRepresentation(new LazyString("20 bytes"));

14.7 Anti-Patterns

// ❌ BAD — string interpolation for a fixed domain (use DisplayTables)
$"0x{etherType:x4}"

// ❌ BAD — string.Format anywhere on the hot path
string.Format("Protocol: {0}", protocol)

// ❌ BAD — ToString() on numeric types in the hot path
checksum.ToString("x4")

// ❌ BAD — per-packet string allocation in Parse() for display text
string summary = $"UDP, Src: {srcPort}, Dst: {dstPort}";  // Allocates every packet

// ✅ GOOD — deferred with FormatLazy (allocates the string only if accessed)
LazyString summary = LazyString.FormatLazy(
    (srcPort, dstPort),
    static s => string.Create(CultureInfo.InvariantCulture, $"UDP, Src: {s.srcPort}, Dst: {s.dstPort}"));

14.8 Precomputed Flags Formatters

Every protocol that has a flags container field must provide a dedicated *FlagsFormatter class with a precomputed lookup table indexed by the packed flag bits. This eliminates all per-packet allocations for the common case of displaying which flags are set.

Two display patterns

Field type	Output format	Example
`NoneField` (grouping container)	`[FLAG1, FLAG2]`	`[SYN, ACK]`
`U64Field` (flags register with numeric value)	`0xNN [FLAG1, FLAG2]`	`0x12 [SYN, ACK]`

Both patterns use [None] when no flags are set.

Canonical reference implementation

Tcp/TcpFlagsFormatter.cs is the canonical example. Follow its exact structure:

// ✅ Canonical pattern — zero-allocation precomputed table
internal static class TcpFlagsFormatter
{
    private static readonly string[] FlagsTable = BuildFlagsTable();

    internal static string Format(ushort flags) => FlagsTable[flags & 0x1FF];

    private static string[] BuildFlagsTable()
    {
        string[] table = new string[512]; // 2^N entries, N = number of flag bits
        for (int i = 0; i < 512; i++)
        {
            table[i] = BuildFlagString((ushort)i);
        }
        return table;
    }

    private static string BuildFlagString(ushort flags)
    {
        if (flags == 0) return "[None]";

        // ... compute totalLen ...

        return string.Create(totalLen, flags, static (chars, f) =>
        {
            // Write "[FLAG1, FLAG2]" into chars without any allocation
        });
    }
}

Key design rules

Static read-only table — built once in the static initializer, never mutated.
Direct array index — use FlagsTable[key] without bounds checks in release (array size == full domain of the key).
string.Create with a value-type state — avoids boxing; pass a struct or primitive (e.g. ushort flags) as the state argument.
[None] sentinel — always return the pre-allocated literal "[None]" for the zero-flags case (no string.Create call).
Compact key extraction — when the flag bits are not contiguous (e.g. DNS), pack them into a synthetic byte key with explicit shift-and-OR logic and document each mapping in a comment.
Prefix token for frame-type flags — when a flag set always implies a specific frame type (e.g. CAN FD's "FD" prefix, CAN XL's "XLF" prefix), embed it as the first token in every non-[None] table entry.
U64Field formatters include the hex prefix — the formatter returns "0x12 [SYN, ACK]" directly (consistent with DisplayTables.FormatHexU8/U16 conventions) so the call site does not need to concatenate.
NoneField container formatters return brackets only — "[SYN, ACK]" or "[None]". For NoneField containers, the hex prefix has no meaning.
DNS / mixed-field case — when a 16-bit word mixes boolean flags with multi-bit numeric fields (opcode, RCODE), build a synthetic 8-bit key from only the boolean bits, build a 256-entry table, and return the brackets string only. The caller concatenates hex + brackets:
```
LazyString.FormatLazy(flags, static f => string.Create(CultureInfo.InvariantCulture,
    $"{DisplayTables.FormatHexU16(f)} {DnsFlagsFormatter.Format(f)}"))
```

Formatter placement

Place the formatter in the same sub-folder as the protocol:

Protocol	Formatter file
TCP	`Tcp/TcpFlagsFormatter.cs`
CAN / CAN FD / CAN XL	`Can/CanFlagsFormatter.cs`
IPv4	`IPv4/IPv4FlagsFormatter.cs`
FlexRay	`FlexRayFlagsFormatter.cs`
ICMPv6 NDP	`Icmpv6/Icmpv6NdpFlagsFormatter.cs`
HTTP/2	`Http2/Http2FlagsFormatter.cs`
DNS	`Dns/DnsFlagsFormatter.cs`

15. Error Handling

15.1 ParseResult Encoding

ParseResult is a compact 4-byte struct:

Encoded value	Meaning
`> 0`	Success: consumed bytes = value − 1
`≤ 0`	Error: details in thread-local `ParseError.LastError`

Implicit conversions make it transparent:

return 42;                          // Success: consumed 42 bytes
return ParseError.InvalidData(...); // Error: stored in TLS

15.2 Never Throw on the Hot Path

Exceptions are forbidden during parsing. Use ParseResult for all error propagation. Exceptions are reserved for registration time only.

// ✅ GOOD — ParseResult with implicit conversion
if (data.Length < HeaderSize)
{
    return ParseError.InsufficientDataWithInfo(ProtocolName, HeaderSize, (ulong)data.Length);
}

// ✅ GOOD — propagate dispatch errors
ParseResult result = parentField.TryCallNextProtocolU64(_TableId, key, payload, in context);
if (result.TryPropagateError(out ParseResult error))
{
    return error;
}

// ❌ BAD — exception on the hot path
throw new InvalidOperationException("Invalid header");

// ❌ BAD — try/catch for flow control
try { ParseHeader(span); }
catch (IndexOutOfRangeException) { return ParseError.InsufficientData(ProtocolName); }

15.3 ParseError Factory Methods

Factory	Usage
`ParseError.InsufficientDataWithInfo(proto, expected, actual)`	Data too short for header/payload
`ParseError.InvalidData(proto, message)`	Structurally invalid data (bad version, corrupt length)
`ParseError.Custom(proto, message)`	Protocol-specific errors
`ParseError.InternalError(proto, message)`	Unexpected programming error

Always pass ProtocolName (the auto-generated constant) as the first argument.

15.4 Error Propagation from Dispatch

All TryCallNextProtocol*() return ParseResult with three named variants: Ok (consumed bytes, including Ok(0)), NotDispatched (table present, no protocol for this key — not an error and not Ok(0)), and Error. Consume with two methods in this order: TryPropagateError first, then TryGetConsumed. After a false TryPropagateError, a false TryGetConsumed is exactly NotDispatched.

ParseResult result = parentField.TryCallNextProtocolU64(_TableId, key, payload, in context);
if (result.TryPropagateError(out ParseResult error))
{
    return error;
}

if (!result.TryGetConsumed(out int consumed))
{
    // no protocol for this key — try another key or stop
}
else
{
    // Ok, including Ok(0) when consumed == 0
}

A dispatch method whose key type differs from the key type of the table (TryCallNextProtocolString on a numeric table, for example) is a configuration fault, not a key miss: it returns an error result of kind ProtocolTableMissing whose text names the table, the key type it has, and the key type of the method, so TryPropagateError hands it to the packet and the fault is seen at once instead of reading as "no protocol for this key".

16. Binary Header Decoding by Hand

16.1 The header struct

Fixed-size protocol headers are internal readonly structs with plain integer properties in wire order and a handwritten TryParse:

internal readonly struct UdpHeader
{
    /// <summary>Source port number (bytes 0..1).</summary>
    public ushort SrcPort { get; init; }

    /// <summary>Destination port number (bytes 2..3).</summary>
    public ushort DstPort { get; init; }

    /// <summary>Length of the UDP datagram (header + payload) in bytes (bytes 4..5).</summary>
    public ushort Length { get; init; }

    /// <summary>One's complement checksum (bytes 6..7).</summary>
    public ushort Checksum { get; init; }

    /// <summary>Serialized header size in bytes (8).</summary>
    internal const int HeaderSize = 8;

    /// <summary>Decodes the 8-byte header; <see langword="false"/> with a default value when fewer bytes are present.</summary>
    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public static bool TryParse(ReadOnlySpan<byte> source, out UdpHeader value)
    {
        if (source.Length < HeaderSize)
        {
            value = default;
            return false;
        }

        // Exact-size slice: the JIT knows every index below is in range and drops the bounds checks.
        ReadOnlySpan<byte> h = source[..HeaderSize];
        value = new UdpHeader
        {
            SrcPort = BinaryPrimitives.ReadUInt16BigEndian(h),
            DstPort = BinaryPrimitives.ReadUInt16BigEndian(h[2..]),
            Length = BinaryPrimitives.ReadUInt16BigEndian(h[4..]),
            Checksum = BinaryPrimitives.ReadUInt16BigEndian(h[6..]),
        };
        return true;
    }
}

16.2 The rules

One length check, one exact-size slice, BinaryPrimitives reads. Network byte order uses the …BigEndian methods; little-endian formats use …LittleEndian.
Offsets are literals with a comment naming the bytes (h[4..] for bytes 4..5); the declaration order of the properties mirrors the wire layout for the reader only.
Short input is an expected failure: TryParse returns false and a default value, never an exception. Longer input is accepted and only the fixed header is read.
Test every field: a test per struct feeds bytes with a distinct value per field and compares each property, plus the one-byte-short case.

16.3 Bit fields

Sub-byte fields (flags, version nibbles) are decoded with shifts and masks from the byte or word that holds them; each line names the bit positions:

byte versionAndIhl = h[0];
Version = (byte)(versionAndIhl >> 4),      // bits 7..4
Ihl = (byte)(versionAndIhl & 0x0F),        // bits 3..0

16.4 Header struct conventions

Always internal readonly struct.
Always include a HeaderSize / MinHeaderSize constant.
Include an ASCII-art header diagram in the XML doc comment.
Use init properties (not set).

17. FieldValue Factories

All field values are constructed via static factories:

// Numeric types
FieldValue.NewU64(42)              // unsigned 64-bit
FieldValue.NewI64(-1)              // signed 64-bit
FieldValue.NewF64(3.14)            // 64-bit float
FieldValue.NewBool(true)           // boolean

// Address types
FieldValue.NewMacAddress(mac)      // 48-bit MAC
FieldValue.NewIPv4(addr)           // 32-bit IPv4
FieldValue.NewIPv6(addr)           // 128-bit IPv6

// Data types
FieldValue.NewBytes(memory)        // ReadOnlyMemory<byte>
FieldValue.NewString("text")       // string (implicit LazyString)
FieldValue.NewTimestamp(ts)        // nanosecond timestamp

// Special
FieldValue.None                    // empty / container with no intrinsic value

Display text override:

// Override how the value is displayed in the UI
string csumText = DisplayTables.FormatHexU16(checksum);
container.AppendWithCustomText(_ChecksumFieldId, FieldValue.NewU64(checksum), csumText);

// Override the container value's representation
FieldValue headerValue = FieldValue.NewBytes(data[..20])
    .WithCustomRepresentation(new LazyString("20 bytes"));

18. Protocol Registration

18.1 Registration Flow

public static void RegisterStandardProtocols(this IStackBuilder builder)
{
    // Frame — entry point, owns frame.link_type table.
    // The frame protocol is discovered automatically during StackBuilder.Build()
    // by convention: the protocol whose Name is "frame" becomes the packet entry point.
    FrameProtocol frame = new();
    ProtocolId frameId = builder.RegisterProtocol(frame);
    frame.RegisterFields(builder, frameId);

    // Ethernet — auto-registered at frame.link_type via [RegisterAtTable]
    EthernetProtocol ethernet = new();
    ProtocolId ethId = builder.RegisterProtocol(ethernet);
    ethernet.RegisterFields(builder, ethId);

    // VLAN — auto-registered at eth.type = 0x8100, 0x88A8
    VlanProtocol vlan = new();
    ProtocolId vlanId = builder.RegisterProtocol(vlan);
    vlan.RegisterFields(builder, vlanId);

    // IPv4 — registered at eth.type = 0x0800, owns ip.proto table
    IPv4Protocol ipv4 = new();
    ProtocolId ipv4Id = builder.RegisterProtocol(ipv4);
    ipv4.RegisterFields(builder, ipv4Id);

    // IPv6 — registered at eth.type = 0x86DD, uses ip.proto table
    IPv6Protocol ipv6 = new();
    ProtocolId ipv6Id = builder.RegisterProtocol(ipv6);
    ipv6.RegisterFields(builder, ipv6Id);

    // UDP — registered at ip.proto = 17
    UdpProtocol udp = new();
    ProtocolId udpId = builder.RegisterProtocol(udp);
    udp.RegisterFields(builder, udpId);
}

The production helper also registers DataProtocol (Name = "data") for registered raw payloads (HTTP content-type, WebSocket, …) and SignalMessageRegistration.Register. Frame-level leftover bytes are packet.unparsed_data on PacketProtocol (§8), not a data protocol. SignalMessageRegistration.Register returns IReadOnlyList<SettingsLoadWarning>; RegisterStandardProtocols returns the combined PDU Transport + Signal Message warning list so callers can decide how to handle each entry. Settings/profile JSON paths may be empty. Stream overloads (Register(builder, stream), PduTransportRegistration.TryLoadConfig) add names/messages on top of that file; they do not replace it.

18.2 Registration Order

Registration order does not affect dispatch table resolution. The [UsesTable] attribute uses deferred resolution (WhenProtocolTableRegistered()), so a protocol can reference a table before the owning protocol is registered.

However, for readability, register protocols in stack order (frame → link → network → transport → application).

19. Source Generator Output

The Source Generator produces the following from attributes:

19.1 From `[Protocol("name", "UI Name")]`

// Generated constants
public const string ProtocolName = "udp";
public const string ProtocolUiName = "User Datagram Protocol";

// Generated Protocol.Name and UiName
public override string Name => ProtocolName;
public override string UiName => ProtocolUiName;
public override void OnStart(Stack stack) { _OnStartCustom(stack); }

// Generated ProtocolId field
private ProtocolId _ProtocolId;

19.2 From Field Attributes

// Source:
[U64Field("udp.srcport", "Source Port", IndexGroup = "udp")]
private FieldId _SrcPortFieldId;

// Generated in RegisterFields():
_SrcPortFieldId = builder.RegisterFieldInGroup(
    protocolId, "udp.srcport", "Source Port", FieldType.U64, "udp");

19.3 From `[ProtocolTableU64]`

// Source:
[ProtocolTableU64(PortTableName, "UDP Port")]
private ProtocolTableId _PortTableId;

// Generated in RegisterFields():
_PortTableId = builder.RegisterProtocolTable("udp.port", "UDP Port", ProtocolTableKeyType.U64);

19.4 From `[RegisterAtTable]`

// Source:
[RegisterAtTable(IPv4Protocol.IpProtoTableName, IpProtoKey)]

// Generated in RegisterFields():
builder.RegisterParserInU64TableByName("ip.proto", 17UL, protocolId);

19.5 From `[UsesTable]`

// Source:
[UsesTable(IPv4Protocol.IpProtoTableName)]
private ProtocolTableId _IpProtoTableId;

// Generated in RegisterFields():
builder.WhenProtocolTableRegistered("ip.proto", id => _IpProtoTableId = id);

19.6 Index Group IDs

// Generated fields (one per unique IndexGroup value):
private IndexGroupId _UdpGroupId;
private IndexGroupId _UdpPayloadGroupId;

// Generated in RegisterFields():
_UdpGroupId = builder.GetOrCreateIndexGroup("udp");
_UdpPayloadGroupId = builder.GetOrCreateIndexGroup("udp.payload");

19.7 RegisterFields / OnStart / OnShutdown Hooks

// Generated:
public void RegisterFields(IStackBuilder builder, ProtocolId protocolId)
{
    // ... register fields, tables, settings, RegisterAtTable entries ...

    // Load setting backing fields (so custom hook can read them)
    _VerifyChecksum = builder.GetBoolSetting("udp.verify_checksum", false);

    // Call user-defined custom hook (config loading, deferred registrations)
    RegisterFieldsCustom(builder, protocolId);
}

public override void OnStart(Stack stack)
{
    // Call user-defined custom hook (caches against the frozen stack)
    OnStartCustom(stack);
}

// User writes (optional — for config-driven protocols):
partial void RegisterFieldsCustom(IStackBuilder builder, ProtocolId protocolId)
{
    // Load config file, register parsers in u64 tables, etc.
    // Use builder.WhenProtocolTableRegistered / WhenFieldRegistered
    // for cross-protocol references that may not be registered yet.
}

// User writes (optional — for caches and delegates):
partial void OnStartCustom(Stack stack)
{
    _ = PopulateUdpFields(in container);
    // Build dispatch caches, resolve cross-protocol fields, etc.
}

20. Current Dispatch Table Map

Table Name	Owner	Key Type	Consumers
`frame.link_type`	`FrameProtocol`	`u64` (LinkType enum)	`EthernetProtocol`, `SllProtocol`, `Sll2Protocol`, `CanProtocol`, `FlexRayProtocol`, `LinProtocol`
`eth.type`	`EthernetProtocol`	`u64` (EtherType)	`IPv4Protocol`, `IPv6Protocol`, `VlanProtocol`, `ArpProtocol`, `LlcProtocol`
`eth.ieee8023`	`EthernetProtocol`	`u64` / any	`LlcProtocol` (catch-all for IEEE 802.3 frames)
`ip.proto`	`IPv4Protocol`	`u64` (IP protocol number)	`TcpProtocol`, `UdpProtocol`, `IcmpProtocol`, `Icmpv6Protocol`; shared by `IPv6Protocol` via `[UsesTable]`
`tcp.port`	`TcpProtocol`	`u64` (port number)	`DnsProtocol`, `TlsProtocol`, `HttpProtocol`, `Http2Protocol`, `SomeIpProtocol`, …
`udp.port`	`UdpProtocol`	`u64` (port number)	`DnsProtocol`, `DhcpProtocol`, `Dhcpv6Protocol`, `DtlsProtocol`, `SomeIpProtocol`, `PduTransportProtocol` (when `pdu_transport.udp_dispatch_ports` contains a matching port)
`pdu_transport.id`	`PduTransportProtocol`	`u64` (PDU ID)	Signal Message via `dispatch_bindings`
`http.content_type`	`HttpProtocol`	`string`	`JsonProtocol`, `TextProtocol`
`http.upgrade`	`HttpProtocol`	`string`	`WebSocketProtocol`
`can.id`	`CanProtocol`	`u64`	Signal Message and sub-protocols by standard 11-bit CAN ID (classic/FD, EFF clear). Extended frames never use this table.
`can.extended_id`	`CanProtocol`	`u64`	Extended 29-bit CAN IDs (classic/FD, EFF set) and CAN XL acceptance-field sub-protocols. Standard frames never use this table.
`flexray.id`	`FlexRayProtocol`	`u64` (slot + channel + cycle)	Signal Message and sub-protocols; key = `FlexRayLinkTypeFrame.EncodeDispatchKey(slot, channelB, cycle)` — bits `[10:0]` slot, bit `11` channel B, bits `[17:12]` cycle. Null frames (`NFI=0`) are not dispatched.
`lin.id`	`LinProtocol`	`u64` (6-bit frame ID, not the protected ID)	Signal Message and sub-protocols by LIN 6-bit frame ID. Skipped for Event (`msgType=3`) and when the capture error-flags byte is non-zero.
`someip.messageid`	`SomeIpProtocol`	`u64`	Payload deserializers by SOME/IP Message ID
`tcp.heuristic`	`TcpProtocol`	heuristic	`HttpProtocol`, `TlsProtocol`, `Http2Protocol` (content-based)
`udp.heuristic`	`UdpProtocol`	heuristic	tried after no `udp.port` parser consumed the payload; parsers add themselves (the SOME/IP parser registers here when it is registered)

TCP stream settings (group tcp, defaults in parentheses):

tcp.relative_sequence_numbers (true) — tcp.seq / tcp.ack subtract the direction ISN; tcp.seq_raw / tcp.ack_raw stay on the wire.
tcp.stream_key_last_vlan (false) — include the innermost VLAN id in the stream key. Id 0 does not split.
tcp.stream_key_frame (false) — include Frame.InterfaceId in the stream key. An invalid interface does not split.
tcp.max_tracked_streams (10000) — new 5-tuples past the cap parse the header and set tcp.error.stream_limit instead of tcp.stream.
tcp.verify_checksum (false) — check the checksum, including a checksum field of 0.

UDP stream settings (group udp):

udp.max_tracked_streams (10000) — new 4-tuples past the cap are still parsed and dispatched, but get no udp.stream; udp.error.stream_limit names the setting. Tuples already tracked keep their index.
udp.verify_checksum (false) — check the checksum. A zero checksum is not verified over IPv4 ("not computed") and is reported as [Bad] over IPv6, where it is invalid.

Decompression limits (secure by default, range 1 KiB to 128 GiB, default 64 MiB; the result is one array, so about 2 GiB is the hard bound):

http.max_decompressed_bytes (group http) — a gzip, deflate or br body that inflates beyond it is left undecoded (never truncated) and http.error.decompress_limit names the setting.
ws.max_decompressed_bytes (group ws) — the same for permessage-deflate frames; websocket.payload.decompressed.limit names the setting.

Total per packet (secure by default but relaxed: default 256 MiB, four times the per-item default; range 2 KiB to 128 GiB, because the TCP PDU of a message counts too; the SOME/IP-TP total is 64 MiB with a range of 2 KiB to 16 GiB). The decoded, unmasked, inflated or reassembled bytes that a protocol binds to one packet stay alive as long as the packet, and one packet can complete many messages, so each protocol also bounds their sum against the bytes bound to the packet in all (§22.4):

http.max_derived_bytes_per_packet (group http) — all bytes bound to one packet when HTTP binds a decoded (dechunked or inflated) body, the bytes of every protocol and the TCP PDUs included; the body that would pass it is not parsed and packet.error names the setting.
ws.max_derived_bytes_per_packet (group ws) — the same for unmasked and inflated WebSocket payloads; same behavior.
someip.tp_max_derived_bytes_per_packet (group someip) — the same for reassembled SOME/IP-TP messages; same behavior. Its default equals someip.tp_max_buffered_bytes, because a packet can only complete messages that the open sessions already hold.

HTTP/1 also reports the fixed header limits (256 header lines, 8192 bytes per line) as http.error.header_limit instead of silently showing the remaining lines as payload.

Fixed limits of the other application and bus protocols (constants, not settings; each is reported, never applied silently):

DNS over TCP: the reassembly accepts a PDU of up to 65537 bytes, the largest legal message (65535 bytes) plus its 2-byte length prefix.
JSON: a key or string value longer than 65536 (bytes without escapes, characters with escapes) is cut, and json.error.string_limit follows the field and names the limit.
SOME/IP: the eleven limits of the group someip (README, "SOME/IP configuration") are settings, not constants. A message above someip.tcp_max_message_size is never framed from a TCP stream; the drop is reported as tcp.error.reassembly_limit. A unit with more than someip.max_messages_per_unit messages shows the rest as raw trailing bytes under the last decoded message, with someip.error.unit_limit.
Reassembled IPv4 or IPv6 datagrams and framed TCP messages are bound to the packet as additional buffers; a packet holds at most 255 of them, and a reassembly that would exceed it is reported as a parse error that names the limit.

HTTP on TCP ports 80 and 8080 is reassembled with Http1MessageDetector before HttpProtocol.Parse. A port that has more than one registered protocol is not reassembled; every registered parser is still tried.

TCP analysis and reassembly

What the TCP layer adds beyond the header, and the limits behind it:

Analysis (tcp.analysis.*) is derived per direction from an ordered list of unacknowledged segments, serial-number arithmetic (so wrap-around is safe), and the window scale of the SYNs. It reports lost segments, retransmissions (plain, fast and spurious, with the time since the first send in tcp.analysis.rto and the packet in tcp.analysis.rto_frame), out-of-order segments, duplicate ACKs (tcp.analysis.duplicate_ack_num counts until the acknowledgement number moves), zero-window and window-update facts, keep-alives, ambiguous and partial ACKs, tcp.analysis.bytes_in_flight (the sum of the unacknowledged bytes) and the round-trip times. The unacknowledged list of a direction holds at most 10000 segments and is allocated on the first data segment; a segment beyond the bound is analysed but not remembered, so a receiver that never acknowledges cannot grow memory.
Connection state (tcp.analysis.connection_state) follows RFC 9293 for the side that closed first. It is derived from recorded FIN and ACK facts, not from flags alone, so a retransmitted FIN does not move it. A connection joined mid-stream reads ESTABLISHED (handshake not captured). A reset ends it for good.
Completeness (tcp.completeness, tcp.completeness.str, one tcp.completeness.* flag per part) records which parts of the conversation the capture contains. The value at a packet describes the packets before it.
Relative numbers (tcp.seq, tcp.ack, tcp.nxtseq) subtract the initial sequence number of the direction. A connection joined mid-stream uses the first number seen as its base.
Stream lifecycle. A FIN from both sides or an RST ends the reassembly of a stream. A new SYN on the same address and port pair after the close starts a new stream and sets tcp.analysis.reused_ports.
Reassembled protocols. DNS, HTTP/1, TLS, HTTP/2 and SOME/IP are reassembled per direction, each with its own framing detector. A protocol that the heuristic table (tcp.heuristic) picks gets the same reassembly. A reassembled PDU lists the packets that delivered it (tcp.segments), and each segment shows tcp.segment.count. A reassembly that reaches its limit sets tcp.error.reassembly_limit and names the limit; nothing is truncated silently. When a stream loses its message boundary and the protocol's resynchronization heuristic skips bytes to find the next one, the segment that carried them shows tcp.error.resync with the number of skipped bytes, and the message behind them is extracted by the same call (the buffer looks again right after a successful resynchronization, so it does not wait for the next segment). TCP publishes the ports of the segment through TcpPortCache for the whole payload dispatch, first parse and replay alike.

Protocol	Largest PDU	Buffer per direction	Why
DNS	65537 bytes	1 MiB	The largest legal message plus its length prefix.
HTTP/1	16 MiB	16 MiB	A message is delivered whole; open by default, configurable by the registering code.
TLS	2 MiB	4 MiB	One record or a handshake message that spans records; a hostile length cannot fill the buffer.
HTTP/2	frame header + 16 MiB - 1	16 MiB	The largest legal frame; the detector follows the peer's `SETTINGS_MAX_FRAME_SIZE`.

HTTP/2 header blocks are decoded with a stateful HPACK decoder per direction (RFC 7541 dynamic table, at most 64 KiB; header block fragments across CONTINUATION frames are joined up to 256 KiB). The decoder state is built during the first parse and replayed read-only afterwards, like all cross-packet state.

Dispatch chain:

Frame ──[frame.link_type]──► Ethernet ──[eth.type]──► IPv4/IPv6/VLAN/ARP/LLC
                         │                                     │
                         └──► SLL / SLL2                [ip.proto]
                         │                                │         │
                         └──► CAN ──[can.id/can.extended_id]       │
                         │                              UDP        TCP
                         └──► FlexRay ──[flexray.id]   │          │
                         └──► LIN ──[lin.id]           │          │
                                                  [udp.port]  [tcp.port / tcp.heuristic]
                                                   │    │      │         │
                                          DNS/DHCP PDU-Tr. HTTP/1.x   TLS/HTTP2
                                                       │          │
                                              [pdu_transport.id]  [http.upgrade / content_type]
                                                       │               │
                                                Signal Message      WS / JSON

21. Checklist for New Protocols

Class Structure

Header Struct

Create an internal readonly struct with a handwritten TryParse (one length check) for fixed-size headers
Read big-endian fields with BinaryPrimitives.Read…BigEndian on an exact-size slice
Decode sub-byte fields with shifts and masks, one comment per field naming the bits
Include HeaderSize / MinHeaderSize constant
Include ASCII-art header diagram in XML doc

Parse Method

Lazy Populator

Pre-allocate delegate in OnStartCustom() (captures only this)
Re-parse header from container.Value.Data.AsBytes()
Read pre-computed helper fields via TryGetPrev on MutField with materialize: false (bounded walk)
Append all child fields via container.Append() / container.AppendWithCustomText()
May call TryCallNextProtocol*() dispatch methods
Do NOT call context.RecordGroupPresence() inside the populator
Return ParseResult (0 on success)

Display Text

Precomputed static table in DisplayTables.cs for fixed-domain fields
LazyString.FormatLazy for dynamic summary text and packet info
No string.Format(), $"", or .ToString() on the parse hot path
Use DisplayTables.FormatHexU8() / FormatHexU16() for hex formatting
Flags container field uses a precomputed *FlagsFormatter class (see §14.8) with bracket-style display text ([FLAG1, FLAG2] or 0xNN [FLAG1, FLAG2])

Dispatch (if protocol owns a table)

Declare table with [ProtocolTableU64(name, uiName)]
Build dispatch cache in OnStartCustom() (dense or sparse)
Write dispatch helper method with cache + fallback pattern
Consider heuristic table for content-based fallback

Tests

Basic parse tests (standard packets, all fields verified)
Edge case tests (min-length, max-length, boundary values)
Malformed data tests (too short, invalid values, corrupt headers)
Display text format tests
Cross-protocol field access tests (if applicable)
Dispatch chain tests (verify sub-protocol is called)

Documentation

Field tree example in class XML doc comment
ASCII-art header diagram in header struct doc comment
Protocol-specific parsing notes and RFC references

22. Fragment Reassembly

Every protocol that carries one message in several fragments (IPv4, IPv6, SOME/IP-TP, and later ISO-TP, DoIP, or L2CAP) uses the one reassembler of Core, FragmentReassembler<TKey>. A protocol supplies the key type that identifies one datagram and a FragmentReassemblyOptions value; Core supplies the table of open datagrams, the check order, pooled fragment copies, eviction, the limits, and one FragmentOutcome per call. Protocol code never sees a fragment list.

private readonly FragmentReassembler<MyKey> _Reassembler = new(FragmentReassemblyOptions.IPv4);

FragmentOutcome outcome = _Reassembler.Add(in key, byteOffset, moreFragments, fragmentBytes, contributor: packet.Id.Value);
byte[]? datagram = outcome.Kind == FragmentOutcomeKind.Complete ? outcome.Payload : null;

The reassembler is not thread-safe by design: call Add only from the ordered first parse, record the outcome in the packet's effect, and let every later parse replay the recorded outcome (§2). A concurrent re-parse that calls Add corrupts the fragment buffers.

22.1 Options

Option	IPv4 preset	IPv6 preset	SOME/IP-TP	Meaning
`MaxPending`	1,024	1,024	`someip.tp_max_sessions`	open datagrams; at the limit one is evicted (secure by default)
`MaxBufferedBytes`	unbounded	unbounded	`someip.tp_max_buffered_bytes`	stored bytes of all open datagrams together (the IP presets keep today's open default)
`MaxDatagramLength`	65,535	65,535	`someip.tp_max_message_size`	extent limit, measured as offset plus length, so one fragment at a huge offset allocates nothing
`MaxFragments`	8,192	8,192	unbounded	fragments per datagram; `Unbounded` when the alignment already bounds the list
`NonLastAlignment`	1	1	16	every fragment but the last is a non-empty multiple of this (a format fact, 16 for SOME/IP-TP)
`OverlapPolicy`	`KeepLargestDuplicate`	`DiscardDatagram`	`KeepFirst`	`KeepFirst` stores only uncovered pieces and flags differing bytes
`EvictionPolicy`	`OldestCreated`	`OldestCreated`	`LeastRecentlyUsed`	`LeastRecentlyUsed` keeps an active long transfer alive
`RestartOnZeroOffset`	no	no	yes	a second fragment at offset 0 starts the datagram over (a late first fragment does not)
`GapPolicy`	`WaitForCompletion`	`WaitForCompletion`	`PrefixOnLastThenComplete`	`PrefixOnLastThenComplete` reports the decodable prefix once, then the full datagram

TryValidate names the offending option and its range. The constructor throws ArgumentException for invalid options, because a bad value is a programming error of the owner; an owner that exposes a value as a user setting validates it against the setting's own range first.

22.2 Check order of `Add`

Extent (threat: one fragment at a huge offset requests a huge buffer): refused before anything is copied; an open datagram of the key is released.
Alignment: refused without creating a datagram, so an empty flood opens nothing.
Lookup or create, with count eviction; an existing datagram becomes the most recently used one.
Restart when the option is on and the offset is 0.
Budget: other datagrams are evicted in policy order until the fragment fits; the datagram being filled is never the victim; a datagram that alone does not fit is released (BufferLimit).
Insert with the overlap policy (scanning from the tail, so in-order arrival is O(1)).
Completion: the payload is a plain array bound to the packet, the contributor tags come in offset order.
Gap: under PrefixOnLastThenComplete the call that fixed the total length reports the decodable prefix once.
Otherwise Pending, which carries no arrays.

22.3 Using the outcome

`Kind`	Meaning	Carries
`Pending`	stored, not complete	nothing
`Complete`	datagram built, datagram released	`Payload`, `Contributors` (offset order), `FragmentCount`
`PartialPrefix`	last fragment arrived, a hole remains	`Payload` (prefix then covered ranges), `PrefixLength`, `Gaps`, `CoveredAfterGap`, `Contributors`
`Discarded`	fragment refused	`DiscardReason` (`ExtentLimit`, `Alignment`, `BufferLimit`, `FragmentLimit`, `Overlap`, `ConflictingEnd`, `BeyondEnd`)
`NotRecorded`	replay without a recorded result	nothing (the default value)

Flags adds Restarted, OverlapConflict, EvictedForCount, and EvictedForBudget; EvictedCount counts the other datagrams the call evicted. The arrays of a delivering outcome are never written after the call, so an outcome can be stored in a published effect and read from any thread. The contributor is an opaque tag the caller passes per fragment (the packet id): the list returned with a delivering outcome is the list of contributing packets, and it names packets parsed earlier or the current one, never a later one.

BufferedBytes counts stored payload bytes. The pooled arrays behind them can be up to twice as large, because the pool rounds a rental up to a power of two.

22.4 Derived bytes for a nested parse

A reassembled datagram is not the only byte sequence that a protocol builds during Parse and hands to a nested protocol: an HTTP body after dechunking and inflating, a WebSocket payload after unmasking and inflating, and the plaintext of a decrypted record are fresh arrays too. The rule is the same for all of them: bind once, before the nested parse, and hand on the bound slice.

// The packet decides only whether a slot is free; the total of this protocol is applied here, to the sum of all bound bytes.
// Compare with the remaining budget (not "bound + length > limit"): the subtraction cannot wrap.
ulong remaining = _MaxDerivedBytesPerPacket > container.BoundBytes
    ? _MaxDerivedBytesPerPacket - container.BoundBytes
    : 0;
if ((ulong)derivedBytes.Length > remaining)
{
    // Name the setting; an empty slice would reach the next protocol as "no data" without any sign of why.
    return ParseError.Custom(ProtocolName, "Decoded bodies cannot be parsed further: together they would pass the total of one packet (setting myproto.max_derived_bytes_per_packet)");
}

if (!container.TryBindBuffer(derivedBytes, out ReadOnlyMemory<byte> bound, out _))
{
    // The packet already holds 255 additional buffers (or is sealed). Name the limit.
    return ParseError.Custom(ProtocolName, _BoundBufferLimitMessage);
}

ParseResult result = container.TryCallNextProtocolString(_TableId, key, bound, in context);

Why bind. A stateful nested protocol keys its recorded effect on the location of its data slice and refuses a slice that is not a packet buffer (MutField.TryGetEffectLayerKey returns false). A reader of the packet's buffers (Packet.BufferCount, Packet.Buffer(i)) sees exactly the bound buffers, so unbound derived bytes are missing from it.
Bind once, reuse the slice. One message gets one buffer, however many dispatches follow (content-type table, built-in fallback, upgrade table). Intermediate results are not bound: a gzip body inside a chunked body is decoded twice but only the final bytes become a buffer.
Bind only what is derived. A payload that is already a slice of the data argument (a body without a transfer or content coding, an unmasked and uncompressed frame) is dispatched as it is and takes no buffer.
Empty derived bytes (only the terminal chunk, an inflated empty message) have nothing to parse: do not bind and do not dispatch.
Full table. The table holds at most 255 additional buffers (the effect key packs the index into 8 bits); the limit is a constant, not a setting. Report it by name with ParseError.Custom, like the IPv4, IPv6, TCP, SOME/IP-TP, HTTP, and WebSocket binders do. A test fills the table with a probe protocol and asserts the named error and that the nested protocol was not called (BoundBufferLimitTests). A protocol that walks several payloads of one packet (WebSocket frames) keeps walking after the error so the index groups of the later payloads are still recorded.
Populator bytes stay field values. A lazy populator runs after the packet is sealed, where TryBindBuffer refuses with BufferBindFailure.Sealed and returns an empty slice. Bytes it derives (http.payload.decoded, websocket.payload.decompressed, the joined TLS handshake bytes) are field values, not buffers.
Cost. One table entry per derived buffer, and the bytes stay alive for the lifetime of the packet. A packet with many masked WebSocket frames can fill the table (the first frame that finds it full reports the limit; the frames after it are not dispatched).

Total size. The derived buffers of one packet are bounded in sum, not only in count and not only per item, because deflate reaches about 1000:1, one TCP segment can complete many messages, and 255 inflated payloads would otherwise stay alive together. The packet itself decides only whether a slot is free (MutField.TryBindBuffer); it knows no protocol and has no byte limit, but it keeps one sum of all bytes bound so far (MutField.BoundBytes, the bytes of every protocol and of the transport together, restarting at 0 with every parse of a packet so a reparse reaches the same decisions). Every protocol that binds bytes it allocated from untrusted input therefore applies its own total to that sum before it binds, with the comparison above (the built-in protocols share it in the internal helper DerivedBytesLimit.TryBind; a refused buffer is not stored, so the packet is unchanged after a failure). The total is a setting of the protocol, separate from its per-item limit, relaxed by default and validated by the settings layer like every limit (§20). Because the sum counts every protocol, a total bounds the memory the packet pins when that protocol binds, the TCP PDUs of the same packet included:

Protocol	Per-item limit (default)	Total per packet (default)	Range of the total
HTTP	`http.max_decompressed_bytes` (64 MiB)	`http.max_derived_bytes_per_packet` (256 MiB)	2 KiB to 128 GiB
WebSocket	`ws.max_decompressed_bytes` (64 MiB)	`ws.max_derived_bytes_per_packet` (256 MiB)	2 KiB to 128 GiB
SOME/IP-TP	`someip.tp_max_message_size` (10 MiB)	`someip.tp_max_derived_bytes_per_packet` (64 MiB)	2 KiB to 16 GiB

The payload or message that would pass the total is not bound or parsed; the parse error names the setting, the total, and that it is the total per packet (DerivedBytesLimit.Text writes the sentence, so the three protocols read alike), and the later payloads of the packet are not dispatched (a protocol that walks several payloads, such as WebSocket frames, keeps walking so the index groups of the later payloads are still recorded). The total is reported first when both apply; the 255-buffer failure keeps its own text. A payload that is neither masked nor inflated binds nothing and does not count. The two limits are independent: a body that inflates above the per-item limit stays undecoded and is flagged by its own field, and the total is not involved; a total below the per-item limit also limits a single item. Bytes that a bounded reassembly buffer already held (a TCP PDU, an IPv4 or IPv6 datagram) are bound with TryBindBuffer alone and need no total of their own: the stream buffer (StreamReassemblyConfig.MaxBufferSize) or the datagram length bounds their sum already (they still count in BoundBytes, so they take part in the totals of the protocols above). A protocol that binds without applying a total is bounded only by the 255 buffers of the table. A new protocol that binds derived payloads declares its own total, with the Section 4.2 record (value, threat, constant or configurable, posture) at the definition, and a decrypting protocol (see PlaintextHandoff) does the same for the plaintext it binds.

23. Testing A Protocol That Keeps State

A protocol that pairs, reassembles or learns across packets needs two proofs on top of its field tests. SOME/IP is the worked example (SomeIpTsharkTests, SomeIpParallelRedissectTests).

Equivalence with an external decoder from one description. When the protocol has a description file, the test writes the external decoder's configuration tables from the compiled model (TsharkSomeIpUatProfile) instead of keeping a second copy by hand. Two rules make the mapping honest: the tool names a value by the filter string of the table row that declares it, so the generator writes a fresh copy of every type for every usage site (one leaf, one filter string, one map entry from the parser's registered name); and a value the tool cannot express (a UUID) throws instead of dropping out of the comparison. Compare typed values, not display text, except where the tool only has text (a string, the name of an enumeration). The tool is mandatory by default; a machine without it sets the documented escape hatch, which turns the test into a visible skip.

Parallel re-dissection of a mixed capture. Build a deterministic capture (fixed seed, fixed times) that exercises every stateful feature, ingest it in order, take the baseline of every materialized tree, and let eight readers re-parse in random order in two phases: after the ingest, and while the ingest is running with each reader taking only packets whose first parse has completed. The expected tree of a packet never changes after its first parse (every cross-packet fact is stored with the packet), so no interleaving can change the verdict; a reader that sees a different tree has found a shared write. A second test checks that the capture really contains the fields of the features it claims, because a scenario that silently stops triggering would turn the proof into a no-op.

FilesExpand file tree

PROTOCOL_GUIDE.md

Latest commit

History