Skip to content

architecture.md

Latest commit

 

History

History
218 lines (182 loc) · 11.7 KB

architecture.md

File metadata and controls

218 lines (182 loc) · 11.7 KB
Raw
type Architecture
title Bashkit Architecture
description Core interpreter architecture, module boundaries, execution flow, and design principles.
tags
bashkit
architecture
interpreter

Architecture

Status

Implemented

Decision

Official name: "Bashkit" (not "BashKit"). Crate/package identifiers use lowercase bashkit.

Bashkit uses a Cargo workspace with multiple crates:

Crate Purpose
crates/bashkit/ Core library (parser, interpreter, VFS, builtins, tool contract)
crates/bashkit-scripted-tool/ ScriptedTool, ToolDef, ToolRegistry (published, lockstep with core)
crates/bashkit-cli/ CLI binary
crates/bashkit-python/ Python bindings (PyO3)
crates/bashkit-js/ JavaScript bindings (NAPI-RS)
crates/bashkit-capi/ Versioned native C ABI
crates/bashkit-eval/ LLM eval study (mira framework)

Core library modules: parser/, interpreter/, fs/, builtins/, network/, git/, ssh/. ScriptedTool / ToolDef / ToolRegistry live in the separate crates/bashkit-scripted-tool/ crate. See source, structure evolves.

Public API

Main entry points: Bash (library) and BashTool (LLM tool contract). See crates/bashkit/src/lib.rs for the full public API surface.

Per-invocation exec state

ExecOptions carries everything scoped to a single exec_with_options call: streaming callback, builtin extensions, arg0, positional, and stdin. All of it is per-call by construction, not session state.

  • Positional parameters exist only inside a call frame, so the host boundary pushes a synthetic top-level frame before Interpreter::execute and truncates the call stack back to its baseline afterwards, including on error paths, so $# is 0 again on the next exec. $0 defaults to bash when no arg0 is supplied; set -- at top level uses the same synthetic frame and must not change $0.
  • stdin seeds pipeline_stdin, which reset_transient_state clears at the start of every exec, so it is installed after that reset and immediately before execution. A pipe or redirect inside the script still wins for the command it applies to.

Byte-native streams

StreamData is the canonical stdin/stdout/stderr transport value. Its byte buffer is authoritative through ExecOptions, interpreter pipeline state, redirections, BuiltinContext, ExecResult, and streaming callbacks. Output limits truncate by bytes, including in live callbacks. Byte-oriented builtins and bindings use as_bytes()/into_bytes(); text-oriented consumers cross an explicit UTF-8 boundary with text() or text_lossy(). Line-oriented consumers locate delimiters and split the authoritative byte buffer before lossy decoding; offsets into the cached lossy view are never byte offsets.

Shell words and variables remain text. Command substitution therefore removes NUL bytes, as Bash does because variables cannot contain NUL, then decodes the remaining bytes for expansion and strips trailing newlines. JSON tool results are also text-only; native Rust, C, Python, Node, and browser results expose raw stdout/stderr bytes alongside their display strings.

Both are installed late (just before execute) so the size, hook, and parse checks that can return early cannot leave state behind.

Process-local execution suspension

Event-backed host calls reuse the async-first execution model: an ExecutionHandle starts the live execution future, yields an owned request to the host, then resolves a one-shot response when the host resumes it. The first next_event hands the future to a task spawner where the target has one (any async runtime on native, spawn_local on JS-backed wasm) so the interpreter timeout keeps running while the host is parked; a non-JS wasm embedder has no executor, so there the handle keeps polling the future itself and the deadline is enforced on the next poll. A timeout drops the session on every target. On normal completion, into_bash returns the reusable session; dropping a suspended handle aborts the driver and drops the session. This is process-local scheduling, not a serializable continuation; snapshots still capture state only between executions. See Builtin Commands and Snapshot History.

Arithmetic expansion budget

Arithmetic's synchronous evaluator owns one depth/fuel budget per top-level expression. Dollar expansion checks and charges original text before recursing into array subscripts or nested $((...)). Read-only child evaluators borrow that budget while discarding writes; siblings cannot restart fuel. Nameref and associative-key lookups stay in the same budget. Recursive bare-variable evaluation and syntax parsing share the depth counter. Expansion errors propagate to the arithmetic boundary; later host executions receive fresh state. Regression and Bash differential coverage: crates/bashkit/tests/integration/arithmetic_resource_tests.rs (TM-DOS-026).

Trap nesting

Self-sent signals run handlers inline. Every handler entered through run_trap_command charges the existing function-depth counter, also used by functions, source, and child shells; its hard ceiling is 16. A dedicated trap-nesting cap of 3 trips first on small stacks: exactly three nested handler executions are admitted, and the fourth entry is skipped with a diagnostic on the handler stderr. This preserves finite nested signal delivery while bounding direct, mutual, and mixed cycles without relying on command fuel or per-handler AST depth. The first trip is surfaced as a resource-limit error at the exec boundary; unparsable or failing handlers still never disturb the interrupted command (TM-DOS-030). The signal path boxes the handler future to keep each native frame small. Trap state and depth are restored on every path. DEBUG, ERR, and RETURN retain their reentrancy suppression and existing error handling. See Threat Model, TM-DOS-020, and signal_trap_depth_tests plus CLI small-stack regression coverage.

Shared execution budget

Each exec_with_options creates exactly one ExecutionBudget before hooks or parsing. Its Arc-backed counters are cloned, not recreated, by nested parsers, command/process substitutions, pipelines, builtin execution plans, embedded Python/TypeScript/SQLite, traversal/search, archive/compression work, and host callbacks. It meters three distinct resources: monotonic work units, monotonic aggregate consumer input bytes, and RAII live/intermediate byte leases.

Exhaustion, the request deadline, or cancellation poisons the shared budget; later descendants see the first failure and cannot resume with fresh counters. This aggregate layer complements rather than replaces parser fuel, command and loop counters, output caps, VFS quotas, and runtime-specific ceilings. The budget is request-scoped: the next host exec receives a fresh budget, while session limits continue to protect repeated host calls.

Runtime admission is deliberately conservative where a VM does not expose a portable completed-instruction count: Python reserves work from its configured memory allowance and TypeScript from its allocation allowance before entry. Python additionally meters Monty allocation/statement checkpoints. The reservation is not refunded, because refunding would recreate the reset path this layer exists to remove.

Heap buffers grown from untrusted sizes use the budget-aware owning builders in limits.rs. BudgetedVec (including byte buffers) and BudgetedString acquire or grow the live-byte lease before reserving capacity, roll the charge back when the allocator or producer fails, and release it with the buffer. Archive and compression paths use these builders because their expansion ratios and nested buffers make post-hoc leasing unsafe. Atomic compare/exchange admission keeps concurrent descendants from wrapping or temporarily exceeding the shared cap.

Command-local input redirections admit canonical VFS metadata/device read ceilings before loading bytes, including before the empty open for <>. Backend capacity beyond its reported size is charged before decoding. Cached lossy text and fd copies use the existing owning builders; byte/text leases travel with both the numbered-fd slot and surviving stdin. Reopening a numbered fd replaces its old storage, without refunding aggregate input consumption. An input not needed for stdin moves into the fd table instead of being copied. Budget and cancellation errors abort the request; ordinary backend I/O errors remain shell command failures. A custom VFS must report reliable read sizes; the legacy read_file trait cannot prevent allocations inside a backend that underreports or concurrently changes those sizes. Persistent/coprocess fd storage remains a separate ownership boundary.

Shell diagnostic names are truncated to 1,024 UTF-8 bytes before copying, without changing positional $0. Builtin diagnostic prefix rewriting uses BudgetedBytes and fallible StreamData text construction under the same request budget. Both storage leases survive through redirection and hooks; input scanning and every emitted line charge work and check the shared cancellation/deadline. Capture caps remain downstream of a command's own redirections, so command-local redirected diagnostics retain full content. A child shell captures uncaught stderr before an outer redirect on the bash -c invocation sees it; that outer redirect receives the child's capped capture. Put the redirect on the child command to bypass capture. Tilde expansion uses the same owning string builder before each append; it borrows HOME/PWD/OLDPWD instead of cloning the directory on each prefix. Scan and copy work polls the shared deadline and cancellation flag. Assignment preprocessing retains leases for expanded literal buffers and their word copies through consumption; mixed-word segments retain leases through field splitting. The synchronous operand pipeline records a refused reservation in the existing sticky resource-error slot, checked before the completed word is returned. Retained-variable admission remains a separate check after expansion.

set -e contexts

errexit_active() is set -e plus condition_sequence_depth == 0. The depth counts the contexts where bash ignores errexit: an if/while/until condition, a non-final &&/|| element, and a ! pipeline. It is a counter on the interpreter, not a flag on results, so it also covers function bodies and subshells run from those contexts (if f; then, ( false; x ) || y). Forked pipeline stages copy it. A command substitution clears -e unless shopt -s inherit_errexit is set; with that option it stops at the first failure, as bash does.

Design Principles

  1. Async-first: All filesystem and execution is async (tokio)
  2. Virtual: No real filesystem access by default
  3. Multi-tenant safe: Isolated state per Bash instance
  4. Trait-based: FileSystem and Builtin traits for extensibility

Alternatives Considered

  • Single crate: rejected, CLI bloats library; Python/JS packages need separate crates.
  • Sync filesystem: rejected, network ops need async; tokio already a dep.

See also