Skip to content

About

Open source agent harness for local LLM inference

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

5 stars

Watchers

0 watching

Forks

Repository files navigation

Orion

Orion

The Orion Agent Harness - a backend-agnostic agent harness for local LLM inference.

Crates.io docs.rs CI Downloads License: MIT MSRV

Bring your own model runtime - llama.cpp, MLX, cloud APIs, anything. Orion is published as the orion-core crate.

Orion handles the conversation loop so you don't have to: context management, token budgets, streaming events, chat formatting, and an automatic tool-execution loop (the agent parses tool calls, runs your tools, feeds the results back, and repeats until the model gives a final answer - see tools).

How It Works

User sends "Hello"
  → Agent.prompt("Hello")
    → Context pipeline (prune old messages to fit token budget)
      → Format prompt (ChatML template + tool definitions)
        → LlmBackend.generate() (streams tokens one by one)
          → AgentEvent stream (your UI subscribes here)
            → If the model called tools: run them, append results, loop back
              → Done (model returns a tool-free answer)

Agent.inbox() reaches the agent mid-run (read before every model call),
and Agent.resume() carries a conversation on without new user text.

You implement one backend trait for your inference engine - LlmBackend for a local engine fed a formatted prompt, or ChatBackend for a hosted chat API fed messages - and Orion handles everything above it.

Quick Start

use std::sync::Arc;
use orion_core::{Agent, AgentConfig, AgentEvent, LlmBackend};
use tokio::sync::mpsc;

// 1. Implement the backend trait for your engine (see `backend` below)
let backend: Arc<dyn LlmBackend> = Arc::new(MyBackend::new());

// 2. Create an agent
let mut agent = Agent::new(AgentConfig {
    system_prompt: "You are a helpful assistant.".into(),
    ..Default::default()
});

// 3. You supply the event channel; the agent streams events into it
//    while generation runs, then returns when the turn is done.
let (tx, mut rx) = mpsc::unbounded_channel::<AgentEvent>();

// Consume events concurrently - forward them to your UI.
let consumer = tokio::spawn(async move {
    while let Some(event) = rx.recv().await {
        match event {
            AgentEvent::MessageDelta { delta, .. } => print!("{delta}"),
            AgentEvent::GenerationStats { tokens_generated, .. } => {
                println!("\n\nDone: {tokens_generated} tokens");
            }
            AgentEvent::ContextBudget { used_tokens, max_tokens, .. } => {
                println!("Context: {used_tokens}/{max_tokens} tokens");
            }
            AgentEvent::Error { message } => eprintln!("Error: {message}"),
            _ => {}
        }
    }
});

agent.prompt("What is Rust?", backend, tx).await?;
consumer.await?;

A complete, runnable version lives in examples/mock_backend.rs - try it with cargo run --example mock_backend.

Don't want to manage the channel yourself? agent.prompt_stream(text, backend) creates it for you and hands back (receiver, future) - drive the future (e.g. with tokio::join!) while you drain the receiver.

For a real over-the-wire backend, enable the http-backend feature and use the supported OpenAiHttpBackend - a streaming OpenAI-compatible /v1/chat/completions client that works against OpenAI, llama.cpp llama-server, vLLM, LM Studio, and Ollama by changing only the base URL. examples/openai_backend.rs drives it end to end; run with cargo run --example openai_backend --features http-backend.

Modules

agent - The Orchestrator

The Agent struct is the main entry point. It owns the conversation state and drives the prompt → LLM → response loop.

use orion_core::{Agent, AgentConfig, InferenceParams, ContextConfig};

let mut agent = Agent::new(AgentConfig {
    system_prompt: "You are a coding assistant.".into(),
    inference_params: InferenceParams {
        max_tokens: 4096,
        temperature: 0.4,
        context_size: 8192,
        n_threads: 6,
    },
    context_config: ContextConfig {
        max_context_tokens: 8192,
        max_response_tokens: 4096,
        ..Default::default()
    },
    ..Default::default()
});

// Change settings on the fly
agent.set_system_prompt("You are a pirate.");
agent.set_inference_params(InferenceParams { temperature: 1.2, ..Default::default() });

// Conversation management
agent.clear();                              // Reset conversation
agent.replace_messages(saved_messages);     // Restore a saved conversation

// Stop a run: the generation, tools watching ctx.is_aborted(), and child agents
agent.abort();

// Reach the agent mid-run, and carry on without new user text
agent.inbox().push("Also check the README.");
agent.resume(backend, tx).await?;

backend - Bring Your Own LLM

Implement the LlmBackend trait to plug in any inference engine:

use orion_core::{LlmBackend, InferenceParams, GenerationResult, TokenCallback, CoreResult};
use std::sync::atomic::AtomicBool;
use std::sync::Arc;

struct LlamaCppBackend {
    // your engine state
}

impl LlmBackend for LlamaCppBackend {
    fn generate(
        &self,
        prompt: &str,               // Fully formatted (chat template applied)
        params: &InferenceParams,    // max_tokens, temperature, context_size
        abort: Arc<AtomicBool>,      // Check this each token to support cancellation
        on_token: TokenCallback,     // Call with (token_text, count, tokens_per_sec)
    ) -> CoreResult<GenerationResult> {
        // Feed prompt, sample tokens, call on_token for each one.
        // Return final stats when done.
        todo!()
    }

    fn tokenize_count(&self, text: &str) -> CoreResult<u32> {
        // Count tokens without running inference.
        // Used by the context pipeline for budget management.
        todo!()
    }

    fn is_ready(&self) -> bool {
        // Whether a model is loaded and ready for inference.
        todo!()
    }
}

The backend runs on a blocking thread - no async required. Orion handles the async orchestration.

Hosted chat APIs: ChatBackend (feature chat-backend, on by default). A hosted API takes a list of messages and applies the model's own chat template, so handing it Orion's already-formatted prompt either templates twice or collapses every turn into one user message. Implement ChatBackend instead: the agent skips its own templating and hands you the system prompt and the pruned messages. Pruning, the tool loop and the event stream are unchanged, and it is async, since the work is I/O.

use std::sync::Arc;
use std::sync::atomic::AtomicBool;
use async_trait::async_trait;
use orion_core::{ChatBackend, CoreResult, GenerationResult, InferenceParams, Message, TokenCallback};

struct HostedApi; // your HTTP client

#[async_trait]
impl ChatBackend for HostedApi {
    async fn chat(
        &self,
        system: &str,          // System prompt, tool instructions folded in
        messages: &[Message],  // The turns that survived pruning, no system message
        params: &InferenceParams,
        abort: Arc<AtomicBool>, // Check between chunks to support cancellation
        mut on_token: TokenCallback,
    ) -> CoreResult<GenerationResult> {
        // Send `system` + `messages` to the API, call on_token per chunk,
        // and return the usage the response reports.
        todo!()
    }
}

let backend: Arc<dyn ChatBackend> = Arc::new(HostedApi);
let reply = agent.send("Hello", backend).await?;

agent.prompt and agent.send take either kind: an Arc<dyn LlmBackend> and an Arc<dyn ChatBackend> both convert into the Backend enum. send is for a caller that is not streaming - it runs the turn and answers with the final assistant message, returning an Err for an error the run reported.

backends - Ready-Made Backends (feature http-backend)

Don't want to implement LlmBackend yourself? Enable the http-backend feature for OpenAiHttpBackend, a streaming OpenAI-compatible client. One implementation targets OpenAI, llama.cpp llama-server, vLLM, LM Studio, and Ollama - change only the base URL:

use std::sync::Arc;
use orion_core::backends::{OpenAiConfig, OpenAiEndpoint, OpenAiHttpBackend};
use orion_core::LlmBackend;

// Local server (no key); or `.with_api_key("sk-...")` for a cloud endpoint.
let config = OpenAiConfig::new("http://localhost:8080/v1", "local-model");
let backend: Arc<dyn LlmBackend> = Arc::new(OpenAiHttpBackend::new(config).unwrap());
// agent.prompt("Hello", backend, tx).await?;

Tokens stream through on_token as they arrive; the GenerationResult carries the real token counts from the response's usage block. Transport failures surface as CoreError::BackendUnreachable (no response - safe to retry or fail over), while an endpoint that answers with an error status surfaces as CoreError::Backend. It pulls in a blocking HTTP client, so it's off by default.

OpenAiHttpBackend implements both traits, and which one you hand the agent decides what goes over the wire:

  • As a ChatBackend (Arc<dyn ChatBackend>) it posts a real message list, system prompt included, to /v1/chat/completions, and the server applies the model's own template. Use this against hosted APIs and chat servers. with_endpoint is not consulted on this path.
  • As an LlmBackend (Arc<dyn LlmBackend>) it sends Orion's formatted prompt, chosen with .with_endpoint(...): OpenAiEndpoint::Completions posts it verbatim to /v1/completions, which is the right choice for a local instruct model on a completion endpoint. OpenAiEndpoint::Chat (the default) posts it to /v1/chat/completions as a single user message, which templates twice; prefer the ChatBackend path there.

messages - Conversation Data

Messages support five roles covering the full agent lifecycle:

use orion_core::Message;

// Standard conversation
let sys = Message::system("msg-1", "You are helpful.");
let user = Message::user("msg-2", "Hello!");
let asst = Message::assistant("msg-3", "Hi there!");

// Tool interaction
let result = Message::tool_result(
    "msg-4",            // message id
    "call-1",           // tool_call_id (links to the assistant's request)
    "read_file",        // tool name
    "file contents...", // result content
    false,              // is_error
);

Roles: System, User, Assistant, ToolCall, ToolResult

Every message has an id, timestamp, and optional token_count, which the agent leaves unset for a host that counts tokens itself to fill in. Assistant messages can carry tool_calls; tool result messages carry a tool_result. A ToolResult has tool_call_id, tool_name, content, is_error and pending, which marks a placeholder for a result still to come and reads as false when missing, so conversations saved by earlier releases load unchanged.

events - Real-Time UI Updates

The agent emits events as it processes. Subscribe to these for building reactive UIs.

Event sequence for a simple prompt:

AgentStart
MessageStart    { user message }
MessageEnd      { user message }
TurnStart
ContextBudget   { used: 120, max: 4096, included: 5, pruned: 0 }
MessageDelta    { delta: "Hello", tokens: 1, tps: 45.2 }
MessageDelta    { delta: " there", tokens: 2, tps: 46.1 }
MessageDelta    { delta: "!", tokens: 3, tps: 44.8 }
GenerationStats { tokens_generated: 3, prompt_tokens: 120, ... }
MessageEnd      { assistant message }
TurnEnd         { message, tool_results: [] }
AgentEnd        { messages: [...] }

Event sequence with tool calls:

AgentStart
MessageStart    { user message }
MessageEnd      { user message }
TurnStart
ContextBudget   ...
MessageDelta    ...
GenerationStats { tokens_generated, prompt_tokens, ... }
MessageEnd      { assistant message with tool_calls }
ToolExecStart   { tool_call_id, tool_name, args }
ToolExecUpdate  { partial progress }
ToolExecEnd     { result }
MessageStart    { tool_result message }
MessageEnd      { tool_result message }
TurnEnd         { message, tool_results: [...] }
TurnStart                            ← new turn: LLM responds to tool result
MessageDelta    ...
GenerationStats { ... }
MessageEnd      { final assistant message }
TurnEnd         { message, tool_results: [] }
AgentEnd        { messages: [...] }

All event types:

Event When Key Data
AgentStart Processing begins -
AgentEnd All done Every message the run added, other than the prompt's own (also on a failed run)
TurnStart New LLM call begins -
TurnEnd LLM call + tools done Assistant message, tool results
MessageStart A user or tool-result message added (an assistant message streams as MessageDelta instead) Full message
MessageDelta Each streamed token delta, tokens_generated, tokens_per_sec
MessageEnd Message complete Full message
ToolExecStart Tool begins running Tool name, args
ToolExecUpdate Tool streams progress Partial output
ToolExecEnd Tool finished Result, is_error, pending
ToolDenied Approval hook refused a call Tool name, reason
ToolResolved A pending call's result arrived via the inbox Result
Paused Run ended to wait on pending calls Pending calls
Nested A child agent's event, from ToolContext::run_child Call id, the child's event
ContextBudget After context prep Tokens used/max, messages included/pruned
Warning Non-fatal issue Warning text
Error Fatal error Error text

context - Token Budget Management

Handles the hard problem of fitting a conversation into a fixed-size context window.

What it does:

  1. Prunes old messages when the conversation exceeds the token budget (sliding window - keeps system prompt + most recent messages)
  2. Formats the surviving messages into a ChatML prompt string
  3. Reports how many tokens are used and how many messages were pruned
use orion_core::ChatMLTemplate;
use orion_core::context::{prepare_context, ContextConfig};

let token_counter = |text: &str| -> u32 { /* your tokenizer */ 0 };

// Prune to fit the budget *and* format with a chat template in one step.
let prepared = prepare_context(
    &ChatMLTemplate,           // any `ChatTemplate` impl
    "You are helpful.",        // system prompt
    &messages,                 // full conversation history
    &tool_schemas,             // tool schemas to inject (may be empty)
    &ContextConfig::default(),
    &token_counter,
)?;

// `prepared.prompt` is the formatted string to feed your backend.
println!(
    "{} tokens, {} kept, {} pruned",
    prepared.token_count, prepared.messages_included, prepared.messages_pruned,
);

The agent calls this automatically before each LLM call. You don't need to call it directly unless you want custom control.

Prune strategies (ContextConfig::prune_strategy):

  • SlidingWindow (default) - drop the oldest turns first to fit the budget.
  • Summarize - before pruning, the agent folds the oldest overflowing turns into a single pinned summary message (one extra backend call), so their gist survives instead of being dropped. Prior summaries are consolidated, so exactly one accumulates. Best-effort: if summarization fails it falls back to the sliding window.

Pinned messages. Any Message with pinned == true always survives pruning, regardless of budget or strategy. Build one with Message::user(id, text).pinned(), or toggle an existing message via agent.set_pinned(message_id, true). Pruning is turn-aware, so a pinned message keeps its whole turn (no orphaned pairs).

template - Chat Prompt Formats

Each model family wants its prompt wrapped a certain way. Orion ships a ChatTemplate for the common ones and picks the right one automatically.

Supported families: ChatML (default), Llama 3, Llama 2, Mistral / Mixtral, Gemma / Gemma 2, Phi-3, DeepSeek (LLM chat), Command-R / Command-R+, Alpaca, and Vicuna.

  • detect_template(gguf_template) - inspects a GGUF metadata template string and returns the matching impl (falling back to ChatML when nothing matches).
  • template_from_name(name) - resolves a manual-override name (with common aliases, e.g. llama-2, phi-3, cohere) to a template, or None for an unimplemented family so the caller can fall back to auto-detection.
  • Agent::with_template(config, template) / agent.set_template(template) - set or swap the template at runtime.

Every template also implements the per-message and per-system formatting hooks the context pipeline needs for accurate token-budget accounting, and advertises tools through the same tool_call convention (see below).

tools - Give the Model Abilities

Agent::prompt drives the full cycle automatically: it injects your tool schemas into the system prompt, parses the model's tool calls out of its reply, runs the matching tool, appends the result to the conversation, and loops back to the model until it returns a tool-free answer (bounded by AgentConfig::max_tool_iterations, default 8). Each step emits ToolExecStart / ToolExecUpdate / ToolExecEnd events.

Tool-call convention. Templates advertise - and [parse_tool_calls] reads - a fenced JSON block:

```tool_call
{"name": "read_file", "arguments": {"path": "Cargo.toml"}}
```

A JSON array invokes several tools in one turn. Parsing is lenient: a ```json fence, a tag with stray punctuation around it such as ```tool_call>, or a whole-message bare JSON object carrying both name and arguments also count, so smaller models still trigger tools when they drift from the exact format. Register tools with agent.set_tools(vec![Box::new(MyTool)]); with no tools registered, parsing is skipped entirely and replies pass through verbatim.

Define tools the model can call. Each tool has a name, description, JSON Schema for parameters, and an async execute function. execute is handed a ToolContext for the call: its id, the run's stop flag (ctx.is_aborted()), ctx.update(..) for progress, and ctx.run_child(..) for running another agent (see Long-running work).

use orion_core::{CoreResult, Tool, ToolContext, ToolOutput};
use async_trait::async_trait;

struct ReadFileTool;

#[async_trait]
impl Tool for ReadFileTool {
    fn name(&self) -> &str { "read_file" }
    fn label(&self) -> &str { "Read File" }
    fn description(&self) -> &str { "Read a file from disk" }

    fn parameters_schema(&self) -> serde_json::Value {
        serde_json::json!({
            "type": "object",
            "properties": {
                "path": { "type": "string", "description": "File path to read" }
            },
            "required": ["path"]
        })
    }

    async fn execute(
        &self,
        args: serde_json::Value,
        _ctx: &ToolContext,
    ) -> CoreResult<ToolOutput> {
        let path = args["path"].as_str().unwrap_or("");
        let content = std::fs::read_to_string(path)
            .map_err(|e| orion_core::CoreError::Tool(e.to_string()))?;
        Ok(ToolOutput::new(content).with_details(serde_json::json!({"path": path})))
    }
}

// Register with the agent
agent.set_tools(vec![Box::new(ReadFileTool)]);

Tool schemas are automatically injected into the system prompt when formatting context, and Agent::prompt runs the full tool call → execute → feed result → LLM responds cycle for you (see the section intro above).

Opting out. The Tool trait and the execution loop live behind the default tools feature, which pulls in async-trait and turns on tokio's macros feature, which ToolContext::run_child uses. Minimal consumers that only need plain chat can drop it:

orion-core = { version = "0.9", default-features = false }

That also drops the default chat-backend feature and with it ChatBackend; add features = ["chat-backend"] back if you drive a hosted chat API.

Tool-call parsing (parse_tool_calls, ParsedToolCall) and ToolSchema stay available either way - only the Tool trait, ToolOutput, ToolContext, and Agent::set_tools require the feature.

Long-running work

Agent::prompt borrows the agent until the run ends, and a tool's execute holds the run until it returns. Four pieces let work outlast that.

Pending results. A tool that starts something slow can return ToolOutput::pending("Started job 7. Its result will follow.") instead of waiting. The placeholder is recorded as the call's result, so the model knows the work is under way, and the call is listed by agent.pending_calls() until the agent reads its real result from the inbox (not the moment you call resolve; a second result for the same call is dropped with a Warning).

The inbox. agent.inbox() is a cloneable handle for reaching the agent from outside its run. inbox.push(text) queues a user message, and inbox.resolve(call_id, content) or inbox.reject(call_id, reason) delivers a pending call's result. The agent reads it before each model call, and again when the model gives its final answer, so anything that lands mid-run is answered in that run.

Resuming. agent.resume(backend, tx) runs the loop with no new user text: after a result lands between runs, after a pause, or after restoring a saved conversation with replace_messages. Pending calls are worked out from the messages, so they survive a save, and a call a crashed run never finished is marked failed rather than run again. With AgentConfig::pause_on_pending set, a turn that leaves calls pending ends the run with AgentEvent::Paused instead of letting the model answer around them.

let inbox = agent.inbox();
agent.prompt("Run the nightly report", backend.clone(), tx).await?;

// Later, when the job the tool started finishes:
for call in agent.pending_calls() {
    inbox.resolve(&call.id, "Report written to out/nightly.csv");
}
agent.resume(backend, tx2).await?;

Child agents. Inside a tool, ctx.run_child(&mut child, task, backend) runs another agent and answers with its reply. The child's events stream live into the parent's as AgentEvent::Nested under the call that started it, and stopping the parent stops the child.

The code snippets in this README are mirrored by compile-checked doctests on the corresponding API items, so they can't silently drift from the real signatures. Run them with cargo test --doc.

skills - Agent Skills (feature skills)

A skill is a folder holding a SKILL.md: YAML frontmatter with a name and a description, then Markdown instructions, often next to scripts and reference files. It is the format Claude Code plugins and other agents share. The model only sees each skill's name and description, so a large collection stays cheap; when a task matches, it calls the load_skill tool to read the instructions or one of the skill's files.

use orion_core::skills::{discover_skills, SkillSet};

// `~/.agents/skills/<name>/SKILL.md`, one skill per folder.
let skills = SkillSet::new(discover_skills("/home/me/.agents/skills".as_ref(), &["."]));
let mut system_prompt = String::from("You are a helpful assistant.");
if let Some(section) = skills.system_section() {
    system_prompt.push_str("\n\n");
    system_prompt.push_str(&section);
}
agent.set_system_prompt(system_prompt);
agent.set_tools(vec![Box::new(skills.tool())]); // with the `tools` feature

discover_skills(root, places) reads skills the way Claude Code reads a plugin's skills setting: each place is a skill folder or a folder of them (skills/ when none are given), plus a SKILL.md at the root. load_skill never reads outside a skill's folder.

Users can also pick a skill themselves with a /name request message. skills.expand_command(text)? returns Some(SkillCommand) whose message carries the skill's instructions in a <skill name="…"> block followed by the request, ready to send as the user turn, so the model needs no tool call; parse_expanded(message) gets the skill and request back for display.

plugins - Plugin Marketplaces (feature plugins)

Fetch a Claude Code marketplace catalog (marketplace.json, or a bare array of its entries such as AgentHub's plugins.json), install plugins from git into a shared store, and read the two things they ship that work in any agent: MCP servers and skills.

use orion_core::plugins::{Catalog, PluginStore};

let catalog = Catalog::fetch("agenthub", "https://agenthub.nullorder.org/plugins.json").await?;
let entry = catalog.get("superpowers-chrome").expect("listed");

let store = PluginStore::new(PluginStore::default_root().expect("home directory"));
// Blocking: run under `spawn_blocking` in real async code.
let prepared = store.prepare(&catalog.name, entry)?;
for (name, server) in &prepared.contents.mcp_servers {
    println!("MCP server {name}: {:?} {:?}", server.command, server.args);
}
let installed = store.commit(prepared)?;
  • The store (~/.agents/plugins by default) keeps each plugin's git checkout at a pinned commit under <marketplace>/<plugin>/, the list of installs in installed.json, and the last catalog of each marketplace in catalogs/. Any tool can read it.
  • Two-step installs. prepare fetches into a staging area (a shallow, sparse checkout for plugins in a subfolder) and reports what the plugin ships, with ${CLAUDE_PLUGIN_ROOT} already pointing at its final folder, so you can show the user exactly what its MCP servers run. commit moves it into place; cancel throws it away. latest_commit tells you when an update is out.
  • Nothing runs on install. Fetching runs no plugin code, git never asks for credentials, and only https:// (and file://) repositories are fetched. npm-sourced plugins are not supported yet.
  • PluginContents reads MCP servers from plugin.json (inline or a path), a root .mcp.json, or the catalog entry; skills via discover_skills; and lists the Claude Code-only parts (commands, agents, hooks, language servers) so you can say what is left out.

error - Error Types

use orion_core::{CoreError, CoreResult};

// Error variants
CoreError::Backend("No model loaded".into())      // LLM backend issues
CoreError::BackendUnreachable("timed out".into()) // No response; safe to retry
CoreError::Context("Token limit exceeded".into()) // Context pipeline issues
CoreError::Tool("File not found".into())          // Tool execution issues
CoreError::Agent("Empty message".into())          // Agent logic issues
CoreError::Aborted                                // User cancelled
CoreError::Skill("no skill named 'x'".into())     // Skill loading (feature `skills`)
CoreError::Plugin("git fetch failed".into())      // Catalogs and installs (feature `plugins`)

All errors are serializable (implements Serialize) for easy transport over IPC.

Architecture

┌──────────────────────────────────────────────────┐
│  Your Application (OrionPod, CLI, server, etc.)  │
├──────────────────────────────────────────────────┤
│  Agent                                           │
│  ├── prompt("Hello") / resume()                  │
│  ├── Inbox (messages + late tool results)        │
│  ├── Conversation state (Vec<Message>)           │
│  ├── System prompt                               │
│  ├── Registered tools                            │
│  └── AgentConfig (inference params, context cfg) │
├──────────────────────────────────────────────────┤
│  Context Pipeline                                │
│  └── prepare_context() - prune + template format │
├──────────────────────────────────────────────────┤
│  LlmBackend (trait) ← you implement this         │
│  ├── generate() - run inference, stream tokens   │
│  ├── tokenize_count() - count tokens             │
│  └── is_ready() - check model status             │
├──────────────────────────────────────────────────┤
│  Your inference engine                           │
│  (llama.cpp, MLX, ONNX, cloud API, etc.)         │
└──────────────────────────────────────────────────┘

Events flow upward through an unbounded channel (tokio::sync::mpsc). Your UI or application layer subscribes to AgentEvents and reacts in real time.

Stability

Orion follows SemVer. While 0.x, a minor bump may carry breaking changes and a patch bump is additive/fixes only. CoreError and AgentEvent are #[non_exhaustive] - match them with a wildcard arm so new variants don't break your build. The MSRV is Rust 1.85 (raised only in a minor release), and that guarantee covers the default feature set; optional features like http-backend and plugins may need a newer toolchain. See CONTRIBUTING.md for the full policy.

License

MIT © Kumar Anirudha. See LICENSE.

About

Open source agent harness for local LLM inference

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

5 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages