llama-dash is a dashboard and API gateway for your self-hosted llama-swap + llama.cpp stack. Manage models, inspect requests, and control client access from one place.
Quick start · Manual setup · Client setup · Documentation
OpenAI SDK / Claude Code / Continue / Open WebUI
│
▼
llama-dash :3000
dashboard · auth · logs · routing · metrics
│ │
▼ ▼
llama-swap :8080 direct /v1 upstreams
llama.cpp models · peers OpenAI · Anthropic
- Monitor traffic, token usage, cost estimates, and CPU/GPU health.
- Inspect request history with filters, sorting, shareable time ranges, payloads, and timing details. Usage rollups outlive raw request logs.
- Load and unload models, inspect capabilities and history, pause inference (llama-swap v268+), and edit validated configuration.
- Control access with dashboard login, API keys, rate limits, model permissions, and MCP relay allow-lists.
- Route requests to local models, peers, or direct upstreams. Rewrite model names, reject requests, inject credentials or system prompts, and limit request sizes.
- Test chat, image, speech, transcription, and realtime voice models in the playground.
The bundled stack runs llama-dash and llama-swap, with the dashboard and client API at http://localhost:3000. Choose AMD/Vulkan or NVIDIA/CUDA below. Both use the published ghcr.io/ndom91/llama-dash:latest image.
git clone https://github.com/ndom91/llama-dash.git
cd llama-dash
cp .env.example .env
cp config/config.example.yaml config/config.yaml- Set
BETTER_AUTH_SECRETin.envto the output ofopenssl rand -base64 33. Compose loads.envand won't start without this secret. - Place a GGUF model in
./models/and update its path and model ID inconfig/config.yaml. The example expects./models/llama-3.ggufwith the IDllama3. - If you plan to store upstream provider credentials, also set
CREDENTIAL_ENCRYPTION_KEYin.env.
The example enables logTimeFormat: rfc3339nano so the Logs viewer can read original wall-clock timestamps from llama-swap's own messages, including replayed history. Existing configs can add this top-level option. This setting does not timestamp raw model output such as llama.cpp's process-uptime logs. Without an embedded timestamp, each source's initial batch shows — and subsequent live batches show browser receipt time, identified in the timestamp tooltip. Since llama-swap does not mark the history boundary, a source with no history also leaves its first live batch untimed. Reconnecting reloads history and resets this tracking.
docker compose -f docker-compose.amd.yaml pull
docker compose -f docker-compose.amd.yaml up -dUses llama-swap's unified-vulkan image. The stack passes /dev/dri to both services for inference and dashboard GPU stats.
Requires the NVIDIA Container Toolkit on the host.
docker compose -f docker-compose.nvidia.yaml pull
docker compose -f docker-compose.nvidia.yaml up -dUses llama-swap's unified-cuda13 image with gpus: all. It supports Turing (RTX 20xx, GTX 16xx) and newer GPUs. For Pascal cards such as the GTX 10xx or P40, change the llama-swap image to ghcr.io/mostlygeek/llama-swap:unified-cuda.
Open http://localhost:3000, create the first dashboard user, then load your model from Models. llama-dash runs database migrations automatically at startup.
Image versions, local builds, and config reloads
Set LLAMA_DASH_IMAGE_TAG in .env to a published image tag, such as a commit SHA tag, to pin a specific build.
To build locally, add -f docker-compose.build.yaml and --build to either vendor's up command:
docker compose -f docker-compose.amd.yaml -f docker-compose.build.yaml up -d --buildBoth setups use llama-swap's unified containers and share the ./config/ directory between services. This lets llama-dash save config.yaml atomically and llama-swap reload it with -watch-config. Model files are mounted from ./models/.
For an existing llama-swap instance, install Node 24+ and pnpm, then run these commands from the repository:
cp .env.example .env # edit INFERENCE_BASE_URL to point at your instance
pnpm install
pnpm db:migrate # creates data/dash.db
pnpm dev # http://localhost:5173See the environment variables reference for all variables, defaults, and deployment notes.
The Endpoints page has connection examples for curl, Python, TypeScript, Home Assistant, Claude Code, opencode, Continue, and Open WebUI.
See the Claude Code & Anthropic passthrough guide for proxy setup.
This project was developed with significant assistance from LLMs. Architecture decisions, implementation, and documentation were all shaped through human-AI collaboration.
MIT







