Skip to content

About

Dockprom-AI is an out-of-the-box observability and AIOps stack for Docker hosts, containers, and LLMs. It pairs Prometheus, Grafana, Loki, Tempo, and OpenTelemetry with Logporter, host Ollama via Envoy, native Grafana/Prometheus MCP servers, Keep AIOps, Mattermost, and pgvector RAG with Chainlit and Generative UI

Resources

Stars

1 star

Watchers

1 watching

Forks

Repository files navigation

Dockprom-AI Logo

Production-Grade AI Observability & AIOps Platform
Prometheus • Grafana • Loki • Tempo • OpenTelemetry • Host Ollama Gateway • MCP Servers • pgvector OpenUI • OpenIntelligence UI • Chainlit

Unit Tests Integration Tests E2E Tests


Overview: Dockprom-AI

Dockprom-AI is a production-grade, out-of-the-box monitoring, tracing, and AIOps observability stack designed for Docker hosts, containers, and LLM applications. It modernizes the classic stefanprodan/dockprom architecture by incorporating native AI agent interfaces via the Model Context Protocol (MCP), local host-level LLM inference, vector-enabled incident storage, and generative UI capabilities.


Core Architecture & Components

The repository is structured into two main layers:

1. Core Observability & AIOps Stack (compose.yaml / compose.yml)

  • Metrics & Monitoring:

  • Prometheus v3.15.1: Core metrics database and PromQL query engine.

  • Alertmanager v0.34.1: Deduplication, grouping, and notification routing for alerts.

  • Pushgateway v1.10.0: Ingestion acceptor for ephemeral and batch jobs.

  • Node Exporter v1.12.1: Linux kernel, host hardware, and system OS metrics.

  • Logporter 1.5.0: Modern, lightweight container metrics exporter and log shipper (replacing cAdvisor) with a built-in dashboard UI.

  • Logs & Distributed Tracing:

  • Loki 3.7.9: Scalable structured log aggregation utilizing TSDB schema v13

  • Tempo 3.1.1: High-throughput distributed tracing backend.

  • OpenTelemetry Collector Contrib 0.164.0: Unified pipeline receiving OTLP traces, logs, and metrics (via ports 4317 gRPC and 4318 HTTP).

  • Dashboards & Edge Ingress:

  • Grafana 13.3.0: Pre-provisioned dashboards covering Logporter, Node Exporter, PostgreSQL, OpenTelemetry pipelines, and MCP server health.

  • Caddy 2.11.7: Edge reverse proxy terminating bcrypt-hashed basic authentication for exposed services.

  • Incident Management & Collaboration:

  • Keep AIOps v0.54.3: Incident correlation, noise reduction, and automated alert remediation platform.

  • Mattermost Team Edition 12.0.1: Incident response collaboration workspace supporting webhook alert integration.

  • Database & Persistence:

  • pgvector (PostgreSQL 17): Unified persistence layer for Keep, Mattermost, and vector embeddings storage (rag_storage).

  • Postgres Exporter v0.20.1: Database engine monitoring with pg_stat_statements enabled.

  • AI Telemetry & MCP Endpoints:

  • Ollama Gateway (Envoy v1.40.2): Reverse proxy running on port 11435 that bridges containers to native host Ollama (host.docker.internal:11434), collecting latency statistics and OTLP traces.

  • Grafana MCP Server 2.1.1: Official Model Context Protocol server exposing Grafana dashboard and Loki query tools via Server-Sent Events (SSE) on port 8000.

  • Prometheus MCP Server v0.18.0: Official MCP server exposing PromQL tools via SSE on port 8080.


2. Generative UI & Copilot Companion Layer (compose-genui.yaml)

Designed to run alongside the core stack on the shared monitor-net Docker network:

  • Chainlit AIOps Copilot (:8050): A Python-based conversational SRE assistant orchestrating multi-step incident triage, retrieving runbooks via cosine similarity search in pgvector, executing queries through the MCP servers, and streaming reasoning steps.
  • Open-Intelligent-UI (:3001): A standalone Next.js application implementing the Thesys OpenUI streaming framework (@openuidev/react) to dynamically render interactive diagnostic widgets, forms, and charts generated by the LLM.

Key Highlights & Features

  • Works Out-of-the-Box: Includes automated host provisioning scripts (install-host-prereqs.sh) for Docker Engine, Docker Compose, and native host Ollama setup.
  • Hardware-Accelerated LLMs: Runs Ollama directly on the host for direct GPU access without Dockerized CUDA/ROCm complications.
  • RAG-Ready Incident Management: Combines pgvector with OpenTelemetry and PromQL to ground AI agent recommendations directly in live cluster metrics, logs, and historical postmortems.
  • Generative UI: Combines Chainlit + OpenUI + OpenIntelligent-UI

Pre-Configured Grafana Dashboards

Dockprom-AI provisions dashboards out-of-the-box using Grafana file provisioning (grafana/provisioning/dashboards/dashboards.yml). The catalog combines documented upstream dashboard references with locally generated dashboards for AI workflows, security, agents, and platform operations.

Dashboard Manifest

Category Dashboard Name Grafana Labs ID / Source Datasource(s) Description
Container & Host Logporter Docker Exporter 23848 Prometheus, Loki Official Logporter dashboard: CPU, memory, volume sizes, restart counts, image update alerts, and embedded Loki container logs.
Container & Host Node Exporter Full 1860 Prometheus Comprehensive host hardware/OS telemetry: CPU usage by core, memory commit, disk I/O latency, network traffic, and system load.
AI & MCP Observability MCP Server Observability 25252 Prometheus, Loki, Tempo Full-stack monitoring for mcp-grafana and prometheus-mcp: SSE sessions, message rates, JSON-RPC errors, tool invocation latency, and execution logs.
AI & LLM Gateway Envoy Ollama Proxy Custom / Envoy Stats Prometheus, Tempo Request throughput, p50/p95/p99 inferencing latencies, HTTP error rates, and OTLP traces for host Ollama requests.
Telemetry Ingestion OpenTelemetry Collector 15983 Prometheus Collector operational pipeline metrics: receiver queue sizes, batch processor latency, export drops, and data transmission throughput.
Stack Health Prometheus 3.x System Overview 3662 Prometheus Scraping health, TSDB compaction rate, ingestion head chunks, sample scrape rates, and query latency percentiles.
Stack Health Loki Logging Stack Health 13407 Prometheus, Loki Chunks stored, stream indexing status, query throughput, per-tenant volume rates, and flush operations.
Stack Health Tempo Traces & Storage 15984 Prometheus, Tempo Ingestion block generation, WAL size, span compaction schedules, and search request performance.
Ollama Monitoring Ollama LLM Inference 25086 Prometheus, Ollama Exporter Ollama dashboard for the open-source ollama-exporter.
AI Operations AI Operations Command Center Local JSON Prometheus, Loki, Tempo Cross-stack availability, workflow SLOs, errors, alerts, and slow traces.
AI Observability Ollama Inference and Model Performance Local JSON Prometheus, Loki, Tempo OpenTelemetry GenAI operation latency, TTFT, token usage, context utilization, queueing, and model load duration.
GenUI GenUI End-to-End Experience Local JSON Prometheus, Tempo Chainlit, RAG, Ollama, streaming, and OpenUI user-journey measurements.
RAG RAG and pgvector Quality Local JSON Prometheus, PostgreSQL, Loki Retrieval latency, empty results, similarity quality, runbook coverage, and vector-search failures.
AI Security AI Security Detection Center Local JSON Prometheus, Loki, Tempo Prompt injection, jailbreak, leakage, policy actions, and trace-linked detections.
MCP Security MCP Tool Security and Audit Local JSON Prometheus, Loki, Tempo Tool outcomes, risk, confirmations, JSON-RPC errors, audit events, and execution traces.
Database PostgreSQL and pgvector Operations Local JSON Prometheus, PostgreSQL Connections, transactions, cache, deadlocks, growth, long queries, and vector indexes.
Edge Security Edge Authentication and API Security Local JSON Prometheus, Loki Authentication failures, scanning, upstream errors, TLS, and unexpected public binds.
Incidents Incident Response Effectiveness Local JSON Prometheus, Loki Alert outcomes, MTTA, MTTR, notifications, runbook coverage, and incident timelines.
Telemetry Quality OpenTelemetry Signal Quality Local JSON Prometheus Accepted/refused signals, export failures, queues, identity gaps, and cardinality.
Capacity GPU and Model Capacity Local JSON Prometheus Accelerator utilization, memory, thermals, errors, queueing, and token throughput.
Agent Security Agent Harness Security and Operations Local JSON Prometheus, Loki, Tempo Agent outcomes, policy denials, approvals, delegation, budgets, and safety actions.
Runtime Security Runtime and Supply-Chain Security Local JSON Prometheus, Loki Falco detections, Trivy summaries, image policy, and container compliance.

Dashboard provisioning

The repository contains deterministic dashboard JSON in grafana/dashboards and file provisioning in grafana/provisioning. Regenerate and validate the local dashboards before starting Grafana:

python3 scripts/generate_grafana_dashboards.py
find grafana/dashboards -name '*.json' -print0 | xargs -0 -n1 jq empty
GF_SECURITY_ADMIN_PASSWORD='replace-me' \
GRAFANA_POSTGRES_PASSWORD='replace-me' \
POSTGRES_EXPORTER_DSN='postgresql://user:password@mattermost-db:5432/postgres?sslmode=disable' \
  docker compose config --quiet

See DOCKPROM_AI_ROADMAP.md for the implementation phases and website/docs/deployment/telemetry-contract.md for the metrics and security-event contract.

Architecture

The diagrams are stored with the Docusaurus static assets so the same files are rendered on GitHub and in the documentation site.

Core observability and AIOps stack

Dockprom-AI core observability and AIOps architecture

Generative UI and AIOps copilot stack

Dockprom-AI Generative UI and AIOps copilot architecture

See README_GENUI.md for details about the Generative UI companion stack.

Roadmaps and security controls

About

Dockprom-AI is an out-of-the-box observability and AIOps stack for Docker hosts, containers, and LLMs. It pairs Prometheus, Grafana, Loki, Tempo, and OpenTelemetry with Logporter, host Ollama via Envoy, native Grafana/Prometheus MCP servers, Keep AIOps, Mattermost, and pgvector RAG with Chainlit and Generative UI

Resources

Stars

1 star

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages