diff --git a/TELEMETRY.md b/TELEMETRY.md new file mode 100644 index 0000000..086ddd8 --- /dev/null +++ b/TELEMETRY.md @@ -0,0 +1,99 @@ +# flAPI Telemetry + +flAPI collects **anonymous, aggregate usage telemetry** to help us understand +which capabilities are used and where the product breaks. It is designed to be +privacy-preserving by construction: only bounded enumerations, route +*templates*, and numbers ever leave the machine — **never** your data, queries, +URLs, credentials, or configuration. + +Telemetry is emitted against the shared DataZoo telemetry schema +(`telemetry_schema: 2`) via the [`posthog-telemetry`](https://github.com/DataZooDE/posthog-telemetry) +library, into PostHog's **EU** ingestion endpoint (`eu.i.posthog.com`). Because +flAPI is a long-running server (not a DuckDB extension), events carry +`install_kind = "server"` and a single session id per server uptime. + +## How to turn it off + +Any **one** of the following disables all telemetry — a single guard +short-circuits every event, and nothing leaves the machine: + +- **Environment variable:** `DATAZOO_DISABLE_TELEMETRY=1` (also `true` / `yes`), + or `FLAPI_NO_TELEMETRY=1`. +- **Config file** (`flapi.yaml`): + ```yaml + telemetry: + enabled: false + ``` +- **CLI flag:** `flapi --no-telemetry`. + +For very high-QPS deployments you can down-sample the per-request/per-tool +events (lifecycle events are always sent in full): + +```yaml +telemetry: + sample_rate: 0.1 # emit 10% of rest_endpoint_served / mcp_tool_called; events carry sample_rate +``` + +## What is collected + +Every event carries a common envelope from the library: `product` (`flapi`), +`product_version`, `product_edition` (`oss`/`enterprise`), `telemetry_schema`, +`os`, `arch`, `platform`, `is_ci`, `is_container`, a per-uptime `$session_id`, +a pseudonymous per-machine `distinct_id` (salted SHA-256 of the OS machine id; +identifies a *machine/install*, not a person), and `$groups` once associated. +flAPI additionally stamps `install_kind = "server"` on every event. + +### Events + +| Event | When | Properties (beyond the envelope) | +|---|---|---| +| `server_started` | server boot | `endpoint_count` (number), `auth_kind` ∈ `none`\|`basic`\|`bearer`\|`oidc` | +| `rest_endpoint_served` | a REST endpoint is served | `method` (`GET`/`POST`/…), `route_template` (e.g. `/customers/:id` — **the template, never the filled path**), `status_class` ∈ `2xx`\|`3xx`\|`4xx`\|`5xx`, `duration_ms` (number), `cache_hit` (bool) | +| `mcp_tool_called` | an MCP tool call completes | `tool` (registered tool name), `status_class`, `duration_ms` (number) | +| `auth_enforced` | auth is enforced on a request | `auth_kind` ∈ `basic`\|`bearer`\|`oidc`, `outcome` ∈ `allow`\|`deny` | +| `$exception` | a request fails | `error_class` ∈ `server_error`\|`bad_request` (REST), `feature`, `route_template` (template) | + +Sampled events additionally carry `sample_rate` (number) so aggregate counts +scale back up. + +### Groups + +- **`deployment`** — always associated at boot; key is the pseudonymous + per-machine `distinct_id`. Powers active-deployment and retention analytics. +- **`account`** — associated only when a license id is present + (`FLAPI_LICENSE_ID`); key is `sha256(license_id)`. The raw license id is never + sent. + +## What is **never** collected + +By design, the following never appear in any property — call sites only pass +enums/templates/numbers, and the library additionally clamps every string +property to 512 bytes as a backstop: + +- Filled request paths or query strings (only the **route template**). +- SQL text, rendered templates, or query plans. +- Request or response **bodies**, or any **row data**. +- HTTP **headers**, tokens, passwords, or connection strings. +- Table names, file paths, hostnames, or user names. +- Free-form error messages (only an enumerated `error_class`). + +## Notes on specific fields + +- **`cache_hit`**: flAPI's cache is a materialized DuckLake table that an + endpoint's SQL queries against — there is no per-request "served-from-cache" + flag. This field therefore reports whether the **endpoint is cache-backed** + (`cache.enabled` with a cache table), not a true per-request hit/miss. +- **`$session_id`**: one id per **server uptime**, so PostHog Paths reflect what + a deployment served during a single run. It is never tied to any end-user + identity. + +## Where this lives in the code + +- Facade + gating: `src/flapi_telemetry.{hpp,cpp}` (`flapi::GlobalTelemetry()`). +- Boot / shutdown (server_started, groups, `Flush()` on SIGINT/SIGTERM): + `src/main.cpp`. +- `rest_endpoint_served` + errors: `src/api_server.cpp` (`handleDynamicRequest`). +- `mcp_tool_called`: `src/mcp_tool_handler.cpp` (`executeTool`). +- `auth_enforced`: `src/auth_middleware.cpp` (`before_handle`). +- Tests (incl. a no-leak assertion against the real transport): + `test/cpp/test_flapi_telemetry.cpp`. diff --git a/src/api_server.cpp b/src/api_server.cpp index c420dac..25d68e7 100644 --- a/src/api_server.cpp +++ b/src/api_server.cpp @@ -3,6 +3,9 @@ #include "api_server.hpp" #include "auth_middleware.hpp" #include "database_manager.hpp" +#include "flapi_telemetry.hpp" + +#include #include "config_service.hpp" #include "config_tool_adapter.hpp" #include "open_api_doc_generator.hpp" @@ -198,29 +201,48 @@ void APIServer::setupHeartbeat() { heartbeatWorker->start(); } -void APIServer::handleDynamicRequest(const crow::request& req, crow::response& res) +void APIServer::handleDynamicRequest(const crow::request& req, crow::response& res) { + const auto t0 = std::chrono::steady_clock::now(); std::string path = req.url; // Match endpoint by both path and HTTP method std::string method = crow::method_name(req.method); const auto& endpoint = configManager->getEndpointForPathAndMethod(path, method); + // Emit one rest_endpoint_served with the ROUTE TEMPLATE (never the filled + // path), status class, duration, and whether the endpoint is cache-backed. + // Errors are emitted with an enumerated class only (no message/SQL/body). + auto emit_telemetry = [&](const std::string& route_template, bool cache_hit) { + const double duration_ms = + std::chrono::duration( + std::chrono::steady_clock::now() - t0).count(); + auto& telemetry = flapi::GlobalTelemetry(); + telemetry.restEndpointServed(method, route_template, res.code, duration_ms, cache_hit); + if (res.code >= 500) { + telemetry.error("server_error", "rest_endpoint_served", route_template); + } else if (res.code == 400) { + telemetry.error("bad_request", "rest_endpoint_served", route_template); + } + }; + if (!endpoint) { res.code = 404; res.body = "Not Found"; res.end(); + emit_telemetry("", false); return; } std::vector paramNames; std::map pathParams; - + bool matched = RouteTranslator::matchAndExtractParams(endpoint->urlPath, path, paramNames, pathParams); if (!matched) { res.code = 404; res.body = "Not Found"; res.end(); + emit_telemetry("", false); return; } @@ -243,6 +265,11 @@ void APIServer::handleDynamicRequest(const crow::request& req, crow::response& r } requestHandler.handleRequest(req, res, *endpoint, pathParams, auth_params); + + // flapi has no per-request cache hit/miss signal (the "cache" is a + // materialized DuckLake table the template queries); the honest bounded + // value is whether this endpoint is cache-backed. See TELEMETRY.md. + emit_telemetry(endpoint->urlPath, dbManager->isCacheEnabled(*endpoint)); } crow::response APIServer::getConfig() { diff --git a/src/auth_middleware.cpp b/src/auth_middleware.cpp index 83fd87c..bfdb4d4 100644 --- a/src/auth_middleware.cpp +++ b/src/auth_middleware.cpp @@ -18,6 +18,7 @@ #include "auth_middleware.hpp" #include "password_hasher.hpp" #include "database_manager.hpp" +#include "flapi_telemetry.hpp" #include "duckdb.hpp" #include "duckdb/common/types/blob.hpp" #include "oidc_provider_presets.hpp" @@ -153,12 +154,16 @@ void AuthMiddleware::before_handle(crow::request& req, crow::response& res, cont CROW_LOG_DEBUG << "Auth enabled for endpoint: " << req.url; + // auth_kind is the endpoint's configured scheme (enum, never a secret). + const std::string auth_kind = endpoint->auth.type; + auto auth_header = req.get_header_value("Authorization"); if (auth_header.empty()) { CROW_LOG_DEBUG << "No Authorization header found"; res.code = 401; res.set_header("WWW-Authenticate", "Basic realm=\"flAPI\""); res.end(); + flapi::GlobalTelemetry().authEnforced(auth_kind, /*allow=*/false); return; } @@ -180,6 +185,7 @@ void AuthMiddleware::before_handle(crow::request& req, crow::response& res, cont } else { CROW_LOG_DEBUG << "Authentication successful for user: " << ctx.username; } + flapi::GlobalTelemetry().authEnforced(auth_kind, ctx.authenticated); } bool AuthMiddleware::authenticateBasic(const std::string& auth_header, const EndpointConfig& endpoint, context& ctx) { diff --git a/src/config_manager.cpp b/src/config_manager.cpp index 7b00208..c39761d 100644 --- a/src/config_manager.cpp +++ b/src/config_manager.cpp @@ -136,7 +136,9 @@ void ConfigManager::parseMainConfig() { if (config["telemetry"]) { telemetry_enabled = safeGet(config["telemetry"], "enabled", "telemetry.enabled", true); - CROW_LOG_DEBUG << "Telemetry Enabled: " << telemetry_enabled; + telemetry_sample_rate = safeGet(config["telemetry"], "sample_rate", "telemetry.sample_rate", 1.0); + CROW_LOG_DEBUG << "Telemetry Enabled: " << telemetry_enabled + << ", sample_rate: " << telemetry_sample_rate; } } catch (const std::exception& e) { CROW_LOG_ERROR << "Error in parseMainConfig: " << e.what(); diff --git a/src/flapi_telemetry.cpp b/src/flapi_telemetry.cpp index 3b941ef..36bc39f 100644 --- a/src/flapi_telemetry.cpp +++ b/src/flapi_telemetry.cpp @@ -2,67 +2,239 @@ #include "telemetry.hpp" +#include + +#include +#include +#include #include #include namespace flapi { +namespace { + +// SHA-256 hex digest. Used to hash a license id into the (non-PII) account +// group key. OpenSSL is already linked into flapi-lib. +std::string Sha256Hex(const std::string& input) { + std::array digest{}; + ::SHA256(reinterpret_cast(input.data()), input.size(), + digest.data()); + static const char* const kHex = "0123456789abcdef"; + std::string out; + out.reserve(digest.size() * 2); + for (unsigned char byte : digest) { + out.push_back(kHex[byte >> 4]); + out.push_back(kHex[byte & 0x0f]); + } + return out; +} + +bool EnvDisablesTelemetry() { + const char* val = std::getenv("DATAZOO_DISABLE_TELEMETRY"); + if (val == nullptr) { + return false; + } + std::string s(val); + return (s == "1" || s == "true" || s == "yes"); +} + +} // namespace + // ── PostHogBackend ──────────────────────────────────────────────────────────── -void PostHogBackend::captureStart(const std::string& app_name, - const std::string& app_version) -{ - duckdb::PostHogTelemetry::Instance().CaptureApplicationStart(app_name, app_version); +void PostHogBackend::setProduct(const std::string& name, const std::string& version, + const std::string& edition) { + duckdb::PostHogTelemetry::Instance().SetProduct(name, version, edition); } -void PostHogBackend::captureStop(const std::string& app_name, - const std::string& app_version) -{ - duckdb::PostHogTelemetry::Instance().CaptureApplicationStop(app_name, app_version); +void PostHogBackend::associateGroup(const std::string& type, const std::string& key) { + duckdb::PostHogTelemetry::Instance().AssociateGroup(type, key); +} + +void PostHogBackend::capture(const std::string& event, duckdb::PropertyMap props) { + duckdb::PostHogTelemetry::Instance().Capture(event, std::move(props)); +} + +void PostHogBackend::captureFeature(const std::string& feature, duckdb::PropertyMap props) { + duckdb::PostHogTelemetry::Instance().CaptureFeature(feature, std::move(props)); +} + +void PostHogBackend::captureError(const std::string& error_class, duckdb::PropertyMap props) { + duckdb::PostHogTelemetry::Instance().CaptureError(error_class, std::move(props)); +} + +void PostHogBackend::flush() { + duckdb::PostHogTelemetry::Instance().Flush(); } // ── FlapiTelemetry ──────────────────────────────────────────────────────────── FlapiTelemetry::FlapiTelemetry() - : backend_(std::make_unique()), enabled_(true) -{} + : backend_(std::make_unique()) {} FlapiTelemetry::FlapiTelemetry(std::unique_ptr backend) - : backend_(std::move(backend)), enabled_(true) -{} + : backend_(std::move(backend)) {} -void FlapiTelemetry::setEnabled(bool enabled) -{ +void FlapiTelemetry::setEnabled(bool enabled) { enabled_ = enabled; } -bool FlapiTelemetry::isTelemetryDisabled(bool enabled) -{ - if (!enabled) { +bool FlapiTelemetry::isEnabled() const { + return active(); +} + +bool FlapiTelemetry::active() const { + return enabled_ && !EnvDisablesTelemetry(); +} + +void FlapiTelemetry::setSampling(double rate) { + if (!(rate > 0.0) || rate >= 1.0 || std::isnan(rate)) { + // Out of range / disabled sampling: emit everything. + sample_rate_ = 1.0; + sample_stride_ = 1; + return; + } + sample_rate_ = rate; + // 1-of-N decimation; round to the nearest whole stride. + sample_stride_ = static_cast(std::llround(1.0 / rate)); + if (sample_stride_ < 1) { + sample_stride_ = 1; + } +} + +bool FlapiTelemetry::sampleHot() { + if (sample_stride_ <= 1) { return true; } - const char* val = std::getenv("DATAZOO_DISABLE_TELEMETRY"); - if (!val) { - return false; + // Emit exactly one of every `sample_stride_` calls. Deterministic so the + // decimation is testable and needs no shared RNG on the hot path. + return (sample_counter_++ % sample_stride_) == 0; +} + +void FlapiTelemetry::stampCommon(duckdb::PropertyMap& props, bool hot) const { + props["install_kind"] = INSTALL_KIND; + if (hot && sample_stride_ > 1) { + props["sample_rate"] = sample_rate_; // JSON number, scales counts back up } - std::string s(val); - return (s == "1" || s == "true" || s == "yes"); } -void FlapiTelemetry::notifyStart(const std::string& version) -{ - if (isTelemetryDisabled(enabled_)) { +std::string FlapiTelemetry::statusClass(int code) { + if (code >= 100 && code < 600) { + switch (code / 100) { + case 1: return "1xx"; + case 2: return "2xx"; + case 3: return "3xx"; + case 4: return "4xx"; + case 5: return "5xx"; + default: break; + } + } + return "unknown"; +} + +void FlapiTelemetry::configureProduct(const std::string& version, + const std::string& edition) { + if (!active()) { + return; + } + backend_->setProduct(PRODUCT, version, edition); +} + +void FlapiTelemetry::associateDeployment() { + if (!active()) { + return; + } + // The deployment group key is the pseudonymous per-machine distinct_id. + backend_->associateGroup("deployment", duckdb::PostHogTelemetry::GetDistinctId()); +} + +void FlapiTelemetry::associateAccount(const std::string& license_id) { + if (!active() || license_id.empty()) { return; } - backend_->captureStart(APP_NAME, version); + backend_->associateGroup("account", Sha256Hex(license_id)); } -void FlapiTelemetry::notifyStop(const std::string& version) -{ - if (isTelemetryDisabled(enabled_)) { +void FlapiTelemetry::serverStarted(int endpoint_count, const std::string& auth_kind) { + if (!active()) { return; } - backend_->captureStop(APP_NAME, version); + duckdb::PropertyMap props; + props["endpoint_count"] = endpoint_count; // JSON number + props["auth_kind"] = auth_kind; // enum: none|basic|bearer|oidc + stampCommon(props, /*hot=*/false); + backend_->capture("server_started", std::move(props)); +} + +void FlapiTelemetry::restEndpointServed(const std::string& method, + const std::string& route_template, + int status_code, double duration_ms, + bool cache_hit) { + if (!active() || !sampleHot()) { + return; + } + duckdb::PropertyMap props; + props["method"] = method; // enum: GET|POST|... + props["route_template"] = route_template; // TEMPLATE, never filled path + props["status_class"] = statusClass(status_code); + props["duration_ms"] = duration_ms; // JSON number + props["cache_hit"] = cache_hit; // JSON bool + stampCommon(props, /*hot=*/true); + backend_->captureFeature("rest_endpoint_served", std::move(props)); +} + +void FlapiTelemetry::mcpToolCalled(const std::string& tool, bool success, + double duration_ms) { + if (!active() || !sampleHot()) { + return; + } + duckdb::PropertyMap props; + props["tool"] = tool; // registered, bounded tool name + props["status_class"] = success ? "2xx" : "5xx"; + props["duration_ms"] = duration_ms; // JSON number + stampCommon(props, /*hot=*/true); + backend_->captureFeature("mcp_tool_called", std::move(props)); +} + +void FlapiTelemetry::authEnforced(const std::string& auth_kind, bool allow) { + if (!active()) { + return; + } + duckdb::PropertyMap props; + props["auth_kind"] = auth_kind; // enum: basic|bearer|oidc + props["outcome"] = allow ? "allow" : "deny"; + stampCommon(props, /*hot=*/false); + backend_->captureFeature("auth_enforced", std::move(props)); +} + +void FlapiTelemetry::error(const std::string& error_class, const std::string& feature, + const std::string& route_template) { + if (!active()) { + return; + } + duckdb::PropertyMap props; + props["feature"] = feature; // enum: which capability + props["route_template"] = route_template; // TEMPLATE, never filled path + stampCommon(props, /*hot=*/false); + // error_class must be an enumerated class — never a message/SQL/user data. + backend_->captureError(error_class, std::move(props)); +} + +void FlapiTelemetry::flush() { + if (!active()) { + return; + } + backend_->flush(); +} + +// ── Process-wide instance ───────────────────────────────────────────────────── + +FlapiTelemetry& GlobalTelemetry() { + // Intentionally leaked (never destroyed): matches the underlying library + // singleton lifetime and keeps late captures during teardown safe. + static FlapiTelemetry* instance = new FlapiTelemetry(); + return *instance; } } // namespace flapi diff --git a/src/include/config_manager.hpp b/src/include/config_manager.hpp index f484dff..fbc011b 100644 --- a/src/include/config_manager.hpp +++ b/src/include/config_manager.hpp @@ -561,6 +561,8 @@ class ConfigManager { // request lands in the same JSONL stream. std::shared_ptr getAuditLogger(); bool isTelemetryEnabled() const { return telemetry_enabled; } + double getTelemetrySampleRate() const { return telemetry_sample_rate; } + const AuthConfig& getGlobalAuthConfig() const { return global_auth_config; } // Load MCP server instructions (inline or from file) std::string loadMCPInstructions() const; @@ -626,6 +628,7 @@ class ConfigManager { AuditConfig audit_config; std::shared_ptr audit_logger_; bool telemetry_enabled = true; + double telemetry_sample_rate = 1.0; ExtendedYamlParser yaml_parser; // Extracted classes for delegation (Facade pattern) diff --git a/src/include/flapi_telemetry.hpp b/src/include/flapi_telemetry.hpp index dec28af..4c4e767 100644 --- a/src/include/flapi_telemetry.hpp +++ b/src/include/flapi_telemetry.hpp @@ -3,51 +3,128 @@ #include #include +#include "telemetry.hpp" // duckdb::PostHogTelemetry, PropertyMap, PropertyValue + namespace flapi { -// Pure interface – inject a mock in unit tests, PostHogBackend in production +// ───────────────────────────────────────────────────────────────────────────── +// flapi telemetry facade +// +// flapi is a long-running *server*, not a DuckDB extension, so it emits against +// the shared DataZoo telemetry schema (telemetry_schema: 2) with +// install_kind="server" and one $session_id per uptime (the library already +// mints a per-process session id). See TELEMETRY.md for the exact catalogue of +// events and properties. +// +// Every property flapi attaches is a bounded enum, a template, or a number — +// never a filled URL, SQL text, request/response body, header, or connection +// string. That contract is enforced by construction here (call sites only pass +// enums/templates/numbers) and by the library's 512-byte clamp as a backstop. +// ───────────────────────────────────────────────────────────────────────────── + +// Pure interface – inject a fake in unit tests, PostHogBackend in production. +// Takes the library's typed PropertyMap so numeric/boolean props keep their +// JSON type (load-bearing for HogQL sum()/avg() and boolean filters). struct ITelemetryBackend { virtual ~ITelemetryBackend() = default; - virtual void captureStart(const std::string& app_name, - const std::string& app_version) = 0; - virtual void captureStop(const std::string& app_name, - const std::string& app_version) = 0; + virtual void setProduct(const std::string& name, const std::string& version, + const std::string& edition) = 0; + virtual void associateGroup(const std::string& type, const std::string& key) = 0; + virtual void capture(const std::string& event, duckdb::PropertyMap props) = 0; + virtual void captureFeature(const std::string& feature, duckdb::PropertyMap props) = 0; + virtual void captureError(const std::string& error_class, duckdb::PropertyMap props) = 0; + virtual void flush() = 0; }; -// Production backend: delegates to duckdb::PostHogTelemetry singleton +// Production backend: delegates to the duckdb::PostHogTelemetry singleton. class PostHogBackend : public ITelemetryBackend { public: - void captureStart(const std::string& app_name, - const std::string& app_version) override; - void captureStop(const std::string& app_name, - const std::string& app_version) override; + void setProduct(const std::string& name, const std::string& version, + const std::string& edition) override; + void associateGroup(const std::string& type, const std::string& key) override; + void capture(const std::string& event, duckdb::PropertyMap props) override; + void captureFeature(const std::string& feature, duckdb::PropertyMap props) override; + void captureError(const std::string& error_class, duckdb::PropertyMap props) override; + void flush() override; }; -// flapi-level telemetry facade. -// Checks DATAZOO_DISABLE_TELEMETRY before delegating to the backend. +// flapi-level telemetry facade. All emit methods are gated by a single guard +// (`active()`), so a single opt-out — env, YAML, or CLI flag — short-circuits +// everything. Every method is non-blocking: the library enqueues onto a +// background worker and returns immediately, so nothing runs on the request +// thread beyond building a small property map. class FlapiTelemetry { public: - // Production: creates a PostHogBackend + // Production: creates a PostHogBackend. FlapiTelemetry(); - // Test injection: takes ownership of provided backend + // Test injection: takes ownership of the provided backend. explicit FlapiTelemetry(std::unique_ptr backend); - // Emit application_start event - void notifyStart(const std::string& version); + // Programmatic opt-out (CLI flag / env var / config file resolve to this). + void setEnabled(bool enabled); + bool isEnabled() const; + + // Client-side sampling for the hot per-request/per-tool paths. rate in + // (0,1]; 1.0 (default) emits every event. Events that survive sampling are + // stamped with sample_rate so counts scale back up. Low-volume lifecycle + // events (server_started, auth_enforced, errors) are always emitted. + void setSampling(double rate); - // Emit application_stop event - void notifyStop(const std::string& version); + // ── Boot ──────────────────────────────────────────────────────────────── + // Set product/version/edition and stamp install_kind on the envelope path. + void configureProduct(const std::string& version, const std::string& edition); + // Associate the deployment group (key == pseudonymous per-machine id). + void associateDeployment(); + // Enterprise only: associate the account group with sha256(license_id). + void associateAccount(const std::string& license_id); + // Emit server_started with bounded counts/kinds only. + void serverStarted(int endpoint_count, const std::string& auth_kind); - // Programmatic opt-out (CLI flag, env var, config file) - void setEnabled(bool enabled); + // ── Runtime (hot paths — sampled) ───────────────────────────────────────── + void restEndpointServed(const std::string& method, + const std::string& route_template, + int status_code, double duration_ms, bool cache_hit); + void mcpToolCalled(const std::string& tool, bool success, double duration_ms); + + // ── Runtime (low volume — always emitted) ──────────────────────────────── + void authEnforced(const std::string& auth_kind, bool allow); + void error(const std::string& error_class, const std::string& feature, + const std::string& route_template); + + // Synchronously drain buffered events (call on clean shutdown / SIGTERM — + // the library's at-exit handler *discards* by design). + void flush(); + + // Map an HTTP status code to a bounded status class ("2xx".."5xx"). + static std::string statusClass(int code); private: - static bool isTelemetryDisabled(bool enabled); + // enabled_ AND not disabled via DATAZOO_DISABLE_TELEMETRY. + bool active() const; + // Decimate a hot-path event; returns true if this call should be emitted. + bool sampleHot(); + // Stamp install_kind (and, when sampling, sample_rate) onto a prop map. + void stampCommon(duckdb::PropertyMap& props, bool hot) const; std::unique_ptr backend_; bool enabled_ = true; - static constexpr const char* APP_NAME = "flapi"; + + // Deterministic 1-of-N decimation for the hot paths (counter, not RNG, so + // it is testable and needs no global random state). + double sample_rate_ = 1.0; + uint64_t sample_stride_ = 1; + uint64_t sample_counter_ = 0; + + static constexpr const char* PRODUCT = "flapi"; + static constexpr const char* INSTALL_KIND = "server"; }; +// Process-wide telemetry instance used by server components (api_server, MCP +// tool handler, auth middleware) to emit without constructor plumbing. main() +// configures it at boot and flushes it on shutdown. Never destroyed (leaked +// like the underlying library singleton) so late captures during teardown are +// safe. +FlapiTelemetry& GlobalTelemetry(); + } // namespace flapi diff --git a/src/include/mcp_tool_handler.hpp b/src/include/mcp_tool_handler.hpp index 5e16f4c..f2a4b4d 100644 --- a/src/include/mcp_tool_handler.hpp +++ b/src/include/mcp_tool_handler.hpp @@ -40,7 +40,9 @@ class MCPToolHandler { std::shared_ptr config_manager); ~MCPToolHandler() = default; - // Tool execution + // Tool execution. Thin wrapper that times the call and emits the + // `mcp_tool_called` telemetry event (bounded tool name + status + duration + // only); the real work lives in executeToolImpl. MCPToolExecutionResult executeTool(const MCPToolCallRequest& request); // Tool validation @@ -57,6 +59,9 @@ class MCPToolHandler { const std::unordered_map& context); private: + // Real tool execution body (wrapped by executeTool for telemetry). + MCPToolExecutionResult executeToolImpl(const MCPToolCallRequest& request); + // Helper methods to work with unified EndpointConfig const EndpointConfig* getEndpointConfigByToolName(const std::string& tool_name) const; diff --git a/src/main.cpp b/src/main.cpp index 5009b51..1cc19b3 100644 --- a/src/main.cpp +++ b/src/main.cpp @@ -39,7 +39,42 @@ using namespace flapi; // Add global variable for signal handling std::atomic should_exit(false); std::shared_ptr api_server; -std::shared_ptr flapi_telemetry; + +// Derive a single, bounded auth_kind for the server_started envelope. +// flapi auth is per-endpoint with an optional global block and a separate MCP +// block; we collapse that to one enum {none,basic,bearer,oidc} for telemetry. +// Counts/kinds only — never routes, users, secrets. +static std::string deriveAuthKind(const ConfigManager& cfg) { + auto normalize = [](const std::string& t) -> std::string { + if (t == "basic" || t == "bearer" || t == "oidc") { + return t; + } + return ""; + }; + + if (cfg.isAuthEnabled()) { + std::string t = normalize(cfg.getGlobalAuthConfig().type); + if (!t.empty()) { + return t; + } + } + for (const auto& endpoint : cfg.getEndpoints()) { + if (endpoint.auth.enabled) { + std::string t = normalize(endpoint.auth.type); + if (!t.empty()) { + return t; + } + } + } + const auto& mcp_auth = cfg.getMCPConfig().auth; + if (mcp_auth.enabled) { + std::string t = normalize(mcp_auth.type); + if (!t.empty()) { + return t; + } + } + return "none"; +} void set_log_level(const std::string& log_level) { if (log_level == "debug") { @@ -286,12 +321,13 @@ LONG WINAPI windowsExceptionHandler(EXCEPTION_POINTERS* exceptionInfo) { void signal_handler(int signal) { - if (signal == SIGINT) { - CROW_LOG_INFO << "Received SIGINT, shutting down..."; + if (signal == SIGINT || signal == SIGTERM) { + CROW_LOG_INFO << "Received " << (signal == SIGINT ? "SIGINT" : "SIGTERM") + << ", shutting down..."; should_exit = true; - if (flapi_telemetry) { - flapi_telemetry->notifyStop(FLAPI_VERSION); - } + // Drain buffered telemetry before exit: the library's at-exit handler + // discards in-flight events by design, so a server must flush explicitly. + flapi::GlobalTelemetry().flush(); if (api_server) { api_server->stop(); } @@ -304,12 +340,14 @@ int main(int argc, char* argv[]) #ifdef _WIN32 SetUnhandledExceptionFilter(windowsExceptionHandler); signal(SIGINT, signal_handler); + signal(SIGTERM, signal_handler); #else struct sigaction sa; sa.sa_handler = signal_handler; sigemptyset(&sa.sa_mask); sa.sa_flags = 0; sigaction(SIGINT, &sa, nullptr); + sigaction(SIGTERM, &sa, nullptr); #endif static argparse::ArgumentParser program("flapi"); @@ -577,12 +615,28 @@ int main(int argc, char* argv[]) config_service_token ); - // Initialize telemetry and emit startup event - flapi_telemetry = std::make_shared(); - if (no_telemetry) { - flapi_telemetry->setEnabled(false); + // Initialize telemetry (this is a long-running server: install_kind="server", + // one $session_id per uptime) and emit server_started. A single opt-out — + // CLI flag, env, or YAML, already resolved into no_telemetry — disables + // everything via setEnabled(false). + { + auto& telemetry = flapi::GlobalTelemetry(); + telemetry.setEnabled(!no_telemetry); + telemetry.setSampling(config_manager->getTelemetrySampleRate()); + + const char* edition_env = std::getenv("FLAPI_EDITION"); + const std::string edition = + (edition_env != nullptr && *edition_env != '\0') ? edition_env : "oss"; + telemetry.configureProduct(FLAPI_VERSION, edition); + telemetry.associateDeployment(); + if (const char* lic = std::getenv("FLAPI_LICENSE_ID"); + lic != nullptr && *lic != '\0') { + telemetry.associateAccount(lic); + } + telemetry.serverStarted( + static_cast(config_manager->getEndpoints().size()), + deriveAuthKind(*config_manager)); } - flapi_telemetry->notifyStart(FLAPI_VERSION); // Start unified server std::thread unified_server_thread([config_manager, server = api_server]() { @@ -610,9 +664,9 @@ int main(int argc, char* argv[]) // Wait for server to finish unified_server_thread.join(); - // Emit shutdown event on normal (non-signal) exit; signal_handler handles the SIGINT path - if (!should_exit && flapi_telemetry) { - flapi_telemetry->notifyStop(FLAPI_VERSION); + // Drain buffered telemetry on clean exit; the signal path already flushed. + if (!should_exit) { + flapi::GlobalTelemetry().flush(); } return 0; diff --git a/src/mcp_tool_handler.cpp b/src/mcp_tool_handler.cpp index 1d0e6f8..80fafc7 100644 --- a/src/mcp_tool_handler.cpp +++ b/src/mcp_tool_handler.cpp @@ -5,9 +5,22 @@ #include "mcp_dry_run.hpp" #include "mcp_response_shaper.hpp" +#include "flapi_telemetry.hpp" namespace flapi { +MCPToolExecutionResult MCPToolHandler::executeTool(const MCPToolCallRequest& request) { + const auto t0 = std::chrono::steady_clock::now(); + MCPToolExecutionResult result = executeToolImpl(request); + const double duration_ms = + std::chrono::duration( + std::chrono::steady_clock::now() - t0).count(); + // Bounded, registered tool name + success/duration only — never arguments, + // SQL, or results. + flapi::GlobalTelemetry().mcpToolCalled(request.tool_name, result.success, duration_ms); + return result; +} + MCPToolHandler::MCPToolHandler(std::shared_ptr db_manager, std::shared_ptr config_manager) : db_manager(db_manager), config_manager(config_manager), @@ -17,7 +30,7 @@ MCPToolHandler::MCPToolHandler(std::shared_ptr db_manager, { } -MCPToolExecutionResult MCPToolHandler::executeTool(const MCPToolCallRequest& request) { +MCPToolExecutionResult MCPToolHandler::executeToolImpl(const MCPToolCallRequest& request) { const auto audit_started_at = std::chrono::steady_clock::now(); const auto emit_audit = [&](const std::string& status, std::int64_t row_count) { if (!audit_logger || !audit_logger->isEnabled()) { diff --git a/test/cpp/test_flapi_telemetry.cpp b/test/cpp/test_flapi_telemetry.cpp index 20b5494..8cb4804 100644 --- a/test/cpp/test_flapi_telemetry.cpp +++ b/test/cpp/test_flapi_telemetry.cpp @@ -1,15 +1,17 @@ #include #include "flapi_telemetry.hpp" +#include "telemetry.hpp" #include #include +#include namespace { -// RAII helper to set/restore an env var +// RAII helper to set/restore an env var. struct EnvGuard { - explicit EnvGuard(const char* name, const char* value) : name_(name) { + EnvGuard(const char* name, const char* value) : name_(name) { const char* existing = std::getenv(name); prev_ = existing ? existing : ""; has_prev_ = (existing != nullptr); @@ -27,176 +29,283 @@ struct EnvGuard { bool has_prev_; }; -struct MockBackend : public flapi::ITelemetryBackend { - int start_calls = 0; - int stop_calls = 0; - std::string last_start_app; - std::string last_start_ver; - std::string last_stop_app; - std::string last_stop_ver; - - void captureStart(const std::string& app_name, - const std::string& app_version) override { - start_calls++; - last_start_app = app_name; - last_start_ver = app_version; +// Records every call so tests can assert on event names and property maps. +struct FakeBackend : public flapi::ITelemetryBackend { + struct Call { + std::string kind; // "capture" | "feature" | "error" + std::string name; // event / feature / error_class + duckdb::PropertyMap props; + }; + std::vector calls; + std::vector> groups; // (type, key) + std::string product_name, product_version, product_edition; + int flushes = 0; + + void setProduct(const std::string& name, const std::string& version, + const std::string& edition) override { + product_name = name; + product_version = version; + product_edition = edition; } + void associateGroup(const std::string& type, const std::string& key) override { + groups.emplace_back(type, key); + } + void capture(const std::string& event, duckdb::PropertyMap props) override { + calls.push_back({"capture", event, std::move(props)}); + } + void captureFeature(const std::string& feature, duckdb::PropertyMap props) override { + calls.push_back({"feature", feature, std::move(props)}); + } + void captureError(const std::string& error_class, duckdb::PropertyMap props) override { + calls.push_back({"error", error_class, std::move(props)}); + } + void flush() override { flushes++; } - void captureStop(const std::string& app_name, - const std::string& app_version) override { - stop_calls++; - last_stop_app = app_name; - last_stop_ver = app_version; + const Call* find(const std::string& name) const { + for (const auto& c : calls) { + if (c.name == name) { + return &c; + } + } + return nullptr; } }; -} // anonymous namespace - -TEST_CASE("FlapiTelemetry - notifyStart calls backend once with correct args", "[flapi_telemetry]") { - auto mock = std::make_unique(); - MockBackend* raw = mock.get(); - - flapi::FlapiTelemetry telemetry(std::move(mock)); - telemetry.notifyStart("1.2.3"); - - REQUIRE(raw->start_calls == 1); - REQUIRE(raw->last_start_ver == "1.2.3"); - REQUIRE(raw->last_start_app == "flapi"); +// Build a FlapiTelemetry over a FakeBackend and hand back the raw pointer. +flapi::FlapiTelemetry makeTelemetry(FakeBackend*& raw) { + auto fake = std::make_unique(); + raw = fake.get(); + return flapi::FlapiTelemetry(std::move(fake)); } -TEST_CASE("FlapiTelemetry - notifyStop calls backend once with correct args", "[flapi_telemetry]") { - auto mock = std::make_unique(); - MockBackend* raw = mock.get(); - - flapi::FlapiTelemetry telemetry(std::move(mock)); - telemetry.notifyStop("1.2.3"); - - REQUIRE(raw->stop_calls == 1); - REQUIRE(raw->last_stop_ver == "1.2.3"); - REQUIRE(raw->last_stop_app == "flapi"); +bool hasStringProp(const duckdb::PropertyMap& p, const std::string& key, + const std::string& value) { + auto it = p.find(key); + return it != p.end() && + it->second.kind == duckdb::PropertyValue::Kind::String && + it->second.s == value; } -TEST_CASE("FlapiTelemetry - DATAZOO_DISABLE_TELEMETRY=1 suppresses notifyStart", "[flapi_telemetry]") { - EnvGuard guard("DATAZOO_DISABLE_TELEMETRY", "1"); - - auto mock = std::make_unique(); - MockBackend* raw = mock.get(); +} // namespace - flapi::FlapiTelemetry telemetry(std::move(mock)); - telemetry.notifyStart("1.2.3"); - - REQUIRE(raw->start_calls == 0); +TEST_CASE("statusClass buckets HTTP codes", "[flapi_telemetry]") { + REQUIRE(flapi::FlapiTelemetry::statusClass(200) == "2xx"); + REQUIRE(flapi::FlapiTelemetry::statusClass(204) == "2xx"); + REQUIRE(flapi::FlapiTelemetry::statusClass(301) == "3xx"); + REQUIRE(flapi::FlapiTelemetry::statusClass(404) == "4xx"); + REQUIRE(flapi::FlapiTelemetry::statusClass(500) == "5xx"); + REQUIRE(flapi::FlapiTelemetry::statusClass(0) == "unknown"); } -TEST_CASE("FlapiTelemetry - DATAZOO_DISABLE_TELEMETRY=1 suppresses notifyStop", "[flapi_telemetry]") { - EnvGuard guard("DATAZOO_DISABLE_TELEMETRY", "1"); - - auto mock = std::make_unique(); - MockBackend* raw = mock.get(); - - flapi::FlapiTelemetry telemetry(std::move(mock)); - telemetry.notifyStop("1.2.3"); - - REQUIRE(raw->stop_calls == 0); +TEST_CASE("server_started carries only bounded counts/kinds + install_kind", "[flapi_telemetry]") { + EnvGuard guard("DATAZOO_DISABLE_TELEMETRY", "0"); + FakeBackend* raw = nullptr; + auto tel = makeTelemetry(raw); + + tel.configureProduct("1.2.3", "oss"); + tel.serverStarted(7, "bearer"); + + REQUIRE(raw->product_name == "flapi"); + REQUIRE(raw->product_version == "1.2.3"); + REQUIRE(raw->product_edition == "oss"); + + const auto* ev = raw->find("server_started"); + REQUIRE(ev != nullptr); + REQUIRE(ev->kind == "capture"); + // endpoint_count is a JSON number, not a quoted string. + REQUIRE(ev->props.at("endpoint_count").kind == duckdb::PropertyValue::Kind::Int); + REQUIRE(ev->props.at("endpoint_count").i == 7); + REQUIRE(hasStringProp(ev->props, "auth_kind", "bearer")); + REQUIRE(hasStringProp(ev->props, "install_kind", "server")); } -TEST_CASE("FlapiTelemetry - DATAZOO_DISABLE_TELEMETRY=true suppresses calls", "[flapi_telemetry]") { - EnvGuard guard("DATAZOO_DISABLE_TELEMETRY", "true"); - - auto mock = std::make_unique(); - MockBackend* raw = mock.get(); - - flapi::FlapiTelemetry telemetry(std::move(mock)); - telemetry.notifyStart("0.3.0"); - telemetry.notifyStop("0.3.0"); - - REQUIRE(raw->start_calls == 0); - REQUIRE(raw->stop_calls == 0); +TEST_CASE("associateDeployment associates the deployment group", "[flapi_telemetry]") { + EnvGuard guard("DATAZOO_DISABLE_TELEMETRY", "0"); + FakeBackend* raw = nullptr; + auto tel = makeTelemetry(raw); + + tel.associateDeployment(); + REQUIRE(raw->groups.size() == 1); + REQUIRE(raw->groups[0].first == "deployment"); + // Key is the pseudonymous machine id; may be empty in some CI sandboxes but + // the group type must be exactly "deployment". } -TEST_CASE("FlapiTelemetry - env var not set allows calls through", "[flapi_telemetry]") { - // Ensure the env var is not set +TEST_CASE("associateAccount hashes the license id (never raw)", "[flapi_telemetry]") { EnvGuard guard("DATAZOO_DISABLE_TELEMETRY", "0"); - - auto mock = std::make_unique(); - MockBackend* raw = mock.get(); - - flapi::FlapiTelemetry telemetry(std::move(mock)); - telemetry.notifyStart("0.3.0"); - telemetry.notifyStop("0.3.0"); - - REQUIRE(raw->start_calls == 1); - REQUIRE(raw->stop_calls == 1); + FakeBackend* raw = nullptr; + auto tel = makeTelemetry(raw); + + tel.associateAccount("ACME-LICENSE-123"); + REQUIRE(raw->groups.size() == 1); + REQUIRE(raw->groups[0].first == "account"); + // The raw license id must never appear — only its sha256 hex digest. + REQUIRE(raw->groups[0].second != "ACME-LICENSE-123"); + REQUIRE(raw->groups[0].second.size() == 64); } -TEST_CASE("FlapiTelemetry - multiple notifyStart calls each forwarded", "[flapi_telemetry]") { - auto mock = std::make_unique(); - MockBackend* raw = mock.get(); - - flapi::FlapiTelemetry telemetry(std::move(mock)); - telemetry.notifyStart("0.3.0"); - telemetry.notifyStart("0.3.0"); - - REQUIRE(raw->start_calls == 2); +TEST_CASE("rest_endpoint_served emits template/enum/number props only", "[flapi_telemetry]") { + EnvGuard guard("DATAZOO_DISABLE_TELEMETRY", "0"); + FakeBackend* raw = nullptr; + auto tel = makeTelemetry(raw); + + tel.restEndpointServed("GET", "/customers/:id", 200, 12.5, true); + + const auto* ev = raw->find("rest_endpoint_served"); + REQUIRE(ev != nullptr); + REQUIRE(ev->kind == "feature"); + REQUIRE(hasStringProp(ev->props, "method", "GET")); + // The ROUTE TEMPLATE is passed through verbatim (never a filled path). + REQUIRE(hasStringProp(ev->props, "route_template", "/customers/:id")); + REQUIRE(hasStringProp(ev->props, "status_class", "2xx")); + REQUIRE(ev->props.at("duration_ms").kind == duckdb::PropertyValue::Kind::Double); + REQUIRE(ev->props.at("cache_hit").kind == duckdb::PropertyValue::Kind::Bool); + REQUIRE(ev->props.at("cache_hit").b == true); + REQUIRE(hasStringProp(ev->props, "install_kind", "server")); + + // Guard against leaks: only these keys may ever be present. + for (const auto& kv : ev->props) { + const std::string& k = kv.first; + REQUIRE((k == "method" || k == "route_template" || k == "status_class" || + k == "duration_ms" || k == "cache_hit" || k == "install_kind")); + } } -TEST_CASE("FlapiTelemetry - production constructor compiles and constructs", "[flapi_telemetry]") { - // Just verify the production path can be instantiated without throwing. - // We can't easily verify it calls PostHogTelemetry without network, - // but DATAZOO_DISABLE_TELEMETRY=1 ensures no HTTP is sent. - EnvGuard guard("DATAZOO_DISABLE_TELEMETRY", "1"); - - REQUIRE_NOTHROW(flapi::FlapiTelemetry{}); +TEST_CASE("mcp_tool_called emits bounded tool name + status/duration", "[flapi_telemetry]") { + EnvGuard guard("DATAZOO_DISABLE_TELEMETRY", "0"); + FakeBackend* raw = nullptr; + auto tel = makeTelemetry(raw); + + tel.mcpToolCalled("get_customer", false, 3.0); + const auto* ev = raw->find("mcp_tool_called"); + REQUIRE(ev != nullptr); + REQUIRE(hasStringProp(ev->props, "tool", "get_customer")); + REQUIRE(hasStringProp(ev->props, "status_class", "5xx")); + REQUIRE(ev->props.at("duration_ms").kind == duckdb::PropertyValue::Kind::Double); } -TEST_CASE("FlapiTelemetry - setEnabled(false) suppresses notifyStart", "[flapi_telemetry]") { - auto mock = std::make_unique(); - MockBackend* raw = mock.get(); - - flapi::FlapiTelemetry telemetry(std::move(mock)); - telemetry.setEnabled(false); - telemetry.notifyStart("1.2.3"); - - REQUIRE(raw->start_calls == 0); +TEST_CASE("auth_enforced emits kind + outcome only", "[flapi_telemetry]") { + EnvGuard guard("DATAZOO_DISABLE_TELEMETRY", "0"); + FakeBackend* raw = nullptr; + auto tel = makeTelemetry(raw); + + tel.authEnforced("oidc", false); + const auto* ev = raw->find("auth_enforced"); + REQUIRE(ev != nullptr); + REQUIRE(hasStringProp(ev->props, "auth_kind", "oidc")); + REQUIRE(hasStringProp(ev->props, "outcome", "deny")); } -TEST_CASE("FlapiTelemetry - setEnabled(false) suppresses notifyStop", "[flapi_telemetry]") { - auto mock = std::make_unique(); - MockBackend* raw = mock.get(); - - flapi::FlapiTelemetry telemetry(std::move(mock)); - telemetry.setEnabled(false); - telemetry.notifyStop("1.2.3"); +TEST_CASE("error emits enumerated class + template only", "[flapi_telemetry]") { + EnvGuard guard("DATAZOO_DISABLE_TELEMETRY", "0"); + FakeBackend* raw = nullptr; + auto tel = makeTelemetry(raw); + + tel.error("server_error", "rest_endpoint_served", "/orders/:id"); + const auto* ev = raw->find("server_error"); + REQUIRE(ev != nullptr); + REQUIRE(ev->kind == "error"); + REQUIRE(hasStringProp(ev->props, "feature", "rest_endpoint_served")); + REQUIRE(hasStringProp(ev->props, "route_template", "/orders/:id")); +} - REQUIRE(raw->stop_calls == 0); +TEST_CASE("DATAZOO_DISABLE_TELEMETRY short-circuits every emit", "[flapi_telemetry]") { + EnvGuard guard("DATAZOO_DISABLE_TELEMETRY", "1"); + FakeBackend* raw = nullptr; + auto tel = makeTelemetry(raw); + + tel.configureProduct("1.0.0", "oss"); + tel.associateDeployment(); + tel.serverStarted(3, "none"); + tel.restEndpointServed("GET", "/x", 200, 1.0, false); + tel.mcpToolCalled("t", true, 1.0); + tel.authEnforced("basic", true); + tel.error("bad_request", "rest_endpoint_served", "/x"); + tel.flush(); + + REQUIRE(raw->calls.empty()); + REQUIRE(raw->groups.empty()); + REQUIRE(raw->product_name.empty()); + REQUIRE(raw->flushes == 0); } -TEST_CASE("FlapiTelemetry - setEnabled(false) takes precedence over env var not set", "[flapi_telemetry]") { +TEST_CASE("setEnabled(false) short-circuits every emit", "[flapi_telemetry]") { EnvGuard guard("DATAZOO_DISABLE_TELEMETRY", "0"); + FakeBackend* raw = nullptr; + auto tel = makeTelemetry(raw); + tel.setEnabled(false); + + tel.serverStarted(3, "none"); + tel.restEndpointServed("GET", "/x", 200, 1.0, false); + REQUIRE(raw->calls.empty()); + REQUIRE(tel.isEnabled() == false); +} - auto mock = std::make_unique(); - MockBackend* raw = mock.get(); +TEST_CASE("sampling decimates the hot path and stamps sample_rate", "[flapi_telemetry]") { + EnvGuard guard("DATAZOO_DISABLE_TELEMETRY", "0"); + FakeBackend* raw = nullptr; + auto tel = makeTelemetry(raw); + tel.setSampling(0.5); // emit 1 of every 2 - flapi::FlapiTelemetry telemetry(std::move(mock)); - telemetry.setEnabled(false); - telemetry.notifyStart("1.0.0"); - telemetry.notifyStop("1.0.0"); + for (int i = 0; i < 10; ++i) { + tel.restEndpointServed("GET", "/x", 200, 1.0, false); + } + REQUIRE(raw->calls.size() == 5); + // Surviving events are stamped with sample_rate so counts scale back up. + REQUIRE(raw->calls.front().props.at("sample_rate").kind == duckdb::PropertyValue::Kind::Double); - REQUIRE(raw->start_calls == 0); - REQUIRE(raw->stop_calls == 0); + // Low-volume events are never sampled out. + tel.authEnforced("basic", true); + REQUIRE(raw->find("auth_enforced") != nullptr); } -TEST_CASE("FlapiTelemetry - setEnabled(true) allows calls through", "[flapi_telemetry]") { +// End-to-end no-leak check against the REAL library transport: drive the +// production PostHogBackend through a test transport that captures the exact +// serialized batch, and assert the outgoing JSON contains only bounded props — +// no filled path, SQL, body, or header ever appears. +TEST_CASE("real transport payload contains no URL/SQL/body leaks", "[flapi_telemetry]") { EnvGuard guard("DATAZOO_DISABLE_TELEMETRY", "0"); - auto mock = std::make_unique(); - MockBackend* raw = mock.get(); + auto& lib = duckdb::PostHogTelemetry::Instance(); + lib.ResetShutdownForTesting(); + lib.SetEnabled(true); + lib.SetAutoFlushEnabledForTesting(false); + + std::vector payloads; + lib.SetTransportForTesting( + [&](const std::string&, const std::string&, + const std::vector& evs) { + for (const auto& e : evs) { + payloads.push_back(e.GetPropertiesJson()); + } + }); + + { + flapi::FlapiTelemetry tel; // real PostHogBackend + tel.setEnabled(true); + tel.configureProduct("9.9.9", "oss"); + tel.restEndpointServed("GET", "/customers/:id", 200, 4.2, true); + tel.mcpToolCalled("get_customer", true, 1.0); + tel.flush(); + } - flapi::FlapiTelemetry telemetry(std::move(mock)); - telemetry.setEnabled(true); - telemetry.notifyStart("1.0.0"); - telemetry.notifyStop("1.0.0"); + lib.SetTransportForTesting({}); // restore real transport + + REQUIRE_FALSE(payloads.empty()); + std::string all; + for (const auto& p : payloads) { + all += p; + } - REQUIRE(raw->start_calls == 1); - REQUIRE(raw->stop_calls == 1); + // The bounded, expected content is present. + REQUIRE(all.find("route_template") != std::string::npos); + REQUIRE(all.find("/customers/:id") != std::string::npos); + REQUIRE(all.find("\"install_kind\"") != std::string::npos); + REQUIRE(all.find("server") != std::string::npos); + + // Simulated sensitive material that the API never accepts must be absent. + REQUIRE(all.find("/customers/12345") == std::string::npos); // filled path + REQUIRE(all.find("SELECT") == std::string::npos); // SQL + REQUIRE(all.find("Authorization") == std::string::npos); // header + REQUIRE(all.find("password") == std::string::npos); // body/secret } diff --git a/third_party/posthog-telemetry b/third_party/posthog-telemetry index 4cd2256..bcb9ba1 160000 --- a/third_party/posthog-telemetry +++ b/third_party/posthog-telemetry @@ -1 +1 @@ -Subproject commit 4cd225671471a2c668c398cce05ec8aaeedf4833 +Subproject commit bcb9ba11b230cee4f37a2430fd0f85d0c97d0abb