Skip to content

feat(loki): add gcx logs stats and datasources loki stats - #1620

Open
L2D2Grafana wants to merge 2 commits into
mainfrom
l2d2/logs-stats
Open

L2D2Grafana wants to merge 2 commits into
mainfrom
l2d2/logs-stats

Conversation

@L2D2Grafana

Copy link
Copy Markdown
Contributor

Summary 📝

  • Adds gcx logs stats and gcx datasources loki stats, which show the estimated bytes a LogQL query would scan, from Loki's index/stats endpoint, without running the query. Table output is a single line (<size> would be scanned); -o json returns streams, chunks, bytes and entries. It only reports the estimate and never blocks or delays a query.
  • Selectors are extracted quote-aware (double quotes and backticks) so the endpoint receives bare matchers, one call per selector, concurrent and summed. The window is widened for range vectors and offsets, including negative offsets.
  • Replaces the stats work in feat(stats): add loki stats and skip-stats #1310, split out so the pre-flight check on query and metrics can be decided separately (see feat(logs): require approval for large or unknown Loki scans #1405).

Test 🧪

  • GCX_AGENT_MODE=false go test ./internal/datasources/... ./internal/query/loki/... ./internal/providers/logs/... ./cmd/... passes, including new tests for the client, formatter, command, and window resolution (TestResolveStatsWindow)
  • golangci-lint run ./... reports 0 issues
  • Manually verified against a real Grafana Cloud stack: gcx logs stats -d grafanacloud-logs '{app="..."}' --since 7d prints the estimate, datasources loki stats ... -o json returns the four fields, and gcx logs query is unchanged and prints no stats line

🤖 Generated with Claude Code

@L2D2Grafana
L2D2Grafana marked this pull request as ready for review October 20, 2026 20:02
@L2D2Grafana
L2D2Grafana requested review from a team as code owners October 20, 2026 20:02

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Intent. Adds gcx logs stats and gcx datasources loki stats, which ask Loki's index/stats endpoint how many bytes a LogQL query would scan without running it. Because that endpoint takes a bare matcher and knows nothing about the window a range vector actually reaches, the change also adds quote-aware selector extraction and lookback widening.

1103 added lines is large for two leaf commands, but 589 of them are tests, and the doGet extraction collapses four near-identical request bodies in internal/query/loki/client.go into one. Pulling ExprOpts out of SharedOpts is the smallest move that gives stats the --expr behaviour without inheriting --step and --error-on-empty. Judged against its problem, the size is earned.

I read the diff for the correctness pass and ran the triggers in .agents/skills/contribute/references/self-review.md that fire for it (T1, T2, T4, T5, T6, T8, T12) plus the four compliance levels. This environment cannot build, run tests, or regenerate docs, so every finding below rests on static reading of the head tree.

Blocking.

  1. FormatIndexStatsTable puts the ⚠ status marker on the -o table result line — internal/query/loki/formatter.go:251.
  2. stats sends an unvalidated time range to index/stats, where both sibling paths reject a reversed one — internal/datasources/loki/stats.go:134.

Fix summary. Drop the cmdio.Yellow("⚠ ") prefix from FormatIndexStatsTable along with the cmdio import it added at formatter.go:16, then add the start.Before(end) check that query_patterns.go:124 already carries. Together that is about five lines changed and one import removed; no structure moves.

Verdict. The approach holds up. The now-1m/now default matches exactly what buildQueryBody already sends for an instant query, the masking and extraction helpers are covered by table tests that would catch a real regression, and summing per-selector stats is documented as deliberately conservative rather than exact. Both blocking findings are small and local, and nothing here argues for a reduced version of the change. The one place I would push back on scope is the negative-offset direction in MaxLookback, noted below as a nit.

Comments below are prefixed required (2) and nit (4).

Comment @claude review for a fresh review.

Comment thread internal/query/loki/formatter.go Resolved
Comment thread internal/datasources/loki/stats.go Resolved
Comment thread internal/datasources/query/opts.go Resolved
Comment thread internal/datasources/loki/stats.go Resolved
Comment thread internal/query/loki/rangevector.go Resolved
Comment thread internal/datasources/loki/stats.go Resolved
@radiohead

Copy link
Copy Markdown
Contributor

@L2D2Grafana I am not sure there's any use for the stats command. If we want to give agents / humans the ability to estimate log volume processed by a particular query, I suggest instead adding appropriate flags for logs query / datasources loki query, see #1405 for reference.

edwardcqian added a commit that referenced this pull request Oct 21, 2026
…rgets

Before `gcx logs query|metrics` (and the `gcx datasources loki` equivalents)
run, estimate the scan with Loki's index/stats and assess it with the shared
lokiguard module against the stack's query policy: selective stream
selectors, a time-range cap including range vectors, and a byte budget.

A query over budget fails with a "Query blocked" error that says how to narrow
it, in agent and terminal mode alike. There is no prompt and no bypass flag.
When no verdict is possible (an unparseable query or a failed estimate), agent
mode blocks and a terminal warns and runs. Self-hosted targets are not guarded.
When the policy can't be fetched, a built-in 50 GiB / 24h / enforce default
applies. Loki requests now carry X-Query-Tags: Source=gcx.

Stacked on #1620 (gcx logs stats), whose stats client and selector helpers the
guard uses.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@trevorwhitney

Copy link
Copy Markdown

@radiohead the introduction of gcx gives us the opportunity to deprecate logcli (see https://github.com/grafana/loki-private/issues/2941), as I think having both creates confusion for customers. I'd advocate we start by bringing everything over and hit feature parity, then track usage statistics to see what's worth removing in the future.

edwardcqian added a commit that referenced this pull request Oct 21, 2026
…rgets

Before `gcx logs query|metrics` (and the `gcx datasources loki` equivalents)
run, estimate the scan with Loki's index/stats and assess it with the shared
lokiguard module against the stack's query policy: selective stream
selectors, a time-range cap including range vectors, and a byte budget.

A query over budget fails with a "Query blocked" error that says how to narrow
it, in agent and terminal mode alike. There is no prompt and no bypass flag.
When no verdict is possible (an unparseable query or a failed estimate), agent
mode blocks and a terminal warns and runs. Self-hosted targets are not guarded.
When the policy can't be fetched, a built-in 50 GiB / 24h / enforce default
applies. Loki requests now carry X-Query-Tags: Source=gcx.

Stacked on #1620 (gcx logs stats), whose stats client and selector helpers the
guard uses.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@radiohead

Copy link
Copy Markdown
Contributor

@radiohead the introduction of gcx gives us the opportunity to deprecate logcli (see grafana/loki-private#2941), as I think having both creates confusion for customers. I'd advocate we start by bringing everything over and hit feature parity, then track usage statistics to see what's worth removing in the future.

@trevorwhitney thank you for providing extra context! The plan sounds good, but I do want to double-check the "bring over, remove later" approach. If the users need to switch tools, they'd have to migrate anyway, which means we could re-think what we want to keep going forward, instead of migrating all the features from logcli and then throwing away some of them, potentially.

In my experience it's easy to add new stuff but it's REALLY hard to remove it, because someone somewhere will end up using every piece of functionality the tool has 😂

I also want to make it clear that I don't oppose having more features in GCX but in my experience making the command tree larger may negatively affect agentic performance, hence my concerns.

I would love to hear what others in @grafana/grafana-gcx have to say, let's wait for a few more folks to chime in.

@dafydd-t

Copy link
Copy Markdown
Contributor

I agree that it will be very hard to remove commands in the future - I think this move from logcli to gcx is the perfect opportunity to drop logcli commands that aren't useful / popular

@matyax

matyax commented Oct 22, 2026

Copy link
Copy Markdown

If you think adding this behavior behind something like --estimate-scan would represent an expected CLI API, we can probably go in that direction.

@radiohead

radiohead commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

If you think adding this behavior behind something like --estimate-scan would represent an expected CLI API, we can probably go in that direction.

I think that'd work quite well!

@L2D2Grafana

L2D2Grafana commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor Author

My understanding is that the loki team would like to add parity with logcli into gcx and mcp so all the normal api calls. Let's add this stats call. Then I do believe @edwardcqian PR adding guardrails is closest to what in mcp already for limiting large loki queries #1619

@matyax
matyax requested a balanced review from Copilot October 29, 2026 16:29

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

L2D2Grafana and others added 2 commits October 29, 2026 11:22
Show the estimated bytes a LogQL query would scan, from Loki's index/stats
endpoint, without executing it. Selectors are extracted quote-aware and the
window is widened for range vectors and offsets. The command only reports
the estimate; no query is ever blocked or delayed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Keep the status glyph off the table result line, reject a reversed time
range, quote the selector in errors, and fix the help text paths and examples.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@trevorwhitney

Copy link
Copy Markdown

@radiohead @dafydd-t thank you for the feedback, I agree. I don't think we need to or should pull over every command. Let me see what I can find regarding data around usage/popularity of the various logcli command.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants