Skip to content

Commit 0dd0b0a

Browse files
authored
Merge pull request #21 from NDDev-OpenNetwork/docs/drakkars-observability-skills
docs(skills): teach native fleet correlation
2 parents 847e952 + d0451c3 commit 0dd0b0a

3 files changed

Lines changed: 30 additions & 2 deletions

File tree

‎skills/canonical/gds-ci-performance-verify/SKILL.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,10 @@ Verify performance with comparable evidence rather than isolated job duration.
3030
Use only naturally occurring project runs; never create synthetic,
3131
benchmark, soak, canary, rerun, or empty-commit traffic to fill a cohort.
3232
3. Decompose median, p90, and p95 latency into queue, provision, setup, execute, upload/deploy, and teardown stages.
33+
Use `queue.assigned`/`queue.running` spans for queue lifecycle and
34+
`provider.create_instance`/`provider.delete_instance` for provisioning and
35+
teardown. Join by `runner_name`; retain `queue_job_uuid`, workflow run and
36+
`incus_member` so member pressure is not attributed to the services host.
3337
4. Compare throughput, utilization, success, retry, infrastructure-failure, flaky-failure, and missing-telemetry rates.
3438
5. Identify critical-path changes and resource contention; keep priority classes separate.
3539
6. Reject conclusions with insufficient samples or incomparable cohorts and label them `NOT_PROVEN`.

‎skills/canonical/gds-drakkars-audit-fleet/SKILL.md‎

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,17 @@ Explain fleet health and bottlenecks from correlated, time-bounded evidence.
3737
correlation fields. Distinguish the bounded pre-reconciliation account-only
3838
window from intent records that remain unbound after authoritative queued
3939
and already-running reconciliation.
40-
7. Separate confirmed faults, saturation, waste, and `NOT_PROVEN` gaps.
40+
Treat `gha_fleet_queue_missing_runner_request_id` as incomplete
41+
pre-execution correlation only. A running direct-JIT job with UUID,
42+
workflow-run, numeric GitHub-runner and runner-name identity belongs to
43+
`gha_fleet_queue_direct_jit_without_runner_request_id`; GitHub emits no
44+
AcquireJobs request ID for that path.
45+
7. In OpenObserve, query job-start logs by `msg = 'job correlation accepted'`.
46+
Join `github_*`, `runner_name` and `instance_name` to `fleet_traces` using
47+
trace search type `traces`; queue spans expose `queue_job_uuid`, while
48+
provider spans expose `incus_member`. Do not use the provider process
49+
resource host as the compute placement member.
50+
8. Separate confirmed faults, saturation, waste, and `NOT_PROVEN` gaps.
4151

4252
## Safety
4353

‎skills/canonical/gds-drakkars-triage-run/SKILL.md‎

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,12 +26,26 @@ Trace one run from event receipt to terminal result without changing state.
2626
## Workflow
2727

2828
1. Capture repository, run, attempt, job, event, requested labels, commit, and timestamps.
29-
2. Follow the same identifiers through admission, priority, assignment, provisioning, runner registration, setup, execution, upload, and teardown.
29+
2. Follow the same identifiers through admission, priority, assignment,
30+
provisioning, runner registration, setup, execution, upload, and teardown.
31+
Start from the native job-start log fields (`github_repository`,
32+
`github_workflow_run_id`, `github_run_attempt`, `github_job_name`,
33+
`github_workflow_ref`, `github_commit_sha`, `runner_name`, `instance_name`).
34+
Join traces by `runner_name`: `queue.*` supplies `queue_job_uuid`, and
35+
provider create/delete supplies `incus_member`.
3036
3. Locate the first abnormal stage; distinguish queue pressure, missing capability, infrastructure failure, deterministic test failure, flaky failure, timeout, and observability loss.
3137
4. Compare with adjacent successful runs using the same workflow and toolchain.
3238
5. If a transient infrastructure error is proven, recommend a bounded rerun; do not execute it here.
3339
6. Preserve running jobs and label unsupported claims `NOT_PROVEN`.
3440

41+
## Direct-JIT identity
42+
43+
Do not require `github_runner_request_id` from a running direct-JIT job. It is
44+
an AcquireJobs identity GitHub does not emit on that path. Require queue UUID,
45+
workflow run, numeric GitHub runner, runner name and instance/member correlation
46+
instead. A missing request ID remains actionable only before the execution path
47+
has converged or when those authoritative identities are also absent.
48+
3549
## Output
3650

3751
Return the causal timeline, first failing stage, supporting links or private evidence locations, impact, and safest recovery path.

0 commit comments

Comments
 (0)