[DO NOT MERGE] [LONG TERM] Gate heavy CI with reusable workflows

[gyohuangxin]

Related to #1920

Summary

• split ATOM, ATOM vLLM, and ATOM SGLang heavy test jobs into reusable workflows
• keep the existing public workflow files as lightweight gate wrappers
• call the reusable test workflows only when the heavy CI gate says the run should execute
• add duplicate-success detection to the shared gate so review-triggered duplicate runs can stop at the gate

Why

The current duplicate-skip approach can avoid rerunning expensive jobs, but the UI still expands many skipped matrix jobs. This version keeps duplicate cases cleaner: the wrapper runs the gate, reports an explicit already-passed job, and does not invoke the reusable heavy test workflow.

Notes

• ATOM offline smoke test remains in the wrapper and stays ungated, preserving the existing always-on PR behavior.
• Duplicate detection still requires a previous successful Accuracy job for the same workflow and PR head SHA; partial or in-progress runs are not treated as duplicates.

Testing

• /tmp/actionlint-1.7.7/actionlint -color -shellcheck "" -pyflakes "" .github/workflows/atom-test.yaml .github/workflows/atom-test-reusable.yaml .github/workflows/atom-vllm-test.yaml .github/workflows/atom-vllm-test-reusable.yaml .github/workflows/atom-sglang-test.yaml .github/workflows/atom-sglang-test-reusable.yaml
• bash -n .github/scripts/check_heavy_ci_gate.sh
• git diff --check
• simulated PR 1916 duplicate vLLM gate and verified reason=duplicate-successful-run
• simulated a new unseen SHA with ci:vllm and verified reason=label-present

[github-actions]

🏷️ CI Guide

Runs automatically on every eligible PR before approval:

• ✅ Pre Checkin: Black, Ruff, catalog schema validation, non-GPU unit tests

Heavy model tests:

• ✅ Run after the PR is approved and Pre Checkin passes
• ✅ Run immediately when an approval review is submitted
• ✅ Can be requested before approval with labels

Label	Tests
ci:full	Run all heavy PR model tests: native ATOM, vLLM, and SGLang
ci:atom	Run native ATOM model accuracy tests
ci:vllm	Run ATOM vLLM OOT model accuracy tests
ci:sglang	Run ATOM SGLang model accuracy tests

Heavy jobs are skipped when the PR is not approved and no matching ci:* label is present.
Add labels via the sidebar or gh pr edit 1922 --add-label <label>

[Copilot]

Pull request overview

This PR refactors ATOM’s heavy CI (accuracy / OOT) workflows to run via reusable workflows behind a lightweight “heavy CI gate”, and adds duplicate-success detection so review-triggered re-runs can stop early without expanding large skipped matrices.

Changes:

• Convert the ATOM / vLLM / SGLang heavy test jobs into workflow_call reusable workflows, keeping existing public workflow files as thin wrappers.
• Extend the shared heavy CI gate script to optionally detect an already-successful prior run for the same workflow + PR head SHA and short-circuit the run.
• Add a small “duplicate-success” reporting job in wrapper workflows when the gate determines the run is a duplicate.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
.github/workflows/atom-test.yaml	Turns the workflow into a gate + duplicate reporting + reusable-workflow caller (offline smoke test remains in wrapper).
.github/workflows/atom-test-reusable.yaml	New reusable workflow containing the heavy ATOM accuracy matrix and dashboard publishing logic.
.github/workflows/atom-vllm-test.yaml	Turns the workflow into a gate + duplicate reporting + reusable-workflow caller for vLLM OOT accuracy.
.github/workflows/atom-vllm-test-reusable.yaml	New reusable workflow containing the heavy vLLM OOT accuracy matrix.
.github/workflows/atom-sglang-test.yaml	Turns the workflow into a gate + duplicate reporting + reusable-workflow caller for SGLang accuracy.
.github/workflows/atom-sglang-test-reusable.yaml	New reusable workflow containing the heavy SGLang accuracy matrix.
.github/scripts/check_heavy_ci_gate.sh	Adds optional duplicate-success detection and emits duplicate_run_url as a gate output.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Suppressed comments (5)

.github/workflows/atom-test.yaml:116

• run-tests calls a reusable workflow that uses the GitHub Actions API (via gh in check_signal.sh / wheel download scripts). Reusable workflows cannot elevate GITHUB_TOKEN permissions beyond what the caller job grants, but this run-tests job does not set permissions, so it may run with the repo default (often missing actions: read) and fail when the called workflow queries workflow runs/jobs.

  run-tests:
    if: ${{ needs.ci-gate.outputs.should_run == 'true' }}
    needs: [ci-gate]
    name: Run ATOM accuracy tests
    uses: ./.github/workflows/atom-test-reusable.yaml
    with:

.github/workflows/atom-vllm-test.yaml:94

• run-tests invokes a reusable workflow that uses gh api to inspect workflow runs/jobs. The called workflow cannot request higher GITHUB_TOKEN permissions than this caller job provides; since run-tests has no explicit permissions, it may lack actions: read and fail inside the reusable workflow.

  run-tests:
    if: ${{ needs.ci-gate.outputs.should_run == 'true' }}
    needs: [ci-gate]
    name: Run ATOM vLLM tests
    uses: ./.github/workflows/atom-vllm-test-reusable.yaml
    with:

.github/workflows/atom-sglang-test.yaml:94

• run-tests invokes a reusable workflow that uses gh api (actions endpoints) as part of the heavy test flow. Since reusable workflows cannot elevate token permissions, this caller job should explicitly grant at least actions: read (and contents: read) to avoid failures when the called workflow queries runs/jobs or downloads artifacts.

  run-tests:
    if: ${{ needs.ci-gate.outputs.should_run == 'true' }}
    needs: [ci-gate]
    name: Run ATOM SGLang tests
    uses: ./.github/workflows/atom-sglang-test-reusable.yaml
    with:

.github/workflows/atom-test-reusable.yaml:546

• accuracy-dashboard auto-pushes benchmark data to gh-pages via benchmark-action/github-action-benchmark, which requires contents: write. With reusable workflows, permissions are constrained by the caller job, and this job currently doesn't declare the needed permissions, so it can fail in repos configured with restricted default GITHUB_TOKEN permissions.

  accuracy-dashboard:
    name: Update accuracy dashboard
    needs: [atom-test]
    if: always() && inputs.is_main_ref && (inputs.caller_event_name == 'push' || inputs.caller_event_name == 'schedule')

.github/workflows/atom-test.yaml:74

• The duplicate-detection regex currently requires Accuracy ( but in the called workflow (atom-test-reusable.yaml) the matrix job is named Accuracy (no explicit parentheses). This can make duplicate detection miss successful prior runs depending on how the Jobs API reports matrix job names. Broadening the pattern to match both Accuracy and Accuracy (...) avoids false negatives.

        id: gate
        env:
          CI_GATE_DUPLICATE_SUCCESS_JOB_PATTERN: '(^| / )Accuracy \('
          CI_GATE_LABELS: ci:full,ci:atom