Skip to content

Optimize /api/runs/list job loading#3986

Merged
peterschmidt85 merged 4 commits into
masterfrom
fix/runs-list-performance
Jun 25, 2026
Merged

Optimize /api/runs/list job loading#3986
peterschmidt85 merged 4 commits into
masterfrom
fix/runs-list-performance

Conversation

@peterschmidt85

@peterschmidt85 peterschmidt85 commented Jun 24, 2026

Copy link
Copy Markdown
Contributor

Fixes #3983

Problem

/api/runs/list powers both the Runs UI and dstack ps. UI/CLI list views normally need only the latest job submission, but the server still eager-loaded the full RunModel.jobs history and trimmed it later during response serialization.

On runs with many retries/submissions, one page could materialize tens of thousands of JobModel rows. Admin/all-project requests also had extra overhead because the service loaded accessible projects into Python and filtered runs with a large project_id IN (...) list.

Fix

  • Stop eager-loading RunModel.jobs in the runs-list path.
  • After selecting the requested page of runs, load only the job rows needed for the response.
  • When job_submissions_limit is set, load the latest N submissions per (run_id, replica_num, job_num) with a SQL window query.
  • Also load the latest terminated submission with a termination reason per logical job, so status messages such as retrying stay correct without loading full history.
  • Replace Python-built project ID lists with direct project lookup/subqueries for single-project, admin all-project, and member all-project paths.
  • Ensure lightweight UI/CLI list callers pass job_submissions_limit=1.

@peterschmidt85 peterschmidt85 requested a review from r4victor June 24, 2026 20:23
@peterschmidt85 peterschmidt85 force-pushed the fix/runs-list-performance branch from 2bb1a37 to 95fb10f Compare June 25, 2026 07:49
@peterschmidt85 peterschmidt85 requested a review from jvstme June 25, 2026 10:07
Comment thread src/dstack/_internal/server/services/runs/__init__.py Outdated
Comment thread src/dstack/_internal/server/services/runs/__init__.py Outdated
@peterschmidt85 peterschmidt85 merged commit 412a58b into master Jun 25, 2026
22 checks passed
@peterschmidt85 peterschmidt85 deleted the fix/runs-list-performance branch June 25, 2026 13:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: /api/runs/list loads all job submissions before applying job_submissions_limit

3 participants