[earakely-scale]

The manual publish runs on main keep failing on base-image pulls from public.ecr.aws. On the last run, 7 of the 8 build jobs passed, and the gateway amd64 job got 429 Too Many Requests through every retry. ECR Public rate-limits anonymous pulls per source IP, and some hosted-runner IPs stay throttled longer than retries inside one job can wait.

Change

• The infra-build job pulls its image's bases from mirror.gcr.io first. A new step runs .github/scripts/mirror_base_images.py "$DOCKERFILE" before the build. It pulls each public.ecr.aws/docker/library/… base from mirror.gcr.io and names it the way the Dockerfile does. The build doesn't pass --pull, so it uses that local copy.
• Falling back: if the mirror doesn't give a base after three tries, the step prints a warning and leaves it to the build. The build then pulls from public.ecr.aws with the existing 429 retries, so failing needs both registries to fail.
• The Dockerfiles stay on public.ecr.aws. Google documents mirror.gcr.io only as a daemon registry-mirrors target, and keeps an image only while it's asked for. Our pinned digests are already older than their tags' current ones. A FROM naming the mirror directly would have no fallback once a digest ages out, and these Dockerfiles ship in the wheel for users to build.
• local-backends.yml uses the same script. It replaces its two inline steps: the containerd store switch and the mirror pull loop. That logic now lives in one place, and the step still runs before the local registry starts.

The script ports the local-backends steps. Docker 28 resolves a digest-pinned FROM only from an image under that exact name, and docker tag refuses digests, so a pinned base is also named in containerd. Docker reads containerd names only on the containerd image store, so dockerd is switched to it (keeping the existing daemon.json) when a base is pinned and it isn't already on that store. A job whose bases are all tag-only, such as service-db, stays on the classic store.

Verified on a VM like a hosted runner (Ubuntu 24.04, Docker 28.0.4, classic store with a daemon.json):

• All four public.ecr.aws-based infra images, one at a time as the jobs run them, with public.ecr.aws blocked in /etc/hosts:
  • each built and passed infra_build.py smoke;
  • each pushed to a local registry as a plain manifest.v2;
  • the gateway's tag merged into an index with imagetools create, and its digest reads back with {{json .Manifest}}.
• Stores: service-db stayed on the classic store; the gateway switched dockerd to the containerd store and kept log-driver.
• Fallback: with mirror.gcr.io blocked and the postgres base removed, the script warned after three tries and the build pulled postgres:16-alpine from public.ecr.aws.
• The local-backends call (src tst) pulled and named every base in the tree.
• Tests: new unit tests for the script, a workflow test that the mirror step comes before the build, and the full unit tier (8,815 passed).

After this merges I'll dispatch the workflow on main again.

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge; no actionable issues were found.

Summary

The PR adds mirror_base_images.py to pull base images from mirror.gcr.io under the names the Dockerfiles already use.

• CI image builds try mirrored base images before Docker builds.

_{Reviews (2) · Last reviewed commit: "fix(release): pull the infra images' bas..." · Reviewed by Greptile}