fix(deployment): mirror kubernetes-secret-generator chart via GHCR - #7199
Conversation
helm.mittwald.de times out intermittently (mittwald/kubernetes-secret-generator#115, filed a year ago, still unfixed), and install_secret_generator() re-runs `helm repo add`/`repo update` against it on every cluster setup, so this was flaking integration-test CI runs regularly. Mirrored kubernetes-secret-generator 3.4.1 as an OCI artifact at ghcr.io/loculus-project/kubernetes-secret-generator and switched deploy.py to pull from there instead, pinned to that version. Upstream releases are rare (3.4.0 -> 3.4.1 was a 3-year gap) so pinning costs little, and OCI charts skip the repo-add/update step entirely.
|
Claude finished @corneliusroemer's task in 1m 54s —— View job Code reviewThe change is the right fix for the flake: OCI refs skip the repo index entirely, so dropping Two inline comments on the in-code refresh recipe and the pin's maintenance story. Beyond those: Docs are now inconsistent — Scope of the fix — this mirrors the chart only. The controller image the chart references is still pulled from its upstream registry at pod-start, so image-pull flakes remain possible; they're just far less likely than the Nothing blocking. I could not verify the mirrored artifact is anonymously pullable (
|
There was a problem hiding this comment.
Pull request overview
This PR improves deployment and CI reliability by eliminating intermittent timeouts when installing the kubernetes-secret-generator Helm chart from https://helm.mittwald.de. Instead, it switches installation to a mirrored OCI-hosted chart on GHCR and pins the chart version to avoid reliance on a Helm repo index.
Changes:
- Remove
helm repo add/helm repo updateusage frominstall_secret_generator()to avoid flaky network dependency. - Install
kubernetes-secret-generatorfromoci://ghcr.io/loculus-project/kubernetes-secret-generatorand pin to version3.4.1. - Add inline instructions in
deploy.pyfor how to refresh the mirrored chart after an upstream bump.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
|
CI failure in dev test ubuntu is unrelated to the code touched here. I'm making a separate PR to deflake that particular CI failure |
@anna-parker noticed CI failure due to helm chart pull failing: https://loculus.slack.com/archives/C05G172HL6L/p1787904000499649?thread_ts=1787903822.341089&cid=C05G172HL6L
helm.mittwald.de(the repo hosting thekubernetes-secret-generatorchart used bysecrets.yaml) times out intermittently onhelm repo add/helm repo update, andinstall_secret_generator()runs that on every./deploy.py clusterinvocation, i.e. every integration-test run — see mittwald/kubernetes-secret-generator#115, which I filed a year ago and is still open with no maintainer response.OCI chart references don't need a repo index, so this also removes the
helm repo add/helm repo updatecalls entirely rather than just pointing them somewhere else.Mirroring the chart
Done once, manually, rather than via a GitHub Action, to keep this simple. To refresh after a version bump upstream:
The package defaults to private on first push — needs
gh api --method PATCH orgs/loculus-project/packages/container/kubernetes-secret-generator -f visibility=public(or the same via the package's GitHub settings page) afterward, otherwise CI's anonymous pull 401s.🚀 Preview: Add
previewlabel to enable