[Mirror] Kubernetes/securitycontexts - #7117
chaoran-chen wants to merge 10 commits into
Conversation
…nd container levels
…l relevant templates
|
Claude finished @chaoran-chen's task in 4m 24s —— View job Review
Solid, well-scoped change. The helper mirrors the existing Seven inline comments. The ones worth acting on:
One gap not attributable to the original work: I could not run |
| "podSecurityContext": { | ||
| "$ref": "#/definitions/podSecurityContextSpec", | ||
| "description": "Pod-level security context values keyed by component name. See https://kubernetes.io/docs/tasks/configure-pod-container/security-context/." | ||
| }, | ||
| "containerSecurityContext": { | ||
| "$ref": "#/definitions/containerSecurityContextSpec", | ||
| "description": "Container-level security context values keyed by component name. See https://kubernetes.io/docs/tasks/configure-pod-container/security-context/." | ||
| }, |
There was a problem hiding this comment.
These two properties are missing a groups key, so they won't show up in the generated docs. SchemaDocs.astro:70 only emits a row when definition.groups includes the requested group — the sibling podScheduling (line 1415) has "groups": ["general"] for exactly this reason.
| "podSecurityContext": { | |
| "$ref": "#/definitions/podSecurityContextSpec", | |
| "description": "Pod-level security context values keyed by component name. See https://kubernetes.io/docs/tasks/configure-pod-container/security-context/." | |
| }, | |
| "containerSecurityContext": { | |
| "$ref": "#/definitions/containerSecurityContextSpec", | |
| "description": "Container-level security context values keyed by component name. See https://kubernetes.io/docs/tasks/configure-pod-container/security-context/." | |
| }, | |
| "podSecurityContext": { | |
| "groups": ["general"], | |
| "$ref": "#/definitions/podSecurityContextSpec", | |
| "description": "Pod-level security context values keyed by component name. See https://kubernetes.io/docs/tasks/configure-pod-container/security-context/." | |
| }, | |
| "containerSecurityContext": { | |
| "groups": ["general"], | |
| "$ref": "#/definitions/containerSecurityContextSpec", | |
| "description": "Container-level security context values keyed by component name. See https://kubernetes.io/docs/tasks/configure-pod-container/security-context/." | |
| }, |
Note that a "type": "object" may also be needed alongside for the Type column to render.
| "silo-importer": { "$ref": "#/definitions/podSecurityContextValue" }, | ||
| "backend": { "$ref": "#/definitions/podSecurityContextValue" }, | ||
| "preprocessing": { "$ref": "#/definitions/podSecurityContextValue" }, | ||
| "minio": { "$ref": "#/definitions/podSecurityContextValue" }, | ||
| "keycloak-database": { "$ref": "#/definitions/podSecurityContextValue" }, | ||
| "database": { "$ref": "#/definitions/podSecurityContextValue" }, | ||
| "config-processor": { "$ref": "#/definitions/podSecurityContextValue" } |
There was a problem hiding this comment.
silo-importer and config-processor are never their own pods — silo-importer is a second container in the silo pod (silo-deployment.yaml:82) and config-processor is an init container injected into the backend/website/keycloak/lapis/silo pods. loculus.podSecurityContext is never called with either key, so setting podSecurityContext.silo-importer or podSecurityContext.config-processor validates fine and then silently does nothing.
Suggest dropping both from podSecurityContextSpec (they're correctly kept in containerSecurityContextSpec).
| securityContext: | ||
| {{ toYaml $values.containerSecurityContext.default | indent 2 }} | ||
| {{- end }} | ||
| {{- end }} No newline at end of file |
There was a problem hiding this comment.
Missing trailing newline at EOF.
Also, the two defines are byte-identical apart from the values key. Worth collapsing into one helper to avoid the two drifting apart:
{{- define "loculus.securityContext" -}}
{{- $component := index . 0 -}}
{{- $scopeMap := index . 1 -}}
{{- if $scopeMap -}}
{{- $sc := or (index $scopeMap $component) (index $scopeMap "default") -}}
...
and call it as (list "backend" $.Values.podSecurityContext). Not blocking — the current shape does mirror loculus.resources, so keeping it is defensible.
| {{- if and $values.podSecurityContext (hasKey $values.podSecurityContext $componentName) }} | ||
| securityContext: | ||
| {{ toYaml (index $values.podSecurityContext $componentName) | indent 2 }} | ||
| {{- else if and $values.podSecurityContext (hasKey $values.podSecurityContext "default") }} | ||
| securityContext: | ||
| {{ toYaml $values.podSecurityContext.default | indent 2 }} | ||
| {{- end }} |
There was a problem hiding this comment.
Two things about the default fallback:
-
No way to opt a component out. Once
defaultis set, every component gets it and the only override is a full replacement. There's no value that disables the context for a single component —podSecurityContext.backend: {}renderssecurityContext: {}(harmless but not "unset"), andnullfails schema validation since the$refrequires an object. If someone needs one workload (e.g.minio,database) to run without the hardened default, they currently can't express it. Consider documenting this or acceptingnull/falseas an explicit disable. -
Full replace vs. merge.
values.yamldocuments this, but amergeOverwriteofdefaultwith the component entry would be much less error-prone for the common "same base, one field differs" case. If full-replace is deliberate, it's worth a one-line comment here in the template too, since this is where a future reader will look.
Neither is a bug, but (1) is the one likely to be hit in practice.
| {{- include "loculus.resources" (list "ingest-init" $Values) | nindent 6 }} | ||
| {{- include "loculus.containerSecurityContext" (list "ingest" $Values) | nindent 6 }} |
There was a problem hiding this comment.
Init containers reuse the app container's key. Here loculus.resources already distinguishes ingest-init from ingest, but the security context lumps version-check (busybox), wait-for-no-other-ingest (alpine/kubectl) and the snakemake ingest container under a single ingest key. Same pattern in keycloak-deployment.yaml:38 (keycloak-theme-prep), ena-submission-deployment.yaml:37 (flyway), taxonomy-deployment.yaml:30 (download-taxonomy-db).
That matters because the headline hardening knobs (runAsUser, runAsNonRoot, readOnlyRootFilesystem) frequently need to differ between an init container and the app it prepares — e.g. flyway and the ena-submission image don't necessarily run as the same UID. Right now there's no way to express that without dropping the setting for both.
Suggest reusing the keys the resources helper already defines (ingest-init) and adding equivalents for the other init containers, at least for those where a divergence is plausible.
| {{- include "loculus.podSecurityContext" (list "backend" $.Values) | nindent 6 }} | ||
| initContainers: | ||
| {{- include "loculus.configProcessor" (dict "name" "loculus-backend-config" "dockerTag" $dockerTag "imagePullPolicy" $.Values.imagePullPolicy) | nindent 8 }} | ||
| {{- include "loculus.configProcessor" (dict "name" "loculus-backend-config" "dockerTag" $dockerTag "imagePullPolicy" $.Values.imagePullPolicy "Values" $.Values) | nindent 8 }} |
There was a problem hiding this comment.
Indentation of this line changed from column 0 to 4 while adding the "Values" key. Harmless (the {{- trims it and nindent 8 sets the real indent), but it now matches neither keycloak-deployment.yaml:29 (column 0) nor silo-deployment.yaml:36 / lapis-deployment.yaml:30 (column 8). Same in loculus-website.yaml:28. Worth reverting to keep the diff to the functional change.
| limits: | ||
| cpu: 500m | ||
| memory: 256Mi | ||
| {{- include "loculus.containerSecurityContext" (list "config-processor" .Values) | nindent 2 }} |
There was a problem hiding this comment.
This makes Values a required key of the loculus.configProcessor dict — a caller that omits it fails with nil pointer evaluating interface {}.containerSecurityContext. All five current callers were updated, so this is correct today, but the contract is now implicit.
Worth documenting the expected dict keys in a comment above the define (the way _ingest-pod-spec.tpl:6-12 does), so the next person adding a caller doesn't have to discover it from a render failure.
| # Optional pod-level securityContext. Supports a `default` entry applied to all workloads, | ||
| # plus per-component overrides (e.g. backend, silo, keycloak). A component-specific entry | ||
| # fully replaces `default`. To change a single field for one component, specify the | ||
| # complete security context for it. | ||
| podSecurityContext: {} | ||
| # Optional container-level securityContext. Same structure and override behavior as podSecurityContext above. | ||
| containerSecurityContext: {} |
There was a problem hiding this comment.
Both default to {}, so this PR is a no-op at render time for every values file in the repo — CI will exercise the {{- end }} branch of both helpers and nothing else. A typo in a nindent level or a bad key would ship undetected.
Consider setting a non-trivial containerSecurityContext.default (and at least one component override) in values_e2e_and_dev.yaml so the e2e cluster actually renders and schedules pods with these fields. That also validates the interesting real-world question — whether the images tolerate runAsNonRoot / readOnlyRootFilesystem — which the current default of {} leaves entirely untested.
2f72f49 to
f9e87aa
Compare
This is a mirror of #7033 to run the CI. Please continue discussions in the original PR, I'll try to keep this in sync for CI and preview.
🚀 Preview: Add
previewlabel to enable