[RFC-0014] Advanced Support for Short-Lived Cryptographic Material

@stealthybox

In RFC-0010 we implemented object-level workload identity for cloud providers leveraging ServiceAccount tokens. However, Flux can still improve in terms of workload identity/short-lived-credentials forms of authentication that do not depend on any cloud providers, but rather on open standards (e.g. OIDC, SPIFFE).

A couple of examples:

The Kustomization and HelmRelease APIs support vendor-agnostic workload identity through the generic provider inside .spec.kubeConfig.configMapRef -> .data.provider. This generic provider issues a ServiceAccount token and uses it for authentication with the remote cluster. The remote cluster must be configured with external OIDC authentication for this to work.
This discussion was recently started asking for SPIFFE integration with the Zot container registry. While the specific form with which the request was made does not align well with the Flux multi-tenancy model, it does inspire the idea I'd like to discuss here.

I've been working with the Zot container registry to introduce OIDC authentication for workloads (Zot already supports OIDC for humans going through the OAuth2 authorization flow in a browser, which is an entirely different thing). With this in mind, I thought about introducing a new authentication pattern for short-lived credentials across Flux, starting with the OCIRepository and ImageRepository APIs.

Proposal

I'd like to introduce the spec field .spec.credential. Valid values would be:

ServiceAccountToken. When the field is set to this value, we do the exactly the same we did for Kustomization and HelmRelease. Users can optionally specify also .spec.serviceAccountName in order not to use the ServiceAccount of the respective controller. The open standard here is obviously OIDC. Again, we leverage Kubernetes for this.
SpiffeJWT. When the field is set to this value, we issue a short-lived SPIFFE SVID JWT. Here, (@stealthybox will love it) the identity will not be based on a ServiceAccount! Kubernetes can only issue OIDC JWTs for ServiceAccounts, but we have freedom to issue a SPIFFE JWT where the identity is the Flux object itself, including kind, name and namespace. The cluster will be identified by the helper flag --spiffe-issuer, which will be the iss claim of the JWT.
SpiffeCertificate. When the field is set to this value, we issue a short-lived SPIFFE SVID x509 certificate for client authentication via mTLS. Both OCIRepository and ImageRepository already support client authentication via mTLS, so this should be easy to implement.

For the JWT credentials, i.e. ServiceAccountToken and SpiffeJWT, we can also introduce the field .spec.audiences, which will be required for these credentials.

For both Spiffe* values of the field, we can add the binary flag --spiffe-secret-name for watching a secret containing the private key required for issuing the short-lived credentials, and also the CA in case of certificates.

Note that this proposal introduces a pattern that is significantly different from the configuration API we used for Kustomization and HelmRelease. However, these two APIs have their own very specific circumstances:

They are the only ones that use a ConfigMap reference.
They target only Kubernetes clusters, which is a third-party service that already supports the OIDC standard and therefore will not require support for SPIFFE.

Why a new field and not `.spec.provider`

The word provider calls the idea of a cloud provider, which is not what we are doing here. Here we are implementing authentication standards, and not targeting cloud vendors. Credentials is a better word for this.
I want the values for the new field to follow PascalCase, like most enums in Kubernetes (especially because of the ServiceAccountToken value). The .spec.provider field uses lowercase initials, e.g. aws, gcp.
Say we wanted to use .spec.provider: generic like in the Kustomization and HelmRelease APIs. This is a strong no-go because this already has a very specific behavior in the OCI APIs.

How trust should be established for SPIFFE

For JWTs, users must host the issuer and JWKS documents somewhere reachable from the third-party services they are integrating with (e.g. from Zot in this first use case).

For certificates, users must configure the third-party service (e.g. Zot) with the CA provided via --spiffe-secret-name.

@Vad1mo

I think Harbor also has/will have support for OIDC tokens, according to this issue where I interacted with them:

goharbor/harbor#22027

@Vad1mo can confirm

If we agree to implement this, I could easily implement ServiceAccountToken for Flux 2.8 (shipping around one month from now), but SPIFFE will have to wait until Flux 2.9.

Zot has implemented OIDC workload identity federation here: project-zot/zot#3711

I think Harbor also has/will have support for OIDC tokens, according to this issue where I interacted with them:

goharbor/harbor#22027

@Vad1mo can confirm

Yes, we are working on Workflow Identity Federation for Harbor.
There is a little demo/tutorial online for k8s and GitLab/GitHub. Once FluxCD is ready we can add one there too.

https://github.com/container-registry/harbor-workload-identity-federation

@Vad1mo

@Vad1mo Are all these claims required? The ServiceAccount tokens issued by Fux do not have a pod name:

@Vad1mo

@Vad1mo Are all these claims required? The ServiceAccount tokens issued by Fux do not have a pod name:

IMO only ISS and AUD, Everything else is optional. @bupd knows it for sure

@Vad1mo

I was able to use Zot and Flux (fluxcd/source-controller#1962) entirely via GitOps:

https://github.com/matheuscscp/flux-operator-zot-example

NAMESPACE     NAME            URL                                                        READY   STATUS                                                                                                                                                                                                                                                   AGE
backend       backend         oci://zot.zot.svc.cluster.local:5000/backend               True    stored artifact for digest 'latest@sha256:ce719eca3139b730aed4cc9dd120b1ee92b9c9beda309ed61320af4658f01797'                                                                                                                                              56s
backend       frontend        oci://zot.zot.svc.cluster.local:5000/frontend              False   failed to determine artifact digest: GET http://zot.zot.svc.cluster.local:5000/v2/frontend/manifests/latest: DENIED: requested access to the resource is denied; map[description:The access controller denied access for the operation on a resource.]   56s
frontend      backend         oci://zot.zot.svc.cluster.local:5000/backend               False   failed to determine artifact digest: GET http://zot.zot.svc.cluster.local:5000/v2/backend/manifests/latest: DENIED: requested access to the resource is denied; map[description:The access controller denied access for the operation on a resource.]    56s
frontend      frontend        oci://zot.zot.svc.cluster.local:5000/frontend              True    stored artifact for digest 'latest@sha256:ce719eca3139b730aed4cc9dd120b1ee92b9c9beda309ed61320af4658f01797'                                                                                                                                              56s

This example shows the workload identity multi-tenancy at play: each tenant has access to its own OCI repo in Zot, but gets permission error in the other tenant's repo.

@Vad1mo I'm going to try Harbor next with a similar example

@Vad1mo

@Vad1mo @bupd Is there a declarative way to configure the feature or is it only via UI? The Zot example I built is fully declarative, users can experiment it locally with a single bootstrap command. Manual config steps would not be the best UX for this example

@Vad1mo

I deployed the Harbor Helm chart and can't find the Federated IDPs feature: container-registry/harbor-workload-identity-federation#1

From the first line in the README.md (and since this repo example is under the container-registry GH org) I thought this was available only in the https://container-registry.com/ SaaS:

Repository with examples demonstrating how to use Harbor/8gears Container Registry with Workload Identity Federation, eliminating the need for static secrets in CI/CD pipelines and Kubernetes.

I signed up for the free plan and can't see the feature there, looks like this is a shared account with limited options:

I signed up for the paid plan free trial and can't see the feature there either:

@Vad1mo Is this feature really released anywhere?

It will be part of 2.15, I can send you a link where it is running.

@Vad1mo

@Vad1mo Nice! I'd love to test the feature

I was able to test this with the yet-to-come Harbor feature as well 🤗

https://github.com/matheuscscp/flux-operator-harbor-example/

NAMESPACE     NAME            URL                                                            READY   STATUS                                                                                                                                                                                                                                                                                AGE
backend       backend         oci://silly-snyder.container-registry.com/backend/artifacts    True    stored artifact for digest 'latest@sha256:9ddb3a533a18a738a120a8ad7ddb622f4a7d08afc5265e6e1487b27f43a06a74'                                                                                                                                                                           2m57s
backend       frontend        oci://silly-snyder.container-registry.com/frontend/artifacts   False   failed to determine artifact digest: GET https://silly-snyder.container-registry.com/v2/frontend/artifacts/manifests/latest: UNAUTHORIZED: unauthorized to access repository: frontend/artifacts, action: pull: unauthorized to access repository: frontend/artifacts, action: pull   2m57s
frontend      backend         oci://silly-snyder.container-registry.com/backend/artifacts    False   failed to determine artifact digest: GET https://silly-snyder.container-registry.com/v2/backend/artifacts/manifests/latest: UNAUTHORIZED: unauthorized to access repository: backend/artifacts, action: pull: unauthorized to access repository: backend/artifacts, action: pull      2m57s
frontend      frontend        oci://silly-snyder.container-registry.com/frontend/artifacts   True    stored artifact for digest 'latest@sha256:9ddb3a533a18a738a120a8ad7ddb622f4a7d08afc5265e6e1487b27f43a06a74'                                                                                                                                                                           2m57s

In this example, Harbor (actually a test instance in the container-registry.com service) has to be manually configured. I put a ConfigMap dependency in the ResourceSet to serve as a manual gate while I was manually configuring the instance, and then created the ConfigMap to kick off the pipeline. After that, I was able to see the output above 👍

@Vad1mo

Thanks very much @Vad1mo for the test instance!

What's the current status? Which version of flux will be able to use against which version of harbor?

Either the next one (Flux 2.9) or the one after that. I think Harbor got the feature in the SaaS already, not sure when it will land in the opensource upstream. Probably better opening an issue in their repo.

Implementation will now commence: #6103

Navigation Menu

Repository navigation

[RFC-0014] Advanced Support for Short-Lived Cryptographic Material #5681

Proposal

Why a new field and not `.spec.provider`

How trust should be established for SPIFFE

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[RFC-0014] Advanced Support for Short-Lived Cryptographic Material #5681

Description

Proposal

Why a new field and not .spec.provider

How trust should be established for SPIFFE

Activity

matheuscscp commented on Jan 18, 2026

matheuscscp commented on Jan 18, 2026

matheuscscp commented on Jan 25, 2026

Vad1mo commented on Jan 25, 2026

matheuscscp commented on Jan 25, 2026

Vad1mo commented on Jan 25, 2026

matheuscscp commented on Jan 31, 2026

matheuscscp commented on Feb 1, 2026

matheuscscp commented on Feb 1, 2026

Vad1mo commented on Feb 1, 2026

matheuscscp commented on Feb 1, 2026

matheuscscp commented on Feb 5, 2026

matheuscscp commented on Feb 5, 2026

vicaya commented on May 7, 2026

matheuscscp commented on May 7, 2026

matheuscscp commented on Oct 6, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Why a new field and not `.spec.provider`