Skip to content

Remove kolla entrypoint pattern, harden SecurityContext - #984

Open
stuggi wants to merge 1 commit into
openstack-k8s-operators:mainfrom
stuggi:remove-kolla
Open

Remove kolla entrypoint pattern, harden SecurityContext#984
stuggi wants to merge 1 commit into
openstack-k8s-operators:mainfrom
stuggi:remove-kolla

Conversation

@stuggi

@stuggi stuggi commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Remove the kolla_start/kolla_set_configs/config.json staging pattern from all three telemetry services (aodh, ceilometer, cloudkitty) and replace with direct exec commands and SubPath volume mounts at final destinations.

Key changes:

Aodh (autoscaling):

  • Replace /bin/bash -c kolla_start with direct exec: aodh-api: /usr/sbin/httpd -DFOREGROUND aodh-evaluator/notifier/listener: direct binary with --logfile aodh-dbsync: /usr/bin/aodh-dbsync
  • Rewrite volumes.go with per-container SubPath mount functions (getAPIVolumeMounts, getEvaluatorVolumeMounts, getWorkerVolumeMounts) replacing the staging-area whole-secret mount pattern
  • Fix TLS cert mount paths: set CertMount/KeyMount before CreateVolumeMounts() so certs mount at /etc/pki/tls/ final paths instead of lib-common's staging path (was a latent bug masked by kolla's copy step)
  • Add run-httpd and var-log-httpd emptyDir volumes for httpd
  • Remove scripts Secret (only contained common.sh kolla merge helper)

Ceilometer:

  • Replace kolla_start with direct exec: ceilometer-polling --polling-namespaces central --logfile /dev/stdout ceilometer-agent-notification --logfile /dev/stdout
  • Keep scripts Secret for centralhealth.py and notificationhealth.py health check scripts (complex psutil-based logic, not replaceable with simple probes), mounted via SubPath instead of whole-dir mount
  • Rename log-httpd volume to var-log-httpd for cross-operator consistency
  • Remove common.sh from scripts Secret AdditionalTemplate (not used by any consumer including EDPM ceilometer-compute/ipmi)

CloudKitty:

  • Replace kolla_start with direct exec: cloudkitty-api: /usr/sbin/httpd -DFOREGROUND cloudkitty-processor: /usr/bin/cloudkitty-processor --logfile cloudkitty-dbsync: /usr/bin/cloudkitty-dbsync upgrade cloudkitty-storageinit: /usr/bin/cloudkitty-storage-init
  • Replace healthcheck.py (psutil process check) with pgrep -f probe
  • Mount loki certs at final path /etc/cloudkitty/certs instead of staging at /var/lib/openstack/loki-certs
  • Add run-httpd and var-log-httpd emptyDir volumes for cloudkitty-api
  • Remove scripts Secret entirely (healthcheck.py replaced, run-on-host was dead code, common.sh was kolla merge helper)
  • Fix TLS cert mount paths for cloudkitty-api

All services:

  • Replace local UID constants (AodhUserID, CeilometerUserID, CloudKittyUserID) with users.XxxUID/XxxGID from lib-common/users package (sourced from s2i-openstack-containers registry)
  • Remove dead CloudKittyUserID/GroupID (42408) from api/v1beta1 types (conflicted with actual internal value 42406)
  • Add pod.RestrictivePodSecurityContext with service UID/GID and ApacheGID (48) supplemental group for httpd-fronted services
  • Add pod.RestrictiveSecurityContext on all containers (Drop ALL capabilities, SeccompProfile RuntimeDefault, RunAsNonRoot)
  • Add AutomountServiceAccountToken: false on all workloads
  • Standardize DefaultMode to 0440 for config Secrets, 0550 for scripts
  • Remove KOLLA_CONFIG_STRATEGY and KOLLA_BOOTSTRAP env vars
  • Remove KollaConfigCentral/KollaConfigNotification dead constants
  • Delete all 11 kolla config.json template files
  • Delete common.sh (kolla merge helper) and run-on-host (dead nsenter)
  • Update httpd.conf templates: User/Group to service user, PidFile to /run/httpd/httpd.pid, ErrorLog to /dev/stderr
  • Update kuttl test assertions for new commands, mounts, and security

Jira: OSPRH-33504
Jira: OSPRH-33503

Depends-On: openstack-k8s-operators/lib-common#728

Remove the kolla_start/kolla_set_configs/config.json staging pattern
from all three telemetry services (aodh, ceilometer, cloudkitty) and
replace with direct exec commands and SubPath volume mounts at final
destinations.

Key changes:

Aodh (autoscaling):
- Replace /bin/bash -c kolla_start with direct exec:
  aodh-api: /usr/sbin/httpd -DFOREGROUND
  aodh-evaluator/notifier/listener: direct binary with --logfile
  aodh-dbsync: /usr/bin/aodh-dbsync
- Rewrite volumes.go with per-container SubPath mount functions
  (getAPIVolumeMounts, getEvaluatorVolumeMounts, getWorkerVolumeMounts)
  replacing the staging-area whole-secret mount pattern
- Fix TLS cert mount paths: set CertMount/KeyMount before
  CreateVolumeMounts() so certs mount at /etc/pki/tls/ final paths
  instead of lib-common's staging path (was a latent bug masked by
  kolla's copy step)
- Add run-httpd and var-log-httpd emptyDir volumes for httpd
- Remove scripts Secret (only contained common.sh kolla merge helper)

Ceilometer:
- Replace kolla_start with direct exec:
  ceilometer-polling --polling-namespaces central --logfile /dev/stdout
  ceilometer-agent-notification --logfile /dev/stdout
- Keep scripts Secret for centralhealth.py and notificationhealth.py
  health check scripts (complex psutil-based logic, not replaceable
  with simple probes), mounted via SubPath instead of whole-dir mount
- Rename log-httpd volume to var-log-httpd for cross-operator consistency
- Remove common.sh from scripts Secret AdditionalTemplate (not used by
  any consumer including EDPM ceilometer-compute/ipmi)

CloudKitty:
- Replace kolla_start with direct exec:
  cloudkitty-api: /usr/sbin/httpd -DFOREGROUND
  cloudkitty-processor: /usr/bin/cloudkitty-processor --logfile
  cloudkitty-dbsync: /usr/bin/cloudkitty-dbsync upgrade
  cloudkitty-storageinit: /usr/bin/cloudkitty-storage-init
- Replace healthcheck.py (psutil process check) with pgrep -f probe
- Mount loki certs at final path /etc/cloudkitty/certs instead of
  staging at /var/lib/openstack/loki-certs
- Add run-httpd and var-log-httpd emptyDir volumes for cloudkitty-api
- Remove scripts Secret entirely (healthcheck.py replaced, run-on-host
  was dead code, common.sh was kolla merge helper)
- Fix TLS cert mount paths for cloudkitty-api

All services:
- Replace local UID constants (AodhUserID, CeilometerUserID,
  CloudKittyUserID) with users.XxxUID/XxxGID from lib-common/users
  package (sourced from s2i-openstack-containers registry)
- Remove dead CloudKittyUserID/GroupID (42408) from api/v1beta1 types
  (conflicted with actual internal value 42406)
- Add pod.RestrictivePodSecurityContext with service UID/GID and
  ApacheGID (48) supplemental group for httpd-fronted services
- Add pod.RestrictiveSecurityContext on all containers (Drop ALL
  capabilities, SeccompProfile RuntimeDefault, RunAsNonRoot)
- Add AutomountServiceAccountToken: false on all workloads
- Standardize DefaultMode to 0440 for config Secrets, 0550 for scripts
- Remove KOLLA_CONFIG_STRATEGY and KOLLA_BOOTSTRAP env vars
- Remove KollaConfigCentral/KollaConfigNotification dead constants
- Delete all 11 kolla config.json template files
- Delete common.sh (kolla merge helper) and run-on-host (dead nsenter)
- Update httpd.conf templates: User/Group to service user, PidFile to
  /run/httpd/httpd.pid, ErrorLog to /dev/stderr
- Update kuttl test assertions for new commands, mounts, and security

Jira: OSPRH-33504
Jira: OSPRH-33503

Depends-On: openstack-k8s-operators/lib-common#728

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Martin Schuppert <mschuppert@redhat.com>
@openshift-ci

openshift-ci Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: stuggi

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci

openshift-ci Bot commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

@stuggi: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/telemetry-operator-build-deploy-kuttl 4d627ab link true /test telemetry-operator-build-deploy-kuttl

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/32f5954160b6469d9a2fbcdf033799bd

✔️ telemetry-openstack-meta-content-provider-master SUCCESS in 3h 47m 32s
telemetry-operator-multinode-cloudkitty FAILURE in 1h 16m 34s
✔️ telemetry-openstack-meta-content-provider-all-services-master SUCCESS in 3h 27m 57s
telemetry-operator-multinode-master FAILURE in 1h 21m 21s
✔️ openstack-k8s-operators-content-provider SUCCESS in 3h 14m 53s
✔️ telemetry-operator-multinode-default-telemetry SUCCESS in 1h 38m 12s
functional-tests-osp18 FAILURE in 1h 54m 53s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant