feat(otel-collector): add OpenTelemetryCollector CRD, controller, and render#4979
Draft
tianfeng92 wants to merge 20 commits into
Draft
feat(otel-collector): add OpenTelemetryCollector CRD, controller, and render#4979tianfeng92 wants to merge 20 commits into
tianfeng92 wants to merge 20 commits into
Conversation
tianfeng92
force-pushed
the
PMREQ-822-otel-collector
branch
from
69eaaa5 to
48c6ba9
Compare
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| Action: v3.Allow, | ||
| Protocol: &networkpolicy.TCPProtocol, | ||
| Destination: v3.EntityRule{ | ||
| Ports: networkpolicy.Ports(uint16(p)), |
marvin-tigera
added
docs-pr-required
release-note-required
and removed
docs-pr-required
release-note-required
labels
Replace the fluentd DaemonSet with fluent-bit for log collection and forwarding. The LogCollector controller now renders the calico-fluent-bit DaemonSet (Linux and Windows) in calico-system, and pkg/render/fluentd.go is replaced by fluentbit.go. - Ship fluent-bit logs to Linseed through its built-in http output. - Rename the FluentdDaemonSet* API types to FluentBitDaemonSet* (fluentd_daemonset_types.go -> fluentbit_daemonset_types.go). Preserve the deprecated fluentdDaemonSet override field name/json tag as an alias and widen its enums to accept both the new calico-fluent-bit* names and the legacy fluentd names so existing LogCollector specs still validate; translateLegacyFluentdOverrides remaps the legacy names. - Warn on invalid fluent-bit-filters ConfigMap content (e.g. left in fluentd <filter> syntax) instead of silently dropping it. - Drop the bogus "calico-fluent-bit" entry from the manager cluster-wide namespace list; fluent-bit runs in calico-system, which is already listed. - Regenerate deepcopy, the operator CRD and enterprise versions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Jiawei Huang <jiawei@tigera.io>
Forward flow, DNS and policy-activity logs from non-cluster hosts through voltron to the in-cluster calico-fluent-bit http input, and on to Linseed. - Grant dnslogs (alongside flowlogs and policyactivity) on the non-cluster-host ClusterRole so the minted host token passes voltron's SubjectAccessReview for the DNS ingestion path instead of 403ing. - Set VOLTRON_LOG_COLLECTOR_CA_BUNDLE_PATH on the manager so voltron verifies the calico-fluent-bit http input's TLS server certificate against the trusted CA bundle it already mounts (the config default /etc/pki/tls/certs/ca.crt is not mounted, so the handshake otherwise fails). - Pass NonClusterHost to the Windows fluent-bit configuration so the Linux and Windows renders produce the shared allow-calico-fluent-bit NetworkPolicy identically. Otherwise, on clusters with Windows nodes, the port-9880 ingress rule (voltron -> http input) flapped on every reconcile and intermittently dropped voltron's access. Adds a controller regression test. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Jiawei Huang <jiawei@tigera.io>
pkg/render/fluentbit.go had grown to ~1900 lines. Move the fluent-bit / EKS log-forwarder rendering into a new pkg/render/logcollector package, split across focused files (logcollector core, config, outputs, daemonset, rbac, networkpolicy, eks_log_forwarder) plus the moved tests. A small set of symbols stays in package render (new pkg/render/logcollector.go) to avoid a render -> render/logcollector import cycle, since Guardian, Manager, compliance, apiserver, dex and intrusion detection reference them: the log-collector network-policy identity (FluentBitSourceEntityRule, EKSLogForwarderEntityRule, LogCollectorNamespace, the fluent-bit node names, FluentBitInputService), the shared Linseed-token constants, and the TrustedBundleVolume helper. The logcollector package aliases these. The shared pod helper setNodeCriticalPod is exported as SetNodeCriticalPod (matching its sibling SetClusterCriticalPod). Pure code move; no behavior change. Build, vet, unit tests, format-check and gen-files all pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Jiawei Huang <jiawei@tigera.io>
…t cert coupling Live validation surfaced that every fluent-bit Prometheus target was down with scrape timeouts: the calico-system.prometheus egress policy still allowed only the fluentd-era metrics port 9081 (which stays — it is also the elasticsearch-metrics port) and was never taught fluent-bit's 2020. Add the egress rule and update the policy fixtures. Verified end-to-end on a GCP kubeadm cluster: targets went from 0 up / 6 down to all up. Since the fluent-bit metrics endpoint is scraped over plain HTTP, the monitor controller has no use for the fluent-bit certificate: drop it from Prometheus's trusted bundle and remove the fluent-bit and EKS-log-forwarder secret watches (the latter was dead even in the fluentd era — never in the bundle), decoupling the monitor controller from the LogCollector controller entirely. Linseed's trust of the fluent-bit client certificate (logstorage secrets controller) is unchanged — that hop is real mTLS. Also use the ServiceMonitor name constant in the watch error message (review feedback) and refresh serviceMonitorFluentBit's godoc, which still explained a TLS-era rationale. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…EKS RBAC
Legacy overrides (review: the container rename is a breaking change):
route the deprecated fluentdDaemonSet field through the shared
containerNameAliases mechanism in pkg/render/common/components instead
of a bespoke translation, mapping the fluentd-era container and
init-container names to their calico-fluent-bit equivalents, and add
render tests for the alias path and for calicoFluentBitDaemonSet
precedence when both fields are set.
EKS forwarder RBAC: the legacy-cleanup list deleted the cluster-scoped
eks-log-forwarder ClusterRoleBinding that the EKS render re-creates
under the same (fluentd-era) name; creates are processed before deletes,
so on EKS clusters every reconcile deleted the forwarder's Linseed RBAC.
The binding is reused in place, so drop it from the cleanup list (and
add the missed fluentd-node-windows Linseed token secret).
Additional-store parity with fluentd (verified against ee_entrypoint.sh,
syslog-config.sh, splunk-config.sh and the out-s3-*.conf templates):
- hostScope gates only cluster *flow* logs for all three stores, and
non-cluster flows always ship when a store is enabled; the previous
render dropped every cluster type but NCH flows under NonClusterOnly
and never shipped NCH flows under All.
- S3: preserve the legacy archive prefixes (audit_tsee, ...), gzip the
objects to match their .gz suffix (fluent-plugin-s3's default
store_as), and drop waf, which the entrypoint never archived. NCH
flows get their own non_cluster_flows/ prefix — out_s3's $INDEX is
per-output, so sharing flows/ would overwrite objects.
- syslog: drop syslog_severity_preset "info" — the property is an
integer (atoi("info") = 0 = Emergency) and the default 6 already is
info — and add syslog_hostname_preset ${NODENAME} as the node-name
fallback for tags whose records carry no host key, matching fluentd's
static SYSLOG_HOSTNAME.
- parse audit.* and ids.events event time (time_key) like the fluentd
sources did, so S3 day partitions and syslog timestamps use event
time. Rendered configs (default and all-stores) validated with the
real fluent-bit image via --dry-run.
Windows: pass Tenant/ExternalElastic into the Windows render so the
shared allow-calico-fluent-bit policy cannot flap on multi-tenant
management clusters with Windows nodes, and the Windows Linseed outputs
target the right namespace and tenant header.
Cleanup: remove the ES-era ElasticsearchEksLogForwarderUserSecret and
cluster-config watches (nothing reads them since the ES output was
dropped) and the unused /etc/fluent-bit/certs trusted-bundle mounts.
Add a controller test for the fluent-bit-filters warning path
(SetWarning on unparseable content, cleared once fixed, rendering
continues) and tighten the store-output render assertions.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… render Add the full OTel Collector operator support: CRD types, controller with license gating, render component, deployment override validation, fluentd integration, and unit tests for all layers. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace the fluentforward protocol with a custom fluentdhttp receiver that accepts Fluentd's out_http JSON format, enabling future mTLS support via the OTel confighttp framework. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…pipeline Route metrics from prometheus receiver to a dedicated prometheusremotewrite exporter instead of sharing log exporters. Add Prometheus port 9090 to network policy egress rules when metrics are enabled. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…tests Verify the metrics pipeline uses the dedicated prometheusremotewrite exporter and that the network policy includes Prometheus port egress when metrics are enabled. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…nd mTLS Add prometheus receiver with Kubernetes SD and mTLS for scraping calico-node metrics. Render TLS volume mounts from certificate manager. Switch config generation from string builder to Go template for maintainability. Add memory_limiter processor and resource limits. Wire OTel log types through to fluent-bit output rendering. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fluent-bit's native opentelemetry output plugin is already compiled in, so we skip the bridge phase and go straight to OTLP end-to-end. Remove LogForwarderProtocol abstraction, FluentForwardPort, and fluentdhttp references. Fluent-bit now uses out_opentelemetry targeting port 4318. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…tion Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Guard body field access with IsMap(body) to prevent "log bodies of type Str cannot be indexed" warnings when logs arrive as plain strings. Simplify audit classification to match all audit logs by auditID only. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…te otel-collector image Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…n and logcollector controller The OTel collector has its own controller and render path — the fluentd render code doesn't need OTelCollectorEnabled or OTelLogTypes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…metrics, and dynamic egress rules Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ess rule Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…prometheus The "no metrics disabled" test checked for absence of "prometheus:" which now always appears in the telemetry.metrics.readers block. Assert on "scrape_configs:" instead, which is specific to the prometheus receiver. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The rebased component handler clears ignore-annotation warnings via status.ClearWarning during CreateOrUpdateOrDelete. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
tianfeng92
force-pushed
the
PMREQ-822-otel-collector
branch
from
5585581 to
f084ce5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
OpenTelemetryCollectorCRD with support for logs (OTLP receiver) and metrics (Prometheus receiver) pipelinesLogCollectortypes to carry OTel-related fields for fluent-bit integrationCombinedCalicoImage) consistent with upstream refactorChanges
api/v1/otelcollector_types.go,api/v1/logcollector_types.go,zz_generated.deepcopy.gopkg/controller/otelcollector/controller.go,internal/controller/pkg/render/otelcollector/component.gopkg/common/validation/otelcollector/validation.gopkg/controller/logcollector/logcollector_controller.gopkg/render/otelcollector/component_test.go,pkg/controller/otelcollector/otelcollector_controller_test.gopkg/imports/crds/operator/Test plan
make ut UT_DIR=./pkg/render/otelcollectormake ut UT_DIR=./pkg/controller/otelcollectormake ci)🤖 Generated with Claude Code