diff --git a/ROADMAP.md b/ROADMAP.md index 937b9df06..3c3bccfb7 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -512,7 +512,7 @@ also brings WebRTC DataChannel support into the release scope. | Goal | Status | Notes | |------|--------|-------| -| (a) CI TLS performance gates (HTTPS churn / keep-alive / large-transfer / RTT) | 🔄 In Progress | Gates landed in #309 but never passed: the absolute backstops came from fast CPUs and healthy code failed on the EPYC 7763 runner, so no history accumulated either. Recalibrated around hardware-normalised https/http ratios; disabling the #307 fix fails RTT (0.08 → 43 ms) and churn ratio (0.118 → 0.052). Done once green on dev. Tracked in #306 | +| (a) CI TLS performance gates (HTTPS churn / keep-alive / large-transfer / RTT) | 🔄 In Progress | Gates landed in #309 but never passed: the absolute backstops came from fast CPUs and healthy code failed on the EPYC 7763 runner, so no history accumulated either. Recalibrated on https/http ratios plus same-CPU history; disabling the #307 fix fails the RTT gate (0.08 → 43 ms) on any CPU. The churn-ratio backstop had to be loosened after the first Intel runner (healthy at 0.071 vs 0.11-0.13 on AMD). Done once green on dev. Tracked in #306 | | (b) TLS observability in Prometheus `/metrics` (handshake counts, TLS version/cipher counters, resumption vs full-handshake ratio) | ✅ Done | `cwist_tls_handshakes_total`, `cwist_tls_handshakes_resumed_total`, `cwist_tls_connections_active`, `cwist_tls_handshakes_tls12_total`, `cwist_tls_handshakes_tls13_total`, `cwist_tls_ciphers_{aes128_gcm,aes256_gcm,chacha20,other}_total`; covered by `test_https_metrics` | | (c) Settle issue #294 with measured data | ✅ Done | #294 closed as completed. An earlier comment there claimed `cwist_app_listen` never uses the sharded handshake shepherds; that was wrong (it confused `cwist_app_multiport`'s inline path with `cwist_app_listen`). Measured: `cwist_app_listen` adds one thread per `CWIST_HTTPS_HS_SHARDS` shard on the first HTTPS request, in both C1M and classic mode. The default floor of 4 shards / cap of 16 is already on dev, and #294's data shows no gain beyond 4 | | (d) WebRTC DataChannel support | ✅ Done | #310: SDP offer/answer, ICE-lite, DTLS (vendored BoringSSL), SCTP DataChannels (`lib/usrsctp`) on the cwist reactor. Echo verified against headless Chromium (`make test_webrtc_browser`); Firefox not yet tested | diff --git a/benchmarks/https_gates.json b/benchmarks/https_gates.json index e774cdfc2..0ba014f5a 100644 --- a/benchmarks/https_gates.json +++ b/benchmarks/https_gates.json @@ -133,5 +133,20 @@ "http_big_gbps": 12.97, "https_churn_ratio": 0.118, "https_big_ratio": 0.324 + }, + { + "timestamp": "2026-10-05T11:21:50+0000", + "runner_hw": "4 vCPU | INTEL(R) XEON(R) PLATINUM 8573C", + "tls_rtt_p50_ms": 0.08, + "tls_rtt_p90_ms": 0.123, + "https_churn_rps": 1440.53, + "http_churn_rps": 20430.4, + "https_keepalive_rps": 143567.15, + "http_keepalive_rps": 213915.29, + "https_keepalive_ratio": 0.671, + "https_big_gbps": 3.37, + "http_big_gbps": 7.96, + "https_churn_ratio": 0.071, + "https_big_ratio": 0.423 } ] diff --git a/scripts/ci/https_gates_eval.py b/scripts/ci/https_gates_eval.py index fbf4ba411..f13339942 100644 --- a/scripts/ci/https_gates_eval.py +++ b/scripts/ci/https_gates_eval.py @@ -11,20 +11,24 @@ The absolute throughput numbers swing ~1.8x between the CPUs GitHub hands out, so their backstops only catch breakage that is wrong on any hardware. -The TLS-specific gates are the https/http *ratios*: both sides are measured -on the same machine in the same run, so the hardware cancels out. Recorded -healthy range (7 CI runs on EPYC 7763 / 9V74 / 9V45, plus Ryzen 5600X): +The https/http *ratios* cancel most of the hardware out (both sides are +measured in the same run), but not all of it: TLS handshake cost relative +to a plaintext accept differs between CPU vendors. Recorded healthy range: - ratio healthy backstop pre-#307 tree (Ryzen) - churn_ratio 0.111-0.127 0.08 0.052 - keepalive_ratio 0.552-0.656 0.45 0.705 (unaffected) - big_ratio 0.337-0.519 0.25 0.300 (unaffected) + ratio AMD (EPYC 7763/9V74/9V45, Ryzen) Intel Xeon 8573C backstop + churn_ratio 0.111-0.127 0.071 0.04 + keepalive_ratio 0.552-0.656 0.671 0.45 + big_ratio 0.337-0.519 0.423 0.25 -Regression check, Ryzen 5600X, with #307's TCP_QUICKACK re-arm disabled: -tls_rtt_p50_ms 0.083 -> 43.0 and churn_ratio 0.118 -> 0.052, so both the -RTT gate and the churn-ratio gate fail it; https_churn_rps (677/s) also -falls under its backstop, but that one alone would not catch it on a fast -CPU. test_https_gates_eval.py replays both rows. +The churn-ratio backstop therefore cannot separate a regression from a +vendor difference (a regressed AMD tree measures 0.052, a healthy Intel one +0.071); it catches large drops, and the same-CPU check below catches the +rest once a CPU has history. + +#307-class regressions are caught by tls_rtt_p50_ms, which does not depend +on the hardware: Ryzen 5600X with #307's TCP_QUICKACK re-arm disabled went +from 0.083 to 43.0 ms (healthy runs measure 0.07-0.13 ms on every CPU seen). +test_https_gates_eval.py replays that row. Usage: https_gates_eval.py RESULT.json [HISTORY.json] # evaluate, exit 1 on fail @@ -53,7 +57,7 @@ GATES = [ ("tls_rtt_p50_ms", "max", 5.0, 2.0), # TLS-path gates, hardware-normalised (see module docstring). - ("https_churn_ratio", "min", 0.08, 1.20), + ("https_churn_ratio", "min", 0.04, 1.20), ("https_keepalive_ratio", "min", 0.45, 1.20), ("https_big_ratio", "min", 0.25, 1.20), # Absolute throughput: catastrophic-only backstops. diff --git a/scripts/ci/test_https_gates_eval.py b/scripts/ci/test_https_gates_eval.py index 367380dfd..01985d858 100644 --- a/scripts/ci/test_https_gates_eval.py +++ b/scripts/ci/test_https_gates_eval.py @@ -44,17 +44,27 @@ def test_healthy_tree_passes(self): failures, _ = evaluate([], HEALTHY) self.assertEqual(failures, []) - def test_pre_307_regression_fails_on_rtt_and_churn_ratio(self): + def test_pre_307_regression_fails_on_rtt(self): failures, _ = evaluate([], PRE_307) - self.assertTrue({'tls_rtt_p50_ms', 'https_churn_ratio'} <= failed_metrics(failures), - failures) - - def test_churn_ratio_alone_catches_it_when_absolute_churn_looks_fine(self): - # On a faster CPU the regressed absolute churn can clear its backstop; - # the ratio must still fail. - fast = dict(PRE_307, https_churn_rps=1100.0, http_churn_rps=21000.0, tls_rtt_p50_ms=0.1) - failures, _ = evaluate([], fast) - self.assertEqual(failed_metrics(failures), {'https_churn_ratio'}) + self.assertIn('tls_rtt_p50_ms', failed_metrics(failures)) + + def test_pre_307_regression_fails_same_cpu_churn_ratio(self): + # With Ryzen history, the churn-ratio drop (0.118 -> 0.052) fails the + # same-CPU check even where the absolute backstop is too loose to. + history = [dict(HEALTHY, timestamp=f'2026-10-0{i}') for i in range(1, 6)] + failures, _ = evaluate(history, dict(PRE_307, tls_rtt_p50_ms=0.1)) + self.assertIn('https_churn_ratio', failed_metrics(failures)) + + def test_healthy_intel_runner_passes_without_history(self): + # Run 37301931499: lower churn ratio than any AMD runner, healthy code. + intel = { + 'runner_hw': '4 vCPU | INTEL(R) XEON(R) PLATINUM 8573C', 'tls_rtt_p50_ms': 0.08, + 'https_churn_rps': 1440.53, 'http_churn_rps': 20430.4, + 'https_keepalive_rps': 143567.15, 'http_keepalive_rps': 213915.29, + 'https_big_gbps': 3.37, 'http_big_gbps': 7.96, + } + failures, _ = evaluate([], intel) + self.assertEqual(failures, []) def test_every_recorded_run_passes_against_the_rest(self): history = json.loads(HISTORY.read_text())