fix(shared): clean up zero-count entries in ConnectionLimiter to prevent memory leak

[chill-czar]

Closes #3448

Summary

• Automatically evict zero-count key entries from ConnectionLimiter map when all active connections for a key are released.
• Add atomic eviction callback RemoveCb in Release(key) within packages/shared/pkg/connlimit/limiter.go.
• Add unit test coverage in packages/shared/pkg/connlimit/limiter_test.go asserting zero-count map key cleanup and fresh re-acquisition behavior.

Why

ConnectionLimiter in packages/shared/pkg/connlimit tracks per-key concurrent connections in an smap.Map[*atomic.Int64]. Previously, Release(key) decremented the counter to 0 via CompareAndSwap, but never removed the key entry from the underlying concurrent map.

For generic HTTP proxy connection limiting and client-IP rate limiting where there is no sandbox termination event to invoke Remove(key) manually, every historic unique key remained in memory with count 0. On long-running edge proxy services (client-proxy and orchestrator/pkg/tcpfirewall), this resulted in steady heap memory accumulation over time.

smap.RemoveCb checks v.Load() == 0 under smap's internal shard lock, guaranteeing zero-count keys are safely removed without race conditions against concurrent TryAcquire calls.

Test Plan

• Unit tests pass: go test -race -v ./packages/shared/pkg/connlimit/...
• Verified zero-count key eviction via limiter.connections.Count(). assertions
• Verified concurrent acquire/release under Go race detector (-race)
• Formatted code cleanly: go fmt ./...
• No regressions across packages/shared test suite

/cc @jakubno @dobrac @ValentaTomas @arkamar @tvi

[chill-czar]

@codex

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 119faf42c5

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Keep zero-count cleanup from deleting a newly acquired counter

When the last Release runs concurrently with a new request for the same key, TryAcquire can get the existing zero-valued counter from Upsert but increment it only after this RemoveCb has observed v.Load() == 0; the callback then deletes the map entry even though the new connection successfully acquired the slot. In the proxy/firewall paths that use this limiter, that live connection is no longer tracked, so later releases can undercount a different counter or connection limits can be bypassed for the sandbox lifecycle.

AGENTS.md reference: AGENTS.md:L22-L26

Useful? React with 👍 / 👎.

[chill-czar]

Good catch! Fixed in commit e4f15b5 using atomic tombstone state (-1):

1. In Release: When a counter decrements from 1 to 0, Release attempts counter.CompareAndSwap(0, -1) to mark the counter instance as tombstoned. If CAS(0, -1) succeeds, Release calls RemoveCb to remove key from the map if it still references that tombstoned counter instance. If a concurrent TryAcquire acquired the counter before CAS(0, -1), CAS(0, -1) fails and Release leaves the map entry intact.
2. In TryAcquire: If a counter has state -1 (current < 0), TryAcquire detects the tombstoned counter and loops to re-fetch/insert a fresh counter instance via getCounter. getCounter atomically replaces tombstoned counters in smap.Upsert.
3. Tests: Added TestConnectionLimiter_Concurrent/concurrent_acquire_during_release_does_not_orphan_connections in limiter_test.go to verify zero untracked connections under high concurrency.