Add pre-create duplicate check and reduce sleep - #58
Open
mweibel wants to merge 2 commits into
Open
Conversation
mweibel
force-pushed
the
fix/eliminate-sleep-precheck
branch
from
August 28, 2026 15:26
b3d37ea to
61ca492
Compare
mweibel
force-pushed
the
fix/duplicate-lb-mutex
branch
from
August 28, 2026 15:45
12a02c9 to
98b6400
Compare
mweibel
force-pushed
the
fix/eliminate-sleep-precheck
branch
from
August 28, 2026 15:48
61ca492 to
ecb5f5d
Compare
mweibel
force-pushed
the
fix/duplicate-lb-mutex
branch
from
August 31, 2026 06:20
98b6400 to
137ebee
Compare
mweibel
force-pushed
the
fix/eliminate-sleep-precheck
branch
from
August 31, 2026 06:20
ecb5f5d to
4ac07c2
Compare
alakae
reviewed
Aug 31, 2026
| // Wait between 5-7.5 seconds between state fetches | ||
| // #nosec G404 | ||
| wait := time.Duration(5000+rand.Intn(2500)) * time.Millisecond | ||
| wait := 500 * time.Millisecond |
Contributor
There was a problem hiding this comment.
IMHO this is too aggressive. Open to discuss and test going form 5-7.5s to lets say 2-4s, but I'd recommend against 500ms.
Collaborator
Author
There was a problem hiding this comment.
given that we had more integration test failures with this: I'll revert this change.
Comment on lines
+54
to
+65
| existing, err := client.LoadBalancers.List(ctx) | ||
| if err == nil { | ||
| for _, lb := range existing { | ||
| if lb.Name == a.lb.Name { | ||
| klog.InfoS("lb already exists, skipping create", | ||
| "name", a.lb.Name, "uuid", lb.UUID) | ||
|
|
||
| return Refresh, nil | ||
| } | ||
| } | ||
| } | ||
|
|
Contributor
There was a problem hiding this comment.
if the mutex in the other PR works correctly, when would you expected that this case is still relevant?
Collaborator
Author
There was a problem hiding this comment.
it's more of a safety check. We can also remove it.
mweibel
force-pushed
the
fix/duplicate-lb-mutex
branch
from
September 1, 2026 09:07
137ebee to
66acf8a
Compare
Prevents duplicate LB creation when multiple goroutines process the same service concurrently (e.g. EnsureLoadBalancer + UpdateLoadBalancer triggered by node sync). Uses sync.Map keyed by service UID. Locks are not cleaned up on service deletion to avoid issues with late-arriving goroutines. Includes a failing unit test that reproduces the concurrent creation race.
mweibel
force-pushed
the
fix/duplicate-lb-mutex
branch
from
September 1, 2026 09:08
66acf8a to
519158d
Compare
Before creating a load balancer, list existing ones and skip creation if a LB with the same name already exists. This prevents duplicate creation even if the race window is hit. Reduce the reconciliation sleep from 5-7.5s to 500ms. With the pre-check in place, the long delay is no longer required for correctness.
mweibel
force-pushed
the
fix/eliminate-sleep-precheck
branch
from
September 1, 2026 10:08
4ac07c2 to
ef4e6cc
Compare
mweibel
force-pushed
the
fix/duplicate-lb-mutex
branch
from
September 1, 2026 13:19
519158d to
c9e310e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Before creating a load balancer, list existing ones and skip creation if a LB with the same name already exists. This prevents duplicate creation even if the race window is hit.
Reduce the reconciliation sleep from 5-7.5s to 500ms. With the pre-check in place, the long delay is no longer required for correctness.