Repository navigation
MinimalHttpAsyncClient: keep the connect cancellable when the lease completes synchronously - #951
Merged
Merged
Conversation
…ompletes synchronously When the pool hands out a new endpoint right away, the lease callback starts the connect and registers it with the result future before lease() returns. The lease future, which is already done, then replaced it, so cancelling the request did not reach the connect. When the connect later finished, the result future was already cancelled and the endpoint was never released back to the pool. Register the lease through a holder installed before the lease starts.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #943, the same overwrite in
MinimalHttpAsyncClient#leaseEndpoint.When the pool has room,
manager.lease()completes synchronously. The lease callback starts the connect and callsresultFuture.setDependency(connectFuture)beforelease()returns, and theresultFuture.setDependency(leaseFuture)right after it replaces that with the lease, which is already done. Cancelling the request during the connect doesn't reach the connect. When the connect finishes,resultFuture.completed()is ignored because the future is already cancelled, so the endpoint is never released. Every such cancel takes one connection out of the pool for good.The change uses the same
ComplexCancellableholder as #943.How I tested it
HttpAsyncClients.createMinimal(cm), max 20 per route, against a local TLS server that finishes the handshake after 1 s. 20 requests, each cancelled after 300 ms, then one more request with a 2 s connection request timeout.With this change alone the cancelled TLS sessions still stay open until their handshake ends. That part is #950.
Same thing under load, 150 s per build: 10 threads, every request cancelled if it takes more than 300 ms, one request per connection, and the server delays the handshake by 2 s on about 10% of new connections. Prometheus scrapes every 2 s.
On master all 20 connections are leased from the first samples and stay that way. Only 196 requests succeed in 150 s; the rest wait for a lease until they're cancelled. With the change it's 15.2K successful requests (~97 req/s) and no pending leases. The client's open sockets still grow on this branch, which is the handshake issue fixed in #950.
TestMinimalHttpAsyncClient#testCancellationReachesConnectAfterSynchronousLeasefails on master and passes with the change. httpclient5verify: 1161 tests, 0 failures. httpclient5-testing async tests: 407, 0 failures.While testing I also noticed that cancelling a
MinimalHttpAsyncClientrequest after the exchange has started doesn't abort it.execute()only links the cancellable to the lease future, which is complete by then, so the server still sends the response 3 s later and the connection goes back to the pool.HttpAsyncClientcloses the connection in the same case. Is that intended for the minimal client, or should I look into it?