feat(lapis): distinguish a SILO timeout from SILO being unreachable - #1867
feat(lapis): distinguish a SILO timeout from SILO being unreachable#1867corneliusroemer wants to merge 1 commit into
Conversation
|
@corneliusroemer-agent is attempting to deploy a commit to the cov-spectrum Team on Vercel. A member of the Team first needs to authorize it. |
There was a problem hiding this comment.
🟡 Changes recommended
User-facing error messages now include full SILO URIs (host/details) that are returned in API responses, which can leak internal service addresses.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR improves SILO error reporting in the LAPIS service by distinguishing request/connection timeouts from true “SILO not reachable” scenarios, and by mapping timeouts to HTTP 503 to align with other SILO-unavailable cases.
Changes:
- Introduces
SiloTimeoutExceptionand throws it onHttpTimeoutExceptionwith a more specific timeout message. - Updates “could not connect” messaging to be more accurate and less assumption-based for non-connect exceptions.
- Adds a MockServer-based test to verify slow
/inforesponses triggerSiloTimeoutException.
File summaries
| File | Description |
|---|---|
lapis/src/main/kotlin/org/genspectrum/lapis/silo/SiloClient.kt |
Adds timeout-specific exception and messaging; refines connect vs other failure messages. |
lapis/src/main/kotlin/org/genspectrum/lapis/controller/ExceptionHandler.kt |
Maps SiloTimeoutException to HTTP 503 with warning-level logging. |
lapis/src/test/kotlin/org/genspectrum/lapis/silo/SiloClientTest.kt |
Adds coverage ensuring slow /info triggers the new timeout exception. |
Review details
Suppressed comments (2)
lapis/src/main/kotlin/org/genspectrum/lapis/silo/SiloClient.kt:257
- This generic SILO error message also embeds the full URI (including host). Since Throwable messages are returned in error responses (including 500s), consider removing host details from the message to avoid exposing internal topology to clients.
val message = "Error talking to silo at $uri: ${exception::class} ${exception.message}"
lapis/src/main/kotlin/org/genspectrum/lapis/silo/SiloClient.kt:295
- The timeout message includes the full SILO URI (including host) and is returned to clients as the 503 body. Consider limiting it to the endpoint path to avoid leaking internal service addresses while still distinguishing timeout vs connect-timeout.
val elapsedMillis = (System.nanoTime() - startedAtNanos) / 1_000_000
return when (exception) {
is HttpConnectTimeoutException -> "Timed out connecting to silo at $uri after ${elapsedMillis}ms"
else -> "Timed out waiting for a response from silo at $uri after ${elapsedMillis}ms"
}
- Files reviewed: 3/3 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| } | ||
| } catch (connectException: ConnectException) { | ||
| val message = "Could not connect to silo: ${connectException::class} ${connectException.message}" | ||
| val message = "Could not connect to silo at $uri: ${connectException::class} ${connectException.message}" |
A timeout was reported as "Could not connect to silo", which is wrong and misleads whoever reads it. HttpTimeoutException means LAPIS did connect, sent the request and gave up waiting - SILO is reachable and may be answering other callers perfectly well. Only ConnectException means not reachable. Seen in the wild: LAPIS evicted itself from its Kubernetes Service after its /sample/info readiness probes failed, while SILO was answering /info in under a millisecond throughout. The message pointed at networking; the cause was callInfo()'s 100 ms budget. The new message therefore also names the target URI and how long the call actually took, since neither is otherwise visible - a /sample/info that fails in ~140 ms gives no hint that the bound was 100 ms rather than the probe's 5 s. SiloTimeoutException maps to 503, the same status the other SILO-unavailable paths use. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2237e0a to
36ea810
Compare
A timeout was reported as "Could not connect to silo", which is wrong and misleads whoever reads it. HttpTimeoutException means LAPIS did connect, sent the request and gave up waiting - SILO is reachable and may be answering other callers perfectly well. Only ConnectException means not reachable.
Related to #1866 in that the misleading exception came up while figuring out what caused info to return silo connection error.
SiloTimeoutException maps to 503, the same status the other SILO-unavailable paths use.
Also edits the message for an uncaught silo exception: the problem isn't necessarily connection failure, could be anything so message shouldn't assume.
PR Checklist
llms.txt.