Skip to content

fix(spanner): do not cache retryable errors in AbstractLazyInitializer - #14752

Open
sakthivelmanii wants to merge 2 commits into
mainfrom
fix/lazy-initializer-retryable-errors
Open

sakthivelmanii wants to merge 2 commits into
mainfrom
fix/lazy-initializer-retryable-errors

Conversation

@sakthivelmanii

@sakthivelmanii sakthivelmanii commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Problem

When AbstractLazyInitializer.get() is invoked and initialize() throws an exception, the failure was previously permanently cached in this.error with this.initialized = true regardless of whether the error was transient or retryable. In MultiplexedSessionDatabaseClient.getDialect() and DatabaseClientImpl.getStatementFactory(), an initial transient failure during session creation or query execution (DEADLINE_EXCEEDED, RESOURCE_EXHAUSTED, or UNAVAILABLE) caused all subsequent calls to immediately rethrow the cached exception without retrying, even after multiplexed session creation succeeded on retry.

Solution

Update AbstractLazyInitializer to cache only resource-not-found exceptions (DatabaseNotFoundException, InstanceNotFoundException, SessionNotFoundException) and non-retryable errors. Retryable SpannerException errors (matching MultiplexedSessionDatabaseClient.RETRYABLE_ERROR_CODES: DEADLINE_EXCEEDED, RESOURCE_EXHAUSTED, UNAVAILABLE) are rethrown without setting initialized = true so subsequent get() calls retry initialization.

  • Make MultiplexedSessionDatabaseClient.RETRYABLE_ERROR_CODES package-private so AbstractLazyInitializer shares the same retryable error set.
  • In DatabaseClientImpl.statementFactorySupplier.initialize(), unwrap ExecutionException.getCause() in SpannerExceptionFactory.asSpannerException and convert TimeoutException via SpannerExceptionFactory.propagateTimeout (DEADLINE_EXCEEDED).
  • Add unit tests in LazySpannerInitializerTest and DatabaseClientImplTest covering retryable error recovery, non-retryable error caching, and concurrent retries.

When AbstractLazyInitializer.get() is called and initialize() fails with an exception, the failure was previously cached permanently regardless of whether the underlying error was transient or retryable. In MultiplexedSessionDatabaseClient and DatabaseClientImpl, this caused calls to getDialect() or getStatementFactory() to permanently cache transient startup or query errors (such as DEADLINE_EXCEEDED, RESOURCE_EXHAUSTED, or UNAVAILABLE) and continually rethrow the cached exception even after session creation succeeded on subsequent retries.

This change modifies AbstractLazyInitializer to selectively cache only resource-not-found exceptions (DatabaseNotFoundException, InstanceNotFoundException, SessionNotFoundException) and non-retryable errors. Retryable SpannerExceptions (with error codes matching MultiplexedSessionDatabaseClient.RETRYABLE_ERROR_CODES: DEADLINE_EXCEEDED, RESOURCE_EXHAUSTED, UNAVAILABLE) are rethrown without setting initialized = true, allowing subsequent get() calls to retry initialization.

In addition:
- Relaxed visibility of MultiplexedSessionDatabaseClient.RETRYABLE_ERROR_CODES from private to package-private to share the retryable error classification.
- In DatabaseClientImpl.statementFactorySupplier, unwrapped ExecutionException to surface the underlying cause to SpannerExceptionFactory.asSpannerException, and converted TimeoutException via SpannerExceptionFactory.propagateTimeout to produce DEADLINE_EXCEEDED.
- Added comprehensive unit tests in LazySpannerInitializerTest and DatabaseClientImplTest verifying retry behavior on transient errors and concurrent retry handling.
@sakthivelmanii
sakthivelmanii requested review from a team as code owners October 23, 2026 15:18

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates AbstractLazyInitializer to allow retrying initialization when transient, retryable errors occur, rather than caching them indefinitely. It also refactors exception handling in DatabaseClientImpl when retrieving the database dialect and exposes RETRYABLE_ERROR_CODES from MultiplexedSessionDatabaseClient. Comprehensive tests have been added to verify recovery behavior. The review feedback suggests simplifying the shouldCacheError method in AbstractLazyInitializer by removing redundant checks for specific exception subclasses, as they are already covered by the more general SpannerException check.

Comment on lines +58 to +69
private static boolean shouldCacheError(Exception e) {
if (e instanceof DatabaseNotFoundException
|| e instanceof InstanceNotFoundException
|| e instanceof SessionNotFoundException) {
return true;
}
if (e instanceof SpannerException) {
return !MultiplexedSessionDatabaseClient.RETRYABLE_ERROR_CODES.contains(
((SpannerException) e).getErrorCode());
}
return true;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The explicit checks for DatabaseNotFoundException, InstanceNotFoundException, and SessionNotFoundException are redundant. These classes all extend SpannerException and have the error code ErrorCode.NOT_FOUND. Since NOT_FOUND is not present in MultiplexedSessionDatabaseClient.RETRYABLE_ERROR_CODES (which only contains DEADLINE_EXCEEDED, RESOURCE_EXHAUSTED, and UNAVAILABLE), the subsequent SpannerException check will naturally return true for them. Removing these redundant checks simplifies the code and reduces unnecessary class coupling.

  private static boolean shouldCacheError(Exception e) {
    if (e instanceof SpannerException) {
      return !MultiplexedSessionDatabaseClient.RETRYABLE_ERROR_CODES.contains(
          ((SpannerException) e).getErrorCode());
    }
    return true;
  }

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simplified shouldCacheError to rely on the SpannerException check directly.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant