Skip to content

[BUG] Introduce Bloom Filter in UIN existence check to reduce DB calls during generation #1841

Description

@dhanendra06

Summary:
During bulk UIN generation, UinGeneratorImpl.generateId() calls uinService.uinExist() for every candidate UIN to check against previously assigned UIns. This results in one DB query per candidate, which
becomes a significant bottleneck as the assigned UIN pool grows into the millions.

Component: kernel-idgenerator-service
Files Affected:

  • UinServiceImpl.java — uinExist() method
  • UinGeneratorImpl.java — generateId() loop

Problem:
// UinGeneratorImpl.java:129 — DB hit on every loop iteration
if (uinFilterUtils.isValidId(generatedUIN) && !uinService.uinExist(generatedUIN)) {

// UinServiceImpl.java:144 — always goes to DB
public boolean uinExist(String uin) {
Optional uinEntityAssignedOptional = uinRepositoryAssigned.findById(uin);
return uinEntityAssignedOptional.isPresent();
}

Proposed Solution:
Introduce a BloomFilter (Guava, already transitively available) in UinServiceImpl:

  1. On startup — populate the Bloom filter by loading all existing UIns from UinEntityAssigned
  2. On uinExist() — check Bloom filter first:
    - Returns false → UIN is definitely new, skip DB query entirely
    - Returns true → confirm with DB (handles rare false positives)
  3. On transferUin() — add newly assigned UIns to the Bloom filter to keep it in sync

Expected Benefit:

  • Eliminates ~99.9% of DB existence checks during bulk UIN generation (at 0.1% false positive rate)
  • ~120MB RAM for 100M UIns — acceptable for a long-running service
  • No change needed in UinGeneratorImpl — logic stays encapsulated in uinExist()
  • No new Maven dependency — Guava is already available via Spring Boot

Acceptance Criteria:

  • Bloom filter initialized at startup from UinEntityAssigned table
  • uinExist() checks Bloom filter before hitting DB
  • Bloom filter updated whenever a UIN is transferred/assigned
  • Unit test covering false positive path (Bloom says exists → DB confirms not exists → treated as new)
  • Benchmark showing reduced DB call count during a generation run

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    Priority

    None yet

    End Date

    None yet

    Complexity

    None yet

    Start Date

    None yet

    Original Estimate

    None yet

    Time Tracking

    None yet

    Resolved By

    None yet

    Story Points

    None yet

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions