Skip to content

Improve retrieval quality with reranking layer #204

Closed

Description

@Ayush-kathil

Improve retrieval quality with simple reranking

Currently we rely on vector similarity alone to pick top-k documents. In some cases this returns chunks that are close semantically but not the most useful for answering the query.

A simple improvement would be to introduce a reranking step after retrieval:

  • expand the candidate pool slightly
  • rerank using additional signals (e.g. keyword overlap, metadata)
  • select top-k after reranking

This should improve answer quality without adding much complexity.

I can work on an initial version of this if this direction makes sense.

Activity

  1. Flux07-gezz commented on May 14, 2026

    @Flux07-gezz

    Hey @Ayush-kathil !
    I'd like to work on this improvement.

    I explored the current retrieval flow and found search_kubeflow_docs() where Milvus currently returns the top-k results directly. My plan is to expand the retrieval candidate pool, add a lightweight reranking step using semantic similarity + keyword overlap (and potentially metadata signals later), then select the final top-k results after reranking.

    Starting with a simple implementation first to keep complexity low.

  2. tanich04 commented on Jun 4, 2026

    @tanich04

    hey @Ayush-kathil ,I’d like to work on this.

    Proposed first PR scope:

    • retrieve a slightly larger candidate pool from vector search
    • build a lightweight lexical ranking over the retrieved candidates
    • fuse vector rank and lexical rank using Reciprocal Rank Fusion (RRF)
    • keep the implementation dependency-free for now
    • add unit tests for RRF and lexical reranking behavior
    • document the reranking logic briefly

    Does this scope sound useful?

  3. chasecadet commented on Nov 20, 2026

    @chasecadet
    Contributor

    Closing as completed: TEI cross-encoder reranking and extractive context compression shipped on main in #262.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions