Skip to content

fix(joiners): treat non-finite scores as zero when merging documents - #13805

Draft
DZH166 wants to merge 1 commit into
deepset-ai:mainfrom
DZH166:fix/document-joiner-non-finite-scores
Draft

DZH166 wants to merge 1 commit into
deepset-ai:mainfrom
DZH166:fix/document-joiner-non-finite-scores

Conversation

@DZH166

@DZH166 DZH166 commented Nov 18, 2026

Copy link
Copy Markdown

Problem

DocumentJoiner merges the weighted contributions with a plain sum:

merged_score = sum(score * weight for score, weight in contributions)

A non-finite input score therefore propagates instead of being neutralised:

  • NaN poisons the merged score (and, because comparisons with NaN are false, the document keeps its position - sorting becomes insertion-order dependent);
  • +inf dominates every real document even with a weight of 0.1;
  • -inf vetoes the document entirely (its merged score is -inf regardless of the other backend).

Scores like these arrive from remote embedding/reranking backends and from user-supplied Document(score=...); they are not scores, and the joiner's contract is to merge comparable relevance values.

before: merged score of the NaN document is NaN; 'non-finite' sorted before 'finite' (0.5)
        +inf -> inf, -inf -> -inf
after : non-finite contributions count as 0; finite documents keep their exact merged scores and order

Fix

Weight a non-finite contribution as zero (0.0 if not math.isfinite(score) else score * weight); finite inputs are unchanged.

Tests

test/components/joiners/test_document_joiner.py (NaN, +inf, -inf with weights):

pytest test/components/joiners -q    # before: 3 failed, 121 passed   after: 124 passed

@DZH166
DZH166 requested a review from a team as a code owner November 18, 2026 14:57
@DZH166
DZH166 requested review from anakin87 and removed request for a team November 18, 2026 14:57
@vercel

vercel Bot commented Nov 18, 2026

Copy link
Copy Markdown
Contributor

@DZH166 is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@HaystackBot

Copy link
Copy Markdown
Contributor

Hi @DZH166, thanks a lot for your contribution! 🙏

We noticed that the Contributor License Agreement (CLA) check (license/cla) hasn't passed yet, so we've temporarily moved this PR to draft and paused the review assignment.

To get your PR reviewed, please sign the CLA via the link in the license/cla check below (or in the CLA bot comment). As soon as the check turns green, this PR will automatically be marked ready for review again and a reviewer will be re-assigned.

@HaystackBot
HaystackBot removed the request for review from anakin87 November 18, 2026 16:21
@HaystackBot HaystackBot added the cla-pending PR is in draft until the contributor signs the CLA label Nov 18, 2026
@HaystackBot
HaystackBot marked this pull request as draft November 18, 2026 16:21
@HaystackBot

Copy link
Copy Markdown
Contributor

Hi @DZH166, this PR is still waiting for the Contributor License Agreement (CLA) to be signed. Please note that if the CLA isn't signed within the next 24 hours, this PR will be automatically closed. Signing only takes a minute via the link in the license/cla check, and the PR will then automatically be marked ready for review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-pending PR is in draft until the contributor signs the CLA topic:tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants