Send accented search terms to arXiv unchanged - #16771
Pranav-error wants to merge 1 commit into
Conversation
callApi stripped accents from the query, with a comment saying the arXiv API had problems with them. That is no longer true, and the workaround now guarantees zero results for any accented search: arXiv returns one match for ti:"slice theorem for Fréchet" and none for the stripped form. The other two stripAccents calls are untouched. One skips accented terms when building the broad fallback query, the other normalises titles for comparison rather than for the request.
PR Summary by QodoPreserve accented terms in arXiv search queries
AI Description
Diagram
High-Level Assessment
Files changed (1)
|
Code Review by Qodo
1. Legacy links in Markdown Javadoc
|
![qodo-free-for-open-source-projects[bot]](https://avatars.githubusercontent.com/in/1068000?s=60&v=4){kind=small}
| /// <br/><br/> | ||
| /// ArXiv-issued DOIs are identifiers associated with every ArXiv entry. They are composed of a fixed | ||
| /// [#DOI_PREFIX] + the entry's ArXiv ID | ||
| /// {@link #DOI_PREFIX} + the entry's ArXiv ID |
There was a problem hiding this comment.
1. Legacy links in markdown javadoc 📘 Rule violation ⚙ Maintainability
The changed /// documentation introduces legacy {@link ...} tags instead of Markdown links. This
directly violates the required Markdown Javadoc syntax.
Agent Prompt
## Issue description
Changed `///` documentation uses legacy `{@link ...}` syntax instead of Markdown Javadoc links.
## Issue Context
Use Markdown links such as `[#DOI_PREFIX]` and `[#getAutomaticDoi(ArXivIdentifier)]` consistently in all four changed documentation lines.
## Fix Focus Areas
- jablib/src/main/java/org/jabref/logic/importer/fetcher/ArXivFetcher.java[175-175]
- jablib/src/main/java/org/jabref/logic/importer/fetcher/ArXivFetcher.java[185-185]
- jablib/src/main/java/org/jabref/logic/importer/fetcher/ArXivFetcher.java[203-203]
- jablib/src/main/java/org/jabref/logic/importer/fetcher/ArXivFetcher.java[222-222]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
| // The arXiv API has problems with accents, so we remove them (i.e. Fréchet -> Frechet) | ||
| if (StringUtil.isNotBlank(searchQuery)) { | ||
| uriBuilder.addParameter("search_query", StringUtil.stripAccents(searchQuery)); | ||
| uriBuilder.addParameter("search_query", searchQuery); |
There was a problem hiding this comment.
2. Accent fix lacks passing test 📘 Rule violation ☼ Reliability
The logic change has no corresponding test change, and the existing accented-title integration test still cannot pass because it compares against a citation key the current path does not return. Consequently, the new request behavior is not protected by a green regression test.
Agent Prompt
## Issue description
The changed arXiv query behavior lacks a corresponding passing regression test.
## Issue Context
The existing live test searches for `Fréchet`, but its full-entry expectation includes `Diez:2013fdp`; the PR description confirms the current result instead receives a DOI-based key. Add or adapt coverage so the accented query behavior is verified by a green, deterministic-enough live endpoint test without weakening unrelated assertions or mocking arXiv.
## Fix Focus Areas
- jablib/src/main/java/org/jabref/logic/importer/fetcher/ArXivFetcher.java[601-602]
- jablib/src/test/java/org/jabref/logic/importer/fetcher/ArXivFetcherTest.java[308-310]
- jablib/src/test/java/org/jabref/logic/importer/fetcher/ArXivFetcherTest.java[110-124]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
| // The arXiv API has problems with accents, so we remove them (i.e. Fréchet -> Frechet) | ||
| if (StringUtil.isNotBlank(searchQuery)) { | ||
| uriBuilder.addParameter("search_query", StringUtil.stripAccents(searchQuery)); | ||
| uriBuilder.addParameter("search_query", searchQuery); |
There was a problem hiding this comment.
3. Accented search fix undocumented 📘 Rule violation ⚙ Maintainability
Passing accented queries unchanged restores user-visible arXiv search results, but this PR adds no entry under Unreleased → Fixed. The missing changelog entry violates the required user-facing release documentation.
Agent Prompt
## Issue description
The user-visible accented arXiv search fix is missing from the unreleased changelog.
## Issue Context
Add one sentence of at most 20 words under `Unreleased` → `Fixed`, beginning with `We fixed` and describing only the user-visible result. Link the relevant issue if one exists; otherwise use the permitted TODO fallback rather than inventing an issue number.
## Fix Focus Areas
- CHANGELOG.md[104-110]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
github-actions
Bot
added
the
status: changes-required
|
Your pull request conflicts with the target branch. Please merge |
|
Fair enough — closing this. The bug and the evidence are in #16772; happy for anyone to take it from there, or I'll reopen a PR written properly. |
|
This pull requests was closed without merging. You have been unassigned from the respective issue #16772. In case you closed the PR for yourself, you can re-open it. Please also check After submission of a pull request in CONTRIBUTING.md. |
Summary
ArXivFetcher.callApistripped accents from the search query, with the comment "The arXiv API has problems with accents, so we remove them (i.e. Fréchet -> Frechet)". That is no longer true, and the workaround now guarantees zero results for any accented search — arXiv matches only the accented form:export.arxiv.orgti:"slice theorem for Fr%C3%A9chet"(UTF-8)ti:"slice theorem for Frechet"(what JabRef sent)Fr%E9chet(Latin-1)URIBuilderalready percent-encodes the parameter as UTF-8, so passing the query through unchanged is enough.The other two
stripAccentscalls in this file are deliberately untouched:buildBroadTitleQueryskips accented terms when assembling the broad fallback query, andnormalizeForTitleMatchingnormalises for comparison rather than for the request.Steps to test
./gradlew :jablib:fetcherTest --tests "*ArXivFetcherTest*"searchEntryByPartOfTitleWithAcuteAccentcurrently fails onmainwithexpected: <[...]> but was: <[]>— the search returns nothing at all. With this change it returns the correct paper.It does not yet make that test green, and I would rather say so than hide it. After this fix the assertion fails on one remaining field, the citation key:
@article{Diez:2013fdp,@article{https://doi.org/10.48550/arxiv.1405.2249,That is a second, independent problem in
infuseWithInspireCitationKeyIfMissing, not something this change introduces — before it, the search returned an empty list, so the key path was never exercised. INSPIRE still serves the expected key (GET /api/arxiv/1405.2249?fields=texkeys→["Diez:2013fdp", "Diez:2014ppa"]), andINSPIREFetcherTestpasses 3/3, so the fetcher itself is healthy — the lookup fromArXivFetcheris what comes back empty, and itsFetcherClientExceptionbranch logs at TRACE, so it fails silently.Happy to chase that in this PR or a separate one, whichever you prefer. I did not want to bundle a guess at it with a fix I can actually demonstrate.
Related issues and pull requests
Closes #16772
AI usage
Claude Code (model
claude-opus-5), used to run the fetcher test task, read the failure report, and check the arXiv and INSPIRE APIs directly. AIL3 — AI-assisted, human-directed and human-owned.Checklist
CHANGELOG.mddescribing the change from the user's point of view (if the change is visible to the user)