deposition: correctly handle case where only raw reads are updated not consensus sequence

[anna-parker]

[P2] Replacing only the raw reads leaves the ENA assembly linked to the old run

I looked into the revision path and reproduced an edge case here. RUN_REF now comes from raw_reads_table, but has_assembly_data_changed still compares the consensus, flat-file metadata, and mapped fields in seq_metadata. It doesn't see changes to the generated run accession.

For example:

1. Version 1 deposits reads as ERR111 and an assembly referencing ERR111.
2. Version 2 replaces the read files but keeps the consensus and assembly metadata unchanged, producing ERR222.
3. Assembly creation retrieves ERR222 here, then takes the unchanged-assembly shortcut at lines 525–528 and copies the previous assembly result without submitting an updated manifest.

Loculus consequently reports the new run, while the ENA assembly remains linked to ERR111. The raw-read workflow can also request suppression of that old run. Adding reads to an existing assembly without otherwise changing it has the same missing-link-update problem.

I reproduced the shortcut using the actual PR functions with database calls mocked: the new run was present, but the previous assembly was copied and no new manifest was generated. The existing raw-read revision test misses this because get_revisions defaults to modify_assembly=True, so it changes assembly metadata too.

It would be good to compare the previous and current effective run references before reusing an assembly result, and add a test with modify_raw_reads=True, modify_assembly=False. If updating the link needs manual handling, that could be flagged explicitly instead.

This could be turned into a follow-up issue if you'd prefer to handle it separately.

Originally posted by @theosanderson-agent in #6816 (comment)

[claude]

This issue is related to ongoing work in the raw reads revision area. Similar or related issues:

• feat(deposition): Handle raw reads revision on ENA #6877 - General raw reads revision handling on ENA
• deposition: Unchanged revision copies previous version's assembly result without checking it succeeded #6985 - Unchanged revision copies previous assembly result without checking success
• ENA deposition: carry forward stable INSDC external metadata across revisions; keep versioned accession revision-specific #6804 - Carrying forward INSDC external metadata across revisions

While each addresses different aspects of the revision workflow, they're all in the same problem space and may benefit from coordinated fixes.

[theosanderson-agent]

I've opened a draft PR with a fix and tests: #7246

It targets the raw_reads_ena branch of #6816, since the RUN_REF-from-raw_reads_table code only exists there.

Summary of the change:

• get_run_ref / run_ref_diff look up the effective RUN_REF for the previous version in raw_reads_table and compare it with the one about to go into the manifest.
• has_assembly_data_changed returns True when the RUN_REF differs (replaced reads ERR111 → ERR222, or reads added None → ERRxxx), so the assembly is resubmitted with the new link instead of copying the previous result.
• With allow_revision_with_manifest_changes=False, a changed RUN_REF is flagged as HAS_ERRORS for manual revision, like other manifest field changes.
• Unit tests with mocked DB calls (passing locally), plus the suggested integration test with modify_raw_reads=True, modify_assembly=False which asserts a new manifest with the new run accession is submitted. I couldn't run the integration test locally (needs ENA dev credentials + Postgres), so CI will need to confirm it.

[theosanderson]

Ah lol, this post automatically tagged the agent triggering it to go to work

[anna-parker]

@theosanderson no worries! the code actually looks pretty decent - also the integration test looks good!