Update single-sample metagenomic binning workflow by paulzierep · Pull Request #1453 · galaxyproject/iwc

paulzierep · 2026-10-20T09:23:27Z

Summary

Publish Single-sample binning 1.0 as a reusable workflow accepting paired-end
reads and corresponding assemblies from any assembler. Each sample's reads are
mapped only against its own assembly. For a pre-grouped/co-assembled input, this
means mapping the group's merged reads against its assembly.

This is the lower-cost coverage strategy: it avoids the all-samples × all-assemblies
mapping used by #1452. Prefer it for unrelated samples, large sample counts, or
limited storage/compute; use multi-coverage when cross-sample abundance patterns
are useful and resources permit.

Changes and interface

Inputs: list:paired reads and a list of assemblies, aligned by position and
in the same order. Reads should already be quality-filtered and host-removed.
Five independently optional binners: SemiBin, CONCOCT, MetaBAT2, MaxBin2 and
COMEBin. All default to on; keep at least one enabled.
Refine with Binette (first/default), DAS Tool, or Both. Binette's
standalone completeness/contamination-weight defaults remain 40 / 2.
Expose SemiBin's supported single-mode environment selector (default global).
Compute CONCOCT read length automatically from SeqKit statistics across both
mates: total bases / total reads, rounded to the nearest integer. This is a
read-count-weighted mean, not an average of per-file means.
Publish coverage-depth matrices, raw binner bins and selected refiner outputs.
Use editor-compatible optional-output bridges and separate coverage, binning and refinement frames
with Markdown documentation. Yes/No binner selectors feed internal boolean gates to avoid invalid
text-to-boolean condition connections in the editor.

This standalone workflow stops at binning/refinement: grouping, assembly, dRep,
abundance estimation and final reporting belong to the parent, not this PR.

Validation

Test fixtures/assertions cover Binette with COMEBin enabled and DAS Tool
with COMEBin disabled. Fixtures are referenced by URL; collection minima are
retained rather than weakened.
gxwf validate: 36 tool states pass, zero failures/skips.
Imported on usegalaxy.eu and included in the actual Galaxy editor-terminal
audit: 457 connections checked across the three deliverables and embedded
workflows, zero invalid connections.
The latest default/presentation follow-up preserves the accepted computational
graph and exact test assertions. Fresh default-on EU checks and GitHub CI are
tracked separately; pending runs are not claimed as passes.

Context and related PRs

This is part of the FAIRyMAGs workflow modularization: publish reusable binning
workflows alongside an end-to-end MAG-generation parent, rather than maintaining
separate copies of the same binning logic. Each deliverable has its own branch
and PR so it can be reviewed independently.

Update single-sample metagenomic binning workflow #1453 — single-sample / single-coverage binning
Update multi-sample metagenomic binning workflow #1452 — multi-sample / multi-coverage binning
Modularize MAG generation with read grouping and selectable binning strategies #1454 — modular end-to-end MAG generation

Suggested review/merge order: Single → Multi → Parent. The parent embeds its
children, so importing it does not require importing the standalone workflows first.

Developed within FAIRyMAGs, with links
to the related workflows and learning resources in the README.

IWC checklist

FOR CONTRIBUTOR:

I have read the Adding workflows guidelines
I have added myself to the CODEOWNERS File for the workflow
License permits unrestricted use (educational + commercial)
Please also take note of the reviewer guidelines below to facilitate a smooth review process.

FOR REVIEWERS:

.dockstore.yml: file is present and aligned with creator metadata in workflow. ORCID identifiers are strongly encouraged in creator metadata. The .dockstore.yml file is required to run tests
Workflow is sufficiently generic to be used with lab data and does not hardcode sample names, reference data and can be run without reading an accompanying tutorial.
In workflow: annotation field contains short description of what the workflow does. Should start with This workflow does/runs/performs … xyz … to generate/analyze/etc …
In workflow: workflow inputs and outputs have human readable names (spaces are fine, no underscore, dash only where spelling dictates it), no abbreviation unless it is generally understood. Altering input or output labels requires adjusting these labels in the the workflow-tests.yml file as well
In workflow: name field should be human readable (spaces are fine, no underscore, dash only where spelling dictates it), no abbreviation unless generally understood
Workflow folder: prefer dash (-) over underscore (_), prefer all lowercase. Folder becomes repository in iwc-workflows organization and is included in TRS id
Readme explains what workflow does, what are valid inputs and what outputs users can expect. If a tutorial or other resources exist they can be linked. If a similar workflow exists in IWC readme should explain differences with existing workflow and when one might prefer one workflow over another
Changelog contains appropriate entries
Large files (> 100 KB) are uploaded to zenodo and location urls are used in test file

github-actions · 2026-10-20T10:30:53Z

Test Results (powered by Planemo)

Test Summary

Test State	Count
Total	2
Passed	0
Error	2
Failure	0
Skipped	0

Errored Tests

❌ single-sample-binning.ga_0

Execution Problem:

Unexpected HTTP status code: 400: {"err_msg":"Workflow was not invoked; the following required tools are not installed: toolshed.g2.bx.psu.edu/repos/iuc/seqkit_stats/seqkit_stats/2.8.1+galaxy0 (version 2.8.1+galaxy0), toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_cat/0.1.0 (version 0.1.0), toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_awk_tool/9.11+galaxy0 (version 9.11+galaxy0)","err_code":0}

❌ single-sample-binning.ga_1

Execution Problem:

Unexpected HTTP status code: 400: {"err_msg":"Workflow was not invoked; the following required tools are not installed: toolshed.g2.bx.psu.edu/repos/iuc/seqkit_stats/seqkit_stats/2.8.1+galaxy0 (version 2.8.1+galaxy0), toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_cat/0.1.0 (version 0.1.0), toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_awk_tool/9.11+galaxy0 (version 9.11+galaxy0)","err_code":0}

github-actions · 2026-10-20T11:38:40Z

Test Results (powered by Planemo)

Test Summary

Test State	Count
Total	2
Passed	0
Error	2
Failure	0
Skipped	0

Errored Tests

❌ single-sample-binning.ga_0

Execution Problem:

Unexpected HTTP status code: 400: {"err_msg":"Workflow was not invoked; the following required tools are not installed: toolshed.g2.bx.psu.edu/repos/iuc/seqkit_stats/seqkit_stats/2.8.1+galaxy0 (version 2.8.1+galaxy0), toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_cat/0.1.0 (version 0.1.0), toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_awk_tool/9.11+galaxy0 (version 9.11+galaxy0)","err_code":0}

❌ single-sample-binning.ga_1

Execution Problem:

Unexpected HTTP status code: 400: {"err_msg":"Workflow was not invoked; the following required tools are not installed: toolshed.g2.bx.psu.edu/repos/iuc/seqkit_stats/seqkit_stats/2.8.1+galaxy0 (version 2.8.1+galaxy0), toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_cat/0.1.0 (version 0.1.0), toolshed.g2.bx.psu.edu/repos/bgruening/text_processing/tp_awk_tool/9.11+galaxy0 (version 9.11+galaxy0)","err_code":0}

COMEBin runs the longest of all binners and the workflow tests stay close to the CI time limit. Turn it off in the published test cases while keeping the other binners, both refinement tools and every existing assertion unchanged. COMEBin stays enabled by default in the workflow.

paulzierep · 2026-10-21T07:38:36Z

Tests: turn COMEBin off in the published test case to keep the suite inside the CI time limit.

COMEBin is the longest binning step: in the last CI runs its five assembly jobs took ~3h20m here (~4h30m for multi-sample), which brought the suite close to the six hour limit. With COMEBin disabled the test now covers SemiBin, MetaBAT2, MaxBin2 and CONCOCT plus the Binette and DAS Tool refinement paths, and all existing output assertions are unchanged. COMEBin stays enabled by default in the workflow and its behaviour is verified separately.

github-actions · 2026-10-21T11:34:52Z

Test Results (powered by Planemo)

Test Summary

Test State	Count
Total	2
Passed	1
Error	0
Failure	1
Skipped	0

Failed Tests

❌ single-sample-binning.ga_0

Passed Tests

✅ single-sample-binning.ga_1

github-actions · 2026-10-21T14:42:59Z

Test Results (powered by Planemo)

Test Summary

Test State	Count
Total	2
Passed	1
Error	0
Failure	1
Skipped	0

Failed Tests

❌ single-sample-binning.ga_0

Passed Tests

✅ single-sample-binning.ga_1

bebatut

Similar comments would apply as in #1452:

Removing in annotation the text that is already in the label of the step
Changing the order in the inputs
Having sub-boxing corresponding to "coverage computation", "binning", "refinement" steps with the corresponding explanation in each box

In addition to that (that would be also valid for #1452), it would be nice to have the different steps as sections in the README with then more details in each. The README is the first thing users will see when they are looking for workflows, so we need to make the different steps stand out, especially in workflows that can be quite complex.

Replace the five boolean binner toggles with Yes/No selectors gated through map_param_value steps, reorder the inputs so reads and assemblies come first, group the workflow into annotated frames, and rewrite step annotations and the README to explain what each step does.

paulzierep · 2026-10-22T18:41:31Z

Thanks @bebatut. Applied the same improvements here: shorter explanatory annotations, assemblies and reads before the binner/Binette controls, Yes/No binner selectors, and separate coverage, binning and refinement frames. The README now introduces each analysis stage before its settings and outputs.

bebatut

Thanks @paulzierep. It seems good on my side (do not have the time to review in details 😅 ), just one comment about changes in the CHANGELOG file

paulzierep · 2026-11-02T14:58:06Z

Please hold this PR until galaxyproject/galaxy#24665 is resolved. This workflow contains or embeds mapped pick_value bridges that may encounter the reported duplicate-scheduling bug when job caching is enabled.

A minimal two-step example on EU reproduced the problem: 1,001 input datasets generated 3,000 jobs before cancellation, repeatedly scheduling the same inputs. Caching is a suspected factor, not yet a confirmed cause.

paulzierep · 2026-11-02T16:03:05Z

Addressed the inline comments in 7eab17b32d: changed “Publish” to “Expose” in the CHANGELOG, simplified the introduction to describe the workflow's function, and replaced the relative workflow links with IWC website links.

The README now explains that running DAS Tool and Binette together benefits from their different refinement approaches, requires downstream dereplication of the combined results, and can still leave contigs in multiple retained bins because dereplication does not enforce exclusive contig assignments. The embedded README is synchronized.

bebatut

Thanks a lot @paulzierep. It looks good to me!
Waiting now for galaxyproject/galaxy#24665

paulzierep and others added 4 commits October 19, 2026 21:00

Expose standalone single-binning refinement and SemiBin controls

e53ff48

Derive CONCOCT read length and expose default-on binner controls

6347b0a

Fold pending single-sample binning changes into the first 1.0 release

edc8251

Drop development-process note from the single-sample binning submission

628b1be

paulzierep requested a review from SantaMcCloud as a code owner October 20, 2026 09:23

paulzierep mentioned this pull request Oct 20, 2026

Modularize MAG generation with read grouping and selectable binning strategies #1454

Open

4 of 13 tasks

Default single-sample bin refinement to Binette

eb0805c

paulzierep mentioned this pull request Oct 20, 2026

Update multi-sample metagenomic binning workflow #1452

Merged

4 of 13 tasks

paulzierep changed the title ~~Agentic foundry mags single~~ Add reusable single-sample metagenomic binning workflow Oct 20, 2026

paulzierep changed the title ~~Add reusable single-sample metagenomic binning workflow~~ Update single-sample metagenomic binning workflow Oct 20, 2026

paulzierep added 2 commits October 20, 2026 13:49

Fix Tool Shed installation metadata for automatic read length

e305f78

Clarify workflow step labels, embedded documentation and output tags

b52aec8

Assert empty COMEBin collections when the binner is disabled

4f43bfe

bebatut reviewed Oct 22, 2026

View reviewed changes

bebatut reviewed Oct 23, 2026

View reviewed changes

Comment thread workflows/microbiome/single-sample-binning/CHANGELOG.md Resolved

Comment thread workflows/microbiome/single-sample-binning/CHANGELOG.md Resolved

Publish binning changes as release 1.1 and preserve 1.0 history

12acaae

paulzierep commented Nov 2, 2026

View reviewed changes

Comment thread workflows/microbiome/single-sample-binning/CHANGELOG.md Resolved

Comment thread workflows/microbiome/single-sample-binning/README.md Resolved

Comment thread workflows/microbiome/single-sample-binning/README.md Resolved

Address README review and explain combined refiner caveats

7eab17b

bebatut approved these changes Nov 3, 2026

View reviewed changes

paulzierep merged commit b34c991 into galaxyproject:main Nov 3, 2026
11 checks passed

Conversation

paulzierep commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes and interface

Validation

Context and related PRs

IWC checklist

Uh oh!

github-actions Bot commented Oct 20, 2026

Test Results (powered by Planemo)

Test Summary

Uh oh!

github-actions Bot commented Oct 20, 2026

Test Results (powered by Planemo)

Test Summary

Uh oh!

paulzierep commented Oct 21, 2026

Uh oh!

github-actions Bot commented Oct 21, 2026

Test Results (powered by Planemo)

Test Summary

Uh oh!

github-actions Bot commented Oct 21, 2026

Test Results (powered by Planemo)

Test Summary

Uh oh!

bebatut left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

paulzierep commented Oct 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bebatut left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

paulzierep commented Nov 2, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

paulzierep commented Nov 2, 2026

Uh oh!

bebatut left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

paulzierep commented Oct 20, 2026 •

edited

Loading

bebatut left a comment •

edited

Loading

paulzierep commented Oct 22, 2026 •

edited

Loading