Update multi-sample metagenomic binning workflow

workflows/microbiome/multi-sample-binning/CHANGELOG.md

@@ -1,5 +1,22 @@
 # Changelog
 
+## [1.1] - 2026-10-24
+
+- Incorporate the COMEBin 1.1.0+galaxy1 upgrade from IWC PR #1435.
+- Replace read interlacing/de-interlacing with native Flatten, Nested Cross Product and Apply Rules operations. Preserve the assembly × read-sample BAM matrix while reusing the original read datasets instead of writing intermediate FASTQ files.
+- Replace manual CONCOCT read length with automatic estimation from both mates of all original samples.
+- Make all five binners independently optional and enabled by default, with editor-compatible optional-output bridges.
+- Enclose workflow tools in a native frame with Markdown documentation.
+- Make Binette the first and default bin-refinement choice.
+- Expose Binette minimum completeness and contamination weight while preserving the standalone defaults (40 and 2).
+- Publish coverage depth matrices for inspection and regression testing.
+- Document the absence of an environment selector in SemiBin co mode.
+- Disable COMEBin in the CI tests, because it runs the longest of all binners and keeps the test suite close to the CI time limit. The other binners and all refinement outputs remain covered, and the workflow still enables COMEBin by default.
+- Reduce the CI test fixture to two mock samples, so the cross-sample alignment matrix stays small and the test runs well within the time limit. Coverage-matrix assertions are updated to the two-sample column count.
+- Replace the binner toggles with explicit `Yes`/`No` selections (default: `Yes`), converted to the boolean that gates each binner branch.
+- Order the inputs as assemblies, reads, binners, refinement and Binette settings, and group the tools in frames for inputs, coverage, binning and refinement, each with its own explanation.
+- Describe what each step does in its annotation instead of repeating the step label, and document every step in its own README section.
+
 ## [1.0] - 2026-09-23
 
 - First release

workflows/microbiome/multi-sample-binning/README.md

@@ -1,67 +1,92 @@
 # Multi-sample binning workflow
 
-This workflow uses paired-end reads and their corresponding assemblies, generated by any assembler, to produce metagenomic bins using multiple binning tools. Each binner uses a different approach to identify and group contigs into bins.
+This workflow turns metagenomic assemblies into metagenomic bins with five binners (CONCOCT, MaxBin2, MetaBAT2, SemiBin and COMEBin), using cross-sample read coverage, and refines the bins with DAS Tool and/or Binette.
 
-In contrast to single-sample binning, the multi-sample approach uses information across multiple samples during binning. Reads from each sample are mapped against the assemblies from all samples to generate cross-sample coverage information. This provides the binners with additional information that can help distinguish contigs based on their abundance patterns across samples.
+It performs cross-sample coverage estimation, binning and bin refinement for the [MAGs generation workflow](https://iwc.galaxyproject.org/workflow/mags-building-main/).
+A dedicated [learning pathway](https://galaxyproject.github.io/training-material/learning-pathways/mags.html) provides detailed guidance on MAG generation in Galaxy, including the binning tools used here.
 
-In this workflow, each sample's reads are mapped against each sample's assembly, and each assembly is subsequently binned separately using the cross-sample coverage information. This differs from multi-sample approaches that concatenate assemblies and perform a joint binning step before splitting the resulting bins by sample.
+## How multi-sample binning works
 
-The multi-sample approach is more computationally demanding than single-sample binning because it requires additional read mappings and processes a larger amount of data. When the reads of N samples are mapped against N assemblies, the number of mapping jobs, the number of BAM files and the number of intermediate read copies all scale as N², so storage requirements grow quickly with the number of samples. However, the resulting cross-sample coverage information can improve binning, particularly when organisms show different relative abundances across samples. For a large number of samples, or for samples from unrelated environments where cross-sample coverage adds little information, single-sample binning is the cheaper choice.
+In single-sample binning, each assembly is binned with the coverage of its own reads. In multi-sample binning, the reads of every sample are mapped against the assembly of every sample, so each contig gets one coverage profile per sample. Binners can then separate organisms that have similar composition but different abundance patterns across samples.
 
-After binning is complete, the workflow uses one or both of the available refinement tools to combine and refine the results from the individual binners, with the goal of generating higher-quality bins.
+Two details are worth knowing before starting:
 
-This workflow is the binning section, with the added multi-sample logic, of the [MAGs generation workflow](../mags-building/), extracted so that MAGs generation and other workflows can reuse it instead of maintaining their own copy (check the [FAIRyMAGs repository](https://github.com/usegalaxy-eu/FAIRyMAGs) for the complete workflow suite and its capabilities).
-A dedicated [learning pathway](https://galaxyproject.github.io/training-material/learning-pathways/mags.html) provides detailed guidance on MAG generation in Galaxy, including the binning tools used here.
+- Each sample's reads are mapped against each sample's assembly, and every assembly is binned on its own. The workflow does not concatenate all assemblies into one co-assembly and split the resulting bins afterwards.
+- The mapping workload grows as N² for N samples: N read sets × N assemblies. Storage grows quickly too, even though the workflow reuses the original FASTQ datasets instead of writing intermediate read files.
+
+**Note on sample count:** Multi-sample binning is most effective with up to ~20 samples. Performance usually peaks around 20 samples ([Mineeva et al., *Nat. Commun.* 2025](https://www.nature.com/articles/s41467-026-71521-w)). Using more than 20 samples can lead to extensive resource usage (N² mappings and substantial storage requirements). For many samples or samples from unrelated environments where cross-sample coverage adds little, [single-sample binning](https://iwc.galaxyproject.org/workflow/single-sample-binning-main/) is often the cheaper choice.
 
 ## Inputs
 
-- **Paired-end reads**: a `list:paired` collection with one element per sample.
-  - Reads should be quality-trimmed and host-removed, e.g. with the IWC [short-read quality control](../../read-preprocessing/short-read-qc-trimming/) and [host contamination removal](../host-contamination-removal/host-contamination-removal-short-reads/) workflows.
-  - The sample names are used as coverage columns for the binners.
 - **Assemblies**: a `list` collection of assemblies from any assembler (e.g. MEGAHIT or metaSPAdes).
-  - Every sample's reads are mapped against every assembly, so the element identifiers of the two collections do not need to match. The number of assemblies does not need to equal the number of read samples either: a single co-assembly binned with the coverage of all samples is a valid input.
-- **Run COMEBin?** (default: yes): COMEBin is a deep-learning binner that often improves the results, but it is the most time- and resource-consuming step of the workflow.
-- **Read length (CONCOCT)** (default: 100): the mean read length, required by CONCOCT to calculate contig coverage from the mapped reads. It can be estimated with FastQC.
-- **Bin refinement tool**: `DAS Tool`, `Binette` or `Both`.
+  - Every sample's reads are mapped against every assembly, so the element identifiers of the reads and assemblies do not need to match.
+  - The number of assemblies does not need to equal the number of read samples: a single co-assembly binned with the coverage of all samples is a valid input.
+- **Paired-end reads**: a `list:paired` collection with one element per sample.
+  - Reads should be quality-trimmed and host-removed, e.g. with the IWC [short-read quality control](https://iwc.galaxyproject.org/workflow/short-read-qc-trimming-main/) and [host contamination removal](https://iwc.galaxyproject.org/workflow/host-contamination-removal-short-reads-main/) workflows.
+  - The sample names become the coverage columns of the binners.
+  - **Use with caution for large datasets:** multi-sample binning is most effective with up to ~20 samples. Performance usually peaks around 20 samples ([Mineeva et al., *Nat. Commun.* 2025](https://www.nature.com/articles/s41467-026-71521-w)). Using more than 20 samples can lead to extensive resource usage, as the mapping workload grows as N² (N read sets × N assemblies) and storage requirements increase substantially. For many samples or unrelated environments, [single-sample binning](https://iwc.galaxyproject.org/workflow/single-sample-binning-main/) is often the cheaper choice.
+- **Run COMEBin?**, **Run SemiBin?**, **Run CONCOCT?**, **Run MetaBAT2?** and **Run MaxBin2?** (default: `Yes`): enable or skip each binner independently.
+  - Keep at least one binner enabled.
+  - COMEBin is the most time- and resource-consuming binner.
+- **Bin refinement tool** (default: `Binette`): `Binette`, `DAS Tool` or `Both`.
 - **CheckM2 Database**: the CheckM2 reference database Binette uses to assess bin quality.
+- **Minimum MAG completeness percentage** (default: 40): the minimum bin completeness used by Binette. It does not filter DAS Tool output.
+- **Contamination weight (Binette)** (default: 2): the contamination penalty used in Binette's scoring.
 
-## Outputs
+## Step 1: Coverage
 
-- **CONCOCT bins**, **MaxBin2 bins**, **MetaBAT2 bins**, **SemiBin bins** and **COMEBin bins** (if COMEBin was run): one list of bins per assembly.
-- **DAS Tool bins**, **DAS Tool summary** and **DAS Tool contigs2bin report** (if DAS Tool was run): the refined bins, a per-bin summary (completeness and redundancy based on single-copy genes) and the contig-to-bin assignments.
-- **Binette bins**, **Binette input bin quality reports** and **Binette final bin quality report** (if Binette was run): the refined bins, the CheckM2 quality of the bins from each binner, and the CheckM2 quality of the final bins.
+This step produces the coverage information every binner needs.
 
-## Why use this workflow
+1. The read pairs of all samples are flattened into one flat list of files, and the assemblies are used as they are.
+2. A nested cross product pairs each assembly with every flattened read file.
+3. Apply Rules rebuilds, for each assembly, the paired reads of each sample and the matching assembly reference. These are collection operations on the original datasets: no interlaced or de-interlaced FASTQ file is written.
+4. Bowtie2 maps every sample against every assembly.
+5. Contig depths are summarised per assembly with `jgi_summarize_bam_contig_depths`, which yields one coverage matrix per assembly with the coverage and variance of every sample.
+6. CONCOCT additionally needs the coverage of its contig fragments, which is computed separately, and the mean read length, which is estimated from both mates of every sample with SeqKit statistics. The estimate is the read-count-weighted mean (total bases divided by total reads), not an average of the per-sample means. No manual read length is required.
 
-This workflow provides a standardized and modular workflow for metagenomic binning that can be integrated into larger metagenomic analysis workflows. Its main purpose is to separate the binning step from other parts of MAG analysis, making complex workflows easier to understand, maintain, and adapt.
+The depth table produced by `jgi_summarize_bam_contig_depths` starts with `contigName` (contig identifier), `contigLen` (contig length) and `totalAvgDepth` (average depth across the input BAM files). Each read sample then contributes a depth column and a variance column. This gives each assembly's contigs a coverage profile across all read samples; “JGI depth table” refers to this format, not to an additional input database.
 
-Using binning as an independent workflow provides better control over the inputs, binning parameters, and outputs. It also allows users to reuse the same binning strategy in different analysis pipelines without having to include unnecessary upstream or downstream steps. This modular design is particularly useful when the generated bins are intended for a specific downstream analysis, such as taxonomic classification, genome annotation, functional analysis, or benchmarking.
+## Step 2: Binning
 
-In contrast, the IWC Metagenome-Assembled Genomes (MAGs) generation workflow (https://iwc.galaxyproject.org/workflow/mags-building-main/) is designed as a comprehensive end-to-end solution. It performs metagenome assembly and multi-tool binning of paired short reads and optional long reads, followed by dereplication and analysis of MAG quality and abundance. This makes it well suited for users who want to generate and evaluate MAGs without requiring a specialized downstream workflow.
+Each assembly is binned separately by every enabled binner.
 
-This binning workflow should therefore be preferred when binning needs to be incorporated as one component of a larger or more specialized analysis. It reduces workflow complexity, improves the overview of individual analysis steps, and makes it easier to modify or replace downstream analyses independently.
+| Binner | Inputs used |
+| --- | --- |
+| SemiBin | Contig composition and cross-sample coverage, in co-binning mode |
+| CONCOCT | Contig fragments, their coverage table and the estimated read length; fragment clusters are merged back to contigs |
+| MetaBAT2 | Contig composition and cross-sample coverage |
+| MaxBin2 | Contig composition, cross-sample coverage and marker genes |
+| COMEBin | Deep-learning binning from contig sequence and coverage; optional |
 
-It should also be noted that this workflow can restrict Upstream. For example, it is possible to restrict the workflow so that the User can only use one of the two refinement tools or can restrict it to only one specific refinement tool.
+A binner set to `No` is skipped, and its candidate results are replaced by empty collections so that the refinement steps keep their structure. Raw binner outputs are only available for binners that ran.
 
-## Workflow logic
+## Step 3: Refinement
 
-- Preprocess reads and assemblies for binning.
-- Generate cross-sample coverage information:
-
-  - Map the reads from each sample against the assemblies from all samples.
+The candidate bins are refined with one or both refinement tools:
+
+- **DAS Tool** scores the contig-to-bin assignments of all enabled binners and selects a non-redundant set of bins.
+- **Binette** evaluates the candidate FASTA bins with CheckM2 and refines them with the selected completeness and contamination settings.
+
+Selecting **Both** runs DAS Tool and Binette independently on the candidate bins, allowing you to benefit from their different refinement approaches. The two output sets must then be combined and dereplicated downstream; this workflow does not perform dereplication. Dereplication reduces redundant genomes but does not enforce unique contig membership, so contigs can still end up in multiple retained bins. If exclusive contig assignments are required, inspect and resolve overlaps explicitly.
+
+## Outputs
+
+- **Coverage depth matrices**: one depth table per assembly, with the coverage of all original samples.
+- **CONCOCT bins**, **MaxBin2 bins**, **MetaBAT2 bins**, **SemiBin bins** and **COMEBin bins** (when COMEBin ran): one list of bins per assembly.
+- **DAS Tool bins**, **DAS Tool summary** and **DAS Tool contigs2bin report** (when DAS Tool ran): the refined bins, a per-bin summary of completeness and redundancy based on single-copy genes, and the contig-to-bin assignments.
+- **Binette bins**, **Binette input bin quality reports** and **Binette final bin quality report** (when Binette ran): the refined bins, the CheckM2 quality of the bins from each binner, and the CheckM2 quality of the final bins.
+
+## Why use this workflow
 
-- Perform binning using multiple binners:
+Binning is a separate workflow here, so it can be reused by larger analysis pipelines without copying or rewiring it.
 
-  - CONCOCT
-  - MaxBin2
-  - MetaBAT2
-  - SemiBin
-  - COMEBin
+- It keeps the binning parameters and outputs visible and controllable.
+- It can be combined with different upstream and downstream workflows, e.g. taxonomic classification, genome annotation, functional analysis or benchmarking.
+- It avoids duplicating binning logic inside every pipeline that needs it.
 
-- Refine the resulting bins using one or both refinement tools:
+The IWC [MAGs generation workflow](https://iwc.galaxyproject.org/workflow/mags-building-main/) is the comprehensive alternative: it assembles reads, bins with multiple tools, dereplicates the MAGs and reports their quality and abundance in one workflow. Use it when you want MAGs and their evaluation without a specialised downstream workflow, and use this binning workflow when binning is one component of a larger analysis.
 
-  - DAS Tool
-  - Binette
+Through the workflow editor, an upstream or downstream workflow can also restrict this one, for example to a single refinement tool.
 
 ## Funding