Container image management

container-image-management

Summary

Deploy and manage containerized bioinformatics workflows using Docker or Singularity runtimes to ensure reproducible LC-HRMS metabolomics analysis across heterogeneous compute environments. This skill enables portable execution of MS-DIAL and MSFLO pipelines without dependency resolution or system-specific compilation.

When to use

Your LC-HRMS metabolomics data (.mzML or .abf files) must be processed reproducibly across multiple machines (local workstations, HPCs, cloud) without manual tool installation, or when you need to enforce identical computational environments for peer review and long-term archival.

When NOT to use

• Your input data is already a processed feature table (.csv, .tsv) — container deployment is unnecessary for downstream statistical analysis.
• You require real-time interactive parameter tuning within the MS-DIAL GUI; containerized workflows enforce static configurations.
• Your compute environment prohibits container runtimes entirely (neither Docker nor Singularity available); fall back to native tool installation.

Inputs

• .mzML or .abf raw mass spectrometry data files
• MS-DIAL parameter configuration file (msdial_params.txt)
• MS-FLO parameter configuration file (msflo_params.ini)
• MS1 spectral library (ms1_lib.txt)
• MS2 spectral library (ms2_lib.msp)

Outputs

• Aligned feature table (.tsv / .msdial)
• Annotated metabolite features (post-MSFLO)
• Execution report (execution_report.html)
• Execution timeline (execution_timeline.html)
• Process-level logs (logs/execution.log)

How to apply

Select your target execution environment: Docker for local macOS/Linux machines with direct container runtime access, or Singularity for HPC systems (e.g., SLURM-managed clusters) where unprivileged container execution is required. Install the Nextflow workflow (≥22.10.0) and the corresponding container runtime. Prepare your .mzML input files in the data/raw_data/ directory, along with MS-DIAL configuration (msdial_params.txt), MS-FLO configuration (msflo_params.ini), and reference spectral libraries (ms1_lib.txt, ms2_lib.msp). Invoke the Nextflow pipeline with the -profile docker or -profile singularity flag; Nextflow will automatically pull, build, and execute the MS-DIAL and MSFLO container images, routing aligned features through both stages and collecting the final feature table (.tsv) output. Validate that container layer caching and process parallelization completed without errors by inspecting the HTML execution reports (execution_report.html, execution_timeline.html) and confirming output file schemas match expected metabolomics feature tables.

Related tools

• Nextflow (Workflow orchestration engine that manages container lifecycle, process parallelization, and resource scheduling across Docker/Singularity runtimes) — https://www.nextflow.io/
• Docker (Container runtime for local execution on macOS and Linux workstations; provides isolated, reproducible MS-DIAL and MSFLO execution environments) — https://docs.docker.com/engine/installation/
• Singularity (Container runtime for HPC systems; enables unprivileged container execution on SLURM-managed clusters and shared computing facilities) — https://www.sylabs.io/guides/3.0/user-guide/
• MS-DIAL (Containerized peak detection and feature alignment engine for LC-HRMS metabolomics data; first stage of the two-stage pipeline) — https://github.com/Nextflow4Metabolomics/nextflow4ms-dial
• MSFLO (Containerized annotation and statistical processing module; second stage of the pipeline that processes MS-DIAL-aligned features) — https://github.com/Nextflow4Metabolomics/nextflow4ms-dial

Examples

nextflow run main.nf -profile docker > logs/execution.log

Evaluation signals

• Container image pull and build logs appear in execution.log with no download/layer resolution errors.
• All processes complete with exit code 0 and produce non-empty .tsv feature tables in the results directory.
• execution_report.html and execution_timeline.html show expected process DAG (MS-DIAL → MSFLO) without task retries or failures.
• Output feature table schema matches expected metabolomics format: rows = features, columns = samples + m/z + RT + annotation metadata.
• Reproducibility check: identical input data + configuration produces identical output feature tables across separate Nextflow runs (checksums match).

Limitations

• Container image builds may require 30–60 min on first execution; subsequent runs benefit from layer caching.
• File name special characters (other than underscores) in input .mzML files cause silent process failures; strict naming convention required.
• SLURM CPU overallocation errors occur if --max_cpus (not --cpus) is not set correctly in the HiPerGator.config; per-process resource requests must not exceed cluster allocation.
• Only .mzML and .abf input formats supported; other raw MS formats (e.g., .raw, .d) require pre-conversion with ProteoWizard msConvert or Reifycs Abf Converter.
• Reference spectral libraries (ms1_lib.txt, ms2_lib.msp) must be present and correctly formatted; missing or malformed libraries cause downstream annotation to fail silently.

Evidence

• [readme] Both Docker and Singularity (for high-performance computing) are supported: "Both Docker and Singularity (for high-performance computing) are supported"
• [readme] Nextflow to run MS-DIAL-based analyses reproducibly across local computers, containers, and high-performance computing environments: "It uses Nextflow to run MS-DIAL-based analyses reproducibly across local computers, containers, and high-performance computing environments."
• [other] containerized workflow MS-DIAL -> MSFLO: "containerized workflow MS-DIAL -> MSFLO"
• [readme] Remove the example data and place your raw data files in data/raw_data/. The workflow accepts .mzML and .abf files.: "Remove the example data and place your raw data files in data/raw_data/. The workflow accepts .mzML and .abf files."
• [readme] Docker configuration is defined in conf/base.config. High-performance computing and Singularity configuration is defined in conf/HiPerGator.config.: "Docker configuration is defined in conf/base.config. High-performance computing and Singularity configuration is defined in conf/HiPerGator.config."
• [readme] To avoid unexpected errors, do not use special characters in file names. Underscores are safe to use.: "To avoid unexpected errors, do not use special characters in file names. Underscores are safe to use."
• [readme] Use --max_cpus, not --cpus, in the configuration file to define the CPUs available to each process.: "Use --max_cpus, not --cpus, in the configuration file to define the CPUs available to each process."
• [readme] Add the MS-DIAL and MS-FLO configuration files to the data/ folder and name them msdial_params.txt and msflo_params.ini.: "Add the MS-DIAL and MS-FLO configuration files to the data/ folder and name them msdial_params.txt and msflo_params.ini."