C module integration

c++-module-integration

Summary

Integrate a format-specific adapter layer into a format-agnostic C++ API to enable lossless and lossy compression of mass spectrometry data in heterogeneous file formats (mzML, mzXML). This skill bridges the gap between standardized mass spectrometry schemas and a unified internal data representation.

When to use

You have raw mass spectrometry data in mzML or mzXML format and need to compress it using a format-agnostic compressor that expects a standardized spectral data contract. The adapter is necessary when the compressor's internal representation differs from the on-disk XML schema, requiring explicit mapping of spectral arrays (m/z and intensity) and metadata.

When NOT to use

• The input mass spectrometry data is already in the compressor's native internal format or in a non-XML binary format (e.g., NetCDF, HDF5) without conversion via msconvert first.
• The m/z arrays in the source file are not monotonically increasing (a current implementation limitation that will cause incorrect compression behavior).
• The mzXML file uses block-based I/O or 64-bit representation, which is not supported by the current mzXML adapter.
• The spectral arrays are pre-compressed (e.g., with zlib or msnumpress) and have not been decompressed or converted using msconvert.

Inputs

• mzML file (XML-formatted mass spectrometry data with spectral records containing m/z and intensity arrays)
• mzXML file (XML-formatted mass spectrometry data, precursor to mzML, limited to 32-bit representation)
• C++ source code defining the format-agnostic API interface
• Example mzML/mzXML handler implementations

Outputs

• Adapter layer (C++ code) mapping mzML/mzXML spectral records to the compressor's internal data representation
• Validated compressed spectra ready for lossless or lossy compression stage
• Decoded mzML/mzXML file (functionally equivalent to input, whitespace-normalized)

How to apply

First, load and review the format-agnostic API interface (defined in include/mspack.h) to understand the expected spectral data contract—specifically the field names, types, and invariants for m/z and intensity arrays. Second, examine the provided mzXML and mzML handler examples to identify the reader/writer patterns used to parse XML and extract spectral records. Third, implement or instantiate an adapter layer that maps parsed mzML/mzXML spectral records (including raw, uncompressed m/z and intensity arrays) to the compressor's internal representation. Fourth, validate that spectra from both formats parse without schema errors and that m/z arrays are monotonically increasing (a requirement in the current implementation). Finally, confirm that no data loss occurs during the round-trip: compress and decompress a test file, and verify that the output XML is functionally equivalent to the input (allowing for whitespace normalization and SHA1 recalculation).

Related tools

• mspack (C++ program providing the format-agnostic compression API and reference mzML/mzXML adapter implementations; target system for integration) — https://github.com/fhanau/mspack
• msconvert (Utility to decompress pre-compressed spectral arrays (zlib, msnumpress) before mspack adapter ingestion)
• tinyxml2 (C++ XML parsing library used by mspack to read and write mzML/mzXML files)
• gzip / bsc (Compression backends available to mspack; gzip is default, bsc improves compression ratio at the cost of speed) — https://github.com/IlyaGrebnov/libbsc

Examples

./mspack --mzmle examples/BSA1.mzml BSA.mgz

Evaluation signals

• Spectra from both mzML and mzXML formats are successfully parsed by the adapter without schema validation errors or exceptions.
• m/z arrays extracted from source files are monotonically increasing; any violation is logged and prevents compression.
• Round-trip test (compress → decompress) produces an output XML file that is functionally equivalent to the input (diffable after whitespace normalization and SHA1 recalculation).
• Compression ratio and performance metrics (speed, file size reduction) for adapted spectra match or exceed baseline results for native mzML/mzXML compression in mspack examples.
• No data loss in spectral intensity or m/z values when comparing input and decoded output (bit-level equivalence for lossless mode, specified error tolerance for lossy mode).

Limitations

• Whitespace is not preserved in decoded files due to the XML library; however, this does not cause functional differences as XML is invariant to whitespace.
• The current implementation requires m/z values to be monotonically increasing; files with additional dimensions (e.g., ion mobility) in addition to m/z and intensity may violate this assumption.
• Only raw, uncompressed m/z and intensity arrays are supported; files with zlib or msnumpress compression must first be converted using msconvert.
• mzXML support is limited compared to mzML: mzXML is restricted to 32-bit files and does not support block-based I/O.
• SHA1 checksum tags in XML are recalculated during compression/decompression, so byte-for-byte round-trip equivalence is not guaranteed.

Evidence

• [other] mspack provides both a format-agnostic API and example implementations for mzXML and mzML formats to adapt spectra for compression.: "mspack provides both a format-agnostic API and example implementations for mzXML and mzML formats to adapt spectra for compression"
• [readme] The API is defined in include/mspack.h. Users are encouraged to use the API for advanced usage.: "The API is defined in include/mspack.h. Users are encouraged to use the API for advanced usage"
• [readme] The implementation can only use raw mz and intensity arrays. If the arrays are compressed with e.g. zlib or msnumpress, the file can be converted using msconvert.: "The implementation can only use raw mz and intensity arrays. If the arrays are compressed with e.g. zlib or msnumpress, the file can be converted using msconvert"
• [readme] The current implementation depends on the mz values being increasing.: "The current implementation depends on the mz values being increasing"
• [readme] Due to the XML library we use, whitespace is not preserved in the decoded file, although this does not cause a functional difference for the decoded file as XML is invariant to the addition or removal of whitespace.: "whitespace is not preserved in the decoded file, although this does not cause a functional difference for the decoded file as XML is invariant"
• [readme] mzXML is limited to 32-bit files and does not support the block-based I/O feature.: "mzXML is limited to 32-bit files and does not support the block-based I/O feature"