Mass spectrometry spectrum preprocessing

mass-spectrometry-spectrum-preprocessing

Summary

Normalize and format experimental mass spectrometry spectra to match the input requirements of a trained neural encoder before inference. This is a critical preprocessing step that ensures real LC–QTOF spectra are compatible with deep learning models trained on pseudo SMILES-spectrum pairs.

When to use

When you have raw LC–QTOF wastewater spectra (or other real experimental mass spectrometry data) and need to feed them into a pretrained MSGO model for molecular structure prediction. The trigger is a mismatch between raw spectrum format (raw m/z arrays, intensity values, metadata) and the model's expected tensor input shape and normalization convention.

When NOT to use

• Input spectra are already pseudo SMILES-spectrum pairs generated by cfmid (use them directly for training, not preprocessing)
• You have no reference model or training protocol defining the expected spectrum format (preprocessing requires a known input schema)

Inputs

• Raw LC–QTOF spectrum (CSV, NetCDF, or vendor format with m/z and intensity columns)
• Experimental wastewater sample spectra (300+ spectra in real datasets)
• Model configuration or metadata describing expected input shape and normalization

Outputs

• Preprocessed spectrum tensor compatible with MSGO model input layer
• Normalized intensity values and aligned m/z ranges
• Preprocessed spectrum in CSV or HDF5 format suitable for batch inference

How to apply

Load raw experimental spectra from CSV or vendor-specific formats (e.g., LC–QTOF output). Normalize intensity values and align m/z ranges to match the format used during MSGO model training on 30k+ pseudo SMILES-spectrum pairs. Apply any mass accuracy corrections or intensity scaling required by the model (e.g., log-intensity transformation, m/z binning). Remove or flag low-intensity noise below a model-specific threshold. Convert the formatted spectrum into a tensor representation compatible with Torch 1.7.1. Verify shape and data type match the model's input layer (typically a 2D array of m/z–intensity pairs or a binned mass spectrum vector). This ensures inference produces valid confidence-ranked SMILES predictions rather than numerical errors or out-of-domain outputs.

Related tools

• MSGO model (Pretrained deep learning model that consumes preprocessed spectra to output ranked molecular structure predictions) — github.com/aaronma2020/MSGO
• Torch (Deep learning framework (version 1.7.1) used to load model and convert preprocessed spectra to tensors for inference)
• Python (Scripting language (version 3.7) for implementing spectrum loading, normalization, and tensor conversion workflows)

Examples

# After downloading and extracting LC–QTOF wastewater spectra to ./data/wastewater.csv, preprocess as follows:
python tools/eval_standard.py --log_path ckpts/pfas --real_csv ./data/wastewater.csv --out_csv ./wastewater_results.csv --beam_size 500 --polar neg

Evaluation signals

• Preprocessed spectrum tensor shape matches MSGO model's input_size parameter (e.g., [batch_size, spectrum_dim])
• Intensity values fall within model's expected range (e.g., [0, 1] if min-max normalized, or log-transformed to typical range observed in training data)
• m/z values are aligned to model's expected mass range (no NaN, inf, or out-of-bounds values)
• Inference on preprocessed spectrum produces valid confidence scores (non-zero, between 0 and 1) and ranked SMILES predictions rather than errors or trivial outputs
• Predictions on known reference compounds or literature-annotated wastewater compounds show non-negligible confidence (>0.01) for correct or near-correct structures

Limitations

• Preprocessing assumes the model's training distribution (30k+ cfmid-generated pseudo spectra) is representative of real experimental LC–QTOF spectra; domain shift or unusual ionization modes may degrade downstream prediction quality
• Spectrum quality and instrument calibration directly impact preprocessing fidelity; poorly calibrated or noisy raw spectra may fail normalization or produce low-confidence predictions
• No single preprocessing formula is provided in the README; practitioners must infer normalization strategy from model training details or empirical validation against known compounds

Evidence

• [other] Preprocess the experimental spectra to match the input format expected by the model.: "Preprocess the experimental spectra to match the input format expected by the model."
• [readme] For Training, we use 30k+ pseudo smiles-specturm pairs generated by cfmid: "For Training, we use 30k+ pseudo smiles-specturm pairs generated by cfmid"
• [readme] we use one LC–QTOF dataset for wastewater samples to verify our model: "we use one LC–QTOF dataset for wastewater samples to verify our model"
• [other] Load the pre-trained MSGO model (Torch 1.7.1) and the LC–QTOF wastewater dataset containing 300+ real spectra.: "Load the pre-trained MSGO model (Torch 1.7.1) and the LC–QTOF wastewater dataset containing 300+ real spectra."