Neural network input preparation

neural-network-input-preparation

Summary

Prepare peak matrix batches from raw LCMS data for neural network training by loading mzML and feature tables into a NeatMS Experiment, then instantiating a NN_handler to generate training/test/validation splits with configurable matrix dimensions, margin encoding, and class balancing. This skill ensures that input peak matrices conform to expected dimensions and class distributions before model training.

When to use

When you have annotated representative LCMS samples (raw mzML files + labeled feature tables in mzmine CSV format) and need to convert them into balanced or unbalanced peak matrix batches with fixed dimensions for neural network training. Use this when you want to verify that preprocessing parameters (matrix_size, margin, min_scan_num) produce the expected peak matrix shape and class distribution before feeding data to a neural network trainer.

When NOT to use

• Feature table is not in CSV format or not mzmine-compatible; use format conversion tools first
• Raw mzML files are missing or cannot be accessed; batch creation requires both raw data and metadata
• Peak annotations are incomplete or majority of peaks are unlabeled; minimum sample requirements (10–20 pooled representative samples recommended) must be met

Inputs

• Raw mzML files (LCMS mass spectrometry data)
• Labeled feature table (CSV format, mzmine-compatible)
• Data folder path containing raw mzML files
• Feature table file path

Outputs

• Training batch arrays (peak matrices with shape 2×120 per peak)
• Test batch arrays
• Validation batch arrays
• Batch metadata (peak counts per class per batch)
• Peak matrix with binary encoding (margin=0, peak=1)

How to apply

Load raw mzML files and corresponding feature table (CSV format, mzmine-compatible) into a NeatMS Experiment object. Instantiate an NN_handler with specified matrix dimensions (matrice_size=120 by default), margin width (1 by default), and minimum scan threshold (min_scan_num=5 by default). Call create_batches() with split ratios (80% training, 10% test, 10% validation) and set normalise_class=False to preserve original class imbalance or normalise_class=True to equalize class counts (bounded by smallest class size). Inspect resulting batch arrays to verify that peak matrices are 2×120 (binary encoding margin vs. peak regions), with margin occupying first and last 40 values and peak signal in the middle 40 values. Validate class distributions match the normalise_class parameter setting.

Related tools

• NeatMS (Core package providing Experiment, NN_handler, and batch creation methods for peak matrix generation and class normalization) — https://github.com/bihealth/NeatMS
• NumPy (Array manipulation and inspection of peak matrix dimensions and values)
• pandas (Feature table I/O and metadata handling)
• Python (Programming language for orchestrating Experiment and NN_handler objects)

Examples

from neatms import Experiment, NN_handler
exp = Experiment('/path/to/raw_data', '/path/to/feature_table.csv', input_data_type='mzmine')
nn_handler = NN_handler(exp, matrice_size=120, margin=1, min_scan_num=5)
nn_handler.create_batches(validation_split=0.1, normalise_class=True)

Evaluation signals

• Peak matrices have expected shape of 2×120 (binary dimension and scan dimension)
• First and last 40 values of each peak matrix correspond to margin regions (encoding value 0)
• Middle 40 values correspond to peak signal regions (encoding value 1)
• When normalise_class=True, all three classes have equal peak counts within each batch (count = smallest class size); when normalise_class=False, class counts reflect original dataset imbalance
• Training, test, and validation batches have correct sample counts reflecting 80%, 10%, 10% split ratios

Limitations

• Default minimum scan requirement (min_scan_num=5) will filter out peaks with fewer scans; peaks below this threshold cannot be included in batch matrices
• Class normalization (normalise_class=True) discards peaks from larger classes to match the smallest class; this reduces effective training data size and may lose minority peak variants
• Requires at least 10–20 pooled representative samples for optimal training dataset construction; smaller datasets may not provide sufficient diversity for generalization
• Matrix size (matrice_size=120) is fixed at instantiation; different peak widths or scan resolutions may require re-instantiation with adjusted parameters

Evidence

• [methods] Load raw mzML and feature table, instantiate NN_handler with matrix parameters, generate train/test/validation batches: "Create a NeatMS Experiment object by loading raw mzML files from the example data folder and the corresponding feature table (csv format) from the github repository, specifying the input format as"
• [methods] Peak matrices encode margin vs. peak regions as binary values: "confirm that the binary dimension correctly encodes margin (0) vs. peak (1) regions. 5. Validate that the first and last 40 values correspond to margin portions and the middle 40 values correspond to"
• [methods] normalise_class parameter controls class distribution in batches: "When normalise_class is set to True, the create_batches() method ensures that every class has an equal number of peaks in the resulting training batches, with the total number of peaks per class"
• [methods] min_scan_num filters peaks below threshold: "min_scan_number argument will filter out all peaks that have a number of point (scan) lower than this value. 5 is the default value"
• [readme] NeatMS is designed for untargeted LCMS peak filtering: "NeatMS is an open source python package for untargeted LCMS signal labelling and filtering. NeatMS enables automated filtering of false positive MS¹ peaks reported by commonly used"