Spectrum preprocessing for similarity analysis

spectrum-preprocessing-for-similarity-analysis

Summary

Preprocesses MS/MS spectra by removing low-intensity noise peaks to improve downstream spectral similarity and entropy-based compound identification. Applied before entropy similarity or other spectral comparison algorithms to ensure reliable quantification of spectral features.

When to use

When you have raw MS/MS spectra with residual noise or low-intensity peaks and plan to calculate spectral entropy, entropy similarity, or perform spectral library matching. Apply this before any spectral comparison algorithm to maximize identification performance, especially when using entropy-based methods that are sensitive to peak intensity distributions.

When NOT to use

• Spectra that have already undergone vendor-specific noise filtering or normalization—re-applying this threshold may over-filter and lose weak but genuine signal.
• Analysis pipelines that require retention of all detected peaks for absolute intensity quantitation or spectral imaging where weak peaks carry spatial information.
• High-resolution or high-sensitivity MS experiments where weak peaks near the 1% threshold are intentionally retained for trace-level or targeted detection.

Inputs

• raw MS/MS spectra with peak intensity values
• peak lists (m/z, intensity pairs) from mass spectrometry experiments

Outputs

• noise-filtered MS/MS spectra with low-intensity peaks removed
• preprocessed peak lists suitable for spectral similarity calculation

How to apply

Load preprocessed MS/MS spectra and filter peaks by removing those with intensity less than 1% of the maximum peak intensity in the spectrum. This noise removal step is recommended before calculating spectral similarity metrics. The rationale is that low-intensity peaks contribute noise rather than signal to entropy and similarity calculations; removing them preserves the informative portion of the spectrum (peaks ≥1% max intensity) while discarding spurious signals. Apply this threshold uniformly to all input spectra prior to computing entropy similarity scores or other distance metrics.

Related tools

• MSEntropy (Host package providing spectral_similarity module with preprocessing integration) — https://github.com/YuanyueLi/MSEntropy
• spectral_similarity (Module that operates on preprocessed spectra to compute entropy similarity scores) — https://github.com/YuanyueLi/MSEntropy
• Entropy Search GUI (User-facing tool that applies preprocessing internally before spectral file comparison) — https://github.com/YuanyueLi/EntropySearch

Examples

import numpy as np
import ms_entropy as me
peaks_raw = np.array([[69.071, 0.5], [86.066, 100], [120.0, 2.0]])
max_intensity = np.max(peaks_raw[:, 1])
peaks_filtered = peaks_raw[peaks_raw[:, 1] >= 0.01 * max_intensity]
similarity = me.spectral_entropy_similarity(peaks_filtered, peaks_reference)

Evaluation signals

• Verify that all retained peaks have intensity ≥ 1% of the maximum intensity in each spectrum.
• Confirm that the number of peaks decreases (noise removed) compared to the raw spectrum, but the top-intensity peaks remain unchanged.
• Entropy similarity score for known spectrum pairs matches reference outputs after preprocessing.
• Calculated entropy similarity value falls within the expected range [0, 1] as per the spectral_similarity module specification.
• Peak counts and intensity distributions in output metadata match the filtered peak set (no peaks below 1% threshold present).

Limitations

• The 1% intensity threshold is a general heuristic and may require adjustment for low-abundance compounds or noisy instrumentation; no adaptive threshold selection is provided.
• Preprocessing alone does not account for m/z calibration errors, isotope patterns, or in-source fragmentation artifacts that can confound spectral comparison.
• Removing low-intensity peaks is irreversible; spectra cannot be recovered to their original raw state once filtered.
• Performance improvement from preprocessing depends on the quality of the raw spectra; already clean spectra will show minimal benefit.

Evidence

• [other] Before calculating spectral similarity, it's highly recommended to remove spectral noise: "Before calculating spectral similarity, it's highly recommended to remove spectral noise"
• [other] Peaks below 1% max intensity removed to improve identification performance: "peaks have intensity less than 1% maximum intensity can be removed to improve identificaiton performance"
• [methods] Preprocessing involves filtering noise from input spectra: "Load two preprocessed MS/MS spectra (with noise removed: peaks <1% of maximum intensity filtered) from input files."
• [intro] Entropy similarity operates on preprocessed spectra for reliable quantification: "our package includes spectral entropy, entropy similarity, and many other functions"