Peak intensity normalization weighted aggregation

peak-intensity-normalization-weighted-aggregation

Summary

Normalize peak intensities to maximum = 1.0 per spectrum, then apply square-root weighting during spectral embedding aggregation to preserve intensity information while preventing high-intensity peaks from dominating fragment ion relationships in Word2Vec models. This technique is critical for learning balanced spectral embeddings that correlate structure and fragmentation patterns.

When to use

When training Word2Vec embeddings on mass spectra represented as peak-word documents, and you need to preserve the quantitative intensity relationships between fragments without allowing a single dominant peak to overwhelm the learned word associations. Essential when building Spec2Vec models where fragment intensity encodes chemical relationship information across a large spectral corpus (>90,000 spectra).

When NOT to use

• Input spectra are already reduced to presence/absence (binary) — intensity information would be lost; use unweighted token occurrence instead.
• Spectra have inconsistent or unreliable intensity calibration across a dataset — normalization artifacts may propagate; validate intensity data first.
• Goal is to match spectra based on exact fragment masses and NOT incorporate intensity relationships into similarity (e.g., traditional cosine score workflows) — use cosine or modified cosine without intensity weighting.

Inputs

• Mass spectra with annotated peaks (m/z and intensity pairs)
• Precursor m/z values (for neutral loss calculation)
• Cleaned spectral dataset with ≥10 fragment peaks per spectrum
• Peak intensity range [0, 1] after normalization

Outputs

• Weighted spectral documents ([email protected] and [email protected] words with √intensity weights)
• Trained Word2Vec model with fragment ion and neutral loss embeddings
• Embedding vectors (typically 100–300 dimensions) encoding learned spectral relationships

How to apply

For each spectrum: (1) normalize all peak intensities by dividing by the maximum intensity in that spectrum, so the highest peak = 1.0. (2) Convert each peak to a word token in format '[email protected]' (2-decimal m/z binning) and neutral losses to '[email protected]' tokens. (3) During Word2Vec training (CBOW or Skip-gram), weight each word occurrence by the square-root of the normalized peak intensity (weight = √normalized_intensity). (4) Use gensim's Word2Vec with window_size=500, negative=5, epochs=15–50 to learn embeddings that capture fragmentation relationships balanced across the intensity spectrum. The square-root transformation prevents intensity-based dominance while retaining relative intensity ordering; it serves as a biological/chemical rationale: fragments with higher relative intensity indicate more probable fragmentation pathways, but that relationship is sublinear rather than linear.

Related tools

• gensim (Word2Vec model training with custom intensity-based word weighting for peak and neutral loss embeddings)
• Spec2Vec (Spectral similarity scoring using weighted word embeddings derived from normalized and intensity-weighted spectral documents) — https://github.com/iomega/spec2vec
• matchms (Spectral data import, metadata cleaning, and peak filtering prior to intensity normalization and document conversion) — https://github.com/matchms/matchms
• NumPy (Vectorized intensity normalization and square-root transformation across all spectra)

Examples

import numpy as np; from gensim.models import Word2Vec; spectra_docs = [['[email protected]', '[email protected]', '[email protected]'] for s in spectra]; weights = [[np.sqrt(0.8), np.sqrt(0.6), np.sqrt(1.0)]] * len(spectra_docs); model = Word2Vec(sentences=spectra_docs, window=500, negative=5, epochs=15); similarities = [np.dot(model.wv['[email protected]'], model.wv['[email protected]']) for _ in range(1)]

Evaluation signals

• Verify all normalized intensities fall in [0.0, 1.0] with maximum intensity per spectrum = 1.0 (exact equality check).
• Confirm weighted word tokens are generated in correct format '[email protected]' (or '[email protected]') with weights = √(normalized_intensity) in [0.0, 1.0].
• Check that trained Word2Vec model embeddings capture expected fragmentation relationships: high-intensity fragments should co-occur more frequently in low-dimensional space than low-intensity fragments, and neutral losses should cluster with their corresponding peak m/z values.
• Validate that Spec2Vec similarity scores computed from intensity-weighted embeddings show stronger correlation with structural Tanimoto similarity (≥0.5 Spearman rho at top 0.1% scoring pairs) compared to unweighted or cosine-only baselines.
• Confirm missing-fraction metric (uncovered weighted intensity) is <0.05 threshold, indicating >95% of peak intensity in spectra is represented by words with known embeddings in the trained model.

Limitations

• Spec2Vec requires training data with sufficient feature coverage (fragments and neutral losses); if a spectrum contains peaks far outside the learned vocabulary, missing-fraction will increase and similarity scores become unreliable (mitigation: apply missing-fraction threshold <0.05 or retrain model on new experimental batches).
• Square-root weighting assumes sublinear relationship between peak intensity and fragmentation importance; this may not hold uniformly across all compound classes or ionization conditions — validation on domain-specific datasets recommended.
• Normalization by maximum intensity per spectrum can be unstable if spectra have very low signal-to-noise or contain single dominant peaks (e.g., molecular ion); pre-filtering spectra for minimum peak count (≥10) and noise filtering is essential.
• Word2Vec CBOW with intensity weights is not demonstrated for GC-MS data where neutral losses are not reliably measured; the method is validated only on LC-MS positive ionization mode.
• Model generalization suffers if training corpus m/z range, resolution, or instrument type differs substantially from query spectra (e.g., training on high-resolution Orbitrap but querying low-resolution ion trap).

Evidence

• [methods] Normalize peak intensities to maximum = 1 for each spectrum: "Normalize peak intensities to maximum = 1 for each spectrum."
• [methods] Weighted sum of word embeddings with weight = normalized intensity, sqrt applied: "weighted sum of word embeddings (weight = normalized intensity, sqrt applied)"
• [methods] Peaks represented as words in [email protected] format with neutral losses as [email protected]: "every peak is represented by a word that contains its position up to a defined decimal precision ("[email protected]"). In addition to all peaks of a spectrum, neutral losses between 5.0 and 200.0 Da were"
• [methods] Word2Vec CBOW training with window-size 500, negative=5, 15-50 epochs: "train from scratch using CBOW with window-size 500, negative sampling (negative=5), 15–50 epochs on spectrum documents"
• [methods] Missing-fraction threshold <0.05 filters spectra outside learned vocabulary: "By setting a threshold for the missing fraction (e.g. <0.05), returning Spec2Vec similarity scores for spectra far outside the learned peaks (and losses) can be avoided"
• [methods] Missing-fraction = 1 − (Σ√w_i for words in model) / (Σ√w_i for all words): "missing_fraction = 1 − (Σ√w_i for words in model) / (Σ√w_i for all words), where w_i is peak intensity"
• [readme] Word2Vec learns fragment relationships inspired by natural language processing: "Spec2Vec is a novel spectral similarity score inspired by a natural language processing algorithm -- Word2Vec. Where Word2Vec learns relationships between words in sentences, spec2vec does so for"