Figure table interpretation

figure_table_interpretation

License: restricted — no clear open-source license detected for the underlying tool; verify licensing before commercial use or redistribution. <!-- asb-license-banner -->

Summary

Extract and interpret numerical results, model predictions, and quantitative comparisons from figures, tables, and structured output (MGF, CSV, PKL formats) produced by mass spectrometry prediction workflows. This skill validates that predictions match expected ranges, statistical distributions, and domain constraints (e.g., precursor types, atom counts).

When to use

After running a 3DMolMS model (or similar MS/MS prediction pipeline) that outputs predictions in MGF, CSV, or PKL format, use this skill to verify prediction quality by examining peak intensities, m/z values, precursor ions, and similarity metrics against ground truth or reference spectra. Triggers include: (1) need to assess whether predicted spectra are chemically plausible; (2) evaluation of model performance via cosine similarity or other metrics; (3) filtering or ranking predictions by confidence; (4) checking whether predicted ions fall within expected instrument-specific mass ranges or precursor type distributions.

When NOT to use

• Input is already a pre-computed similarity matrix or aggregated summary statistic (e.g., AUC, F1 score already reported); this skill is for extracting and computing per-prediction metrics from raw output files.
• Ground truth spectra are unavailable or incompletely documented; comparison-based interpretation requires reference labeling.
• Molecules violate hard constraints (>300 atoms, unsupported atom types, unsupported precursor types) — these should be filtered out during preprocessing, not interpreted post-hoc.

Inputs

• predicted_mgf_file (MGF format spectra from molnet_engine.pred_msms)
• ground_truth_spectra (MGF, SDF, or reference library in MoNA/NIST format)
• prediction_dataframe (CSV or PKL output from pred_msms, pred_rt, or pred_ccs)
• test_pkl_file (preprocessed molecule dataset with ground truth labels)
• model_checkpoint (PyTorch .pt file for reproducibility)

Outputs

• evaluation_results_csv (per-spectrum metrics: cosine similarity, peak accuracy, precursor error)
• similarity_histogram (matplotlib plot of cosine similarity distribution)
• prediction_dataframe_annotated (input predictions + computed metrics + pass/fail flags)
• summary_statistics (mean, std, quantiles of similarity, RT error, CCS error by instrument/precursor type)

How to apply

Load the predicted spectra (MGF or PKL) and ground truth reference data into a structured format (DataFrame or dictionary). Compute per-spectrum metrics: peak count, intensity distribution, precursor m/z accuracy, and for comparison tasks, cosine similarity or spectral entropy. Filter predictions using constraints extracted from the workflow: atom number ≤300, atom types in {C, O, N, H, P, S, F, Cl, B, Br, I, Na}, and precursor types in {[M+H]+, [M-H]-, [M+H-H2O]+, [M+Na]+}. Plot histograms or scatter plots of similarity scores, retention time residuals, or CCS prediction errors to assess systematic biases. Rationale: The 3DMolMS model learns molecular representations through MS/MS prediction; interpreting its outputs against these domain constraints validates that the learned representations encode chemically and instrumentally meaningful features.

Related tools

• molnetpack (Python package providing MolNet class with pred_msms(), pred_rt(), pred_ccs(), and evaluate() methods to load predictions and ground truth data for interpretation) — https://pypi.org/project/molnetpack/
• matplotlib (Visualization library for plotting histograms, scatter plots, and heatmaps of prediction metrics (similarity, error distributions))
• Pandas (Data structure library for loading, filtering, and aggregating prediction DataFrames and computing per-spectrum statistics)
• PyTorch (Deep learning framework underlying model checkpoints; used to load and reproducibly apply models during prediction interpretation) — https://pytorch.org/get-started/locally/
• MoNA (Reference spectral library (MoNA-export-All_LC-MS-MS_QTOF.sdf, MoNA-export-All_LC-MS-MS_Orbitrap.sdf) for ground truth comparison in similarity computation)
• NIST20 (Reference library (hr_msms_nist.SDF, available under academic license) for evaluating MS/MS prediction accuracy)

Examples

molnet_engine.evaluate(test_pkl="./data/qtof_etkdgv3_test.pkl", pred_mgf="./output_msms.mgf", result_path="./eval_results.csv", plot_path="./eval_similarity_hist.png")

Evaluation signals

• Cosine similarity distribution across test set is unimodal, centered > 0.7, with no unexpected bimodality suggesting model failure on a subset
• Per-spectrum peak count in predicted MGF matches order of magnitude in ground truth (e.g., both have 10–100 peaks for singly-charged ions)
• Precursor m/z values in predictions fall within ±0.01 Da (QTOF) or ±5 ppm (Orbitrap) of input SMILES-derived m/z after accounting for adduct type
• Retention time (RT) and collision cross section (CCS) predictions show residuals with median close to zero and inter-quartile range <2× the training set standard deviation
• Histogram of similarity scores shows no spike at 0 or <0.3 suggesting systematic underperformance on specific subsets (e.g., molecules with unsupported atom types)

Limitations

• Model predictions are restricted to molecules with ≤300 atoms and atom types in {C, O, N, H, P, S, F, Cl, B, Br, I, Na}; interpretation is undefined for inputs outside these constraints.
• Precursor types supported: [M+H]+, [M-H]−, [M+H-H2O]+, [M+Na]+; predictions on other adducts (e.g., [M+2Na]2+) are not validated.
• Ground truth spectra quality depends on reference library provenance (MoNA, NIST20) and may reflect instrument-specific fragmentation patterns; QTOF and Orbitrap show different spectral distributions and require separate model checkpoints.
• Cosine similarity as a single metric does not capture peak intensity ordering errors or missing low-abundance fragments; supplementary metrics (spectral entropy, peak recall) may be needed for interpretation.

Evidence

• [readme] 3D Molecular Network for Mass Spectra Prediction (3DMolMS) is a deep neural network model to predict the MS/MS spectra of compounds from their 3D conformations.: "3D Molecular Network for Mass Spectra Prediction (3DMolMS) is a deep neural network model to predict the MS/MS spectra of compounds from their 3D conformations."
• [readme] Predict and save to MGF (checkpoints are downloaded automatically) pred_df = molnet_engine.pred_msms(path_to_results="./output_msms.mgf", instrument="qtof"): "Predict and save to MGF (checkpoints are downloaded automatically) pred_df = molnet_engine.pred_msms(path_to_results="./output_msms.mgf", instrument="qtof")"
• [readme] Evaluate against ground truth results_df = molnet_engine.evaluate(test_pkl="./data/qtof_etkdgv3_test.pkl", pred_mgf="./output_msms.mgf", result_path="./eval_results.csv", plot_path="./eval_similarity_hist.png"): "Evaluate against ground truth results_df = molnet_engine.evaluate(test_pkl="./data/qtof_etkdgv3_test.pkl", pred_mgf="./output_msms.mgf", result_path="./eval_results.csv","
• [methods] Atom types: 'C', 'O', 'N', 'H', 'P', 'S', 'F', 'Cl', 'B', 'Br', 'I', 'Na': "Atom types: 'C', 'O', 'N', 'H', 'P', 'S', 'F', 'Cl', 'B', 'Br', 'I', 'Na'"
• [methods] Precursor types: '[M+H]+', '[M-H]-', '[M+H-H2O]+', '[M+Na]+': "Precursor types: '[M+H]+', '[M-H]-', '[M+H-H2O]+', '[M+Na]+'"
• [readme] This model's molecular representation, learned through MS/MS prediction tasks, can be further applied to enhance performance in other molecular-related tasks, such as predicting retention times (RT) and collision cross sections (CCS).: "This model's molecular representation, learned through MS/MS prediction tasks, can be further applied to enhance performance in other molecular-related tasks, such as predicting retention times (RT)"