Python module integration and pipeline extension

Python module integration and pipeline extension

Summary

Integrate new Python-implemented scoring components into an existing MS/MS spectral matching pipeline by implementing scoring functions, composing them with existing similarity metrics via a random forest re-ranker, and validating through unit tests and regression testing. This skill enables extension of spectral library search tools with novel scoring logic without disrupting core functionality.

When to use

You have developed new scoring logic (e.g., average inchikey score, neighbourhood score) for an MS/MS spectral analogue search tool and need to embed these components into an existing production pipeline that uses a random forest to combine multiple features for candidate re-ranking.

When NOT to use

• Your new scoring logic is not yet validated or lacks a formal algorithm specification — prototype and validate independently first.
• The scoring component is intended to replace (not augment) the random forest re-ranker; this skill assumes composition with existing features, not replacement.
• You have not established unit test coverage expectations or do not have access to known test cases; the skill requires testability verification.

Inputs

• Candidate match set (library spectra matched to query spectrum)
• Scoring algorithm specification (e.g., inchikey similarity rules, structural neighbourhood definitions)
• Existing MS2Query pipeline module with random forest re-ranker
• Unit test specifications for expected scoring outputs

Outputs

• Scoring component functions (Python callables returning numeric scores)
• Updated MS2Query pipeline with new scoring features integrated into random forest
• Unit test suite validating each scoring component
• CHANGELOG.md entry documenting new scoring metrics
• Pull request on iomega/ms2query with integrated code and tests

How to apply

First, implement each new scoring component as a Python function that computes a numeric score across a set of candidate matches, following the scoring algorithm specification (e.g., PR #78). Second, integrate the new scoring functions into the match ranking pipeline by adding them as features to the random forest re-ranker that combines the score with existing similarity metrics (MS2Deepscore, spectral similarity). Third, write unit tests in Python to verify each scoring component produces correct outputs for known input candidates. Fourth, run the existing test suite using python setup.py test to confirm no regression in core functionality. Fifth, update CHANGELOG.md documenting the new scoring metrics and their functionality. Finally, create a pull request documenting the integration.

Related tools

• MS2Query (The spectral library matching pipeline into which scoring components are integrated; implements the random forest re-ranker that combines scoring features.) — https://github.com/iomega/ms2query
• Python (Implementation language for scoring component functions, unit tests, and integration into the MS2Query module.)
• GitHub (Version control and pull request workflow for submitting integrated code changes to the iomega/ms2query repository.) — https://github.com/iomega/ms2query

Evaluation signals

• Each scoring component function returns numeric output (0–1 range or similar) for all candidate match inputs without error.
• Unit tests pass for known input candidates with expected output values; new tests document the scoring logic.
• Existing test suite runs without regression: python setup.py test passes all tests before and after integration.
• The random forest re-ranker successfully incorporates the new scoring features alongside existing metrics (MS2Deepscore, spectral similarity) and produces re-ranked candidate lists.
• CHANGELOG.md is updated with new scoring component names, descriptions, and functionality; pull request is accepted and merged into main branch.

Limitations

• MS2Query does not perform peak picking or clustering of similar MS2 spectra; input spectra should be preprocessed (e.g., via MZMine) to reduce redundancy before pipeline integration.
• The random forest re-ranker combines features trained on a specific library (e.g., GNPS 2021-12-15); new scoring components may require retraining on new model data if performance degrades.
• Scoring components must be defined for both positive and negative ionization modes; separate model and embedding files are required for each mode.

Evidence

• [other] Implement scoring component functions in Python that compute average inchikey score across candidate matches and neighbourhood score based on structural similarity.: "Implement scoring component functions in Python that compute average inchikey score across candidate matches and neighbourhood score based on structural similarity."
• [other] Integrate both scoring components into the MS2Query match ranking pipeline to combine with existing similarity metrics.: "Integrate both scoring components into the MS2Query match ranking pipeline to combine with existing similarity metrics."
• [other] Write unit tests in Python to verify each scoring component produces correct outputs for known input candidates.: "Write unit tests in Python to verify each scoring component produces correct outputs for known input candidates."
• [other] make sure the existing tests still work by running python setup.py test: "make sure the existing tests still work by running python setup.py test"
• [readme] MS2Query optimizes re-ranking the best analogue or exact match at the top by using a random forest that combines 5 features.: "MS2Query optimizes re-ranking the best analogue or exact match at the top by using a random forest that combines 5 features."