Python automated test execution

Skill HolobiomicsLab/asb-skill-collections/collections/metabolomics/v2/skills/python-automated-test-execution

Curated, evidence-grounded skill and software-tool collections for scientific AI agents, generated by the AgenticScienceBuilder

Install

npx -y skills add HolobiomicsLab/asb-skill-collections --skill python-automated-test-execution

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

14 stars14 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Use when when contributing code changes to a Python project (fork, feature branch, or pull request) that uses a setup.py-based test suite, before pushing changes to the remote repository or merging into the main branch.

The file declares its own license as CC-BY-4.0. That is the author’s claim about this one file, and it is not the same thing as the license GitHub reports for the repository, which is listed with the other numbers below.

SKILL.md

6.3 KB, as published. Nobody here has run it

python-automated-test-execution

Summary

Execute Python test suites within continuous integration workflows to verify that code changes do not break existing functionality. This skill ensures software reliability by automating test discovery and execution on each commit or pull request.

When to use

When contributing code changes to a Python project (fork, feature branch, or pull request) that uses a setup.py-based test suite, before pushing changes to the remote repository or merging into the main branch. Apply this skill to validate that new code or modifications do not introduce regressions in existing test cases.

When NOT to use

Input is a compiled binary or pre-built package—test suite must be executed from source.
Project does not use setup.py or has no defined test suite—use language-specific test runners (pytest, unittest, tox) directly instead.
Tests require external services or datasets not available in the CI environment—mock or skip integration tests that cannot run in CI.

Inputs

Python source code repository with setup.py configuration
Test suite (pytest, unittest, or custom tests referenced in setup.py)
Git commit or pull request containing code changes

Outputs

Test execution report (console output with pass/fail counts)
Exit code (0 for all tests passing, non-zero for failures)
GitHub Actions workflow status (optional: automated badge reflecting test outcome)

How to apply

Run the test suite using python setup.py test from the repository root after making code changes. This command discovers and executes all tests defined in the project's test configuration. Verify that all tests pass (exit code 0) before proceeding with commits or pull requests. In a GitHub Actions workflow context, integrate this step into the CI pipeline by adding a job that runs the same command; the workflow will trigger automatically on push and pull request events and report test results via a status badge. If tests fail, review the error output to identify which test cases are broken and remediate the code changes accordingly. The test execution should be repeatable and deterministic—running the same commit twice should yield the same test results.

Related tools

GitHub Actions (Orchestrates automated test execution on push and pull request events; reports CI status and generates status badge) — https://github.com/iomega/ms2query/actions/workflows/CI_build.yml
Python setuptools (Provides setup.py test command for discovering and running test suite)
Sonarcloud (Optional static analysis step integrated into CI workflow to scan code quality alongside test execution)

Examples

python setup.py test

Evaluation signals

Exit code is 0 (all tests passed) after running python setup.py test
GitHub Actions workflow status badge displays 'passing' and links to successful workflow run
All test case names are listed in console output with PASSED status; no FAILED or ERROR entries
Test count reported in output matches the expected number of test cases in the project
Repeated execution of the same commit produces identical test results (determinism check)

Limitations

Tests must be self-contained and not require manual environment setup beyond conda/pip dependencies listed in setup.py; external databases or API keys are not available in standard CI environments.
MS2Query is tested on MacOS, Windows, and Ubuntu for Python 3.9 and 3.10 only—tests may fail on other Python versions or operating systems.
Test execution time increases with the number of tests; large test suites may exceed CI runner time limits.
Flaky or non-deterministic tests (e.g., those depending on system clock, network I/O, or random seeds) may pass or fail intermittently despite no code changes.

Evidence

[methods] make sure the existing tests still work by running python setup.py test: "make sure the existing tests still work by running python setup.py test"
[other] MS2Query uses a GitHub Actions workflow (CI_build.yml) that triggers on pull requests and integrates with Sonarcloud, producing a GitHub Workflow Status badge: "MS2Query uses a GitHub Actions workflow (CI_build.yml) that triggers on pull requests and integrates with Sonarcloud, producing a GitHub Workflow Status badge"
[readme] MS2Query is tested by continous integration on MacOS, Windows and Ubuntu for python version 3.9 and 3.10: "MS2Query is tested by continous integration on MacOS, Windows and Ubuntu for python version 3.9 and 3.10"
[other] Configure GitHub Actions workflow file to trigger on push and pull request events to the ms2query repository.: "Configure GitHub Actions workflow file to trigger on push and pull request events to the ms2query repository"
[other] Define a matrix job that runs Python test suite using python setup.py test to verify existing tests pass.: "Define a matrix job that runs Python test suite using python setup.py test to verify existing tests pass"