agentsclimarketplace

Synthetic rca scenario pytest marker

Skill kjuhwa/skills-hub/skills/testing/synthetic-rca-scenario-pytest-marker

Self-correcting knowledge corpus for Claude Code — 9 stable shape clusters, bias-correction pipeline baked into contribution flow. 47 papers, 45 techniques, 1.1k skills.

Install
npx -y skills add kjuhwa/skills-hub --skill synthetic-rca-scenario-pytest-marker

Assembled from the repository path, not quoted from the project. Check it against their README if it does not work.

One thing to look at

  • 0 stars0 stars. Stars are a popularity signal and not a quality one, but at this level it is likely that nobody has read this closely except its author, and you would be relying on your own review.

What its author says it does

Copied from the file, not written here

Use a custom pytest marker (e.g. "synthetic") to gate non-deterministic LLM-based tests away from CI test-cov, while still enabling them via opt-in pytest -m synthetic and Makefile targets.

SKILL.md

2.4 KB, as published. Nobody here has run it

Synthetic RCA Scenario Pytest Marker

When to use

You have two test categories: deterministic unit/integration tests (run on every CI build) and slow non-deterministic LLM-based scenario tests (run nightly or on demand). You want one repo, one pytest invocation, but easy filtering.

How it works

  • Register the marker in pyproject.toml so pytest doesn't warn: 
    [tool.pytest.ini_options]
markers = [
  "synthetic: LLM-based synthetic RCA scenario tests (non-deterministic)",
]
  • Apply with @pytest.mark.synthetic (or pytestmark = pytest.mark.synthetic at module level).
  • Coverage runs exclude both the marker AND the directory because some collected tests don't have the mark explicitly set: 
    test-cov:
    pytest -n auto -v --cov=app --cov-report=term-missing \
      --ignore=tests/synthetic -m "not synthetic"

test-synthetic:
    pytest -m synthetic -v tests/synthetic/

Example

import pytest

@pytest.mark.synthetic
def test_root_cause_for_oom_alert(rca_runner, oom_fixture):
    result = rca_runner.invoke(oom_fixture)
    # Soft assertion — non-deterministic
    assert "memory" in result.root_cause.lower() or "oom" in result.root_cause.lower()

Gotchas

  • The --ignore=tests/synthetic plus -m "not synthetic" belt-and-suspenders approach catches synthetic tests that someone forgot to mark.
  • Use pytest-xdist (-n auto) for the deterministic suite; synthetic tests with real LLM calls usually need to run sequentially to respect rate limits.
  • Document the marker in the same markers list so pytest --markers is self-documenting for new contributors.
  • Pair with a tests/synthetic/run_suite.py script with a --scenario flag so users can target a single scenario for debugging.

Keep looking

Skills are one crate of 328,083. Ordering is by how many stacks a row turns up in, so the top of any crate is what has actually been picked rather than what has the most stars.