Python modernize

What I do

• Assess the project's Python version against the EOL schedule and recommend an upgrade path
• Recommend Python 3.14 free-threaded (no-GIL) build for CPU-bound threading workloads, or standard Python 3.14+ otherwise
• Migrate packaging metadata to pyproject.toml (PEP 621 [project] table)
• Set up uv as the package and environment manager
• Configure ruff as the single tool for linting and formatting (replaces black, flake8, isort, pyupgrade)
• Modernize Dockerfile — multi-stage build, slim-bookworm base, non-root user, uv-based installs
• Run security and vulnerability audits (pip-audit for dependency CVEs, bandit for source code, trivy for container images)
• Update CI configs and pre-commit hooks to use the new tools
• Add a .python-version file to pin the Python version
• Run the formatter and linter to bring the codebase to a clean state

When to use me

Use this skill when the project has any of:

• setup.py or setup.cfg (legacy packaging)
• requirements.txt without a pyproject.toml
• Pipfile / Pipfile.lock (pipenv)
• Separate black, flake8, isort, pyupgrade configurations
• No virtual environment tooling specified
• Python version that is EOL or approaching EOL (see reference table below)
• A Dockerfile using a full-fat base image, pip install, or running as root
• No dependency vulnerability scanning in place
• The user wants to evaluate Python 3.14's free-threaded (no-GIL) mode

Step-by-step workflow

---

1. Assess the Python version

1a. Determine current version

python3 --version
# Also check config files:
cat .python-version 2>/dev/null
grep -i "python" pyproject.toml setup.cfg setup.py Dockerfile 2>/dev/null | head -20

1b. Check against the EOL reference table

Version	Status	EOL Date	Notes
3.8	EOL	Oct 2024	No longer receives any patches — upgrade immediately
3.9	EOL	Oct 2025	No longer receiving security patches — upgrade immediately
3.10	Security-only	Oct 2026	Only critical security fixes; plan upgrade
3.11	Security-only	Oct 2027	Acceptable floor for existing projects
3.12	Bugfix	Oct 2028	Solid choice; actively maintained
3.13	Bugfix	Oct 2029	Experimental free-threaded support (PEP 703)
3.14	Current	Oct 2030	Officially supported free-threaded build (PEP 779); recommended for new projects
3.15	Prerelease	Oct 2031	Not yet released

Rule of thumb: The minimum requires-python for any actively maintained project should be >= 3.11. For new projects, target >= 3.12 or >= 3.13.

1c. Decide: standard vs. free-threaded Python 3.14

Python 3.14 (released Oct 2025) introduces an officially supported free-threaded build that disables the GIL (Global Interpreter Lock), enabling true multi-core parallelism for threads.

Choose the free-threaded (python3.14t) build when the project:

• Performs CPU-bound work across multiple threads (data processing, numerical computation, ML inference)
• Currently uses multiprocessing to work around the GIL and would benefit from shared-memory threading
• Runs backend APIs or inference servers that need concurrent CPU-intensive request handling

Stay with the standard (GIL-enabled) build when:

• The workload is primarily I/O-bound (network, filesystem) — the GIL is already released during I/O
• The application is single-threaded
• Critical C-extension dependencies have not yet been updated for free-threading compatibility
• Benchmarks show single-threaded performance regression (~5-10% overhead is expected in no-GIL mode)

How to verify free-threading at runtime:

import sys

# Check if the interpreter was built with free-threading
if hasattr(sys, "_is_gil_enabled"):
    print(f"GIL enabled: {sys._is_gil_enabled()}")
else:
    print("Standard (GIL-enabled) build")

To install via uv:

# Standard build
uv python install 3.14

# Free-threaded build
uv python install 3.14t

To force GIL back on in a free-threaded build (for compatibility testing):

PYTHON_GIL=1 python3.14t myapp.py
# or
python3.14t -X gil=1 myapp.py

---

2. Audit the current project state

Check for all legacy files and existing configs:

ls -la setup.py setup.cfg requirements*.txt Pipfile pyproject.toml .flake8 .black tox.ini Dockerfile docker-compose.yml .dockerignore 2>/dev/null

Read each file to understand:

• Project name, version, description, author, license
• All dependencies (runtime and dev/test)
• Any existing linter/formatter configuration
• Python version requirements
• Docker setup (base image, install method, user)

---

3. Verify uv is available

uv --version

If not installed:

curl -LsSf https://astral.sh/uv/install.sh | sh
# then reload shell: source ~/.bashrc or source ~/.zshrc

---

4. Create or migrate pyproject.toml

If pyproject.toml does not exist, create it with PEP 621 structure:

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "your-project-name"
version = "0.1.0"
description = "A short description of the project"
readme = "README.md"
license = { text = "MIT" }
authors = [
    { name = "Author Name", email = "[email protected]" },
]
requires-python = ">=(minimum_supported_version)"
dependencies = [
    # migrate from requirements.txt here
]

[project.optional-dependencies]
dev = [
    "pytest>=8.0",
    "pytest-cov>=5.0",
    "pip-audit>=2.7",
    "bandit>=1.7",
]

[project.urls]
Homepage = "https://github.com/org/repo"
Repository = "https://github.com/org/repo"

If pyproject.toml exists but uses [tool.poetry], migrate the [tool.poetry.dependencies] section to [project.dependencies] format:

• Poetry format: requests = "^2.28" → PEP 621: "requests>=2.28"
• Poetry dev deps → [project.optional-dependencies] dev group

---

5. Configure ruff in pyproject.toml

Add the [tool.ruff] section — this replaces black, flake8, isort, and pyupgrade:

[tool.ruff]
target-version = "py3XX"  # match requires-python minimum
line-length = 88           # same default as black

[tool.ruff.lint]
select = [
    "E",   # pycodestyle errors
    "W",   # pycodestyle warnings
    "F",   # pyflakes
    "I",   # isort
    "B",   # flake8-bugbear
    "C4",  # flake8-comprehensions
    "UP",  # pyupgrade — modernize Python syntax
    "N",   # pep8-naming
    "SIM", # flake8-simplify
    "TCH", # flake8-type-checking
    "S",   # flake8-bandit (security rules)
    "RUF", # ruff-specific rules
]
ignore = [
    "E501",  # line too long — handled by formatter
    "B008",  # do not perform function calls in default arguments
]

[tool.ruff.lint.isort]
known-first-party = ["your_package_name"]

[tool.ruff.format]
# Matches black defaults
quote-style = "double"
indent-style = "space"
skip-magic-trailing-comma = false
line-ending = "auto"

---

6. Initialize uv and lock dependencies

# Initialize uv in the project (creates .venv and uv.lock)
uv sync

# If migrating from requirements.txt:
uv add $(cat requirements.txt | grep -v '^#' | grep -v '^$' | tr '\n' ' ')

# If migrating from Pipfile:
# Manually map Pipfile [packages] to uv add commands

# Add dev dependencies:
uv add --dev pytest pytest-cov ruff pip-audit bandit

---

7. Add a .python-version file

# Pin to the target Python version decided in Step 1
echo "3.14" > .python-version
# Or for free-threaded:
# echo "3.14t" > .python-version

This is read by uv, pyenv, and many CI systems automatically.

---

8. Run security and vulnerability audits

8a. Scan dependencies for known CVEs

# Audit installed packages against PyPI Advisory Database + OSV
uv run pip-audit

# If vulnerabilities are found, pip-audit will list them with CVE IDs.
# Fix by upgrading the affected package:
uv add "package_name>=fixed_version"

# Re-run to confirm:
uv run pip-audit

8b. Scan source code for security anti-patterns

# Run bandit on the project source (exclude tests and venv)
uv run bandit -r src/ -x tests/ --severity-level medium

# Common findings to address:
# - B101: assert used in production code
# - B105/B106/B107: hardcoded passwords
# - B301/B302: pickle usage
# - B608: SQL injection via string formatting
# - B603: subprocess calls with shell=True

8c. Scan container image for OS-level vulnerabilities (if Docker is used)

# Install trivy if not present (one-liner)
# On Debian/Ubuntu:
sudo apt-get install -y trivy 2>/dev/null || \
  curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | sh -s -- -b /usr/local/bin

# Scan the built image
trivy image your-project:latest --severity HIGH,CRITICAL

# Scan the Dockerfile itself for misconfigurations
trivy config Dockerfile

---

9. Modernize Docker (if Dockerfile exists)

9a. Audit the existing Dockerfile

Look for these red flags:

• Full-fat base image (python:3.X instead of python:3.X-slim-bookworm)
• EOL Python version in FROM line
• Running as root (no USER directive)
• pip install without --no-cache-dir
• No .dockerignore file
• Single-stage build with build tools in the final image
• COPY . . before dependency install (busts layer cache on every code change)

9b. Modernized Dockerfile template (multi-stage, uv-based)

Replace or rewrite the Dockerfile using this pattern:

# ============================================================
# Stage 1: Builder — install dependencies in an isolated layer
# ============================================================
FROM python:3.14-slim-bookworm AS builder

# Install uv for ultra-fast dependency resolution
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/

WORKDIR /app

# Copy only dependency manifests first (layer cache optimization)
COPY pyproject.toml uv.lock ./

# Install dependencies into the system Python (no venv needed in container)
RUN uv sync --frozen --no-dev --no-install-project

# Copy application code
COPY . .

# Install the project itself
RUN uv sync --frozen --no-dev

# ============================================================
# Stage 2: Runtime — minimal image with only what's needed
# ============================================================
FROM python:3.14-slim-bookworm AS runtime

# Security: run as non-root
RUN groupadd --gid 1000 appuser && \
    useradd --uid 1000 --gid appuser --shell /bin/bash --create-home appuser

WORKDIR /app

# Copy the virtual environment from builder
COPY --from=builder /app/.venv /app/.venv

# Copy application code
COPY --from=builder /app .

# Put the venv on PATH
ENV PATH="/app/.venv/bin:$PATH"

# Drop privileges
USER appuser

EXPOSE 8000

CMD ["python", "-m", "your_app"]

For free-threaded builds, use python:3.14t-slim-bookworm (when available) or build from source with --disable-gil in the builder stage.

9c. Create or update .dockerignore

.git
.gitignore
.venv
venv
__pycache__
*.pyc
*.pyo
.env
.env.*
*.egg-info
dist
build
node_modules
.mypy_cache
.pytest_cache
.ruff_cache
*.md
!README.md
Dockerfile
docker-compose*.yml
.dockerignore

9d. Update docker-compose.yml (if present)

Ensure it references the new image tag and any changed port/volume mappings. If the compose file pins a Python version in environment variables or build args, update those to the target version from Step 1.

---

10. Run ruff to fix and format the codebase

# Fix all auto-fixable lint issues (includes isort, pyupgrade, etc.)
ruff check --fix .

# Format all Python files (replaces black)
ruff format .

Review any remaining lint errors that could not be auto-fixed:

ruff check .

Fix remaining issues manually, file by file.

---

11. Remove legacy config files

After confirming ruff and uv work correctly, remove obsolete files:

# Only remove after confirming everything works
rm -f setup.py           # if fully migrated to pyproject.toml
rm -f setup.cfg          # if fully migrated to pyproject.toml
rm -f Pipfile Pipfile.lock
rm -f .flake8            # replaced by [tool.ruff] in pyproject.toml
rm -f .isort.cfg         # replaced by [tool.ruff.lint.isort]
rm -f pyproject-black.toml  # replaced by [tool.ruff.format]

Caution: Do not remove requirements.txt if it is still referenced in Dockerfile, CI, or deployment scripts without updating those references first.

---

12. Update CI configuration

GitHub Actions example — replace old workflow steps:

# Before (legacy):
- run: pip install -r requirements.txt
- run: black --check .
- run: flake8 .
- run: isort --check .

# After (modern):
- uses: astral-sh/setup-uv@v4
  with:
    enable-cache: true
- run: uv sync --frozen
- run: ruff check .
- run: ruff format --check .
- run: uv run pip-audit
- run: uv run bandit -r src/ -x tests/ --severity-level medium
- run: uv run pytest

Add container scanning to CI (if Docker is used):

- name: Build Docker image
  run: docker build -t ${{ github.repository }}:ci .

- name: Scan image for vulnerabilities
  uses: aquasecurity/trivy-action@master
  with:
    image-ref: ${{ github.repository }}:ci
    severity: HIGH,CRITICAL
    exit-code: 1

---

13. Update or create .pre-commit-config.yaml

repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.11.4
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format

---

14. Test and validate

# Run the full test suite
uv run pytest

# Verify Docker build succeeds (if applicable)
docker build -t project-name:test .
docker run --rm project-name:test python --version

# Final security audit
uv run pip-audit

---

15. Commit the modernization

git add -A
git commit -m "chore: modernize Python tooling"

Rules and guardrails

• Always read pyproject.toml, setup.py, setup.cfg, and all requirements*.txt fully before migrating — missing a dependency will break the project
• Do not remove requirements.txt if it is referenced in Dockerfile, CI, or deployment scripts without updating those references first
• The ruff format output is intentionally identical to black output — no manual formatting changes needed
• If the project uses tox, update tox.ini to call uv run instead of direct python or pip commands
• Test the project after migration: uv run pytest must pass before committing
• If setup.py has custom build logic (not just metadata), that logic may need to be preserved in a hatch_build.py or similar — do not delete it blindly
• Docker: Never use :latest tag for base images — always pin to a specific Python version and Debian codename (e.g., python:3.14-slim-bookworm)
• Docker: Always verify the app starts correctly inside the container after Dockerfile changes (docker run --rm)
• Docker: If the existing Dockerfile has custom system-level dependencies (apt-get install), preserve them in the builder stage
• Security: Do not ignore HIGH or CRITICAL CVEs from pip-audit — either upgrade the package or document a justification if no fix is available
• Security: bandit findings at MEDIUM or above should be reviewed; false positives can be silenced with # nosec inline comments (with justification)
• Python version: Never upgrade past a version where all critical dependencies have published compatible wheels — check PyPI first
• Free-threading: If recommending the free-threaded build, audit all C-extension dependencies for compatibility; if any critical extension is incompatible, stay on the standard build and note it for future re-evaluation