Python packaging

Python Packaging

Comprehensive guide to creating, structuring, and distributing Python packages using modern packaging tools, pyproject.toml, and publishing to PyPI.

When to Use This Skill

• Creating Python libraries for distribution
• Building command-line tools with entry points
• Publishing packages to PyPI or private repositories
• Setting up Python project structure
• Creating installable packages with dependencies
• Building wheels and source distributions
• Versioning and releasing Python packages
• Creating namespace packages
• Implementing package metadata and classifiers

Core Concepts

1. Package Structure

• Source layout: src/package_name/ (recommended)
• Flat layout: package_name/ (simpler but less flexible)
• Package metadata: pyproject.toml, setup.py, or setup.cfg
• Distribution formats: wheel (.whl) and source distribution (.tar.gz)

2. Modern Packaging Standards

• PEP 517/518: Build system requirements
• PEP 621: Metadata in pyproject.toml
• PEP 660: Editable installs
• pyproject.toml: Single source of configuration

3. Build Backends

• setuptools: Traditional, widely used
• hatchling: Modern, opinionated
• flit: Lightweight, for pure Python
• poetry: Dependency management + packaging

4. Distribution

• PyPI: Python Package Index (public)
• TestPyPI: Testing before production
• Private repositories: JFrog, AWS CodeArtifact, etc.

Quick Start

Minimal Package Structure

my-package/
├── pyproject.toml
├── README.md
├── LICENSE
├── src/
│   └── my_package/
│       ├── __init__.py
│       └── module.py
└── tests/
    └── test_module.py

Minimal pyproject.toml

[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"

[project]
name = "my-package"
version = "0.1.0"
description = "A short description"
authors = [{name = "Your Name", email = "[email protected]"}]
readme = "README.md"
requires-python = ">=3.8"
dependencies = [
    "requests>=2.28.0",
]

[project.optional-dependencies]
dev = [
    "pytest>=7.0",
    "black>=22.0",
]

Package Structure Patterns

Pattern 1: Source Layout (Recommended)

my-package/
├── pyproject.toml
├── README.md
├── LICENSE
├── .gitignore
├── src/
│   └── my_package/
│       ├── __init__.py
│       ├── core.py
│       ├── utils.py
│       └── py.typed          # For type hints
├── tests/
│   ├── __init__.py
│   ├── test_core.py
│   └── test_utils.py
└── docs/
    └── index.md

Advantages:

• Prevents accidentally importing from source
• Cleaner test imports
• Better isolation

pyproject.toml for source layout:

[tool.setuptools.packages.find]
where = ["src"]

Pattern 2: Flat Layout

my-package/
├── pyproject.toml
├── README.md
├── my_package/
│   ├── __init__.py
│   └── module.py
└── tests/
    └── test_module.py

Simpler but:

• Can import package without installing
• Less professional for libraries

Pattern 3: Multi-Package Project

project/
├── pyproject.toml
├── packages/
│   ├── package-a/
│   │   └── src/
│   │       └── package_a/
│   └── package-b/
│       └── src/
│           └── package_b/
└── tests/

Detailed patterns and worked examples

Detailed pattern documentation lives in references/details.md. Read that file when the navigation tier above is insufficient.