Library module organization and accessibility

library-module-organization-and-accessibility

Summary

Organizing a scientific Python library into well-documented subpackages and utilities ensures that functions are discoverable, importable, and properly integrated into the published API. This skill covers how to structure cooltools so that utilities like adaptive_coarsegrain are accessible via established import paths (e.g., cooltools.lib) and appear in generated documentation.

When to use

You are building or extending a multi-module Python library for scientific computation (e.g., Hi-C analysis, image processing) and need to ensure that new utility functions are discoverable by end-users through stable import paths, appear in auto-generated API documentation, and pass code coverage and style linting checks as part of the library's continuous integration.

When NOT to use

• The function is a private internal utility that should not be exposed to end-users; keep it in a private module without exporting it in init.py.
• The library uses lazy importing or plugin discovery mechanisms that do not rely on explicit init.py registration; follow the library's established pattern instead of forcing standard module organization.
• Documentation is generated by an alternative system (e.g., MkDocs, pdoc) that does not use Sphinx; adapt the documentation build step accordingly.

Inputs

• Python source file containing a new utility function (e.g., adaptive_coarsegrain.py)
• cooltools library repository with setup.py or pyproject.toml
• Existing cooltools.lib subpackage init.py
• Pytest configuration (pytest.ini or pyproject.toml [tool.pytest.ini_options])
• Sphinx configuration (conf.py) and documentation source files

Outputs

• Function callable via stable import path (cooltools.lib.adaptive_coarsegrain)
• Function entry in cooltools.lib API reference documentation (HTML/PDF from Sphinx)
• pytest test report with coverage metrics and style compliance
• Updated module docstring reflecting new utilities

How to apply

Install the library in editable development mode using pip install -e . to ensure that module changes are immediately reflected in the Python environment. Verify that the function can be imported from its documented subpackage path (e.g., from cooltools.lib import adaptive_coarsegrain), confirming that it is callable and registered in the module's namespace. Run the full pytest suite on the lib package unit tests to verify function behavior, code coverage, and code style (using pytest-cov and pytest-flake8). Build the Sphinx documentation using make docs to confirm that the function and its Numpy-style docstring appear in the cooltools.lib API reference. Generate a test report documenting successful import, execution, and documentation inclusion. If any of these steps fail—import errors, missing docstring, low coverage, style violations, or missing documentation—revise the module structure, add or fix docstrings, and re-run the checks.

Related tools

• pytest (Execute unit tests on the lib package to verify function behavior and measure code coverage) — https://docs.pytest.org/en/latest
• pytest-cov (Measure code coverage for unit tests to ensure new utilities have adequate test coverage)
• pytest-flake8 (Check code style compliance (PEP 8) during test execution to enforce consistent formatting)
• Sphinx (Generate API reference documentation from Numpy-style docstrings, verifying function appears in cooltools.lib docs) — http://www.sphinx-doc.org/en/stable
• pip (Install the library in editable development mode to enable live import testing)

Examples

cd cooltools && pip install -e . && python -c "from cooltools.lib import adaptive_coarsegrain; print(adaptive_coarsegrain)" && pytest cooltools/lib --cov=cooltools.lib --flake8 && make docs

Evaluation signals

• The function can be imported without error using the documented import path: from cooltools.lib import adaptive_coarsegrain returns a callable object.
• Pytest runs successfully on the lib package with no test failures, and coverage report shows the new function has ≥80% line coverage.
• Sphinx build completes without warnings or errors, and the function name and docstring appear in the generated API reference HTML under cooltools.lib.
• pytest-flake8 reports no style violations (E* or W* codes) in the new module.
• The function's Numpy-style docstring is properly formatted (Parameters, Returns, Examples sections present) and renders correctly in the documentation.

Limitations

• The article notes that new functionality for smoothing P(s) and derivatives has an API that is not yet stable, suggesting that utility organization may need to be revisited if the API changes; mark unstable functions with deprecation warnings or notes in docstrings.
• The provided section text ends mid-sentence and contains no functional details about adaptive_coarsegrain's implementation; verification of correct integration must rely on runtime inspection and documentation rendering rather than source-level inspection alone.
• Integration into cooltools.lib assumes the library uses standard Python package conventions (setup.py, init.py registration); non-standard or plugin-based package structures may require adapted organization steps.

Evidence

• [other] install in editable development mode using pip install -e .: "install in "editable" (i.e. development) mode using the -e option"
• [other] Run pytest on the lib package unit tests to verify function behavior and code coverage are satisfactory.: "you can just cd to the root of your repository and run pytest"
• [other] Use pytest-cov and pytest-flake8 to check code coverage and style.: "We use pytest as our unit testing framework with the pytest-cov extension to check code coverage and pytest-flake8 to check code style"
• [other] Build Sphinx documentation and verify the function appears in the cooltools.lib API reference.: "We use Numpy style docstrings and Sphinx to document this library"
• [other] Build documentation using make docs.: "To build the documentation: make docs"