Yocto python packaging

Yocto Python3 Packaging (Scarthgap/5.0+)

Follow these patterns to package modern Python modules using the OpenEmbedded PEP 517 build infrastructure.

1. Core Build System Selection

Do not use inherit setuptools3 for packages containing a pyproject.toml. Select the correct class based on the [build-system] defined in the source.

Map the build backend (in pyproject.toml) to the appropriate Yocto class to inherit:

• hatchling.build -> inherit python_hatchling
• setuptools.build_meta -> inherit python_setuptools_build_meta
• flit_core.buildapi -> inherit python_flit_core
• poetry.core.masonry.api -> inherit python_pep517

2. Error Resolution Procedures

Error: ModuleNotFoundError: No module named 'setuptools.command.bdist_wheel'

• Diagnosis: The recipe is likely using inherit setuptools3 on a package that requires a PEP 517 build frontend (like python3-build-native).
• Action: Switch to inherit python_setuptools_build_meta. This ensures python3-wheel-native and python3-installer-native are present in the sysroot.

Error: Circular Dependencies in python3-build-native

• Diagnosis: Attempting to manually create native build-tool recipes that require themselves to install.
• Action: Always leverage the upstream python_pep517.bbclass. It utilizes a "bootstrap" mechanism (Stage 0) to break the dependency loop between the installer and the builder. Do not attempt to override python3-build-native manually.

3. Dependency Management

Modern Python packaging differentiates between build-time and runtime dependencies. Implement them as follows:

• Build-time (DEPENDS): Use for requirements in the native sysroot to create the wheel.
  • Example: DEPENDS += "python3-hatchling-native"
• Runtime (RDEPENDS): Use for requirements on the target image for the package to function.
  • Example: RDEPENDS:${PN} += "python3-compression python3-core"

4. Best Practices for Scarthgap

Apply the following practices when developing recipes:

1. Prefer PyPI: Use inherit pypi whenever possible to automate SRC_URI construction and versioning.
2. Verify Backend: Always check the source pyproject.toml before writing the recipe. If multiple backends are supported, hatchling is the modern preference for many core utilities.
3. Use Layered Definitions: Ensure meta-python is updated to the Scarthgap branch to avoid version mismatches between the class logic and the recipes.

5. Fetch Accurate PyPI Data via Python

When curl or jq are unavailable, use Python to reliably extract the correct source URLs and SHA256 hashes for Yocto recipes.

• Execute script:

  ./scripts/fetch_pypi_data.py <package_name> <version>
  

  (Replace <package_name> and <version> with the target package details).

6. Fix SPDX Compliance via .bbappend

Patch upstream or external recipes to fix license issues without modifying their source by using a .bbappend in the local layer.

• Example command:

  echo 'LICENSE = "BSD-3-Clause"' > meta-tcat/recipes-python/jupyter-console/python3-jupyter-console_%.bbappend
  

7. Anticipate Packaging Quirks and Provider Conflicts

• Static Assets: When dealing with UI, web, or Jupyter components, preemptively check FILES:${PN} and prepare for static assets landing outside of the standard Python site-packages directory.
• Conflicting Providers: Do not create competing recipes (e.g., a custom fork recipe alongside the original). Inspect the system's core-image.bb first to understand existing provider constraints.
  • Apply custom forks via SRC_URI patches to the existing recipe.
  • Alternatively, utilize PREFERRED_PROVIDER globally rather than battling package manager conflicts in the image rootfs.