Unity python envs

Unity Python Envs

Use this subskill for Unity-specific questions about Python environments, package installation, wheel-based installs, notebook kernel setup, and GPU-dependent package builds.

Default Pattern

• Prefer module load mamba/25.11.0 for Python environment work on Unity.
• Use a user-owned prefix environment instead of relying on a shared module-owned env root.
• Install GPU-dependent packages on a compute node, not the login node.
• If you already have a built wheel, install the wheel rather than rebuilding from source on the cluster.

Modules On Unity

• ml avail and ml spider <term> are the fastest ways to inspect what the cluster exposes.
• Load the environment tooling before creating or activating envs.
• If the shell complains about activation, initialize the shell hook once in the current shell or batch script.

Typical shell setup:

module load mamba/25.11.0
eval "$(mamba shell hook --shell bash)"

User-Owned Prefix Envs

Prefix envs are the safest default on Unity.

mamba create --prefix $HOME/envs/unityenv python=3.12 pip -y
mamba activate $HOME/envs/unityenv

Notes:

• Activate prefix envs by full path, not by name.
• mamba activate unityenv only works for named envs that were created as named envs.
• If mamba env list shows a path but the name lookup fails, use the full path.
• Keep envs in a user-writable location such as $HOME/envs/<name> or $HOME/.conda/envs/<name>.

Installing Packages

For a normal Python package install:

python -m pip install --upgrade pip setuptools wheel
python -m pip install <package>

For a local wheel:

python -m pip install --upgrade /path/to/dist/package.whl

If you are testing your own code on Unity:

• rebuild the wheel locally after code changes
• copy the new wheel to the path the cluster job will use
• reinstall the wheel into the target env before rerunning the job

Compute-Node Installs For GPU Packages

GPU-dependent packages can behave differently depending on the runtime and hardware they are validated on.

Use a compute node when:

• the package compiles native extensions
• the package installs CUDA-specific wheels or GPU-sensitive dependencies
• you are validating a GPU build path for torch, torchvision, bitsandbytes, or similar packages

Practical rule:

• build or install on the same GPU family you expect to run on if you can
• a successful install on one GPU family does not prove another family has enough VRAM or the same runtime behavior

Notebook And Kernel Setup

If the env will be used from Jupyter or VS Code notebooks:

• install ipykernel into the env first
• register the kernel before launching the notebook server

Example:

python -m pip install ipykernel
python -m ipykernel install --user --name=unityenv --display-name="Python (unityenv)"

Common Unity Pitfalls

• Shared module roots can be read-only or managed by the site; do not assume you can write into them.
• A path shown by mamba env list does not mean you can safely activate it by name.
• Login-node shells and batch-job shells do not always have the same environment variables or working directory behavior.
• ~ is fine for a shell prompt, but code that saves files should use an absolute path or a path derived from a known project root.
• If a batch job is using the wrong Python, it is usually because the job never activated the env you thought it did.

Failure Triage

• CondaError: Run 'conda init' before 'conda deactivate': shell initialization/hook issue, usually not a broken environment.
• EnvironmentNameNotFound: you tried to activate a prefix env by name; use the full path.
• Permission denied or read-only write errors: you are targeting a shared root you do not own.
• ImportError after a successful install: confirm the job is running the same env where the package was installed.
• No module named ipykernel: install ipykernel inside the env, then register the kernel.
• CUDA OOM during package/model load: the env is probably fine; the GPU is too small or the model is too large.
• Install succeeds on the login node but fails on a compute node: the runtime is not the same; repeat the install or verification on the target compute node.

Minimal Unity Workflow

module load mamba/25.11.0
eval "$(mamba shell hook --shell bash)"
mamba create --prefix $HOME/envs/unityenv python=3.12 pip -y
mamba activate $HOME/envs/unityenv
python -m pip install --upgrade pip setuptools wheel
python -m pip install --upgrade /path/to/dist/package.whl

If this env will be used for notebooks, add ipykernel before you start the server.