Cooler file format handling

cooler-file-format-handling

Summary

Load, validate, and interact with cooler format files—a hierarchical HDF5-based storage format for high-resolution Hi-C contact matrices—using the cooler Python library. This skill is essential for accessing, inspecting metadata, and preparing contact data for downstream genomic analyses.

When to use

You have a Hi-C dataset stored in cooler format (e.g., from a public repository, GEO, or Zenodo, or generated by a contact-calling pipeline) and need to programmatically load it, inspect its structure (bins, chromosomes, resolution), verify data integrity, or pass it to analysis functions like cooltools.coverage or cooltools.insulation.

When NOT to use

• Input is already loaded as a numpy/scipy sparse matrix or pandas DataFrame and you do not need cooler's metadata or hierarchical access features.
• Your Hi-C data is in a different format (e.g., text-based .hic, Juicer, or raw BAM files); use format conversion tools first.
• You require real-time streaming or random-access queries on datasets larger than available memory and do not have HDF5 infrastructure in place.

Inputs

• cooler file path (.cool or .mcool format)
• genomic region specification (optional; chromosome, start, end)
• resolution level (for .mcool files only)

Outputs

• cooler object (Python cooler.Cooler or cooler.MultiResolutionCooler instance)
• bin table (pandas DataFrame with bin coordinates and metadata)
• contact matrix (sparse or dense array for specified genomic region)
• metadata dictionary (resolution, bin count, chromsome list, balance weights)

How to apply

Use the cooler Python API to open a .cool or .mcool file, inspect bin-level metadata (coordinates, sizes) and matrix properties, and validate that the file structure matches expected schemas. The cooler library provides a Pythonic interface to the hierarchical HDF5 format, allowing you to fetch contact matrices by genomic range and inspect balance weights or other stored annotations. Verify file integrity by checking row counts, coordinate ranges, and the presence of required columns (chrom, start, end, and contact matrix values). For multi-resolution datasets (mcool), specify the desired resolution when loading.

Related tools

• cooler (Primary Python library for reading, writing, and manipulating cooler format files; provides hierarchical HDF5 access to binned contact matrices) — https://github.com/open2c/cooler
• cooltools (Companion library offering high-level functions (e.g., coverage, insulation) that accept cooler objects as input) — https://github.com/open2c/cooltools
• Python (Runtime environment for executing cooler API calls and data manipulation)

Examples

import cooler; c = cooler.Cooler('data.cool'); bins = c.bins[:]; matrix = c.matrix(balance=False).fetch('chr1:0-5000000')

Evaluation signals

• cooler object loads without HDF5 read errors and bin table has expected row count matching (genome_size / resolution)
• Bin table contains required columns: chrom, start, end; all coordinates are non-negative integers and ordered
• Contact matrix shape matches (nbins, nbins) and contains numeric contact counts (non-negative for valid Hi-C data)
• Metadata attributes (resolution, binsize, nchroms) are consistent and match file specifications
• Genomic range queries return non-empty or empty matrices as expected; balance weights (if present) are numeric and non-zero

Limitations

• cooler format requires HDF5 support; some systems or environments may not have HDF5 libraries installed.
• Very large cooler files (.mcool with many resolutions or genome-wide high-resolution maps) may require substantial disk I/O; random-access queries can be slow without proper indexing.
• cooler format stores binned contact data; original fragment-level or alignment information is not preserved.
• Balance weights and other stored annotations may be missing or incomplete depending on how the cooler file was generated; validation must account for optional fields.

Evidence

• [intro] The recently-introduced cooler format readily handles storage of high-resolution datasets: "The recently-introduced cooler format readily handles storage of high-resolution datasets"
• [intro] Larger datasets increase computational challenges at each step: storage, memory, and researchers' time: "Larger datasets increase the challenges at each step of computational analysis, from storage, to memory, to researchers' time."
• [other] cooler Python API for loading and inspecting cooler files: "The recently-introduced cooler format"
• [other] cooltools provides computational tools for analyzing high-resolution Hi-C datasets stored in the cooler format: "Cooltools provides computational tools for analyzing high-resolution Hi-C datasets stored in the cooler format, enabling extraction of quantitative genomic features."