Contact frequency smoothing log space

contact-frequency-smoothing-log-space

Summary

Apply logarithmic binning and smoothing to precomputed expected contact frequency tables to produce log-binned, smoothed contact probability P(s) curves suitable for Hi-C distance decay analysis. This skill transforms raw distance-frequency pairs into log-spaced bins with smoothed contact frequencies, essential for characterizing the relationship between genomic distance and contact probability.

When to use

You have a precomputed expected contact frequency table (TSV with columns: dist_bp, contact_frequency, n_valid) derived from cooler Hi-C matrices and need to generate a smoothed, log-binned P(s) curve for downstream analysis such as TAD detection, contact probability visualization, or comparison of contact decay across conditions or cell types.

When NOT to use

• Input is already a log-binned or smoothed contact frequency table; applying this skill twice will over-smooth and lose resolution.
• Raw contact matrix (cooler file) is available and you need to compute expected frequency de novo; use cooler's built-in expected/observed computation first.
• Analysis requires linear (not logarithmic) binning of distances, e.g., fixed-width bins for regulatory element analysis.

Inputs

• precomputed expected contact frequency table (TSV format with columns: dist_bp, contact_frequency, n_valid)
• cooler-derived Hi-C dataset (optional, if generating expected table de novo)

Outputs

• log-binned and smoothed P(s) table (TSV format with columns: dist_bp, count.avg.smoothed, bin statistics)
• log-space contact probability curve suitable for visualization and downstream analysis

How to apply

Load the expected_cis table (TSV format) into Python and apply the cooltools logbin_expected() function to perform logarithmic binning on distance values and apply smoothing to contact frequencies within each log bin. The function uses a logarithmic scale to group genomic distances such that relative changes in distance are equally represented, which is critical for Hi-C contact decay typically follows a power law. Configure the logarithmic bin spacing (e.g., base 2 or custom log factor) to balance resolution at short distances with coverage at long distances. The smoothing algorithm within logbin_expected() estimates mean smoothed contact frequency for each log bin, reducing noise while preserving the overall decay shape. Output the result as a TSV with columns for binned distance (dist_bp), mean smoothed contact frequency (count.avg.smoothed), and associated bin statistics (e.g., bin count, variance). Validate that the output curve is monotonically decreasing and spans the full input distance range.

Related tools

• cooltools (Provides logbin_expected() function to perform logarithmic binning and smoothing on precomputed expected contact frequency tables) — https://github.com/open2c/cooltools
• cooler (Stores and retrieves Hi-C contact matrices in cooler format; used to generate or access the precomputed expected_cis table) — https://github.com/open2c/cooler
• Python (Host language for cooltools API and data manipulation (NumPy, Pandas for table I/O and processing))

Examples

from cooltools.expected import logbin_expected; import pandas as pd; expected = pd.read_csv('expected_cis.tsv', sep='\t'); smoothed = logbin_expected(expected, logbase=2.0); smoothed.to_csv('expected_cis.logbin.tsv', sep='\t', index=False)

Evaluation signals

• Output TSV contains expected columns (dist_bp, count.avg.smoothed, bin statistics) with correct data types and no missing values.
• Contact frequency values are monotonically decreasing or non-increasing as a function of distance (no spurious oscillations introduced by smoothing).
• Log bin boundaries follow logarithmic spacing (e.g., bin width in log space is constant); verify by computing log ratios of consecutive bin edges.
• Number of output bins is significantly smaller than input rows, confirming binning aggregation; typical reduction to 20–50 bins for mammalian genome.
• Smoothed curve visually and statistically resembles the power-law or inverse-power decay expected for Hi-C contact probability; fit to power law should be monotonically decreasing with coefficient of determination (R²) > 0.95.

Limitations

• The API for smoothing P(s) and its derivatives is not yet stable in cooltools, meaning function signature, parameter names, or output format may change across versions.
• Smoothing algorithm details (kernel type, bandwidth selection, edge handling) are not fully documented; users cannot easily customize or replicate smoothing independently.
• Log binning inherently sacrifices resolution at long genomic distances; short-range contacts may be underrepresented if bin width is too coarse.
• The skill assumes a unimodal, monotone-decreasing contact decay; it may produce artifacts or misleading smoothing for complex, multimodal distance-frequency distributions (e.g., if topologically associating domains create secondary peaks).

Evidence

• [other] Load precomputed expected_cis table (TSV format with columns dist_bp, contact_frequency, n_valid) from a deposited cooler-derived dataset.: "Load precomputed expected_cis table (TSV format with columns dist_bp, contact_frequency, n_valid) from a deposited cooler-derived dataset."
• [other] Apply logbin_expected function with logarithmic binning to group distance values into log-spaced bins and smooth contact frequency within each log bin.: "Apply logbin_expected function with logarithmic binning to group distance values into log-spaced bins. 3. Smooth contact frequency within each log bin using the smoothing algorithm provided by the"
• [other] Generate output table containing binned distance (dist_bp), mean smoothed contact frequency (count.avg.smoothed), and bin statistics.: "Generate output table containing binned distance (dist_bp), mean smoothed contact frequency (count.avg.smoothed), and bin statistics."
• [discussion] New functionality for smoothing P(s) and derivatives (API is not yet stable): "New functionality for smoothing P(s) and derivatives (API is not yet stable)"
• [other] Cooltools includes new functionality for smoothing P(s) and its derivatives, though the API for this smoothing mechanism is not yet stable.: "Cooltools includes new functionality for smoothing P(s) and its derivatives, though the API for this smoothing mechanism is not yet stable."