Probability distribution inverse sampling

probability-distribution-inverse-sampling

Summary

Use inverse complementary cumulative distribution function (CDF) sampling to probabilistically select a fixed number of compounds from a scored population, where compounds with higher scaled similarity scores have proportionally higher selection probability. This enables stochastic filtering that preserves high-quality compounds while maintaining diversity across generations of reaction network expansion.

When to use

When you need to reduce a large compound set generated by reaction rule application while (1) retaining compounds ranked highest by Tanimoto similarity to target structures, (2) avoiding hard thresholds that may lose borderline-useful compounds, and (3) maintaining stochastic diversity to explore multiple regions of chemical space across multiple generations.

When NOT to use

• You require deterministic, reproducible filtering: inverse sampling is stochastic and will produce different results across runs unless random seed is fixed.
• You must guarantee that all compounds above a known threshold are retained: inverse CDF sampling can exclude high-scoring compounds if sample_size is small relative to population size.
• You lack target compound fingerprints or Tanimoto similarity scores are not meaningful for your chemical space (e.g., scaffolds with no analogues in the target set will have near-zero scores and near-zero selection probability).

Inputs

• compound set (SMILES strings or RDKit molecule objects)
• target compound fingerprints (RDKit fingerprints, typically Morgan or similar)
• sample_size (integer, number of compounds to retain)
• weighting exponent (numeric, default 4 for T^4 scaling)

Outputs

• filtered compound set (subset of input, size = sample_size)
• selection probability distribution (scaled similarities used for sampling)

How to apply

For each compound in the population, compute its maximum Tanimoto similarity score to a target compound set using RDKit fingerprints. Scale each similarity score using a weighting function (default T^4, where T is the Tanimoto coefficient) to emphasize high-similarity compounds. Compute the complementary CDF (1 - CDF) of the scaled distribution, then use inverse CDF sampling to draw sample_size compounds without replacement. Compounds with higher scaled scores have higher cumulative probability mass and thus higher selection probability, but lower-ranked compounds retain a nonzero chance of selection proportional to their score.

Related tools

• RDKit (compute Tanimoto similarity scores between compound fingerprints and apply molecular fingerprinting) — https://rdkit.org/docs/api-docs.html
• pytest (write and execute unit tests to verify correct similarity calculation, weight scaling, and inverse CDF sampling behavior) — https://docs.pytest.org/en/stable/
• Python (scipy or numpy) (implement inverse complementary CDF sampling and probability distribution operations)

Evaluation signals

• Verify that compounds with higher maximum Tanimoto similarity scores have higher selection probability under the T^4 weighting function.
• Confirm that exactly sample_size compounds are retained after filtering (no duplicates, no under/over-selection).
• Check that the complementary CDF is correctly computed from scaled scores (i.e., CDF values are monotonically increasing from 0 to 1).
• Validate that inverse sampling respects the scaled distribution: compounds selected multiple times across independent runs should show frequency proportional to their scaled score.
• Test edge cases: compounds with zero similarity, empty target set, sample_size >= population size.

Limitations

• Performance degrades for very large compound sets (>1M) due to pairwise fingerprint similarity computation; consider batch processing or approximate nearest-neighbor methods.
• The T^4 default weighting function is arbitrary and may require tuning for different chemical spaces; weak discriminators (e.g., highly similar target set) may compress the distribution and reduce effective filtering.
• Stochastic sampling introduces variance across runs; reproducibility requires fixed random seed, which may complicate parallel or distributed reaction network generation.
• Inverse CDF sampling requires fully sorted or cumulative probability computation; not suitable for streaming or online filtering scenarios.

Evidence

• [other] scales each compound's maximum Tanimoto similarity score using a weighting function (default T^4): "scales each compound's maximum Tanimoto similarity score using a weighting function (default T^4)"
• [other] apply inverse complementary CDF sampling to select N compounds (sample_size) from the scaled distribution for the next generation: "apply inverse complementary CDF sampling to select N compounds (sample_size) from the scaled distribution for the next generation"
• [intro] This tanimoto score is scaled and then the distribution is sampled by inverse complementary distribution function sampling to select N compounds: "This tanimoto score is scaled and then the distribution is sampled by inverse complementary distribution function sampling to select N compounds"
• [other] compute the maximum Tanimoto similarity for each compound to the target set using RDKit fingerprints: "compute the maximum Tanimoto similarity for each compound to the target set using RDKit fingerprints"
• [intro] Specified filters are applied before each generation (and at the end of the run if specified) to reduce the number of compounds to be expanded: "Specified filters are applied before each generation (and at the end of the run if specified) to reduce the number of compounds to be expanded"