Spearman correlation statistical analysis

Spearman-Correlation Statistical Analysis

Summary

Compute Spearman rank correlation coefficients (SCC) between predicted and observed metabolite abundances to quantify prediction accuracy across cross-validated folds. This non-parametric correlation metric is essential for evaluating microbiome-metabolome prediction models when the underlying relationship may be monotonic but not necessarily linear.

When to use

You have paired predicted and observed metabolite abundance vectors from a predictive model (e.g., neural network, Elastic Net, Random Forest) evaluated over multiple cross-validation folds, and you need to measure how well the model's predictions rank-correlate with ground truth, accounting for potential non-linearity and robustness to outliers in compositional data.

When NOT to use

• Input is qualitative (presence/absence) rather than quantitative relative abundance; use rank-based association measures instead.
• Sample size is extremely small (n < 10) and cross-validation folds cannot be meaningfully stratified.
• Metabolite abundances are already log-ratio transformed and you require raw correlation interpretation (document the transformation in results).

Inputs

• Predicted metabolite abundance matrix (samples × metabolites)
• Observed metabolite abundance matrix (samples × metabolites)
• Cross-validation fold assignments or iteration metadata

Outputs

• Mean Spearman correlation coefficient (SCC) per model ± standard deviation
• Per-metabolite SCC values across all cross-validation iterations
• Background SCC distribution (from shuffled data, ≥100 iterations)
• 95th percentile threshold from background distribution
• Binary classification of well-predicted metabolites (SCC > 95th percentile)

How to apply

For each metabolite across all cross-validation iterations, calculate the Spearman rank correlation coefficient between predicted and observed relative abundances. Aggregate SCC values across metabolites and iterations to compute mean ± standard deviation. Use the mean SCC as the primary performance metric to compare models. Additionally, generate a background SCC distribution by training models on shuffled samples (with randomly reordered sample labels) across the same cross-validation protocol, then define well-predicted metabolites as those with SCC above the 95th percentile of background correlations. This empirical threshold distinguishes truly predictable features from noise-driven spurious correlations.

Related tools

• MiMeNet (Neural network framework that trains models to predict metabolomic profiles from microbiome data and applies SCC evaluation across 10-fold cross-validation iterations) — https://github.com/YDaiLab/MiMeNet
• MelonnPan (Elastic Net linear regression baseline model evaluated using identical Spearman correlation and cross-validation protocol for benchmarking) — https://github.com/biobakery/melonnpan
• scikit-learn (Provides random forest regression and cross-validation utilities; SCC can be computed via scipy.stats.spearmanr)

Examples

from scipy.stats import spearmanr; scc_per_metabolite = [spearmanr(predicted[:, i], observed[:, i])[0] for i in range(predicted.shape[1])]; mean_scc = np.mean(scc_per_metabolite); bg_scc_threshold = np.percentile(background_correlations, 95); well_predicted = [scc > bg_scc_threshold for scc in scc_per_metabolite]

Evaluation signals

• SCC values range from −1 to +1; mean SCC for well-performing models on real data should be substantially higher than the 95th percentile of the background distribution generated from shuffled samples.
• Standard deviation of SCC across iterations should be reasonable relative to mean (coefficient of variation typically <50% for stable predictions).
• The number of well-predicted metabolites (SCC > 95th percentile threshold) should be substantially higher than expected by chance; compare to random baseline.
• External validation datasets (held-out cohorts) should show similar or slightly lower mean SCC compared to internal cross-validation, indicating generalizability.
• Model comparison (e.g., MiMeNet vs. MelonnPan) should show statistically meaningful differences in mean SCC; report effect sizes and confidence intervals across iterations.

Limitations

• Not all metabolites are biologically associated with the microbiome, so some will inherently have low SCC even in well-performing models, lowering the overall mean correlation across all features.
• SCC is rank-based and may mask absolute prediction error magnitude; two models with different error distributions can yield identical correlations.
• The 95th percentile background threshold is empirically derived and may be sensitive to sample size, number of background iterations, and the degree of data shuffling; recommend at least 100 background iterations.
• Centered log-ratio (CLR) or relative abundance transformations alter the correlation landscape compared to raw counts; results are not directly comparable across different normalization strategies.
• The method assumes that shuffled-sample correlations reflect true noise, but strong compositional structure (e.g., spike taxa) can inflate background correlations.

Evidence

• [methods] the predictive performance of a model is measured by the average Spearman correlation coefficients (SCCs) between the predicted and the observed abundances: "the predictive performance of a model is measured by the average Spearman correlation coefficients (SCCs) between the predicted and the observed abundances"
• [results] MiMeNet achieves mean Spearman correlation coefficients that increase from 0.108 to 0.309 (IBD PRISM), 0.276 to 0.457 (Cystic Fibrosis), and -0.272 to 0.264 (Soil) compared to MelonnPan: "MiMeNet achieves mean Spearman correlation coefficients that increase from 0.108 to 0.309 (IBD PRISM), 0.276 to 0.457 (Cystic Fibrosis), and -0.272 to 0.264 (Soil) compared to MelonnPan"
• [methods] We then defined a metabolite as well-predicted if its SCC is above the 95th percentile of the background correlations: "We then defined a metabolite as well-predicted if its SCC is above the 95th percentile of the background correlations"
• [results] MiMeNet then generates a background distribution of SCCs through multiple iterations of shuffling the dataset and performing a cross-validation on the shuffled set: "MiMeNet then generates a background distribution of SCCs through multiple iterations of shuffling the dataset and performing a cross-validation on the shuffled set"
• [abstract] Using ten iterations of 10-fold cross-validation on three paired microbiome-metabolome datasets: "Using ten iterations of 10-fold cross-validation on three paired microbiome-metabolome datasets"
• [discussion] since not all metabolites may be associated with microbes, some metabolites will have lower prediction correlations, which resulted in an overall lower mean correlation across all metabolites: "since not all metabolites may be associated with microbes, some metabolites will have lower prediction correlations, which resulted in an overall lower mean correlation across all metabolites"