Parametric nonparametric model selection

parametric-nonparametric-model-selection

License: restricted — no clear open-source license detected for the underlying tool; verify licensing before commercial use or redistribution. <!-- asb-license-banner -->

Summary

Select between parametric ComBat, non-parametric ComBat, two-stage (ber), and bagging-based batch correction models by comparing their performance on metabolomics data using adjusted R² scores and diagnostic plots. This skill enables practitioners to choose the statistical model that best fits their dataset's batch structure before applying correction.

When to use

You have preprocessed, log₂-scaled metabolomics data in CSV format (samples × features) with batch assignment labels and need to decide which batch effect correction model to apply. Use this skill when you want to compare how well parametric ComBat, non-parametric ComBat, ber, and ber-bagging models explain batch-induced variance before committing to one for downstream analysis.

When NOT to use

• Input data is not log₂-scaled or not normalized; preprocessing must precede model selection.
• Metabolomics dataset contains >2000 features; dbnormSCORE and Visodbnorm are recommended for <2000 features for computational speed.
• Batch assignment is unknown or ambiguous; model selection requires clear batch labels in the input data.

Inputs

• preprocessed metabolomics matrix (CSV format: rows=samples, columns=features)
• batch assignment vector (first column of input CSV)
• log₂-scaled abundance data

Outputs

• adjusted R² coefficient matrix for each model and feature
• performance score table ranking models by maximum adjusted R²
• diagnostic PDF (PCA, Scree, RLA, Correlation plots)
• batch-corrected CSV matrices for each tested model

How to apply

Load your normalized, log₂-scaled metabolomics matrix (rows=samples, columns=features, batch labels in first column) into dbnorm. Run dbnormSCORE() to compute adjusted R² for each feature under each of the four models (parametric ComBat, non-parametric ComBat, ber, ber-bagging) in both raw and batch-corrected data. Inspect the resulting Score table to identify which model achieves the highest and most consistent adjusted R² across features, indicating best fit to your batch structure. For datasets with <2000 features, also run Visodbnorm() to visually compare PCA, Scree, and RLA plots across models. Select the model that maximizes variance explained by batch removal while minimizing feature-to-feature inconsistency in performance.

Related tools

• dbnorm (primary package providing dbnormSCORE and Visodbnorm functions for model comparison and batch correction) — https://github.com/NBDZ/dbnorm
• sva (source package providing parametric and non-parametric ComBat models integrated into dbnorm)
• R (execution environment for dbnorm functions and statistical model evaluation)
• ber (archived R package (v4.0) providing two-stage batch effect removal procedure implemented in dbnorm) — http://cran.us.r-project.org

Examples

dbnormSCORE(data); # Generates adjusted R² scores and performance comparison table for all four models
Visodbnorm(data);  # Generates PCA, Scree, and RLA diagnostic plots for model visual comparison (features < 2000)

Evaluation signals

• Adjusted R² should decrease substantially in batch-corrected data compared to raw data, indicating batch variance removal.
• One model should show consistently high adjusted R² across >80% of features, indicating stable model fit.
• PCA plots should show reduced batch clustering and improved inter-batch sample mixing after correction with selected model.
• Score table should identify a clear winner (model with highest maximum adjusted R²) with <20% performance variance across features.
• RLA (Relative Log Abundance) plots should show centered distributions around zero median log-fold-change post-correction.

Limitations

• Model selection is empirical and data-dependent; no single model is universally optimal across all batch structures.
• Computational speed degrades substantially for >2000 features; dbnormSCORE and Visodbnorm are not recommended at that scale.
• Selection depends on correct batch label specification; mislabeled or misaligned batch assignments will produce misleading scores.
• Adjusted R² alone does not account for biological signal preservation; visual inspection of diagnostic plots is essential.
• ber and ber-bagging models were originally developed for microarray data; performance on metabolomics may differ from parametric/non-parametric ComBat.

Evidence

• [readme] dbnormSCORE function provides adjusted coefficient of determination for each variable estimated in a regression model for its dependency to the batch level: "This function gives a quick notification about the performance of the statistical models, two-stage procedure [DOI:10.1007/s12561-013-9081-1] and/or empirical Bayes methods in two setting of"
• [readme] Score table facilitates quick comparison of models for selecting the most appropriate: "Immediately, the performance of applied models are presented by a score calculated based on the maximum variability explained by the batch level, notify the consistency of model performance for all"
• [readme] Visodbnorm visualizes three batch correction models via PCA, Scree, and RLA plots for datasets with <2000 features: "This function performs batch effect adjustment via three statistical models implemented in the dbnorm, namely two-stage procedure as described by Giordan (2013)[DOI:10.1007/s12561-013-9081-1]"
• [readme] Input data must be normalized and log₂-scaled to account for high-abundance features: "Data to be uploaded must be normalized and scaled on the log2 to account for the high abundance features (variables) by which technical heterogeneity might be overlooked."
• [readme] dbnorm implements four distinct batch correction models for model selection comparison: "dbnorm includes several statistical models such as, ComBat(parametric and non-parametric)-model [PMID:16632515] from sva package [PMID:22257669] ,that was already in use for metabolomics data"