Why Doesn’t My Model Work?

The article, based on the author's 20 years of ML experience and the paper 'How to avoid machine learning pitfalls: a guide for academic researchers', outlines common mistakes that lead to models failing in production. First, misleading data: hidden variables (e.g., body orientation in Covid chest scans) and spurious correlations (e.g., background pixels in tank images) cause models to learn irrelevant patterns. Adversarial attacks exploit these spurious correlations. Labeling errors, even at a few percent, can distort benchmark comparisons. Second, data leakage: performing data-dependent preprocessing (centering, scaling, feature selection, data augmentation) on the full dataset before splitting test data leaks information, inflating performance. Look-ahead bias in time series forecasting and iterative overfitting of the test set (training to the test set) are common leaks. Third, inappropriate metrics: using accuracy on imbalanced data can be misleading. The author recommends the REFORMS checklist for ML-based science and practices like examining saliency maps, using multiple benchmarks, and ensuring proper data splits.