TabFM vs XGBoost: 5 Facts Not in the Release
Google/DeepMind
This article analyzes the new TabFM model from Google Research, which performs table predictions without training. The author checks the claims, reveals limitations, and shares independent benchmarks. The key takeaway: zero-shot TabFM can outperform tuned gradient boosting, but hidden caveats like context size limits and synthetic training data need consideration.
The article by Sergey Proshchaev, a Tech Lead from FinTech & E-commerce, discusses TabFM, a model released by Google Research on June 30, 2026. TabFM reads a table as a single context, using frozen weights and in-context learning, eliminating the need for training on your data. It is the successor to TimesFM for time series and follows the line of TabPFN (2022). The model was pre-trained on hundreds of millions of synthetic datasets generated by structural causal models, not on real corporate data. The article highlights that the scikit-learn wrapper has default limits of 500 features and 100 context rows, so the model samples if the table is larger. Independent benchmarks by Yash Raj Pandey show TabFM outperforming tuned XGBoost on small and medium tables, but some wins vanished when considering multiple seeds. The article notes memory issues: the claimed 22.75 GB of video memory was mostly an XLA allocator artifact, with real usage around 16.95 GB; enabling bf16 and activation chunking in PyTorch reduced memory to 3-7 GB. The author suggests a proper evaluation protocol with multiple seeds and equal budgets for hyperparameter tuning. The article concludes that while TabFM shows promise, its behavior on dirty corporate tables and distribution drift is uncharacterized, so validation on your own data is essential.
- Abbreviations
- XGBoost — eXtreme Gradient Boosting
- CatBoost — CatBoost (gradient boosting on decision trees)
- LightGBM — Light Gradient Boosting Machine
- TabFM — Tabular Foundation Model
- TabPFN — Tabular Prior-Data Fitted Network
- TabICL — Tabular In-Context Learning
- CARTE — CARTE model for tabular data
- CLS = Classification Token — токен классификации
- SVD = Singular Value Decomposition — сингулярное разложение
- MNC = Non-negative Least Squares — неотрицательный метод наименьших квадратов
Source: Habr — хаб ИИ —
original