Google Introduces TabFM: A Foundation Model for Tabular Data with Zero-Shot Learning
Google/DeepMind
Google Research has unveiled TabFM, a foundation model for tabular data that uses zero-shot in-context learning to perform classification and regression without manual training or feature engineering. The model is trained entirely on hundreds of millions of synthetic datasets generated via structural causal models and outperforms traditional algorithms like XGBoost on the TabArena benchmark. TabFM will be integrated into Google BigQuery, allowing users to run predictions with a simple SQL command.
Google Research has introduced TabFM, a new foundation model specifically designed for tabular data classification and regression. The model uses an in-context learning (ICL) approach, treating the entire dataset as a unified prompt to interpret relationships between columns and rows at inference time, eliminating the need for hyperparameter tuning and feature engineering. TabFM synthesizes strengths from architectures like TabPFN and TabICL into a novel hybrid design to handle the two-dimensional, orderless nature of tables. Trained entirely on hundreds of millions of synthetic datasets generated using structural causal models (SCMs), TabFM generalizes well to unseen real-world data. Evaluated on the TabArena benchmark across 38 classification and 13 regression datasets, it consistently outperforms heavily tuned supervised algorithms like XGBoost. TabFM will be integrated into Google BigQuery, enabling users to perform advanced regression and classification with a simple AI.PREDICT SQL command.
- Сокращения
- TabFM = Tabular Foundation Model — Фундаментальная модель для табличных данных
- ICL = In-Context Learning — Обучение в контексте
- SCM = Structural Causal Model — Структурная каузальная модель
- LLM = Large Language Model — Большая языковая модель
Source: Google Research —
original