Yelp Launches Training Orchestrator for Unified Management of Machine Learning Model Training

Yelp has unveiled Training Orchestrator, a new internal framework that replaces the independently written Spark training scripts used by various teams. Previously, each applied machine learning team created its own orchestration, leading to code duplication, inconsistent configurations, and fragile custom monitoring. Training Orchestrator separates 'what to run' from 'how to run' using Pydantic-based configuration objects, which are validated at creation time to avoid wasted compute. It builds a directed acyclic graph (DAG) from declared step dependencies and executes steps in topological order. Shared Spark/MLflow context allows steps to run unchanged in local, Jupyter, or production environments. Schema validation catches mismatches in seconds, and complete run configurations are logged as MLflow artifacts for reproducibility. The framework integrates with Yelp's ML platform, including feature stores, shared neural networks, gradient boosting libraries, and MLflow. Yelp plans to add explicit steps for model evaluation, comparison, and explanation, as well as lineage tracking. The DAG-based approach decouples training logic from cluster runtime, supporting local runs and unit tests. Yelp acknowledges upfront investment in schemas and migration but believes it avoids repeated costs across teams.