From Manual Rules to Model: How to Automate Subsidies Without Dropping A/B Metrics

Avito Podrabotka, a temporary employment service, used to manually assign 'ontop' subsidies to gig workers for less attractive shifts, funded from the platform's budget. As the service grew, manual management became inefficient, leading to overspending in inelastic segments and underspending where subsidies were needed, and the goal was set to reduce SubGMV (subsidization level) without harming business metrics. The team evaluated various approaches including business rules, binary uplift, bandits, reinforcement learning, and Bayesian methods, but chose a price-response model due to constraints: skewed data, limited experiment capability (cannot zero out ontop in tests), low frequency and delayed feedback, and need for interpretability. The model predicts the probability of a slot closing within a specific day, based on price (base rate plus ontop), shift parameters, geolocation, market, and historical platform statistics. To combat data bias, they expanded training data with A/B test data and formulated the task as a three-class classification (contact in group A, contact in group B, no contact), later collapsing to two classes in production. They applied a monotonic constraint on price features to ensure that higher subsidies do not decrease closing probability, implemented via decomposing CatBoost into two binary models. Key evaluation metrics include LogLoss 0.44, Brier 0.29, Fill Rate deviation -0.6 percentage points, and a symmetry gap below 0.01. The model's calibration is crucial for the budget allocation algorithm to distribute subsidies effectively without degrading Completed slots and Fill Rate.