Platform Is Not an End in Itself: When to Unify ML Development and When to Stop, Discussed at 'IT Picnic'
At the 'IT Picnic' conference, Sergey Kuznetsov, head of MWS recommendation and search platform development, shared insights on when to build a full ML platform versus when to stop at a library level. He illustrated with three cases: unifying recommendation systems, scoring models, and dynamic pricing. The key takeaway is that a platform should only be built if it justifies its own costs through repeated tasks.
Sergey Kuznetsov from MWS (part of MTS) discussed the evolution of ML development, noting that as teams grow, they accumulate practices that can be turned into libraries and then platforms. He presented three cases: first, unifying about ten recommendation systems into a single platform, which standardized data formats, experiments, and pipelines; second, using a platform for scoring models to automate data preparation and training, which accumulated thousands of verified features over time; third, a failed attempt to create a universal dynamic pricing platform, which was deemed too costly compared to developing a custom solution for a large client. He emphasized that platformization carries risks and costs, and it's only worthwhile when the same process repeats frequently. The most valuable assets are not specific models, which change quickly, but the infrastructure, tools, and interfaces that persist.
- Abbreviations
- ML = Machine Learning — машинное обучение
- MWS = MTS Web Services — веб-сервисы МТС
Source: Habr — хаб ML —
original