Error Grouping and Root Cause Analysis (RCA) with AI
OpenAI
The article on Habr discusses the importance of data preparation for AI in QA testing, focusing on grouping failures before root cause analysis. The author demonstrates with playwright-ai/auto-debug that AI debugging without grouping multiplies analysis time. Existing experiments and infrastructure like Allure Report are highlighted for their role in preprocessing.
The article argues that while AI accelerates code development, QA throughput hasn't improved, partly because AI analysis of failures requires context and data preparation. The author runs tests on a simple banner component with a changed locator, causing two test failures. AI analysis (via playwright-ai/auto-debug) provides separate hypotheses for each failure, but without grouping, time is multiplied. Existing RCA experiments (GPTrace, FlakyCat, BuildSheriff, LogSage) show that grouping is often done algorithmically or with AI, but AI preprocessing is costly (e.g., GPTrace took 10 hours for 300k vectors using OpenAI). The article emphasizes the need for grouping failures before AI analysis, and highlights Allure Report as a structured data source that simplifies preprocessing, with built-in grouping by error messages and support for custom categories. In conclusion, AI in testing requires mature data practices, not just model tuning.
- Abbreviations
- RCA = Root Cause Analysis — Анализ корневых причин
- QA = Quality Assurance — Обеспечение качества
- IDE = Integrated Development Environment — Интегрированная среда разработки
- JS = JavaScript — JavaScript
- DOM = Document Object Model — Объектная модель документа
- HTML = HyperText Markup Language — Язык разметки гипертекста
- CSS = Cascading Style Sheets — Каскадные таблицы стилей
Source: Habr — хаб ИИ —
original