MLSecOps: Different Priorities for Different AI Use Cases
GitHub
Cursor
Anthropic
OpenAI
Generative AI is widely used both as a developer tool and as a product component, each with distinct security risks. MLSecOps extends DevSecOps to cover AI-specific threats like prompt injection and data leakage, requiring control points close to the developer for coding assistants and runtime protection for AI-integrated products.
Generative AI has become part of development, used both as a coding assistant (like Copilot, Cursor, Claude, ChatGPT) and as an embedded product component. These scenarios have different risks, control points, and costs of mistakes. MLSecOps is needed to integrate security into the AI system lifecycle, from prompts and code to datasets, models, inference APIs, runtime protection, and production monitoring. In the developer tool scenario, key threats include generation of insecure code, secret leaks, indirect prompt injection, and compliance auditing; controls should be placed near the developer, including an LLM/AI firewall, IDE checks, PR review, and CI/CD as a last line. In the product integration scenario, AI becomes a runtime attack surface with threats like prompt injection and jailbreaks, model extraction, ML supply chain compromise, and attacks on AI agents. Agents should be treated as privileged users with minimal rights and sandboxing; assessing the 'lethal triad' (private data access, untrusted input, external channel) helps evaluate risk. The article emphasizes that traditional AppSec tools miss these AI-specific vulnerabilities, so MLSecOps extends rather than replaces DevSecOps.
- Сокращения
- MLSecOps = Machine Learning Security Operations — операции безопасности машинного обучения
- LLM = Large Language Model — большая языковая модель
- RAG = Retrieval-Augmented Generation — генерация с дополнением извлечением
- API = Application Programming Interface — программный интерфейс приложения
- IDE = Integrated Development Environment — интегрированная среда разработки
- CI/CD = Continuous Integration/Continuous Delivery — непрерывная интеграция/непрерывная доставка
- SAST = Static Application Security Testing — статическое тестирование безопасности приложений
- DAST = Dynamic Application Security Testing — динамическое тестирование безопасности приложений
- AppSec = Application Security — безопасность приложений
- SQL = Structured Query Language — язык структурированных запросов
- XSS = Cross-Site Scripting — межсайтовый скриптинг
- SSRF = Server-Side Request Forgery — подделка запросов на стороне сервера
- PR = Pull Request — запрос на слияние
- CISO = Chief Information Security Officer — директор по информационной безопасности
- HTTPS = Hypertext Transfer Protocol Secure — защищённый протокол передачи гипертекста
- TLS = Transport Layer Security — протокол защиты транспортного уровня
- ML = Machine Learning — машинное обучение
- CRM = Customer Relationship Management — управление взаимоотношениями с клиентами
- IPI = Indirect Prompt Injection — косвенная инъекция промптов
- HTTP = Hypertext Transfer Protocol — протокол передачи гипертекста
Source: Habr — хаб ИИ —
original