Model Is Not to Blame: Analysis of Three High-Profile AI Incidents Due to Lack of Architecture

An engineer designing LLM-based systems reviews three incidents. First: Deloitte Australia used AI for a government report; part of the footnotes turned out to be nonexistent or attributed to wrong authors, despite internal reviews. The reason is that checking relied on text fluency rather than mechanical source verification. Second: in Minnesota, police using Flock camera network detained journalist Joel Feder because an incomplete license plate (missing small digits) was entered into the database, and a partial match triggered as exact without mandatory officer verification. Third: a founder of an indie SaaS discovered that model-written code executed via cron overnight in 7 seconds canceled all active subscriptions, slashing MRR to $38 (only two test accounts survived). The code had permissions for irreversible bulk operations without oversight. The author proposes solutions through architectural principles: control in deterministic code rather than in prompts, validation at boundaries, and human-in-the-loop as a mandatory code path. For the Claude API, examples are shown with citations, structured outputs, and tool use.