Self-hosting vs API: how to calculate the real cost of a solved task in the era of open models

The author criticizes the common practice of comparing self-hosting and API costs purely on a per-token basis, pointing out that self-hosting is a fixed cost while API is variable, and that the relevant metric is cost per solved task, not per token. Open models have caught up in quality on benchmarks like SWE-bench Verified, making the choice an economic one. Part 1 derives a break-even analysis: blending input/output token prices for API (e.g., Claude Opus 4.8 at $15.00/M tokens for long generation, Kimi K3 at $9.00, DeepSeek V3.2 at $0.286) and estimating self-hosting costs for an 8×H200 node (amortized ~$7.7-10.3k/month plus electricity ~$580-880/month). Break-even occurs when monthly volume V exceeds cost_per_million_self / (blended_price_api - marginal_cost_self), but also requires sufficient throughput capacity; low utilization makes API cheaper. Part 2 introduces cost per solved task = (token_budget_per_attempt * cost_per_token) / solve_rate, using SWE-bench Verified scores and a fixed 1.5M token budget. DeepSeek V3.2 via API costs $0.49 per task, while self-hosted at given utilization costs $4.37, showing that cheap API beats self-hosting for the same model. Self-hosting only wins against expensive closed APIs like Claude Opus 4.8 ($11.54) or Kimi K3 ($6.57) at high utilization. Part 3 notes non-cost factors: data residency (forces self-hosting), vendor lock-in and price volatility (e.g., Moonshot AI paused Kimi K3 signups after 48h due to demand), asymmetry of updates (API gets free upgrades, self-hosting costs effort), and hardware depreciation tied to NVIDIA release cycles rather than accounting schedules. The conclusion is that the API vs self-hosting decision depends on infrastructure utilization, data requirements, and operational constraints, not token price alone.