PolyAI Releases Dialog-RSN-1: An Audio-Native Dialog Model That Fuses Turn-Taking, Speech Recognition, Function Calling, And Response

PolyAI has introduced Dialog-RSN-1, a dialog model that perceives the caller's audio directly instead of reading a transcript. It fuses turn-taking, speech recognition, function calling and response generation into one audio-native model, and is already handling live production calls. Key takeaways include that Dialog-RSN-1 is audio-aware on the input side only; TTS stays separate, so the output voice remains controllable. It runs as a request-based LLM probed on demand, not an always-on stream that pins a GPU. Turn-taking is the model's first output token: EMPTY, ONGOING, or COMPLETE. PolyAI reports sub-300ms responses, +11% relative containment at a restaurant group, and -37% latency at an insurer. English only at launch, delivered through PolyAI's platform rather than open weights or a public API. The model is deployable only through PolyAI; existing customers can enable it today, new customers can request early access. Target customers are large, high-call-volume enterprises, with 100+ enterprise customers and 2,000+ live deployments at its $86M Series D in December 2025. Industries include restaurants, insurance, financial services, healthcare, hotels, retail, telecom, travel and utilities. Applications include booking and reservations, billing and payments, authentication, call routing, order management and troubleshooting. Dialog-RSN-1 is audio-aware on input only: one model reasons over raw audio and hands generation to a separate, promptable TTS system. It is probed on demand rather than streamed: a high-recall VAD plus a few timers decide when to run it, and the first token of the reply settles whether the agent should speak. The model was post-trained from open-weight multimodal models with supervised and reinforcement finetuning on in-house data. The pipeline is broadly base-model agnostic; PolyAI evaluated Gemma, GPT-OSS, Qwen and Mistral. Targeting sub-300ms on A100 GPUs puts candidates in the 8B dense to 30B sparse range. Latency work includes prefilling the attention cache while the user speaks, an append-only prompt template, routing each caller to the same GPU, a finetuned speculative drafter with mean acceptance of 3.9 tokens, and auto-reasoning learned during RFT. Transcription runs last, after the response or tool call, in parallel with speech generation. PolyAI evaluated on Dialog-Eval, an internal benchmark it plans to open-source. Each example is a call truncated at one decision point, scoring a single atomic next step. Dialog-RSN-1 is the highest-scoring real-time capable model, the cascaded Audio-score ceiling is near 77, and GPT Realtime 2.1 scores on par with cascades on audio-aware examples. On transcription, gpt-4o-transcribe's WER improved from 7.8% to 6.9% once given the same context, with Dialog-RSN-1 lower still. For this release PolyAI focused on English; Raven 3.5 remains its recommendation for non-English and rich web chat. A technical report and a Dialog-Eval paper are planned.