Ming-Flash-Omni: Key Technologies and Practices of a Unified Full-Modality Model

Guo Qingpei from Ant Group presented the technical architecture and practices behind Ming-Flash-Omni, a full-modality unified model with hundreds of billions of parameters, at QCon Global Software Development Conference 2026 Beijing. The model unifies audio, video, image, and text understanding with generation tasks in a single framework. To address modality unification, the team designed a multi-router mechanism to handle modality-specific expert routing and AnyExperts, which dynamically allocates experts based on token importance, reducing redundancy in image and video tokens. They also introduced a collaborative training strategy with data ratio planning and dynamic learning rate adjustment based on convergence speed. For task unification, they built on a frozen multimodal understanding backbone, adding image generation and speech synthesis capabilities, supplemented by fine-grained visual knowledge through generative segmentation training, a dual-stream information approach for identity preservation, and separate extraction of typographic features to avoid garbled text. Speech generation was enhanced with dialect, emotion, and custom timbre controls, and simultaneous generation of speech, sound, and music. The model achieved state-of-the-art results across modalities, matching or exceeding specialized models, and is the first open-source thousand-billion-parameter full-modality unified model. Engineering optimizations included a full-modality sequence packing scheme and flexible parallel strategies, improving training efficiency by 67%. The team also explored representation-level unification with open-source projects MingTok and MingTok-Audio, demonstrating a unified framework for both image and audio understanding and generation.