Xiaomi open-sourced a speech recognition model called Xiaomi-CocktailASR-1 on September 11, built specifically to solve the 'cocktail party problem': when a room full of people talk at once, the model transcribes only the one person you want to hear. The code sits under the xiaomi-research organization on GitHub, the weights are on Hugging Face, and the technical report was submitted to arXiv on September 10.
Its input includes one extra piece most speech recognition models don't take: a reference audio clip of the target speaker. The model uses that clip as a voiceprint cue, picks that person's voice out of the overlapping audio track, and transcribes it — everyone else's speech produces no text at all.
Numbers
On the multi-speaker test sets, the report gives these word error rates: 4.11% for two-way overlap on LibriMix, 12.29% for three-way overlap on LibriMix3mix, and 2.90% on LibriSpeechMix2mix. In that same three-speaker test, general-purpose speech recognition models scored between 76% and 121%. A rate above 100% means the model inserted more wrong characters than the original speech contained — it blended two or three people's words into a single garbled stream.
For real-world conditions, the team used AliMeeting conference recordings; under far-field conditions the target-speaker word error rate (TS-WER) was 20.63%, nearly seven percentage points below the previous best published result of 27.5%. On clean single-speaker audio, LibriSpeech, it scored 1.73%, roughly on par with mainstream general-purpose models — a result that matters because it determines whether the model can serve as a general ASR tool rather than just a niche component for overlapping speech.
There's also a rejection metric. When the target speaker never actually speaks, the model is supposed to output empty text. On an English negative-sample set, its rejection rate was 79.59%; the report benchmarks this against Gemini-2.5-pro, whose false-rejection rate was 21.31% — meaning it mistakenly judged 'nobody spoke' in cases where it should have produced a transcript.
A Simple Structure, a Large Amount of Data
The architecture has three parts: a self-supervised audio encoder based on Data2Vec2 with roughly 60 million parameters that outputs 1280-dimensional frame-level embeddings; a linear adapter that projects the audio embeddings into the language model's hidden space; and a Qwen3-8B backbone. The whole thing is an end-to-end large language model architecture, not the traditional ASR setup of a separate acoustic model bolted to a language model.
Training ran in four stages; the second stage alone used more than 400,000 hours of multi-speaker data and 600,000 hours of single-speaker data. That scale suggests the model's capability comes mainly from data volume and the coverage of synthesized speech mixtures, rather than from architectural novelty — which lowers the bar for outside reproduction.
The model also supports a chain-of-thought mode, first reasoning out how many people are in the clip and which segment matches the target voiceprint before transcribing. The report says turning on CoT drops the word error rate by another 0.24 percentage points (absolute). In practical terms, that 0.24-point gain is unlikely to outweigh the extra inference cost — the mode's real value is more about interpretability, letting you see where the model's judgment went wrong when it fails.
What It Means for Adoption in China
China's speech recognition market isn't short on commercial products; what's missing is an open-source component that can reliably lock onto one person in a multi-speaker conversation. Meeting transcripts, in-car voice assistants for multi-passenger cabins, dual-recording quality checks in customer service, and hearing devices all run into the same wall: the microphone picks up more than one person.
Xiaomi's own use case isn't hard to guess — voice assistants in cars and IoT devices both have to deal with multiple people sharing the same space. But this time the full weights and code are released as open source, so third parties can run it directly without waiting for some cloud service to open an API. All the test sets in the report are public datasets too, which lowers the bar for others to reproduce the results and poke holes in them.
One caveat: datasets like LibriMix are clean speech artificially mixed together, and don't fully match the reverberation, directionality, and interruptions of a real room. The AliMeeting figure of 20.63% is the more realistic benchmark, and it's still some distance from 'production-ready.' Xiaomi also hasn't separately published the model's results on Chinese-language multi-speaker scenarios.
Sources: Xiaomi open-source repository xiaomi-research/xiaomi-cocktailasr-1, CocoLoop, arXiv technical report 2609.11274, ITHome; word error rate and rejection rate figures follow the technical report.