Taobao's MNN team open-sourced MNN-Pocket-0.3B, an on-device language model, on November 6. The model has roughly 300 million parameters, focuses on Chinese and English, and is built to run inside mobile apps. The team says everything, from architecture, vocabulary and training data to quantization and the inference engine, was designed together and trained from scratch.
Running fully offline on an iPhone 18 Pro, it averages about 350 tokens per second in decoding. The INT4 deployment file is 173MB and includes the weights, tokenizer, compute graph and configuration.
Set the targets first, then work backward to the model
The team began by setting four hard targets for itself: decoding faster than 200 tokens per second; peak memory below 300MB; energy per inference no more than one ten-thousandth of the reference phone's battery, meaning 100 runs should use no more than 1% of charge; and the highest capability possible within those limits.
The parameter count was derived backward. The team tried a range of dense configurations from about 50 million to 1.3 billion parameters, and found that roughly 300 million was the largest size that met both the memory and speed constraints. Going from 16 to 28 layers improved capability; at 36 layers the score stopped rising and decoding speed fell from 355 to 323 tokens per second, so the team settled on 28 layers. The vocabulary was trimmed from the Qwen3 tokenizer's roughly 151,000 entries to 64,000, a cut of about 57%, which also lowered the cost of the embedding and output layers.
Benchmarks and comparison
Comparison on an iPad Pro M5 (Metal backend, 512-token configuration):
| Test | MNN-Pocket-0.3B | Qwen3-0.6B |
|---|---|---|
| Decoding speed | 466 tokens/s | 286 tokens/s |
| INT4 package | 173MB | 340MB |
| Peak memory increase | 228MB | 427MB |
On the CPU backend of a Snapdragon 8 Gen 5, decoding reaches 251 tokens per second. For the energy test, the team used a OnePlus 13 with a 4-thread CPU, a 1,024-token input and a 128-token output, at about 2.7 joules per run. Converted against a 6,000mAh battery, 100 runs use roughly 0.3% of charge, well below the team's own 1% ceiling.
On capability, the model scores 73.20 on English IFEval, 74.59 on the Chinese instruction-following test IFEval-zh, 51.20 on the Chinese math test MGSM-zh and 53.66 on HumanEval. Across 11 benchmarks covering knowledge, reasoning, math, coding and instruction following, it averages 50.00, which the team says is higher than MiniCPM4-0.5B and Qwen3-0.6B. IFEval-zh is an evaluation set the team built itself, and no external reproduction exists yet.
Six versions have been released in total: Base, SFT, an instruction-tuned version, an on-device version trained with W4A16 quantization, DFlash2, a draft model for speculative decoding (up to 2.06 times faster decoding on certain tasks), and Dec, a decision head for ranking candidates. The code is in the alibaba/MNN repository on GitHub.
What it means for app developers in China
MNN is Alibaba's in-house on-device inference engine, open-sourced in 2019, and apps such as Mobile Taobao have used it to run local models for years. With the model and engine now released together, Chinese app teams get a complete package they can drop straight into a client, without having to trim, quantize and speed-tune an open-source large model themselves.
A 300-million-parameter model cannot handle complex reasoning. It suits high-frequency small tasks such as intent recognition, short-text rewriting, search-term completion and candidate ranking. If these requests used to go to the cloud, each one carried network latency and API fees; run locally, they work offline and user input never leaves the phone. At 173MB, the package is still a burden for e-commerce and utility apps, and downloading it on demand may be the more realistic approach.
Of the six versions, one is a dedicated Dec decision head, which suggests the team's target use cases lean toward ranking and selection tasks. The release materials do not say which Taobao features already use this model.
Sources: ITHome, CocoLoop, the MNN open-source repository; verified the decoding speeds on iPhone 18 Pro and iPad Pro M5, the INT4 package size, and how the benchmark scores were measured.