The local inference library llama.cpp has shipped v1.0.0, and its underlying tensor library, ggml, has moved to 1.0 alongside it. In a post on the Hugging Face blog, the project team said the release is meant to give dependent developers a steadier foundation: from now on, any incompatible change to the public API or ABI in llama.cpp or ggml will show up as a major version bump.
For the past few years, llama.cpp has been released almost entirely through build numbers prefixed with "b". Every merge into the main branch produced a new build, and the counter now stands above b12000. The build numbers do not go away after 1.0 and will keep appearing as pre-releases. What changes is that there is now a formal version line that comes with a commitment, and users can choose which one to follow.
Version numbers now carry meaning
The rules in the blog post are plain. MAJOR corresponds to breaking changes in the public C interface under the ./include directory, MINOR covers backward-compatible new features or support for new models, and PATCH covers bug fixes that leave the interface untouched. The scheme applies to every project under the ggml-org umbrella. The team wrote that as more people depend on the libraries, it will be stricter about keeping those boundaries visible.
To get off to a clean start, 1.0 includes a round of housekeeping: a batch of public interfaces that had long been marked deprecated were removed, some functions were renamed, and new fields were added to the context parameter struct. On the ggml side, deprecated APIs and the old DIAG_MASK operator were also dropped. For developers who link llama.cpp into their own products as a library, this upgrade means changing code once. After that, following minor releases should no longer carry the risk of an interface vanishing in a particular build.
What else is in this release
Beyond the versioning rules, 1.0 itself brings a number of updates:
- The KV cache now supports FP8, and there is a new MoE expert cache held in host memory, with a configurable share of work handed to the CPU;
- Support has been added for models including Kimi-K3.1-Pro, and Kimi-K3.1 can handle image input;
- llama-server gains a /v1/audio/speech text-to-speech endpoint, to be used with the drama-3 TTS model;
- The server has fixes for SSRF, CORS and DNS rebinding issues. Tool calls without authentication and access to private URLs now have to be switched on explicitly, and HF_TOKEN is sent only to huggingface.co.
The last item matters most to people who run their own servers. llama-server is often mounted on an internal network as a local OpenAI-compatible endpoint, and sometimes exposed directly to the public internet. If tool calling is open by default, the attack surface grows with it. Requiring an explicit opt-in hands the choice back to whoever is deploying it.
The MoE expert cache is aimed at the shape of today's models. Most large models this year use a mixture-of-experts design, with total parameter counts often in the hundreds of billions and only a small fraction active at each step. When video memory cannot hold every expert, moving the idle ones into host memory and giving the CPU a set share of the compute is a common compromise for running a big MoE model on a consumer graphics card. It has now landed in the main branch, so users no longer need to hunt for a third-party fork.
Set against the data-center side
Inference frameworks have split into two tracks. Frameworks such as vLLM and SGLang target data-center GPUs and compete on batch throughput and concurrency. llama.cpp aims to get models running on phones, laptops and workstations, using quantization to squeeze large models into limited memory and mixing CPU and GPU.
The backend list shows this clearly: CUDA, Metal, Vulkan, WebGPU, AMD's HIP, Intel's SYCL, Snapdragon Hexagon and Moore Threads' MUSA are all on it, and each backend can be built as a dynamic library and loaded at runtime on demand. According to the blog, it already supports more than 150 model architecture variants. Data-center frameworks chase the newest hardware, while llama.cpp chases the long tail of devices, so the two user bases overlap little.
On GitHub the project has roughly 135,000 stars and 25,600 forks. The team is now based at Hugging Face, and the blog thanks HF for the CI compute and also lists hardware support from NVIDIA, AMD, Qualcomm, Arm, IBM and Intel. Whether the versioning promise holds will depend on whether the next few minor releases quietly change the interface.
Sources: Hugging Face blog, llama.cpp GitHub release page, CocoLoop, llama.cpp repository README; the release page was used to verify the new 1.0 features, the scope of the interface cleanup and the backend list.