Google Cloud announced at PyTorch Conference 2026 that TorchTPU, developed together with Meta's PyTorch team, is now open source on GitHub. According to Google, TPUs are now a native PyTorch device option alongside GPUs and CPUs, and existing code should run with few or no changes.
The announcement is signed by Mark Lohmeyer, Google Cloud's VP for AI and compute infrastructure, and Bill Jia, its VP of engineering. The repository sits under the google-pytorch organization, while the accompanying vLLM and SGLang backends live in those projects' own organizations.
One open-source release, a string of frameworks
TorchTPU supports eager-mode debugging and can also go through torch.compile to raise TPU utilization and cut memory overhead. A number of upstream and downstream projects were connected on the same day:
- Inference: vLLM and SGLang each get a native PyTorch backend. The former supports continuous batching, quantization and multi-chip parallelism, while the latter uses RadixAttention to reuse the KV cache. llm-d 1.0, released together with Red Hat and IBM, handles prefix-cache-aware routing and prefill/decode disaggregation.
- Training: TorchTitan has validated models including Llama 3 70B/405B, Qwen 3 8B/32B, Qwen 3 235B (MoE) and FLUX.1 12B. The reinforcement-learning framework VeRL has merged a TPU backend and a GRPO reference recipe, currently in preview.
- Ecosystem: Hugging Face Transformers v5 loads models onto TPU in one line with
device_map="tpu", and a nightly TPU continuous-integration run has been set up for Transformers. Ray can combine CPU, GPU and TPU in a single pipeline.
Google gives two sets of performance numbers. TorchTitan reaches 52% MFU on Qwen 3 8B and scales nearly linearly on up to 8,192 Ironwood (TPU v7x) chips. Partner Zenteiq, using llm-d with GKE, raised daily generation per 1,000 TPU v6e chips from 1.47 billion tokens to 5.97 billion, an increase of more than 4x. Both sets come from tests run by Google or its partners, and the announcement does not include the full configurations needed to reproduce them.
Why Meta is on stage
Jana van Greunen, Meta's director of PyTorch engineering, offered a quote in the announcement:
"We cannot be constrained by a single hardware architecture."
She said TorchTPU already supports "critical production scenarios" across Meta's large models, ads and recommendation systems. How many TPUs are involved, and what share of Meta's compute they represent, was not disclosed this time. Hugging Face, Inferact (behind vLLM) and RadixArk (behind SGLang) also issued statements, all centered on hardware diversity.
The same week, AWS delivered its own
Put TorchTPU next to TorchNeuron, which AWS announced the day before, and the two companies are clearly taking the same road. TorchNeuron is a native PyTorch backend for Trainium, open-sourced under Apache 2.0 and in public beta since October 20. It supports Trainium2 and Trainium3, and users only need .to('neuron'). It likewise fronts Transformers v5 and TorchTitan, and the same Meta PyTorch engineering director wrote an endorsement for AWS too.
The difference is in the details. AWS published its test methodology: a blind test across 100,000 Hugging Face models, running PyTorch's upstream tests for about 154,000 results, of which roughly 39,000 were skipped deliberately by Neuron, covering features the first release does not support such as sparse tensors and dynamic shapes. Google lists only what is supported; the announcement has no description at the same level of detail about which operators fall back to the CPU or which features are missing.
Both rely on the device-agnostic interfaces the PyTorch community has built in recent years, such as the PrivateUse1 extension point and torch.accelerator. Until now, PyTorch users who wanted TPUs mostly relied on PyTorch/XLA, which translates through the XLA compiler, and its debugging experience has long drawn complaints. With a native backend, the migration barrier drops from "change your code" to "change the device name".
For developers, the most practical test is the set of models that are not on the officially validated list. The TorchTPU documentation names only v6e and Ironwood as supported TPU generations; whether older generations work, and how they perform, Google has not yet said. The accompanying TPU agent plugin is also still in early preview.
Sources: Google Cloud official blog, PyTorch official blog (TorchNeuron announcement), CocoLoop, TorchTPU GitHub repository. MFU and throughput figures reflect Google's and its partners' own testing.