clauding.de
DE
• 2 min read AI-generated

AMD Instella-MoE: A Fully Open MoE Model — Trained on AMD GPUs

Copy article as Markdown

AMD released Instella-MoE-16B-A3B: a Mixture-of-Experts model that opens up not just its weights, but every training stage, the data mixtures, and the code. And it was trained entirely on AMD's own Instinct GPUs — which is the real headline.

Open Source AMD LLM
Featured image for "AMD Instella-MoE: A Fully Open MoE Model — Trained on AMD GPUs"

Amid all the frontier models from Anthropic, OpenAI and Google, it’s easy to miss a release coming from a very different corner: AMD has unveiled Instella-MoE-16B-A3B — a fully open Mixture-of-Experts model. And for once, “fully open” really does mean fully.

What’s inside

Instella-MoE is a MoE model with 16 billion parameters in total, but only around 2.8 billion active per token. That’s the classic MoE trick: the capacity of a large model at the runtime cost of a small one. Architecturally, AMD leans on a few innovations like Gated Multi-head Latent Attention and a technique called FarSkip-Collective to make training and inference more efficient.

On benchmarks, the base model lands at an average score of 76.7, beating open models like SmolLM3-3B and OLMo-3-7B. Not a frontier killer — but solid for its class. There are three variants: Base, SFT, and a “Think” version for reasoning.

The real point: openness

What sets Instella-MoE apart isn’t the score, it’s the level of transparency. AMD is publishing the weights from every training stage — not just the finished model. On top of that come the data mixtures, the training configs, and the inference code. For researchers that’s gold: you can trace exactly how the model came together step by step, instead of just downloading a black box.

One catch: the weights ship under a ResearchRAIL license — research and academic use only, not a drop-in for products. The training code, on the other hand, is MIT-licensed and free to use.

Why it matters

The most interesting part is almost buried in the fine print: Instella-MoE was trained end to end on AMD’s own Instinct GPUs (MI300X and MI325X) with the ROCm stack. That’s a statement. The entire AI boom runs on NVIDIA hardware, and here AMD shows it can build a competitive, fully open model from scratch entirely on its own platform.

For us as users, a single 16B model isn’t the news. The news is that the market for training hardware looks a little less like a monopoly. More competition among GPUs eventually means more affordable models for everyone. And a model that lays out its entire path from scratch is exactly the kind of contribution the open AI world runs on.

Sources: