Mistral Unveils "Le Chonk" as an Open Model with One Trillion Parameters

French startup Mistral AI has unveiled its new "Mistral Large 4" model, nicknamed "Le Chonk", an open-weight model based on the Mixture-of-Experts architecture featuring 1.05 trillion parameters, of which 49 billion are active, according to Open Data Science. The model supports a context window of up to one million tokens.

The model was trained from scratch over roughly two months using around 4,000 Nvidia Grace Blackwell processing units in European data centers, and supports more than 160 languages, including all official European Union languages. A preview version is available via Mistral Studio, with a promise to release the full weights on October 27.

The company says the model leads open models developed outside China, while acknowledging that it still lags behind some frontier systems in coding tasks. Co-founder and Chief Scientist Guillaume Lample believes open models help governments and companies defend against attackers using modified closed models.

The reveal comes as Europe competes to build sovereign computing capacity away from total reliance on US and Chinese providers. Open weights serve as an attractive factor for governmental and research entities, as they allow running the model within their own infrastructure instead of relying on external cloud services.

What do these terms mean?

Mixture of Experts (MoE): An architecture that divides the model into specialized parts, with only a small portion active at a time, reducing operational costs. Parameters: The numerical values learned by the model during training; the higher their count, the greater its capability along with higher operational costs. Open weights: Publishing the trained model's files so any entity can run and modify it within its own infrastructure.