Baidu releases ERNIE 5.5 Flash Base, a 280-billion-parameter open model

Baidu released the ERNIE 5.5 Flash Base model as an open foundation model, without fine-tuning or post-training, according to what The Decoder published.

The model relies on a Mixture of Experts architecture with a total of 280 billion parameters, activating only 14 billion parameters per input, meaning that only part of the parameters works with each input.

Baidu trained the model on 33.5 trillion tokens, and made its weights available on the Hugging Face platform in FP8 and BF16 formats, with a post-trained version to follow.

Open models allow researchers and companies to build their own applications without training from scratch, which makes large foundation models part of the competition among AI companies.

What do these terms mean?

Foundation model: A large artificial intelligence model trained on public data on which specialised applications are then built.

Parameters: The internal numerical values the model learns during training and that determine its behaviour.

Mixture of Experts (MoE): A design that runs only part of the model's components for each input instead of running the whole model.

Fine-tuning: Additional training of a ready model on specialised data to improve its performance on a specific task.

Token: A small unit of text that the model processes, which may be a word or part of a word.

Model weights: The files containing the trained values that allow the model to be run or reused.