M3.1 Flash Scores 61.7 on OSWorld, But Results Lack Independent Review

Chinese AI company MiniMax published the benchmark results and technical specifications of its M3.1 Flash model, after launching it on Monday as a faster and cheaper update to its M3 model released last June.

The company said the model reduces the cost of running agent tasks by 70% compared to the previous version, with inference being nearly three times faster, reaching a peak daily throughput of over 200 tokens per second, according to SiliconAngle.

The model features 428 billion parameters, offers five levels of inference effort from low to maximum, accepts text, image, and video inputs, and can handle prompts up to one million tokens. The company attributed the extra speed to increased pruning of unnecessary weights during inference.

In the company's own benchmarks, M3.1 Flash scored 61.7 points on OSWorld V2, 76.2 on BankerToolBench, and 82% on CyberGym, and these results have not been independently reviewed. The company set pricing at 15 cents per million input tokens and 60 cents per million output tokens, with the model offered via the company's API, M Plan subscription, and the MiniMax Code application, and it has not yet released the model's weights.

What do these terms mean?

Parameters: The internal numerical values learned by the model during training, and the model size is usually measured by their count.

Inference: Running the trained model to answer a prompt or perform a task, which is the stage where it consumes computing power.

Weights: The numerical values learned by the model that determine how it generates responses; not releasing them means the model is not available for local use.

Token: The smallest text unit processed by the model, which may correspond to a word or part of a word, used to measure prompt size and cost.