--- title: "A pre-training metric is not product capability, and this model cannot even be called yet." url: https://wakenex.com/notes/a-pre-training-metric-is-not-product-capability-and-this/ language: en type: note section: "Systems Research" published: 2026-10-18T10:44:13+07:00 author: "WAKENEX" id: "note:a-pre-training-metric-is-not-product-capability-and-this" x_post: https://x.com/AAwakened32198/status/2111664620754845833 alternate_language: https://wakenex.com/notes/a-pre-training-metric-is-not-product-capability-and-this/index.zh.md license: "© WAKENEX. You may quote and summarise with attribution and a link to the canonical URL." cite_as: "WAKENEX. \"A pre-training metric is not product capability, and this model cannot even be called yet.\" wakenex.com, 2026-10-18. https://wakenex.com/notes/a-pre-training-metric-is-not-product-capability-and-this/" --- # A pre-training metric is not product capability, and this model cannot even be called yet. **In one line:** A pre-training metric is not product capability. **WAKENEX concept:** [Results alone don't prove capability](https://wakenex.com/concepts/results-dont-prove-capability/) ![A single small figure stands at the foot of a vast bare concrete wall at night; one low floodlight throws its shadow, hugely enlarged, across the whole wall. The reach of the shadow dwarfs the small figure casting it. AI-generated image; a concept, not evidence.](https://wakenex.com/assets/notes/a-pre-training-metric-is-not-product-capability-and-this/cover-1600.jpg?v=9c04c074b4) *AI-generated image · concept, not evidence* Microsoft AI said on Oct 15 it finished pre-training MAI-Base-2, and its chief, Mustafa Suleyman, told an internal town hall it beats the base model behind GPT-5.6 Sol using about a quarter of the compute. The headline writes itself: Microsoft built a better model than OpenAI's for a fraction of the cost. Before you rank anyone on it, look at what was actually measured. What Microsoft reported is a pre-training result. MAI-Base-2 is a 2.8-trillion-parameter mixture-of-experts with 80 billion active, trained on 16,000 GB200 GPUs over 38 days on a 50-trillion-token mixture. Its claims are 4 to 12 times the FLOPs-efficiency of DeepSeek-V4-Pro-Base across four held-out evaluations, and compression scores of 0.215 bits-per-byte on code and 0.41 on question-answering. Those are real engineering numbers. They are also the wrong kind of number for the claim being made. Bits-per-byte measures how tightly a model compresses text. FLOPs-efficiency measures how much you get per unit of training compute. Neither is a task score, and neither is a product. A base model is the raw engine before post-training, the tuning and alignment that turn next-token prediction into something usable. Strong pre-training often fades through that stage. And the comparison to OpenAI comes from private tests on held-out loss that only Microsoft can run; OpenAI has not confirmed it. There is no API, no price, no release date, and no public benchmark. The mechanism worth watching is who controls the scoreboard. If a pre-training-efficiency number is accepted as a capability claim, "who leads AI" gets decided on private held-out loss that only the vendor can measure. That weakens outside scrutiny and rewards whoever owns the evals, a second-order shift that matters more than any single model. The strongest case the other way: a strong base model usually does predict a strong product once post-training is done, and licensing the weights to partners through Foundry is a concrete delivery path, not vapor. If the pre-training lead carries through to tasks, enterprise partners could see a real edge. That case wins over months, once the model ships and is independently tested, not on the day of a town-hall remark. If you are ranking labs to guide your own model choices, this is a reason to defer, not to decide. The engine on the bench may be excellent; you still cannot drive it. Bits-per-byte measures compression, FLOPs-efficiency measures training cost, neither is a product. We'd revise if a shipped, post-trained MAI model scores at or above GPT-5.6 Sol on independent task benchmarks, not loss or bits-per-byte. ## Sources & notes - [Primary source: microsoft.ai/news/frontier-pre-training-at-microsoft-ai](https://microsoft.ai/news/frontier-pre-training-at-microsoft-ai/) --- Cite as: WAKENEX. "A pre-training metric is not product capability, and this model cannot even be called yet." wakenex.com, 2026-10-18. https://wakenex.com/notes/a-pre-training-metric-is-not-product-capability-and-this/ Web page: https://wakenex.com/notes/a-pre-training-metric-is-not-product-capability-and-this/