Soofi S is one of the first large language models trained entirely on Deutsche Telekom's Industrial AI Cloud in Munich. The open 30B model uses a lean hybrid architecture and a training mix deliberately weighted toward German.
Soofi S is a mixture-of-experts model. It contains 31.6 billion parameters in total but activates only about 3.2 billion per generated token. That puts its compute cost closer to a 3B model than a conventional 30B model. The consortium adopts the architecture of Nvidia's Nemotron 3 Nano：https://the-decoder.com/nvidias-nemotron-3-ultra-becomes-the-smartest-open-us-model-but-china-still-leads/ without modification, a hybrid design combining Mamba-2 layers with standard attention layers. Ad
The key difference from typical transformers is memory behavior. In conventional models, the KV cache that stores previous tokens for attention computation grows linearly with context length. With long inputs and many parallel requests, reloading that cache becomes a bottleneck. Only 6 of Soofi S's 52 layers maintain such a cache at all. Ad DEC_D_Incontent-1
The practical payoff shows up in generation throughput. At a context length of 40,000 tokens with 32 parallel requests, Soofi S generates roughly eight times more tokens per second per GPU than dense models in the 14 to 24 billion parameter range. While throughput drops significantly for conventional models as context grows, Soofi S stays nearly flat from 4,000 to 256,000 tokens. The only model that shows similar behavior in the measurements is Alibaba's Qwen3.5 35B-A3B：https://the-decoder.com/alibabas-open-qwen-3-5-takes-aim-at-gpt-5-mini-and-claude-sonnet-4-5-at-a-fraction-of-the-cost/, which also uses a hybrid architecture.
The consortium processed about 27 trillion tokens in total, split across three phases. In the first phase, the model learns language fundamentals from roughly 20 trillion tokens drawn from a broad mix of web, code, math, and domain-specific texts. A second phase follows with about 6 trillion tokens from higher-quality sources, designed to sharpen the patterns learned earlier. A shorter third phase then extends the context window by training on very long documents of up to one million tokens. Ad
The deliberate focus on German is central. In the first phase, German makes up 7.2 percent of the training mix; in the second phase, that share rises to 15.3 percent. In Nvidia's Nemotron reference recipe, all non-English languages combined account for only about 5 percent.
For data sources, the consortium combines German web text from HPLT, the openly licensed German Commons corpus, German portions of FinePDFs and FineWiki, and the commercially licensed Genios corpus containing 193 million newspaper articles from 916 German publications. Machine-translated and synthetically generated German texts round out the mix. Ad DEC_D_Incontent-2
In evaluations against 16 other open models, Soofi S leads all fully open models on aggregate scores for both German and English, according to the report. That includes OLMo 3 32B from the Allen Institute for AI and Apertus 70B from ETH Zurich and EPFL. Against every European sovereign baseline, the model comes out ahead on all German benchmarks in the suite, sometimes by double-digit margins. Ad
On code benchmarks, Soofi S scores 73.8 percent on HumanEval, 70.2 on MBPP, and 84.2 on the German MBPP variant, the best results among open-source peers. On INCLUDE-DE, a test for Germany-specific regional knowledge, Soofi S ties for first place at 61.2 points with the larger Qwen3.5 35B-A3B. Compared to the Nemotron baseline, the German data recipe improves language proficiency by 15.1 points and the science test GPQA-Diamond by 9.6 points, without sacrificing English performance.
Soofi S doesn't do as well on German competition math, where it scores 56 points on Minerva MATH-DE, well behind Qwen3.5 35B-A3B (76.5) and Gemma 3 27B：https://the-decoder.com/google-releases-new-gemma-3-open-model-family/ (65.6). It also lags on open factual retrieval in NaturalQuestions. The latter likely relates to having only 3 billion active parameters, which can store less world knowledge：https://the-decoder.com/sinas-open-model-vibethinker-3b-aims-to-show-reasoning-compresses-well-but-factual-knowledge-doesnt/ than a dense 27B model.
The RULER long-context test also reveals a specific weakness: When the model has to extract frequently occurring words from a long text, Soofi S's hit rate drops to around 3 percent beyond 32,000 tokens of context, while the comparable Nemotron model still manages 60 to 64 percent. The authors attribute this to the fact that their long-context training data contains many long documents but lacks synthetic data designed for extraction tasks. On the remaining twelve RULER tasks, both models perform about the same.
The training run took place between March and May on up to 512 Nvidia B200 GPUs at Deutsche Telekom's Industrial AI Cloud：https://the-decoder.com/10000-nvidia-blackwell-gpus-set-to-increase-germanys-ai-capacity-by-50-percent/ in Munich, totaling about 253,000 GPU-hours. According to the report, the facility runs entirely on renewable energy, is cooled with water from the Eisbach canal, and feeds waste heat into the surrounding Tucherpark neighborhood. Soofi S was one of the first major training runs on this infrastructure.
Behind Soofi is a consortium of German research institutions and companies, coordinated by the German AI Association and funded by the German Federal Ministry for Economic Affairs and Energy as part of the European IPCEI-CIS program.
Participants include the Fraunhofer Institutes IAIS and IIS, the German Research Center for Artificial Intelligence (DFKI), TU Darmstadt, the University of Würzburg, the L3S Research Center, the Berlin University of Applied Sciences, and AI companies Ellamind and Merantix Momentum. The project's goal is to build an open European AI model family that can run on sovereign infrastructure and be tested in industrial applications.
The researchers are releasing model weights along with selected intermediate checkpoints：https://huggingface.co/collections/Soofi-Project/soofi-s-beta-models, the complete training and evaluation code, and a detailed data inventory listing raw token counts, epoch numbers, and effective contributions per source. Sources that were reviewed but excluded are also documented. According to the team, this means Soofi S meets the Open Source AI Definition 1.0 from the Open Source Initiative：https://the-decoder.com/open-source-initiative-releases-first-formal-definition-of-open-source-ai/.
A stricter proposal for a European open-data definition, which would require every single training token to be freely distributable, isn't met because of the 1.3 percent share of Genios data, which carries a commercial license. The report says about 99 percent of the training mix can be independently reconstructed. The exact license for the model's release hasn't been finalized yet.
As lead author Michael Fromm writes：https://x.com/effi288/status/2075904321707798699, Soofi S positions itself between broadly multilingual European sovereignty projects like EuroLLM or Teuken：https://the-decoder.com/eu-project-releases-7b-model-that-speaks-24-european-languages/, which cover many languages, and the highest-performing international open-weight models. According to the project website, the consortium is looking for industry partners for the next phase to test the model in applications involving technical documents, code generation, and agent-based systems.
Stay in the loop on AI. Clear, useful, no fluff.
Follow The Decoder for AI news, background stories and expert analyses.
扫码直达文章详情