That is not a sentence I expected to write, but I think it is pretty interesting :)
Before anyone moves their production AI workloads to Hetzner: this is very much an experiment . There is no billing, no SLA, no production guarantee, and currently only one model. Hetzner says it wants to learn whether people actually want this, how the system scales, which features matter, and what kind of load it can handle.
So this is not a finished product launch. It is Hetzner putting something early in front of users and seeing what happens. I really like that approach.
Hetzner Inference ：https://experiments.hetzner.com is an OpenAI-compatible API running on Hetzner's own infrastructure. You create an API token in the Experiments dashboard, point an OpenAI client at Hetzner's base URL, and use it like most other inference APIs.
Right now, the only available model is Qwen/Qwen3.6-35B-A3B-FP8 ：https://huggingface.co/Qwen/Qwen3.6-35B-A3B-FP8. It is a 35-billion-parameter Mixture-of-Experts model with 3 billion active parameters. It accepts text and images, has a 262K context window, and uses FP8-quantized weights.
That is a perfectly reasonable model for an experiment. It is small enough to serve without a ridiculous GPU cluster, but still useful enough to test the API with real workloads.
Hetzner also published a short tutorial for connecting OpenCode to the API ：https://community.hetzner.com/tutorials/opencode-with-hetzner-inference-api-systemd-sandbox/, if you want to try it without writing any code.
Because the API is OpenAI-compatible, there is almost nothing special about the integration:
The enable_thinking option is worth mentioning. Without it, the model can spend a surprising amount of the completion budget reasoning before it returns a visible answer. The option worked in my tests, but it is not documented by Hetzner, so I would not build anything important around that exact request shape yet.
I ran a few small tests on July 23, 2026. I do not want to turn this post into a giant benchmark report, because the product is experimental and a benchmark against it will probably age badly. But the rough numbers were:
That is fast! It is also just one test from one client at one point in time. It is not an SLA, and it says almost nothing about what happens when many people use the service at once.
The model itself was roughly what I expected. It followed most formatting and retrieval instructions, handled an image correctly, and failed two very simple arithmetic questions. So: a small, slightly shitty LLM :D
The Qwen endpoint is fun, but I do not think the current model is the interesting part.
The interesting part is why Hetzner is testing inference in the first place.
Everything from here on is purely my speculation. I have no insider information, and nobody at Hetzner told me what they are planning. I am just looking at the product and trying to connect a few dots.
Open-weight inference is a commodity market. Everyone can download the same weights, run more or less the same serving software, and expose an OpenAI-compatible API. Switching providers is also easy, especially with products like OpenRouter or LiteLLM for those you self-host.
That makes it hard to build huge margins unless you have some kind of advantage. Usually that means:
Hetzner is very good at buying hardware, putting it into its own data centers, and operating it with a brutally efficient cost structure. That is basically the whole company. If anyone can turn inference into another low-margin infrastructure product, Hetzner is at least a believable candidate.
There is also a nice utilization story here. A rented bare-metal GPU belongs to one customer, whether that customer uses it or not. An inference API can share GPU capacity across many users and keep the hardware busy. If Hetzner has spare GPU capacity — or plans to build a much larger GPU fleet — an inference product could help turn that capacity into revenue.
Again, I have no idea whether this is actually what they are doing. It would just make economic sense to me.
This is where I am not yet convinced.
Hetzner's current public dedicated GPU server lineup ：https://docs.hetzner.com/robot/dedicated-server/server-lines/gpu-server/ uses two GPU types:
Those are capable GPUs, and the 96 GB RTX PRO 6000 is a pretty nice inference machine for small and medium-sized models. The FP8 files for Hetzner's current Qwen model are around 38 GB, with actual VRAM use landing somewhere above that depending on context length, cache size, and serving setup.
But these are workstation GPUs, not the dense multi-GPU systems you need for the really large open models.
Take GLM-5 ：https://huggingface.co/zai-org/GLM-5 as an extreme example. It has 754 billion parameters, and the official serving recipe splits it across eight GPUs. Even with aggressive quantization, you are talking about hundreds of gigabytes of VRAM. Realistically, that is B200/B300-class hardware, or something similar, with very fast links between multiple GPUs.
Hetzner does not currently offer that kind of hardware in its public bare-metal lineup.
Of course, that does not tell us what sits behind the experimental API. Hetzner may use completely different internal hardware, and a public inference product does not have to mirror its dedicated-server catalogue.
Still, this is the part I am watching.
If Hetzner keeps serving one or two smaller models, I do not really see it becoming an important inference provider. That would be a cool experiment, but not much more.
If this experiment is the first step toward larger GPU clusters, a proper model catalogue, and B200/B300-class hardware, then it gets much more interesting. Hetzner already has the data centers, network, hardware experience, European positioning, and reputation for aggressive pricing. That combination could make it a serious competitor.
For now, the API is fast, free, and fun to try. For me, the interesting question isn't which small model they add next, but whether Hetzner invests in the hardware needed to serve the big ones.
Jonas Scholz is one of the co-founders of Sliplane, where he focuses mainly on infrastructure and system reliability.
Sliplane is a cloud platform that makes deploying and scaling your apps simple. Launch in minutes and grow as you go, containers included.
Sliplane makes it simple to deploy and scale your apps in the cloud. Start with a container or your favorite framework and grow from there. Try it now and get started in minutes!
Sliplane is not associated with Docker Inc. or any of its products. Prices do not include VAT.
扫码直达文章详情