一个模型，以视觉智能为核心，能够生成图像、视频和音频——并驱动生产线上的机器人。内容创作和物理 AI 是同一基础的两种应用。
An early version of FLUX 3, our new multimodal foundation model：https://bfl.ai/blog/flux-3, is now running on robots. We gave mimic robotics：https://www.mimicrobotics.com/ early access to FLUX.3. Their strength in robot learning and deployment, combined with the model's world knowledge and BFL's foundation model expertise, produced FLUX-mimic: the next generation of video-action models.
FLUX 1 and FLUX 2 generate images. FLUX 3 expands into multimodality and generates audio-visual content jointly - and, at the same time, provides the foundation of FLUX-mimic: A video-action model, developed in collaboration with mimic, running robots that have been tested and deployed at Audi.
At first glance, producing convincing visual content and controlling robots seem to have little in common. One requires generating pixels, the other an understanding of how the physical world responds when you touch and manipulate it. If one model does both, it was never really only a content creation model. It is a model of how the world behaves, and content creation is one thing one can do with it.
FLUX 3 is one model, jointly trained across images, video and audio from the beginning. The most demanding part of that training - accounting for over 95% of the total compute costs - is video prediction. To generate realistic videos, a model has no choice but to learn contact, motion, weight, cause and effect; get any of them wrong and it looks wrong. Learning to render the world accurately means learning how the world behaves.
Relatively speaking, audio is the easy modality. Low dimensional and far less detailed than video, it makes up less than 0.5% of the tokens in a 720p video with audio. Once a model has done the hard work of learning video understanding, it will learn the causal relationship between video and audio to predict speech synchronized to lip movement and audio effects synchronized to the physical events causing them.
Actions follow the same shape: a low dimensional representation of a robot's state, tightly coupled to visual observations. Actions, audio and video frames are all partial representations of a single underlying physical reality. After the model has learned about the physical processes behind video and audio, action prediction is not a new departure - it is one more view of the reality it already models.
If that framing is correct, teaching FLUX 3 to predict actions should not incur lasting costs: we expect a brief phase of disturbance as the model has to learn the structure of the action space and align its internal representation of the world to it, before returning to full performance. That is exactly what we observe.
In a large-scale training run, we added action prediction to the curriculum and observed the effect on video generation quality. Human ratings on text-to-video and image-to-video initially fell by up to 10% as the model started to incorporate the new action modality. After 3500 steps, the model had regained its full previous quality on video generation tasks while now also predicting actions.
Each series is normalized to its own quality before action prediction was added. Higher is better.
The model had to integrate actions into its inputs and outputs - but doing so didn't cost it capacity permanently. It merely had to learn how this new modality relates to its existing model of the world. Once this was figured out, the performance penalty on its existing capabilities was gone. Video generation and action prediction don't need separate foundations. The same backbone carries both.
This makes Physical AI a natural extension of our roadmap at Black Forest Labs rather than a change in direction. Content creation is what our multimodal FLUX 3 backbone does with image, video and audio. Physical AI is what it does with actions. One foundation model, with visual intelligence at its core, enabling two families of applications. We didn't build a separate foundation model. We focused on the hard thing: building a model that understands the world. Acting in it is what that understanding makes possible.
What happens when we point the FLUX 3 backbone at real automation tasks on real production lines?
That's the question mimic and BFL created FLUX-mimic to answer. mimic builds their own robots and brings expertise in robot learning, dexterous manipulation and production deployment; BFL builds visual foundation models and brings multimodal training and modeling expertise. Together, we built a next-generation model for general-purpose manipulation - adapted to industry requirements and integrated into mimic's full-stack deployment system. FLUX-mimic is a video-action model built on the FLUX 3 backbone.
Our thesis is that FLUX 3 has to learn an internal representation of the world to be able to generate videos. FLUX-mimic follows through on this thesis and decodes actions from the learned world representation of the FLUX backbone. This approach, pioneered in mimic-video, trains a lightweight action decoder on top of intermediate features extracted from the video prediction path of FLUX.
Architecture overview how FLUX-mimic is built on top of FLUX 3
The success of this approach depends on two related but different aspects: the quality of the world model learned by FLUX and the quality of the feature representation of this world model. The quality of the world model is directly related to the generation quality: if a model does not understand how the world behaves, it cannot simulate it. However, even the best world model does not help an action decoder if it is inaccessible: if the feature space keeps the causal relationships between modalities entangled nonlinearly, understanding those relationships from the feature representation remains as difficult as understanding them from the raw inputs - representation quality matters.
Generation quality and representation quality have long been studied and approached in isolation from each other. Generative approaches result in high-quality world models that enable simulations and they exhibit scaling laws for predictable returns on compute investments. However, compared to more specialized approaches for representation learning they produce less disentangled representations, which puts a ceiling on their usefulness for tasks that require world understanding.
As generative models themselves rely on their own features, this divergence in their representation quality seems counter-intuitive. Improved representations within generative models should improve the quality of their world model and make them more usable for downstream tasks. In our work, Self-Flow：https://bfl.ai/research/self-flow, we demonstrated how to unify generation and representation learning in a single framework and observed exactly this reciprocal improvement: the world model improved - as measured by generation quality across video, image and audio - and its representation quality improved - as measured by success rate for robot control tasks in simulation.
Self-Flow vs. Flow Matching (FM). Left: generation error (Fréchet distance) per modality, each normalized to FM = 100 (lower is better). Right: success rate on manipulation tasks averaged over four task groups through finetuning (higher is better).
Scaling laws remain true with Self-Flow, and FLUX 3 is the application of that: the scaled-up version of Self-Flow. It is trained on tens of millions of hours of general video content to learn world dynamics as broadly as possible from day one, and on hundreds of thousands of hours of video content focused on human and robot manipulation tasks to be ready as a backbone for visual intelligence. This scaling is what translates the success of Self-Flow from the lab to reality. mimic deployed FLUX-mimic in real factory use cases spanning the daily reality of production and logistics work: kitting parts into structured trays, inserting electronic control units into tight-fitting fixtures, assembling components together, and handling soft, flexible materials like seals and cables that conventional automation has never been able to touch.
Benchmarks demonstrate that the action decoder outperforms previous vision-language-action models, even with a completely frozen FLUX backbone - a setting where previous vision-language-action models fail to succeed. This highlights how scaling gives our backbone strong knowledge of the world and how to act in it, and how Self-Flow makes this knowledge readily decodable from the backbone's feature representations. When finetuning the backbone together with the action decoder, FLUX-mimic achieves state-of-the-art success rates.
Dashed line marks each model's median success rate across 20 autonomous trials. Higher is better.
A backbone exposing world knowledge in decodable representations changes what it takes to teach a robot a new task. If the physics is already in the representation and readily accessible, adapting to a task is no longer a matter of teaching the model how the world works - it only has to learn how this particular task maps onto what it already knows. The expensive part is done before the robot ever moves.
This shows up directly in how much demonstration data a new task requires. In our Self-Flow experiments, action prediction reached a given success rate in half the training steps compared to a video model without Self-Flow - better representations make the world knowledge easier to extract, so less data is needed to reach the same capability. The mimic-video paper reports up to 10x sample efficiency for video-action models over vision-language-action models; FLUX-mimic combines both effects.
The same benefit shows up in behavior. FLUX-mimic naturally recovers from failure: a robot that misses a grasp corrects itself, grasps again, and completes the task. No demonstration set can cover every possible way a task can go wrong. Recovery that was never demonstrated has to come from somewhere else - from a model that already knows how the world behaves.
The backbone's predicted future (top), alongside the rollout the robot produced from the decoded actions (bottom).
Real-world deployment sets a hard constraint: the model has to act as fast as the world moves. The dominant compute cost for FLUX-mimic sits in the backbone. It is the largest component of the model and, in mimic's optimized deployment stack, its latency effectively sets the ceiling for the whole system.
This is where our methodology pays off a second time. Better representations mean more capability per parameter: a model that has learned a well-structured world model needs less capacity to reach a given level of performance than one that has not. For deployment, this translates directly into being able to run a smaller backbone - and a smaller backbone is a faster backbone.
CLIP score at 1.0M training steps; higher is better. Backbone depth is the dominant driver of deployment latency, so fewer layers means a faster model.
As a result, the backbone of FLUX-mimic can be optimized to run from input to world representation in less than 80ms on a single NVIDIA RTX 5090 GPU - which puts it on the same order of magnitude as human visual reaction time.
Real-world deployments require additional optimizations of the full deployment stack to avoid adding any additional latency: mimic's optimizations range from the action decoder, through cutting inter-process latency between sensors, the model and actuators, to real-time chunking such that prediction and execution overlap and keep the robots running smoothly without jitter. The end result is a self-contained robot system with reaction times of 101ms.
All of this leads back to the place where automation matters: the factory. Audi runs one of the most automated production networks in the automotive industry - which gives it a precise view of where conventional automation still stops. Despite decades of robotics investment, tasks with flexible parts and fine manipulation have stayed manual, largely for economic reasons: the variant diversity of premium production makes conventionally programmed robot cells too costly to re-engineer for each case. Learning-based systems change that math.
"In partnership with mimic, Audi has been testing and deploying FLUX-mimic. We have seen these robots solve complex soft-body manipulation work that would have been simply impossible with conventional robotics. This can have a major impact in assisting our employees, increasing efficiency, and expanding flexible automation across production and logistics operations. For us, partnering with pioneering companies such as mimic and Black Forest Labs is essential in pushing the frontier of physical AI and validating these innovations in real-world production environments." — Christoph Schneider, Audi Production Lab
FLUX-mimic is a purpose-built robotics model - and a proof point for what's to come. Its sample efficiency and robustness come from the FLUX 3 backbone and the quality of the representations it exposes, not from task-specific engineering. That is what lets the approach transfer across tasks, industries, and hardware. Read more through mimic：https://www.mimicrobotics.com/blog/introducing-flux-mimic.
One model, with visual intelligence at its core, generating image, video and audio - and driving robots on a production line. Content creation and physical AI are two applications of the same foundation.
情报判断
Aioga 编辑摘要
Black Forest Labs 公开 FLUX 3 早期版本，该模型从训练之初联合处理图像、视频与音频，并与 mimic 合作形成视频动作模型 FLUX-mimic，相关机器人已在奥迪测试和部署。
扫码直达文章详情