Mistral AI 发布 Robostral Navigate：8B 参数单 RGB 摄像头具身导航模型

Mistral AI 推出首个具身导航模型 Robostral Navigate。

该 8B 参数模型仅使用单 RGB 摄像头，即可根据自然语言指令在办公室、住宅等复杂环境中自主移动。

在 R2R-CE 基准的 validation unseen 分项上达到 76.6% 成功率，比最佳单摄像头方案高 9.7 个百分点。

模型采用"指向"决策方法预测目标图像坐标，目标出视野时回退到位移指令。

训练数据完全来自仿真，包含约 40 万条轨迹、覆盖 6000 个场景。

基于前缀缓存和树状注意力掩码的训练策略将训练 token 量压缩 22 倍，后续在线强化学习阶段进一步提升成功率 3.2%。

Mistral AI has released Robostral Navigate：https://mistral.ai/news/robostral-navigate/ , its first model built for embodied navigation. The 8B model takes RGB images and a plain-language instruction, then moves a robot. Notably, it reaches 76.6% success on R2R-CE validation unseen using only a single RGB camera.

Robostral Navigate is an 8B model for robotic navigation through complex environments. These environments include offices, residential buildings, commercial buildings, and outdoor settings. You give it one instruction, and it completes the whole task alone. For example: ‘Leave the lobby, walk through the corridor, enter the supply room, and stop to face the second shelf. ‘ Importantly, it moves through live spaces full of people and obstacles never seen in training.

Moreover, most navigation systems use depth sensors, LiDAR, or several cameras together. By contrast, Robostral Navigate uses one ordinary RGB camera and no depth sensors. Fewer sensors also make the system more efficient than multi-sensor approaches.

Beyond hardware, the next question is how the model decides where to move. To choose each move, the model uses a method called pointing . Given a task and observation history, it predicts where the robot moves next. Specifically, it infers the image coordinates of the target in the current camera view. It also predicts the desired orientation upon arrival.

This choice has a clear benefit. Unlike commands based on metric displacements, pointing stays robust to camera-intrinsic and world-scale changes. However, pointing fails when the target sits outside the current field of view. In that case, the model falls back to displacements in the robot’s local coordinate frame. For example: ‘Move 2 meters forward, 1.5 meters to the left, and turn 25 degrees left. ‘

Notably, Robostral Navigate does not rely on existing open-source VLMs. Instead, it starts from Mistral’s vision-language model built for grounding tasks. Those tasks include pointing, counting, and object localization. Navigation then emerges as a natural extension of grounding. Once the model knows where things are, it learns how to move.

For training data, Mistral built a generation pipeline entirely in simulation. This pipeline produced roughly 400,000 trajectories collected across 6,000 scenes.

Beyond data, training efficiency was core element to the design. The method uses an algorithm based on prefix-caching. A tree-based attention-masking strategy compresses an entire episode into one sequence. Consequently, the model trains on all time steps in a single forward pass. It also prevents information leakage between time steps.

As a result, this approach cuts training tokens by 22× while keeping all learning signals. In practice, runs that once took months now finish in days. After supervised training, Mistral applied CISPO, an online reinforcement learning algorithm. This stage lets the model learn from trial and error, recover from failures, and acquire exploratory behaviors. It also mitigates the distribution shift issue of vanilla behavior cloning. On its own, this step raised success rate by 3.2%.

On evaluation, Robostral Navigate reaches state-of-the-art results on R2R-CE. R2R-CE means Room-to-Room in Continuous Environments, a standard instruction-following benchmark. It is built on Matterport3D, where an agent follows language instructions in a continuous 3D environment. The ‘validation unseen’ split holds out environments to measure generalization. The benchmark scores agents with Success Rate, Oracle Success Rate, Success weighted by Path Length, and Navigation Error.

It scores 79.4% success rate on validation seen. It scores 76.6% on validation unseen. Furthermore, it beats the best single-camera approach by 9.7 points. It also beats the best depth or multi-camera system by 4.5 points.

To place these results in context, the table below contrasts Robostral Navigate with typical multi-sensor systems.

The right column describes common practice in vision-and-language navigation, not one named system.

Given these traits, several applications become practical. In manufacturing, a robot can carry parts between stations from one instruction. In delivery and logistics, a wheeled robot can move packages across a warehouse. In hospitality, a robot can guide a guest from lobby to room. Because the model runs on wheeled, legged, and flying robots, one fleet can share it. Additionally, it stays robust to differences in camera intrinsics across those robots.

To make the pointing loop concrete, the pseudo-code below shows the decision path. It is illustrative, not the official API.

Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us ：https://forms.gle/wbash1wF6efRj8G58