GPT-6 Astra 在 LEGO-Bench 的领先地位与其他观察结果一致。AI 研究员 Yoav Artzi 认为该模型在空间理解方面取得了重大飞跃，并怀疑它是通过大量 3D 数据（如 Blender 场景）训练的：https://the-decoder.com/gpt-6-astra-appears-to-show-a-step-change-in-spatial-reasoning-based-on-early-benchmarks/。3D 软件制造商已经在为这类智能体做准备。Unity 发布了针对 Claude Code 和 Codex 的官方插件：https://the-decoder.com/unity-launches-official-plugins-for-claude-code-and-openai-codex-to-stop-ai-agents-from-using-outdated-tutorials/。其他方法则完全跳过编码，直接在模型内部重建场景，例如 World Labs 的 Atlas 世界模型：https://the-decoder.com/world-labs-unveils-atlas-a-single-ai-model-that-generates-reconstructs-and-simulates-3d-worlds-from-just-a-few-photos/。Google Deepmind 则采用另一种路线，使用视频模型进行深度估计和分割的 GenCeption：https://the-decoder.com/google-deepmind-argues-video-generators-already-contain-the-world-models-computer-vision-has-been-missing/，其性能可与专用模型匹敌。
保持对 AI 的关注。清晰、有用，无废话。
关注 The Decoder 获取 AI 新闻、背景故事和专家分析。
解码器：https://the-decoder.com/
Coding agents can build an editable 3D scene from a single photo by writing and refining code step by step. The benchmark that comes with this work reveals where the process still breaks down: self-assessment and geometric accuracy.
A 3D reconstruction from a single photo is most useful when it exists as an executable program you can inspect, edit, and query. That's the premise behind LEGO-Anything, a project from researchers at the University of Maryland and AWS.
The approach is called "Image-to-Code." A coding agent receives a single image and writes code for Blender, the widely used 3D software. Rather than generating the scene in one pass, the agent works iteratively: it writes code, runs it, looks at the result, and revises until the scene matches the original.
Because the output is a program, it captures objects, geometry, layout, and camera position explicitly. You can run, check, and modify the scene like any other piece of code.
To measure how well agents perform, the team introduces LEGO-Bench. It contains 208 images from 104 indoor and outdoor scenes and uses 443 registered assets.
Real photos don't provide a precise 3D ground truth to compare against, according to the researchers：https://lego-anything.com/. Simple synthetic scenes look unrealistic. So LEGO-Bench splits the difference by rendering its images from professionally built simulator scenes. The inputs look natural, while the exact geometry, depth, and object assignments stay hidden and serve as the answer key for automated scoring. Scene complexity can also be ramped up without changing lighting or camera settings.
The benchmark scores each scene on three axes: validity checks whether a usable scene artifact was delivered at all. Reconstruction measures how accurate the visible geometry is. Appearance captures how closely the look matches the original by re-rendering the submitted scene and comparing it pixel by pixel against the reference image.
All six tested GPT configurations delivered a working scene almost every time. Accuracy varied wildly, though. GPT-6 Astra, the best tested model, hit 53.4 percent on indoor scenes and 39.6 percent on outdoor scenes. Weaker configurations scored around 15 percent.
The more complex a scene, the more accuracy drops, and outdoor scenes are harder than interiors. When the researchers increased the models' reasoning budget, the GPT-6 variants improved a lot. Astra's score on an office test subset jumped from 32.3 to 61.8 percent.
To understand why, the researchers analyzed the agents' work steps. They found poor initial attempts, revisions that undid earlier progress, and unreliable self-assessment as the most common issues.
That last point is the most telling. When models had to pick which of two versions better matched the original, their geometric judgments landed near or below chance level. An agent basically can't tell whether its own scene has gotten better. The researchers conclude that refinement should rely on concrete measurements, not the agent's own judgment.
That insight led the authors to build LEGO-Plugin, an extension that needs no extra training. It anchors the starting scene in the reference image, swaps the unreliable self-judgment for concrete measurements, and shields correct progress from regressive edits. The plugin improved all six models. Weaker agents saw the biggest gains, with boosts up to 62.7 percent. The already strong top model gained only about two percentage points.
Finally, the team tested whether reconstructed scenes could serve as a basis for standard vision tasks. Because each scene is an executable program, object detection, segmentation, and depth estimation can be pulled directly from it.
Without any extra training, the scenes produced usable but unremarkable results across all three tasks. Object detection fared best, with the reconstructed scenes reaching roughly half the performance of the specialized model DINO：https://the-decoder.com/meta-makes-its-state-of-the-art-dinov3-image-analysis-model-available-for-commercial-projects/. For segmentation and depth estimation, the gap to specialized models like SAM 3：https://the-decoder.com/metas-sam-3-segmentation-model-blurs-the-boundary-between-language-and-vision/ and Depth Anything 3 was larger. The authors say executable scene programs from current coding agents show promise but aren't accurate enough. A big gap remains between a working result and a faithful reconstruction.
GPT-6 Astra's commanding lead in LEGO-Bench lines up with other observations. AI researcher Yoav Artzi sees the model as a major leap in spatial understanding and suspects it was trained on large amounts of 3D data like Blender scenes：https://the-decoder.com/gpt-6-astra-appears-to-show-a-step-change-in-spatial-reasoning-based-on-early-benchmarks/. 3D software makers are already gearing up for these kinds of agents. Unity has released official plugins for Claude Code and Codex：https://the-decoder.com/unity-launches-official-plugins-for-claude-code-and-openai-codex-to-stop-ai-agents-from-using-outdated-tutorials/. Other approaches skip code entirely and reconstruct scenes directly inside the model, like the Atlas world model from World Labs：https://the-decoder.com/world-labs-unveils-atlas-a-single-ai-model-that-generates-reconstructs-and-simulates-3d-worlds-from-just-a-few-photos/. Google Deepmind takes yet another route with GenCeption, using a video model for depth estimation and segmentation：https://the-decoder.com/google-deepmind-argues-video-generators-already-contain-the-world-models-computer-vision-has-been-missing/ that matches the performance of specialized models.
Stay in the loop on AI. Clear, useful, no fluff.
Follow The Decoder for AI news, background stories and expert analyses.
扫码直达文章详情