前 Meta 首席 AI 科学家 Yann LeCun 更进一步，认为生成视频模型是死胡同。Meta 的 V-JEPA 2：https://the-decoder.com/metas-latest-model-highlights-the-challenge-ai-faces-in-long-term-planning-and-causal-reasoning/ 遵循他倡导的替代方法，预测抽象概念而不是像素。清华大学的一个基准测试：https://the-decoder.com/ 指出了像素预测的局限性。Sora 2、Seedance 2.0 和 Veo 3.1 即使输出看起来很可信，也在基础物理和逻辑测试中反复失败。
GenCeption 将视频生成用于更狭窄的目的。研究人员并不要求 Wan2.1 预测世界的行为，而是用它来提取用于特定任务的特征，例如深度估计和分割，其中这些学习到的特征可以超越专门的系统。Google Deepmind 也在用 Genie 3：https://the-decoder.com/google-deepminds-genie-3-generates-interactive-3d-worlds-that-stay-consistent-for-multiple-minutes/ 探索不同方法，构建用于训练 AI 智能体的交互式 3D 环境。
保持关注 AI 动态。清晰、有用，无废话。
关注 The Decoder 获取 AI 新闻、背景故事和专家分析。
解码器：https://the-decoder.com/
A new model from Google Deepmind called GenCeption uses a pre-trained video generation model as the basis for classic computer vision tasks. It achieves state-of-the-art performance in depth estimation, segmentation, and 3D pose estimation while needing very little training data.
Language models became versatile processing systems almost as a byproduct of learning to predict the next word. That task appears to require models to absorb grammar, world knowledge, and contextual relationships during training. This is the prevailing explanation for the emergent capabilities：https://the-decoder.com/openais-sora-is-the-gpt-1-of-video-with-plans-to-scale-and-unlock-emergent-ai-capabilities/ of large language models.
Computer vision still lacks an equivalent training method. Specialized models dominate the field, including "Segment Anything" for segmentation and "Depth Anything" for depth estimation. Each uses its own architecture. Ad
In a new paper：https://the-decoder.com/, Deepmind researchers argue that large text-to-video models could bridge this gap. Generating realistic videos requires understanding the spatial geometry of a scene, how objects move, and basic physics. These models also use text descriptions during training, which links language to visual content. They can train on vast datasets because labeling the data is relatively cheap. Ad DEC_D_Incontent-1
GenCeption builds on Alibaba's open source Wan2.1 video model：https://the-decoder.com/alibaba-launches-wan2-2-its-improved-open-source-video-generation-model/. The main change is a simpler architecture. Diffusion models typically generate video from noise through many small steps. GenCeption instead produces a prediction in one forward pass, making it fast enough for practical computer vision tasks.
The researchers use a simple method to make one model handle many tasks. GenCeption represents every output as a standard three channel RGB image, whether the result is a depth map, surface normal map, or segmentation mask. It also converts camera movement into an image representation. Ad
A text prompt tells the model which task to perform, much like an instruction given to a chatbot. The team adds trainable modules for tasks such as 3D keypoint prediction, which don't produce images.
Training uses one loss function across all tasks. Rather than changing the architecture for each task, the researchers process the training data to account for task specific requirements. Ad DEC_D_Incontent-2
Most of the training data came from a synthetic dataset containing just 7,500 videos. The team combined 800 digital human models with 200 motion sequences from a motion capture dataset. It then rendered the results in Blender with different backgrounds and camera angles. Real videos were used only for language guided segmentation. Ad
According to the study, GenCeption matches or beats state-of-the-art results across many benchmarks despite using one architecture for every task. Its depth estimates match those from specialized models such as DepthAnything 3. GenCeption beats NormalCrafter and Lotus-2 on surface normal estimation. It also outperforms Genmo and TRAM on 3D pose recognition. On complex language guided segmentation, it matches Meta's SAM 3 combined with Gemini 3.5 Flash.
Models such as D4RT and VGGT Omega trained on millions of videos. GenCeption reaches similar results using 7 to 500 times less data. When tested under the same conditions, pretraining on video generation also beats methods such as V-JEPA and VideoMAE V2. The authors attribute those results to the generation task rather than data volume alone. They argue that video generation helps models learn useful representations of space and movement.
GenCeption trained almost entirely on synthetic videos showing one person at a time, but it still works on real videos with several people in the frame, as well as unrelated categories such as animals and humanoid robots. According to the study, some outputs contain more detail than the Blender renderings used for training. The results can preserve a cat's whiskers and the edges of individual strands of hair.
Joint training across all tasks does hurt 3D keypoint estimation. The researchers suspect that the extra components needed for this task interfere with mechanisms learned by the base model during pretraining.
Their conclusion is that as few changes as possible should be made to the original model's architecture. Processing speed also needs work. The smaller model takes about six seconds to process a video with 81 frames, the larger model has 14 billion parameters and takes about ten seconds.
The authors reject the idea that video generators are only entertainment tools. They argue that these models already contain a type of universal "world model"：https://the-decoder.com/openais-sora-2-answers-science-questions-directly-in-its-generated-videos/ that could support a foundation model for computer vision. Such a system could fill the same role in image processing that large language models fill in text.
Whether that framing holds up is debated. An international research team recently proposed a uniform definition：https://the-decoder.com/researchers-define-what-counts-as-a-world-model-and-text-to-video-generators-do-not/ with OpenWorldLib：https://the-decoder.com/ and explicitly excluded text-to-video models because they lack feedback from the real world.
Former Meta chief AI scientist Yann LeCun has gone further, arguing that generative video models are a dead end. Meta's V-JEPA 2：https://the-decoder.com/metas-latest-model-highlights-the-challenge-ai-faces-in-long-term-planning-and-causal-reasoning/ follows the alternative he has championed, predicting abstract concepts rather than pixels. A benchmark from Tsinghua University：https://the-decoder.com/ points to the limits of pixel prediction. Sora 2, Seedance 2.0, and Veo 3.1 repeatedly failed tests of basic physics and logic even when their output looked convincing.
GenCeption uses video generation for a narrower purpose. The researchers don't ask Wan2.1 to predict how the world will behave. They use it to extract features for specific tasks such as depth estimation and segmentation, where those learned features can outperform specialized systems. Google Deepmind is also exploring a different approach with Genie 3：https://the-decoder.com/google-deepminds-genie-3-generates-interactive-3d-worlds-that-stay-consistent-for-multiple-minutes/, which builds interactive 3D environments for training AI agents.
Stay in the loop on AI. Clear, useful, no fluff.
Follow The Decoder for AI news, background stories and expert analyses.
扫码直达文章详情