今天，我们推出了自主视频理解（Agentic Video Understanding）：https://ai.google.dev/gemini-api/docs/video-understanding#agentic-video-understanding，适用于我们最新的模型：Gemini 3.7 Flash、3.6 Flash 和 3.5 Flash-Lite。这一新功能在显著降低视频分析的 token 使用量和成本的同时，提升了准确性。类似于自主视觉（Agentic Vision）：https://blog.google/innovation-and-ai/technology/developers-tools/agentic-vision-gemini-3-flash/，该技术将代码执行与 Gemini 模型的原生图像理解结合，自主视频理解使用 Gemini 的原生视频工具来提升性能，并为视频处理解锁新功能，例如亚秒级片段检索、更准确的异常检测、精确计数等。
该功能现已通过 Google AI Studio 和 Gemini 企业代理平台的 Gemini API 对视频上传和 YouTube 视频开放。
与当前的“静态”处理不同，静态处理模式下模型以固定帧率（默认 1 FPS，可通过 API 调整）摄取视频，自主视频理解将模型的核心推理与原生视频工具相结合，动态搜索、扫描和检查目标视频片段的视觉帧、音频和转录文本。在标准视频分析基准测试中，配备自主视频理解的 Gemini 模型可将分析成本降低高达 66%，token 消耗降低高达 88%，同时准确率提升高达 7%。
这些效率提升在长视频上尤其明显（从 10 分钟的操作指南到 90 分钟的讲座以及数小时的录制），因为静态处理迫使开发者在高 token 成本和可能丢失关键细节的处理技术之间做出选择。
启用自主视频理解后，使用 Gemini 3.7 Flash 的 token 消耗可降低高达 88%，准确率提升可达 7%。
虽然这些提升适用于所有三种支持的模型，但配备自主理解的 Gemini 3.7 Flash 提供了整体最佳质量及最佳质量与成本效率的组合，使其在视频理解的测试模型中处于准确率与成本的帕累托前沿。
使用自主视频理解使 Gemini 3.7 Flash 在视频分析中达到了准确率与成本的帕累托前沿。
与模型以固定帧率静态处理媒体流不同，智能视频理解使 Gemini 能够主动、目标导向地决定观看内容的方式、速度以及通过哪种模态（帧、音频或文字记录）观看，只提取所需的时刻和信号。虽然开发者以前可以手动完成此操作，但通过智能视频理解，Gemini 可以通过智能循环调用内部工具加载视频文件的相关部分，从而显著减少开发开销。
智能视频理解改变了开发者在各种高要求应用中处理长视频内容的方式。
高效令牌长视频分析
看看 Gemini 3.7 Flash 在 LongVideoBench（长视频理解基准）上有无智能视频理解的表现。注意令牌的大幅减少以及准确性的提升。
动态帧速下的快速动作精准分析
通过智能视频理解，3.7 Flash 可以通过按需扫描和重放不同帧率的视频，准确统计快速动作的次数。
高效令牌中的大海捞针搜索
使用智能视频理解，Gemini 3.7 能够基于视频内容准确回答复杂问题，同时相比静态分析消耗的令牌显著减少。
我们的许多早期访问合作伙伴在测试智能视频理解时看到了强劲的性能表现。他们这样说：
智能视频理解可通过 Google AI Studio 的 Gemini API 间接访问：https://ai.google.dev/gemini-api/docs/video-understanding#agentic-video-understanding 以及 Gemini 企业代理平台：https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/capabilities/video-understanding，已在 Gemini 3.7 Flash、3.6 Flash 和 3.5 Flash-Lite 上推出。它使用标准的 Gemini API 令牌计费，无额外功能费用。
启用方法简单，只需在 API 配置中将处理模式设置为“agentic”。阅读我们的开发者指南：http://ai.dev/learn/agentic-video-understanding-with-gemini 以获取功能更多见解及入门方法。
我们还将代理视频理解的效率和质量提升带给谷歌产品中的数十亿用户。该功能将很快在 Gemini 应用的 Flash 和 Flash-Lite 模型中向所有用户推出。在接下来的几个月中，代理视频理解还将为 YouTube 的“Ask YouTube”: https://support.google.com/youtube/answer/14110396?hl=en&co=GENIE.Platform%3DAndroid 功能提供支持，该功能位于视频观看页面，利用 Gemini 提供基于视觉内容的更高质量答案。
感谢以下人员对这项工作的贡献：Sergi Caelles、Filip Pavetić、Ahmet Iscen、Suhas Yogin 以及代理视觉团队。
请检查您的收件箱以确认您的订阅。
Today, we’re launching agentic video understanding：https://ai.google.dev/gemini-api/docs/video-understanding#agentic-video-understanding across our latest models: Gemini 3.7 Flash, 3.6 Flash and 3.5 Flash-Lite. This new capability improves accuracy while dramatically reducing token usage and costs for video analysis. Similar to agentic vision：https://blog.google/innovation-and-ai/technology/developers-tools/agentic-vision-gemini-3-flash/, which combines code execution with Gemini models’ native image understanding, agentic video understanding uses Gemini’s native video tools to improve performance and unlock new capabilities for video processing like sub-second moment retrieval, more accurate anomaly detection, precise counting and more.
The feature is available today for video uploads and YouTube videos via the Gemini API in Google AI Studio and the Gemini Enterprise Agent Platform.
Unlike current ‘static’ processing, where the model ingests the video at a fixed frames-per-second rate (default 1 FPS, adjustable via API), agentic video understanding pairs the model’s core reasoning with native video tools to dynamically search, scan, and inspect target video segments across visual frames, audio, and transcripts. Across standard video analysis benchmarks, Gemini models with agentic video understanding reduce analysis costs by up to 66% and token consumption by up to 88%, while improving accuracy by up to 7%.
These efficiency gains are especially pronounced on long-form video (from 10-minute how-to guides to 90-minute lectures and multi-hour recordings), where static processing forces developers to choose between high token costs or techniques that drop critical details.
Activating agentic video understanding drops token consumption by up to 88% and boosts accuracy by up to 7% with Gemini 3.7 Flash.
While these gains span all three supported models, Gemini 3.7 Flash with agentic understanding offers the best possible quality overall and the best combination of quality and cost efficiency, putting it at the accuracy-to-cost pareto frontier among tested models for video understanding.
Using agentic video understanding places Gemini 3.7 Flash at the accuracy-to-cost pareto frontier for video analysis.
Instead of static processing where the model ingests media streams at a fixed frame rate, agentic video understanding enables Gemini to take an active, goal-directed role in determining what to watch, at what speed, and through which modality (frames, audio, or transcript), fetching only the moments and signals needed. While developers could previously do this manually, with agentic video understanding, Gemini can accomplish it through an agentic loop, invoking an internal tool to load the relevant part of the video file, significantly reducing development overheads.
Agentic video understanding transforms how developers can process long-form video content across a variety of demanding applications.
Token-efficient long-form video analysis
See how Gemini 3.7 Flash performs with and without agentic video understanding on LongVideoBench, a long-form video understanding benchmark. Notice the large token reductions and accuracy improvements.
Accurate fast action analysis with dynamic FPS
With agentic video understanding, 3.7 Flash is able to accurately count a fast-paced movement by scanning and rewatching the video at different frames per second, as needed.
Token-efficient needle-in-a-haystack search
Using agentic video understanding, Gemini 3.7 is able to accurately answer complex questions based on the content of the video while consuming a significantly lower number of tokens compared to static analysis.
Many of our early access partners saw strong performance while testing with agentic video understanding. Here’s what they have to say:
Agentic video understanding is available via the Gemini API in Google AI Studio：https://ai.google.dev/gemini-api/docs/video-understanding#agentic-video-understanding and Gemini Enterprise Agent Platform：https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/capabilities/video-understanding, launching across Gemini 3.7 Flash, 3.6 Flash, and 3.5 Flash-Lite. It uses standard Gemini API token pricing with no additional feature fee.
To enable it, simply set processing to "agentic" in the API configuration. Read our developer guide：http://ai.dev/learn/agentic-video-understanding-with-gemini to get more insights into the feature and how to get started.
We are also bringing the efficiency and quality improvements of agentic video understanding to billions of users across Google products. The feature will roll out to all users in the Gemini app across Flash and Flash-Lite models soon. And in the coming months, agentic video understanding will also power YouTube's ‘Ask YouTube’：https://support.google.com/youtube/answer/14110396?hl=en&co=GENIE.Platform%3DAndroid feature on the video watch page, leveraging Gemini to deliver higher-quality answers grounded in the visuals.
Acknowledgement for their contribution to this work: Sergi Caelles, Filip Pavetić, Ahmet Iscen, Suhas Yogin, and the Agentic Vision team.
Check your inbox to confirm your subscription.
扫码直达文章详情