Google 详解 Harness 工程：如何评估、迭代和守护 AI 编码智能体

当开发人员第一次开始为具代理性的编码系统进行工具开发时，他们经常会陷入同一个陷阱：他们运行像 Terminal-Bench 和 DeepSWE 这样的常见端到端基准，看到综合得分变化了几个百分点，但根本不知道为什么会变化。

端到端基准实际上是评估模型性能和确定哪些方面需要深入调查的标准方法，但问题在于这些调查的成本很高。

行为评估通常是更好的方法，它可以衡量你期望的行为是否真正发生，以及你是在朝正确方向前进还是由于回退或新模型变更而退步。它们可以作为你的迭代伙伴，帮助你洞察为什么某些变化会以某种方式影响结果。

这是我们对行为评估的看法，包括帮助我们在模型演化时保持代理系统可靠的方法。

大多数团队评估 AI 代理的方式就像评估一个参加考试的学生。他们交给代理一个庞大的代码库，给它一个时间限制，并根据有多少测试通过或失败来衡量其成功。

当得分下降时，到底哪里出了问题？

端到端基准通常不能直接回答这些问题。

行为评估的功能类似于提高代理工具操作的集成测试。当你拥有足够丰富的行为评估集时，你就有了针对代理目标行为的基线，并且可以迭代改进提示以达到目标。

行为评估不是衡量代理是否完成了整个多文件重构，而是衡量离散、可观察的动作：

不要在第一天就建立复杂的评估工具，而是利用这段时间跟随你的直觉并进行实验。

在从零开始引导一个代理时，你首先依靠开发者的直觉和自我测试。除非你已经构建了一个能够测试自身代码库、处理样板代码、编写自己的 Markdown 渲染器并执行常规开发任务的代理，否则运行评估没有意义。

评估属于开发的第二阶段：确保前进并防止回退。

评估套件的主要目的是不是为了在你让代理提升2%时庆祝；而是为了让你对新的提示调整、工具模式更改或模型升级没有整体上使代理变得更差有不可动摇的信心。

一个健壮的评估框架将行为断言分解为快速、确定性、类单元的检查，可在本地运行。

将你的注意力转向这些更小、可观察的动作会创建可靠的安全网。你可以自信地迭代你的系统提示或切换到不同的模型，因为你会立即知道是否不小心破坏了核心行为。

行为评估关注中间执行步骤，例如特定的工具调用或文件修改，而不是最终字符串的相等性：

借助丰富的行为评估套件，你可以自动化你的提示工程。例如，你可以设置一个循环，让大型语言模型调整自己的系统提示，迭代直到失败的测试最终通过，同时你的其余测试套件表现得类似于 CI/CD 风格的护栏运作。这有助于确保更改不会破坏任何现有功能。

从一开始，你可以做一些事情来让这个过程可重复。我建议你从一个三步行为测试循环开始：

你的代理不需要更高的基准分数就可以开始。它需要的是一个保持诚实的评估框架。

要构建一个稳定、具有弹性的评估工具，你必须停止将模型当作一个参加期末考试的黑箱来看待，而是开始将你的评估框架当作需要单元测试和集成测试的标准软件来处理。

虽然行为评估是评估框架工程的核心支柱，但它们不能替代更大规模的端到端评估套件。它们实际上是互补的。宏观基准验证最终目标，而微观行为评估作为伙伴，支持安全、快速的迭代。当你采用两者时，你在迭代时（如修改提示、开发新功能或部署全新模型）会有更高的信心水平。

宣布 Kotlin 版 ADK 1.0：在 Kotlin、Android 及更多平台构建生产就绪的 AI 代理

如何在 ADK 中评估实时和语音代理

推动开发者卓越：深入了解项目冲刺

When developers first work on harness engineering for agentic coding systems, they often fall into the same trap: they run common end-to-end benchmarks like Terminal-Bench and DeepSWE, watch a composite score move by a few percentage points, and have no idea why it changed.

End-to-end benchmarks are the de facto for evaluating model performance and determining what needs deeper investigation, but the challenge is that those investigations come at a high cost.

Behavioral evaluations are often a better measure of confidence on whether the behaviors you expect actually do happen and whether you’re moving in the right direction instead of backsliding when it comes to regressions or new model changes. They can serve as your iteration partner and help give insight into why certain changes move the needle in one way or another.

Here’s our take on behavioral evaluation, including approaches that have helped us keep agent systems reliable as models evolve.

Most teams evaluate AI agents like they would evaluate a student taking an exam. They hand the agent a large codebase, give it a time limit, and measure its success based on how many tests pass or fail.

When that score drops, what went wrong?

End-to-end benchmarks don’t typically directly answer these questions.

Behavioral evaluations function like integration tests for improving agent harness operation. When you have a rich enough behavioral eval set, you have a baseline for the behavior you're targeting from your agent, and you're able to iteratively improve the prompt to get there.

Instead of measuring whether the agent solved an entire multi-file refactor, a behavioral eval measures discrete, observable actions:

Instead of setting up a complex evaluation harness on day one, use this time to follow your hunches and run experiments.

When bootstrapping an agent from scratch, you start with developer instinct and dogfooding . Until you have built an agent capable of dogfooding its own codebase, handling boilerplate, writing its own markdown renderer, and executing routine developer tasks, it doesn’t make sense to run evaluations.

Evals belong to the second phase of development: ensuring forward progress and guarding against regressions .

The primary purpose of an evaluation suite is not to celebrate when you make the agent 2% better; it is to give you unshakeable confidence that a new prompt tweak, tool schema change, or model upgrade did not make the agent holistically worse .

A robust harness evaluation framework separates behavioral assertions into fast, deterministic, unit-style checks that run locally.

Shifting your focus to these smaller, observable actions creates a reliable safety net. You can confidently iterate on your system prompts or switch to a different model, because you’ll know immediately if you've accidentally broken a core behavior.

Behavioral evals assert on intermediate execution steps, like specific tool calls or file modifications, instead of final string equality:

With a rich suite of behavioral evals, you can automate your prompt engineering. For example, you can set up a loop where an LLM tweaks its own system prompt, iterating until a failing test finally passes, all while the rest of your test suite acts similar to how a CI/CD-style guardrail operates. This helps you ensure that the changes don’t break any existing features.

There are a few things you can do from the start to make this process repeatable. I suggest you start small with a three-step behavioral testing loop:

Your agent doesn't need a higher benchmark score to get started. It needs an evaluation harness that keeps it honest.

To build a stable, resilient harness, you have to stop treating your model like a black box passing a final exam, and start treating your harness like standard software that requires unit and integration testing.

While behavioral evaluations are a core pillar of harness engineering, they aren’t a replacement for larger, end-to-end evaluation suites. They’re actually complementary. Macro benchmarks verify the final destination and micro behavioral evals serve as a partner that enables safe, rapid iteration. When you adopt both, you'll have higher confidence levels when iterating, like when making prompt changes, building out new features, or even deploying brand-new models.

Announcing ADK for Kotlin 1.0: Building Production-Ready AI Agents in Kotlin, Android, and Beyond

How to Evaluate Live & Voice Agents in ADK

Driving Developer Excellence: Inside the Program Sprints