{"@context":"https://schema.org","@type":"NewsArticle","generatedAt":"2026-09-21T02:00:36.969Z","headline":"Google AI 团队分享如何为 LLM-as-a-Judge 评测编写可靠的评分标准","description":"Google AI 团队发布教程，讲解如何为 LLM-as-a-Judge 评测编写可靠的布尔式评分标准，指出模糊提示会导致评估不一致和浪费 token。文中给出四条经验：问题保持原子化且互不重叠、只让评判模型评估客观事实（可用 RFC 2119 术语如 MUST 表述）、只评 prompt 中明确要求的内容、用专家标注的 golden set 校准评判模型直至与人类评分一致。","url":"https://www.aioga.com/news/cmtkbz92801nmrowy61g2fsob/","mainEntityOfPage":"https://www.aioga.com/news/cmtkbz92801nmrowy61g2fsob/","datePublished":"2026-09-02T16:35:00.000Z","dateModified":"2026-09-02T16:35:00.000Z","inLanguage":"zh-CN","publisher":{"@type":"NewsMediaOrganization","name":"Aioga","url":"https://www.aioga.com"},"citation":["https://dev.to/googleai/how-to-write-reliable-rubrics-for-llm-as-a-judge-evaluations-ndp","https://aihot.virxact.com/items/cmtkbz92801nmrowy61g2fsob"],"canonicalUrl":"https://www.aioga.com/news/cmtkbz92801nmrowy61g2fsob/","directAnswer":{"@type":"Answer","text":"Google AI 团队发布教程，介绍如何为 LLM-as-a-Judge 评测编写可靠的布尔式评分标准。教程指出，模糊提示可能造成评估不一致并浪费 token，并总结了四条编写经验。","url":"https://www.aioga.com/news/cmtkbz92801nmrowy61g2fsob/","dateCreated":"2026-09-02T16:35:00.000Z","author":{"@type":"Organization","@id":"https://www.aioga.com/authors/aioga-editorial/#editorial-team","name":"Aioga Editorial Team","url":"https://www.aioga.com/authors/aioga-editorial/"}},"evidence":[{"@type":"CreativeWork","name":"dev.to source article","url":"https://dev.to/googleai/how-to-write-reliable-rubrics-for-llm-as-a-judge-evaluations-ndp","datePublished":"2026-09-02T16:35:00.000Z","provider":{"@type":"Organization","name":"dev.to","url":"https://dev.to/googleai/how-to-write-reliable-rubrics-for-llm-as-a-judge-evaluations-ndp"}},{"@type":"CreativeWork","name":"AIHot archive record","url":"https://aihot.virxact.com/items/cmtkbz92801nmrowy61g2fsob","datePublished":"2026-09-02T16:35:00.000Z","provider":{"@type":"Organization","name":"AIHot","url":"https://aihot.virxact.com/items/cmtkbz92801nmrowy61g2fsob"}}],"aggregationSource":"Google AI：DEV 作者专属（RSS）","originalPublisher":{"name":"dev.to","url":"https://dev.to/googleai/how-to-write-reliable-rubrics-for-llm-as-a-judge-evaluations-ndp"},"geoDeepAnswer":null,"article":{"id":"cmtkbz92801nmrowy61g2fsob","slug":"cmtkbz92801nmrowy61g2fsob","url":"https://www.aioga.com/news/cmtkbz92801nmrowy61g2fsob/","title":"Google AI 团队分享如何为 LLM-as-a-Judge 评测编写可靠的评分标准","title_en":"","summary":"Google AI 团队发布教程，讲解如何为 LLM-as-a-Judge 评测编写可靠的布尔式评分标准，指出模糊提示会导致评估不一致和浪费 token。文中给出四条经验：问题保持原子化且互不重叠、只让评判模型评估客观事实（可用 RFC 2119 术语如 MUST 表述）、只评 prompt 中明确要求的内容、用专家标注的 golden set 校准评判模型直至与人类评分一致。","source":"Google AI：DEV 作者专属（RSS）","sourceUrl":"https://dev.to/googleai/how-to-write-reliable-rubrics-for-llm-as-a-judge-evaluations-ndp","aiHotUrl":"https://aihot.virxact.com/items/cmtkbz92801nmrowy61g2fsob","publishedAt":"2026-09-02T16:35:00.000Z","category":"行业动态","score":72,"selected":true,"articleBody":["Follow up to Part 1: How to Design AI Evaluations You Can Actually Trust：https://dev.to/googleai/how-to-design-ai-evaluations-you-can-actually-trust-41c3","At Google, we are publishing a suite of Agent Skills for Google products and technologies on GitHub：https://github.com/google/skills. My team is interested in measuring their performance to understand how they perform. Deterministic tests, like checking if generated code compiles, are ideal. Unfortunately they can't be easily created at scale for nuanced, generative responses, such as answers to open-ended questions or information retrieval tasks.","In my previous post we looked at what you test：https://dev.to/googleai/how-to-design-ai-evaluations-you-can-actually-trust-41c3, which means the evaluations that are the actions you ask the agent to perform. The next step is to look at how you assert whether the agent succeeded. This means creating reliable and accurate assessments of the agent's response.","To assess complex outputs at scale, especially when topics cover broad areas with nuanced parts, we use an \"LLM-as-a-judge\" approach. Responses are assessed against a structured rubric using a model based grader. The judge evaluates each response using a set of true/false questions. Aggregated, these answers provide an accuracy score for a response.","Giving an LLM a vague prompt or subjective questions leads to ambiguity in its response. This ambiguity introduces noisy data and causes inconsistent evaluations. Ultimately, it wastes your token budget on metrics that aren't useful.","To make these assessments more reliable, you must treat your rubrics like formal specifications. By constraining the judge to evaluate strict, objective boolean truths, you reduce the chance of hallucinations. Because evaluating strict boolean truths is a less complex task, you can even use smaller, faster models for grading.","Here are four lessons we learned to help you write robust rubric questions for your LLM-as-a-judge scorer.","Evaluating multiple requirements in a single question, such as \"Does the response contain a metadata property and format the output as JSON?\" , forces the LLM judge to guess which clause is more important. This ambiguity leads to inconsistent grading and wasted tokens.","The rubric-based approach exists because giving an LLM judge a full prose prompt to evaluate a complex response leads to inconsistent numbers. If you ask the judge subjective questions like \"Was this a comprehensive answer?\" or ask it to explain \"Why did the agent do this?\" , you introduce ambiguity that creates noisy, unrepeatable data.","When building rubrics, it is easy to accidentally evaluate an agent on requirements that were never stated in the prompt. Doing this creates false negatives and reduces the accuracy of your measurement.","Even if you follow these rules and write perfectly atomic, objective questions, your LLM judge might still misinterpret your scoring instructions and rubrics. To guarantee your pipeline generates consistent ratings and reliable signals, you must prove that the judge's scoring aligns with how a human subject matter expert would evaluate the exact same response through calibration .","Once you have this reliable data, the next step is making it visible. In AI Evals at a Glance: Heatmaps for Stakeholders：https://dev.to/googleai/ai-evals-at-a-glance-heatmaps-for-stakeholders-2mki Joe Spiro explains how to take these raw measurements and visualize evaluations.","When building our agent skills, we learned that ambiguous evaluation rubrics do not provide useful signals and feedback. Forcing your LLM judge to evaluate strict, boolean facts eliminates this noise. It makes your tests repeatable, optimizes your token spend, and lets you confidently measure whether your AI tool is actually improving.","Photo by William Warby：https://unsplash.com/@wwarby on Unsplash：https://unsplash.com/photos/gray-and-yellow-measures-WahfNoqbYnM","Templates let you quickly answer FAQs or store snippets for re-use.","Are you sure you want to hide this comment? It will become hidden in your post, but will still be visible via the comment's permalink：#.","For further actions, you may consider blocking this person and/or reporting abuse：/report-abuse","Google AI Studio is the fastest way to start building with Gemini. Ready to build?","DEV Community：/ — A space to discuss and keep up software development and manage your software career","Built on Forem：https://www.forem.com — the open source：https://dev.to/t/opensource software that powers DEV：https://dev.to and other inclusive communities.","Made with love and Ruby on Rails：https://dev.to/t/rails. DEV Community © 2016 - 2026.","We're a place where coders share, stay up-to-date and grow their careers."],"articleImages":[{"sourceUrl":"https://media2.dev.to/dynamic/image/width=256,height=,fit=scale-down,gravity=auto,format=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F8j7kvp660rqzt99zui8e.png","alt":"pic","afterParagraph":13,"url":"/media/articles/cmtkbz92801nmrowy61g2fsob/f75d1e7bc8b434f4.webp"},{"sourceUrl":"https://assets.dev.to/assets/multi-unicorn-b44d6f8c23cdd00964192bedc38af3e82463978aa611b4365bd33a0f1f4f3e97.svg","alt":"","afterParagraph":21,"url":"/media/articles/cmtkbz92801nmrowy61g2fsob/ae2a1867ed3e47a7.jpg"},{"sourceUrl":"https://assets.dev.to/assets/exploding-head-daceb38d627e6ae9b730f36a1e390fca556a4289d5a41abb2c35068ad3e2c4b5.svg","alt":"","afterParagraph":21,"url":"/media/articles/cmtkbz92801nmrowy61g2fsob/c9631abcb96b6799.jpg"}],"mediaStatus":"ok","articleBodyZh":["跟进第一部分：如何设计你实际上可以信任的 AI 评估：https://dev.to/googleai/how-to-design-ai-evaluations-you-can-actually-trust-41c3","在 Google，我们正在 GitHub 上发布一系列适用于 Google 产品和技术的 Agent 技能：https://github.com/google/skills。我的团队有兴趣衡量它们的性能，以了解它们的表现情况。确定性的测试，比如检查生成的代码是否能编译，是理想的。不幸的是，对于细微的生成式响应（例如对开放性问题的回答或信息检索任务）来说，它们无法轻松大规模创建。","在我上一篇文章中，我们研究了你测试的内容：https://dev.to/googleai/how-to-design-ai-evaluations-you-can-actually-trust-41c3，这意味着评估是你要求代理执行的操作。下一步是看看你如何断定代理是否成功。这意味着需要创建对代理响应的可靠且准确的评估。","为了大规模评估复杂输出，尤其是当主题涉及广泛领域且包含细微部分时，我们使用“以大型语言模型（LLM）作为裁判”的方法。响应将根据结构化评分标准由模型评分器评估。裁判使用一组是/否问题来评估每个响应。汇总这些答案会为响应提供准确率分数。","给 LLM 提供模糊提示或主观问题会导致其响应出现不确定性。这种不确定性会引入噪声数据，导致评估不一致。最终，它会浪费你的代币预算在没有用处的指标上。","为了使这些评估更可靠，你必须将评分标准视为正式规范。通过将裁判限制为评估严格的、客观的布尔真值，你可以降低幻觉的可能性。由于评估严格布尔真值的任务较为简单，你甚至可以使用更小、更快的模型进行评分。","以下是我们学到的四条经验，帮助你为 LLM 作为裁判的评分器撰写稳健的评分标准问题。","在单个问题中评估多个要求，例如“响应是否包含元数据属性并将输出格式化为 JSON？”会迫使 LLM 裁判猜测哪个子句更重要。这种模糊性会导致评分不一致并浪费代币。","基于评分标准的方法存在的原因是，如果给一个大语言模型作为评审者一个完整的散文提示来评估复杂的回答，会导致评分结果不一致。如果您问评审者主观性问题，比如“这是一个全面的答案吗？”或者让它解释“为什么代理会这样做？”，就会引入模糊性，从而产生嘈杂、不具重复性的数据。","在建立评分标准时，很容易不小心在提示中未提出的要求上评估代理。这样会产生假阴性，降低测量的准确性。","即使您遵循这些规则并编写完美的、原子化的、客观的问题，您的大语言模型评审者仍可能误解您的评分说明和评分标准。为了保证您的流程生成一致的评分和可靠的信号，您必须通过校准证明评审者的评分与人类主题专家对完全相同回答的评估一致。","一旦您拥有这些可靠的数据，下一步就是让其可见。在《AI Evals at a Glance: Heatmaps for Stakeholders》中：https://dev.to/googleai/ai-evals-at-a-glance-heatmaps-for-stakeholders-2mki，Joe Spiro 讲解了如何将这些原始测量数据可视化为评估结果。","在构建我们的代理技能时，我们了解到，模糊的评估评分标准无法提供有用的信号和反馈。强迫大语言模型评审者评估严格的布尔事实可以消除这些噪声。它使测试可重复，优化令牌消耗，并让您能够自信地衡量您的 AI 工具是否真正提升了性能。","照片由 William Warby 拍摄：https://unsplash.com/@wwarby，来自 Unsplash：https://unsplash.com/photos/gray-and-yellow-measures-WahfNoqbYnM","模板让您能够快速回答常见问题或存储可重复使用的片段。","您确定要隐藏此评论吗？它会在您的帖子中隐藏，但通过评论的永久链接仍可见：#。","对于进一步操作，您可以考虑屏蔽此人和/或举报滥用行为：/report-abuse","Google AI Studio 是开始构建 Gemini 的最快方式。准备好开始了吗？","DEV 社区：/ — 一个用于讨论、追踪软件开发并管理软件职业的平台","构建于 Forem：https://www.forem.com — 这个开源软件：https://dev.to/t/opensource 支撑着 DEV：https://dev.to 以及其他包容性社区。","用爱和 Ruby on Rails 制作：https://dev.to/t/rails。DEV 社区 © 2016 - 2026。","我们是一个程序员分享、保持更新并发展职业的平台。"],"translationStatus":"translated","bodyOrigin":"source-page","editorial":{"summary":"Google AI 团队发布教程，介绍如何为 LLM-as-a-Judge 评测编写可靠的布尔式评分标准。教程指出，模糊提示可能造成评估不一致并浪费 token，并总结了四条编写经验。","background":"文章称，确定性测试适合检查代码是否编译，但开放式问答和信息检索等细腻的生成式响应难以大规模测试，因此采用模型评判器依据结构化评分标准，以一组真或假问题评估响应。","viewpoint":"Aioga 判断：将评分标准视为形式化规格，有助于把评测要求拆解为更明确的客观判断，但可靠性仍取决于问题设计和校准过程。","implications":"可能影响：原子化、互不重叠且只覆盖明确要求的评分问题，可能减少歧义；但这不代表模型评判天然等同于人工评分，仍需要专家标注的 golden set 校准。","nextStep":"后续观察：应关注相关评分标准是否落实四条经验，以及评判模型经过专家标注集校准后，能否与人类评分保持一致。","evidenceRefs":["title","summary","articleBody","source"],"status":"published","aiGenerated":true,"autoApproved":true,"generatedBy":"aioga-editorial:gpt-5.6-sol","reviewedBy":"aioga-editorial-review:gpt-5.6-sol","generatedAt":"2026-09-02T17:17:15.057Z","sourceHash":"7a3de5dc8423ffa3","review":{"approved":true,"groundedness":94,"clarity":92,"duplicationRisk":18,"blockingIssues":[],"notes":["“可能减少歧义”属于基于来源结论的合理概括；如需更严格，可改为“有助于减少歧义和幻觉风险”。","候选内容提及四条经验，但所给正文摘录仅直接展开了原子化且互不重叠这一条；其余经验由来源摘要支持。"]},"validation":{"passed":true,"mode":"ai-auto","revisions":0,"checks":["schema","length","source-attribution","editorial-labels","inference-boundary","low-source-overlap","no-html","independent-ai-review"]}},"tags":["行业动态","Google AI：DEV 作者专属（RSS）"],"translations":{"zh-CN":{"title":"Google AI 团队分享如何为 LLM-as-a-Judge 评测编写可靠的评分标准","summary":"Google AI 团队发布教程，讲解如何为 LLM-as-a-Judge 评测编写可靠的布尔式评分标准，指出模糊提示会导致评估不一致和浪费 token。文中给出四条经验：问题保持原子化且互不重叠、只让评判模型评估客观事实（可用 RFC 2119 术语如 MUST 表述）、只评 prompt 中明确要求的内容、用专家标注的 golden set 校准评判模型直至与人类评分一致。","category":"行业动态","source":"dev.to","aggregationSource":"Google AI：DEV 作者专属（RSS）","pageTitle":"Google AI 团队分享如何为 LLM-as-a-Judge 评测编写可靠的评分标准 - Aioga AI资讯","description":"Google AI 团队发布教程，讲解如何为 LLM-as-a-Judge 评测编写可靠的布尔式评分标准，指出模糊提示会导致评估不一致和浪费 token。文中给出四条经验：问题保持原子化且互不重叠、只让评判模型评估客观事实（可用 RFC 2119 术语如 MUST 表述）、只评 prompt 中明确要求的内容、用专家标注的 golden set 校准评判模型...","url":"https://www.aioga.com/news/cmtkbz92801nmrowy61g2fsob/","articleBody":["跟进第一部分：如何设计你实际上可以信任的 AI 评估：https://dev.to/googleai/how-to-design-ai-evaluations-you-can-actually-trust-41c3","在 Google，我们正在 GitHub 上发布一系列适用于 Google 产品和技术的 Agent 技能：https://github.com/google/skills。我的团队有兴趣衡量它们的性能，以了解它们的表现情况。确定性的测试，比如检查生成的代码是否能编译，是理想的。不幸的是，对于细微的生成式响应（例如对开放性问题的回答或信息检索任务）来说，它们无法轻松大规模创建。","在我上一篇文章中，我们研究了你测试的内容：https://dev.to/googleai/how-to-design-ai-evaluations-you-can-actually-trust-41c3，这意味着评估是你要求代理执行的操作。下一步是看看你如何断定代理是否成功。这意味着需要创建对代理响应的可靠且准确的评估。","为了大规模评估复杂输出，尤其是当主题涉及广泛领域且包含细微部分时，我们使用“以大型语言模型（LLM）作为裁判”的方法。响应将根据结构化评分标准由模型评分器评估。裁判使用一组是/否问题来评估每个响应。汇总这些答案会为响应提供准确率分数。","给 LLM 提供模糊提示或主观问题会导致其响应出现不确定性。这种不确定性会引入噪声数据，导致评估不一致。最终，它会浪费你的代币预算在没有用处的指标上。","为了使这些评估更可靠，你必须将评分标准视为正式规范。通过将裁判限制为评估严格的、客观的布尔真值，你可以降低幻觉的可能性。由于评估严格布尔真值的任务较为简单，你甚至可以使用更小、更快的模型进行评分。","以下是我们学到的四条经验，帮助你为 LLM 作为裁判的评分器撰写稳健的评分标准问题。","在单个问题中评估多个要求，例如“响应是否包含元数据属性并将输出格式化为 JSON？”会迫使 LLM 裁判猜测哪个子句更重要。这种模糊性会导致评分不一致并浪费代币。","基于评分标准的方法存在的原因是，如果给一个大语言模型作为评审者一个完整的散文提示来评估复杂的回答，会导致评分结果不一致。如果您问评审者主观性问题，比如“这是一个全面的答案吗？”或者让它解释“为什么代理会这样做？”，就会引入模糊性，从而产生嘈杂、不具重复性的数据。","在建立评分标准时，很容易不小心在提示中未提出的要求上评估代理。这样会产生假阴性，降低测量的准确性。","即使您遵循这些规则并编写完美的、原子化的、客观的问题，您的大语言模型评审者仍可能误解您的评分说明和评分标准。为了保证您的流程生成一致的评分和可靠的信号，您必须通过校准证明评审者的评分与人类主题专家对完全相同回答的评估一致。","一旦您拥有这些可靠的数据，下一步就是让其可见。在《AI Evals at a Glance: Heatmaps for Stakeholders》中：https://dev.to/googleai/ai-evals-at-a-glance-heatmaps-for-stakeholders-2mki，Joe Spiro 讲解了如何将这些原始测量数据可视化为评估结果。","在构建我们的代理技能时，我们了解到，模糊的评估评分标准无法提供有用的信号和反馈。强迫大语言模型评审者评估严格的布尔事实可以消除这些噪声。它使测试可重复，优化令牌消耗，并让您能够自信地衡量您的 AI 工具是否真正提升了性能。","照片由 William Warby 拍摄：https://unsplash.com/@wwarby，来自 Unsplash：https://unsplash.com/photos/gray-and-yellow-measures-WahfNoqbYnM","模板让您能够快速回答常见问题或存储可重复使用的片段。","您确定要隐藏此评论吗？它会在您的帖子中隐藏，但通过评论的永久链接仍可见：#。","对于进一步操作，您可以考虑屏蔽此人和/或举报滥用行为：/report-abuse","Google AI Studio 是开始构建 Gemini 的最快方式。准备好开始了吗？","DEV 社区：/ — 一个用于讨论、追踪软件开发并管理软件职业的平台","构建于 Forem：https://www.forem.com — 这个开源软件：https://dev.to/t/opensource 支撑着 DEV：https://dev.to 以及其他包容性社区。","用爱和 Ruby on Rails 制作：https://dev.to/t/rails。DEV 社区 © 2016 - 2026。","我们是一个程序员分享、保持更新并发展职业的平台。"]},"en":{"title":"Google AI team shares how to write reliable scoring criteria for LLM-as-a-Judge evaluation","summary":"The Google AI team released a tutorial explaining how to write reliable Boolean scoring criteria for evaluating LLM-as-a-Judge, pointing out that vague prompts can lead to inconsistent assessments and wasted tokens. The article provides four pieces of advice: keep questions atomic and non-overlapping, have the judging model evaluate only objective facts (which can be expressed using RFC 2119 terms like MUST), evaluate only the content explicitly requested in the prompt, and calibrate the judging model with an expert-labeled golden set until it aligns with human scoring.","category":"Industry","source":"Google AI：DEV 作者专属（RSS）","aggregationSource":"Google AI：DEV 作者专属（RSS）","pageTitle":"Google AI team shares how to write reliable scoring criteria for LLM-as-a-Judge evaluation - Aioga AI News","description":"The Google AI team released a tutorial explaining how to write reliable Boolean scoring criteria for evaluating LLM-as-a-Judge, pointing out that vague prompts can lead to inconsis...","url":"https://www.aioga.com/en/news/cmtkbz92801nmrowy61g2fsob/","contentTranslated":true,"sourceHash":"35ee1c62e4ae7343","translatedAt":"2026-09-02T17:04:16.407Z"},"ja":{"title":"Google AI チームが LLM-as-a-Judge の評価のために信頼できる評価基準を作成する方法を共有","summary":"Google AI チームがチュートリアルを公開し、LLM-as-a-Judge 評価のために信頼できるブール型スコア基準を作成する方法を説明しました。曖昧なプロンプトは評価の不一致やトークンの無駄遣いを引き起こすと指摘しています。文中では4つの経験則が示されています：質問は原子的かつ重複しないようにすること、審査モデルには客観的事実のみを評価させること（RFC 2119 の用語、例えば MUST を使用可能）、プロンプトで明確に要求された内容のみを評価すること、専門家がラベル付けしたゴールデンセットを使って審査モデルを人間の評価と一致するまでキャリブレーションすること。","category":"業界動向","source":"Google AI：DEV 作者专属（RSS）","aggregationSource":"Google AI：DEV 作者专属（RSS）","pageTitle":"Google AI チームが LLM-as-a-Judge の評価のために信頼できる評価基準を作成する方法を共有 - Aioga AIニュース","description":"Google AI チームがチュートリアルを公開し、LLM-as-a-Judge 評価のために信頼できるブール型スコア基準を作成する方法を説明しました。曖昧なプロンプトは評価の不一致やトークンの無駄遣いを引き起こすと指摘しています。文中では4つの経験則が示されています：質問は原子的かつ重複しないようにすること、審査モデルには客観的事実のみを評価させること（R...","url":"https://www.aioga.com/ja/news/cmtkbz92801nmrowy61g2fsob/","contentTranslated":true,"sourceHash":"35ee1c62e4ae7343","translatedAt":"2026-09-02T17:04:26.787Z"},"ko":{"title":"Google AI 팀이 LLM-as-a-Judge 평가를 위한 신뢰할 수 있는 채점 기준 작성 방법을 공유하다","summary":"구글 AI 팀이 튜토리얼을 발표하여 LLM-as-a-Judge 평가를 위한 신뢰할 수 있는 불리언식 점수 기준 작성 방법을 설명하고, 모호한 프롬프트가 평가 불일치와 토큰 낭비를 초래할 수 있다고 지적했습니다. 글에서는 네 가지 경험을 제시합니다: 문제를 원자화하고 서로 겹치지 않게 유지, 평가 모델이 객관적 사실만 평가하도록 제한(RFC 2119 용어인 MUST 등 사용 가능), 프롬프트에서 명확히 요구한 내용만 평가, 전문가가 표시한 golden set으로 평가 모델을 보정하여 인간 평가와 일치시킬 때까지 조정.","category":"업계 동향","source":"Google AI：DEV 作者专属（RSS）","aggregationSource":"Google AI：DEV 作者专属（RSS）","pageTitle":"Google AI 팀이 LLM-as-a-Judge 평가를 위한 신뢰할 수 있는 채점 기준 작성 방법을 공유하다 - Aioga AI 뉴스","description":"구글 AI 팀이 튜토리얼을 발표하여 LLM-as-a-Judge 평가를 위한 신뢰할 수 있는 불리언식 점수 기준 작성 방법을 설명하고, 모호한 프롬프트가 평가 불일치와 토큰 낭비를 초래할 수 있다고 지적했습니다. 글에서는 네 가지 경험을 제시합니다: 문제를 원자화하고 서로 겹치지 않게 유지, 평가 모델이 객관적 사실만 평가...","url":"https://www.aioga.com/ko/news/cmtkbz92801nmrowy61g2fsob/","contentTranslated":true,"sourceHash":"35ee1c62e4ae7343","translatedAt":"2026-09-02T17:05:23.344Z"},"es":{"title":"El equipo de Google AI comparte cómo redactar criterios de evaluación confiables para LLM-como-juez","summary":"El equipo de Google AI publicó un tutorial que explica cómo redactar criterios de puntuación booleanos confiables para la evaluación de LLM como juez, señalando que los indicadores ambiguos pueden provocar evaluaciones inconsistentes y desperdicio de tokens. El artículo presenta cuatro experiencias: mantener las preguntas atomizadas y sin superposición, hacer que el modelo juez evalúe solo hechos objetivos (puede usar términos de RFC 2119 como MUST), evaluar únicamente lo que se solicita explícitamente en el prompt, y calibrar el modelo juez con un conjunto dorado etiquetado por expertos hasta que coincida con la puntuación humana.","category":"Industria","source":"Google AI：DEV 作者专属（RSS）","aggregationSource":"Google AI：DEV 作者专属（RSS）","pageTitle":"El equipo de Google AI comparte cómo redactar criterios de evaluación confiables para LLM-como-juez - Aioga Noticias de IA","description":"El equipo de Google AI publicó un tutorial que explica cómo redactar criterios de puntuación booleanos confiables para la evaluación de LLM como juez, señalando que los indicadores...","url":"https://www.aioga.com/es/news/cmtkbz92801nmrowy61g2fsob/","contentTranslated":true,"sourceHash":"35ee1c62e4ae7343","translatedAt":"2026-09-02T17:05:17.541Z"},"fr":{"title":"L'équipe Google AI partage comment rédiger des critères d'évaluation fiables pour évaluer LLM en tant que juge","summary":"L'équipe Google AI a publié un tutoriel expliquant comment rédiger des critères de notation booléens fiables pour l'évaluation des LLM en tant que juge, en soulignant que des instructions vagues peuvent entraîner des évaluations incohérentes et un gaspillage de tokens. L'article donne quatre conseils : garder les questions atomiques et non chevauchantes, ne faire évaluer par le modèle juge que des faits objectifs (en utilisant éventuellement les termes RFC 2119 tels que MUST), ne traiter que ce qui est explicitement demandé dans le prompt, et calibrer le modèle juge avec un ensemble de données golden annoté par des experts jusqu'à obtenir une concordance avec les évaluations humaines.","category":"Industrie","source":"Google AI：DEV 作者专属（RSS）","aggregationSource":"Google AI：DEV 作者专属（RSS）","pageTitle":"L'équipe Google AI partage comment rédiger des critères d'évaluation fiables pour évaluer LLM en tant que juge - Aioga Actualités IA","description":"L'équipe Google AI a publié un tutoriel expliquant comment rédiger des critères de notation booléens fiables pour l'évaluation des LLM en tant que juge, en soulignant que des instr...","url":"https://www.aioga.com/fr/news/cmtkbz92801nmrowy61g2fsob/","contentTranslated":true,"sourceHash":"35ee1c62e4ae7343","translatedAt":"2026-09-02T17:06:19.969Z"},"de":{"title":"Das Google AI-Team teilt, wie man zuverlässige Bewertungskriterien für LLM-as-a-Judge-Tests erstellt","summary":"Das Google AI-Team hat ein Tutorial veröffentlicht, das erklärt, wie man zuverlässige boolesche Bewertungskriterien für die Bewertung von LLM-as-a-Judge erstellt, und weist darauf hin, dass unklare Aufforderungen zu inkonsistenten Bewertungen und Verschwendung von Tokens führen. Im Text werden vier Erfahrungen angegeben: Die Fragen atomar und nicht überlappend halten, das Bewertungsmodell nur objektive Fakten beurteilen lassen (kann mit RFC 2119-Termini wie MUST ausgedrückt werden), nur die Inhalte bewerten, die im Prompt ausdrücklich verlangt werden, und das Bewertungsmodell mit einem von Experten gekennzeichneten Golden Set kalibrieren, bis es mit menschlichen Bewertungen übereinstimmt.","category":"行业动态","source":"Google AI：DEV 作者专属（RSS）","aggregationSource":"Google AI：DEV 作者专属（RSS）","pageTitle":"Das Google AI-Team teilt, wie man zuverlässige Bewertungskriterien für LLM-as-a-Judge-Tests erstellt - Aioga KI-News","description":"Das Google AI-Team hat ein Tutorial veröffentlicht, das erklärt, wie man zuverlässige boolesche Bewertungskriterien für die Bewertung von LLM-as-a-Judge erstellt, und weist darauf...","url":"https://www.aioga.com/de/news/cmtkbz92801nmrowy61g2fsob/","contentTranslated":true,"sourceHash":"35ee1c62e4ae7343","translatedAt":"2026-09-02T17:06:06.944Z"},"pt-BR":{"title":"A equipe de IA do Google compartilha como criar critérios de avaliação confiáveis para avaliar LLM como juiz","summary":"A equipe de IA do Google publicou um tutorial explicando como escrever critérios de pontuação booleanos confiáveis para avaliação de LLM-as-a-Judge, apontando que instruções vagas podem levar a avaliações inconsistentes e desperdício de tokens. O artigo apresenta quatro experiências: manter as questões atomizadas e não sobrepostas, fazer o modelo avaliador julgar apenas fatos objetivos (podendo usar termos do RFC 2119 como MUST), avaliar apenas o que é explicitamente solicitado no prompt e calibrar o modelo avaliador com um conjunto dourado de anotações de especialistas até que sua avaliação seja consistente com a pontuação humana.","category":"行业动态","source":"Google AI：DEV 作者专属（RSS）","aggregationSource":"Google AI：DEV 作者专属（RSS）","pageTitle":"A equipe de IA do Google compartilha como criar critérios de avaliação confiáveis para avaliar LLM como juiz - Aioga Notícias de IA","description":"A equipe de IA do Google publicou um tutorial explicando como escrever critérios de pontuação booleanos confiáveis para avaliação de LLM-as-a-Judge, apontando que instruções vagas...","url":"https://www.aioga.com/pt-BR/news/cmtkbz92801nmrowy61g2fsob/","contentTranslated":true,"sourceHash":"35ee1c62e4ae7343","translatedAt":"2026-09-02T17:07:06.041Z"},"ru":{"title":"Команда Google AI делится тем, как разработать надежные критерии оценки для LLM в роли судьи","summary":"Команда Google AI выпустила руководство, объясняющее, как составлять надежные булевые критерии оценки для LLM-as-a-Judge, указывая, что неопределённые подсказки приводят к несогласованной оценке и трате токенов. В статье приведены четыре рекомендации: вопросы должны оставаться атомарными и не перекрываться; позволять модели оценивать только объективные факты (можно использовать термины RFC 2119, такие как MUST); оценивать только то, что явно требуется в подсказке; калибровать модель оценки с помощью золотого набора, размеченного экспертами, до достижения согласия с оценками человека.","category":"行业动态","source":"Google AI：DEV 作者专属（RSS）","aggregationSource":"Google AI：DEV 作者专属（RSS）","pageTitle":"Команда Google AI делится тем, как разработать надежные критерии оценки для LLM в роли судьи - Aioga Новости ИИ","description":"Команда Google AI выпустила руководство, объясняющее, как составлять надежные булевые критерии оценки для LLM-as-a-Judge, указывая, что неопределённые подсказки приводят к несоглас...","url":"https://www.aioga.com/ru/news/cmtkbz92801nmrowy61g2fsob/","contentTranslated":true,"sourceHash":"35ee1c62e4ae7343","translatedAt":"2026-09-02T17:07:12.897Z"},"ar":{"title":"شارك فريق جوجل للذكاء الاصطناعي كيفية كتابة معايير تقييم موثوقة لتقييم LLM بصفتها قاضياً","summary":"نشرت فريق Google AI دليلًا يشرح كيفية كتابة معايير تقييم منطقية موثوقة لتقييم LLM-as-a-Judge، مشيرًا إلى أن التعليمات الغامضة قد تؤدي إلى تقييم غير متسق وإهدار الرموز. وقدم المقال أربع خبرات: الحفاظ على الأسئلة ذرية وغير متداخلة، جعل نموذج التقييم يقيم الحقائق الموضوعية فقط (يمكن استخدام مصطلحات RFC 2119 مثل MUST)، تقييم المحتوى المطلوب بوضوح فقط في التعليمات، استخدام مجموعة ذهبية موسومة من قبل الخبراء لمعايرة نموذج التقييم حتى يتوافق مع التقييم البشري.","category":"行业动态","source":"Google AI：DEV 作者专属（RSS）","aggregationSource":"Google AI：DEV 作者专属（RSS）","pageTitle":"شارك فريق جوجل للذكاء الاصطناعي كيفية كتابة معايير تقييم موثوقة لتقييم LLM بصفتها قاضياً - Aioga أخبار الذكاء الاصطناعي","description":"نشرت فريق Google AI دليلًا يشرح كيفية كتابة معايير تقييم منطقية موثوقة لتقييم LLM-as-a-Judge، مشيرًا إلى أن التعليمات الغامضة قد تؤدي إلى تقييم غير متسق وإهدار الرموز. وقدم المقال...","url":"https://www.aioga.com/ar/news/cmtkbz92801nmrowy61g2fsob/","contentTranslated":true,"sourceHash":"35ee1c62e4ae7343","translatedAt":"2026-09-02T17:08:05.631Z"},"hi":{"title":"गूगल एआई टीम ने साझा किया कि LLM-ए-ए-जज मूल्यांकन के लिए विश्वसनीय मूल्यांकन मानदंड कैसे तैयार करें","summary":"Google AI टीम ने एक ट्यूटोरियल जारी किया, जिसमें यह समझाया गया कि LLM-as-a-Judge मूल्यांकन के लिए विश्वसनीय बूलियन-शैली के स्कोरिंग मानदंड कैसे तैयार किए जाएं, और यह बताया कि अस्पष्ट संकेतों से असंगत मूल्यांकन और टोकन की बर्बादी हो सकती है। लेख में चार अनुभव बताए गए हैं: प्रश्नों को परमाणु और आपस में ओवरलैप न होने दें, केवल न्यायाधीश मॉडल से वस्तुनिष्ठ तथ्यों का मूल्यांकन करवाएं (RFC 2119 जैसी शब्दावली जैसे MUST का उपयोग किया जा सकता है), केवल प्रॉम्प्ट में स्पष्ट रूप से मांगी गई सामग्री का ही मूल्यांकन करवाएं, और विशेषज्ञों द्वारा चिह्नित गोल्डन सेट का उपयोग करके न्यायाधीश मॉडल को मानव स्कोरिंग के साथ मेल खाने तक कैलिब्रेट करें।","category":"行业动态","source":"Google AI：DEV 作者专属（RSS）","aggregationSource":"Google AI：DEV 作者专属（RSS）","pageTitle":"गूगल एआई टीम ने साझा किया कि LLM-ए-ए-जज मूल्यांकन के लिए विश्वसनीय मूल्यांकन मानदंड कैसे तैयार करें - Aioga AI समाचार","description":"Google AI टीम ने एक ट्यूटोरियल जारी किया, जिसमें यह समझाया गया कि LLM-as-a-Judge मूल्यांकन के लिए विश्वसनीय बूलियन-शैली के स्कोरिंग मानदंड कैसे तैयार किए जाएं, और यह बताया कि अस्पष...","url":"https://www.aioga.com/hi/news/cmtkbz92801nmrowy61g2fsob/","contentTranslated":true,"sourceHash":"35ee1c62e4ae7343","translatedAt":"2026-09-02T17:08:09.411Z"},"it":{"title":"Il team di Google AI condivide come scrivere criteri di valutazione affidabili per la valutazione LLM-as-a-Judge","summary":"Il team di Google AI ha pubblicato un tutorial che spiega come scrivere criteri di valutazione booleani affidabili per la valutazione di LLM come giudice, sottolineando che prompt vaghi possono portare a valutazioni incoerenti e allo spreco di token. L'articolo fornisce quattro esperienze: mantenere le domande atomiche e non sovrapposte, far valutare al modello giudice solo fatti oggettivi (utilizzando termini RFC 2119 come MUST), limitarsi a quanto richiesto esplicitamente nel prompt, calibrare il modello giudice con un set d'oro annotato da esperti finché non corrisponde alla valutazione umana.","category":"行业动态","source":"Google AI：DEV 作者专属（RSS）","aggregationSource":"Google AI：DEV 作者专属（RSS）","pageTitle":"Il team di Google AI condivide come scrivere criteri di valutazione affidabili per la valutazione LLM-as-a-Judge - Aioga Notizie IA","description":"Il team di Google AI ha pubblicato un tutorial che spiega come scrivere criteri di valutazione booleani affidabili per la valutazione di LLM come giudice, sottolineando che prompt...","url":"https://www.aioga.com/it/news/cmtkbz92801nmrowy61g2fsob/","contentTranslated":true,"sourceHash":"35ee1c62e4ae7343","translatedAt":"2026-09-02T17:08:59.729Z"},"nl":{"title":"Het Google AI-team deelt hoe je betrouwbare beoordelingscriteria opstelt voor LLM-as-a-Judge evaluaties","summary":"Het Google AI-team heeft een handleiding uitgebracht waarin wordt uitgelegd hoe betrouwbare Booleaanse beoordelingscriteria kunnen worden opgesteld voor LLM-as-a-Judge evaluaties, en wijst erop dat vage prompts tot inconsistente evaluaties en verspilling van tokens leiden. In het artikel worden vier lessen gegeven: houd vragen atomair en niet-overlappend, laat het beoordelende model alleen objectieve feiten beoordelen (kan worden uitgedrukt met termen uit RFC 2119 zoals MUST), laat het alleen de in de prompt expliciet gevraagde inhoud beoordelen, en kalibreer het beoordelende model met een door experts gemarkeerde golden set totdat het overeenkomt met menselijke beoordelingen.","category":"行业动态","source":"Google AI：DEV 作者专属（RSS）","aggregationSource":"Google AI：DEV 作者专属（RSS）","pageTitle":"Het Google AI-team deelt hoe je betrouwbare beoordelingscriteria opstelt voor LLM-as-a-Judge evaluaties - Aioga AI-nieuws","description":"Het Google AI-team heeft een handleiding uitgebracht waarin wordt uitgelegd hoe betrouwbare Booleaanse beoordelingscriteria kunnen worden opgesteld voor LLM-as-a-Judge evaluaties,...","url":"https://www.aioga.com/nl/news/cmtkbz92801nmrowy61g2fsob/","contentTranslated":true,"sourceHash":"35ee1c62e4ae7343","translatedAt":"2026-09-02T17:08:54.823Z"},"tr":{"title":"Google AI ekibi, LLM-as-a-Judge değerlendirmesi için güvenilir puanlama kriterlerinin nasıl yazılacağını paylaştı","summary":"Google AI ekibi bir rehber yayımladı; bu rehber, LLM-as-a-Judge değerlendirmesi için güvenilir boolean tabanlı puanlama kriterlerinin nasıl yazılacağını anlatıyor ve belirsiz ipuçlarının değerlendirmede tutarsızlığa ve token israfına yol açacağını belirtiyor. Makalede dört deneyim paylaşılıyor: soruları atomik ve birbirini örtmeyecek şekilde tutmak, yalnızca yargılayan modelin nesnel gerçekleri değerlendirmesine izin vermek (RFC 2119 terimleri gibi MUST kullanılabilir), yalnızca prompt'ta açıkça talep edilen içeriği değerlendirmek ve uzmanların işaretlediği altın seti kullanarak değerlendirme modelini insan puanlarıyla uyumlu olana kadar kalibre etmek.","category":"行业动态","source":"Google AI：DEV 作者专属（RSS）","aggregationSource":"Google AI：DEV 作者专属（RSS）","pageTitle":"Google AI ekibi, LLM-as-a-Judge değerlendirmesi için güvenilir puanlama kriterlerinin nasıl yazılacağını paylaştı - Aioga AI Haberleri","description":"Google AI ekibi bir rehber yayımladı; bu rehber, LLM-as-a-Judge değerlendirmesi için güvenilir boolean tabanlı puanlama kriterlerinin nasıl yazılacağını anlatıyor ve belirsiz ipuçl...","url":"https://www.aioga.com/tr/news/cmtkbz92801nmrowy61g2fsob/","contentTranslated":true,"sourceHash":"35ee1c62e4ae7343","translatedAt":"2026-09-02T17:09:56.529Z"},"vi":{"title":"Nhóm Google AI chia sẻ cách viết tiêu chí đánh giá đáng tin cậy cho LLM-as-a-Judge","summary":"Đội ngũ Google AI đã phát hành hướng dẫn, giải thích cách viết tiêu chuẩn đánh giá kiểu Boolean đáng tin cậy cho LLM-as-a-Judge, nêu rõ rằng các gợi ý mơ hồ sẽ dẫn đến đánh giá không nhất quán và lãng phí token. Bài viết đưa ra bốn kinh nghiệm: giữ các câu hỏi nguyên tử và không chồng chéo nhau, chỉ để mô hình đánh giá một cách khách quan các sự kiện (có thể dùng thuật ngữ RFC 2119 như MUST để diễn đạt), chỉ đánh giá những nội dung được yêu cầu rõ ràng trong prompt, dùng bộ dữ liệu vàng được chuyên gia gán nhãn để hiệu chuẩn mô hình đánh giá cho đến khi đạt kết quả nhất quán với đánh giá của con người.","category":"行业动态","source":"Google AI：DEV 作者专属（RSS）","aggregationSource":"Google AI：DEV 作者专属（RSS）","pageTitle":"Nhóm Google AI chia sẻ cách viết tiêu chí đánh giá đáng tin cậy cho LLM-as-a-Judge - Tin tức AI Aioga","description":"Đội ngũ Google AI đã phát hành hướng dẫn, giải thích cách viết tiêu chuẩn đánh giá kiểu Boolean đáng tin cậy cho LLM-as-a-Judge, nêu rõ rằng các gợi ý mơ hồ sẽ dẫn đến đánh giá khô...","url":"https://www.aioga.com/vi/news/cmtkbz92801nmrowy61g2fsob/","contentTranslated":true,"sourceHash":"35ee1c62e4ae7343","translatedAt":"2026-09-02T17:10:00.317Z"},"id":{"title":"Tim Google AI berbagi bagaimana menyusun kriteria penilaian yang dapat diandalkan untuk evaluasi LLM-als-Judge","summary":"Tim Google AI merilis tutorial yang menjelaskan cara menulis kriteria penilaian berbasis boolean yang andal untuk evaluasi LLM-as-a-Judge, menunjukkan bahwa petunjuk yang ambigu akan menyebabkan evaluasi yang tidak konsisten dan membuang token. Artikel ini memberikan empat pengalaman: pertahankan pertanyaan tetap atomik dan tidak saling tumpang tindih, hanya biarkan model penilai menilai fakta objektif (dapat menggunakan istilah RFC 2119 seperti MUST), hanya nilai konten yang secara eksplisit diminta dalam prompt, gunakan set emas yang diberi anotasi ahli untuk mengkalibrasi model penilai hingga sesuai dengan penilaian manusia.","category":"行业动态","source":"Google AI：DEV 作者专属（RSS）","aggregationSource":"Google AI：DEV 作者专属（RSS）","pageTitle":"Tim Google AI berbagi bagaimana menyusun kriteria penilaian yang dapat diandalkan untuk evaluasi LLM-als-Judge - Berita AI Aioga","description":"Tim Google AI merilis tutorial yang menjelaskan cara menulis kriteria penilaian berbasis boolean yang andal untuk evaluasi LLM-as-a-Judge, menunjukkan bahwa petunjuk yang ambigu ak...","url":"https://www.aioga.com/id/news/cmtkbz92801nmrowy61g2fsob/","contentTranslated":true,"sourceHash":"35ee1c62e4ae7343","translatedAt":"2026-09-02T17:10:49.561Z"},"th":{"title":"ทีม Google AI แบ่งปันวิธีการสร้างเกณฑ์การให้คะแนนที่เชื่อถือได้สำหรับการประเมิน LLM ในฐานะผู้ตัดสิน","summary":"ทีม Google AI เผยแพร่คู่มือ อธิบายวิธีการเขียนเกณฑ์การให้คะแนนแบบบูลีนที่น่าเชื่อถือสำหรับการประเมิน LLM-as-a-Judge โดยระบุว่าคำแนะนำที่คลุมเครืออาจทำให้การประเมินไม่สอดคล้องกันและเสีย token ในบทความได้ให้ประสบการณ์สี่ข้อ: ทำให้คำถามเป็นอะตอมและไม่ทับซ้อนกัน, ให้โมเดลประเมินเฉพาะข้อเท็จจริงเชิงวัตถุประสงค์ (สามารถใช้คำศัพท์ RFC 2119 เช่น MUST), ประเมินเฉพาะเนื้อหาที่ระบุอย่างชัดเจนใน prompt, ใช้ชุดข้อมูล golden set ที่ได้รับการทำเครื่องหมายโดยผู้เชี่ยวชาญปรับเทียบโมเดลประเมินจนสอดคล้องกับการให้คะแนนของมนุษย์","category":"行业动态","source":"Google AI：DEV 作者专属（RSS）","aggregationSource":"Google AI：DEV 作者专属（RSS）","pageTitle":"ทีม Google AI แบ่งปันวิธีการสร้างเกณฑ์การให้คะแนนที่เชื่อถือได้สำหรับการประเมิน LLM ในฐานะผู้ตัดสิน - ข่าว AI Aioga","description":"ทีม Google AI เผยแพร่คู่มือ อธิบายวิธีการเขียนเกณฑ์การให้คะแนนแบบบูลีนที่น่าเชื่อถือสำหรับการประเมิน LLM-as-a-Judge โดยระบุว่าคำแนะนำที่คลุมเครืออาจทำให้การประเมินไม่สอดคล้องกันและ...","url":"https://www.aioga.com/th/news/cmtkbz92801nmrowy61g2fsob/","contentTranslated":true,"sourceHash":"35ee1c62e4ae7343","translatedAt":"2026-09-02T17:11:03.675Z"},"pl":{"title":"Zespół Google AI dzieli się, jak tworzyć wiarygodne kryteria oceniania dla oceny LLM jako sędziego","summary":"Zespół Google AI opublikował samouczek, w którym wyjaśnia, jak tworzyć wiarygodne kryteria oceny w formie logicznej dla LLM jako sędziego, wskazując, że niejasne podpowiedzi prowadzą do niespójnych ocen i marnowania tokenów. W artykule podano cztery doświadczenia: utrzymywać pytania atomowymi i niepokrywającymi się, pozwalać modelowi oceniać tylko fakty obiektywne (można używać terminów z RFC 2119, takich jak MUST), oceniać tylko to, czego wyraźnie wymaga prompt, kalibrować model oceniający za pomocą zestawu referencyjnego oznaczonego przez ekspertów, aż będzie zgodny z oceną ludzką.","category":"行业动态","source":"Google AI：DEV 作者专属（RSS）","aggregationSource":"Google AI：DEV 作者专属（RSS）","pageTitle":"Zespół Google AI dzieli się, jak tworzyć wiarygodne kryteria oceniania dla oceny LLM jako sędziego - Aioga Wiadomości AI","description":"Zespół Google AI opublikował samouczek, w którym wyjaśnia, jak tworzyć wiarygodne kryteria oceny w formie logicznej dla LLM jako sędziego, wskazując, że niejasne podpowiedzi prowad...","url":"https://www.aioga.com/pl/news/cmtkbz92801nmrowy61g2fsob/","contentTranslated":true,"sourceHash":"35ee1c62e4ae7343","translatedAt":"2026-09-02T17:11:58.932Z"}},"evidenceTier":"verified-news","reviewStatus":"editorial-selected","indexable":true,"editorialCover":""}}