{"@context":"https://schema.org","@type":"NewsArticle","generatedAt":"2026-08-21T19:20:11.661Z","headline":"测量语音识别中的基准优化：Hugging Face 新测试揭示 ASR 模型\"刷分\"现象","description":"Hugging Face 最新研究引入三项测试量化语音识别中的基准优化（benchmaxxing）现象。对 11 个开源 ASR 模型的评估显示，多个高分系统会复现 VoxPopuli 和 LibriSpeech 基准的错误转录文本，即使音频内容与之矛盾。部分模型甚至依赖声学线索识别基准来源，导致其得分高估了真实转录能力。 🔗 阅读原文 via AIHOT · https://aihot.virxact.com/items/cmt30vskr0e0wro6t19q13yic","url":"https://www.aioga.com/news/cmt30vskr0e0wro6t19q13yic/","mainEntityOfPage":"https://www.aioga.com/news/cmt30vskr0e0wro6t19q13yic/","datePublished":"2026-08-21T00:00:00.000Z","dateModified":"2026-08-21T00:00:00.000Z","inLanguage":"zh-CN","publisher":{"@type":"NewsMediaOrganization","name":"Aioga","url":"https://www.aioga.com"},"citation":["https://huggingface.co/blog/asr-benchmark-optimization","https://aihot.virxact.com/items/cmt30vskr0e0wro6t19q13yic"],"canonicalUrl":"https://www.aioga.com/news/cmt30vskr0e0wro6t19q13yic/","directAnswer":{"@type":"Answer","text":"Hugging Face 研究评估了 11 个开源语音识别模型，发现部分高分系统会复现 VoxPopuli 和 LibriSpeech 的错误参考文本，分数未必代表真实转录能力。","url":"https://www.aioga.com/news/cmt30vskr0e0wro6t19q13yic/","dateCreated":"2026-08-21T00:00:00.000Z","author":{"@type":"Organization","@id":"https://www.aioga.com/authors/aioga-editorial/#editorial-team","name":"Aioga Editorial Team","url":"https://www.aioga.com/authors/aioga-editorial/"}},"evidence":[{"@type":"CreativeWork","name":"huggingface.co source article","url":"https://huggingface.co/blog/asr-benchmark-optimization","datePublished":"2026-08-21T00:00:00.000Z","provider":{"@type":"Organization","name":"huggingface.co","url":"https://huggingface.co/blog/asr-benchmark-optimization"}},{"@type":"CreativeWork","name":"AIHot archive record","url":"https://aihot.virxact.com/items/cmt30vskr0e0wro6t19q13yic","datePublished":"2026-08-21T00:00:00.000Z","provider":{"@type":"Organization","name":"AIHot","url":"https://aihot.virxact.com/items/cmt30vskr0e0wro6t19q13yic"}}],"aggregationSource":"Hugging Face：Blog（RSS","originalPublisher":{"name":"huggingface.co","url":"https://huggingface.co/blog/asr-benchmark-optimization"},"geoDeepAnswer":null,"article":{"id":"cmt30vskr0e0wro6t19q13yic","slug":"cmt30vskr0e0wro6t19q13yic","url":"https://www.aioga.com/news/cmt30vskr0e0wro6t19q13yic/","title":"测量语音识别中的基准优化：Hugging Face 新测试揭示 ASR 模型\"刷分\"现象","title_en":"","summary":"Hugging Face 最新研究引入三项测试量化语音识别中的基准优化（benchmaxxing）现象。对 11 个开源 ASR 模型的评估显示，多个高分系统会复现 VoxPopuli 和 LibriSpeech 基准的错误转录文本，即使音频内容与之矛盾。部分模型甚至依赖声学线索识别基准来源，导致其得分高估了真实转录能力。 🔗 阅读原文 via AIHOT · https://aihot.virxact.com/items/cmt30vskr0e0wro6t19q13yic","source":"Hugging Face：Blog（RSS","sourceUrl":"https://huggingface.co/blog/asr-benchmark-optimization","aiHotUrl":"https://aihot.virxact.com/items/cmt30vskr0e0wro6t19q13yic","publishedAt":"2026-08-21T00:00:00.000Z","category":"行业动态","score":72,"selected":true,"articleBody":["Reference disagreement (VoxPopuli case study) ：#reference-disagreement-voxpopuli-case-study Masked Entity Retrieval ：#masked-entity-retrieval Orthographic Switching ：#orthographic-switching Localizing the switches ：#localizing-the-switches Conclusion ：#conclusion Public voice AI benchmarks increasingly suggest that models are performing at human levels. Yet those scores don't always reflect how models work in the real-world. Since public benchmarks are open and widely used, models can also become optimized for the tests themselves. Their scores may improve because they have learned benchmark-specific patterns and not because they have become better at the underlying task.","One reason is that traditional benchmarks overlook many of the conditions and qualities that make voice systems reliable, natural, contextually appropriate, and effective in practice. That's why we recently introduced held-out sets in Real World VoiceEQ：https://huggingface.co/spaces/HumeAI/rw-voice-eq, the Open-ASR Leaderboard：https://huggingface.co/blog/open-asr-leaderboard-private-data, and the Far-field ASR Leaderboard：https://huggingface.co/spaces/treble-technologies/ffasr: to measure more of what matters in real-world use.","However, broader measurement alone does not solve the problem. This phenomenon, sometimes called benchmark optimization or \"benchmaxxing,\" is often discussed around machine learning, however, it has been difficult to measure in speech recognition.","Our latest research introduces three tests to help quantify it. We evaluated 11 widely used open-source ASR models and found that several of the highest-scoring systems reproduced benchmark transcripts from the VoxPopuli：https://huggingface.co/datasets/facebook/voxpopuli English and LibriSpeech：https://huggingface.co/datasets/openslr/librispeech_asr (clean, other) datasets – even when the audio contradicted them, relevant words had been silenced, or the audio equally supported two different written forms.","In some cases, models appeared to rely not only on what was said, but also on subtle acoustic cues that indicated which benchmark they were being tested on. As a result, their scores overstated how well they could transcribe speech more generally.","VoxPopuli is known to contain a high number of transcription errors (which is why Artificial Analysis released a cleaned version：https://huggingface.co/datasets/ArtificialAnalysis/VoxPopuli-Cleaned-AA). Our consensus disagreement probe tests what happens when leading ASR models encounter these errors: Do they accurately transcribe what the audio says, or reproduce the benchmark's incorrect reference transcript?","To test this at scale, we use an ensemble of independent models selected for their low phoneme error rate (PER). PER measures how closely a written transcription matches the sounds in the audio, making it a useful proxy for how faithfully a model transcribes what it hears. The ensemble results can be used to flag cases in which the models unanimously disagree with the benchmark's reference transcript. We then compare a sample of those flagged cases against human annotations to validate the corrected transcripts.","For example, one VoxPopuli clip audibly includes the phrase \"Thank you, Mr. President,\" but the reference transcript omits \"Thank you.\" Six of the 11 models we tested reproduced the benchmark's erroneous transcript—giving the \"expected\" answer even though it contradicted the audio. On the real clip, the formatting follows the same pattern: models that omit \"Thank you\" also reproduce the benchmark's punctuation style, writing \"Mr\" without a period, while models that include the audible phrase tend to write \"Mr.\" with the period.","When we present the same content in newly collected voices from EU parliamentary recordings or generic voices, this behavior often weakens or disappears. In the below samples, all but one model flips back to transcribing the audio-faithful transcript for a clone of a new parliamentary recording. This suggests that the models are responding to acoustic cues that help them identify the benchmark membership and thus produce the expected transcript even if it contradicts the audio.","The reference transcript for this clip reads \"Mr President, I have another complaint about this procedure, which is that it is not secret.\" The audio in all three clips below actually says the same thing, preceded by an audible \"Thank you,\"—the clones are text-to-speech renditions of that true sentence, so the courtesy is audible in all three. Green highlighting and ✅ mark a transcript that includes the audible \"Thank you\"; red highlighting and ❌ mark a transcript that reproduces the benchmark's erroneous omission. All transcripts are raw model output, prior to any normalization—casing and punctuation are preserved exactly as generated, including lowercase output from some models.","Clone of a parliament speaker recorded after every model's training cutoff","Parakeet is the only model that flips between reproducing the benchmark on the real clip and getting it right on the same-speaker clone. Phi-4 is the only model still dropping the courtesy on the ep-fresh clone. When we instead resynthesize the sentence in a generic TTS voice unconnected to any parliamentary recording, all eleven models restore the courtesy.","The results suggest that this problem is both widespread and meaningful. Our methodology flagged potential reference errors in 40% of the VoxPopuli test clips we analyzed, affecting roughly 3% of all reference words.","Models exhibiting benchmark-optimized behavior reproduced erroneous reference transcripts 18–30% of the time. The scatterplot below compares VoxPopuli word error rate (WER) on the x-axis with the rate at which each model reproduces the benchmark's incorrect reference instead of the consensus correction. The models with the lowest WER—and therefore the strongest reported benchmark performance—are also the most likely to reproduce these errors.","To build on the consensus disagreement probe, we deliberately silence numbers in the audio samples of test datasets and ask the models to transcribe what it hears. The number is literally absent from the audio, so models should not output any number, much less the exact number in the text.","Some of these numbers are semi-predictable (although still unlikely for a model to predict), yet others are quite surprising. The following clip combines both probes, showing both how models recreate reference transcript errors including an incorrect number and one model even autocompletes a relatively random year (2011) despite it being silenced. In each model's row below:","Recovery rates were highest on the public benchmarks and lower on held-out or newly collected audio (ep-fresh and libri-fresh below). On LibriSpeech, some of the strongest benchmark-performing models reproduced masked numbers in roughly 30–40% of examples, even though the number itself had been removed. The effect weakened on freshly collected data for several models, suggesting that the surrounding benchmark-associated audio—not only textual autocomplete—helped the models recover the reference.","Our orthographic switching probe tests whether models reproduce the exact spelling used in a benchmark's reference transcript despite it not being clear in the audio. Orthographic variants are words that are semantically and phonetically identical but can be spelled different ways (1 vs one, Mr. vs mister, John vs Jon, Honor vs Honour, etc). In theory, models should consistently prefer one spelling over another, or alternate between them at roughly random rates. If models systematically switch to match what is in each benchmark's reference transcript, that suggests the models are picking up on which spelling the test expects.","Transcription: \"I URGED ON THE BOYS THAT WHATEVER HAPPENED WE SHOULD NOT SHOOT ANY ONE\" — models using \"any one\": 6/11, models using \"anyone\": 5/11","Transcription: \"CAMOUFLAGE WAS NOT A WORD THE CAPTAIN OR ANYONE ELSE OF HIS TIME YET UNDERSTOOD\" — models using \"any one\": 2/11, models using \"anyone\": 9/11","Within LibriSpeech, we test one intra-dataset switch involving an older spacing convention: some reference transcripts use \"any one\", while others use \"anyone.\" We measure the minimum accuracy for a given variant, which we call \"switch rate\". If a model only uses one variant it would have a 0% switch rate; a model which picks randomly would be expected to have a 50% switch rate. A model which knows which variant to use in every test sample would earn a 100% switch rate.","Our second probe tests an inter-dataset switch , in which each benchmark uses a different spelling convention consistently across its test corpus. For example, VoxPopuli uses the abbreviation \"Mr.,\" while LibriSpeech spells out \"Mister.\"","Multiple models exceed the 50% random-choice baseline, with some reaching roughly 90% switch accuracy. This suggests that the models can identify which dataset an audio sample comes from and select the spelling convention that benchmark expects, even though both forms sound identical.","To test whether these behaviors generalize beyond the public benchmarks, we also collected fresh data from the same source domains but after the models' training cutoffs: recent European Parliament recordings for VoxPopuli and recordings from newly active LibriVox narrators for LibriSpeech. However, when presented with recently collected data from the same domain, many models stop matching the reference transcript and revert to more audio faithful transcriptions.","Other interventions point to the same conclusion. Phrases which are present in the audio but are omitted in the reference transcript can reappear when a model is asked to translate the audio or when its attention is restricted to the relevant frames. Trimming away surrounding benchmark context, or appending ordinary conversational audio, can also restore the faithful transcript. Appending VoxPopuli audio can have the opposite effect, making otherwise faithful synthetic or mined samples more likely to match the benchmark reference.","Together, these results suggest that models are able to faithfully transcribe the literal spoken words, but are using surrounding acoustic context to decide whether to follow the audio or a benchmark-specific transcription policy.","Our findings suggest that, on two major open-source datasets, some models detect dataset-associated acoustic cues and adjust their transcription behavior accordingly. Specifically, models may reproduce words that are absent from the audio but present in the reference transcript, recover silenced numbers at elevated rates, or use surrounding acoustic context to select the written variant expected by a particular benchmark.","For people selecting models, these findings underscore the importance of using fully held-out evaluation sets, as RW-Voice-EQ Bench and the Open ASR Leaderboard do, and of looking beyond word error rate on a single public benchmark. To this end, a \"Benchmark fitting\" tab has been added to the Open ASR Leaderboard：https://huggingface.co/spaces/hf-audio/open_asr_leaderboard, which includes two of the above analyses across all models: quantifying (1) reference error rates from VoxPopuli and (2) orthographic switching across all public datasets. The relevant scripts are open-sourced on GitHub：https://github.com/huggingface/open_asr_leaderboard/tree/main/benchmark_fitting as well as the un-normalized model outputs：https://huggingface.co/buckets/hf-audio/asr_leaderboard_h200.","Our findings also suggest that benchmark developers should avoid simple independent and identically distributed test splits in favor of temporal, speaker, or other metadata-based separation. Greater transparency around training data and model-selection procedures would also help researchers understand how these behaviors arise.","Public benchmarks remain valuable: they are transparent, repeatable, easy to run, and well understood by the research community. But they are most useful when we can distinguish genuine transcription improvements from benchmark-specific gains that do not generalize to new audio.","For more information, we encourage you to read our full report：https://huggingface.co/papers/2608.19936.","Explore and compare speech model leaderboards with audio samples","Explore speech recognition model benchmarks across datasets","Far-Field ASR — clean / noisy / reverberant benchmark"],"articleImages":[{"sourceUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6384db7fb2906edaf835a91d/MOTXxaOmjlTZ8wONYifnD.jpeg","alt":"","afterParagraph":0,"url":"/media/articles/cmt30vskr0e0wro6t19q13yic/fd4789dda847dae4.webp"},{"sourceUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/r6F20gfbuflBGbpmMZ7e7.png","alt":"","afterParagraph":0,"url":"/media/articles/cmt30vskr0e0wro6t19q13yic/37ca03096b90ac52.webp"}],"mediaStatus":"ok","articleBodyZh":["参考分歧（VoxPopuli 案例研究）：#reference-disagreement-voxpopuli-case-study 掩码实体检索：#masked-entity-retrieval 文字形式切换：#orthographic-switching 定位切换：#localizing-the-switches 结论：#conclusion 公共语音 AI 基准测试越来越多地表明，模型的表现达到了人类水平。然而，这些分数并不总能反映模型在现实世界中的工作情况。由于公共基准是开放的并被广泛使用，模型也可能针对测试本身进行优化。它们的分数可能会提高，因为它们学会了基准特定的模式，而不是因为它们在基础任务上变得更好。","其中一个原因是，传统基准忽略了许多使语音系统在实践中可靠、自然、符合上下文并有效的条件和特性。这就是为什么我们最近在 Real World VoiceEQ：https://huggingface.co/spaces/HumeAI/rw-voice-eq、Open-ASR 排行榜：https://huggingface.co/blog/open-asr-leaderboard-private-data 和远场 ASR 排行榜：https://huggingface.co/spaces/treble-technologies/ffasr 中引入了保留集：以衡量在现实世界使用中更重要的方面。","然而，仅仅更广泛的测量并不能解决问题。这个现象，有时称为基准优化或“benchmaxxing”，在机器学习中经常被讨论，但在语音识别中很难测量。","我们最新的研究引入了三项测试来帮助量化这个问题。我们评估了 11 个广泛使用的开源 ASR 模型，发现几种得分最高的系统在 VoxPopuli：https://huggingface.co/datasets/facebook/voxpopuli 英文和 LibriSpeech：https://huggingface.co/datasets/openslr/librispeech_asr （clean、other）数据集上再现了基准转录文本——即使音频与之相矛盾、相关词被静音，或者音频同样支持两种不同的书面形式。","在某些情况下，模型似乎不仅依赖于所说内容，还依赖于微妙的声学提示，这些提示表明它们正在测试的基准。因此，它们的分数夸大了它们一般转录语音的能力。","VoxPopuli 众所周知包含大量转录错误（这也是 Artificial Analysis 发布了清理版本的原因：https://huggingface.co/datasets/ArtificialAnalysis/VoxPopuli-Cleaned-AA）。我们的共识分歧探测测试用于检验在顶级 ASR 模型遇到这些错误时会发生什么：它们是准确转录音频所说内容，还是重现基准数据中错误的参考转录文本？","为了在大规模上测试这一点，我们使用了一个独立模型的集成，这些模型是根据低音素错误率（PER）选出的。PER 测量书面转录与音频中声音的匹配程度，是评估模型如何忠实转录所听内容的有用指标。集成结果可用于标记那些模型一致不同意基准参考转录的情况。然后我们将这些标记案例的样本与人工标注进行对比，以验证已纠正的转录文本。","例如，一个 VoxPopuli 剪辑中可听到短语“Thank you, Mr. President”，但参考转录省略了“Thank you”。我们测试的 11 个模型中有 6 个重现了基准的错误转录——给出了“预期”的答案，即使这与音频内容相矛盾。在实际剪辑中，格式也遵循相同模式：省略“Thank you”的模型也沿用了基准的标点风格，将“Mr”写作无句号，而包含该可听短语的模型倾向于写作带句号的“Mr.”。","当我们在新收集的欧盟议会录音或通用声音中呈现相同内容时，这种行为往往减弱或消失。在下面的样本中，除了一个模型之外，其余模型都回到了为新议会录音克隆忠实转录音频的方式。这表明，模型正在响应帮助它们识别基准所属信息的声学线索，从而即使与音频内容相矛盾也生成预期的转录文本。","该片段的参考文本记录为“总统先生，我对这一程序有另一个抱怨，那就是它不保密。”下面三个片段中的音频实际上说的都是同样的话，并在前面有一个可听到的“谢谢”，——这些克隆是该真实句子的文本转语音版本，所以在三个片段中都可以听到礼貌用语。以绿色高亮和✅标记的文本包含可听到的“谢谢”；以红色高亮和❌标记的文本则再现了基准文本中错误的遗漏。所有文本均为模型原始输出，尚未进行任何标准化——大小写和标点完全按生成时的原样保留，包括某些模型输出的小写字母。","每个模型训练截止后录制的议会发言人克隆","鹦鹉模型是唯一在真实片段上复现基准文本与在同一发言者克隆上正确复现之间切换的模型。Phi-4 是唯一在 ep-fresh 克隆上仍然省略礼貌用语的模型。当我们改用与任何议会录音无关的通用 TTS 语音重新合成该句时，所有十一种模型都恢复了礼貌用语。","结果表明，这一问题既广泛又具有重要意义。我们的方法在分析的 VoxPopuli 测试片段中，发现了 40% 存在潜在参考错误，影响了大约 3% 的所有参考词汇。","显示基准优化行为的模型有 18–30% 的时间复现了错误的参考文本。下方散点图将 x 轴上的 VoxPopuli 单词错误率（WER）与每个模型复现基准错误参考而非共识修正的频率进行比较。WER 最低、因此报告的基准表现最强的模型，也是最可能复现这些错误的模型。","为了进一步探查共识分歧，我们故意在测试数据集的音频样本中将数字静音，并要求模型转录所听内容。音频中确实没有数字，因此模型不应输出任何数字，更不能输出文本中的确切数字。","这些数字中有些是半可预测的（尽管模型仍然不太可能预测到），而其他一些则相当令人惊讶。以下片段结合了两种探测，展示了模型如何重现参考转录错误，包括一个错误的数字，甚至有一个模型在数字被静音的情况下自动补全了一个相对随机的年份（2011）。在下面每个模型的行中：","在公共基准测试上恢复率最高，而在保留或新收集的音频（下文的 ep-fresh 和 libri-fresh）上较低。在 LibriSpeech 上，一些基准表现最佳的模型在大约 30–40% 的示例中再现了被掩盖的数字，即使该数字本身已被删除。在若干模型的新收集数据上的效果减弱，这表明不仅仅是文本自动补全，而是与基准相关的周围音频帮助模型恢复参考内容。","我们的正字法切换探测测试模型是否会重现基准参考转录中使用的确切拼写，即使音频中不清楚。正字法变体是指语义和发音相同但拼写不同的词（1 与 one，Mr. 与 mister，John 与 Jon，Honor 与 Honour 等）。理论上，模型应始终偏好一种拼写，或以大致随机的比例交替使用。如果模型系统性地切换以匹配每个基准的参考转录，这表明模型正在学习测试所期望的拼写。","抄写：\"我督促男孩们，无论发生什么，我们都不应该开枪射击任何人\"——使用“any one”的模型：6/11，使用“anyone”的模型：5/11","抄录：“伪装”不是船长或他那个时代的任何人所理解的词 — 使用“any one”的模型：2/11，使用“anyone”的模型：9/11","在 LibriSpeech 中，我们测试一个涉及旧间距约定的同数据集内切换：一些参考转录使用 \"any one\"，而其他使用 \"anyone\"。我们测量某个变体的最小准确率，称为“切换率”。如果模型只使用一种变体，则切换率为 0%；随机选择的模型预期切换率为 50%。如果模型在每个测试样本中都知道要使用哪个变体，则切换率为 100%。","我们的第二次探测测试了跨数据集切换，其中每个基准在其测试语料库中始终使用不同的拼写规范。例如，VoxPopuli 使用缩写“Mr.”，而 LibriSpeech 则拼写为“Mister”。","多种模型的表现都超过了 50% 的随机选择基线，其中一些模型的切换准确率约为 90%。这表明模型可以识别音频样本来自哪个数据集，并选择基准所期望的拼写规范，尽管两种形式在发音上完全相同。","为了测试这些行为是否会超越公开基准，我们还从相同来源域收集了模型训练截止日期之后的新数据：最近的欧洲议会录音用于 VoxPopuli，以及新活跃的 LibriVox 朗读者录音用于 LibriSpeech。然而，当呈现来自相同领域的最近收集的数据时，许多模型会停止匹配参考转录，并恢复更忠实于音频的转录。","其他干预措施也指向相同的结论。存在于音频中但在参考转录中被省略的短语，当模型被要求翻译音频或其注意力被限制在相关帧时，可能会重新出现。修剪周围的基准上下文，或附加普通对话音频，也可以恢复忠实的转录。附加 VoxPopuli 音频可能产生相反效果，使原本忠实的合成或挖掘样本更可能匹配基准参考。","综上，这些结果表明模型能够忠实地转录字面上的口语，但会利用周围的声学上下文来决定是遵循音频还是特定基准的转录策略。","我们的研究结果表明，在两个主要开源数据集上，一些模型能够检测与数据集相关的声学信号，并相应地调整其转录行为。具体来说，模型可能会复现音频中不存在但参考转录中出现的词语，提高恢复被静音的数字的比率，或利用周围的声学上下文选择某一特定基准所期望的书写形式。","对于选择模型的人来说，这些发现强调了使用完全独立的评估集的重要性，就像 RW-Voice-EQ Bench 和 Open ASR Leaderboard 那样，并且需要超越单一公开基准的词错误率。因此，Open ASR Leaderboard 中新增了一个“基准拟合”标签：https://huggingface.co/spaces/hf-audio/open_asr_leaderboard，其中包括对所有模型的两项上述分析：量化 (1) VoxPopuli 的参考错误率，以及 (2) 所有公开数据集的正字法切换。相关脚本已在 GitHub 上开源：https://github.com/huggingface/open_asr_leaderboard/tree/main/benchmark_fitting，以及未归一化的模型输出：https://huggingface.co/buckets/hf-audio/asr_leaderboard_h200。","我们的研究结果也表明，基准开发者应避免简单的独立同分布测试集划分，而应选择基于时间、说话人或其他元数据的分离。对训练数据和模型选择过程的更大透明度也将有助于研究人员理解这些行为是如何产生的。","公开基准仍然有价值：它们透明、可重复、易于运行，并且研究社区对其有很好的理解。但它们最有用的情况是，当我们能够区分真正的转录改进和仅在基准上表现良好的无法推广到新音频的成果时。","更多信息，请查阅我们的完整报告：https://huggingface.co/papers/2608.19936。","探索并比较带有音频样本的语音模型排行榜","跨数据集探索语音识别模型基准","远场语音识别 — 干净 / 嘈杂 / 混响 基准"],"translationStatus":"translated","bodyOrigin":"source-page","editorial":{"summary":"Hugging Face 研究评估了 11 个开源语音识别模型，发现部分高分系统会复现 VoxPopuli 和 LibriSpeech 的错误参考文本，分数未必代表真实转录能力。","background":"研究提出三项测试，分别考察模型面对参考文本分歧、被遮蔽实体和正字法差异时的表现。材料还提到，部分模型可能利用暗示基准来源的声学线索完成识别。","viewpoint":"Aioga 判断，公开基准的高分应与音频忠实度和真实场景表现结合解读。若模型重复错误答案或依赖测试线索，单一榜单排名可能被高估。","implications":"这一发现可能推动语音识别评测更多采用隐藏测试集、人工校验和面向真实环境的指标。模型开发者也需要关注训练或评测数据中的错误转录与特定模式。","nextStep":"值得关注后续研究是否公开三项测试的完整结果、各模型差异及人工校验范围，并观察 Open-ASR、远场 ASR 等评测是否持续采用独立或隐藏数据。","evidenceRefs":["title","summary","articleBody","source"],"status":"published","aiGenerated":true,"autoApproved":true,"generatedBy":"aioga-editorial:gpt-5.6-sol","reviewedBy":"aioga-editorial-review:gpt-5.6-sol","generatedAt":"2026-08-21T15:03:57.293Z","sourceHash":"72e5fc7f1dd5e743","review":{"approved":true,"groundedness":93,"clarity":91,"duplicationRisk":18,"blockingIssues":[],"notes":["“被遮蔽实体”是对 Masked Entity Retrieval 的概括，建议在正式发布时沿用原文测试名称或补充定义，以避免歧义。","“Aioga 判断”属于明确标注的观点，不构成事实冒充。","“可能推动”“值得关注”等表述属于合理推论和后续观察，不应视为来源已证实的事实。"]},"validation":{"passed":true,"mode":"ai-auto","revisions":0,"checks":["schema","length","source-attribution","low-source-overlap","no-html","independent-ai-review"]}},"tags":["行业动态","Hugging Face：Blog（RSS）"],"translations":{"zh-CN":{"title":"测量语音识别中的基准优化：Hugging Face 新测试揭示 ASR 模型\"刷分\"现象","summary":"Hugging Face 最新研究引入三项测试量化语音识别中的基准优化（benchmaxxing）现象。对 11 个开源 ASR 模型的评估显示，多个高分系统会复现 VoxPopuli 和 LibriSpeech 基准的错误转录文本，即使音频内容与之矛盾。部分模型甚至依赖声学线索识别基准来源，导致其得分高估了真实转录能力。 🔗 阅读原文 via AIHOT · https://aihot.virxact.com/items/cmt30vskr0e0wro6t19q13yic","category":"行业动态","source":"huggingface.co","aggregationSource":"Hugging Face：Blog（RSS","pageTitle":"测量语音识别中的基准优化：Hugging Face 新测试揭示 ASR 模型\"刷分\"现象 - Aioga AI资讯","description":"Hugging Face 最新研究引入三项测试量化语音识别中的基准优化（benchmaxxing）现象。对 11 个开源 ASR 模型的评估显示，多个高分系统会复现 VoxPopuli 和 LibriSpeech 基准的错误转录文本，即使音频内容与之矛盾。部分模型甚至依赖声学线索识别基准来源，导致其得分高估了真实转录能力。 🔗 阅读原文 via AIHOT...","url":"https://www.aioga.com/news/cmt30vskr0e0wro6t19q13yic/","articleBody":["参考分歧（VoxPopuli 案例研究）：#reference-disagreement-voxpopuli-case-study 掩码实体检索：#masked-entity-retrieval 文字形式切换：#orthographic-switching 定位切换：#localizing-the-switches 结论：#conclusion 公共语音 AI 基准测试越来越多地表明，模型的表现达到了人类水平。然而，这些分数并不总能反映模型在现实世界中的工作情况。由于公共基准是开放的并被广泛使用，模型也可能针对测试本身进行优化。它们的分数可能会提高，因为它们学会了基准特定的模式，而不是因为它们在基础任务上变得更好。","其中一个原因是，传统基准忽略了许多使语音系统在实践中可靠、自然、符合上下文并有效的条件和特性。这就是为什么我们最近在 Real World VoiceEQ：https://huggingface.co/spaces/HumeAI/rw-voice-eq、Open-ASR 排行榜：https://huggingface.co/blog/open-asr-leaderboard-private-data 和远场 ASR 排行榜：https://huggingface.co/spaces/treble-technologies/ffasr 中引入了保留集：以衡量在现实世界使用中更重要的方面。","然而，仅仅更广泛的测量并不能解决问题。这个现象，有时称为基准优化或“benchmaxxing”，在机器学习中经常被讨论，但在语音识别中很难测量。","我们最新的研究引入了三项测试来帮助量化这个问题。我们评估了 11 个广泛使用的开源 ASR 模型，发现几种得分最高的系统在 VoxPopuli：https://huggingface.co/datasets/facebook/voxpopuli 英文和 LibriSpeech：https://huggingface.co/datasets/openslr/librispeech_asr （clean、other）数据集上再现了基准转录文本——即使音频与之相矛盾、相关词被静音，或者音频同样支持两种不同的书面形式。","在某些情况下，模型似乎不仅依赖于所说内容，还依赖于微妙的声学提示，这些提示表明它们正在测试的基准。因此，它们的分数夸大了它们一般转录语音的能力。","VoxPopuli 众所周知包含大量转录错误（这也是 Artificial Analysis 发布了清理版本的原因：https://huggingface.co/datasets/ArtificialAnalysis/VoxPopuli-Cleaned-AA）。我们的共识分歧探测测试用于检验在顶级 ASR 模型遇到这些错误时会发生什么：它们是准确转录音频所说内容，还是重现基准数据中错误的参考转录文本？","为了在大规模上测试这一点，我们使用了一个独立模型的集成，这些模型是根据低音素错误率（PER）选出的。PER 测量书面转录与音频中声音的匹配程度，是评估模型如何忠实转录所听内容的有用指标。集成结果可用于标记那些模型一致不同意基准参考转录的情况。然后我们将这些标记案例的样本与人工标注进行对比，以验证已纠正的转录文本。","例如，一个 VoxPopuli 剪辑中可听到短语“Thank you, Mr. President”，但参考转录省略了“Thank you”。我们测试的 11 个模型中有 6 个重现了基准的错误转录——给出了“预期”的答案，即使这与音频内容相矛盾。在实际剪辑中，格式也遵循相同模式：省略“Thank you”的模型也沿用了基准的标点风格，将“Mr”写作无句号，而包含该可听短语的模型倾向于写作带句号的“Mr.”。","当我们在新收集的欧盟议会录音或通用声音中呈现相同内容时，这种行为往往减弱或消失。在下面的样本中，除了一个模型之外，其余模型都回到了为新议会录音克隆忠实转录音频的方式。这表明，模型正在响应帮助它们识别基准所属信息的声学线索，从而即使与音频内容相矛盾也生成预期的转录文本。","该片段的参考文本记录为“总统先生，我对这一程序有另一个抱怨，那就是它不保密。”下面三个片段中的音频实际上说的都是同样的话，并在前面有一个可听到的“谢谢”，——这些克隆是该真实句子的文本转语音版本，所以在三个片段中都可以听到礼貌用语。以绿色高亮和✅标记的文本包含可听到的“谢谢”；以红色高亮和❌标记的文本则再现了基准文本中错误的遗漏。所有文本均为模型原始输出，尚未进行任何标准化——大小写和标点完全按生成时的原样保留，包括某些模型输出的小写字母。","每个模型训练截止后录制的议会发言人克隆","鹦鹉模型是唯一在真实片段上复现基准文本与在同一发言者克隆上正确复现之间切换的模型。Phi-4 是唯一在 ep-fresh 克隆上仍然省略礼貌用语的模型。当我们改用与任何议会录音无关的通用 TTS 语音重新合成该句时，所有十一种模型都恢复了礼貌用语。","结果表明，这一问题既广泛又具有重要意义。我们的方法在分析的 VoxPopuli 测试片段中，发现了 40% 存在潜在参考错误，影响了大约 3% 的所有参考词汇。","显示基准优化行为的模型有 18–30% 的时间复现了错误的参考文本。下方散点图将 x 轴上的 VoxPopuli 单词错误率（WER）与每个模型复现基准错误参考而非共识修正的频率进行比较。WER 最低、因此报告的基准表现最强的模型，也是最可能复现这些错误的模型。","为了进一步探查共识分歧，我们故意在测试数据集的音频样本中将数字静音，并要求模型转录所听内容。音频中确实没有数字，因此模型不应输出任何数字，更不能输出文本中的确切数字。","这些数字中有些是半可预测的（尽管模型仍然不太可能预测到），而其他一些则相当令人惊讶。以下片段结合了两种探测，展示了模型如何重现参考转录错误，包括一个错误的数字，甚至有一个模型在数字被静音的情况下自动补全了一个相对随机的年份（2011）。在下面每个模型的行中：","在公共基准测试上恢复率最高，而在保留或新收集的音频（下文的 ep-fresh 和 libri-fresh）上较低。在 LibriSpeech 上，一些基准表现最佳的模型在大约 30–40% 的示例中再现了被掩盖的数字，即使该数字本身已被删除。在若干模型的新收集数据上的效果减弱，这表明不仅仅是文本自动补全，而是与基准相关的周围音频帮助模型恢复参考内容。","我们的正字法切换探测测试模型是否会重现基准参考转录中使用的确切拼写，即使音频中不清楚。正字法变体是指语义和发音相同但拼写不同的词（1 与 one，Mr. 与 mister，John 与 Jon，Honor 与 Honour 等）。理论上，模型应始终偏好一种拼写，或以大致随机的比例交替使用。如果模型系统性地切换以匹配每个基准的参考转录，这表明模型正在学习测试所期望的拼写。","抄写：\"我督促男孩们，无论发生什么，我们都不应该开枪射击任何人\"——使用“any one”的模型：6/11，使用“anyone”的模型：5/11","抄录：“伪装”不是船长或他那个时代的任何人所理解的词 — 使用“any one”的模型：2/11，使用“anyone”的模型：9/11","在 LibriSpeech 中，我们测试一个涉及旧间距约定的同数据集内切换：一些参考转录使用 \"any one\"，而其他使用 \"anyone\"。我们测量某个变体的最小准确率，称为“切换率”。如果模型只使用一种变体，则切换率为 0%；随机选择的模型预期切换率为 50%。如果模型在每个测试样本中都知道要使用哪个变体，则切换率为 100%。","我们的第二次探测测试了跨数据集切换，其中每个基准在其测试语料库中始终使用不同的拼写规范。例如，VoxPopuli 使用缩写“Mr.”，而 LibriSpeech 则拼写为“Mister”。","多种模型的表现都超过了 50% 的随机选择基线，其中一些模型的切换准确率约为 90%。这表明模型可以识别音频样本来自哪个数据集，并选择基准所期望的拼写规范，尽管两种形式在发音上完全相同。","为了测试这些行为是否会超越公开基准，我们还从相同来源域收集了模型训练截止日期之后的新数据：最近的欧洲议会录音用于 VoxPopuli，以及新活跃的 LibriVox 朗读者录音用于 LibriSpeech。然而，当呈现来自相同领域的最近收集的数据时，许多模型会停止匹配参考转录，并恢复更忠实于音频的转录。","其他干预措施也指向相同的结论。存在于音频中但在参考转录中被省略的短语，当模型被要求翻译音频或其注意力被限制在相关帧时，可能会重新出现。修剪周围的基准上下文，或附加普通对话音频，也可以恢复忠实的转录。附加 VoxPopuli 音频可能产生相反效果，使原本忠实的合成或挖掘样本更可能匹配基准参考。","综上，这些结果表明模型能够忠实地转录字面上的口语，但会利用周围的声学上下文来决定是遵循音频还是特定基准的转录策略。","我们的研究结果表明，在两个主要开源数据集上，一些模型能够检测与数据集相关的声学信号，并相应地调整其转录行为。具体来说，模型可能会复现音频中不存在但参考转录中出现的词语，提高恢复被静音的数字的比率，或利用周围的声学上下文选择某一特定基准所期望的书写形式。","对于选择模型的人来说，这些发现强调了使用完全独立的评估集的重要性，就像 RW-Voice-EQ Bench 和 Open ASR Leaderboard 那样，并且需要超越单一公开基准的词错误率。因此，Open ASR Leaderboard 中新增了一个“基准拟合”标签：https://huggingface.co/spaces/hf-audio/open_asr_leaderboard，其中包括对所有模型的两项上述分析：量化 (1) VoxPopuli 的参考错误率，以及 (2) 所有公开数据集的正字法切换。相关脚本已在 GitHub 上开源：https://github.com/huggingface/open_asr_leaderboard/tree/main/benchmark_fitting，以及未归一化的模型输出：https://huggingface.co/buckets/hf-audio/asr_leaderboard_h200。","我们的研究结果也表明，基准开发者应避免简单的独立同分布测试集划分，而应选择基于时间、说话人或其他元数据的分离。对训练数据和模型选择过程的更大透明度也将有助于研究人员理解这些行为是如何产生的。","公开基准仍然有价值：它们透明、可重复、易于运行，并且研究社区对其有很好的理解。但它们最有用的情况是，当我们能够区分真正的转录改进和仅在基准上表现良好的无法推广到新音频的成果时。","更多信息，请查阅我们的完整报告：https://huggingface.co/papers/2608.19936。","探索并比较带有音频样本的语音模型排行榜","跨数据集探索语音识别模型基准","远场语音识别 — 干净 / 嘈杂 / 混响 基准"]},"en":{"title":"Measuring benchmark optimization in speech recognition: Hugging Face's new test reveals the ASR model \"score brushing\" phenomenon","summary":"Hugging Face's latest research introduces three tests to quantify the phenomenon of benchmaxing in speech recognition. An evaluation of 11 open-source ASR models showed that multiple high-scoring systems reproduce incorrect transcription texts from VoxPopuli and LibriSpeech benchmarks, even if the audio content contradicts them. Some models even rely on acoustic cue identification as a benchmark source, resulting in scores that overestimate true transcription capability. 🔗 Read the original article via AIHOT · https://aihot.virxact.com/items/cmt30vskr0e0wro6t19q13yic","category":"Industry","source":"Hugging Face：Blog（RSS","aggregationSource":"Hugging Face：Blog（RSS","pageTitle":"Measuring benchmark optimization in speech recognition: Hugging Face's new test reveals the ASR model \"score brushing\" phenomenon - Aioga AI News","description":"Hugging Face's latest research introduces three tests to quantify the phenomenon of benchmaxing in speech recognition. An evaluation of 11 open-source ASR models showed that multip...","url":"https://www.aioga.com/en/news/cmt30vskr0e0wro6t19q13yic/","contentTranslated":true,"sourceHash":"189656fc6a4d3191","translatedAt":"2026-08-21T14:42:07.527Z"},"ja":{"title":"音声認識におけるベンチマーク最適化の測定:Hugging Faceの新テストでASRモデルの「スコアブラシング」現象が明らかになる","summary":"Hugging Faceの最新研究では、音声認識におけるベンチマックス現象を定量化するための3つのテストを導入しています。 11のオープンソースASRモデルの評価では、複数の高得点システムがVoxPopuliおよびLibriSpeechベンチマークから誤った文字起こしテキストを再現していることが示されました。たとえ音声内容が矛盾していてもです。 一部のモデルは音響手がかりの識別を基準源として利用しており、その結果、真の転写能力を過大評価するスコアが出ることもあります。 🔗 原文記事はAIHOTより読むことができます。 https://aihot.virxact.com/items/cmt30vskr0e0wro6t19q13yic","category":"業界動向","source":"Hugging Face：Blog（RSS","aggregationSource":"Hugging Face：Blog（RSS","pageTitle":"音声認識におけるベンチマーク最適化の測定:Hugging Faceの新テストでASRモデルの「スコアブラシング」現象が明らかになる - Aioga AIニュース","description":"Hugging Faceの最新研究では、音声認識におけるベンチマックス現象を定量化するための3つのテストを導入しています。 11のオープンソースASRモデルの評価では、複数の高得点システムがVoxPopuliおよびLibriSpeechベンチマークから誤った文字起こしテキストを再現していることが示されました。たとえ音声内容が矛盾していてもです。 一部のモデル...","url":"https://www.aioga.com/ja/news/cmt30vskr0e0wro6t19q13yic/","contentTranslated":true,"sourceHash":"189656fc6a4d3191","translatedAt":"2026-08-21T14:42:07.704Z"},"ko":{"title":"음성 인식에서 벤치마크 최적화 측정: Hugging Face의 새로운 테스트가 ASR 모델의 '점수 브러시링' 현상을 밝혀냈습니다","summary":"Hugging Face의 최신 연구는 음성 인식에서 벤치맥싱 현상을 정량화하기 위한 세 가지 테스트를 소개합니다. 11개의 오픈 소스 ASR 모델을 평가한 결과, 여러 고점수 시스템이 VoxPopuli 및 LibriSpeech 벤치마크에서 오디오 내용이 모순되더라도 잘못된 전사 텍스트를 재현하는 것으로 나타났습니다. 일부 모델은 음향 단서 식별을 기준 소스로 삼아 실제 전사 능력을 과대평가하는 점수를 내기도 합니다. 🔗 원문 기사는 AIHOT를 통해 읽을 수 있습니다. https://aihot.virxact.com/items/cmt30vskr0e0wro6t19q13yic","category":"업계 동향","source":"Hugging Face：Blog（RSS","aggregationSource":"Hugging Face：Blog（RSS","pageTitle":"음성 인식에서 벤치마크 최적화 측정: Hugging Face의 새로운 테스트가 ASR 모델의 '점수 브러시링' 현상을 밝혀냈습니다 - Aioga AI 뉴스","description":"Hugging Face의 최신 연구는 음성 인식에서 벤치맥싱 현상을 정량화하기 위한 세 가지 테스트를 소개합니다. 11개의 오픈 소스 ASR 모델을 평가한 결과, 여러 고점수 시스템이 VoxPopuli 및 LibriSpeech 벤치마크에서 오디오 내용이 모순되더라도 잘못된 전사 텍스트를 재현하는 것으로 나타났습니다. 일부...","url":"https://www.aioga.com/ko/news/cmt30vskr0e0wro6t19q13yic/","contentTranslated":true,"sourceHash":"189656fc6a4d3191","translatedAt":"2026-08-21T14:42:16.445Z"},"es":{"title":"Medición de la optimización de benchmarks en reconocimiento de voz: La nueva prueba de Hugging Face revela el fenómeno del modelo ASR llamado \"cepillado de puntuaciones\"","summary":"La última investigación de Hugging Face introduce tres pruebas para cuantificar el fenómeno del benchmaxing en el reconocimiento de voz. Una evaluación de 11 modelos ASR de código abierto mostró que múltiples sistemas de alta puntuación reproducen textos de transcripción incorrectos de los benchmarks de VoxPopuli y LibriSpeech, incluso si el contenido de audio los contradice. Algunos modelos incluso dependen de la identificación de pistas acústicas como fuente de referencia, lo que resulta en puntuaciones que sobreestiman la capacidad verdadera de transcripción. 🔗 Lee el artículo original a través de AIHOT · https://aihot.virxact.com/items/cmt30vskr0e0wro6t19q13yic","category":"Industria","source":"Hugging Face：Blog（RSS","aggregationSource":"Hugging Face：Blog（RSS","pageTitle":"Medición de la optimización de benchmarks en reconocimiento de voz: La nueva prueba de Hugging Face revela el fenómeno del modelo ASR llamado \"cepillado de puntuaciones\" - Aioga Noticias de IA","description":"La última investigación de Hugging Face introduce tres pruebas para cuantificar el fenómeno del benchmaxing en el reconocimiento de voz. Una evaluación de 11 modelos ASR de código...","url":"https://www.aioga.com/es/news/cmt30vskr0e0wro6t19q13yic/","contentTranslated":true,"sourceHash":"189656fc6a4d3191","translatedAt":"2026-08-21T14:42:16.469Z"},"fr":{"title":"Mesurer l’optimisation des benchmarks en reconnaissance vocale : Le nouveau test de Hugging Face révèle le phénomène du modèle ASR « score brushing »","summary":"Les dernières recherches de Hugging Face introduisent trois tests pour quantifier le phénomène du benchmaxing en reconnaissance vocale. Une évaluation de 11 modèles ASR open source a montré que plusieurs systèmes à haut score reproduisent des textes de transcription incorrects issus des benchmarks VoxPopuli et LibriSpeech, même si le contenu audio les contredit. Certains modèles s’appuient même sur l’identification des indices acoustiques comme source de référence, ce qui aboutit à des scores qui surestiment la véritable capacité de transcription. 🔗 Lisez l’article original via AIHOT · https://aihot.virxact.com/items/cmt30vskr0e0wro6t19q13yic","category":"Industrie","source":"Hugging Face：Blog（RSS","aggregationSource":"Hugging Face：Blog（RSS","pageTitle":"Mesurer l’optimisation des benchmarks en reconnaissance vocale : Le nouveau test de Hugging Face révèle le phénomène du modèle ASR « score brushing » - Aioga Actualités IA","description":"Les dernières recherches de Hugging Face introduisent trois tests pour quantifier le phénomène du benchmaxing en reconnaissance vocale. Une évaluation de 11 modèles ASR open source...","url":"https://www.aioga.com/fr/news/cmt30vskr0e0wro6t19q13yic/","contentTranslated":true,"sourceHash":"189656fc6a4d3191","translatedAt":"2026-08-21T14:42:25.053Z"},"de":{"title":"Messung der Benchmark-Optimierung bei der Spracherkennung: Hugging Faces neuer Test enthüllt das ASR-Modell \"Score Brushing\"-Phänomen","summary":"Die neueste Forschung von Hugging Face stellt drei Tests vor, um das Phänomen des Benchmaxings in der Spracherkennung zu quantifizieren. Eine Auswertung von 11 Open-Source-ASR-Modellen zeigte, dass mehrere hochbewertete Systeme fehlerhafte Transkriptionstexte aus VoxPopuli- und LibriSpeech-Benchmarks reproduzieren, selbst wenn der Audioinhalt ihnen widerspricht. Einige Modelle verlassen sich sogar auf akustische Hinweiserkennung als Benchmark-Quelle, was zu Werten führt, die die wahre Transkriptionsfähigkeit überschätzen. 🔗 Lesen Sie den Originalartikel über AIHOT · https://aihot.virxact.com/items/cmt30vskr0e0wro6t19q13yic","category":"行业动态","source":"Hugging Face：Blog（RSS","aggregationSource":"Hugging Face：Blog（RSS","pageTitle":"Messung der Benchmark-Optimierung bei der Spracherkennung: Hugging Faces neuer Test enthüllt das ASR-Modell \"Score Brushing\"-Phänomen - Aioga KI-News","description":"Die neueste Forschung von Hugging Face stellt drei Tests vor, um das Phänomen des Benchmaxings in der Spracherkennung zu quantifizieren. Eine Auswertung von 11 Open-Source-ASR-Mode...","url":"https://www.aioga.com/de/news/cmt30vskr0e0wro6t19q13yic/","contentTranslated":true,"sourceHash":"189656fc6a4d3191","translatedAt":"2026-08-21T14:42:25.120Z"},"pt-BR":{"title":"Medindo otimização de benchmarks no reconhecimento de fala: O novo teste do Hugging Face revela o fenômeno do modelo ASR \"escovação de pontuação\"","summary":"A pesquisa mais recente do Hugging Face apresenta três testes para quantificar o fenômeno do benchmaxing no reconhecimento de fala. Uma avaliação de 11 modelos ASR de código aberto mostrou que múltiplos sistemas de alta pontuação reproduzem textos de transcrição incorretos dos benchmarks VoxPopuli e LibriSpeech, mesmo que o conteúdo de áudio os contradiga. Alguns modelos até dependem da identificação acústica de pistas como fonte de referência, resultando em escores que superestimam a verdadeira capacidade de transcrição. 🔗 Leia o artigo original via AIHOT · https://aihot.virxact.com/items/cmt30vskr0e0wro6t19q13yic","category":"行业动态","source":"Hugging Face：Blog（RSS","aggregationSource":"Hugging Face：Blog（RSS","pageTitle":"Medindo otimização de benchmarks no reconhecimento de fala: O novo teste do Hugging Face revela o fenômeno do modelo ASR \"escovação de pontuação\" - Aioga Notícias de IA","description":"A pesquisa mais recente do Hugging Face apresenta três testes para quantificar o fenômeno do benchmaxing no reconhecimento de fala. Uma avaliação de 11 modelos ASR de código aberto...","url":"https://www.aioga.com/pt-BR/news/cmt30vskr0e0wro6t19q13yic/","contentTranslated":true,"sourceHash":"189656fc6a4d3191","translatedAt":"2026-08-21T14:42:33.820Z"},"ru":{"title":"Измерение оптимизации бенчмарков в распознавании речи: новый тест Hugging Face раскрывает феномен модели ASR «brushing score brushing»","summary":"Последние исследования Hugging Face включают три теста для количественного определения феномена бенчмаксинга в распознавании речи. Оценка 11 открытых моделей ASR показала, что несколько систем с высокими баллами воспроизводят некорректные транскрипционные тексты из бенчмарков VoxPopuli и LibriSpeech, даже если аудиоконтент им противоречит. Некоторые модели даже полагаются на акустическую идентификацию сигналов в качестве источника ориентира, что приводит к оценкам, переоценивающим истинную способность транскрипции. 🔗 Прочитайте оригинальную статью на сайте AIHOT · https://aihot.virxact.com/items/cmt30vskr0e0wro6t19q13yic","category":"行业动态","source":"Hugging Face：Blog（RSS","aggregationSource":"Hugging Face：Blog（RSS","pageTitle":"Измерение оптимизации бенчмарков в распознавании речи: новый тест Hugging Face раскрывает феномен модели ASR «brushing score brushing» - Aioga Новости ИИ","description":"Последние исследования Hugging Face включают три теста для количественного определения феномена бенчмаксинга в распознавании речи. Оценка 11 открытых моделей ASR показала, что неск...","url":"https://www.aioga.com/ru/news/cmt30vskr0e0wro6t19q13yic/","contentTranslated":true,"sourceHash":"189656fc6a4d3191","translatedAt":"2026-08-21T14:42:33.793Z"},"ar":{"title":"قياس تحسين المعايير في التعرف على الكلام: اختبار Hugging Face الجديد يكشف عن ظاهرة \"فرشاة الدرجات\" في نموذج ASR","summary":"تقدم أحدث أبحاث موقع Hugging Face ثلاثة اختبارات لقياس ظاهرة القفز الثقيل في التعرف على الكلام. أظهر تقييم ل 11 نموذجا مفتوح المصدر لإعادة الكتابة المبكرة أن عدة أنظمة ذات درجات عالية تعيد إنتاج نصوص نسخ خاطئة من معايير VoxPopuli وLibriSpeech، حتى لو كان المحتوى الصوتي يتعارض معها. تعتمد بعض النماذج حتى على تحديد الإشارة الصوتية كمصدر مرجعي، مما يؤدي إلى درجات تبالغ في تقدير القدرة الحقيقية على النسخ الصحيح. 🔗 اقرأ المقال الأصلي عبر AIHOT · https://aihot.virxact.com/items/cmt30vskr0e0wro6t19q13yic","category":"行业动态","source":"Hugging Face：Blog（RSS","aggregationSource":"Hugging Face：Blog（RSS","pageTitle":"قياس تحسين المعايير في التعرف على الكلام: اختبار Hugging Face الجديد يكشف عن ظاهرة \"فرشاة الدرجات\" في نموذج ASR - Aioga أخبار الذكاء الاصطناعي","description":"تقدم أحدث أبحاث موقع Hugging Face ثلاثة اختبارات لقياس ظاهرة القفز الثقيل في التعرف على الكلام. أظهر تقييم ل 11 نموذجا مفتوح المصدر لإعادة الكتابة المبكرة أن عدة أنظمة ذات درجات عا...","url":"https://www.aioga.com/ar/news/cmt30vskr0e0wro6t19q13yic/","contentTranslated":true,"sourceHash":"189656fc6a4d3191","translatedAt":"2026-08-21T14:42:42.557Z"},"hi":{"title":"वाक् पहचान में बेंचमार्क अनुकूलन को मापना: हगिंग फेस के नए परीक्षण से एएसआर मॉडल \"स्कोर ब्रशिंग\" घटना का पता चलता है","summary":"हगिंग फेस के नवीनतम शोध में वाक् पहचान में बेंचमैक्सिंग की घटना को मापने के लिए तीन परीक्षण किए गए हैं। 11 ओपन-सोर्स एएसआर मॉडल के मूल्यांकन से पता चला है कि कई उच्च-स्कोरिंग सिस्टम VoxPopuli और LibriSpeech बेंचमार्क से गलत ट्रांसक्रिप्शन टेक्स्ट को पुन: पेश करते हैं, भले ही ऑडियो सामग्री उनका खंडन करती हो। कुछ मॉडल बेंचमार्क स्रोत के रूप में ध्वनिक क्यू पहचान पर भी भरोसा करते हैं, जिसके परिणामस्वरूप ऐसे स्कोर होते हैं जो सही ट्रांसक्रिप्शन क्षमता को कम करते हैं। 🔗 AIHOT के माध्यम से मूल लेख पढ़ें · https://aihot.virxact.com/items/cmt30vskr0e0wro6t19q13yic","category":"行业动态","source":"Hugging Face：Blog（RSS","aggregationSource":"Hugging Face：Blog（RSS","pageTitle":"वाक् पहचान में बेंचमार्क अनुकूलन को मापना: हगिंग फेस के नए परीक्षण से एएसआर मॉडल \"स्कोर ब्रशिंग\" घटना का पता चलता है - Aioga AI समाचार","description":"हगिंग फेस के नवीनतम शोध में वाक् पहचान में बेंचमैक्सिंग की घटना को मापने के लिए तीन परीक्षण किए गए हैं। 11 ओपन-सोर्स एएसआर मॉडल के मूल्यांकन से पता चला है कि कई उच्च-स्कोरिंग सिस्ट...","url":"https://www.aioga.com/hi/news/cmt30vskr0e0wro6t19q13yic/","contentTranslated":true,"sourceHash":"189656fc6a4d3191","translatedAt":"2026-08-21T14:42:42.487Z"},"it":{"title":"Misurare l'ottimizzazione dei benchmark nel riconoscimento vocale: il nuovo test di Hugging Face rivela il fenomeno del modello ASR \"score brushing\"","summary":"Le ultime ricerche di Hugging Face introducono tre test per quantificare il fenomeno del benchmaxing nel riconoscimento vocale. Una valutazione di 11 modelli ASR open source ha mostrato che molteplici sistemi ad alto punteggio riproducono testi di trascrizione errati dai benchmark VoxPopuli e LibriSpeech, anche se il contenuto audio li contraddice. Alcuni modelli si affidano persino all'identificazione acustica dei segnali come fonte di benchmark, ottenendo punteggi che sovrastimano la vera capacità di trascrizione. 🔗 Leggi l'articolo originale su AIHOT · https://aihot.virxact.com/items/cmt30vskr0e0wro6t19q13yic","category":"行业动态","source":"Hugging Face：Blog（RSS","aggregationSource":"Hugging Face：Blog（RSS","pageTitle":"Misurare l'ottimizzazione dei benchmark nel riconoscimento vocale: il nuovo test di Hugging Face rivela il fenomeno del modello ASR \"score brushing\" - Aioga Notizie IA","description":"Le ultime ricerche di Hugging Face introducono tre test per quantificare il fenomeno del benchmaxing nel riconoscimento vocale. Una valutazione di 11 modelli ASR open source ha mos...","url":"https://www.aioga.com/it/news/cmt30vskr0e0wro6t19q13yic/","contentTranslated":true,"sourceHash":"189656fc6a4d3191","translatedAt":"2026-08-21T14:42:51.006Z"},"nl":{"title":"Benchmarkoptimalisatie meten in spraakherkenning: De nieuwe test van Hugging Face onthult het ASR-model \"score brushing\"-fenomeen","summary":"Het nieuwste onderzoek van Hugging Face introduceert drie tests om het fenomeen benchmaxing in spraakherkenning te kwantificeren. Een evaluatie van 11 open-source ASR-modellen toonde aan dat meerdere hoogscorende systemen onjuiste transcriptieteksten van VoxPopuli- en LibriSpeech-benchmarks reproduceren, zelfs als de audio-inhoud deze tegenspreekt. Sommige modellen vertrouwen zelfs op akoestische cue-identificatie als benchmarkbron, wat resulteert in scores die de ware transcriptiecapaciteit overschatten. 🔗 Lees het originele artikel via AIHOT · https://aihot.virxact.com/items/cmt30vskr0e0wro6t19q13yic","category":"行业动态","source":"Hugging Face：Blog（RSS","aggregationSource":"Hugging Face：Blog（RSS","pageTitle":"Benchmarkoptimalisatie meten in spraakherkenning: De nieuwe test van Hugging Face onthult het ASR-model \"score brushing\"-fenomeen - Aioga AI-nieuws","description":"Het nieuwste onderzoek van Hugging Face introduceert drie tests om het fenomeen benchmaxing in spraakherkenning te kwantificeren. Een evaluatie van 11 open-source ASR-modellen toon...","url":"https://www.aioga.com/nl/news/cmt30vskr0e0wro6t19q13yic/","contentTranslated":true,"sourceHash":"189656fc6a4d3191","translatedAt":"2026-08-21T14:42:51.250Z"},"tr":{"title":"Konuşma tanıma alanında kıyaslama optimizasyonunu ölçmek: Hugging Face'in yeni testi, ASR modeli \"skor fırçalama\" olgusunu ortaya koyuyor","summary":"Hugging Face'in son araştırması, konuşma tanıma alanında benchmaxing olgusunu niceliklendirmek için üç test sunuyor. 11 açık kaynak ASR modelinin değerlendirilmesi, birden fazla yüksek puanlı sistemin VoxPopuli ve LibriSpeech kıyaslamalarından yanlış transkripsiyon metinlerini yeniden ürettiğini, ses içeriği bu verilerle çelişse bile, gösterdi. Bazı modeller akustik işaret tanımlamasını bir kıyaslama kaynağı olarak kullanır, bu da gerçek transkripsiyon yeteneğini fazla tahmin eden puanlar ortaya çıkarır. 🔗 Orijinal makaleyi AIHOT üzerinden okuyun · https://aihot.virxact.com/items/cmt30vskr0e0wro6t19q13yic","category":"行业动态","source":"Hugging Face：Blog（RSS","aggregationSource":"Hugging Face：Blog（RSS","pageTitle":"Konuşma tanıma alanında kıyaslama optimizasyonunu ölçmek: Hugging Face'in yeni testi, ASR modeli \"skor fırçalama\" olgusunu ortaya koyuyor - Aioga AI Haberleri","description":"Hugging Face'in son araştırması, konuşma tanıma alanında benchmaxing olgusunu niceliklendirmek için üç test sunuyor. 11 açık kaynak ASR modelinin değerlendirilmesi, birden fazla yü...","url":"https://www.aioga.com/tr/news/cmt30vskr0e0wro6t19q13yic/","contentTranslated":true,"sourceHash":"189656fc6a4d3191","translatedAt":"2026-08-21T14:42:59.327Z"},"vi":{"title":"Đo lường tối ưu hóa chuẩn trong nhận dạng giọng nói: Bài kiểm tra mới của Hugging Face tiết lộ hiện tượng \"chải điểm\" của mô hình ASR","summary":"Nghiên cứu mới nhất của Hugging Face giới thiệu ba bài kiểm tra để định lượng hiện tượng benchmaxing trong nhận dạng giọng nói. Một đánh giá 11 mô hình ASR mã nguồn mở cho thấy nhiều hệ thống có điểm số cao tái tạo các văn bản phiên âm sai từ các bài kiểm tra VoxPopuli và LibriSpeech, ngay cả khi nội dung âm thanh mâu thuẫn với chúng. Một số mô hình thậm chí dựa vào nhận dạng tín hiệu âm học làm nguồn chuẩn, dẫn đến điểm số đánh giá quá cao khả năng phiên âm thực sự. 🔗 Đọc bài viết gốc qua AIHOT · https://aihot.virxact.com/items/cmt30vskr0e0wro6t19q13yic","category":"行业动态","source":"Hugging Face：Blog（RSS","aggregationSource":"Hugging Face：Blog（RSS","pageTitle":"Đo lường tối ưu hóa chuẩn trong nhận dạng giọng nói: Bài kiểm tra mới của Hugging Face tiết lộ hiện tượng \"chải điểm\" của mô hình ASR - Tin tức AI Aioga","description":"Nghiên cứu mới nhất của Hugging Face giới thiệu ba bài kiểm tra để định lượng hiện tượng benchmaxing trong nhận dạng giọng nói. Một đánh giá 11 mô hình ASR mã nguồn mở cho thấy nhi...","url":"https://www.aioga.com/vi/news/cmt30vskr0e0wro6t19q13yic/","contentTranslated":true,"sourceHash":"189656fc6a4d3191","translatedAt":"2026-08-21T14:43:00.040Z"},"id":{"title":"Mengukur optimasi tolok ukur dalam pengenalan suara: Tes baru Hugging Face mengungkap fenomena \"scorebrushing\" model ASR","summary":"Penelitian terbaru Hugging Face memperkenalkan tiga tes untuk mengukur fenomena benchmaxing dalam pengenalan suara. Evaluasi terhadap 11 model ASR open-source menunjukkan bahwa beberapa sistem dengan skor tinggi mereproduksi teks transkripsi yang salah dari tolok ukur VoxPopuli dan LibriSpeech, meskipun konten audio bertentangan dengan mereka. Beberapa model bahkan mengandalkan identifikasi isyarat akustik sebagai sumber tolok ukur, menghasilkan skor yang melebih-lebihkan kemampuan transkripsi yang sebenarnya. 🔗 Baca artikel asli melalui AIHOT · https://aihot.virxact.com/items/cmt30vskr0e0wro6t19q13yic","category":"行业动态","source":"Hugging Face：Blog（RSS","aggregationSource":"Hugging Face：Blog（RSS","pageTitle":"Mengukur optimasi tolok ukur dalam pengenalan suara: Tes baru Hugging Face mengungkap fenomena \"scorebrushing\" model ASR - Berita AI Aioga","description":"Penelitian terbaru Hugging Face memperkenalkan tiga tes untuk mengukur fenomena benchmaxing dalam pengenalan suara. Evaluasi terhadap 11 model ASR open-source menunjukkan bahwa beb...","url":"https://www.aioga.com/id/news/cmt30vskr0e0wro6t19q13yic/","contentTranslated":true,"sourceHash":"189656fc6a4d3191","translatedAt":"2026-08-21T14:43:07.904Z"},"th":{"title":"การวัดการเพิ่มประสิทธิภาพมาตรฐานในการรู้จําเสียงพูด: การทดสอบใหม่ของ Hugging Face เผยปรากฏการณ์ \"การปัดคะแนน\" ของโมเดล ASR","summary":"งานวิจัยล่าสุดของ Hugging Face นําเสนอการทดสอบสามแบบเพื่อวัดปรากฏการณ์การเบนช์แม็กซ์ในระบบรู้จําเสียงพูด การประเมินโมเดล ASR แบบโอเพ่นซอร์ส 11 แบบแสดงให้เห็นว่าระบบที่มีคะแนนสูงหลายระบบสามารถทําซ้ําข้อความถอดความที่ไม่ถูกต้องจากการทดสอบ VoxPopuli และ LibriSpeech แม้ว่าเนื้อหาเสียงจะขัดแย้งกับข้อความเหล่านั้น บางรุ่นยังใช้การระบุสัญญาณเสียงเป็นแหล่งอ้างอิง ส่งผลให้คะแนนประเมินความสามารถในการถอดเสียงจริงเกินจริง 🔗 อ่านบทความต้นฉบับผ่าน AIHOT · https://aihot.virxact.com/items/cmt30vskr0e0wro6t19q13yic","category":"行业动态","source":"Hugging Face：Blog（RSS","aggregationSource":"Hugging Face：Blog（RSS","pageTitle":"การวัดการเพิ่มประสิทธิภาพมาตรฐานในการรู้จําเสียงพูด: การทดสอบใหม่ของ Hugging Face เผยปรากฏการณ์ \"การปัดคะแนน\" ของโมเดล ASR - ข่าว AI Aioga","description":"งานวิจัยล่าสุดของ Hugging Face นําเสนอการทดสอบสามแบบเพื่อวัดปรากฏการณ์การเบนช์แม็กซ์ในระบบรู้จําเสียงพูด การประเมินโมเดล ASR แบบโอเพ่นซอร์ส 11 แบบแสดงให้เห็นว่าระบบที่มีคะแนนสูงหลา...","url":"https://www.aioga.com/th/news/cmt30vskr0e0wro6t19q13yic/","contentTranslated":true,"sourceHash":"189656fc6a4d3191","translatedAt":"2026-08-21T14:43:07.871Z"},"pl":{"title":"Pomiar optymalizacji benchmarków w rozpoznawaniu mowy: Nowy test Hugging Face ujawnia zjawisko \"szczotkowania wyników\" modelu ASR","summary":"Najnowsze badania Hugging Face wprowadzają trzy testy mające na celu ilościowe określenie zjawiska benchmaxingu w rozpoznawaniu mowy. Ocena 11 otwartych modeli ASR wykazała, że wiele systemów o wysokiej ocenie odtwarza błędne teksty transkrypcyjne z benchmarków VoxPopuli i LibriSpeech, nawet jeśli treści audio im przeczą. Niektóre modele opierają się nawet na identyfikacji sygnałów akustycznych jako źródle odniesienia, co skutkuje wynikami przeceniającymi możliwości prawdziwej transkrypcji. 🔗 Przeczytaj oryginalny artykuł za pośrednictwem AIHOT · https://aihot.virxact.com/items/cmt30vskr0e0wro6t19q13yic","category":"行业动态","source":"Hugging Face：Blog（RSS","aggregationSource":"Hugging Face：Blog（RSS","pageTitle":"Pomiar optymalizacji benchmarków w rozpoznawaniu mowy: Nowy test Hugging Face ujawnia zjawisko \"szczotkowania wyników\" modelu ASR - Aioga Wiadomości AI","description":"Najnowsze badania Hugging Face wprowadzają trzy testy mające na celu ilościowe określenie zjawiska benchmaxingu w rozpoznawaniu mowy. Ocena 11 otwartych modeli ASR wykazała, że wie...","url":"https://www.aioga.com/pl/news/cmt30vskr0e0wro6t19q13yic/","contentTranslated":true,"sourceHash":"189656fc6a4d3191","translatedAt":"2026-08-21T14:43:16.527Z"}},"evidenceTier":"verified-news","reviewStatus":"editorial-selected","indexable":true,"editorialCover":""}}