{"@context":"https://schema.org","@type":"NewsArticle","generatedAt":"2026-07-23T06:40:50.084Z","headline":"SetwiseEvalKit：面向文档集合交互质量的评测基准与优化框架","description":"研究者提出SetwiseEvalKit，一个覆盖短文本与长文本场景、包含约28K条高质量评估准则的三级九维度文档集合评测基准。对12种重排序器的评估显示，最佳方法覆盖率不足45%，跨文档协调维度普遍薄弱。基于此提出的免训练方法Rubric4Setwise将准则转化为集合选择信号，在更少文档和搜索轮次下取得最优下游生成性能，是唯一在两个场景均保持SOTA的方法。","url":"https://www.aioga.com/news/cmrwz3b2v03y0robh2j8sh6gx/","mainEntityOfPage":"https://www.aioga.com/news/cmrwz3b2v03y0robh2j8sh6gx/","datePublished":"2026-07-22T00:00:00.000Z","dateModified":"2026-07-22T00:00:00.000Z","inLanguage":"zh-CN","publisher":{"@type":"NewsMediaOrganization","name":"Aioga","url":"https://www.aioga.com"},"citation":["https://arxiv.org/abs/2607.19747","https://aihot.virxact.com/items/cmrwz3b2v03y0robh2j8sh6gx"],"canonicalUrl":"https://www.aioga.com/news/cmrwz3b2v03y0robh2j8sh6gx/","directAnswer":{"@type":"Answer","text":"Aioga 编辑摘要：研究者提出SetwiseEvalKit，一个覆盖短文本与长文本场景、包含约28K条高质量评估准则的三级九维度文档集合评测基准。 Aioga 将其归入「论文研究」方向，重点关注它对真实使用和行业竞争的影响。","url":"https://www.aioga.com/news/cmrwz3b2v03y0robh2j8sh6gx/","dateCreated":"2026-07-22T00:00:00.000Z","author":{"@type":"Organization","@id":"https://www.aioga.com/authors/aioga-editorial/#editorial-team","name":"Aioga Editorial Team","url":"https://www.aioga.com/authors/aioga-editorial/"}},"evidence":[{"@type":"CreativeWork","name":"arXiv source article","url":"https://arxiv.org/abs/2607.19747","datePublished":"2026-07-22T00:00:00.000Z","provider":{"@type":"Organization","name":"arXiv","url":"https://arxiv.org/abs/2607.19747"}},{"@type":"CreativeWork","name":"AIHot archive record","url":"https://aihot.virxact.com/items/cmrwz3b2v03y0robh2j8sh6gx","datePublished":"2026-07-22T00:00:00.000Z","provider":{"@type":"Organization","name":"AIHot","url":"https://aihot.virxact.com/items/cmrwz3b2v03y0robh2j8sh6gx"}}],"aggregationSource":"HuggingFace Daily Papers（社区热门论文）","originalPublisher":{"name":"arXiv","url":"https://arxiv.org/abs/2607.19747"},"article":{"id":"cmrwz3b2v03y0robh2j8sh6gx","slug":"cmrwz3b2v03y0robh2j8sh6gx","url":"https://www.aioga.com/news/cmrwz3b2v03y0robh2j8sh6gx/","title":"SetwiseEvalKit：面向文档集合交互质量的评测基准与优化框架","title_en":"Beyond Relevance-Centric Retrieval： Rubric-Oriented Document Set Selection and Ranking","summary":"研究者提出SetwiseEvalKit，一个覆盖短文本与长文本场景、包含约28K条高质量评估准则的三级九维度文档集合评测基准。对12种重排序器的评估显示，最佳方法覆盖率不足45%，跨文档协调维度普遍薄弱。基于此提出的免训练方法Rubric4Setwise将准则转化为集合选择信号，在更少文档和搜索轮次下取得最优下游生成性能，是唯一在两个场景均保持SOTA的方法。","source":"HuggingFace Daily Papers（社区热门论文）","sourceUrl":"https://arxiv.org/abs/2607.19747","aiHotUrl":"https://aihot.virxact.com/items/cmrwz3b2v03y0robh2j8sh6gx","publishedAt":"2026-07-22T00:00:00.000Z","category":"论文研究","score":67,"selected":false,"articleBody":["arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.","Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs ：https://info.arxiv.org/labs/index.html."],"articleImages":[{"sourceUrl":"https://arxiv.org/static/base/1.0.1/images/funders/simons-foundation.png","alt":"Simons Foundation","afterParagraph":1,"url":"/media/articles/cmrwz3b2v03y0robh2j8sh6gx/e2d7f38d62f5ca91.png"},{"sourceUrl":"https://arxiv.org/static/base/1.0.1/images/funders/simons-foundation-international.png","alt":"Simons Foundation International","afterParagraph":1,"url":"/media/articles/cmrwz3b2v03y0robh2j8sh6gx/1d56e29c5557cbdc.png"},{"sourceUrl":"https://arxiv.org/static/base/1.0.1/images/funders/schmidt-sciences.png","alt":"Schmidt Sciences","afterParagraph":1,"url":"/media/articles/cmrwz3b2v03y0robh2j8sh6gx/8e18212b8219c104.png"}],"mediaStatus":"ok","articleBodyZh":["arXivLabs 是一个框架，允许合作者直接在我们的网站上开发和分享新的 arXiv 功能。","有一个可以为 arXiv 社区增加价值的项目想法吗？了解更多关于 arXivLabs 的信息：https://info.arxiv.org/labs/index.html。"],"translationStatus":"translated","bodyOrigin":"source-page","editorial":{"summary":"Aioga 编辑摘要：研究者提出SetwiseEvalKit，一个覆盖短文本与长文本场景、包含约28K条高质量评估准则的三级九维度文档集合评测基准。 Aioga 将其归入「论文研究」方向，重点关注它对真实使用和行业竞争的影响。","background":"背景分析：模型与研究类动态需要结合能力边界、开放方式、成本、可用性和真实任务表现判断，单项指标领先不等于已经形成稳定采用。","viewpoint":"Aioga 判断：这条动态更适合作为行业观察信号，当前信息足以建立线索，但不足以推导长期结论。","implications":"影响分析：对相关团队而言，短期应先核对来源、可用范围和实际成本，再判断是否值得接入或跟进。","nextStep":"后续观察：继续观察官方文档、实际可用性、价格变化、开发者反馈和竞品回应。","evidenceRefs":["title","summary","articleBody"],"confidence":"medium","status":"published","aiGenerated":false,"autoApproved":true,"generatedBy":"rule-safe-fallback","generatedAt":"2026-07-23T06:49:19.056Z","sourceHash":"49c94071f1716380","validation":{"passed":true,"mode":"rule-safe-fallback","checks":["schema","length","source-attribution","no-html"]}},"tags":["论文研究","HuggingFace Daily Papers（社区热门论文）"],"translations":{"zh-CN":{"title":"SetwiseEvalKit：面向文档集合交互质量的评测基准与优化框架","summary":"研究者提出SetwiseEvalKit，一个覆盖短文本与长文本场景、包含约28K条高质量评估准则的三级九维度文档集合评测基准。对12种重排序器的评估显示，最佳方法覆盖率不足45%，跨文档协调维度普遍薄弱。基于此提出的免训练方法Rubric4Setwise将准则转化为集合选择信号，在更少文档和搜索轮次下取得最优下游生成性能，是唯一在两个场景均保持SOTA的方法。","category":"论文研究","source":"arXiv","aggregationSource":"HuggingFace Daily Papers（社区热门论文）","pageTitle":"SetwiseEvalKit：面向文档集合交互质量的评测基准与优化框架 - Aioga AI资讯","description":"研究者提出SetwiseEvalKit，一个覆盖短文本与长文本场景、包含约28K条高质量评估准则的三级九维度文档集合评测基准。对12种重排序器的评估显示，最佳方法覆盖率不足45%，跨文档协调维度普遍薄弱。基于此提出的免训练方法Rubric4Setwise将准则转化为集合选择信号，在更少文档和搜索轮次下取得最优下游生成性能，是唯一在两个场景均保持SOTA的方法...","url":"https://www.aioga.com/news/cmrwz3b2v03y0robh2j8sh6gx/"},"en":{"title":"SetwiseEvalKit: Evaluation Benchmark and Optimization Framework for Interaction Quality in Document Collections","summary":"Researchers proposed SetwiseEvalKit, a three-level, nine-dimension document set evaluation benchmark covering both short-text and long-text scenarios, containing approximately 28K high-quality evaluation criteria. Evaluation of 12 re-rankers showed that the best method had a coverage of less than 45%, and cross-document coordination dimensions were generally weak. Based on this, the training-free method Rubric4Setwise was proposed, which converts criteria into set selection signals, achieving optimal downstream generation performance with fewer documents and search rounds, and is the only method that maintains SOTA in both scenarios.","category":"Research","source":"HuggingFace Daily Papers（社区热门论文）","aggregationSource":"HuggingFace Daily Papers（社区热门论文）","pageTitle":"SetwiseEvalKit: Evaluation Benchmark and Optimization Framework for Interaction Quality in Document Collections - Aioga AI News","description":"Researchers proposed SetwiseEvalKit, a three-level, nine-dimension document set evaluation benchmark covering both short-text and long-text scenarios, containing approximately 28K...","url":"https://www.aioga.com/en/news/cmrwz3b2v03y0robh2j8sh6gx/","contentTranslated":true,"sourceHash":"874c2dca683db9eb","translatedAt":"2026-07-23T04:00:08.519Z"},"ja":{"title":"SetwiseEvalKit：ドキュメント集合におけるインタラクション品質の評価ベンチマークと最適化フレームワーク","summary":"研究者はSetwiseEvalKitを提案した。これは短文および長文のシナリオをカバーし、約28K件の高品質評価基準を含む三層九次元の文書集合評価ベンチマークである。12種類の再ランキング手法の評価では、最良の方法でもカバレッジは45%に満たず、文書間の協調次元は全般的に弱いことが示された。これに基づき提案された学習不要の手法Rubric4Setwiseは、基準を集合選択シグナルに変換し、より少ない文書と検索ラウンドで最適な下流生成性能を達成し、両方のシナリオでSOTAを維持する唯一の方法である。","category":"論文研究","source":"HuggingFace Daily Papers（社区热门论文）","aggregationSource":"HuggingFace Daily Papers（社区热门论文）","pageTitle":"SetwiseEvalKit：ドキュメント集合におけるインタラクション品質の評価ベンチマークと最適化フレームワーク - Aioga AIニュース","description":"研究者はSetwiseEvalKitを提案した。これは短文および長文のシナリオをカバーし、約28K件の高品質評価基準を含む三層九次元の文書集合評価ベンチマークである。12種類の再ランキング手法の評価では、最良の方法でもカバレッジは45%に満たず、文書間の協調次元は全般的に弱いことが示された。これに基づき提案された学習不要の手法Rubric4Setwiseは、...","url":"https://www.aioga.com/ja/news/cmrwz3b2v03y0robh2j8sh6gx/","contentTranslated":true,"sourceHash":"874c2dca683db9eb","translatedAt":"2026-07-23T04:00:20.745Z"},"ko":{"title":"SetwiseEvalKit: 문서 집합 상호작용 품질을 위한 평가 기준 및 최적화 프레임워크","summary":"연구자들은 SetwiseEvalKit을 제안했으며, 이는 단문과 장문 시나리오를 포괄하고 약 28K개의 고품질 평가 기준을 포함한 3단계 9차원 문서 집합 평가 벤치마크이다. 12종의 재정렬기를 평가한 결과, 최상의 방법의 커버리지는 45%에도 미치지 못했으며, 문서 간 조정 차원은 전반적으로 약했다. 이를 기반으로 제안된 학습 없는 방법 Rubric4Setwise는 기준을 집합 선택 신호로 변환하여 더 적은 문서와 검색 라운드에서 최적의 다운스트림 생성 성능을 달성했으며, 두 시나리오 모두에서 SOTA를 유지한 유일한 방법이다.","category":"연구","source":"HuggingFace Daily Papers（社区热门论文）","aggregationSource":"HuggingFace Daily Papers（社区热门论文）","pageTitle":"SetwiseEvalKit: 문서 집합 상호작용 품질을 위한 평가 기준 및 최적화 프레임워크 - Aioga AI 뉴스","description":"연구자들은 SetwiseEvalKit을 제안했으며, 이는 단문과 장문 시나리오를 포괄하고 약 28K개의 고품질 평가 기준을 포함한 3단계 9차원 문서 집합 평가 벤치마크이다. 12종의 재정렬기를 평가한 결과, 최상의 방법의 커버리지는 45%에도 미치지 못했으며, 문서 간 조정 차원은 전반적으로 약했다. 이를 기반으로 제안...","url":"https://www.aioga.com/ko/news/cmrwz3b2v03y0robh2j8sh6gx/","contentTranslated":true,"sourceHash":"874c2dca683db9eb","translatedAt":"2026-07-23T04:01:09.688Z"},"es":{"title":"SetwiseEvalKit: Marco de referencia y optimización para la evaluación de la calidad de la interacción con conjuntos de documentos","summary":"Los investigadores propusieron SetwiseEvalKit, un benchmark de evaluación de colecciones de documentos de tres niveles y nueve dimensiones, que cubre escenarios de textos cortos y largos, e incluye aproximadamente 28,000 criterios de evaluación de alta calidad. La evaluación de 12 tipos de reordenadores mostró que la mejor cobertura de método fue inferior al 45%, y que la coordinación a través de dimensiones de documentos generalmente era débil. Basándose en esto, se propuso el método sin entrenamiento Rubric4Setwise, que convierte los criterios en señales de selección de colecciones, logrando el mejor rendimiento en generación descendente con menos documentos y rondas de búsqueda, siendo el único método que mantiene el estado del arte en ambos escenarios.","category":"Investigación","source":"HuggingFace Daily Papers（社区热门论文）","aggregationSource":"HuggingFace Daily Papers（社区热门论文）","pageTitle":"SetwiseEvalKit: Marco de referencia y optimización para la evaluación de la calidad de la interacción con conjuntos de documentos - Aioga Noticias de IA","description":"Los investigadores propusieron SetwiseEvalKit, un benchmark de evaluación de colecciones de documentos de tres niveles y nueve dimensiones, que cubre escenarios de textos cortos y...","url":"https://www.aioga.com/es/news/cmrwz3b2v03y0robh2j8sh6gx/","contentTranslated":true,"sourceHash":"874c2dca683db9eb","translatedAt":"2026-07-23T04:01:03.761Z"},"fr":{"title":"SetwiseEvalKit : Benchmark et cadre d'optimisation pour la qualité de l'interaction avec les ensembles de documents","summary":"Les chercheurs ont proposé SetwiseEvalKit, une référence d'évaluation de collections de documents à trois niveaux et neuf dimensions, couvrant à la fois les scénarios de textes courts et longs, contenant environ 28 000 critères d'évaluation de haute qualité. L'évaluation de 12 types de réordonneurs montre que la meilleure méthode ne couvre pas plus de 45 % des cas et que la coordination entre documents est généralement faible. Sur cette base, la méthode sans entraînement Rubric4Setwise transforme les critères en signaux de sélection de collections, obtenant les performances optimales de génération en aval avec moins de documents et moins de cycles de recherche, et étant la seule méthode à maintenir l'état de l'art dans les deux scénarios.","category":"Recherche","source":"HuggingFace Daily Papers（社区热门论文）","aggregationSource":"HuggingFace Daily Papers（社区热门论文）","pageTitle":"SetwiseEvalKit : Benchmark et cadre d'optimisation pour la qualité de l'interaction avec les ensembles de documents - Aioga Actualités IA","description":"Les chercheurs ont proposé SetwiseEvalKit, une référence d'évaluation de collections de documents à trois niveaux et neuf dimensions, couvrant à la fois les scénarios de textes cou...","url":"https://www.aioga.com/fr/news/cmrwz3b2v03y0robh2j8sh6gx/","contentTranslated":true,"sourceHash":"874c2dca683db9eb","translatedAt":"2026-07-23T04:01:53.471Z"},"de":{"title":"SetwiseEvalKit: Bewertungsstandard und Optimierungsrahmen für die Interaktionsqualität von Dokumentensammlungen","summary":"Die Forscher haben SetwiseEvalKit vorgestellt, ein Benchmark zur Bewertung von Dokumentensammlungen, das Szenarien mit kurzen und langen Texten abdeckt und rund 28.000 hochwertige Bewertungskriterien in einer dreistufigen, neun-dimensionalen Struktur enthält. Die Bewertung von 12 verschiedenen Re-Ranker-Modellen zeigt, dass die beste Methode eine Abdeckung von weniger als 45 % erreicht und die bereichsübergreifende Koordination zwischen den Dokumenten generell schwach ist. Darauf aufbauend wurde die trainingsfreie Methode Rubric4Setwise vorgeschlagen, die die Kriterien in Signals für die Auswahl von Sammlungen umwandelt und bei weniger Dokumenten und Suchrunden die beste nachgelagerte Generierungsleistung erzielt; sie ist die einzige Methode, die in beiden Szenarien den SOTA-Status beibehält.","category":"论文研究","source":"HuggingFace Daily Papers（社区热门论文）","aggregationSource":"HuggingFace Daily Papers（社区热门论文）","pageTitle":"SetwiseEvalKit: Bewertungsstandard und Optimierungsrahmen für die Interaktionsqualität von Dokumentensammlungen - Aioga KI-News","description":"Die Forscher haben SetwiseEvalKit vorgestellt, ein Benchmark zur Bewertung von Dokumentensammlungen, das Szenarien mit kurzen und langen Texten abdeckt und rund 28.000 hochwertige...","url":"https://www.aioga.com/de/news/cmrwz3b2v03y0robh2j8sh6gx/","contentTranslated":true,"sourceHash":"874c2dca683db9eb","translatedAt":"2026-07-23T04:01:53.058Z"},"pt-BR":{"title":"SetwiseEvalKit: Um marco de avaliação e framework de otimização voltado para a qualidade da interação com coleções de documentos","summary":"Os pesquisadores propuseram o SetwiseEvalKit, um benchmark de avaliação de conjuntos de documentos de três níveis e nove dimensões, cobrindo cenários de textos curtos e longos, contendo aproximadamente 28 mil critérios de avaliação de alta qualidade. A avaliação de 12 tipos de reordenadores mostrou que a melhor abordagem alcança menos de 45% de cobertura, sendo geralmente fraca na dimensão de coordenação entre documentos. Com base nisso, o método sem treinamento Rubric4Setwise transforma os critérios em sinais de seleção de conjuntos, alcançando o melhor desempenho na geração de resultados subsequentes com menos documentos e rodadas de busca, sendo o único método a manter o estado da arte em ambos os cenários.","category":"论文研究","source":"HuggingFace Daily Papers（社区热门论文）","aggregationSource":"HuggingFace Daily Papers（社区热门论文）","pageTitle":"SetwiseEvalKit: Um marco de avaliação e framework de otimização voltado para a qualidade da interação com coleções de documentos - Aioga Notícias de IA","description":"Os pesquisadores propuseram o SetwiseEvalKit, um benchmark de avaliação de conjuntos de documentos de três níveis e nove dimensões, cobrindo cenários de textos curtos e longos, con...","url":"https://www.aioga.com/pt-BR/news/cmrwz3b2v03y0robh2j8sh6gx/","contentTranslated":true,"sourceHash":"874c2dca683db9eb","translatedAt":"2026-07-23T04:02:34.490Z"},"ru":{"title":"SetwiseEvalKit: Бенчмарк и оптимизационный фреймворк для оценки качества взаимодействия с коллекциями документов","summary":"Исследователи предложили SetwiseEvalKit — эталон для оценки коллекций документов с тремя уровнями и девятью измерениями, охватывающий как короткие, так и длинные тексты, включающий около 28 тысяч высококачественных критериев оценки. Оценка 12 различных методов повторной сортировки показала, что у лучших методов коэффициент покрытия не превышает 45%, а координация между документами по измерениям в целом слаба. На основе этого была предложена методика без обучения Rubric4Setwise, которая преобразует критерии в сигналы выбора для коллекций и достигает оптимальной производительности в генерации на последующих этапах с меньшим количеством документов и раундов поиска, являясь единственным методом, поддерживающим SOTA в обоих сценариях.","category":"论文研究","source":"HuggingFace Daily Papers（社区热门论文）","aggregationSource":"HuggingFace Daily Papers（社区热门论文）","pageTitle":"SetwiseEvalKit: Бенчмарк и оптимизационный фреймворк для оценки качества взаимодействия с коллекциями документов - Aioga Новости ИИ","description":"Исследователи предложили SetwiseEvalKit — эталон для оценки коллекций документов с тремя уровнями и девятью измерениями, охватывающий как короткие, так и длинные тексты, включающий...","url":"https://www.aioga.com/ru/news/cmrwz3b2v03y0robh2j8sh6gx/","contentTranslated":true,"sourceHash":"874c2dca683db9eb","translatedAt":"2026-07-23T04:02:36.586Z"},"ar":{"title":"SetwiseEvalKit: إطار عمل للتقييم والتحسين لجودة التفاعل مع مجموعة من المستندات","summary":"اقترح الباحثون SetwiseEvalKit، وهو معيار تقييم لمجموعة المستندات من ثلاثة مستويات وتسعة أبعاد يغطي سيناريوهات النصوص القصيرة والطويلة، ويحتوي على حوالي 28 ألف معيار تقييم عالي الجودة. أظهرت التقييمات لـ 12 نوعًا من إعادة الترتيب أن أفضل طريقة لا تتجاوز نسبة التغطية 45٪، وكانت أبعاد التنسيق عبر المستندات ضعيفة بشكل عام. بناءً على ذلك، اقترحوا طريقة Rubric4Setwise بدون تدريب، والتي تحول المعايير إلى إشارات اختيار المجموعة، وتحقق أفضل أداء في توليد المخرجات على عدد أقل من المستندات وجولات البحث، وهي الطريقة الوحيدة التي تحافظ على مستوى SOTA في كل السيناريوهين.","category":"论文研究","source":"HuggingFace Daily Papers（社区热门论文）","aggregationSource":"HuggingFace Daily Papers（社区热门论文）","pageTitle":"SetwiseEvalKit: إطار عمل للتقييم والتحسين لجودة التفاعل مع مجموعة من المستندات - Aioga أخبار الذكاء الاصطناعي","description":"اقترح الباحثون SetwiseEvalKit، وهو معيار تقييم لمجموعة المستندات من ثلاثة مستويات وتسعة أبعاد يغطي سيناريوهات النصوص القصيرة والطويلة، ويحتوي على حوالي 28 ألف معيار تقييم عالي الجو...","url":"https://www.aioga.com/ar/news/cmrwz3b2v03y0robh2j8sh6gx/","contentTranslated":true,"sourceHash":"874c2dca683db9eb","translatedAt":"2026-07-23T04:03:19.179Z"},"hi":{"title":"SetwiseEvalKit: दस्तावेज़ संग्रह इंटरैक्शन गुणवत्ता के लिए मूल्यांकन मानक और अनुकूलन ढांचा","summary":"शोधकर्ताओं ने SetwiseEvalKit पेश किया, जो छोटे और लंबे पाठ परिदृश्यों को कवर करता है और लगभग 28K उच्च गुणवत्ता वाले मूल्यांकन मानदंडों वाला तीन-स्तरीय नौ-आयामी दस्तावेज़ संग्रह मूल्यांकन मानक है। 12 प्रकार के पुनर्संगठनकर्ताओं के मूल्यांकन से पता चला कि सबसे अच्छी विधि की कवरेज 45% से कम है, और दस्तावेज़ों के बीच समन्वय आयाम आमतौर पर कमजोर हैं। इसके आधार पर प्रस्तावित बिना प्रशिक्षण की विधि Rubric4Setwise मानदंडों को चयन संकेत में परिवर्तित करती है, और कम दस्तावेज़ और खोज चक्रों में सर्वोत्तम डाउनस्ट्रीम जनरेशन प्रदर्शन प्राप्त करती है, यह एकमात्र विधि है जो दोनों परिदृश्यों में SOTA बनाए रखती है।","category":"论文研究","source":"HuggingFace Daily Papers（社区热门论文）","aggregationSource":"HuggingFace Daily Papers（社区热门论文）","pageTitle":"SetwiseEvalKit: दस्तावेज़ संग्रह इंटरैक्शन गुणवत्ता के लिए मूल्यांकन मानक और अनुकूलन ढांचा - Aioga AI समाचार","description":"शोधकर्ताओं ने SetwiseEvalKit पेश किया, जो छोटे और लंबे पाठ परिदृश्यों को कवर करता है और लगभग 28K उच्च गुणवत्ता वाले मूल्यांकन मानदंडों वाला तीन-स्तरीय नौ-आयामी दस्तावेज़ संग्रह मूल...","url":"https://www.aioga.com/hi/news/cmrwz3b2v03y0robh2j8sh6gx/","contentTranslated":true,"sourceHash":"874c2dca683db9eb","translatedAt":"2026-07-23T04:03:21.728Z"},"it":{"title":"SetwiseEvalKit: Benchmark e framework di ottimizzazione per la qualità dell'interazione con raccolte di documenti","summary":"I ricercatori propongono SetwiseEvalKit, un benchmark di valutazione di collezioni di documenti a tre livelli e nove dimensioni, che copre scenari di testi brevi e lunghi e contiene circa 28.000 criteri di valutazione di alta qualità. La valutazione su 12 tipi di riorganizzatori mostra che il metodo migliore copre meno del 45% dei casi e le dimensioni di coordinazione inter-documenti risultano generalmente deboli. Sulla base di ciò, il metodo senza addestramento Rubric4Setwise trasforma i criteri in segnali di selezione della collezione, ottenendo le migliori prestazioni nella generazione downstream con meno documenti e cicli di ricerca ed è l'unico metodo a mantenere lo SOTA in entrambi gli scenari.","category":"论文研究","source":"HuggingFace Daily Papers（社区热门论文）","aggregationSource":"HuggingFace Daily Papers（社区热门论文）","pageTitle":"SetwiseEvalKit: Benchmark e framework di ottimizzazione per la qualità dell'interazione con raccolte di documenti - Aioga Notizie IA","description":"I ricercatori propongono SetwiseEvalKit, un benchmark di valutazione di collezioni di documenti a tre livelli e nove dimensioni, che copre scenari di testi brevi e lunghi e contien...","url":"https://www.aioga.com/it/news/cmrwz3b2v03y0robh2j8sh6gx/","contentTranslated":true,"sourceHash":"874c2dca683db9eb","translatedAt":"2026-07-23T04:04:06.294Z"},"nl":{"title":"SetwiseEvalKit: een benchmark- en optimalisatieframework voor de interactiekwaliteit van documentverzamelingen","summary":"Onderzoekers hebben SetwiseEvalKit voorgesteld, een benchmark voor documentverzamelingsevaluatie met drie niveaus en negen dimensies, die zowel korte als lange tekstscenario's beslaat en ongeveer 28.000 hoogwaardige evaluatiecriteria bevat. Evaluatie van 12 herordenaars toont aan dat de beste methode een dekkingsgraad van minder dan 45% bereikt, en dat de coördinatie tussen documenten over de dimensies over het algemeen zwak is. Op basis hiervan is de training-vrije methode Rubric4Setwise voorgesteld, die criteria omzet in verzamelfluitsignalen, en optimale downstream generatieve prestaties behaalt met minder documenten en zoekrondes, en de enige methode is die in beide scenario's de SOTA-niveau handhaaft.","category":"论文研究","source":"HuggingFace Daily Papers（社区热门论文）","aggregationSource":"HuggingFace Daily Papers（社区热门论文）","pageTitle":"SetwiseEvalKit: een benchmark- en optimalisatieframework voor de interactiekwaliteit van documentverzamelingen - Aioga AI-nieuws","description":"Onderzoekers hebben SetwiseEvalKit voorgesteld, een benchmark voor documentverzamelingsevaluatie met drie niveaus en negen dimensies, die zowel korte als lange tekstscenario's besl...","url":"https://www.aioga.com/nl/news/cmrwz3b2v03y0robh2j8sh6gx/","contentTranslated":true,"sourceHash":"874c2dca683db9eb","translatedAt":"2026-07-23T04:04:11.583Z"},"tr":{"title":"SetwiseEvalKit: Belge Koleksiyonu Etkileşim Kalitesi için Değerlendirme Standardı ve Optimizasyon Çerçevesi","summary":"Araştırmacılar, kısa ve uzun metin sahnelerini kapsayan ve yaklaşık 28 bin yüksek kaliteli değerlendirme kriteri içeren üç seviyeli dokuz boyutlu bir belge koleksiyonu değerlendirme standardı olan SetwiseEvalKit'i önerdiler. 12 tür yeniden sıralayıcı ile yapılan değerlendirmeler, en iyi yöntemin kapsama alanının %45’in altında olduğunu ve belge çapında koordinasyon boyutlarının genel olarak zayıf olduğunu gösterdi. Buna dayanarak önerilen eğitim gerektirmeyen yöntem Rubric4Setwise, kriterleri koleksiyon seçim sinyaline dönüştürmekte ve daha az belge ve arama turunda en iyi alt seviye üretim performansını elde etmekte olup, her iki sahnede de SOTA’yı koruyan tek yöntemdir.","category":"论文研究","source":"HuggingFace Daily Papers（社区热门论文）","aggregationSource":"HuggingFace Daily Papers（社区热门论文）","pageTitle":"SetwiseEvalKit: Belge Koleksiyonu Etkileşim Kalitesi için Değerlendirme Standardı ve Optimizasyon Çerçevesi - Aioga AI Haberleri","description":"Araştırmacılar, kısa ve uzun metin sahnelerini kapsayan ve yaklaşık 28 bin yüksek kaliteli değerlendirme kriteri içeren üç seviyeli dokuz boyutlu bir belge koleksiyonu değerlendirm...","url":"https://www.aioga.com/tr/news/cmrwz3b2v03y0robh2j8sh6gx/","contentTranslated":true,"sourceHash":"874c2dca683db9eb","translatedAt":"2026-07-23T04:04:55.334Z"},"vi":{"title":"SetwiseEvalKit: Bộ khung đánh giá và tối ưu chất lượng tương tác hướng tới tập hợp tài liệu","summary":"Các nhà nghiên cứu đã đề xuất SetwiseEvalKit, một bộ tiêu chuẩn đánh giá tài liệu ba cấp chín chiều, bao phủ cả kịch bản văn bản ngắn và dài, chứa khoảng 28.000 tiêu chí đánh giá chất lượng cao. Đánh giá trên 12 bộ tái xếp hạng cho thấy, phương pháp tốt nhất chỉ đạt độ bao phủ dưới 45%, các chiều phối hợp giữa các tài liệu nói chung còn yếu. Dựa trên đó, phương pháp không cần huấn luyện Rubric4Setwise đã biến các tiêu chí thành tín hiệu chọn lựa bộ, đạt hiệu suất sinh downstream tối ưu với ít tài liệu và lượt tìm kiếm hơn, là phương pháp duy nhất duy trì SOTA ở cả hai kịch bản.","category":"论文研究","source":"HuggingFace Daily Papers（社区热门论文）","aggregationSource":"HuggingFace Daily Papers（社区热门论文）","pageTitle":"SetwiseEvalKit: Bộ khung đánh giá và tối ưu chất lượng tương tác hướng tới tập hợp tài liệu - Tin tức AI Aioga","description":"Các nhà nghiên cứu đã đề xuất SetwiseEvalKit, một bộ tiêu chuẩn đánh giá tài liệu ba cấp chín chiều, bao phủ cả kịch bản văn bản ngắn và dài, chứa khoảng 28.000 tiêu chí đánh giá c...","url":"https://www.aioga.com/vi/news/cmrwz3b2v03y0robh2j8sh6gx/","contentTranslated":true,"sourceHash":"874c2dca683db9eb","translatedAt":"2026-07-23T04:04:56.180Z"},"id":{"title":"SetwiseEvalKit: Kerangka Evaluasi dan Optimalisasi Kualitas Interaksi untuk Kumpulan Dokumen","summary":"Para peneliti mengajukan SetwiseEvalKit, sebuah tolok ukur evaluasi koleksi dokumen tiga tingkat dan sembilan dimensi yang mencakup skenario teks pendek dan panjang, serta berisi sekitar 28 ribu kriteria evaluasi berkualitas tinggi. Evaluasi terhadap 12 jenis re-ranker menunjukkan bahwa metode terbaik hanya mencapai cakupan kurang dari 45%, dengan dimensi koordinasi antar-dokumen umumnya lemah. Berdasarkan hal ini, metode tanpa pelatihan Rubric4Setwise diusulkan, yang mengubah kriteria menjadi sinyal pemilihan koleksi, mencapai kinerja generasi downstream optimal dengan lebih sedikit dokumen dan putaran pencarian, serta merupakan satu-satunya metode yang mempertahankan SOTA di kedua skenario.","category":"论文研究","source":"HuggingFace Daily Papers（社区热门论文）","aggregationSource":"HuggingFace Daily Papers（社区热门论文）","pageTitle":"SetwiseEvalKit: Kerangka Evaluasi dan Optimalisasi Kualitas Interaksi untuk Kumpulan Dokumen - Berita AI Aioga","description":"Para peneliti mengajukan SetwiseEvalKit, sebuah tolok ukur evaluasi koleksi dokumen tiga tingkat dan sembilan dimensi yang mencakup skenario teks pendek dan panjang, serta berisi s...","url":"https://www.aioga.com/id/news/cmrwz3b2v03y0robh2j8sh6gx/","contentTranslated":true,"sourceHash":"874c2dca683db9eb","translatedAt":"2026-07-23T04:05:39.600Z"},"th":{"title":"SetwiseEvalKit: กรอบการประเมินและการปรับปรุงคุณภาพการโต้ตอบสำหรับชุดเอกสาร","summary":"นักวิจัยได้เสนอ SetwiseEvalKit ซึ่งเป็นชุดเกณฑ์การประเมินเอกสารสามระดับเก้ามิติที่ครอบคลุมสถานการณ์ข้อความสั้นและข้อความยาว รวมถึงเกณฑ์การประเมินคุณภาพสูงประมาณ 28,000 รายการ การประเมินตัวจัดเรียงลำดับใหม่ 12 ชนิดแสดงให้เห็นว่าวิธีที่ดีที่สุดมีอัตราการครอบคลุมน้อยกว่า 45% และมิติการประสานงานข้ามเอกสารค่อนข้างอ่อนแอ บนพื้นฐานนี้ วิธีการไม่ต้องฝึกที่เสนอ Rubric4Setwise แปลงเกณฑ์เป็นสัญญาณเลือกชุดเอกสาร โดยสามารถทำให้เกิดประสิทธิภาพการสร้างผลลัพธ์ลำดับต่อไปที่ดีที่สุดแม้ใช้เอกสารและรอบการค้นหาน้อย และเป็นวิธีเดียวที่สามารถรักษาระดับ SOTA ในทั้งสองสถานการณ์","category":"论文研究","source":"HuggingFace Daily Papers（社区热门论文）","aggregationSource":"HuggingFace Daily Papers（社区热门论文）","pageTitle":"SetwiseEvalKit: กรอบการประเมินและการปรับปรุงคุณภาพการโต้ตอบสำหรับชุดเอกสาร - ข่าว AI Aioga","description":"นักวิจัยได้เสนอ SetwiseEvalKit ซึ่งเป็นชุดเกณฑ์การประเมินเอกสารสามระดับเก้ามิติที่ครอบคลุมสถานการณ์ข้อความสั้นและข้อความยาว รวมถึงเกณฑ์การประเมินคุณภาพสูงประมาณ 28,000 รายการ การปร...","url":"https://www.aioga.com/th/news/cmrwz3b2v03y0robh2j8sh6gx/","contentTranslated":true,"sourceHash":"874c2dca683db9eb","translatedAt":"2026-07-23T04:05:42.811Z"},"pl":{"title":"SetwiseEvalKit: benchmark oceny i ramy optymalizacji jakości interakcji w zbiorach dokumentów","summary":"Badacze zaproponowali SetwiseEvalKit, zestaw punktów odniesienia do oceny dokumentów obejmujący scenariusze krótkiego i długiego tekstu, zawierający około 28 tys. wysokiej jakości kryteriów, składający się z trzech poziomów i dziewięciu wymiarów. Ocena 12 różnych re-rankerów wykazała, że najlepsza metoda osiągała pokrycie poniżej 45%, a wymiary koordynacji między dokumentami były generalnie słabe. W oparciu o to zaproponowano metodę bezuczeniową Rubric4Setwise, która przekształca kryteria w sygnały wyboru w zbiorze, osiągając najlepszą wydajność przy generowaniu w niższej liczbie dokumentów i rund wyszukiwania, będąc jedyną metodą utrzymującą stanowiące SOTA wyniki w obu scenariuszach.","category":"论文研究","source":"HuggingFace Daily Papers（社区热门论文）","aggregationSource":"HuggingFace Daily Papers（社区热门论文）","pageTitle":"SetwiseEvalKit: benchmark oceny i ramy optymalizacji jakości interakcji w zbiorach dokumentów - Aioga Wiadomości AI","description":"Badacze zaproponowali SetwiseEvalKit, zestaw punktów odniesienia do oceny dokumentów obejmujący scenariusze krótkiego i długiego tekstu, zawierający około 28 tys. wysokiej jakości...","url":"https://www.aioga.com/pl/news/cmrwz3b2v03y0robh2j8sh6gx/","contentTranslated":true,"sourceHash":"874c2dca683db9eb","translatedAt":"2026-07-23T04:06:36.317Z"}}}}