LEAD: Breaking the No-Recovery Bottleneck in Long-Horizon Reasoning
Authors Denys Pushkin†, Emmanuel Abbé†
View publication：https://arxiv.org/abs/2603.06870
Long-horizon execution in Large Language Models (LLMs) remains unstable even when high-level strategies are provided. Evaluating on controlled algorithmic puzzles, we demonstrate that while decomposition is essential for stability, extreme decomposition creates a “no-recovery bottleneck”. We show that this bottleneck becomes critical due to highly non-uniform error distribution, where consistent errors on a few “hard” steps become irreversible. To address this, we propose Lookahead-Enhanced Atomic Decomposition (LEAD). By incorporating short-horizon future validation and aggregating overlapping rollouts, LEAD provides enough isolation to maintain stability while retaining enough local context to correct errors. This enables the o4-mini model to solve Checkers Jumping up to complexity n = 13, whereas extreme decomposition fails beyond n = 11.
Divide-or-Conquer? Which Part Should You Distill Your LLM?
October 25, 2024 research area Speech and Natural Language Processing：/research/?domain=Speech%20and%20Natural%20Language%20Processing conference EMNLP：/research/?event=EMNLP
Recent methods have demonstrated that Large Language Models (LLMs) can solve reasoning tasks better when they are encouraged to solve subtasks of the main task first. In this paper we devise a similar strategy that breaks down reasoning tasks into a problem decomposition phase and a problem solving phase and show that the strategy is able to outperform a single stage solution. Further, we hypothesize that the decomposition should be easier to…
Joint Learning of Portrait Intrinsic Decomposition and Relighting
July 28, 2021 research area Computer Vision：/research/?domain=Computer%20Vision
Inverse rendering is the problem of decomposing an image into its intrinsic components, i.e. albedo, normal and lighting. To solve this ill-posed problem from single image, state-of-the-art methods in shape from shading mostly resort to supervised training on all the components on either synthetic or real datasets. Here, we propose a new self-supervised training paradigm that 1) reduces the need for full supervision on the decomposition task and…
Our research in machine learning breaks new ground every day.
扫码直达文章详情