Sakana AI 的 Error Diffusion 训练符合 Dale 原则的双流网络，无需反向传播

反向传播主导了深度学习，但它使用的机制大脑可能无法实现。具体来说，反向传播需要前向权重矩阵的精确转置。这就是权重传输问题。Sakana AI 的新论文《Diffusing Blame》：https://arxiv.org/pdf/2606.31700，直接应对了这一限制。研究团队训练了遵循戴尔（Dale）原则的网络，同时完全避免了权重传输。

误差扩散（Error Diffusion, ED）是一种局部学习规则，最早由金子（Kaneko, 2000）提出。每次权重更新只依赖三种信号：前突触活动、后突触激活导数以及一个全局错误信号。因此，ED 从不传递前向权重的转置，也不使用随机反馈矩阵。这种局部性使 ED 与戴尔原则天然兼容。然而，以往的工作仅在二分类和 MNIST 上演示了 ED。

为了满足这一限制，研究团队将每一层分为两条通路。一条为兴奋性（p），另一条为抑制性（n）。前向传播为每条通路计算兴奋性减抑制性的预激活值：

在这里，所有四个权重矩阵的元素都保持非负。偏置 bp 和 bn 是例外，因为它们不必非负。此外，W_np 和 W_pn 前的取负符号是结构性的，而非通过学习得到的。因此，跨通路连接保持抑制性，而所有可学习的权重保持非负。这一设计每层需要四个权重子矩阵。因此，它使用的参数量大约是单通路网络的 4 倍。对于相同架构，约为 DFA 的 3200 万参数，而 DFA 为 800 万。

在该架构确定后，主要扩展是模运算误差路由（modulo error routing）。这使误差扩散（ED）超越了二分类问题。对于隐藏单元 i，研究团队定义路由 r(i) = i mod C，其中 C 是输出维度。该单元然后从路由的误差分量中学习。简言之，每个隐藏单元被分配一个固定的输出通道。与 DFA 的随机反馈矩阵不同，ED 使用这种结构化对应关系。

基于该路由，研究团队为多类分类添加了三个改进：

通过这三项创新，误差扩散（ED）在MNIST上的准确率达到96.7%，在CIFAR-10上达到61.7%。相比之下，没有这些创新的种子ED准确率降至50.4%和11.6%。DFA在两个任务上的得分都更高，但违反了Dale原则，使用了约2.84M个负权重。值得注意的是，这是ED首次训练卷积网络。此前，Fujita（2026）使用扁平化的MLP在CIFAR-10上仅达到约55.2%。即便如此，61.7%仍远低于标准梯度方法的水平。

有趣的是，这些创新的重要性在不同任务间发生了反转。在MNIST上，移除层特定宽度是灾难性的（−71.4个百分点），准确率接近随机水平。批量中心化几乎没有影响（−0.3个百分点）。然而在CIFAR-10上，顺序则相反。移除批量中心化误差造成最大下降（−47.9个百分点），使五个种子中的四个崩溃。这种反转揭示了单一基准评估无法看到的任务依赖性信用分配瓶颈。

除了分类任务，研究团队将ED与近端策略优化（PPO）相结合。他们将结果称为ED-PPO，并在Brax运动和Craftax上进行测试。在这里，策略输出误差按输出通道路由到隐藏单元。对于标量值网络，误差广播到所有单元。值得注意的是，ED-PPO完全去掉了三项分类创新。在五个种子上，ED-PPO在HalfCheetah上优于BP-PPO（5494 vs 3520；p < 0.001），并且与DFA-PPO持平。在Ant任务上，它与两种PPO变体表现相当。在Craftax上，DFA-PPO反而是最弱的方法（19.8 vs BP-PPO 27.0）。因此，对于分类足够的随机反馈，在开放式强化学习中可能失效。

三个设置使这一点具体化：

提出的方法与其他无反向传播且受生物启发的学习规则对比。“符合Dale”的意思是存在分开的兴奋性/抑制性群体且权重非负。方法名称链接到主要来源。

这些方程可转化为一个紧凑、具有示例性的更新循环：

需要与我们合作以推广您的GitHub仓库、Hugging Face页面、产品发布或网络研讨会等吗？请联系我们：https://forms.gle/wbash1wF6efRj8G58

Asif Razzaq 是 Marktechpost Media Inc. 的首席执行官。作为一个有远见的企业家和工程师，Asif 致力于利用人工智能的潜力为社会带来积极影响。他最近的努力是推出人工智能媒体平台 Marktechpost，该平台以对机器学习和深度学习新闻的深入报道而著称，这些报道既技术上可靠，又易于广大受众理解。该平台每月浏览量超过 200 万次，显示了其在观众中的受欢迎程度。

构建一个自主事件场地运营商 [完整代码]：https://pxllnk.co/twdn5

谢谢！我们的团队会尽快与您联系

Backpropagation dominates deep learning, yet it uses a mechanism the brain likely cannot. Specifically, the backward pass needs exact transposes of forward weight matrices. This is the weight transport problem . Sakana AI’s new paper, Diffusing Blame ：https://arxiv.org/pdf/2606.31700, confronts this constraint directly. The research team trains networks that obey Dale’s principle while avoiding weight transport entirely.

Error Diffusion (ED) is a local learning rule, first proposed by Kaneko (2000). Each weight update depends on three signals only. These are presynaptic activity, a postsynaptic activation derivative, and a single global error sign. Consequently, ED never transports transposed forward weights or uses random feedback matrices. That locality makes ED naturally compatible with Dale’s principle. However, prior work demonstrated ED only on binary classification and MNIST.

To satisfy that constraint, the research team split each layer into two streams. One stream is excitatory ( p ), and the other is inhibitory ( n ). The forward pass computes excitatory-minus-inhibitory preactivations for each stream:

Here, all four weight matrices stay non-negative element-wise. The biases b p and b n are the exception, since they need not be non-negative. Moreover, the negation signs before W np and W pn are structural, not learned. Therefore cross-stream connections remain inhibitory while all learnable weights stay non-negative. This design needs four weight sub-matrices per layer. As a result, it uses roughly 4× more parameters than a single-stream network. For the same architecture, that is ∼32M versus ∼8M for DFA.

With that architecture in place, the main extension is modulo error routing . This lifts Error Diffusion (ED) beyond binary classification. For hidden unit i , the research team define the routing r(i) = i mod C . Here, C is the output dimension. That unit then learns from the routed error component. In short, each hidden unit is assigned one fixed output channel. Unlike DFA, whose feedback matrices are random, ED uses this structured correspondence.

Building on that routing, the research team adds three fixes for multi-class classification:

With all three innovations, Error Diffusion (ED) reaches 96.7% on MNIST and 61.7% on CIFAR-10. In contrast, seed ED without them collapses to 50.4% and 11.6%. DFA scores higher on both tasks but violates Dale’s principle, using ∼2.84M negative weights. Notably, this is the first time ED has trained convolutional networks. Previously, Fujita (2026) reached ∼55.2% on CIFAR-10 using a flattened MLP. Even so, 61.7% remains far from standard gradient-based methods.

Interestingly, the innovations’ importance flips between tasks. On MNIST, removing layer-specific widths is catastrophic (−71.4 pp), collapsing accuracy toward chance. Batch-centering barely matters there (−0.3 pp). On CIFAR-10, however, the order reverses. Removing batch-centered error becomes the largest drop (−47.9 pp), collapsing four of five seeds. This reversal exposes task-dependent credit-assignment bottlenecks invisible to single-benchmark evaluation.

Beyond classification, the research team integrate ED with Proximal Policy Optimization (PPO). They call the result ED-PPO and test it on Brax locomotion and Craftax. Here, policy-output error is routed to hidden units by output channel. For the scalar value network, the error is broadcast to all units. Importantly, ED-PPO drops the three classification innovations entirely. Across five seeds, ED-PPO beats BP-PPO on HalfCheetah (5494 vs 3520; p < 0.001) and matches DFA-PPO. On Ant, it stays on par with both PPO variants. On Craftax, meanwhile, DFA-PPO is the weakest method (19.8 vs BP-PPO 27.0). Thus random feedback that suffices for classification can fail on open-ended RL.

Three settings make this concrete :

Proposed approach vs. other backpropagation-free and biologically motivated learning rules. &ldquo;Dale-compliant&rdquo; means separate excitatory/inhibitory populations with non-negative weights. Method names link to primary sources.

The equations translate into a compact, illustrative update loop:

Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us ：https://forms.gle/wbash1wF6efRj8G58

Asif Razzaq is the CEO of Marktechpost Media Inc.. As a visionary entrepreneur and engineer, Asif is committed to harnessing the potential of Artificial Intelligence for social good. His most recent endeavor is the launch of an Artificial Intelligence Media Platform, Marktechpost, which stands out for its in-depth coverage of machine learning and deep learning news that is both technically sound and easily understandable by a wide audience. The platform boasts of over 2 million monthly views, illustrating its popularity among audiences.

Build an Agentic Event Venue Operator [Full Codes]：https://pxllnk.co/twdn5

Thanks! Our team will contact you soon