Towards a quantum computer that learns from its errors

沃洛迪米尔·西瓦克 和 保罗·克利莫夫，谷歌量子人工智能研究科学家，谷歌研究院

通过将强化学习与量子纠错结合，我们展示了量子计算机能够持续适应漂移并在长时间计算中保持稳定。

想象一个交响乐团正在演奏一部复杂的杰作。如果小提琴每几小节就跑音，乐团将不得不停下来重新调音。幸运的是，乐团中情况并非如此，因为乐器能够可靠地保持调音。然而，这正是当前操作量子计算机的现实。

由于量子计算机本质上是易受漂移影响的模拟机器，保持可靠运行需要不断重新校准它们的控制参数，也就是控制量子比特的模拟信号的频率、振幅和相位。如今，这需要完全终止整个量子计算。这种计算与校准的完全脱节代表了未来的一个根本瓶颈，因为有用的量子算法必须连续运行数天甚至数月。

为了解决这个问题，在发表于《自然》杂志（Nature：https://www.nature.com/）的论文《量子纠错的强化学习控制：https://www.nature.com/articles/s41586-026-10759-2》中，我们展示了一个强化学习（RL）框架：https://en.wikipedia.org/wiki/Reinforcement_learning，其中一个自主智能体通过量子错误检测学习，持续控制数千个参数，在计算过程中稳定量子系统以抵抗漂移。简而言之：我们找到了在演奏音乐时调音的方法。

在音乐厅中，一个跑音的乐器会立即被察觉。量子领域没有这种幸运。就像听本身就会破坏演出一样，测量量子比特会塌缩它们的量子叠加状态。为了保留量子信息，我们采用量子纠错（QEC）：https://research.google/blog/making-quantum-error-correction-work/，这是一项利用冗余将许多物理量子比特组合成“逻辑量子比特”的技术，并在物理量子比特上使用专门的奇偶校验，将模拟噪声数字化为二进制错误检测事件。

不幸的是，这些比特仅仅告诉我们在量子电路的某个有界时空区域内发生了错误，而无法指明其确切位置。这就像听到一小段跑调的音符，却不知道是哪位乐手演奏的。为了确定可能的错误位置并计算所需的纠正，我们依赖量子纠错解码器（QEC decoders），例如神经网络解码器 AlphaQubit：https://blog.google/innovation-and-ai/models-and-research/google-deepmind/alphaqubit-quantum-error-correction/（基于真实数据训练）和算法解码器 Tesseract：https://arxiv.org/html/2503.10988v1。如果错误足够稀少，这些解码器可以通过分析错误检测数据成功恢复逻辑量子信息。然而，解码器留下了一个关键问题尚无答案：这些错误最初为什么会发生？

部分错误源于量子系统与周围环境不可避免的相互作用，导致退相干现象：https://physicstoday.aip.org/features/decoherence-and-the-transition-from-quantum-to-classical。这个无情的过程会破坏宏观量子叠加态，实际上将量子计算机转变为经典计算机。这一基本现象如此普遍，以至于导致我们熟悉的经典现实从自然的量子规律中出现。虽然这些环境引起的错误永远无法完全避免，但许多其他错误是由于控制校准不精确和硬件漂移造成的——这些缺陷仍然在我们可控范围内，可以减轻。

传统上，量子校准依赖物理模型。其技术通过数十年的量子控制研究不断完善。然而，在各个技术领域，人类设计的模型不可避免地会遇到性能天花板。例如，早期的计算机视觉在依赖严格几何规则时陷入停滞：https://en.wikipedia.org/wiki/ImageNet#History_of_the_ImageNet_challenge。传统机器人学：https://arxiv.org/abs/2506.13498 仍然在处理运动学方程时遇到困难，因为这些方程无法捕捉接触动力学和摩擦的复杂现实。同样，传统物理模型几十年来难以解决的蛋白质折叠预测问题，直到像 AlphaFold：https://deepmind.google/blog/alphafold-five-years-of-impact/ 这样深度学习系统的出现，才实现了前所未有的精度。在这些领域中，当方法转向直接从数据中学习时，才出现了新的突破。

最近，AlphaQubit：https://blog.google/innovation-and-ai/models-and-research/google-deepmind/alphaqubit-quantum-error-correction/ 超过了最强大的算法量子纠错（QEC）解码器的精度。如今，量子控制也面临同样的天花板。随着量子处理器通过制造和硬件的进步不断改进，其误差越来越多地由传统建模和校准难以处理的复杂现象主导。机器学习能带来新的进展吗？

谷歌研究部门在利用强化学习（RL）解决传统编程无法处理的复杂问题方面有着丰富的先驱历史。与依赖明确指令的算法不同，RL 通过经验运作。一个自主智能体会测试不同的行为，并直接从由此产生的错误中学习，从而完善策略。将 RL 应用于实现精确、连续的量子校准几乎是不可避免的。由于量子纠错（QEC）已经生成了一连串稳定的检测事件，我们只是赋予这些数据一个互补的角色。除了对其进行解码和纠正误差外，我们还将检测事件作为主动学习信号。随着计算的进行，RL 智能体监控这些数据，并学习动态调整控制参数，以抵消漂移并防止新的错误出现。

量子计算通过模拟控制信号实现物理操作。量子误差检测事件被解码器用于推断逻辑校正。在我们的控制框架中，这些事件也被重新利用作为学习信号，教导强化学习（RL）代理在计算过程中持续引导数千个控制参数并稳定量子系统。

我们在旗舰级 Willow：https://blog.google/innovation-and-ai/technology/research/google-willow-quantum-chip/ 超导处理器上验证了这种 RL 量子控制。通过刻意注入控制参数的人工漂移，我们显示 RL 引导提高了我们纠错码的逻辑稳定性 3.5 倍，延长了处理器作为可靠“量子存储”设备的时间。

通常，将处理器调至最佳性能严重依赖“人类在环”方法，科学家运用物理直觉来解决难以自动化的边缘情况。然而，即使在经过这种详尽的专家校准之后，后续的 RL 微调仍系统性地将逻辑错误率额外降低了 20%。

在本实验中，所有技术的综合应用将量子存储中的逻辑错误降至历史最低：在表面码中每千个纠错周期少于一次误差：https://research.google/blog/making-quantum-error-correction-work/，在彩色码中每一百个周期约一次误差：https://research.google/blog/a-colorful-quantum-future/。

量子误差校正（QEC）延长了基于表面码（左）和彩色码（右）的量子存储中逻辑信息的寿命。通过 RL 微调我们的控制器，通过规避复杂物理模型和人类直觉的局限性，从而提高了 QEC 的质量。

机器学习和量子研究人员面临的一个关键问题是，这种 RL 方法是否可以扩展到未来的大型量子计算机。为测试这一点，我们进行了数百量子比特和数万控制参数的数值模拟。

在我们的模拟中，一个校准错误的系统从高物理误差率开始。代理通过学习改进的控制参数，随着时间的推移稳步降低物理误差率，并且由于量子纠错（QEC），逻辑误差率（LER）随系统规模（即物理量子比特的数量）呈指数级下降。关键是，强化学习（RL）降低物理误差率的速度与系统规模无关，这使我们能够将这种方法扩展到更大的系统。

模拟结果证实了我们的预期：所需的RL训练迭代次数（epoch）与系统规模无关，这是因为QEC检测事件对误差的局部敏感性。然而，要充分实现RL框架的潜力，需要更紧密的集成。通过加快代理与量子处理器之间的通信周期，并采用更先进的机器学习方法，我们希望能够释放显著的额外改进空间。

因此，我们的工作建立了一种新范式：能够从其错误中学习且不会停止计算的量子计算机。

我们感谢合著者的贡献，包括建设和维护硬件、软件、低温技术及电子基础设施。这项工作得以实现，得益于谷歌研究院（Google Research）谷歌量子AI团队（Google Quantum AI team），并与谷歌DeepMind团队合作完成。

Volodymyr Sivak and Paul Klimov, Research Scientists, Google Quantum AI, Google Research

By integrating reinforcement learning with quantum error correction, we showed that a quantum computer can continuously adapt to drift and remain stable during long computations.

Imagine a symphony orchestra performing a complex masterpiece. If the violins drifted out of tune every few measures, the ensemble would constantly have to stop and retune their instruments. Thankfully, this doesn't happen in an orchestra because the instruments reliably stay in tune. However, it is the current reality of operating a quantum computer.

Since quantum computers are fundamentally analog machines that are sensitive to drift, maintaining reliable operation requires perpetually recalibrating their control parameters, i.e., the frequencies, amplitudes, and phases of the analog signals choreographing the qubits. Today, this requires fully terminating the entire quantum computation. This complete decoupling of computation and calibration represents a fundamental bottleneck for the future, as useful quantum algorithms must run continuously for days or even months.

To address this, in “Reinforcement learning control of quantum error correction：https://www.nature.com/articles/s41586-026-10759-2”, published in Nature ：https://www.nature.com/, we demonstrated a reinforcement learning：https://en.wikipedia.org/wiki/Reinforcement_learning (RL) framework in which an autonomous agent learns from quantum error detections to continuously steer thousands of control parameters, stabilizing the quantum system against drift during the computation. In short: we found a way to tune the instruments while the music plays.

In a concert hall, a detuned instrument is immediately heard. The quantum realm offers no such luxury. As if the very act of listening ruined the performance, measuring the qubits collapses their quantum superposition states. To preserve the quantum information, we instead employ Quantum Error Correction：https://research.google/blog/making-quantum-error-correction-work/ (QEC), a technique that exploits redundancy to create “logical qubits” out of many physical qubits, and uses specialized parity checks on the physical qubits to digitize the analog noise into binary error detection events.

Unfortunately, these bits only tell us that an error occurred somewhere within a bounded spacetime region of the quantum circuit, not its exact location. It is like hearing a sour note without knowing exactly which musician played it. To pinpoint the likely error locations and calculate the necessary corrections, we rely on QEC decoders, such as the neural network decoder AlphaQubit：https://blog.google/innovation-and-ai/models-and-research/google-deepmind/alphaqubit-quantum-error-correction/ (trained on real data) and algorithmic decoder Tesseract：https://arxiv.org/html/2503.10988v1. If errors are sufficiently rare, these decoders can successfully restore the logical quantum information by analyzing the error detection data. However, decoders leave a crucial question unanswered: why did those errors happen in the first place?

Some errors result from the unavoidable interaction of a quantum system with its surrounding environment, leading to decoherence：https://physicstoday.aip.org/features/decoherence-and-the-transition-from-quantum-to-classical. This ruthless process destroys macroscopic quantum superpositions, effectively turning quantum computers into classical ones. This fundamental phenomenon is so pervasive that it causes our familiar classical reality to emerge from the underlying quantum laws of Nature. While these environmental errors can never be completely prevented, many others are manifestations of imprecise control calibration and hardware drift – flaws that remain within our power to mitigate.

Traditionally, quantum calibration relied on physics models. Its techniques were refined through decades of quantum control research. However, across technological domains, human-crafted models inevitably hit a performance ceiling. Early computer vision stalled：https://en.wikipedia.org/wiki/ImageNet#History_of_the_ImageNet_challenge when relying on strict geometric rules. Traditional robotics：https://arxiv.org/abs/2506.13498 still struggles with kinematic equations that fail to capture the messy reality of contact dynamics and friction. Similarly, the decades-old challenge of predicting protein folding remained largely intractable for traditional physical models until deep learning systems like AlphaFold：https://deepmind.google/blog/alphafold-five-years-of-impact/ achieved unprecedented accuracy. Across these fields, new breakthroughs occurred when the approach shifted toward learning directly from data.

Recently, AlphaQubit：https://blog.google/innovation-and-ai/models-and-research/google-deepmind/alphaqubit-quantum-error-correction/ surpassed the accuracy of the most powerful algorithmic QEC decoders. Now quantum control faces the same ceiling. As quantum processors improve through progress in fabrication and hardware, their errors become dominated by complex phenomena that are challenging for traditional modeling and calibration. Can machine learning bring new advances?

Google Research has a rich history of pioneering RL to solve problems too complex for traditional programming. Unlike algorithms relying on explicit instructions, RL operates through experience. An autonomous agent tests different behaviors and learns directly from resulting errors to refine its strategy. Applying RL to achieve accurate, continuous quantum calibration felt almost inevitable. Since QEC already generates a steady stream of detection events, we simply granted this data a complementary role. In addition to decoding it and correcting the errors, we employ the detection events as an active learning signal. As computation progresses, the RL agent monitors this data and learns to dynamically steer the control parameters, counteracting drift and preventing new errors.

Quantum computation is physically realized via analog control signals. The quantum error detection events are used by the decoder to infer logical corrections. In our control framework, they are also repurposed as a learning signal, teaching the RL agent to continuously steer thousands of control parameters and stabilize the quantum system during the computation.

We validated such RL quantum control on our flagship Willow：https://blog.google/innovation-and-ai/technology/research/google-willow-quantum-chip/ superconducting processor. By deliberately injecting artificial drift of control parameters, we showed that RL steering improved the logical stability of our error-correcting code 3.5-fold, prolonging the time during which the processor acts as a reliable “quantum memory” device.

Typically, tuning the processor to peak performance relied heavily on a “human-in-the-loop” approach, with scientists applying physical intuition to resolve edge cases that are difficult to automate. Yet, even after this exhaustive expert calibration, subsequent RL fine-tuning systematically suppressed the logical error rate by an additional 20%.

The synthesis of all our technologies in this experiment reduced the logical errors in quantum memories to a record low: fewer than one per thousand error correction cycles in the surface code：https://research.google/blog/making-quantum-error-correction-work/, and one per hundred in the color code：https://research.google/blog/a-colorful-quantum-future/.

QEC prolongs the lifetime of logical information in quantum memories based on the surface code ( left ) and color code ( right ). RL fine-tuning of our controller improves the quality of QEC by circumventing the limitations of complex physics models and human intuition.

A critical question from machine learning and quantum researchers is whether this RL approach can scale to large quantum computers of the future. To test this, we conducted numerical simulations with hundreds of qubits and tens of thousands of control parameters.

In our simulation, a miscalibrated system starts at a high physical error rate. The agent steadily reduces it over time by learning improved control parameters, and thanks to QEC the logical error rate (LER) gets suppressed exponentially in system size (i.e. number of physical qubits). Crucially, the speed at which physical error rate is reduced by RL is independent of this size, which allows us to scale this approach to much larger systems.

The simulations confirmed our expectation: the number of required RL training iterations (epochs) is independent of the system size, owing to the local sensitivity of the QEC detection events to errors. However, realizing the full potential of the RL framework requires tighter integration. By speeding up the communication cycle between the agent and the quantum processor, and employing more advanced machine learning methods, we hope to unlock significant additional improvements.

Our work thus enables a new paradigm: a quantum computer that learns from its errors and doesn’t stop computing.

We thank our co-authors for their contributions, including building and maintaining the hardware, software, cryogenics and electronics infrastructure. This work was made possible by the Google Quantum AI team at Google Research, in collaboration with teams from Google DeepMind.