智力并非主要瓶颈：AI 时代医学进步的真实制约

编辑：澄清一下，我并不是说旧金山科技圈里的每个人，甚至多数人，都有这种想法。但确实有一些人是这样的（例如开场的对话是逐字的），而这篇帖子讲述的是一个特定的核心圈层，他们的观点在话语权和政策制定中有着不成比例的影响力。

在一次晚宴上，一位来自人工智能实验室的人直接问我，为什么我做这些事——为什么我写关于医学进展的监管瓶颈，为什么我把时间花在与临床试验相关的政策上。他看着我的眼神大概可以总结为怜悯。

我告诉他我的信念：在人工智能时代，医学将比以往任何时候都更容易被监管和临床试验的繁琐程序所限制，而投入数十亿美元加快临床前研究并不能解决这个问题，尽管它不那么吸引人。我提到了住房问题：我们几十年来一直有建造更好住房的技术，但住房成本却比以往任何时候都高，因为住房问题取决于政治意愿，而非纯粹能力。

他难以置信地看着我。肯定吧，一个像我这样聪明的人应该知道，人工智能，或者更确切地说，通用人工智能很快就会极具说服力——在许多基准测试中，它已经超过了专业辩手的说服能力。我说：“嗯。”他说：“是的，是的”，语气中透出一种深入了解事物的神态，那些像我这样不在AI实验室工作的人，仅凭自身根本无法理解。他再次用那种怜悯的目光看着我，就像看着一只稍有智力缺陷却可爱、即将被宰杀的动物。

那天夜里，我辗转反侧，难以入眠，思考我的生活是否有任何意义。是否我曾经做出的每一个决定，从某种意义上讲，都是一个错误。我本可以做得更好吗？

但当太阳升起时，我又回到了同一个想法：无论人工智能变得多么“智能”，智力往往不是现实世界中事物改变的主要瓶颈。当比你聪明的人，掌握特权信息的人，坚信相反观点时，很难保持这种信念。毕竟，这可能只是来自那只天真等待被宰杀的仓鼠的“自我安慰”。

但是，我仍然会坚持我的信念。

因为这些实验室的人们靠押注曾被认为疯狂的想法赚了大钱，现在世界几乎把他们说的每一句话都当作信仰。我认为，这是一个错误。AI会解决大多数人真正关心的问题，包括医学问题，这并不是理所当然的，除非我们清晰地思考这些问题。而目前，我认为我们还没有做到，或者至少没有做到我们本可以达到的程度。我的论点很简单，来自两个方向。一方面是我对旧金山社会动态的观察，以及那里人们谈论这些话题的方式。另一方面是我对一个我恰好有所了解的领域——医学——所注意到的情况。

最常被引用来为AI风险辩护的承诺之一是它将“治愈疾病”。每个主要的AI实验室CEO都这么说：https://www.timesnownews.com/technology-science/sam-altman-says-ai-could-cure-diseases-but-warns-of-serious-risks-article-153916369，投资者似乎也同意：任何附带AI故事的生物技术初创公司都能获得惊人的估值，即使更传统的生物技术公司在资金筹集上苦苦挣扎甚至倒闭。但正如我长期以来所争论的，这整个事业受到许多与“智能”本身关系不大的因素的瓶颈制约，而且这种情况经常被忽视，这种现象令人感到奇怪。

特别是在生物制药领域，一个很大的瓶颈是临床试验：https://www.macroscience.org/p/to-get-more-effective-drugs-we-need?hide_intro_popup=true（我知道我一直在强调这个问题），而如今每种药物的临床试验大约需要七年时间，成本超过十亿美元。而试验并非仅仅是一个形式。它们生成的数据正是最初用来训练更好模型所需要的：人体数据，而这些数据最终是不可替代的：https://www.asimov.press/p/clinic-loop。

我们有大量证据表明，更快的临床试验对生物医学创新非常重要，既有直接作用，也有间接作用，通过二次效应（例如提高在该行业工作者的风险偏好，并通过快速反馈循环更好地对齐激励）。这些证据涵盖了从强有力的经济学论文显示，如果其他条件相同，更短的试验会大幅增加某个疾病领域的投资，到中国的自然实验。中国正威胁要在生物技术领域超越美国，中国的生物技术公司在大型西方制药公司的授权交易中占据了一半以上：https://www.nytimes.com/2026/05/30/business/china-lung-cancer-drugs-asco.html。仅仅十年前，这一比例为零。这一转变发生在中国在基础科学方面仍然较弱的情况下，主要是由于监管改革：https://worksinprogress.co/issue/the-blood-cancer-that-became-solvable/ 允许利用人体数据进行更快速的迭代学习 1：#footnote-1。

临床试验本身可以变得更短、更具信息性和更好，包括利用人工智能。我尤其看好一个领域，即替代终点和生物标志物：https://ifp.org/proxy-praxis-how-surrogate-endpoints-can-speed-drug-development/，人工智能可能将药物是否起效的离散、粗略读数转化为快速、连续的读数。这反过来将极大地优化试验。对于某些适应症，如果找到合适的替代指标，试验速度和成本的提升可能高达10倍：https://ifp.org/proxy-praxis-why-validating-an-endpoint-took-twelve-years/。而这甚至还未考虑单纯优化试验而不必缩短试验时间的好处：想象一下，如果有合适的生物标志物，能够足够早地告诉你治疗是否有效。

但即便在这里，真正的限制因素并不完全是智能。很多时候，是治理问题。我不断与试图建立这些生物标志物的公司交流，他们一次又一次地遇到的困难，是如何获取基础数据。有些公司已经等待了一年，才希望国家卫生研究院（NIH）发布用于生成更好生物标志物的影像数据集。如果需要与美国食品药品监督管理局（FDA）互动以验证终点，那情况就更糟：我以前写过关于骨密度（BMD）作为骨质疏松症试验替代终点的验证过程花了12年的文章：https://ifp.org/proxy-praxis-why-validating-an-endpoint-took-twelve-years/（!），尽管用于支持它的数据已经完全存在，且所做的分析基本上只是回归分析。

当人工智能领域的人涉及医学监管和治理时，这种参与往往显得相当表面化。我多次看到这样的论点：由于大多数药物因缺乏疗效而失败，因此监管无法解释过去几十年我们见证的医疗进展相比基础科学的放缓：毕竟，批准更多无效药物也无济于事。这种分析表面上听起来有道理，但实际上并非如此。正如我一直所说，当人们谈论监管在减缓生物医学进展中的重要性时，他们很少指最终的批准决策。关注的是整个上游过程：人类数据收集的难易程度，以及收集后数据的实际使用难易程度。

我对临床试验最了解，但治理和政策减缓进展的领域并不仅限于此。另一个例子涉及我们的专利制度结构如何影响药物创新性。

在生物医学领域，我们面临一个叫做目标放牧的问题：https://www.nature.com/articles/d41573-023-00063-3：大多数公司追求相同的生物目标，因为它们风险较低，这意味着我们探索的生物领域远远少于我们能探索的范围。最直接的方法是通过风险来判断原因。药物发现中有两种重要性。目标风险反映了生物学上的不确定性：这种蛋白质是否与疾病有关？击中它是否能帮助患者而不会产生不可接受的毒性？分子设计和优化风险是下游问题，取决于靶点：设计针对靶点2：#footnote-2的新型分子，并提升化学物质（如小分子）或生物（如抗体）的效力、选择性、安全性及众多其他特性。这大多属于化学领域，本质上更易处理和可预测;只要有一个经过验证的靶点，优秀的团队通常能达到一个不错的分子。

专利制度进一步抑制了企业承担目标风险的动力，因为它奖励的是新型化学物质，而非新型生物学。物质的组成是大多数专利保护的部分。那指的是特定分子，意味着它们根本无法保护靶点值得下药的洞察。

目标放牧几乎机械地从上述限制中得出：一方面发现更多此类限制确实更难，另一方面专利制度不奖励这些限制。任何验证新靶点（并隐含地发现新生物学）的人都承担着巨大的目标风险，但由于专利制度的设计方式：一旦首创药物3：#footnote-3发布了令人信服的临床数据，靶点就被验证，而验证实际上是公开的。竞争者随后针对同一降低风险的靶点设计不同的分子，每种快速跟随者都能获得强有力的专利。因此，企业最终会不断地针对已被验证的靶点，而不是验证新的靶点，结果我们接连收到针对同一类型生物学的药物浪潮。

一位著名的非营利组织领导人，资助致力于致命疾病分子基础研究的工作，包括利用人工智能寻找新靶点的努力，告诉我，目前的知识产权体系几乎使他支持的机构无法通过发现新的生物学价值来捕获其创造的价值。他很高兴继续资助这项工作，但认为如果这些价值可以被捕获，大家会受益更多，而且现在的激励机制严重失衡。

讽刺的是，那些筹集最大资金的生物学人工智能公司主要针对的是同样已经得到充分关注的分子设计和优化问题。例如，Chai Discovery 目前估值 38 亿美元：https://techfundingnews.com/chai-discovery-triples-to-3-8b-in-7-months-as-openai-backed-drug-designer-lands-400m-after-novartis-deal/，以早期生物技术标准来看，这是巨大的，Isomorphic Labs 也是如此，它们的核心都是分子设计和优化引擎：给定一个靶点，它们能更快、更准确地设计出更好的化合物。这很好，但正如我和其他人以前所论证的那样，拓展经过验证的靶点前沿，从整体上对社会的价值会更高。然而，这正是专利制度没有激励的部分。结果发现，即使是人工智能驱动的公司，最终也会响应经济和法律激励！

我怀疑同样的原则适用于医学之外的领域。扩散是困难的，技术在经济中实际传播的速度，并不完全或主要取决于其基础进展的速度。一项能力存在，并不等同于该能力被机构吸收，更重要的是，将其转化为人们关心的成果。

有大量证据表明情况确实如此。

今天走在世界各地，你可能会注意到奇怪的是世界变化不大。大型语言模型可以生成代理并编写复杂应用程序，但连客户服务这样平凡的工作仍然没有被很好地自动化。事实上，通常最糟糕的体验正来自自动化聊天机器人。国内生产总值在增长，这是好事，但增幅并不显著。尽管反复有“末日”的警告，入门级工作岗位似乎基本保持完整：https://www.ft.com/content/6cb9570b-dccd-46f5-b42a-4d0b7b5de35a?syn-25a6b1a6=1，甚至有所增加。

对许多人来说，这很令人惊讶。就在前几天，我参加了一个会议，有人评论说，如果他几年前能看到今天的人工智能能力，他会感到震惊——并会以为世界到现在应该已经发生了巨大的变化，国内生产总值增长会高得多。这被作为一个新的观察提出。但事实上，这种看法以前就有人提出过，而且对于那些考虑现实世界传播障碍的人来说，整个发展轨迹是相当可预测的。我记得在2023年的一次进步大会上，经济学家泰勒·科恩（Tyler Cowen）在一屋子的强调AGI的人面前提出了完全相同的论点，其中许多人坦率地表示难以置信。他认为，这些能力的到来会比它们旨在产生的变化更快；国内生产总值增长相对于炒作会令人失望，而且不会有大规模的就业流失。到目前为止，数据似乎支持他的观点。

那些预见到这一点的人，只是对技术实际扩散到世界上的缓慢和不均衡有更好的感觉。奇怪的是，在我看来，核心的旧金山科技圈，充满了极其聪明的人，却没有这种感觉。难道这么多聪明人会共享同一个盲点吗？

我的观点是，旧金山的某些社区（今后我将称之为“旧金山人”，虽然我并不是指旧金山的每个人，甚至大多数人）已经接近形成了一种单一文化，而这种单一文化有两个相互强化的特征。第一个是，至少单独来看，提出AGI可能并非完全具有变革性的观点已经真正变得低地位。那些早期意识到人工智能潜力并且正确的人赚了很多钱，而在大事情上正确已经硬化为一种假设：他们在所有事情上都是对的。质疑最极端的说法就会被标记为愚笨、智商低、不够“AGI-pilled”。人们仍然相信，任何在实验室工作的人，尤其是CEO，掌握着外人无法获取的未来秘密。结果是自我审查：我认识的许多旧金山聪明人看到人工智能传播中的瓶颈，却保持沉默，以显得更“AGI-pilled”。

但是这么聪明的人怎么可能会错……关于任何事情？我不认为这有什么奇怪或不同寻常的地方，在一个更智慧的时代，我们会更容易表达这种观点。我以前说过，最优秀的人类与普通人类相差甚远，但上帝与最优秀的人类相比，更是遥不可及。也就是说，无论多么聪明、强大和有经验的人，没有人拥有所有答案。

不仅如此，我认为人工智能竞赛的赢家们可能存在某种错位，这种错位在他们的核心工作——开发AI时可能有所帮助，但在判断其他事情时则可能适得其反。在初创公司中，妄想常常是适应性的：一种不合理的信念正是让小团队尝试不可能任务并偶尔成功的原因。Byrne Hobart 和 Tobias Huber 在《Boom》中提出了这一论点：https://www.awesomebooks.com/book/9781953953476/boom?dwm=5604a29490861e76dde7c8cb231a6a42&msclkid=5604a29490861e76dde7c8cb231a6a42&utm_source=bing&utm_medium=cpc&utm_campaign=Shopping%20%7C%20UK%20%7C%20New&utm_term=4585719407650487&utm_content=ASRUK30001-100000，他们认为金融泡沫通常被认为是破坏性的，但实际上却是突破性进步的引擎：看似集体的妄想，往往是资助和推动真正进步的力量。但这也是双刃剑。能够在生产性妄想中获胜，并不意味着在其他所有事情上都是正确的。实际上，正是导致成功的这一特质，也可能在其他领域中以非生产性的方式膨胀信心。

这种单一文化的第二个特征是一种伪装的逆向思维。由于外部世界长期对AI的潜力持怀疑态度，旧金山的人们感觉自己像勇敢的异议者。尽管如此，在他们自己的圈子里，持有这些观点却是安全且能够提升地位的选择。相信自己是一个被误解的反叛者令人陶醉，同时也使得根本的信念更难被动摇。

当我还是一名学者的时候，我经常注意到，学术界总体上非常进步，但他们认为自己是反对派。这至少部分原因是大多数人来自比大学更保守的地方。他们会聚在一起，把自己与外部世界比较，而不是彼此比较，并得出结论认为自己的进步主义在某种程度上是勇敢的，尽管这只是房间里显而易见的共识。在我看来，科幻小说界在对待人工智能问题上做了同样的事情。在精英科技圈子里，认真对待通用人工智能并不反潮流；如果说有反差的话，反而是别的方向。但由于比较对象总是别人，例如华盛顿特区的政策人员，这种处于围攻中的异议感即使接触现实也依然存在。

我想问题在于，这些事情是否重要。为什么会有坏处呢？据我所知，在科技泡沫中对AGI的疯狂崇拜或许会继续。但它正以一种可能会重新调整优先事项、不利的方式渗透到公众话语中。

在最基本的层面上，我认为更接近经验事实总比远离事实要好。但还有一个更具体的代价：错误的信念可能导致错误的优先事项。鉴于我确实预计人工智能会带来颠覆（尽管不一定以科幻界认为的方式），在这样的时代制定错误的优先事项可能尤其糟糕。

我再次以医学为例：我们的目标或努力可能集中在错误的瓶颈上。在医学领域，正如我所论证的，监管改革得到的关注远小于对AI驱动的生物学的关注，即便在AI驱动的生物学中，真正能够产生显著效果的干预往往是那些资金较少的项目。

这远远超出了医学领域。我们痴迷于AGI之神，而更平凡的问题却被忽视。社会结构正在明目张胆地瓦解。越来越多的年轻人买不起房子，也维持不了一段关系。人们把青春投入到赌博应用中，其中一些应用现在变得更强大，正是由我们不断讨论的人工智能推动的。人们在把更多思考交给机器的过程中技能下降。我有时说，我更担心的不是人工智能会取代我的工作，而是人们会变得太缺乏好奇心甚至完全不去阅读。这些问题每一个都需要人来解决。沉迷于能够神奇地解决我们所有问题的数字神祇，会让人忽视那些已经出现的、更简单、更具人性的麻烦。

米兰·茨维特科维奇指出，目前尚不清楚以加速人体数据为核心的监管改革是否推动了中国的崛起。2015年，中国进行了一系列政策改革，包括扩大覆盖范围、清理一些监管决策积压等，很难说哪些改革最具影响力。本文认为：https://www.nber.org/papers/w34977 是覆盖范围的扩大。我在这篇X帖中解释了：https://x.com/ruxandrateslo/status/2067027090894098806?s=46 为什么我在某种程度上怀疑那篇文章。无论如何，我认为大家都同意，加快人体数据收集是中国实力的一个重要组成部分（无论是否由监管改革引起），同时中国生物技术崛起的直接原因也是政策变化。你可以在这里看到我与米兰的交流：https://x.com/RuxandraTeslo/status/2079682745840570880?s=20。

实际上，这略微复杂一些，因为并非所有药物都是分子。有些是细胞或基因疗法。然而，即便如此，广泛原则仍然适用。针对已知靶点改进T细胞疗法本质上风险较低。

首创药物往往不是商业上“获胜”的药物，因为追随者通常能更好地优化化学结构，并从首创药物的不足中学习。所以，开创新方法的人很难获得价值。

非常有用的文章。这是我长期尝试强调的一点：许多问题并非靠智慧就能解决！

太棒了！我不知道你是如何将临床试验的难度和旧金山的文化融入到一个连贯的论点中的，但我真的很喜欢阅读这篇文章。

Edit: To be clear, I'm not saying everyone or even a majority of people in the SF tech scene think this way. But some certainly do (e.g. the opening conversation is verbatim) and this post is about a particular core scene whose views carry disproportionate weight in the discourse and in policymaking.

At a recent dinner, someone from an AI lab asked me, quite bluntly , why I do what I do — why I write about the regulatory bottlenecks to medical progress, why I spend my time on policies related to clinical trials. He looked at me with something best summarized as pity.

I told him what I believe: that in the age of AI, medicine will be bottlenecked more than ever by regulation and the grind of clinical trials, and that billions poured into faster pre-clinical research won’t touch that problem, unsexy as it is. I brought up housing: we’ve had the technology to build better housing for decades, yet it’s more expensive than ever, because housing is a question of political will, not pure capability.

He looked at me incredulously. Surely, a smart person like me should know that AI, or better said, AGI will be hyperpersuasive soon – already on a bunch of benchmarks it exceeds professional debaters at persuasion. I said, “Hmm.” He said, “Yes, yes”, with the undertone of a man who knows things deeply, things that mere mortals like me, not being AI lab employees, are simply not well placed to grasp. And he looked at me again with that sense of pity, the way one looks at a slightly mentally impaired but cute animal awaiting its imminent slaughter.

That night I lay awake, twisting and turning, wondering whether my life had any point at all. Whether every decision I’d ever made had been, in some way, a mistake. What could I have done better?

But as the sun came up, I found my way back to the same thought: no matter how “intelligent” AI becomes, intelligence is often not the main bottleneck to things changing in the real world. It is hard to hold on to that conviction when people smarter than you, with access to privileged information, insist otherwise. After all, this could all just be “cope” from the naive hamster awaiting its slaughter.

But, I shall nonetheless stick to my beliefs.

Because the people at these labs made fortunes betting on ideas that once looked insane, the world now takes nearly everything they say on faith. That, I think, is a mistake. It is not a given that AI will solve the problems most people actually care about, including medicine, unless we think about those problems clearly. And at the moment, I don’t believe we are or at least, not to the extent that we could. My case is simple, and it comes from two directions. One is what I observe in the social dynamics of San Francisco, and in how people there talk about all this. The other is what I notice in a field I happen to know something about: medicine.

One of the promises most often invoked to justify AI’s risks is that it will “cure disease.” Every major AI lab CEO says it ：https://www.timesnownews.com/technology-science/sam-altman-says-ai-could-cure-diseases-but-warns-of-serious-risks-article-153916369 , and investors seems to agree: any biotech startup with an AI story attached commands an impressive valuation, even as more conventional biotechs struggle for funding and die. But this whole enterprise, as I have long argued, is bottlenecked by many things that have little to do with “intelligence” as such, and the degree to which that often goes unacknowledged is strange to watch.

In biopharma specifically, one great bottleneck is clinical trials ：https://www.macroscience.org/p/to-get-more-effective-drugs-we-need?hide_intro_popup=true (I know I keep banging on about this), which today consume something like seven years and cost more than a billion dollars per drug. And trials are not a mere formality. They generate precisely the kind of data that would train better models in the first place: human data, which is ultimately irreplaceable ：https://www.asimov.press/p/clinic-loop .

We have extensive evidence faster clinical trials are important for biomedical innovation, both directly and indirectly, through second-order effects (like increasing the risk appetite of those working in the industry and helping better align incentive through fast feedback loops). This ranges from robust economic papers showing shorter trials massively boost investment in a disease area, all else equal, to the natural experiment of China. China is threatening to race ahead of US in biotech, with Chinese biotechs encompassing more than a half of big Western pharma ：https://www.nytimes.com/2026/05/30/business/china-lung-cancer-drugs-asco.html licensing deals. A mere decade ago, this percentage was zero. This transformation happened while China remains worse in terms of basic science, mostly due to regulatory reforms ：https://worksinprogress.co/issue/the-blood-cancer-that-became-solvable/ that allow faster iterative learning using in-human data 1：#footnote-1 .

Trials themselves can be made shorter, more informative and better, including with AI. One area I am particularly bullish on is surrogate endpoints and biomarkers ：https://ifp.org/proxy-praxis-how-surrogate-endpoints-can-speed-drug-development/ , where AI could turn discrete, coarse readouts of whether a drug is working into fast, continuous ones. This would in turn optimize trials immensely. For some indications, the gain could be as high as 10x faster and cheaper trials ：https://ifp.org/proxy-praxis-why-validating-an-endpoint-took-twelve-years/ if the right surrogates are found. And this is not even considering the benefits of simply optimizing the trials without necessarily shortening them: imagine having the right biomarker that tells you whether a therapy is working early enough.

But even here the binding constraint is not entirely intelligence. Quite often, it is governance. I keep talking to companies trying to build exactly these biomarkers, and what they run into, again and again, is how hard it is to access the underlying data. Some have been waiting for a year for the NIH to release imaging datasets they can use to produce better biomarkers. If one needs to interact with the FDA to get their endpoint validated, it is even worse: I have previously written about how the validation of Bone Mineral Density (BMD) for use as a surrogate endpoint in osteoporosis trials took 12 years ：https://ifp.org/proxy-praxis-why-validating-an-endpoint-took-twelve-years/ (!), despite the fact that the data to support it already existed in full and the analyses done were basically regressions.

When people in the AI sphere do engage with regulation and governance in medicine, it often seems to happen in a rather superficial way. I have seen many times that argument that because most drugs fail for lack of efficacy, regulation can’t explain much of the slow-down in medical progress compared to basic science that we have witnessed in the last decades: after all, approving more inefficacious drugs won’t help. This analysis sounds superficially true, but it’s not. As I keep saying, when people talk about the importance of regulation in slowing down biomedical progress, they rarely mean the approval decision at the end. It is about the entire process upstream: how easily human data can be collected, and then how easily, once collected, it can actually be used.

I know most about clinical trials, but this is not the only area where governance and policy slows down progress. Another example is related to how the structure of our patent system shapes how innovative our drugs are.

In biomedicine we have a problem called target herding ：https://www.nature.com/articles/d41573-023-00063-3 : most companies chase the same biological targets because they are de-risked, which means we explore far less of the biological space than we could. The cleanest way to see why is through risk. Two kinds matter in drug discovery. Target risk captures the biological uncertainty: is this protein causally involved in the disease, and will hitting it help a patient without unacceptable toxicity? Molecule design and optimization risk is the downstream problem, conditional on the target: designing a novel molecule against a target 2：#footnote-2 and improving potency, selectivity, safety and a myriad of other features of a chemical (e.g. small molecule) or biological (e.g. antibody). That is mostly in the realm of chemistry and is intrinsically far more tractable and predictable; given a validated target, good teams usually reach a decent molecule.

The patent system then further disincentivizes companies from taking target risk, because it rewards novel chemical matter, not novel biology. Composition of matter is what most patents protect. That refers to a specific molecule, which means that they don’t protect the insight that a target is worth drugging at all.

Target herding then follows almost mechanically from the above mentioned constraints: on one hand the fact that is genuinely harder to uncover more of these and then that the patent system does not reward it. Whoever validates a novel target (and implicitly, discovers new biology) bears the enormous target risk but can’t capture the reward due to the way the patent system is designed: once a first-in-class drug 3：#footnote-3 posts convincing clinical data, the target is validated, and that validation is essentially public. Competitors then design distinct molecules against the same de-risked target, and each fast-follower earns its own strong patent. So firms end up piling onto proven targets instead of validating new ones, so we get wave after wave of drugs aimed at the same type of biology.

A well-known non-profit leader who funds work on the molecular basis of deadly diseases, including AI-driven efforts to find novel targets, told me the current IP system makes it nearly impossible for the institution he supports to capture the value it creates by uncovering new biology. He's glad to keep funding it regardless, but believes everyone would be better served if that value were capturable, and that today's incentives are badly skewed.

The irony is that the AI-for-biology companies drawing the largest rounds are aimed mostly at the same well-served problem of molecule design and optimization. For example, Chai Discovery, now valued at $3.8 billion ：https://techfundingnews.com/chai-discovery-triples-to-3-8b-in-7-months-as-openai-backed-drug-designer-lands-400m-after-novartis-deal/ , enormous by early-stage biotech standards, and Isomorphic Labs are both, at heart, at molecule design and optimization engines: they design better compounds faster and with more accuracy, given a target. This is great, but as I and others have argued before, expanding the frontier of validated targets would be worth more overall to society. Yet this is precisely the part that the patent system does not incentivize. It turns out even AI-driven companies ultimately respond to economic and legal incentives!

The same principle applies, I suspect, well beyond medicine. Diffusion is hard, and the rate at which a technology actually spreads through an economy is not entirely or even mainly predicted by the raw pace of its underlying progress. A capability existing is not the same as a capability being absorbed into institutions, and, perhaps most important of all, translated into outcomes people care about.

And there is expansive evidence this is the case.

Walking around the world today one might notice that it is weirdly unchanged. LLMs can spawn agents and write complex applications, yet something as mundane as customer service still hasn’t been automated well. In fact, often it’s the automated chatbots that are the worst part of the experience. GDP is up, which is good, but it is not dramatically up. And entry-level jobs, despite repeated warnings of an apocalypse, seem largely intact ：https://www.ft.com/content/6cb9570b-dccd-46f5-b42a-4d0b7b5de35a?syn-25a6b1a6=1 , if not increasing.

To many, this is surprising. Just the other day I was at a conference where someone remarked that if he could have seen today’s AI capabilities a few years ago, he would have been astonished — and would have assumed the world by now would look far more transformed, with much higher GDP growth. It was offered as a fresh observation. But in fact, it has been made before, and this whole trajectory was quite predictable to those thinking about real world barriers to diffusion. I remember a Progress Conference in 2023 where the economist Tyler Cowen made exactly this argument to a room full of AGI-pilled attendees, many of whom were frankly incredulous. The capabilities, he suggested, would arrive faster than the changes they were meant to produce; GDP growth would disappoint relative to the hype, and there would be no massive job displacement. So far, the data seem to bear him out.

The people who saw this coming simply have a better feel for friction for how slowly and unevenly a technology actually diffuses into the world. Strangely, it seems to me that the core San Francisco tech scene, full of exceptionally intelligent people, does not. How can so many smart people share this one blind spot?

My view is that certain communities in San Francisco (which I shall henceforth call “SF people”, even though I do not mean literally everyone or even most people in SF) have become something close to a monoculture, and this monoculture has two reinforcing features. The first is that it has become genuinely low-status to suggest AGI might not be wholly transformative, at least not on its own. The people who were early and right about AI’s potential made a great deal of money, and being right about the big thing has hardened into an assumption that they are right about everything. To question the most extreme claims is to be marked as dim, low-IQ, insufficiently “AGI-pilled.” The belief persists that anyone who works at a lab, specially a CEO, holds secrets about the future that no outsider can access. The result is self-censorship: a lot of smart people I know in San Francisco see the bottlenecks to AI’s diffusion and say nothing to seem more “AGI-pilled”.

But how could such smart people be wrong… About anything? I do not think there is anything weird or unusual about that and in a wiser era we would find it easier to articulate. I have said before that the best human is very far from the average human, but God is much further way from the best human still. That is to say that nobody, no matter how smart, powerful and experienced, has all the answers.

Not only that, but I think the winners of the AI race might be miscalibrated in a way that might have helped in their core endeavour, developing AI, but maladaptive when it comes to judging things beyond that. Delusion is often adaptive in startups: an unreasonable conviction is exactly what lets a small group attempt the impossible and occasionally pull it off. Byrne Hobart and Tobias Huber make a version of this case in Boom ：https://www.awesomebooks.com/book/9781953953476/boom?dwm=5604a29490861e76dde7c8cb231a6a42&msclkid=5604a29490861e76dde7c8cb231a6a42&utm_source=bing&utm_medium=cpc&utm_campaign=Shopping%20%7C%20UK%20%7C%20New&utm_term=4585719407650487&utm_content=ASRUK30001-100000 , arguing that financial bubbles, usually maligned as destructive, have in fact been an engine of breakthrough progress: what looks like collective delusion is often what funds and forces genuine advance. But that cuts both ways. Being productively delusional and winning does not make one right about everything else. In fact, the very trait that produced the success also might inflate confidence in unproductive ways in other areas.

The second feature of this monoculture is a kind of counterfeit contrarianism. Because most of the outside world has long been skeptical of AI’s potential, people in San Francisco feel like brave dissenters. This is despite the fact that, within their own circles, holding those views is the safe and status-conferring position. Believing yourself a misunderstood contrarian is intoxicating, and it makes the underlying conviction harder to dislodge.

Back when I was an academic, I often noticed that academics were, on the whole, deeply progressive, yet thought of themselves as contrarians. This was at least in part because most had come from places more conservative than the university. They’d gather, compare themselves to the outside world rather than to each other, and conclude that their progressivism was somehow brave, despite it being the plain consensus of the room. In my view, SF has done the same thing with AI. In elite tech circles it is simply not contrarian to take AGI seriously; if anything the reverse. But because the comparison class is always someone else: for example, a policy person in DC, the feeling of embattled dissent survives contact with reality.

The question, I suppose, is whether any of this matters. Why would it be bad? For all I know, the AGI God frenzy should continue in the tech bubble. But it’s seeping into general public discourse in a way that might realign priorities in a way that is bad.

At the most basic level, I think being closer to the empirical truth is simply better than being further from it. But there is a more concrete cost: mistaken beliefs can lead to mistaken priorities. Given that I do expect AI to be disruptive (albeit not necessarily in the same ways the SF crowd thinks), this might be a particularly bad times to have bad priorities.

Again, I shall bring the example of medicine: we might be aiming or effort at the wrong bottlenecks. In medicine, as I’ve argued, regulatory reform gets a tiny fraction of the attention lavished on AI-enabled biology and even within AI-enabled biology, the interventions that would actually move the needle are the ones that often seem less funded.

And this goes well beyond medicine. We fixate on AGI gods, while more mundane problems pile up unattended. The social fabric is fraying in plain sight. More and more young adults cannot afford a house or sustain a relationship. People pour their youth into gambling apps, some of them now made more powerful by the very AI we keep discussing. People are de-skilling as they hand more thinking to machines. I sometimes say I worry less that AI will take my job than that people will grow too incurious to read at all. Every one of these problems needs people working on it. Being in the thrall of digital deities that will magically dissolve all our problems pulls attention away from the simpler, more human troubles that are already here.

Milan Cvitkovic pointed out that it is not clear that the regulatory reforms around faster in-human data spearheaded China’s rise. In 2015 China did a bunch of policy reforms that included expansion of coverage, clearing up some regulatory decision backlog and others and it is hard to say which ones were the most consequential. This article argues：https://www.nber.org/papers/w34977 it was the coverage expansion. I explained in this X post：https://x.com/ruxandrateslo/status/2067027090894098806?s=46 why I doubt that article to some extent. Either way, I think everyone agrees that faster in-human data is a large component of China’s strength (whether caused by regulatory reforms or not) and also that the proximal cause of China’s rise in biotech was policy change. You can find my exchange with Milan here：https://x.com/RuxandraTeslo/status/2079682745840570880?s=20 .

This is in practice a bit more complicated, as not all drugs are molecules. Some are cells or gene therapies. Nonetheless, even here, the broad principles hold. Improving a T cell therapy against a known target is inherently less risky.

First-in-class drugs are often not the drugs that “win” commercially, as followers usually manage to optimize the chemistry better and learn from deficiencies from the first-in-class. So it’s hard for someone who pioneers a new approach to capture value.

Very useful piece. This is a point I've been trying to make for a long while: many problems are not solvable by intelligence!

Fantastic piece! I don't know how you managed to fit in the difficulty of clinical trials and the culture of San Francisco into one coherent argument, but I really enjoyed reading this.