Skyfall AI 发布 MORPHEUS：面向持续强化学习的持久企业模拟基准

Skyfall AI 推出 MORPHEUS，一个面向持续强化学习（CRL）的持久企业模拟平台。

与每次回合后重置的传统基准不同，MORPHEUS 构建永不重置的持久世界，通过故障注入引擎（11种故障类型，4种预设速率）和异步配置漂移控制器引入非平稳性。

平台采用两阶段策略：先用前沿模型 Gemini 3.1 pro 基于 ReAct 框架收集轨迹，再通过监督微调（SFT）训练 Qwen3-14B，随后所有基线使用 PPO 进行在线后训练。

评估协议包含六个指标（适应速度、遗忘、恢复时间等）。

在动态资源分配和延迟调度两项任务上，EWC、HER、LCM 等算法族表现各异，无单一方法全面占优。

代码已开源。

Most reinforcement learning benchmarks reset the world after every episode. Real operations never reset. Skyfall AI’s MORPHEUS targets that gap. It is a persistent enterprise simulation platform for continual reinforcement learning (CRL).

MORPHEUS is grounded in the Big World Hypothesis (Javed & Sutton, 2024). It says the world’s complexity exceeds any agent’s representational capacity. As a result, the environment looks non-stationary even under fixed dynamics.

To force continual learning, MORPHEUS requires three properties: persistence, non-stationarity, and operational complexity. Persistence means past decisions compound into future dynamics. Non-stationarity means any fixed policy eventually becomes suboptimal. Operational complexity means no fixed optimal policy exists.

Each environment is a self-contained TypeScript world plugin. It exports Operational Descriptors (ODs), a simulation scheduler, seed data, and documentation. An OD defines the step-by-step execution plan for a capability. Agents act through a capability API, and each call triggers an OD execution.

Building on that architecture, non-stationarity comes from two engines. First, a failure injection engine inserts typed disruptions between OD steps. It draws from eleven failure types, including missing_data , dependency_failure , and rate_limit . It runs at four preset rates: light (5%), realistic (8%), moderate (15%), and aggressive (30%).

Second, an asynchronous configuration shift controller changes failure presets and demand at fixed timestamps. It runs independently of the training loop, so shifts never align with gradient updates. This stops the agent from using update periodicity as a proxy clock.

Alongside these engines, reward comes from three operational verifiers logged natively by the platform. These are failure event signals, financial ledger status, and resource throughput. The composite reward combines them. Default weights are w_f = 0.5 and w_l = w_p = 0.25 .

Under the upper-bound assumptions (zero failures, minimum cost, full throughput), the bound per configuration equals 0.50.

Because the action space is large, pure RL from scratch is impractical. Therefore MORPHEUS uses a two-stage pipeline. A frontier model (Gemini 3.1 pro) collects trajectories using the ReAct framework. These traces then fine-tune Qwen3-14B via supervised fine-tuning (SFT).

Consequently, every RL run starts from this shared SFT checkpoint. This isolates continual learning behaviour from basic operational competence. All baselines then use PPO as the base optimizer for online post-training.

With training defined, cumulative reward alone is not enough. A scalar sum hides performance across a non-stationary horizon. So the research team propose six metrics instead. These are per-configuration reward, adaptation speed, forgetting, recovery time, stability, and performance gap.

Among these, adaptation speed is the headline metric. It counts steps until the running-average reward reaches half the upper bound. Two supplementary diagnostics also track relative adaptation advantage (RAA) and plasticity via effective rank.

Using this protocol, the research team tests four algorithm families from the shared SFT checkpoint. Two tasks are defined. Task 1 is dynamic resource allocation under structured drift. Task 2 is scheduling under drift with delayed effects.

Across these results, no single family dominates. On process-outbound Task 1, EWC leads reward and LCM adapts fastest. On Task 2, HER leads reward while LCM loses its edge under delayed reward. Meanwhile, mean performance gaps sit near 1.0 for every method. That signals a large settled-state deficit, not a minor tuning gap.

Notably, PPO and HER generally adapt only in the first configuration. They then fail to adapt in later regimes, even without label signals.

In practice, MORPHEUS suits several reader roles. For AI engineers, it tests whether an agent detects regime shifts without labels. For example, demand switches from low to bursty, and the policy must adapt with no signal.

For data scientists, it stresses delayed credit assignment. For example, On-Time In-Full (OTIF) delivery is observable only days after the dispatch decision. For software engineers, the TypeScript plugin format allows swapping rewards or toggling observability without changing dynamics.

Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us ：https://forms.gle/wbash1wF6efRj8G58

Michal Sutter is a data science professional with a Master of Science in Data Science from the University of Padova. With a solid foundation in statistical analysis, machine learning, and data engineering, Michal excels at transforming complex datasets into actionable insights.