On January 20, 2026, a startup called Overworld published Waypoint-1 on the Hugging Face blog: a “real-time-interactive video diffusion model, controllable and prompted via text, mouse, and keyboard.” Give it starting frames and it does not render a fixed clip — it generates a steerable world, with every generated frame conditioned on your live input. The weights of Waypoint-1-Small, a 2.3B-parameter model, went straight onto the Hub, and the inference stack, WorldEngine, was opened on GitHub alongside it.
For developers, this is the first time a world model has shown up in downloadable, benchmarkable form: not a pre-recorded demo, but 30 FPS interactive control on consumer hardware. The next day, researcher Akshay Gholam Akaliq called it “a meaningful step toward interactive world models that can actually respond to real-time control.”
What Waypoint-1 Is
Waypoint-1 is a video diffusion model prompted and steered by text, mouse, and keyboard. The contrast with ordinary text-to-video models is the control path: those are fine-tuned with brief control inputs, while here, in the team’s words, “you can move the camera freely with the mouse, and input any key on the keyboard.” At the API level it accepts a per-frame control input, CtrlInput(button=…, mouse=[x, y]), combined with text prompts and optional forced frames. The backbone is a frame-causal rectified flow transformer operating in a compressed latent space. The claim is zero-latency control: move the camera or press a key, and the next frame responds to that input directly rather than waiting on a batched generation pass.
10,000 Hours of Gameplay and Diffusion Forcing
Training runs in two stages. Pretraining used 10,000 hours of gameplay footage paired with control inputs and captions, via diffusion forcing: a causal attention mask over the sequence, with each frame noised at random levels and denoised independently. The cost of that recipe is a train/inference mismatch — long rollouts accumulate error and get visibly noisier. The post-training stage therefore switches to self-forcing with DMD (distribution matching distillation) to close the gap, and adds one-pass CFG plus few-step denoising to push inference cost into real-time territory.
WorldEngine: The Inference Engineering Behind 30 FPS
What actually makes “real-time” true is the WorldEngine stack. On an RTX 5090, Waypoint-1-Small (256 tokens per frame) sustains roughly 30,000 token-passes per second on a single denoising pass — which translates to 30 FPS at 4 denoising steps, or 60 FPS at 2. The frame-budget arithmetic explains why every optimization matters: at 30 FPS you have about 33 milliseconds per frame, so the full conditioning, attention, and decoding loop has to fit inside that window on every single frame. The techniques are unapologetically low-level: AdaLN feature caching, a static rolling KV cache built on Flex Attention, fused QKV matmuls, and torch.compile run with fullgraph, max-autotune, and dynamic mode off. In other words, 30 FPS was not bought with a smaller model alone; it came from squeezing every millisecond out of each denoising step.
Open Weights and the Developer Ecosystem
The ecosystem rollout is unusually complete for a first release. Waypoint-1-Small is on the Hub as overworld/Waypoint-1-Small, with Waypoint-1-Medium marked “coming soon.” WorldEngine lives in the Wayfarer-Labs/world_engine repository on GitHub, and a playable demo runs on the company’s site. Launch day came with a hackathon whose prize was an RTX 5090 — a deliberate move to pull the first wave of players and builders into the ecosystem.
How Far From Actually Playable
The limits are equally clear. What shipped first is the 2.3B small model, with Medium still pending; long-rollout consistency in a latent-space model is precisely the problem the training pipeline is engineered around, not one it has solved. Community testers are already playing it with keyboard and mouse in real time, and the direction gets credit — but fidelity and stability are not yet at product grade.
The pragmatic read for product teams: treat this as the first self-hostable baseline for interactive world models — good for interactive-story prototypes, agent training environments, and gameplay validation — not as a game-engine replacement. The API surface (per-frame control inputs, forced frames, text prompts) is exactly the shape you would want for scripting scenarios or piping an agent’s actions into a simulated world, and self-hosting means the data never leaves your infrastructure. The real signal is categorical: world models have moved from papers and closed demos into an open, measurable, iterable phase.
Sources
- Introducing Waypoint-1: Real-time interactive video diffusion from Overworld — Hugging Face
- Waypoint-1 announcement and demos — Akshay Gholam Akaliq on X
- Wayfarer-Labs/world_engine — GitHub
AI-assisted summary compiled from the sources above, reviewed by a human before publishing.