Today we are introducing Helix World, a real-time interactive audio-visual world model built by Noiz AI and The Hong Kong University of Science and Technology (HKUST). You start from a single image, move with WASD and look around with the arrow keys. The model generates the next stretch of video and the matching stereo sound at the same time, at 24 frames per second on a single NVIDIA H800 GPU.
Helix World is a world model that generates an explorable scene and its spatial stereo soundtrack together, conditioned on the user's camera motion and keyboard actions. When the camera turns, the sound field turns with it.
On the WBench navigation split, Helix World scores 79.9, close to the strongest silent world models, while running faster than real time with a real-time factor (RTF) of 0.77. On HelixBench, the audio-visual benchmark we release with the model, its spatial-acoustic consistency score is 41.8, compared with 12.7 for EchoWM, the only other world model in the comparison that generates sound. The paper is on arXiv, the code is on GitHub, a preview checkpoint is on Hugging Face, and the interactive demo is open at helixworld.org.
Why world models need sound
Interactive world models let a user steer a generated environment in real time. Most of them are silent. Yet sound carries information the picture does not: what a surface is made of, how hard two objects hit, and what is happening off screen. In an immersive scene, turning your head should also move the sound between your left and right ear.
Adding audio after the fact does not solve this. A cascaded video-to-audio model never sees the camera trajectory or the user's actions, so its sound ignores ego-motion, and running it after the video adds latency. Bidirectional audio-visual foundation models generate both streams, but they have no control interface, and naively turning them into causal, streaming models makes errors accumulate until both picture and sound degrade. Helix World addresses all three problems: missing training data, missing control, and long-horizon drift.
Explore worlds in real time
In the demo you choose a starting image or upload your own; it is encoded at 768×512. A text prompt is optional: if you leave it empty, the scene is described automatically from the frame. Once the session starts, WASD moves the camera and the arrow keys change the view. Generation runs in chunks of about 1.3 seconds, and each keyboard input or new prompt takes effect from the next chunk, so you can redirect the scene while it is running.
Benchmarks
Visual quality and speed
We measure visual quality and action following on the WBench navigation split. Real-time factor is the seconds of compute needed per second of output video, measured on one NVIDIA H800 including video and audio decoding; below 1.0 means faster than real time. Table 3 of the paper:
Model | Native audio | RTF (H800) ↓ | WBench average ↑ |
|---|---|---|---|
Alaya-EVOKE-Turbo | No | 4.29 | 82.0 |
EchoWM | Yes | 1.98 | 81.0 |
Zing-0.5 | No | 2.81 | 81.0 |
Helix World | Yes | 0.77 | 79.9 |
LingBot-World v2 | No | 5.21 | 79.4 |
SANA-WM | No | 0.79 | 76.0 |
LTX-2.3 (base) | Yes | 14.61 | 74.2 |
Matrix-Game 3 | No | 0.58 | 71.3 |
Among models that run faster than real time, Helix World has the highest score: 79.9, against 76.0 for SANA-WM and 71.3 for Matrix-Game 3. The three models above it need between 1.98 and 4.29 seconds of compute per second of video, and two of them are silent. Helix World is the only real-time model in the table that also generates sound.
Sound that follows the camera
Existing world-model benchmarks evaluate only video, and audio-visual benchmarks use short clips without camera motion. HelixBench fills that gap with 1,015 human-verified cases across six domains and four subsets: Navigation, Perspective, Event, and Subject. Each case provides a starting frame, an audio-visual prompt, continuous camera poses, and a stereo reference. Its spatial score correlates the on-screen position of sound sources with left-right stereo panning, so it directly measures whether the sound field rotates with the camera.
The baselines include EchoWM and cascaded pipelines in which we mute Helix World's video and re-dub it with a video-to-audio model (AudioX, ThinkSound, PrismAudio), so the picture stays fixed and only the audio changes. Table 4 of the paper:
Model | KL ↓ | FAD ↓ | DeSync (s) ↓ | ImageBind ↑ | CLAP ↑ | Spatial ↑ |
|---|---|---|---|---|---|---|
Helix World (causal student) | 1.39 | 2.39 | 0.59 | 0.299 | 0.302 | 41.8 |
Helix World (bidirectional teacher) | 1.54 | 3.08 | 0.50 | 0.270 | 0.268 | 33.2 |
EchoWM | 1.95 | 7.75 | 0.67 | 0.204 | 0.235 | 12.6 |
Helix World + ThinkSound | 2.23 | 6.19 | 0.42 | 0.208 | 0.289 | 14.8 |
Helix World + AudioX | 2.02 | 3.92 | 1.18 | 0.257 | 0.309 | −5.7 |
Helix World + PrismAudio | 2.24 | 6.39 | 0.46 | 0.215 | 0.272 | −9.5 |
LTX-2.3 (base) | 1.80 | 7.64 | 0.40 | 0.278 | 0.258 | 1.0 |
Helix World leads on audio fidelity (KL, FAD), audio-visual semantic alignment (ImageBind), and spatial consistency. It is not best everywhere: LTX-2.3 and the ThinkSound pipeline have lower temporal offset (DeSync), and the AudioX pipeline scores slightly higher on text-audio alignment (CLAP). The re-dubbing pipelines produce negative spatial scores because a video-to-audio model does not know where the sound sources are relative to the camera.
What people preferred
In a user study on full audio-visual rollouts, participants preferred Helix World over EchoWM 67.9% to 20.8%, and over the re-dubbed versions of its own video by 67.9% to 24.5% (AudioX), 80.8% to 9.6% (ThinkSound) and 77.8% to 16.7% (PrismAudio); the remainder were ties. With the sound muted, Helix World was preferred over SANA-WM, Alaya and HY-WorldPlay 1.5, tied with LingBot-World v2 at 39.5% each, and was less preferred than EchoWM, 26.3% to 44.7%. Its advantage comes mainly from the audio.
How we built Helix World
Data: true stereo with metric camera poses
Most video corpora are unusable for this task: the audio is often background music, voiceover, or pseudo-stereo with identical channels. We started from 4,100 hours of raw footage (43.9% real-world video, 34.1% game recordings, 22.0% open-source datasets) and filtered it in four stages, ordered from cheapest to most expensive:
Heuristic filtering removes dual-mono audio, shot cuts, low perceptual quality, text overlays, and static scenes.
Clip standardization cuts audio and video together into 121-frame clips at 24 fps and 768×512, with 48 kHz stereo.
Semantic filtering with a multimodal LLM that watches and listens rejects background music, narration, UI overlays, picture-in-picture, and talking heads, and keeps diegetic sound.
Camera and captioning recovers metric camera poses for every frame and writes three caption tracks: video-only, audio-only, and audio-visual.
The result is 3,000 hours in 2.1 million clips, a 73.1% yield. In an ablation, training with all three caption tracks gave the best audio-visual alignment.
A bidirectional teacher with camera and action control
Helix World builds on the open-source LTX-2.3 audio-visual model. Continuous 6-DoF camera trajectories enter the visual attention through PRoPE, and discrete keyboard actions enter through AdaLN-Zero on the diffusion timestep. Cross-modal attention then binds visual geometry to the audio features, which is how camera motion becomes stereo panning.
Training is progressive: first the control modules, then the video branch, then the full audio-visual backbone. Compared with fine-tuning everything at once, this lowers camera rotation error from 0.1237 to 0.1144 and translation error from 0.0957 to 0.0823, while keeping the pretrained visual quality.
A causal student that streams without drift
The bidirectional teacher sees the whole clip, so it cannot respond to live input. We distill it into a few-step causal student that generates block by block with a sliding key-value cache and fixed sink frames. The student is initialized by regressing the teacher's ODE outputs, then trained with self-forcing on its own rollouts.
Self-forcing with distribution matching (DMD) alone still drifts over long rollouts. Our addition is an online trajectory distillation loss: at each update, with probability p, the student's velocity at an intermediate rollout state is supervised by the teacher's probability-flow ODE target; otherwise the DMD loss is used. A final long-horizon streaming stage trains on extended windows. Together, these raise the WBench score from 77.9 to 79.1 and keep 30-second rollouts stable.
Limitations
Output resolution is 768×512. Real-time speed is reported on an NVIDIA H800 data-center GPU; the paper does not report consumer GPUs.
The public checkpoint is a preview release.
On visual quality alone, Helix World is slightly behind the best silent models (79.9 vs 82.0 WBench average), and with sound muted, viewers preferred EchoWM.
Helix World does not have the lowest DeSync on HelixBench, so tight onset timing still has room to improve.
For a different approach to the same problem, adding streaming stereo to an existing world model, see our WorldSonus post.
Try Helix World
The interactive demo is open at helixworld.org. Read the full paper, "HelixWorld: A Real-time Interactive Audio-Visual World Model" (arXiv:2609.38123, 29 September 2026), on arXiv. Code is on GitHub and the preview weights are on Hugging Face.
Authors: Lei Ke, Jiahao Pan and Zeyue Tian (equal contribution), Jiaming Wang, Haoyuan Huang, Kam Man Wu, Pengjun Fang, Hongyu Liu, Chenyang Qi, Lin Wang, Ruibin Yuan, Weijia Chen, Fangneng Zhan, Qifeng Chen, Wei Xue and Yike Guo. Corresponding authors: Zeyue Tian and Wei Xue. The Hong Kong University of Science and Technology and Noiz AI.