Open a 208 MB web app in your browser and type vim test.txt: the screen instantly switches into the familiar editor interface. Split panes with tmux or launch a game of Snake, and the cursor responds smoothly without hesitation. But deliberately mistype pythoon at the command line, and something bizarre happens: rather than printing “command not found” or freezing up, the terminal seamlessly drops you into an interactive Python REPL prompt. Behind this terminal—dubbed WorldTUI—there is no bash process running and no mounted filesystem. Every single screen update is predicted in real time by a 360M-parameter language model.
With only 360M parameters and zero lines of low-level execution logic, WorldTUI sustains a self-consistent terminal universe, retaining nearly 70% of file contents across sessions. Yet because it only predicts character deltas rather than executing underlying rules, the illusion unravels the moment familiar commands are combined in novel ways.
360M Parameters in a Browser, Running Zero True Code
WorldTUI is built on Alibaba’s open-source Qwen2.5-Coder-0.5B, pruned and fine-tuned down to 360M parameters, running locally via WebGPU inside desktop Chrome or Edge. The author offers a candid self-assessment: “Is this useful? Not really.” The core hypothesis he set out to test originates from DeepMind’s Genie 3—if future software is no longer hand-coded line by line by engineers, but instead hallucinated frame by frame by neural networks conditioned on user input, can such a tiny model sustain a coherent, stateful world?
Figure: WorldTUI predicting cowsay output and directory operations inside the browser. Note the configurable grid dimensions at the top. Source: SimEdw’s Blog (live project screenshot)
As early as 2022, developers were prompting ChatGPT to roleplay Linux terminals. Later, GameNGen simulated Doom frame-by-frame with diffusion models, and NeuralOS attempted to generate graphical desktop interfaces (GUIs). However, pixel-level video world models demand massive GPU clusters and vast video datasets—far beyond the reach of individual hackers. Stepping back from pixel canvases to character grids neatly sidesteps this compute bottleneck: real shells generate virtually infinite, perfectly labeled execution data for free, and pretrained code models already have a solid inductive prior on what ls or cat should print. By shrinking world models from pixel grids down to terminal characters, a single consumer GPU is enough to interrogate state consistency.
333 Tokens and a Single Copy Op: Slashing Scrolling Overhead by 97%
Standard LLMs append tokens sequentially to the end of a sequence, an approach that hits a wall in a terminal environment. In vim, pressing Esc produces no visible text; running clear wipes the entire screen and resets the cursor to the top-left corner. Output is rarely restricted to the bottom row. The author abandoned sequential text generation in favor of predicting screen grid deltas: each session begins with a snapshot of the grid size (e.g., 32×96), current characters, and cursor coordinates. Subsequent keystrokes only predict which rows must be rewritten and where the cursor should land. Because grid dimensions are explicit inputs, the model can resume mid-stream—whether handed an active game of Snake or a half-edited vim buffer.
Keystrokes are fed directly as raw bytes: Enter is \r, Escape is 0x1b, and the left arrow is ESC [ D. If the user is idle, a duration-parameterized WAIT token is injected, providing the internal clock that drives Snake forward. Rather than inventing novel vocabularies for coordinates and characters, the author reused Qwen’s native byte tokens spanning all 256 byte values, mapping row coordinates like 17 to a single token. The real bottleneck was vertical scrolling: whenever the terminal scrolls up by one line, every character moves. Forcing the model to reprint the full screen would demand ~3,000 tokens per Enter keypress.
Figure: WorldTUI executing a multi-line copy in vim, utilizing row-level COPY operations to avoid full-screen redraws. Source: SimEdw’s Blog (demo video frame)
To close this compute sinkhole, the author introduced line-copy primitives like ROW_0 COPY 1 and ROW_1 COPY 2. When scrolling, shifting a line takes just 3 tokens instead of ~96 redraw tokens. This also solved partial updates for screen clears, vim line insertions/deletions, and paging through man pages. Ultimately, the active vocabulary was compressed to just 333 tokens (256 byte tokens, 64 row tokens, plus a few special operators). Coupled with grammar-constrained decoding, every prediction produces a valid screen update. Whether an edge-deployed world model succeeds depends equally on neural scale and the compression efficiency of its state deltas.
75 Hours to Train Anti-Hallucination via Self-Correction
Once the vocabulary was settled, existing terminal recording datasets (like asciinema) proved inadequate because they lack synchronized keystroke-to-frame timings. The author generated synthetic data in Docker containers. Synthetic personas—from software developers to sysadmins—randomly generated directory trees and usernames, chaining 6 to 24 activities together (editing files, running Python, launching tmux). The pipeline simulated realistic typing cadence, Tab completion, a 1.2% human typo rate, and started 35% of sessions in media res. This yielded 336k episodes totaling roughly 2 billion tokens across 32×96, 24×80, and 40×120 grid sizes.
Supervised fine-tuning (SFT) quickly taught the model typing, backspacing, and cursor mechanics. However, long sessions routinely fell apart: a single mistyped filename early on would cascade and corrupt subsequent interactions. Drawing inspiration from robotics and the DAgger (Dataset Aggregation) algorithm, the author devised a self-correcting training loop. The model steps unguided for 1 to 4 steps from a checkpoint in a recorded session; if the diverging screen remains reasonably close to ground truth, a synthetic “correction” action forces the display back to reality on the subsequent keystroke.
A strict invariant governed these corrections: any term introduced in a correction step must have appeared in earlier ground-truth screens or user input history. If the model misremembered a previously seen filename, training it to rectify the spelling reinforced memory; if it hallucinated an entirely fabricated file, forcing it to recover the real filename would perversely reward hallucination. Roughly a third of sampled sessions yielded valid correction data. Running across three iterations over 75 GPU hours on a single RTX 4090, the int4 quantized model saw its PresetBench score (a suite of 47 brittle scripting scenarios) climb from 24/47 to 38/47. Autoregressive world models are notoriously vulnerable to compounding drift; training recovery behaviors directly over the inference distribution is far more effective than naively scaling demonstration data.
4× Cross-Session Retention over Base Models, but Compositional Limits Persist
To evaluate world consistency, the author devised three benchmark suites outside the training distribution. The most grueling was the cross-session memory test: writing random strings to two distinct files across 32 isolated environments, opening a clean session, and reading them back. Passing required both files to match perfectly:
| Model Setup | Parameters | ShellBench (106 commands) | Cross-Session Readback (Memory Test) |
|---|---|---|---|
| WorldTUI (Fine-Tuned World Model) | 360M | 80 / 106 | 9 / 13 |
| Qwen2.5-Coder-0.5B (Prompt Roleplay) | 500M | 52 / 106 | 2 / 13 |
| Qwen3-0.6B (Prompt Roleplay) | 600M | 60 / 106 | 3 / 13 |
On cross-session recall, WorldTUI achieved 9/13—outperforming similarly sized base models roleplaying via system prompts by 3× to 4×. Yet the architectural ceiling was laid bare in the exact same benchmark. While reliable on isolated command patterns, the model immediately falters when familiar commands are chained unconventionally. For instance, executing echo green >> colors to append a string, followed immediately by cat colors, frequently omits the newly appended line. Proponents note that sustaining ~70% state fidelity across sessions purely through attention validates the feasibility of “neural state machines.” Skeptics counter that failing to generalize on basic shell redirection illustrates why probabilistic token prediction cannot substitute for a deterministic OS kernel.
Beyond generalization limits, autoregressive generation nearly torpedoed real-time usability. Token-by-token generation delivered an acceptable ~38ms latency for standard typing. But full-screen redraws triggered by vim, man, or clear demanded up to 3,000 tokens—imposing an agonizing 12-second stall on an RTX 4090. The author noted a structural realization: rows within a single frame are largely independent and need not be produced serially. By treating each row as an independent batch element sharing the prefix KV cache under a customized attention mask, all rows decode in parallel, capped off by a single cursor-prediction step. This compressed full-screen decoding from 3,000 steps down to ~100 steps (the width of the longest row), slashing p95 latency on large frames from 6.7s to 0.87s (though Firefox’s WebGPU implementation remains ~20× slower than Chrome, likely due to browser-level driver differences). Remembering past characters does not equal understanding filesystem semantics; unconstrained neural prediction hits an insurmountable barrier when faced with compositional generalization.
With 75 GPU hours of compute, WorldTUI offers a sobering reality check on the “AI as software” zeitgeist while mapping the true outer bounds of 360M-parameter models. Leveraging a compact 333-token state encoding, DAgger self-correction, and row-parallel decoding, it squeezes an interactive terminal that edits files, splits panes, and plays Snake into a 208 MB browser footprint—without a single line of shell logic. Yet the ghost line that vanishes after echo green >> colors serves as a stark reminder: predicting surface-level visual syntax is fundamentally distinct from mastering state-transition mechanics. The author plans next to train across thousands of procedurally generated micro-apps to test if contextual cues alone can uncover unfamiliar interaction semantics—only then might world models transition from reciting screen recordings to grasping software logic.
References:
- SimEdw’s Blog: WorldTUI: A Terminal World Model
- Hacker News Discussion on WorldTUI
