This is a video transformer I trained to simulate 2D waves. Draw with your mouse or finger — when you lift off, a physics simulator seeds the first 8 frames, then the ~18M-parameter model rolls out the rest on my GPU and streams the ripples back to your canvas in real time, one frame at a time. The model learned its physics in a walled tank — watch the ripples bounce off the edges and interfere with each other on the way back.
Tip: keep it lively with a few spread-out splashes, and draw over a running wave to drop in another — the new splash merges into the ripples already in motion and rings louder than the older waves, which have started to fade. The fade slider sets how fast the water calms (zero lets it ring forever), and speed trades smoothness for slow-motion detail.
No equations — just the path a single frame takes through the network. Scroll to walk down the pipeline, one stage at a time.
The instant you lift off, the splash is frozen as a 128×128 grid of surface heights — a tiny topographic map of the water. A quick physics pass hands the model its first 8 frames of motion to start from.
128 × 128 field · 8 seed frames
Every frame is chopped into little 16×16 tiles, and each tile becomes one token — like turning a picture into words the transformer can read. One frame becomes 64 tokens.
patch 16×16 → 64 tokens / frame
Each tile is expanded into a 384-number fingerprint, then stamped with where it sits on the grid and when in time it happened. Now the model knows each token’s place in space and moment.
linear → 384-d · spatial pos + RoPE time
Ten layers of self-attention. Tokens in the same frame share everything; across time, each moment can only look backward — never at the future. This is where the model learns how a ripple here nudges the water there a step later.
10 layers · 8 heads · block-causal attention
The transformed tokens are turned back into 16×16 tiles and stitched into a full 128×128 frame — the model’s guess for what the water looks like a moment from now.
linear head → unpatchify → 128 × 128
A small convolutional network slides across the rebuilt frame, softening the seams between tiles and sharpening the ripples. It only adds a gentle correction on top of what the transformer produced.
residual Conv2d smoothing
The fresh frame joins the last 8 and the whole loop runs again — one new frame at a time, quantized and streamed live from the GPU to your canvas. That’s the ripple you watch unfold.
autoregressive · ~18M params · live on GPU