Film a clip of a game that doesn't exist — then play it

Here's a neat one: you tell a video model "a courier boy jumping between flying whales above the clouds." You get 15 seconds of gorgeous gameplay. Then an agent turns that clip into a game — same sky, same whales, same jump. And you can play it in the browser.

Why this only works now: agents write game code fine, but they can't draw. Ask for "a whale game" and you get grey boxes and a camera that fights you. Video models are great at exactly the look. On October 29 a skill called Video to Game came out: the video designs how the game looks, and the agent catches up frame by frame.

How it works

• The video is the spec. A script cuts the clip into frames and pulls out the palette and cut times. The agent writes a spec from them: camera, hero, one core mechanic.
• fal makes all the art and sound. Images, 3D models, animations, music — all generated from the video's own frames. Nothing hand-drawn, nothing downloaded.
• The agent plays its own game. A script presses keys, records a run and looks for falls and stuck moments. It fixes them — and only then shows you.
• The output is one index.html with Three.js. No engine, no build. Upload it anywhere and it's a link.

The author built 31 games this way: 1,013 fal calls, $202 total — about $6 per game ($4 to $11).

What you'll learn

• Picture first, code second. You'll see why "show an example" beats "describe it." That goes way beyond games.
• Splitting work between models. The video model owns the look, the agent owns the mechanics. Each does what it's good at.
• Checking before showing. An agent that playtests its own game is a trick worth stealing for any project.

A ready starter prompt

Start with the clip. The video model doesn't need "pretty" — it needs a game it can repeat:

Then hand the clip to an agent with the skill:

Turn gameplay.mp4 into a playable browser game with the video-to-game skill. Match the video's look, generate every asset with fal, and show me side-by-side pairs and a recorded take of you playing it.

What you end up with

Two pictures side by side: your clip on the left, the game on the right in the same light and color. Plus a recording of the agent riding the level itself. You send a friend the link: "here's the video I imagined — and here's how you play it."

Honest limits: it's not a big game, it's one mechanic and one level. You need a fal key (paid, ~$5–10 per game), Python, Node and ffmpeg. And the agent run is long — an evening, not a minute.

A year ago "a game from a video" sounded like sci-fi. Now it's a skill you install with one command.

Source: Video to Game — a skill for Claude Code, Codex and Cursor (GitHub, lovisdotio)