Same prompts. Different models. Put the results next to each other and look closer.
Give it tools.
Let it cook.
Keep the receipts.
0 results, grouped by the prompt that started them.
Give an agent a prompt. Let it build and test.
Bring back what it made.
results/model_name/run-001/index.htmlA different medium. Same curiosity.
Give the agent the whole production. Watch what comes back.
One complete prompt. Tools from the start. Your move.
Independent evaluations of the finished builds.
Numerical benchmarks matter. Research and training need guardrails, and a score is useful evidence. I use them. But equal scores do not mean equal experiences with the models behind them.
Look at this Deep-SWE snapshot. GPT-6 Astra [xhigh] and Gemini 3.8 Flash [high] both display 74%, with different costs, token usage, and step counts. In my hands, these models don't even feel like they're playing the same game.
What I keep missing in benchmark tables is the chance to try what the model built. Open the app. Push the controls. Follow a workflow until something breaks. You start to recognize a model's style, its strengths, and the things it consistently neglects.
That's what I wanted to make easier with Trial.
I chose tasks with enough complexity to make the agent work for it. A couple of model generations ago, I would have budgeted multiple hours for these jobs.
In my own Astra runs so far, each has delivered a build in under an hour. That's what I observed in those runs; correctness still has to earn its own evidence.
Does the physics look plausible? Is the game fun? Does the interface hold together? You can form a useful first impression without being a domain expert.
Then use the formal checks to see whether that impression survives the requirements. Both kinds of judgment matter.
I like a good voxel world. Those Twitter fly-throughs can show real visual flair. But for learning how a model builds, I want something I can push back on.
Disturb the physics. Make decisions in a playable game. Play an instrument. Push an interface until it breaks. That's why I prefer these demos: they expose cause and effect, strengths, and failure modes you can actually reproduce.
Voxel apps can do all of that too. A pretty camera orbit just hasn't demonstrated it yet.
The Video lab takes that curiosity into another medium. Research, script, motion, voice, sound, and the final edit: give the agent the whole production. You know when a film loses you, when a visual makes an idea click, and when the narration gets ahead of the evidence. Start there.
Keep the benchmarks.
Get your hands on the work.
Copy a complete prompt from the library. Keep the evaluator directory separate.
Let the agent plan, write, execute, debug and test. Browser validation belongs in the run. The capstone also asks it to research the web and choose its own idea beyond the other 23 challenges.
results/
model_name/
01-fluid-simulation/
index.html
screenshot.png
evidence/Every HTML challenge delivers one self-contained index.html. Keep the original artifact intact. Metadata and screenshots are optional. Video experiments have their own briefs in the Video lab.
Refresh the gallery. Each prompt gets its own row of model outputs. Open a screenshot to inspect the build, or read the original prompt alongside it.
python tools/new_run.py --model "Model Name" --run 01-fluid-simulation-001 --task 01-fluid-simulation --html work/index.html
Screenshots show the surface. Open the app to inspect its interactions. Record independent checks in report.json; keep the agent's own logs in evidence/.
Bind local reports to the artifact hash. The public showcase publishes HTML and thumbnails; evaluation files are excluded from the website export.