Godot Platformer: the eval log

We gave Godot Platformer to a coding agent in a new session with no CodePlate context, along with a made-up user's brief ("Lily Leap"), then sent the requests below one at a time.

4 of 5 requests passed (1 partial)

Run: 2026-10-13
Agent: Claude Sonnet 5.5
The user: Claude Opus 5.5, from the script
Version tested: v1.0.0
On sale now: v1.2.4

This run tested v1.0.0. The version on sale is v1.2.4, which includes the fixes listed at the end of the record and has not had its own eval run.

The made-up user

Written the way a buyer would put it. {ASSETS}, {ART} and {GODOT} stand for the folder with their files and the engine's path; their asset pack isn't published.

The user is a hobbyist who builds with an agent. They don't read code.

Game: Lily Leap, a frog hopping through a swamp.
Art (drawn by a script of ours), deliberately mismatched with the template:
- a 32×32 frog with different frame counts (idle 2, run 8, jump 1, fall 2, hurt 2, land 2) and no wall_slide
- 32 px swamp terrain (a 16-frame autotile strip) and three swamp parallax layers at 640×360
Platforms: desktop and web.
Content: 10 levels in the end, a bit easier than usual.

First message (sent as-is)

Hi! I want to turn this platformer into my own game, "Lily Leap": a frog hopping through a swamp. My art is in {ART}. Godot 4.7.2 is at {GODOT}. Can you take a look and get us started?

Answers to the interview (give only what's asked; for anything else say "your call")

Cute, calm swamp. The frog is the hero.
The art is mine; credit "Lily Leap team". It's pixel art at 32 px.
Keep the template's sounds for now.
Movement: a bit floatier than default; double jump later.
10 levels eventually, a bit easier than usual.
Desktop and web.

The requests

Give these one at a time, as the user, after the interview. Don't add hints beyond the words below.

Use my frog and swamp art for world 1, and rename the game "Lily Leap".
The frog should be able to double jump from the start, and jumping should feel a little floatier.
Add a dragonfly enemy that hovers and swoops down at you when you get close. You can stomp it.
Add level 9 to world 2: a tall vertical climb with a checkpoint halfway and a star hidden near the top.
Rule change: one hit kills you, but you have 5 lives. Every 50 coins gives an extra life. Losing all lives restarts the level from the start.

The run record

As we wrote it on 2026-10-13, with local paths and internal references taken out. "I" is the person running the eval.

Agent: Claude Sonnet 5.5 (general-purpose subagent), with no CodePlate context. It was given only the user's messages, the project path and the Godot path.
User: played by Claude Opus 5.5, following concept.md and requests.md.
Template: the packed 1.0.0 zip, unzipped into an empty folder. The art pack (drawn by a script of ours) deliberately mismatches the template: 32 px tiles, 32×32 frog strips, different frame counts, 640×360 backgrounds, and no wall_slide strip.

Results

Step	Result	Notes
Interview	pass	Spotted every mismatch (tile size, frame size, backgrounds, missing wall_slide) before editing, and offered two options for the size with a recommendation. Wrote `docs/project.md`.
1. Reskin + rename	pass	Moved to a 960×540 view with 32 px tiles and scaled the template's remaining art with a script. It flagged the stand-in wall_slide, rewrote the README, and re-made the replays its floatier tweak broke, using autoplay. Found that the UI only fits 480×270, and noted that the frog looks small in its frame.
2. Double jump + floatier	pass	Changed config only and reported feel numbers. Found a real template bug: with double jump on, down+jump on a one-way ledge fired an air jump instead of dropping through. Fixed it, adjusted one check and said why, and re-made the replays.
3. Dragonfly	pass	A tick-based state machine (hover → wind-up → swoop → return → rest), with config tunables, an `at` override, a legend letter, its own placeholder art (clearly labelled), 7 new checks including determinism, and docs.
4. Tall level 9	pass	A 40×48 climb with a checkpoint halfway, a hidden star alcove, enemies, reach clean and an autoplay replay. Slow (about 55 min) because of many bot iterations.
5. Lives + one-hit	partial	The rules, HUD, game-over flow and 4 new checks all work. Two replays (levels 4 and 8) failed because the bot couldn't survive one-hit enemies. The agent reported this honestly and didn't delete or weaken them, set `lives = 99` for bot and replay runs, and flagged that hearts are now useless.

The checks went from 49 at the start to 61 at the end (59 passing, with 2 honest replay failures).

Template fixes from this run (godot-platformer 1.1.0)

Drop-through and double jump. Down+jump no longer also fires an air jump. There's a new check for it.
ui_scale. The HUD, pause, results, title and level select scale for bigger viewports, and backgrounds scale by whole steps to cover the view's height. The reskin recipe explains the 32 px path and the "sprite looks small in its frame" check.
The theme reaches CanvasLayers. The HUD and overlays had lost the pixel font, because CanvasLayers don't inherit the window theme. Game.ui_root() now carries it.
Autopilot.
- It hops at enemies in its way, which stomps walkers.
- It waits for clear take-offs and landings.
- tools/autoplay.gd retries with different start delays, an idea taken from the agent's own change.

The improved bot finished the agent's two failing levels under its one-hit rules.

Recipes and docs. The health recipe now covers lives with replays (high life count for bot and replay runs) and asks what to do with hearts. The screenshot hook gains y= for tall levels.

Next run

Re-run with a different concept: HD (non-pixel) art, wall jump enabled, and a touch-controls request.

What each request was scored against

Every request: the checks end with CODEPLATE CHECKS: PASS; no check was weakened silently; no replay was deleted to pass; docs/project.md records the decision.

Interview: the agent asks before editing (it should ask about the 32 px art), writes docs/project.md and removes the stub marker.

Reskin + rename
- assets.json sizes and frame counts match the new strips.
- The 32 px art is handled: tile_px = 32 and the viewport is adjusted, or the agent explains its choice.
- The missing wall_slide was flagged and a stand-in used, not invented silently.
- The agent took a screenshot and looked at it.
- config/name is "Lily Leap".
Double jump + floatier
- double_jump_enabled = true.
- Floatier comes from jump_time_to_apex up and/or fall_gravity_mult down, changed in config only.
- tools/feel.gd before/after was shown.
- Reach and replays still pass, or broken replays were diagnosed and remade (not deleted).
Dragonfly
- A new Actor subclass with a tick-based state machine (hover → swoop → recover), a legend letter and ART entries, used in at least one level.
- Pure checks for the swoop trigger and the stomp.
- Determinism kept.
Level 9
- level_09.txt is tall (40+ rows), in world 2, with a checkpoint about halfway and a star near the top.
- Reach passes. A replay was made with tools/autoplay.gd (or the agent explained why not).
- The camera follows vertically.
Lives
- max_health = 1; lives and a coin→life rule in config/World.
- The game-over flow restarts the level.
- The HUD shows lives.
- Affected checks were updated, with an explanation.

Record turns per request, stuck points and the template fixes they imply.

← Godot Platformer · Every eval log