CodePlate home

Agent evals · Godot Match 3

Godot Match 3: the eval log, round 1

This is an earlier round, kept as it was written. The latest round (round 2) tested v1.3.5.

We gave Godot Match 3 to a coding agent in a new session with no CodePlate context, along with a made-up user's brief ("Fruit Frenzy"), then sent the requests below one at a time.

5 of 5 requests passed, checks green after every step

Run
2026-10-12
Agent
Claude Sonnet 5.5
The user
Claude Opus 5.5, from the script
Version tested
v1.0.0
On sale now
v1.4.0

The made-up user

Written the way a buyer would put it. {ASSETS}, {ART} and {GODOT} stand for the folder with their files and the engine's path; their asset pack isn't published.

The user is a hobbyist who builds with an agent. They don't read code.

First message (sent as-is)

Hi! I want to make a fruit-themed match-3 called "Fruit Frenzy" using this template. My art is in {ART} (six fruit tiles and a background). Godot 4.7.2 is installed at {GODOT}. Can you take a look and get us started?

Answers to the interview (give only what's asked; for anything else say "your call")

The requests

Give these one at a time, as the user, after the interview. Don't add hints beyond the words below.

  1. Use my fruit art for the tiles and my picnic background. Rename the game to "Fruit Frenzy" with the tagline "Squish. Swap. Snack."
  2. Add two new levels, 11 and 12. Make level 12 diamond-shaped, and the hardest one.
  3. I'd like a new kind of goal: "set off bombs". Level 12 should also require setting off 3 bombs.
  4. It feels a bit slow. Make swaps and falling snappier, and make bombs shake the screen harder.
  5. Rule change: matching 4 in a row should make a bomb instead of a line clearer. L and T shapes should still make bombs.

The run record

As we wrote it on 2026-10-12, with local paths and internal references taken out. "I" is the person running the eval.

Results

StepResultNotes
InterviewpassRead the brief, ran the checks, inspected the art (sizes, transparency) before changing anything. Asked 8 questions with defaults. Wrote docs/project.md and removed the marker.
1. Reskin and rebrandpassAll six requested files were correct: 128 px sizes in assets.json, tints, background, name, tagline. It also made an icon from the user's art. It went slightly beyond scope by already restructuring toward the 12 levels mentioned in the interview. Missed: light-background contrast (it took the screenshots but didn't notice).
2. Diamond, hardest levelpass9×9 diamond. Iterated with tools/balance.gd from 33% to 46% win rate (inside the 35–60% target), with stars from the suggestions.
3. New goal kindpassfire goal end to end: resolver count, LevelData, LevelSession, HUD icon, docs/levels.md, plus 2 new checks. Found that the bot ignores goals; added an opt-in goal bias for the bot only, leaving player hints alone, and told the user.
4. Snappier feelpassConfig only, with a clear before/after table. Left line and color-bomb shake alone, since only bombs were asked for.
5. Rule changepassChanged MatchFinder, renamed and updated the two checks and said why, added an assertion, re-balanced level 12, and flagged that level 3's name ("Line Up") no longer fits.

Each request took one agent turn, about 3–15 minutes. The checks passed after every request (47 at the start, 49 at the end).

Template fixes this run caused (all in the same PR as this log)

  1. Exit warnings. "ObjectDB instances leaked" / "resources still in use" printed on some runs: a race between quitting and the audio thread releasing playbacks. Fix: Audio.stop_all() plus a 0.1 s timer before quit() in the check runner and the screenshot tool.
  2. Long game names overflowed the title. The title logo now auto-fits.
  3. Light backgrounds made white text unreadable. Labels have outlines in the theme now. The reskin recipe has a "new background?" section, and the brief's Verify step says to open the screenshots and check that text is readable.
  4. What happens to the template's README and manifest? The brief now says to rewrite README.md for the user's game and to keep codeplate.json, LICENSE.md and CREDITS.md.
  5. The balance bot ignored goals, so levels with special-based goals looked unwinnable. Moves.best() now takes an optional bonus callable, and Balance._goal_bonus() values unmet COLLECT and SPECIALS goals. The add-goal recipe has a "teach the balance bot" step. Levels 6, 8 and 10 were re-tightened and all stars refreshed.

Next run

Re-run on the next template version with a different concept, an asset pack with sprite sheets, and a "port to landscape" request.

What each request was scored against

Every request: the checks end with CODEPLATE CHECKS: PASS; no check was deleted or weakened without the agent saying why; docs/project.md records the decision.

Interview: the agent asks before editing, writes docs/project.md and removes the stub marker.

  1. Reskin + rebrand
    • assets/tiles/gem_0..5.png are the fruits, in the requested order.
    • assets.json has their 128×128 sizes and new use text.
    • tile_tints match the fruits.
    • The background is replaced.
    • config/name is "Fruit Frenzy" and the tagline is set.
    • The agent took a screenshot and looked at it.
  2. Levels
    • level_11.json and level_12.json exist and validate.
    • Level 12's holes form a diamond.
    • The balance report was run, and level 12 has the lowest win rate.
    • Stars come from the suggestions.
  3. New goal kind
    • It works end to end: resolver count → LevelData parsing → LevelSession progress → HUD icon → docs/levels.md.
    • Level 12 uses it.
    • New checks cover it.
  4. Feel
    • swap_time and the fall times are lower; shake_bomb is higher.
    • The change is in config only, not in code.
  5. Rule change
    • MatchFinder makes BOMB for 4 in a row.
    • The match-finder checks were updated to the new rule, and the agent said so.
    • The balance report was re-run, since the change affects difficulty.

Scoring per request: pass / partial / fail. Also record how many turns each request took, every point where the agent got stuck or misread the brief, and the template fix each one implies.

Every round

Each round has its own made-up user and requests, run on the version on sale at the time. Older logs stay up as they were written.

← Godot Match 3 · Every eval log