Agent evals · Print-and-Play Kit
Print-and-Play Kit: the eval log
We gave Print-and-Play Kit to a coding agent in a new session with no CodePlate context, along with a made-up user's brief ("Kettle & Crow"), then sent the requests below one at a time.
3 of 5 requests passed (2 partial)
- Run
- 2026-10-15
- Agent
- Claude Sonnet 5.5
- The user
- The script, relayed turn by turn
- Version tested
- v1.0.3
- On sale now
- v1.1.0
This run tested v1.0.3. The version on sale is v1.1.0, which includes the fixes listed at the end of the record and has not had its own eval run.
The made-up user
Written the way a buyer would put it. {ASSETS}, {ART} and {GODOT} stand for the folder with their files and the engine's path; their asset pack isn't published.
The user is a secondary-school teacher in Ohio who designs small card games in the evenings and builds with an agent. She has playtested Kettle & Crow with paper slips for months and wants a real print-and-play PDF for her friends and 10 copies from a print-on-demand shop for a local convention. She doesn't read code. She has Node and Typst installed (the eval machine has both; if the agent can't find Typst, that's a finding).
- Game: Kettle & Crow, "a push-your-luck game of shiny things", 2–5 players, 15 minutes, ages 8+. Designer: Dana Whitlock.
- Cards: her Google Sheet export
cards.csv(one tab, all cards): Trinket cards (the main deck, 30 printed), Crow cards (one per player, 5) and Event cards (6). Her symbols are written[shiny]and[feather]in the text. - Sizes: Trinkets poker size; Crows tarot ("big, so the art shows"); Events square.
- Printing at home: US Letter, her printer only does short-edge duplex. At the shop: 2.5 × 3.5 in cards with 1/8 in bleed, one 300 dpi PNG per card face, a tuck box for the trinkets.
- Look: copper and sky blue on near-black ("old kitchen at dusk"); her logo is
kettle-crow-logo.svg; her two symbols areshiny.svgandfeather.svg(drawn in black, notcurrentColor). No card art yet: placeholders are fine. - Rules:
rules.md, about 350 words.
First message
Hi! I've been playtesting my card game Kettle & Crow on paper slips and I want to make it properly printable, at home and at a print shop. My card list, logo, symbols and rules are in
{ASSETS}. Can you set it up?
Interview answers
- Game: Kettle & Crow, a push-your-luck game of shiny things, 2–5 players, 15 min, 8+, by Dana Whitlock.
- Decks: trinkets (poker), crows (tarot), events (square); counts are in the Qty column. No tokens, no board for now.
- Printing: US Letter at home; "my printer flips on the short edge". Shop: 2.5 × 3.5 in, 1/8 in bleed, 300 dpi PNGs, a tuck box.
- Look: near-black
#1b1a1f, copper#c4733b, sky blue#7fb7d9, cream#f6efe3; "keep the fonts if they're free to use"; placeholders for art. - Symbols:
[shiny]and[feather], her SVGs. - Rulebook: half-letter booklet.
The requests
- Make it mine: Kettle & Crow, my colours, my logo on the card backs and the box, and take Meadowlight out. I print on US Letter only, and my printer flips on the short edge.
- Put my cards in from
cards.csv: trinkets as poker cards, crows as tarot cards, events as square cards, with the copies from the Qty column. My[shiny]and[feather]symbols should show as my little pictures in the text. Placeholder art is fine for now. - Trinkets need a "Weight" (1–3) in the top right corner, with a feather symbol, and the Magpie's Hoard card's text got squashed or cut off. Fix it so it reads well.
- I'm ordering 10 copies from a shop: cards 2.5 × 3.5 in with 1/8 in bleed, one 300 dpi PNG per card face, and the trinket tuck box as art plus a separate cut line. Get me the files.
- Turn my
rules.mdinto the rulebook as a half-letter booklet (it needs a multiple of 4 pages), with a picture of a real trinket card in it. Also: on my duplex test, the red crosses on the back land about 1 mm too far right and half a millimetre too high.
The run record
As we wrote it on 2026-10-15, with local paths and internal references taken out. "I" is the person running the eval.
- Template: print-and-play-kit 1.0.3, fresh from the packed zip, with the Kettle & Crow pack from
assets/. - Agent: a Claude Sonnet 5.5 subagent with no CodePlate context, on Windows with Node and Typst 0.15. User turns were relayed one at a time from
concept.mdandrequests.md. - Most of the work came before the requests. The first message ends with "Can you set it up?". The agent asked nine questions with defaults: it asked A4 or Letter, flagged that her symbols are drawn in black, warned that the Magpie's Hoard text is too long for a poker card, and offered not to change her wording. The interview answers said "leave Magpie's Hoard for now" and "go ahead", and in that turn it did the setup: card import, colours, logo, US Letter short-edge, shop export and rulebook. Requests 1 and 2 arrived after the work was done.
- Relay note: request 4 ended with a question (see below). The answer and request 5 went in the same message.
- Result: interview + 3/5 requests passed, 2 partial. I re-ran
npm run checkon the finished project:CODEPLATE CHECKS: PASS(tools, assets, data, unit, pdf, faces, sheets, shop, rulebook, preview, budget), with no Typst warnings. Faces: 99 text areas on 27 faces, all fit inside the 4 mm safe zone. Sheets: 83 pieces on US Letter, 41 backs behind their fronts (short edge). Shop: trinkets 825 × 1125 px, crows 900 × 1500, events 825 × 825. Rulebook: 4 pages. No check, fixture or test was changed.
| # | Request | Turns | Result | Notes |
|---|---|---|---|---|
| 0 | Interview | 2 | pass | Nine questions with defaults, and the three problems in her files found before anything changed. During the setup it found a template bug: print.bleed was rounded to 3.18 mm, which made the 300 dpi PNGs 826 px wide instead of 825. It fixed one line in src/load.mjs and wrote the change in docs/project.md. |
| 1 | Make it mine | 1 | partial | Already done in the setup turn; this turn it also removed Meadowlight leftovers, including the screenshots in docs/media/. Title, 2-5 players, 15 min, 8+, Dana Whitlock; her four colours plus tints; the logo listed at 300 × 100 and credited to her, on every back, the box front and the rulebook cover; papers ["us-letter"], duplex "short-edge". grep -ri meadowlight over game.json, data, layout and rulebook finds nothing. The subtitle is "Crows love shiny things." (the first line of her rules), not "a push-your-luck game of shiny things" from the interview. Her logo has no lettering and replaced the title on the box front, so the box front no longer shows the game's name (it is still on the box sides). |
| 2 | Her cards | 1 | pass | Already done; it re-checked all 23 cards against cards.csv with a script. Three decks (poker, tarot, square), 30 + 5 + 6 printed, her wording kept, her Notes column kept as an unprinted #notes column. [shiny]/[feather] became {shiny}/{feather}, and her two SVGs were changed to currentColor (her originals left untouched). Placeholder art from npm run art -- placeholder, a layout per deck. |
| 3 | Weight + Magpie's Hoard | 1 | pass | weight is int:1..3!; a copper badge with the feather and the number sits top right, and the name now wraps clear of it. For the Hoard it kept her text and gave trinkets with an effect over 100 characters a shorter art window; the text now sets at about 7 pt inside the safe zone. It also added a sentence about the badge to the rulebook without asking, and left an unused weight.svg icon behind. |
| 4 | Shop files | 2 | partial | print.bleed "0.125in", npm run export: 12 trinket fronts and back.png, each 825 × 1125, PDFs per deck, box.pdf and box-dieline.pdf for 30 cards, and manifest.csv with the counts; it worked out the quantities for 10 copies. It noticed that crows and events aren't 2.5 × 3.5 in and asked what to do instead of guessing. docs/project.md tells her to check the shop's own spec and tuck box template, but the relayed replies don't mention that the files are RGB or that she should order one proof copy first. |
| 5 | Rulebook + back offset | 1 | pass | The rulebook was already a half-letter booklet of 4 pages in her words, with the Copper Kettle Lid card (and a crow and an event) as pictures. backOffset ["-1mm", "0.5mm"]: crosses too far right, so the backs move left; too high, so they move down, as the home-printing recipe says. The shifted backs then hit the 6 mm margin check, so it raised home.margin to 7 mm, which costs a card per US Letter sheet (9 poker cards become 8; the trinkets still fit on 4 sheets). Re-exported the shop files. |
Where the template fell short (fixed in print-and-play-kit 1.1.0)
- Bleed rounded to 0.01 mm.
src/load.mjsstoredprint.bleedthroughround2, so0.125in(3.175 mm) became 3.18 mm and every 300 dpi poker PNG came out 826 × 1126. The shop check computed the expected size from the same rounded value, so it passed. The loader now keeps the bleed to 4 decimals, and the shop check works out the expected pixels from the bleed as written ingame.json. A loader test reads"0.125in"and gets 825 × 1125 (poker), 900 × 1500 (tarot) and 825 × 825 (square), and"3mm"still gives 821 px. - Float noise in
manifest.csv. The file a shop reads said76.19999999999999x127and174.54999999999998x147.45. Every millimetre value in the manifest now has at most two decimals. A test exports a 3 × 5 in deck with a 0.125 in bleed and checks every size cell. - Any back offset failed the margin check. With 9 poker cards on US Letter at a 6 mm margin there is 0.35 mm to spare, so a 0.5 mm offset put the backs "in the margin" and cost a card per sheet. The margin check now measures each back where it was planned, before
backOffset; the check that each back sits behind its front still includes the offset. The home-printing recipe says the offset doesn't count against the margin and what a 7 mm margin costs on US Letter. A test runs the eval's case: US Letter, 6 mm, short edge,["-1mm", "0.5mm"]. - Meadowlight was wired into the layouts. The card back, the tuck box, the showcase and the rulebook now read only
game.json:game.logo, the palette rolesink,paper,accent(now required) andaccent2with optional light and dark tints,box.art,box.lids,rulebook.coverArtandshowcase.art. Without a picture they fill with the palette's colours. Meadowlight'sgame.jsonsets these to its old colours and files, so its PDFs and screenshots come out the same.npm run art -- sampleredraws only the sample files that still exist and says "no sample art left" when none do. The start-your-own-game recipe lists what to remove and what stays, and that no shared layout orsrc/file needs editing. A test builds a game with none of Meadowlight's files and onlyink,paperandaccent: proof, home sheets, the box and a deck as shop files, rulebook and showcase, with no Typst warnings, with and without a logo. - Box text lived in the layout. The tagline and blurb are now
box.taglineandbox.blurbingame.json, checked as text. The same test gives the box a 300-character blurb, and every text area still fits inside the safe zone. - A logo could push the name off the box. The box front now shows the logo and the title together.
game.logoHasName: truedrops the title text only when the logo spells the name. The replace-the-art and tuck-box recipes say so. - No licence value for the user's own files. Files the user supplies are now
user-owned, credited to their owner, and aCodePlate-originalfile must credit CodePlate. The brief and the replace-the-art, icons and add-cards recipes say which value to use.
Not changed: the RGB and proof-copy advice is already step 5 of the print-shop recipe; the agent didn't pass it on. The subtitle came from the first line of her rules instead of the interview; the start-your-own-game recipe now says to use the subtitle the user gave, but that was the agent's choice, not something the template forced.
What each request was scored against
Every request:
npm run checkends withCODEPLATE CHECKS: PASS(tools, assets, data, unit, pdf, faces, sheets, shop, rulebook, preview, budget), run in full after the change.- No check is weakened, skipped or deleted to pass;
print.safeandprint.bleedare not lowered to make text fit; every text area still usesinfo(no plainplace(text(…))for card text); no Typst warnings. - Card text and numbers live in
data/*.csv, not in.typfiles. - The agent ran
npm run preview -- proof(ornpm run preview) and opened the images for every deck it changed, and said what it saw. docs/project.mdis updated (the stub marker is gone after the interview).- No npm or Typst Universe packages; no third-party art or icons; the fonts stay OFL with their licence files (or new OFL fonts with theirs).
- Make it mine
game.jsongamesays Kettle & Crow, the subtitle, 2–5, 15 min, 8+, Dana Whitlock;paletteholds her four colours under names the layouts use (or the layouts were updated).kettle-crow-logo.svgis inassets/, listed inassets.jsonwithwidth: 300, height: 100(the viewBox) and credited to her; it appears on the card backs and the tuck box front, at a sensible size (not stretched).- No Meadowlight cards, names, art, icons or rulebook text left (
grep -ri meadowlight game.json data layout rulebookclean, apart from anything the user agreed to keep);tests/fixtures/untouched andnpm testpasses. home.papersis["us-letter"],home.duplexis"short-edge"; the back sheets are mirrored top-to-bottom (sheetspasses).
- Her cards
- Three decks in
game.jsonwith the right sizes (poker,tarot,square), CSVs indata/converted fromcards.csv(her wording kept), counts from Qty: 30 trinkets, 5 crows, 6 events printed. [shiny]/[feather]became{shiny}/{feather};assets/icons/shiny.svgandfeather.svgare her files, listed inassets.json(24 × 24), changed tocurrentColor(or the agent explained why they stay black and the user agreed).- Placeholder art from
npm run art -- placeholder …(or none, with the layouts handling no art), listed inassets.json. - A layout per deck (
layout/cards/*.typ) registered inlayout/templates.typ; the proofs show readable cards.
- Three decks in
- Weight and the Magpie's Hoard
- A
weightcolumn (int:1..3!) ingame.jsonand the trinket CSV, drawn top right with{feather}/icon("feather")inside the safe zone. - The Magpie's Hoard text fits: the agent saw the
facesnote or failure (or the proof) and either enlarged the rules area for all trinkets, shortened the text with the user's OK, or both. Not fixed by lowering the smallest size below about 6 pt or moving text outside the safe zone.
- A
- Shop files
print.bleedis"0.125in"(3.175 mm);npm run exportran;out/shop/png/trinkets/has one PNG per trinket front plusback.png, each 825 × 1125 px;manifest.csvlists the counts;box.pdfandbox-dieline.pdfexist for the trinket deck (30 cards).- The agent told her the files are RGB (the shop converts), that she should check the shop's current template, and to order one proof copy.
- Rulebook and back alignment
rulebook/rulebook.typcarries her rules (her wording, tidied only with her OK),#components()lists her decks, a#card-figure("trinkets", …)shows a real card;rulebook.paperisus-half-letter, page count a multiple of 4.home.backOffsetis["-1mm", "0.5mm"](red crosses too far right → move backs left; too high → move them down); the duplex test PDF was rebuilt;sheetspasses with the offset.