Astro Website Starter: the eval log

We gave Astro Website Starter to a coding agent in a new session with no CodePlate context, along with a made-up user's brief ("Rye & Ember"), then sent the requests below one at a time.

5 of 5 requests passed, checks green after every step

Run: 2026-10-13
Agent: Claude Sonnet 5.5
The user: The script, relayed turn by turn
Version tested: v1.0.0
On sale now: v1.0.3

This run tested v1.0.0. The version on sale is v1.0.3, which includes the fixes listed at the end of the record and has not had its own eval run.

The made-up user

Written the way a buyer would put it. {ASSETS}, {ART} and {GODOT} stand for the folder with their files and the engine's path; their asset pack isn't published.

The user runs a small sourdough bakery and builds with an agent. They're comfortable with computers but don't read code.

Site: Rye & Ember, a neighbourhood sourdough bakery in a small town. Pitch: "Slow-fermented bread, baked in a wood-fired oven every morning."
What it's for: show the menu, opening hours and location, take pre-orders for weekend bakes through a form, and post a journal (blog) about baking.
Brand: warm and rustic. Dark rye brown (#3b2618) and ember orange (#d9622b) on a cream background (#f7f0e3). The logo is in assets/ (see its README). Tone: friendly, short sentences, a little playful.
Pages: home, menu, journal, about, contact/pre-order. No pricing tiers and no changelog (it's not software).
Form: pre-order requests go to their Discord channel through a webhook (they'll paste the URL later).
Domain: ryeandember.example, on Cloudflare.

First message

Hi! I want to turn this template into the website for my bakery, "Rye & Ember". My logo is in {ASSETS}. Can you take a look and get us started?

Interview answers

A sourdough bakery; name "Rye & Ember", pitch as above.
Colors as above, the logo is the SVG. Dark mode can stay, with the same feel.
Pages: home, menu, journal (the blog), about, contact. Drop pricing and changelog.
Placeholders are fine for photos and testimonials, clearly marked.
Form: weekend pre-orders, sent to a Discord webhook (URL comes later).
Domain ryeandember.example on Cloudflare; not deploying yet.

The requests

Make it Rye & Ember: my logo, my colors, my words everywhere (the page titles, the footer, the social previews too). Take out pricing and the changelog, and make the home page about the bakery: what we bake, the oven, a few (placeholder) reviews and our opening hours.
Add a Menu page: our breads, pastries and drinks in sections, each with a name, a short description, a price and labels like "vegan" or "contains nuts". Some items are only baked on weekends; show that clearly. I want to edit the menu without touching the page code.
Turn the form into a weekend pre-order form: name, email or phone, pickup day (Saturday or Sunday only), and what they'd like (free text). Send it to my Discord webhook; until I give you the URL, it should still work for testing.
Write our first journal post, "Why we bake with a wood-fired oven", with a photo placeholder and a callout with a tip for home bakers. Put the latest two journal posts on the home page.
Show our opening hours and address on the contact page and in the footer, from one place, and highlight today's hours on the contact page.

The run record

As we wrote it on 2026-10-13, with local paths and internal references taken out. "I" is the person running the eval.

Template: site-starter 1.0.0, fresh from the packed zip.
Agent: a Claude Sonnet 5.5 subagent with no CodePlate context, on Windows (Git Bash + PowerShell). User turns were relayed one at a time from concept.md and requests.md.
Result: interview + 5/5 requests passed.
- npm run check passed after every request.
- I re-ran it independently at the end: env, types, unit, build, links and e2e all pass.
- The sitemap lists only the bakery's pages: no pricing, changelog or /blog/.
- Components contain no hard-coded hex colors.

#	Request	Turns	Result	Notes
0	Interview	1	pass	Read the brief and logo, changed nothing first. Six questions with defaults; refused to invent testimonials or prices.
1	Re-brand	1	pass	Name, titles, footer, OG image, favicon (badge without text), tokens in both themes. Removed pricing and changelog cleanly. Darkened the brand orange for text contrast and said why. Unasked: renamed the blog to "Journal" but left `/blog/` URLs.
2	Menu	1 (+ the `/journal/` rename)	pass	`src/data/menu.ts` with typed sections, items, prices, labels, weekend-only and an `isPlaceholder` flag. Weekend badge, visible placeholder notice, allergen note. Tests in data and e2e. Moving to `/journal/` touched many files ("a bit more work").
3	Pre-order form	1	pass	Server-side validation (email or phone; Saturday/Sunday only). A Discord-formatted payload with `allowed_mentions` disabled, trimmed to Discord's limit. Console adapter by default; tested the webhook path against a local fake listener.
4	Journal post	1	pass	MDX post with an alt-texted placeholder cover and the Callout component. Home shows the newest two from the collection, with a test that they match the Journal page's top two.
5	Hours + address	1	pass	One source in `src/data/site.ts`, used by home, contact and footer. Today's highlight is computed in the browser in the bakery's time zone (`timezone`, set to Europe/London and flagged to confirm). Tests cover week coverage and the highlight. Reviewed screenshots of all pages, desktop and phone, light and dark, only because the user asked.

Where the template fell short (fixed before merge)

Renaming the blog's URL was spread across many files. The base path is now a single constant, and the add-a-blog-post recipe has a "Rename the blog" section.
The rebrand recipe didn't mention the contrast test. tokens.test.ts already checked the token pairs, but the recipe didn't point to it. The contrast math now lives in src/lib/contrast.ts with its own test, and failures name the pair, its ratio and the fix. Rebrand step 5 says to run the test right after picking colors, and what to do when it fails.
The agent didn't review every changed page in every theme and width unless asked. "Seeing your work" now makes that a rule, and npm run shots has a flag to do it in one call.

Not changed:

AGENTS.md's intro still says "Brightside" after a rebrand. The brief describes the template, not the user's site; the README is the part that gets rewritten.
A unit test keeps "Brightside" as sample input for pageTitle(). It's harmless test data.

What each request was scored against

Every request:

npm run check ends with CODEPLATE CHECKS: PASS (env, types, unit, build, links, e2e).
No test is weakened silently; tests for removed pages are removed or replaced on purpose.
The footer credit "Built with CodePlate (codeplate.io)" stays (free license).
docs/project.md is updated.
The agent looked at its screenshots (light, dark and phone width where layout changed).

Re-brand
- Site name, title template, description, Open Graph and RSS metadata say Rye & Ember; a grep shows only intentional leftovers of "Brightside".
- The logo SVG is used in the header and as the favicon; colors come from the design tokens only (no hard-coded hex in components) and are readable in both themes.
- Pricing and changelog pages are gone: no nav links, no broken links (the link check passes), the changelog collection is removed or unused on purpose, the sitemap doesn't list them.
- Home sections come from src/data/, with placeholders clearly marked.
Menu
- Menu data lives in src/data/ (or a content collection) with a typed schema: section, name, description, price, labels, weekend-only.
- The page groups by section, shows labels and a visible weekend-only marker; it's in the nav and the sitemap.
- A unit or e2e test covers the menu (e.g. every item has a price; weekend items are marked).
Pre-order form
- Fields: name, email or phone (at least one required), pickup day limited to Saturday/Sunday, free text.
- Validated on the server endpoint (not only in the browser), with clear errors in the UI.
- Sends through the webhook adapter (FORM_WEBHOOK_URL), formatted for Discord; the console adapter works without the URL; .env.example documents it; no secret in code.
- Unit tests cover validation and the Discord payload; e2e submits the form.
Journal post
- A new MDX post in the blog collection with valid frontmatter (title, date, description, alt text for the image), the callout component, and it's in RSS.
- Home shows the two newest posts, sorted by date, from the collection (not hard-coded).
Hours and address
- One source of truth in src/data/ (e.g. site.ts or hours.ts), used by the contact page and the footer.
- Today's hours are highlighted on the contact page; the weekday logic is tested, and it doesn't break static builds (computed in the browser or with a clear note on time zones).

← Astro Website Starter · Every eval log