Agent evals · Tauri Desktop Starter
Tauri Desktop Starter: the eval log
We gave Tauri Desktop Starter to a coding agent in a new session with no CodePlate context, along with a made-up user's brief ("Crate"), then sent the requests below one at a time.
5 of 5 requests passed, checks green after every step
- Run
- 2026-10-13
- Agent
- Claude Sonnet 5.5
- The user
- The script, relayed turn by turn
- Version tested
- v1.0.0
- On sale now
- v1.1.4
This run tested v1.0.0. The version on sale is v1.1.4, which includes the fixes listed at the end of the record and has not had its own eval run.
The made-up user
Written the way a buyer would put it. {ASSETS}, {ART} and {GODOT} stand for the folder with their files and the engine's path; their asset pack isn't published.
The user collects vinyl records and builds with an agent. They're comfortable with computers but don't read code.
- App: Crate, a desktop app to catalog a record collection and a wishlist, offline, on their own computer.
- The core thing is a record: artist, album title, year, format (LP, 12", 7", cassette, CD), condition, notes. Genres work like tags.
- Brand: dark plum and amber, warm and a little retro. The icon is in
assets/(see its README). - Platforms: Windows now; a Mac version later. Free for now; maybe sold later.
- Data: the collection matters. They want backups and a spreadsheet export.
First message
Hi! I want to turn this template into "Crate", a desktop app for cataloging my vinyl records. My icon is in
{ASSETS}. Can you take a look and get us started?
Interview answers
- Records replace notes. Fields: artist (required), album title (required), year (optional), format (LP, 12", 7", cassette, CD; default LP), condition (mint, very good, good, fair, poor; optional), notes. Genres are the tags.
- Name "Crate", identifier
com.crate.appis fine. Windows first, Mac later. - Colors: dark plum (#2b1d3a) and amber (#e8a33d). Dark mode should stay.
- Data: the collection must never be lost: keep the backup export.
- Free for now.
The requests
- Make it Crate: records instead of notes (artist, album, year, format, condition, notes; genres as tags), my icon, my colors, and my words everywhere: the window, the menus and the installer name too.
- Let me give each record a rating from 1 to 5 stars, and sort the list by artist, year, rating or recently added. Remember the sort I picked.
- I also want a wishlist: records I'm hunting for, kept apart from the ones I own. Put "Collection" and "Wishlist" in the sidebar, with counts, and let me move a record from the wishlist to the collection once I buy it.
- Add "Export to spreadsheet…" to the File menu: it saves my collection as a CSV file I can open in Excel.
- Add a Stats screen: how many records I own, how many there are of each format and from each decade, and my top 5 genres.
The run record
As we wrote it on 2026-10-13, with local paths and internal references taken out. "I" is the person running the eval.
- Template: desktop-app-starter 1.0.0, fresh from the packed zip.
- Agent: a Claude Sonnet 5.5 subagent with no CodePlate context, on Windows (Git Bash + PowerShell), with Playwright through the installed Edge.
- User turns were relayed one at a time, from
concept.mdandrequests.md.
- User turns were relayed one at a time, from
- Result: interview + 5/5 requests passed.
npm run checkpassed every step after each request, including the real app build and the smoke launch.- I verified it independently at the end: all 11 steps pass, with 55 Rust tests and 20 Playwright tests.
- Screenshots of the list, wishlist and stats looked right in light and dark.
| # | Request | Turns | Result | Notes |
|---|---|---|---|---|
| 0 | Interview | 1 | pass | Read the brief and the stub and changed nothing first. Asked 7 questions with defaults, including identifier and publisher. |
| 1 | Make it Crate | 1 | pass | New migrations 0002 (rename) and 0003 (details); 0001 untouched. Required artist and album; format and condition as enums. Icon regenerated from the SVG. Plum and amber themes. Changed com.crate.app to com.crate.desktop, citing the recipe's .app rule. |
| 2 | Rating + sort | 1 | pass | Migration 0004. The sort is an enum mapped to a fixed ORDER BY, stored as a setting with a serde default. Export bumped to v2; v1 still imports. |
| 3 | Wishlist | 1 | pass | Migration 0005. Sidebar shelves with counts, "I got it" with Undo, filters scoped per shelf. Export v3. Found and fixed a stale-editor bug through its own tests. |
| 4 | CSV export | 1 | pass | Built in core with a UTF-8 BOM, RFC 4180 quoting and formula-injection defusing. Native save dialog command, browser fallback, menu item. Core and e2e tests. |
| 5 | Stats | 1 | pass | SQL in core: by format, by decade (with an unknown-year note), top 5 genres. A route plus a sidebar entry. Readable in both themes and narrow windows. |
Where the template fell short (fixed before merge)
- The rename recipe didn't warn about built-in names.
Note→Recordshadowed TypeScript'sRecord<K, V>. The recipe's step 1 and AGENTS.md's pitfalls now list the taken names and say what to do. - add-a-field only showed an optional free-form field.
- New sections cover choice fields, number fields, required fields and making a field searchable (rebuilding
notes_fts,reindex). - In code,
notes.rsnow reads columns by name, not position.snippet()'s column number is a named constant,FTS_BODY_COLUMN, with a test that it points atbody.
- New sections cover choice fields, number fields, required fields and making a field searchable (rebuilding
- The interview didn't ask for identifier and publisher. It's now question 7, with the
.apprule. - The checks' summary gave no reason for a failure. The runner now repeats the end of each failed step's output after the summary.
- The native menu wasn't checked. A unit test now keeps the menu ids in
src-tauri/src/lib.rsandMenuIdinplatform.tsin step. The macOS branch still can't be compiled on Windows; it's noted in the menu recipe. - Settings leaked between e2e tests. The demo seed (
POST /dev/seed) now resets the settings too, sofresh()gives every test defaults. - Demo counts were hard-coded in tests.
demo::COUNT(Rust) and the count returned byfresh()(e2e) replace the literal 6. - The editor kept a stale copy. It's documented in AGENTS.md's pitfalls, with how to remount it.
Not changed:
- The other recipes still say "note" after a rename. The rename recipe already tells the agent to record the mapping in
docs/project.md. - The
npm run shots404 message on a missing item is fine as it is.
What each request was scored against
Every request:
npm run checkends withCODEPLATE CHECKS: PASS(all eleven steps, including smoke).- No test is weakened silently.
- New commands have tests in
core/tests/. docs/project.mdis updated.- The agent looked at its screenshots.
- Re-theme
noteis renamed torecordthrough the stack:- a NEW migration renames or rebuilds the tables and the FTS index;
0001_init.sqlis untouched - core module, types, commands and
COMMANDS; bindings regenerated;api.ts, components,site.ts, demo data (fixed ids kept), e2e tests - a grep shows only intentional leftovers of "note"
- a NEW migration renames or rebuilds the tables and the FTS index;
- Fields:
- artist and album are required, with an error message in the UI
- year is optional and validated (a sensible range)
- format is a fixed choice with LP as the default
- condition is an optional choice
- notes; genres use the existing tags
- Search covers artist and album.
- The export format still imports, or the change is a deliberate, explained bump of
FORMAT_VERSIONthat still reads old files. - Name, window title, identifier and the menu labels say Crate. The icon is regenerated from the SVG (all sizes and
public/icon.png). The plum and amber tokens are readable in both themes.
- Rating and sort
ratingis 1–5 or empty, stored via a migration and validated in core.- Sort is done in SQL in
list(with a whitelist, no string-built ORDER BY from input) and stored as a setting with a serde default. - The UI shows stars and a sort control. The chosen sort survives a restart.
- Tests cover the sorts and rating validation.
- Wishlist
- Owned vs wanted is stored on the record (a migration).
- The sidebar shows Collection and Wishlist counts and filters by them; tags still filter within them.
- A "Got it" or "Move to collection" action exists.
- New records go to the current view. Export and import carry the flag; old exports default it sensibly.
- Tests cover it.
- CSV export
- Built in Rust (core), with correct escaping of commas, quotes and newlines. A UTF-8 BOM, or an explanation, so Excel shows accents.
- Owned records only, or a stated choice.
- The native save dialog goes through a src-tauri command, with a browser fallback download in
platform.ts. - A menu item with an id handled in
App.tsx. - A core test of escaping.
- Stats
- Counts are computed in core with SQL: by format, by decade (with an "unknown year" bucket), and the top 5 tags.
- A command with a test, and a route/screen reachable from the sidebar.
- Readable in both themes and in a small window.