CodePlate home

Agent evals · Tauri Desktop Starter

Tauri Desktop Starter: the eval log

We gave Tauri Desktop Starter to a coding agent in a new session with no CodePlate context, along with a made-up user's brief ("Crate"), then sent the requests below one at a time.

5 of 5 requests passed, checks green after every step

Run
2026-10-13
Agent
Claude Sonnet 5.5
The user
The script, relayed turn by turn
Version tested
v1.0.0
On sale now
v1.1.4

This run tested v1.0.0. The version on sale is v1.1.4, which includes the fixes listed at the end of the record and has not had its own eval run.

The made-up user

Written the way a buyer would put it. {ASSETS}, {ART} and {GODOT} stand for the folder with their files and the engine's path; their asset pack isn't published.

The user collects vinyl records and builds with an agent. They're comfortable with computers but don't read code.

First message

Hi! I want to turn this template into "Crate", a desktop app for cataloging my vinyl records. My icon is in {ASSETS}. Can you take a look and get us started?

Interview answers

The requests

  1. Make it Crate: records instead of notes (artist, album, year, format, condition, notes; genres as tags), my icon, my colors, and my words everywhere: the window, the menus and the installer name too.
  2. Let me give each record a rating from 1 to 5 stars, and sort the list by artist, year, rating or recently added. Remember the sort I picked.
  3. I also want a wishlist: records I'm hunting for, kept apart from the ones I own. Put "Collection" and "Wishlist" in the sidebar, with counts, and let me move a record from the wishlist to the collection once I buy it.
  4. Add "Export to spreadsheet…" to the File menu: it saves my collection as a CSV file I can open in Excel.
  5. Add a Stats screen: how many records I own, how many there are of each format and from each decade, and my top 5 genres.

The run record

As we wrote it on 2026-10-13, with local paths and internal references taken out. "I" is the person running the eval.

#RequestTurnsResultNotes
0Interview1passRead the brief and the stub and changed nothing first. Asked 7 questions with defaults, including identifier and publisher.
1Make it Crate1passNew migrations 0002 (rename) and 0003 (details); 0001 untouched. Required artist and album; format and condition as enums. Icon regenerated from the SVG. Plum and amber themes. Changed com.crate.app to com.crate.desktop, citing the recipe's .app rule.
2Rating + sort1passMigration 0004. The sort is an enum mapped to a fixed ORDER BY, stored as a setting with a serde default. Export bumped to v2; v1 still imports.
3Wishlist1passMigration 0005. Sidebar shelves with counts, "I got it" with Undo, filters scoped per shelf. Export v3. Found and fixed a stale-editor bug through its own tests.
4CSV export1passBuilt in core with a UTF-8 BOM, RFC 4180 quoting and formula-injection defusing. Native save dialog command, browser fallback, menu item. Core and e2e tests.
5Stats1passSQL in core: by format, by decade (with an unknown-year note), top 5 genres. A route plus a sidebar entry. Readable in both themes and narrow windows.

Where the template fell short (fixed before merge)

  1. The rename recipe didn't warn about built-in names. Note→Record shadowed TypeScript's Record<K, V>. The recipe's step 1 and AGENTS.md's pitfalls now list the taken names and say what to do.
  2. add-a-field only showed an optional free-form field.
    • New sections cover choice fields, number fields, required fields and making a field searchable (rebuilding notes_fts, reindex).
    • In code, notes.rs now reads columns by name, not position. snippet()'s column number is a named constant, FTS_BODY_COLUMN, with a test that it points at body.
  3. The interview didn't ask for identifier and publisher. It's now question 7, with the .app rule.
  4. The checks' summary gave no reason for a failure. The runner now repeats the end of each failed step's output after the summary.
  5. The native menu wasn't checked. A unit test now keeps the menu ids in src-tauri/src/lib.rs and MenuId in platform.ts in step. The macOS branch still can't be compiled on Windows; it's noted in the menu recipe.
  6. Settings leaked between e2e tests. The demo seed (POST /dev/seed) now resets the settings too, so fresh() gives every test defaults.
  7. Demo counts were hard-coded in tests. demo::COUNT (Rust) and the count returned by fresh() (e2e) replace the literal 6.
  8. The editor kept a stale copy. It's documented in AGENTS.md's pitfalls, with how to remount it.

Not changed:

What each request was scored against

Every request:

  • npm run check ends with CODEPLATE CHECKS: PASS (all eleven steps, including smoke).
  • No test is weakened silently.
  • New commands have tests in core/tests/.
  • docs/project.md is updated.
  • The agent looked at its screenshots.
  1. Re-theme
    • note is renamed to record through the stack:
      • a NEW migration renames or rebuilds the tables and the FTS index; 0001_init.sql is untouched
      • core module, types, commands and COMMANDS; bindings regenerated; api.ts, components, site.ts, demo data (fixed ids kept), e2e tests
      • a grep shows only intentional leftovers of "note"
    • Fields:
      • artist and album are required, with an error message in the UI
      • year is optional and validated (a sensible range)
      • format is a fixed choice with LP as the default
      • condition is an optional choice
      • notes; genres use the existing tags
    • Search covers artist and album.
    • The export format still imports, or the change is a deliberate, explained bump of FORMAT_VERSION that still reads old files.
    • Name, window title, identifier and the menu labels say Crate. The icon is regenerated from the SVG (all sizes and public/icon.png). The plum and amber tokens are readable in both themes.
  2. Rating and sort
    • rating is 1–5 or empty, stored via a migration and validated in core.
    • Sort is done in SQL in list (with a whitelist, no string-built ORDER BY from input) and stored as a setting with a serde default.
    • The UI shows stars and a sort control. The chosen sort survives a restart.
    • Tests cover the sorts and rating validation.
  3. Wishlist
    • Owned vs wanted is stored on the record (a migration).
    • The sidebar shows Collection and Wishlist counts and filters by them; tags still filter within them.
    • A "Got it" or "Move to collection" action exists.
    • New records go to the current view. Export and import carry the flag; old exports default it sensibly.
    • Tests cover it.
  4. CSV export
    • Built in Rust (core), with correct escaping of commas, quotes and newlines. A UTF-8 BOM, or an explanation, so Excel shows accents.
    • Owned records only, or a stated choice.
    • The native save dialog goes through a src-tauri command, with a browser fallback download in platform.ts.
    • A menu item with an id handled in App.tsx.
    • A core test of escaping.
  5. Stats
    • Counts are computed in core with SQL: by format, by decade (with an "unknown year" bucket), and the top 5 tags.
    • A command with a test, and a route/screen reachable from the sidebar.
    • Readable in both themes and in a small window.

← Tauri Desktop Starter · Every eval log