CodePlate home

Agent evals · Typst Invoices & Quotes

Typst Invoices & Quotes: the eval log

We gave Typst Invoices & Quotes to a coding agent in a new session with no CodePlate context, along with a made-up user's brief ("Marta Kowalczyk Translations"), then sent the requests below one at a time.

5 of 5 requests passed

Run
2026-10-14
Agent
Claude Sonnet 5.5
The user
The script, relayed turn by turn
Version tested
v1.0.0
On sale now
v1.1.4

This run tested v1.0.0. The version on sale is v1.1.4, which includes the fixes listed at the end of the record and has not had its own eval run.

The made-up user

Written the way a buyer would put it. {ASSETS}, {ART} and {GODOT} stand for the folder with their files and the engine's path; their asset pack isn't published.

The user is a freelance translator (Polish ⇄ English) in Edinburgh who builds with an agent. She keeps her jobs in a spreadsheet and writes invoices by hand in a word processor, which takes an evening a month and sometimes has typos in the totals. She doesn't read code. She has Node and Typst installed (the eval machine has both; if the agent can't find Typst, that's a finding).

First message

Hi! I'm a freelance translator and I'm tired of writing invoices by hand. This template looks like what I need. My logo, my October jobs and my client list are in {ASSETS}. Can you set it up for me?

Interview answers

The requests

  1. Make it mine: my name, address, logo and colours, no VAT anywhere, 14-day terms, my bank details, and invoice numbers that carry on from 2026-118. Take out the Fernhill examples.
  2. Make my October invoices from jobs-october.csv: one invoice per client, dated 31 October 2026, every job a line with the word count and the per-word rate, in each client's currency, with their PO number where there is one. Number them from 2026-119 in the order of each client's first job. I'll send a new file like this every month, so make it a one-command import.
  3. I have a minimum fee per job: £35 (or €40, or 150 zł). If a job comes to less, charge the minimum instead and say "minimum fee" on that line. Redo October with it.
  4. Wydawnictwo Lato want their invoices in Polish (and in złoty, as now). Everyone else stays in English.
  5. I overcharged Wydawnictwo Lato on the chapters 1–3 job: it was 17,700 words, not 18,200. That invoice has been sent, so don't change it: make a credit note for the difference that refers to it.

The run record

As we wrote it on 2026-10-14, with local paths and internal references taken out. "I" is the person running the eval.

#RequestTurnsResultNotes
0Interview2passFound Typst first, asked 13 questions: tax (and that it can't advise on it), numbering, terms, bank details, home currency, Polish wording, day-first dates, jobs dated after "today", per-client vs per-job, PO numbers, the rush rate, client legal names. Then built all four October invoices in the same turn.
1Make it mine1passEverything was already in place; cleaned up leftover sample screenshots in docs/media and the manifest's media, and rewrote one line of AGENTS.md. Left the frozen test fixture alone, as the rules say.
2October import, one command1passThe template has no job/CSV importer: the agent wrote scripts/import-jobs.mjs (grouping per client, numbering in order of first job, idempotent, validation errors with line numbers). Numbering continued from the highest number in data/documents.csv, so deleting rows would restart at 2026-001.
3Minimum fee1passNo minimum-fee or per-line note support: added minimumFee per currency in business.json and a "minimum fee" note under the line. Changing it later would also have changed already-sent invoices (see 5).
4Polish for one client1passLabels were English-only, with no per-client language: added a language column, a Polish label set, Polish date and number formats. Flagged that a Polish speaker should review the wording.
5Credit note + freeze sent invoices1passNo credit notes and no notion of a sent invoice: added a CN- series that must reference an existing invoice, and npm run send <no>, which archives the data and PDF; checks fail if an archived invoice's rows change. Verified by editing the phone number and the minimum fee.

Where the template fell short (fixed before merge)

  1. No importer for line items. Ship a one-command import (CSV/JSON → documents), grouped per client, idempotent, with a recipe and tests.
  2. Sent documents weren't immutable. PDFs are rebuilt from current settings, so an address, rate or rule change silently rewrote sent invoices. Ship an "issue/send" step that freezes data and PDF, and a check that a frozen document is unchanged.
  3. No credit notes. Ship a credit-note document type with its own series that must reference an issued invoice.
  4. English only. Ship per-client language with label sets (at least en, plus a documented way to add one), localized dates and numbers.
  5. No line-level rules or notes. Ship a minimum charge per line (per currency) and a line note field.
  6. Numbering depended on surviving rows. Store the last number per series so deleting a draft can't restart numbering.
  7. Stale PDFs in out/. The sample's 2026-001…004 PDFs stayed in out/ after the sample data was removed; the build should remove outputs whose documents no longer exist.
  8. Removing the sample was manual. Ship a "remove the sample" command or recipe that lists every sample-only file (data, docs/media, manifest media).
What each request was scored against

Every request:

  • npm run check ends with CODEPLATE CHECKS: PASS (tools, assets, data, unit, pdf, layout, preview, credit, budget), run in full after the change.
  • No check is weakened, skipped or deleted to pass; the <cp-…> markers are still in the layout; no Typst warnings.
  • Money math stays in src/money.mjs with integers (no floats, no arithmetic in .typ files); new rules have tests in tests/.
  • The agent ran npm run preview and opened the page images for every document it changed, and said what it saw.
  • docs/project.md is updated (the stub marker is gone after the interview).
  • README.md still carries "Built with CodePlate (codeplate.io)"; no npm or Typst Universe packages were added.
  • The agent gives no tax advice beyond what the user asked for (it may say what's commonly required and suggest checking).
  1. Make it mine
    • business.json has her name, address, contacts and colours; legalName is her name (sole trader); no registration line; no tax section, and the pages show no Tax column and no VAT lines.
    • assets/marta-kowalczyk-logo.svg (or a renamed copy) is in assets/, business.json logo points at it, assets.json lists it with width: 320, height: 80 (the viewBox) and a credit to her business; CREDITS.md updated; the Fernhill logo is removed from assets/ and assets.json if unused.
    • The wide logo looks right in the header (not 15 mm tall and stretched across the page; rebrand.md's width tip or similar).
    • documents.invoice.numbering is {yyyy}-{nnn}, dueDays: 14, gapless: true; the quote type removed or left unused, by her choice.
    • Payment lines and the reference wording are hers.
    • The Fernhill sample clients and documents are gone from data/; tests/fixtures/sample/ is untouched and npm test still passes.
  2. October import
    • A converter (scripts/import-*.mjs or similar, Node built-ins and src/csv.mjs only) with an npm script; it maps DD/MM/YYYY, client names to ids, groups by client, keeps 4-decimal rates, puts the PO in reference, and doesn't duplicate documents when run twice (a test or a demonstrated second run).
    • data/clients.csv has the four clients from clients.txt with addresses, tax numbers and currencies (Lato PLN, Kramer EUR, the others GBP by default).
    • Four invoices 2026-119 (Northlight, £624.76), 2026-120 (Lato, PLN 9,420.20 in en-GB formatting), 2026-121 (Kramer, €376.20), 2026-122 (Dr Nowak, £143.10), all issued 2026-10-31, due 2026-11-14; each job is one line with words as the quantity, "word" (or similar) as the unit and the rate as the unit price (£0.0875).
    • npm run list shows those totals exactly.
  3. Minimum fee
    • The rule lives in src/money.mjs (or is applied in src/load.mjs through a money function) per currency, configured in data (business.json), not hard-coded in the layout; tests cover under, at, and over the minimum and each currency.
    • Dr Nowak's 250-word job is charged £35.00 and Lato's blurb 150.00 zł, each marked "minimum fee" on its line (from labels.json); totals become £155.60 and PLN 9,498.00; other invoices unchanged.
    • The agent pointed out that 2026-120 and 2026-122 change and asked or noted whether they had been sent (rule 4).
  4. Polish for Lato
    • A Polish label block in layout/labels/<lang>.json covering every key; a per-client language (and locale pl-PL) wired through src/load.mjs with a test; only Lato's documents use it.
    • Lato's invoice reads "Faktura" (or the word she approves), dates like "31 października 2026", amounts like "9498,00 zł" (Polish groups only from five digits); no English left on that page; the other invoices are still English.
    • The agent suggested that Marta check the Polish legal wording (or noted it can't vouch for it).
  5. Credit note
    • A credit-note type (or similar) with its own numbering (e.g. CN-{yyyy}-{nnn}), labels in English and Polish, no payment block (or a deliberate one).
    • One credit note for Lato in PLN: 500 words × 0.38 = -190,00 zł (negative, Polish formatting), referring to invoice 2026-120; invoice 2026-120 itself is unchanged.
    • Tests cover the new type loading and the negative total; the page reads clearly as a credit (title, total wording).

← Typst Invoices & Quotes · Every eval log