Browser Extension Starter: the eval log

We gave Browser Extension Starter to a coding agent in a new session with no CodePlate context, along with a made-up user's brief ("Price Watch"), then sent the requests below one at a time.

5 of 5 requests passed

Run: 2026-10-14
Agent: Claude Sonnet 5.5
The user: The script, relayed turn by turn
Version tested: v1.0.0
On sale now: v1.0.3

This run tested v1.0.0. The version on sale is v1.0.3, which includes the fixes listed at the end of the record and has not had its own eval run.

The made-up user

Written the way a buyer would put it. {ASSETS}, {ART} and {GODOT} stand for the folder with their files and the engine's path; their asset pack isn't published.

The user shops online a lot and builds with an agent. They're comfortable with computers and browsers but don't read code.

Extension: Price Watch, for Chrome and Edge. Pitch: "Mark a product, and know when its price drops."
What it does: on a shopping page, the user selects the product's price (or right-clicks it) and chooses "Watch this price". The extension remembers the product (page title, address, the price and currency, the date). When the user visits that product page again, it reads the price again, records it, and shows whether it went up or down. The popup lists every watched product with its first, lowest and current price; the side panel shows one product's price history.
Which sites: any shopping site the user marks a product on. No site list, no background scraping, no visits the user didn't make.
Data: everything stays in the browser (local only). No accounts, no server, no analytics. Prices, product titles and addresses are the only things stored.
Browsers: Chrome and Edge now; Firefox maybe later. They have a Chrome Web Store developer account.
Brand: teal (#0f766e) with an orange accent (#f97316) for price drops; plain and trustworthy. Their icon is in assets/ (see its README).

First message

Hi! I want to turn this template into "Price Watch", a browser extension that keeps track of prices of products I mark on shopping sites. My icon is in {ASSETS}. Can you take a look and get us started?

Interview answers

Price Watch: "Mark a product, and know when its price drops." For people who wait for a better price.
Parts: something on the page (mark a price; see "down 12% since you marked it"), the popup (all watched products), the side panel (one product's history), a right-click "Watch this price", settings for which drop counts (default 5%) and showing the badge.
Sites: whatever shopping page they mark; it should only read prices on product pages they've marked.
Data: local only. Prices, product title and address. Nothing leaves the browser. No record of other pages.
Browsers: Chrome and Edge. Firefox later, not now.
Brand: teal and orange as above, light and dark. Icon: the SVG in the assets folder. Tone: short, factual.

The requests

Make it Price Watch: my name, icon and colors everywhere (toolbar icon, popup, side panel, options, the store images), and our words. Take out everything about highlights and notes that doesn't fit, but keep the parts we'll reuse. Show me the screenshots.
Let me watch a price: on a product page I select the price (or right-click it) and choose "Watch this price". Save the product's name, address, the price as a number with its currency, and today's date. If the text I picked isn't a price, tell me on the page instead of saving junk. Use a test product page to prove it works.
When I open a watched product page again, read the current price and add it to the product's history (at most once a day). Show a small badge on the page next to the price: "▼ 12% since 3 Oct" in orange when it's lower, grey when it's the same or higher. If the price can't be found any more, say so in the popup instead of failing.
The popup should list my watched products: name, current price, the lowest price seen and the change since I marked it, sorted by the biggest drop. The side panel shows one product's price history as a simple chart and a table, and lets me stop watching it. Add an export of everything as CSV.
Add settings: the minimum drop that counts as "a drop" (default 5%), and whether to show the badge on pages. Settings saved before this change must keep working. Then get me ready to publish: the store zip, the screenshots with my product data, and the privacy answers I need to fill in.

The run record

As we wrote it on 2026-10-14, with local paths and internal references taken out. "I" is the person running the eval.

Template: browser-extension-starter 1.0.0, fresh from the packed zip, with the Price Watch icon from assets/.
Agent: a Claude Sonnet 5.5 subagent with no CodePlate context, on Windows, with Playwright through Edge. User turns were relayed one at a time from concept.md and requests.md.
Result: interview + 5/5 requests passed. I re-ran npm run check independently at the end, and all eight steps pass. The real right-click menu, the keyboard shortcuts and the browser's permission prompt can't be driven by automation, so they're untested by design.

#	Request	Turns	Result	Notes
0	Interview	1	pass	Six questions with defaults. Proposed the least-alarming permission model: no site access at install, optional per-site access.
1	Rebrand	1	pass	Icons rendered from the SVG; popup, side panel and options rebuilt for products. Removed the highlight machinery but kept messaging, storage, the single writer and the side panel. Looked at everything in light and dark. The default shortcut showed "Not set" in Edge, and the 16 px icon is soft.
2	Watch a price	1	pass	Selection plus context menu, shortcut or popup button. A robust price parser handles many formats and rejects 8 kinds of junk with an on-page message. Uses `activeTab` + `scripting`. Had to invent an e2e-only build (host permissions for localhost/*.test), because activeTab flows can't be granted in automation.
3	Revisit + badge	1	pass	Reads the price at most once a day, places the badge carefully, and remembers the element with a JSON-LD/meta fallback. Optional per-site host permission requested at first watch, and content scripts registered dynamically only for watched URLs. Shows a "price gone" notice. The template has no pattern for optional per-site access.
4	List, chart, CSV	1	pass	Sorted by biggest drop; lowest price comes from history (tested); chart plus table; two-click stop. The CSV has a BOM and defuses formula injection. Fixed an e2e slowdown: Edge's downloads tab won't close.
5	Settings + publish prep	1	pass	Settings v2 migration (unversioned and v1 → v2, per-field reset). Store zip verified; store listing and privacy-policy drafts. Moved permission requests into user-gesture paths. The zip still carried the template's placeholder `homepage`.

Where the template fell short (fixed before merge)

E2E-only build. The template now ships a test build that adds host permissions for localhost and *.test. It's never zipped, and the manifest check verifies it never leaks into the store build.
Optional per-site access. A helper and recipe now cover optional_host_permissions, requesting them inside a user gesture, registering content scripts for specific URLs only, and revoking cleanly.
Store prep. npm run zip refuses a placeholder homepage. The template generates docs/store-listing.md and docs/privacy-policy.md drafts from the manifest's permissions.
Shortcuts. The defaults are chosen not to clash with Chrome or Edge, and the brief explains why a shortcut shows "Not set".
Icons. npm run icons accepts an optional simplified small-size source for the 16/32 px icons.
The e2e harness handles download tabs.

What each request was scored against

Every request:

npm run check ends with CODEPLATE CHECKS: PASS (env, assets, typecheck, lint, unit, build, manifest, e2e).
No test is weakened silently; tests for removed features are removed or replaced on purpose, and new behavior gets tests.
The options page's About keeps "Built with CodePlate (codeplate.io)" (free license); its tests still pass.
docs/project.md is updated (the interview answers first, then each decision).
The agent took screenshots (npm run shots), light and dark, and says it looked at them.
Permissions stay minimal: every permission and host pattern has a reason in REASONS (src/manifest.ts); no tabs, history or <all_urls> is added without a reason the user agreed to.
No remote code, no innerHTML with page text, no secrets.

Rebrand
- src/shared/brand.ts, the package.json name, the README and the manifest say Price Watch; a grep shows only intentional leftovers of "Marginalia" (internal ids it explains).
- The icon PNGs (16/32/48/128) come from the user's SVG (rendered, or faithfully redrawn in scripts/make-icons.mjs); assets.json and CREDITS.md record them; the 16 px icon is legible.
- Tokens in src/ui/styles.css and src/content/ui.css.ts use teal/orange in both themes; the contrast test passes without being loosened.
- Highlight-specific UI that doesn't fit is removed or clearly repurposed; the store images (npm run store) show Price Watch.
Watch a price
- The data model is new and typed (e.g. src/shared/products.ts): id, page key, title, currency, a history of { price, at }; writes go through the service worker with a typed BackgroundApi message.
- Price parsing is a pure, unit-tested function covering at least "$1,299.99", "1.299,99 €", "£12", "12 USD", and rejecting non-prices.
- The selection toolbar or the right-click menu ("Watch this price", contexts: ['selection']) starts it; a non-price shows an in-page message (shadow-root UI) and nothing is stored.
- A test product page in tests/pages/ (original content) and an e2e test mark a price and check storage.
Re-read on visit
- On load of a watched page (content script, no background polling), the price is found again (text anchor with context, or a stored selector) and a history point is added at most once a day (tested with an injectable clock).
- The badge is drawn without breaking the page (shadow root or the highlight API), orange for a drop beyond the threshold, grey otherwise; the date uses the user's locale.
- A price that can't be found is recorded as a status, shown in the popup, and doesn't throw.
- E2E: a second visit with a changed price on the test page adds history and shows the badge.
Popup, side panel, export
- The popup list is sorted by biggest drop and shows current, lowest and change; the empty state explains how to start.
- Side panel: one product's history as a chart drawn by the extension's own code (SVG or canvas, no chart CDN) plus a table; "Stop watching" removes it through the service worker.
- CSV export with a header row, escaped commas and quotes, ISO dates; unit-tested.
- Screenshots of the popup and side panel in both themes, numbers readable.
Settings, migration, publishing
- New settings in src/shared/settings.ts with defaults; if a stored shape changed, SETTINGS_VERSION is bumped with a migration, and a unit test proves older settings still load.
- The options page shows both settings; the content script respects them (badge hidden when off, threshold applied).
- npm run zip produces the upload with no manifest FAILs; npm run store images use the user's product demo data (scripts/lib/demo.mjs, test pages).
- docs/project.md (or the reply) gives the single purpose, a justification per permission (from REASONS), "no remote code", and the data-usage answers (nothing collected; local only), noting a privacy policy isn't required for local-only data but is recommended.

← Browser Extension Starter · Every eval log