From 5edb1f04f62b86c6b5477e218492e26e3873334b Mon Sep 17 00:00:00 2001 From: Sjoerd de Vries Date: Sat, 5 Sep 2026 14:35:57 +0200 Subject: [PATCH] Add implementation plan for korting prijstracker Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_013gwc3aCiyQxmCWQke8RvEz --- .../plans/2026-09-05-korting-prijstracker.md | 1332 +++++++++++++++++ 1 file changed, 1332 insertions(+) create mode 100644 docs/superpowers/plans/2026-09-05-korting-prijstracker.md diff --git a/docs/superpowers/plans/2026-09-05-korting-prijstracker.md b/docs/superpowers/plans/2026-09-05-korting-prijstracker.md new file mode 100644 index 0000000..9eaf2a6 --- /dev/null +++ b/docs/superpowers/plans/2026-09-05-korting-prijstracker.md @@ -0,0 +1,1332 @@ +# Korting prijstracker Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Build a Node.js/Express web app that lets you manage a list of product URLs, checks their price daily, and emails when a price changes. + +**Architecture:** One Express app in one process. `node-cron` runs a daily job in-process; the same check logic is reachable via a "check now" button in the UI. SQLite (`better-sqlite3`) stores products and their last known price. Price extraction tries JSON-LD, then meta tags, then a regex fallback. Email goes out via `nodemailer` over SMTP to the existing Postal server at `mail.sjoerd.app`. + +**Tech Stack:** Node.js 22, Express, better-sqlite3, cheerio, nodemailer, node-cron, express-basic-auth. Tests via Node's built-in `node:test` + `supertest` (devDependency only). + +**Spec:** `docs/superpowers/specs/2026-09-05-korting-prijstracker-design.md` + +## Global Constraints + +- Node.js 22, CommonJS modules (`require`/`module.exports`), no TypeScript, no bundler. +- One SQLite file at `DB_PATH` (default `./data/korting.db` locally; `/data/korting.db` on the Coolify volume in production). +- Daily cron schedule `0 8 * * *`, timezone `Europe/Amsterdam`. +- `last_price` and `last_checked_at` are updated **only** on a fully successful check (price found, and email either not needed or sent successfully). They are left untouched on `fetch_error`, `not_found`, or `mail_error`, so a failure is retried automatically the next day. +- Every route is protected by basic auth (`BASIC_AUTH_USER` / `BASIC_AUTH_PASS`). +- SMTP env vars: `SMTP_HOST`, `SMTP_PORT`, `SMTP_USER`, `SMTP_PASS`, `SMTP_FROM`, `NOTIFY_EMAIL` (recipient). +- Test runner: Node's built-in `node:test` (`node --test test/`). The only devDependency is `supertest`, used solely for the Express app tests. + +--- + +### Task 1: Project scaffolding + +**Files:** +- Create: `package.json` +- Create: `.gitignore` +- Create: `.env.example` +- Create: `test/smoke.test.js` + +**Interfaces:** +- Produces: `npm test` runs `node --test test/`; `npm start` runs `node server.js` (server.js created in Task 8). + +- [ ] **Step 1: Create `package.json`** + +```json +{ + "name": "korting", + "version": "1.0.0", + "private": true, + "description": "Dagelijkse prijstracker die mailt bij prijswijzigingen", + "main": "server.js", + "scripts": { + "start": "node server.js", + "test": "node --test test/" + }, + "dependencies": { + "better-sqlite3": "^11.3.0", + "cheerio": "^1.0.0", + "express": "^4.19.2", + "express-basic-auth": "^1.2.1", + "node-cron": "^3.0.3", + "nodemailer": "^6.9.14" + }, + "devDependencies": { + "supertest": "^7.0.0" + } +} +``` + +- [ ] **Step 2: Install dependencies** + +Run: `npm install` +Expected: completes with no errors, creates `node_modules/` and `package-lock.json`. + +- [ ] **Step 3: Create `.gitignore`** + +``` +node_modules/ +data/ +.env +``` + +- [ ] **Step 4: Create `.env.example`** + +``` +PORT=3000 +DB_PATH=./data/korting.db +BASIC_AUTH_USER=admin +BASIC_AUTH_PASS=change-me +SMTP_HOST=mail.sjoerd.app +SMTP_PORT=587 +SMTP_USER= +SMTP_PASS= +SMTP_FROM=korting@sjoerd.app +NOTIFY_EMAIL=openai@sjoerddev.com +``` + +- [ ] **Step 5: Create `test/smoke.test.js`** + +```js +const test = require('node:test'); +const assert = require('node:assert/strict'); + +test('smoke test runs', () => { + assert.ok(true); +}); +``` + +- [ ] **Step 6: Run the test suite** + +Run: `npm test` +Expected: PASS (1 test) + +- [ ] **Step 7: Commit** + +```bash +git add package.json package-lock.json .gitignore .env.example test/smoke.test.js +git commit -m "chore: project scaffolding" +``` + +--- + +### Task 2: Database module + +**Files:** +- Create: `src/db.js` +- Test: `test/db.test.js` + +**Interfaces:** +- Produces: + - `openDb(dbPath: string) -> Database` — creates parent dir (unless `:memory:`), ensures schema, returns a `better-sqlite3` instance. + - `listProducts(db) -> Array` ordered by `created_at` ascending. + - `getProduct(db, id) -> Product | undefined` + - `addProduct(db, { name, url }) -> Product` (sets `created_at`, `last_price: null`, `last_checked_at: null`) + - `deleteProduct(db, id) -> void` + - `recordCheckSuccess(db, id, { price, checkedAt }) -> void` — sets `last_price`, `last_checked_at`, `last_check_status = 'ok'` + - `recordCheckFailure(db, id, status: string) -> void` — sets only `last_check_status` + - `Product` shape: `{ id, name, url, last_price, last_checked_at, last_check_status, created_at }` + +- [ ] **Step 1: Write the failing tests** + +Create `test/db.test.js`: + +```js +const test = require('node:test'); +const assert = require('node:assert/strict'); +const { + openDb, listProducts, getProduct, addProduct, deleteProduct, + recordCheckSuccess, recordCheckFailure, +} = require('../src/db'); + +function freshDb() { + return openDb(':memory:'); +} + +test('addProduct inserts a row with defaults and returns it', () => { + const db = freshDb(); + const product = addProduct(db, { name: 'Voorbeeld', url: 'https://example.com' }); + assert.equal(product.name, 'Voorbeeld'); + assert.equal(product.url, 'https://example.com'); + assert.equal(product.last_price, null); + assert.equal(product.last_checked_at, null); + assert.ok(product.created_at); + assert.ok(product.id); +}); + +test('listProducts returns all inserted products in insertion order', () => { + const db = freshDb(); + addProduct(db, { name: 'A', url: 'https://a' }); + addProduct(db, { name: 'B', url: 'https://b' }); + const products = listProducts(db); + assert.equal(products.length, 2); + assert.equal(products[0].name, 'A'); + assert.equal(products[1].name, 'B'); +}); + +test('getProduct returns undefined for an unknown id', () => { + const db = freshDb(); + assert.equal(getProduct(db, 999), undefined); +}); + +test('deleteProduct removes the row', () => { + const db = freshDb(); + const product = addProduct(db, { name: 'A', url: 'https://a' }); + deleteProduct(db, product.id); + assert.equal(getProduct(db, product.id), undefined); +}); + +test('recordCheckSuccess updates price, timestamp and status', () => { + const db = freshDb(); + const product = addProduct(db, { name: 'A', url: 'https://a' }); + recordCheckSuccess(db, product.id, { price: 12.5, checkedAt: '2026-09-05T08:00:00.000Z' }); + const updated = getProduct(db, product.id); + assert.equal(updated.last_price, 12.5); + assert.equal(updated.last_checked_at, '2026-09-05T08:00:00.000Z'); + assert.equal(updated.last_check_status, 'ok'); +}); + +test('recordCheckFailure only updates the status, leaving price and timestamp untouched', () => { + const db = freshDb(); + const product = addProduct(db, { name: 'A', url: 'https://a' }); + recordCheckSuccess(db, product.id, { price: 12.5, checkedAt: '2026-09-05T08:00:00.000Z' }); + recordCheckFailure(db, product.id, 'fetch_error'); + const updated = getProduct(db, product.id); + assert.equal(updated.last_price, 12.5); + assert.equal(updated.last_checked_at, '2026-09-05T08:00:00.000Z'); + assert.equal(updated.last_check_status, 'fetch_error'); +}); +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `npm test` +Expected: FAIL with "Cannot find module '../src/db'" + +- [ ] **Step 3: Write `src/db.js`** + +```js +const Database = require('better-sqlite3'); +const fs = require('node:fs'); +const path = require('node:path'); + +const SCHEMA = ` +CREATE TABLE IF NOT EXISTS products ( + id INTEGER PRIMARY KEY AUTOINCREMENT, + name TEXT NOT NULL, + url TEXT NOT NULL, + last_price REAL, + last_checked_at TEXT, + last_check_status TEXT, + created_at TEXT NOT NULL +)`; + +function openDb(dbPath) { + if (dbPath !== ':memory:') { + fs.mkdirSync(path.dirname(dbPath), { recursive: true }); + } + const db = new Database(dbPath); + db.exec(SCHEMA); + return db; +} + +function listProducts(db) { + return db.prepare('SELECT * FROM products ORDER BY created_at ASC').all(); +} + +function getProduct(db, id) { + return db.prepare('SELECT * FROM products WHERE id = ?').get(id); +} + +function addProduct(db, { name, url }) { + const createdAt = new Date().toISOString(); + const info = db.prepare( + 'INSERT INTO products (name, url, created_at) VALUES (?, ?, ?)' + ).run(name, url, createdAt); + return getProduct(db, info.lastInsertRowid); +} + +function deleteProduct(db, id) { + db.prepare('DELETE FROM products WHERE id = ?').run(id); +} + +function recordCheckSuccess(db, id, { price, checkedAt }) { + db.prepare( + 'UPDATE products SET last_price = ?, last_checked_at = ?, last_check_status = ? WHERE id = ?' + ).run(price, checkedAt, 'ok', id); +} + +function recordCheckFailure(db, id, status) { + db.prepare('UPDATE products SET last_check_status = ? WHERE id = ?').run(status, id); +} + +module.exports = { + openDb, listProducts, getProduct, addProduct, deleteProduct, + recordCheckSuccess, recordCheckFailure, +}; +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `npm test` +Expected: PASS (all `db.test.js` tests + smoke test) + +- [ ] **Step 5: Commit** + +```bash +git add src/db.js test/db.test.js +git commit -m "feat: add SQLite-backed products store" +``` + +--- + +### Task 3: Scraper module + +**Files:** +- Create: `src/scraper.js` +- Create: `test/fixtures/json-ld.html` +- Create: `test/fixtures/meta-tag.html` +- Create: `test/fixtures/regex-fallback.html` +- Create: `test/fixtures/no-price.html` +- Test: `test/scraper.test.js` + +**Interfaces:** +- Produces: + - `extractPrice(html: string) -> { price: number | null, method: 'json-ld' | 'meta' | 'regex' | null }` + - `fetchAndExtractPrice(url: string, { fetchImpl? }) -> Promise<{ price: number | null, method: string | null, error: Error | null }>` — `fetchImpl` defaults to the global `fetch`; never throws, errors are returned on the `error` field. + +- [ ] **Step 1: Create fixture files** + +`test/fixtures/json-ld.html`: + +```html + + +

Voorbeeld product

+``` + +`test/fixtures/meta-tag.html`: + +```html + + +

Voorbeeld product

+``` + +`test/fixtures/regex-fallback.html`: + +```html +

Onze prijs: €14,95 vandaag

+``` + +`test/fixtures/no-price.html`: + +```html +

Geen prijs op deze pagina.

+``` + +- [ ] **Step 2: Write the failing tests** + +Create `test/scraper.test.js`: + +```js +const test = require('node:test'); +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const path = require('node:path'); +const { extractPrice, fetchAndExtractPrice } = require('../src/scraper'); + +function loadFixture(name) { + return fs.readFileSync(path.join(__dirname, 'fixtures', name), 'utf8'); +} + +test('extractPrice reads price from JSON-LD Product offers', () => { + const result = extractPrice(loadFixture('json-ld.html')); + assert.deepEqual(result, { price: 19.99, method: 'json-ld' }); +}); + +test('extractPrice reads price from og:price:amount meta tag', () => { + const result = extractPrice(loadFixture('meta-tag.html')); + assert.deepEqual(result, { price: 24.50, method: 'meta' }); +}); + +test('extractPrice falls back to euro-amount regex in page text', () => { + const result = extractPrice(loadFixture('regex-fallback.html')); + assert.deepEqual(result, { price: 14.95, method: 'regex' }); +}); + +test('extractPrice returns null price when nothing matches', () => { + const result = extractPrice(loadFixture('no-price.html')); + assert.deepEqual(result, { price: null, method: null }); +}); + +test('fetchAndExtractPrice returns error when fetch rejects', async () => { + const fetchImpl = async () => { throw new Error('network down'); }; + const result = await fetchAndExtractPrice('https://example.invalid', { fetchImpl }); + assert.equal(result.price, null); + assert.ok(result.error instanceof Error); +}); + +test('fetchAndExtractPrice returns error on non-OK response', async () => { + const fetchImpl = async () => ({ ok: false, status: 404, text: async () => '' }); + const result = await fetchAndExtractPrice('https://example.invalid', { fetchImpl }); + assert.equal(result.price, null); + assert.ok(result.error instanceof Error); +}); + +test('fetchAndExtractPrice extracts price from fetched HTML', async () => { + const html = loadFixture('meta-tag.html'); + const fetchImpl = async () => ({ ok: true, status: 200, text: async () => html }); + const result = await fetchAndExtractPrice('https://example.invalid', { fetchImpl }); + assert.deepEqual(result, { price: 24.50, method: 'meta', error: null }); +}); +``` + +- [ ] **Step 3: Run tests to verify they fail** + +Run: `npm test` +Expected: FAIL with "Cannot find module '../src/scraper'" + +- [ ] **Step 4: Write `src/scraper.js`** + +```js +const cheerio = require('cheerio'); + +function roundMoney(n) { + return Math.round(n * 100) / 100; +} + +function parsePriceValue(raw) { + if (raw == null) return null; + const cleaned = String(raw).trim().replace(',', '.'); + const value = parseFloat(cleaned); + return Number.isFinite(value) ? roundMoney(value) : null; +} + +function searchJsonLdNode(node) { + if (Array.isArray(node)) { + for (const item of node) { + const price = searchJsonLdNode(item); + if (price != null) return price; + } + return null; + } + if (node && typeof node === 'object') { + const type = node['@type']; + const isProduct = type === 'Product' || (Array.isArray(type) && type.includes('Product')); + if (isProduct && node.offers) { + const offers = Array.isArray(node.offers) ? node.offers[0] : node.offers; + const price = parsePriceValue(offers && offers.price); + if (price != null) return price; + } + if (Array.isArray(node['@graph'])) { + const price = searchJsonLdNode(node['@graph']); + if (price != null) return price; + } + } + return null; +} + +function findPriceInJsonLd($) { + const scripts = $('script[type="application/ld+json"]').toArray(); + for (const el of scripts) { + let data; + try { + data = JSON.parse($(el).contents().text()); + } catch { + continue; + } + const price = searchJsonLdNode(data); + if (price != null) return price; + } + return null; +} + +function findPriceInMeta($) { + const selectors = ['meta[property="product:price:amount"]', 'meta[property="og:price:amount"]']; + for (const selector of selectors) { + const price = parsePriceValue($(selector).attr('content')); + if (price != null) return price; + } + return null; +} + +function findPriceInText($) { + const text = $('body').text(); + const match = text.match(/€\s?(\d+[.,]\d{2})/); + return match ? parsePriceValue(match[1]) : null; +} + +function extractPrice(html) { + const $ = cheerio.load(html); + let price = findPriceInJsonLd($); + if (price != null) return { price, method: 'json-ld' }; + price = findPriceInMeta($); + if (price != null) return { price, method: 'meta' }; + price = findPriceInText($); + if (price != null) return { price, method: 'regex' }; + return { price: null, method: null }; +} + +async function fetchAndExtractPrice(url, { fetchImpl = fetch } = {}) { + let response; + try { + response = await fetchImpl(url); + } catch (err) { + return { price: null, method: null, error: err }; + } + if (!response.ok) { + return { price: null, method: null, error: new Error(`HTTP ${response.status}`) }; + } + const html = await response.text(); + const { price, method } = extractPrice(html); + return { price, method, error: null }; +} + +module.exports = { extractPrice, fetchAndExtractPrice, parsePriceValue }; +``` + +- [ ] **Step 5: Run tests to verify they pass** + +Run: `npm test` +Expected: PASS (all `scraper.test.js` tests + previous tests) + +- [ ] **Step 6: Commit** + +```bash +git add src/scraper.js test/scraper.test.js test/fixtures +git commit -m "feat: add price scraper with JSON-LD/meta/regex fallback" +``` + +--- + +### Task 4: Mailer module + +**Files:** +- Create: `src/mailer.js` +- Test: `test/mailer.test.js` + +**Interfaces:** +- Produces: + - `createTransport(env = process.env) -> Transport` — builds a `nodemailer` transport from `SMTP_HOST`/`SMTP_PORT`/`SMTP_USER`/`SMTP_PASS`. + - `sendPriceChangeEmail(transport, { from, to, name, url, oldPrice, newPrice }) -> Promise` — throws/rejects if the transport's `sendMail` rejects. + - `formatPrice(value: number) -> string` — e.g. `19.9 -> "19,90"`. + +- [ ] **Step 1: Write the failing tests** + +Create `test/mailer.test.js`: + +```js +const test = require('node:test'); +const assert = require('node:assert/strict'); +const { sendPriceChangeEmail, formatPrice, createTransport } = require('../src/mailer'); + +test('formatPrice formats with comma decimal separator', () => { + assert.equal(formatPrice(19.9), '19,90'); + assert.equal(formatPrice(24.5), '24,50'); +}); + +test('sendPriceChangeEmail sends mail with expected fields', async () => { + const calls = []; + const fakeTransport = { sendMail: async (opts) => { calls.push(opts); } }; + await sendPriceChangeEmail(fakeTransport, { + from: 'korting@sjoerd.app', + to: 'me@example.com', + name: 'Voorbeeld product', + url: 'https://shop.example/product', + oldPrice: 19.99, + newPrice: 17.5, + }); + assert.equal(calls.length, 1); + assert.equal(calls[0].from, 'korting@sjoerd.app'); + assert.equal(calls[0].to, 'me@example.com'); + assert.equal(calls[0].subject, 'Prijs gewijzigd: Voorbeeld product'); + assert.match(calls[0].text, /19,99/); + assert.match(calls[0].text, /17,50/); + assert.match(calls[0].text, /https:\/\/shop\.example\/product/); +}); + +test('sendPriceChangeEmail propagates transport errors', async () => { + const fakeTransport = { sendMail: async () => { throw new Error('smtp down'); } }; + await assert.rejects( + () => sendPriceChangeEmail(fakeTransport, { + from: 'a@b.com', to: 'c@d.com', name: 'X', url: 'https://x', oldPrice: 1, newPrice: 2, + }), + /smtp down/ + ); +}); + +test('createTransport builds a transport from env vars', () => { + const transport = createTransport({ SMTP_HOST: 'mail.sjoerd.app', SMTP_PORT: '587', SMTP_USER: 'u', SMTP_PASS: 'p' }); + assert.equal(typeof transport.sendMail, 'function'); +}); +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `npm test` +Expected: FAIL with "Cannot find module '../src/mailer'" + +- [ ] **Step 3: Write `src/mailer.js`** + +```js +const nodemailer = require('nodemailer'); + +function createTransport(env = process.env) { + return nodemailer.createTransport({ + host: env.SMTP_HOST, + port: Number(env.SMTP_PORT || 587), + secure: Number(env.SMTP_PORT) === 465, + auth: env.SMTP_USER ? { user: env.SMTP_USER, pass: env.SMTP_PASS } : undefined, + }); +} + +function formatPrice(value) { + return value.toFixed(2).replace('.', ','); +} + +async function sendPriceChangeEmail(transport, { from, to, name, url, oldPrice, newPrice }) { + await transport.sendMail({ + from, + to, + subject: `Prijs gewijzigd: ${name}`, + text: [ + `De prijs van "${name}" is gewijzigd.`, + `Oude prijs: €${formatPrice(oldPrice)}`, + `Nieuwe prijs: €${formatPrice(newPrice)}`, + `Link: ${url}`, + ].join('\n'), + }); +} + +module.exports = { createTransport, sendPriceChangeEmail, formatPrice }; +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `npm test` +Expected: PASS (all `mailer.test.js` tests + previous tests) + +- [ ] **Step 5: Commit** + +```bash +git add src/mailer.js test/mailer.test.js +git commit -m "feat: add SMTP mailer for price-change notifications" +``` + +--- + +### Task 5: Price-check core logic + +**Files:** +- Create: `src/checkProducts.js` +- Test: `test/checkProducts.test.js` + +**Interfaces:** +- Consumes (via dependency injection, no direct import): a `fetchAndExtractPrice`-shaped function (Task 3), a `sendMail(payload) -> Promise`-shaped function (built from Task 4's `sendPriceChangeEmail`), `recordCheckSuccess`/`recordCheckFailure`-shaped functions (Task 2), and a `now() -> string` clock function. +- Produces: + - `checkProduct(product: Product, deps) -> Promise<{ status: 'ok'|'fetch_error'|'not_found'|'mail_error', changed?: boolean }>` + - `checkAllProducts(products: Product[], deps) -> Promise>` — never throws; a single product's failure is caught and reported as `{ status: 'error', error }`. + +- [ ] **Step 1: Write the failing tests** + +Create `test/checkProducts.test.js`: + +```js +const test = require('node:test'); +const assert = require('node:assert/strict'); +const { checkProduct, checkAllProducts } = require('../src/checkProducts'); + +function makeDeps(overrides = {}) { + const failures = []; + const successes = []; + const mailCalls = []; + const deps = { + fetchAndExtractPrice: async () => ({ price: 10, method: 'meta', error: null }), + sendMail: async (payload) => { mailCalls.push(payload); }, + recordCheckFailure: (id, status) => failures.push({ id, status }), + recordCheckSuccess: (id, data) => successes.push({ id, ...data }), + now: () => '2026-09-05T08:00:00.000Z', + ...overrides, + }; + return { deps, failures, successes, mailCalls }; +} + +test('checkProduct stores baseline price without sending mail', async () => { + const { deps, successes, mailCalls } = makeDeps(); + const product = { id: 1, name: 'X', url: 'https://x', last_price: null }; + const result = await checkProduct(product, deps); + assert.equal(result.status, 'ok'); + assert.equal(result.changed, false); + assert.equal(mailCalls.length, 0); + assert.deepEqual(successes, [{ id: 1, price: 10, checkedAt: '2026-09-05T08:00:00.000Z' }]); +}); + +test('checkProduct sends mail and updates price when price changed', async () => { + const { deps, successes, mailCalls } = makeDeps(); + const product = { id: 2, name: 'X', url: 'https://x', last_price: 15 }; + const result = await checkProduct(product, deps); + assert.equal(result.status, 'ok'); + assert.equal(result.changed, true); + assert.equal(mailCalls.length, 1); + assert.deepEqual(mailCalls[0], { name: 'X', url: 'https://x', oldPrice: 15, newPrice: 10 }); + assert.equal(successes.length, 1); +}); + +test('checkProduct does not send mail or update when price unchanged', async () => { + const { deps, successes, mailCalls } = makeDeps(); + const product = { id: 3, name: 'X', url: 'https://x', last_price: 10 }; + const result = await checkProduct(product, deps); + assert.equal(result.status, 'ok'); + assert.equal(result.changed, false); + assert.equal(mailCalls.length, 0); + assert.equal(successes.length, 1); +}); + +test('checkProduct records fetch_error and does not touch price on network failure', async () => { + const { deps, failures, successes } = makeDeps({ + fetchAndExtractPrice: async () => ({ price: null, method: null, error: new Error('down') }), + }); + const product = { id: 4, name: 'X', url: 'https://x', last_price: 10 }; + const result = await checkProduct(product, deps); + assert.equal(result.status, 'fetch_error'); + assert.deepEqual(failures, [{ id: 4, status: 'fetch_error' }]); + assert.equal(successes.length, 0); +}); + +test('checkProduct records not_found when no price could be extracted', async () => { + const { deps, failures, successes } = makeDeps({ + fetchAndExtractPrice: async () => ({ price: null, method: null, error: null }), + }); + const product = { id: 5, name: 'X', url: 'https://x', last_price: 10 }; + const result = await checkProduct(product, deps); + assert.equal(result.status, 'not_found'); + assert.deepEqual(failures, [{ id: 5, status: 'not_found' }]); + assert.equal(successes.length, 0); +}); + +test('checkProduct records mail_error and keeps old price when sending mail fails', async () => { + const { deps, failures, successes } = makeDeps({ + sendMail: async () => { throw new Error('smtp down'); }, + }); + const product = { id: 6, name: 'X', url: 'https://x', last_price: 15 }; + const result = await checkProduct(product, deps); + assert.equal(result.status, 'mail_error'); + assert.deepEqual(failures, [{ id: 6, status: 'mail_error' }]); + assert.equal(successes.length, 0); +}); + +test('checkAllProducts processes every product independently, isolating failures', async () => { + const { deps } = makeDeps(); + let calls = 0; + const throwingDeps = { + ...deps, + fetchAndExtractPrice: async () => { + calls += 1; + if (calls === 1) throw new Error('boom'); + return { price: 10, method: 'meta', error: null }; + }, + }; + const products = [ + { id: 1, name: 'A', url: 'https://a', last_price: null }, + { id: 2, name: 'B', url: 'https://b', last_price: null }, + ]; + const results = await checkAllProducts(products, throwingDeps); + assert.equal(results.length, 2); + assert.equal(results[0].status, 'error'); + assert.equal(results[1].status, 'ok'); +}); +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `npm test` +Expected: FAIL with "Cannot find module '../src/checkProducts'" + +- [ ] **Step 3: Write `src/checkProducts.js`** + +```js +async function checkProduct(product, deps) { + const { fetchAndExtractPrice, sendMail, recordCheckSuccess, recordCheckFailure, now } = deps; + const result = await fetchAndExtractPrice(product.url); + + if (result.error) { + recordCheckFailure(product.id, 'fetch_error'); + return { status: 'fetch_error' }; + } + if (result.price == null) { + recordCheckFailure(product.id, 'not_found'); + return { status: 'not_found' }; + } + + const isBaseline = product.last_price == null; + const changed = !isBaseline && result.price !== product.last_price; + + if (changed) { + try { + await sendMail({ + name: product.name, + url: product.url, + oldPrice: product.last_price, + newPrice: result.price, + }); + } catch { + recordCheckFailure(product.id, 'mail_error'); + return { status: 'mail_error' }; + } + } + + recordCheckSuccess(product.id, { price: result.price, checkedAt: now() }); + return { status: 'ok', changed }; +} + +async function checkAllProducts(products, deps) { + const results = []; + for (const product of products) { + try { + const result = await checkProduct(product, deps); + results.push({ productId: product.id, ...result }); + } catch (err) { + results.push({ productId: product.id, status: 'error', error: err }); + } + } + return results; +} + +module.exports = { checkProduct, checkAllProducts }; +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: `npm test` +Expected: PASS (all `checkProducts.test.js` tests + previous tests) + +- [ ] **Step 5: Commit** + +```bash +git add src/checkProducts.js test/checkProducts.test.js +git commit -m "feat: add price-check orchestration with retry-safe error handling" +``` + +--- + +### Task 6: Scheduler + +**Files:** +- Create: `src/scheduler.js` +- Test: `test/scheduler.test.js` + +**Interfaces:** +- Produces: + - `start(runCheck: () => Promise, { cron? }) -> ScheduledTask` — schedules `runCheck` with cron expression `'0 8 * * *'` and `{ timezone: 'Europe/Amsterdam' }`; `cron` defaults to the real `node-cron` module and is injectable for tests. + - `SCHEDULE` (string constant `'0 8 * * *'`), `TIMEZONE` (string constant `'Europe/Amsterdam'`). + +- [ ] **Step 1: Write the failing test** + +Create `test/scheduler.test.js`: + +```js +const test = require('node:test'); +const assert = require('node:assert/strict'); +const { start, SCHEDULE, TIMEZONE } = require('../src/scheduler'); + +test('start schedules the daily check with the expected cron expression and timezone', () => { + const calls = []; + const fakeCron = { + schedule: (expression, fn, options) => { + calls.push({ expression, fn, options }); + return { stop: () => {} }; + }, + }; + const runCheck = async () => {}; + start(runCheck, { cron: fakeCron }); + assert.equal(calls.length, 1); + assert.equal(calls[0].expression, SCHEDULE); + assert.equal(SCHEDULE, '0 8 * * *'); + assert.equal(calls[0].options.timezone, TIMEZONE); + assert.equal(TIMEZONE, 'Europe/Amsterdam'); + assert.equal(calls[0].fn, runCheck); +}); +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: `npm test` +Expected: FAIL with "Cannot find module '../src/scheduler'" + +- [ ] **Step 3: Write `src/scheduler.js`** + +```js +const cronDefault = require('node-cron'); + +const SCHEDULE = '0 8 * * *'; +const TIMEZONE = 'Europe/Amsterdam'; + +function start(runCheck, { cron = cronDefault } = {}) { + return cron.schedule(SCHEDULE, runCheck, { timezone: TIMEZONE }); +} + +module.exports = { start, SCHEDULE, TIMEZONE }; +``` + +- [ ] **Step 4: Run test to verify it passes** + +Run: `npm test` +Expected: PASS (all `scheduler.test.js` tests + previous tests) + +- [ ] **Step 5: Commit** + +```bash +git add src/scheduler.js test/scheduler.test.js +git commit -m "feat: add daily cron scheduler for price checks" +``` + +--- + +### Task 7: Views + Express app + +**Files:** +- Create: `src/views.js` +- Create: `src/app.js` +- Test: `test/app.test.js` + +**Interfaces:** +- Consumes: nothing directly imported from earlier tasks — `createApp(deps)` takes injected `listProducts`, `addProduct`, `deleteProduct`, `checkAllProducts`, `authUser`, `authPass` functions/values, matching the shapes produced by Tasks 2 and 5 (wired together in Task 8). +- Produces: + - `renderIndex({ products: Product[], message?: string }) -> string` (full HTML document) + - `createApp(deps) -> express.Application` with routes: `GET /`, `POST /products`, `POST /products/:id/delete`, `POST /check-now` — all behind basic auth. + +- [ ] **Step 1: Write the failing tests** + +Create `test/app.test.js`: + +```js +const test = require('node:test'); +const assert = require('node:assert/strict'); +const request = require('supertest'); +const { createApp } = require('../src/app'); + +function makeDeps(overrides = {}) { + let products = []; + let nextId = 1; + return { + listProducts: () => products, + addProduct: ({ name, url }) => { + const product = { id: nextId++, name, url, last_price: null, last_checked_at: null, last_check_status: null }; + products.push(product); + return product; + }, + deleteProduct: (id) => { products = products.filter((p) => p.id !== id); }, + checkAllProducts: async () => [], + authUser: 'admin', + authPass: 'secret', + ...overrides, + }; +} + +test('rejects requests without basic auth credentials', async () => { + const app = createApp(makeDeps()); + const res = await request(app).get('/'); + assert.equal(res.status, 401); +}); + +test('lists products for authenticated requests', async () => { + const deps = makeDeps(); + deps.addProduct({ name: 'Voorbeeld', url: 'https://example.com' }); + const app = createApp(deps); + const res = await request(app).get('/').auth('admin', 'secret'); + assert.equal(res.status, 200); + assert.match(res.text, /Voorbeeld/); +}); + +test('adding a product redirects to the list', async () => { + const deps = makeDeps(); + const app = createApp(deps); + const res = await request(app) + .post('/products') + .auth('admin', 'secret') + .type('form') + .send({ name: 'Nieuw product', url: 'https://shop.example/p' }); + assert.equal(res.status, 302); + assert.equal(res.headers.location, '/'); + const list = await request(app).get('/').auth('admin', 'secret'); + assert.match(list.text, /Nieuw product/); +}); + +test('deleting a product removes it from the list', async () => { + const deps = makeDeps(); + const product = deps.addProduct({ name: 'Weg ermee', url: 'https://shop.example/p2' }); + const app = createApp(deps); + const res = await request(app).post(`/products/${product.id}/delete`).auth('admin', 'secret'); + assert.equal(res.status, 302); + const list = await request(app).get('/').auth('admin', 'secret'); + assert.doesNotMatch(list.text, /Weg ermee/); +}); + +test('check-now triggers checkAllProducts and redirects with confirmation', async () => { + let called = false; + const deps = makeDeps({ checkAllProducts: async () => { called = true; return []; } }); + const app = createApp(deps); + const res = await request(app).post('/check-now').auth('admin', 'secret'); + assert.equal(res.status, 302); + assert.equal(res.headers.location, '/?checked=1'); + assert.equal(called, true); +}); +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: `npm test` +Expected: FAIL with "Cannot find module '../src/app'" + +- [ ] **Step 3: Write `src/views.js`** + +```js +function escapeHtml(str) { + return String(str).replace(/[&<>"']/g, (c) => ({ + '&': '&', '<': '<', '>': '>', '"': '"', "'": ''', + }[c])); +} + +function formatPrice(price) { + return price == null ? '—' : `€${Number(price).toFixed(2).replace('.', ',')}`; +} + +function renderProductRow(product) { + return ` + + ${escapeHtml(product.name)} + bekijken + ${formatPrice(product.last_price)} + ${product.last_checked_at ? escapeHtml(product.last_checked_at) : '—'} + ${escapeHtml(product.last_check_status || '—')} + +
+ +
+ + `; +} + +function renderIndex({ products, message }) { + return ` + + + + Korting — prijstracker + + + +

Korting — prijstracker

+ ${message ? `

${escapeHtml(message)}

` : ''} +
+ + + +
+
+ +
+ + + + + + ${products.map(renderProductRow).join('')} + +
NaamURLPrijsLaatst gecontroleerdStatus
+ +`; +} + +module.exports = { renderIndex, escapeHtml, formatPrice }; +``` + +- [ ] **Step 4: Write `src/app.js`** + +```js +const express = require('express'); +const basicAuth = require('express-basic-auth'); +const { renderIndex } = require('./views'); + +function createApp(deps) { + const { + listProducts, addProduct, deleteProduct, + checkAllProducts, authUser, authPass, + } = deps; + + const app = express(); + app.use(express.urlencoded({ extended: false })); + app.use(basicAuth({ users: { [authUser]: authPass }, challenge: true })); + + app.get('/', (req, res) => { + const products = listProducts(); + const message = req.query.checked ? 'Controle uitgevoerd.' : undefined; + res.send(renderIndex({ products, message })); + }); + + app.post('/products', (req, res) => { + const { name, url } = req.body; + if (name && url) { + addProduct({ name, url }); + } + res.redirect('/'); + }); + + app.post('/products/:id/delete', (req, res) => { + deleteProduct(Number(req.params.id)); + res.redirect('/'); + }); + + app.post('/check-now', async (req, res) => { + const products = listProducts(); + await checkAllProducts(products); + res.redirect('/?checked=1'); + }); + + return app; +} + +module.exports = { createApp }; +``` + +- [ ] **Step 5: Run tests to verify they pass** + +Run: `npm test` +Expected: PASS (all `app.test.js` tests + previous tests) + +- [ ] **Step 6: Commit** + +```bash +git add src/views.js src/app.js test/app.test.js +git commit -m "feat: add web UI for managing products" +``` + +--- + +### Task 8: Composition root (server entry point) + +**Files:** +- Create: `server.js` + +**Interfaces:** +- Consumes: `openDb`, `listProducts`, `addProduct`, `deleteProduct`, `recordCheckSuccess`, `recordCheckFailure` (Task 2); `fetchAndExtractPrice` (Task 3); `createTransport`, `sendPriceChangeEmail` (Task 4); `checkAllProducts` (Task 5); `start` as `startScheduler` (Task 6); `createApp` (Task 7). +- Produces: the running HTTP server (no further consumers — this is the top of the dependency graph). + +No new automated tests here — this file only wires together already-tested pieces. It is verified with a manual smoke check. + +- [ ] **Step 1: Write `server.js`** + +```js +const path = require('node:path'); +const { + openDb, listProducts, addProduct, deleteProduct, + recordCheckSuccess, recordCheckFailure, +} = require('./src/db'); +const { fetchAndExtractPrice } = require('./src/scraper'); +const { createTransport, sendPriceChangeEmail } = require('./src/mailer'); +const { checkAllProducts } = require('./src/checkProducts'); +const { start: startScheduler } = require('./src/scheduler'); +const { createApp } = require('./src/app'); + +const PORT = process.env.PORT || 3000; +const DB_PATH = process.env.DB_PATH || path.join(__dirname, 'data', 'korting.db'); + +const db = openDb(DB_PATH); +const transport = createTransport(process.env); + +function runCheck() { + const products = listProducts(db); + const deps = { + fetchAndExtractPrice, + sendMail: (payload) => sendPriceChangeEmail(transport, { + from: process.env.SMTP_FROM, + to: process.env.NOTIFY_EMAIL, + ...payload, + }), + recordCheckFailure: (id, status) => recordCheckFailure(db, id, status), + recordCheckSuccess: (id, data) => recordCheckSuccess(db, id, data), + now: () => new Date().toISOString(), + }; + return checkAllProducts(products, deps); +} + +startScheduler(runCheck); + +const app = createApp({ + listProducts: () => listProducts(db), + addProduct: (data) => addProduct(db, data), + deleteProduct: (id) => deleteProduct(db, id), + checkAllProducts: runCheck, + authUser: process.env.BASIC_AUTH_USER, + authPass: process.env.BASIC_AUTH_PASS, +}); + +app.listen(PORT, () => { + console.log(`Korting draait op poort ${PORT}`); +}); +``` + +- [ ] **Step 2: Manual smoke check** + +Run: +```bash +cp .env.example .env +# edit .env: fill in a real BASIC_AUTH_USER/PASS at minimum +set -a; source .env; set +a +node server.js +``` +In another terminal: +```bash +curl -u "$BASIC_AUTH_USER:$BASIC_AUTH_PASS" http://localhost:3000/ +curl -u "$BASIC_AUTH_USER:$BASIC_AUTH_PASS" -X POST http://localhost:3000/products \ + -d "name=Testproduct" -d "url=https://example.com" +curl -u "$BASIC_AUTH_USER:$BASIC_AUTH_PASS" http://localhost:3000/ +``` +Expected: first `curl` returns the HTML page; second redirects (302); third shows "Testproduct" in the table. Stop the server with Ctrl-C. + +- [ ] **Step 3: Commit** + +```bash +git add server.js +git commit -m "feat: wire modules together into the running server" +``` + +--- + +### Task 9: Dockerfile + +**Files:** +- Create: `Dockerfile` +- Create: `.dockerignore` + +**Interfaces:** +- Produces: a Docker image that runs `node server.js` on `$PORT` (default 3000), expecting `/data` to be mounted as a persistent volume when `DB_PATH=/data/korting.db` is set. + +- [ ] **Step 1: Write `Dockerfile`** + +`better-sqlite3` needs a native build step, so this uses a two-stage build: the first stage has build tools to compile it, the second stage is the slim runtime image. + +```dockerfile +FROM node:22-slim AS build +RUN apt-get update && apt-get install -y --no-install-recommends python3 build-essential \ + && rm -rf /var/lib/apt/lists/* +WORKDIR /app +COPY package.json package-lock.json ./ +RUN npm ci --omit=dev +COPY . . + +FROM node:22-slim +WORKDIR /app +COPY --from=build /app . +ENV NODE_ENV=production +EXPOSE 3000 +CMD ["node", "server.js"] +``` + +- [ ] **Step 2: Write `.dockerignore`** + +``` +node_modules +data +.env +.git +docs +test +``` + +- [ ] **Step 3: Manual build/run verification** + +Run: +```bash +docker build -t korting . +docker run --rm -p 3000:3000 \ + -e BASIC_AUTH_USER=admin -e BASIC_AUTH_PASS=test \ + -e SMTP_HOST=mail.sjoerd.app -e SMTP_PORT=587 \ + -e SMTP_FROM=korting@sjoerd.app -e NOTIFY_EMAIL=test@example.com \ + -e DB_PATH=/data/korting.db -v korting-data:/data \ + korting +``` +In another terminal: `curl -u admin:test http://localhost:3000/` should return the HTML page. Stop with Ctrl-C. + +- [ ] **Step 4: Commit** + +```bash +git add Dockerfile .dockerignore +git commit -m "chore: add Dockerfile for deployment" +``` + +--- + +### Task 10: README + +**Files:** +- Create: `README.md` + +- [ ] **Step 1: Write `README.md`** + +```markdown +# Korting — prijstracker + +Houdt een lijst productpagina's dagelijks in de gaten en mailt wanneer +de prijs verandert. + +## Lokaal draaien + +\`\`\`bash +npm install +cp .env.example .env # env vars invullen +npm test +npm start +\`\`\` + +De UI staat achter basic auth (`BASIC_AUTH_USER` / `BASIC_AUTH_PASS`) op +`http://localhost:3000`. + +## Environment variables + +| Variabele | Omschrijving | +|---|---| +| `PORT` | HTTP-poort (default `3000`) | +| `DB_PATH` | Pad naar het SQLite-bestand (default `./data/korting.db`) | +| `BASIC_AUTH_USER` / `BASIC_AUTH_PASS` | Login voor de web-UI | +| `SMTP_HOST` / `SMTP_PORT` / `SMTP_USER` / `SMTP_PASS` | SMTP-server (Postal op `mail.sjoerd.app`) | +| `SMTP_FROM` | Afzenderadres van de notificatiemails | +| `NOTIFY_EMAIL` | Ontvangeradres van de notificatiemails | + +## Hoe het werkt + +Elke dag om 08:00 (Europe/Amsterdam) doorloopt de app alle producten, +haalt de huidige prijs op (JSON-LD → meta tags → regex-fallback), en +stuurt een email als de prijs is veranderd sinds de vorige geslaagde +check. Een mislukte poging (pagina onbereikbaar, geen prijs gevonden, +of de mail kon niet verstuurd worden) verandert de opgeslagen prijs +niet, zodat de volgende dag opnieuw wordt geprobeerd. Via de "Nu +controleren"-knop in de UI kan dit ook handmatig getriggerd worden. +``` + +- [ ] **Step 2: Commit** + +```bash +git add README.md +git commit -m "docs: add README" +``` + +--- + +## After this plan + +Deployment (Gitea repo + Coolify application, domain `korting.sjoerd.app`, +persistent volume, env vars, auto-deploy on push) is handled separately +via the `deploying-to-coolify` skill — it is an operational step, not a +coding task.