Files

T

Sjoerd de Vries 5edb1f04f6 Add implementation plan for korting prijstracker

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013gwc3aCiyQxmCWQke8RvEz

2026-09-05 14:35:57 +02:00

41 KiB

Raw Blame History

Korting prijstracker Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Build a Node.js/Express web app that lets you manage a list of product URLs, checks their price daily, and emails when a price changes.

Architecture: One Express app in one process. node-cron runs a daily job in-process; the same check logic is reachable via a "check now" button in the UI. SQLite (better-sqlite3) stores products and their last known price. Price extraction tries JSON-LD, then meta tags, then a regex fallback. Email goes out via nodemailer over SMTP to the existing Postal server at mail.sjoerd.app.

Tech Stack: Node.js 22, Express, better-sqlite3, cheerio, nodemailer, node-cron, express-basic-auth. Tests via Node's built-in node:test + supertest (devDependency only).

Spec: docs/superpowers/specs/2026-09-05-korting-prijstracker-design.md

Global Constraints

Node.js 22, CommonJS modules (require/module.exports), no TypeScript, no bundler.
One SQLite file at DB_PATH (default ./data/korting.db locally; /data/korting.db on the Coolify volume in production).
Daily cron schedule 0 8 * * *, timezone Europe/Amsterdam.
last_price and last_checked_at are updated only on a fully successful check (price found, and email either not needed or sent successfully). They are left untouched on fetch_error, not_found, or mail_error, so a failure is retried automatically the next day.
Every route is protected by basic auth (BASIC_AUTH_USER / BASIC_AUTH_PASS).
SMTP env vars: SMTP_HOST, SMTP_PORT, SMTP_USER, SMTP_PASS, SMTP_FROM, NOTIFY_EMAIL (recipient).
Test runner: Node's built-in node:test (node --test test/). The only devDependency is supertest, used solely for the Express app tests.

Task 1: Project scaffolding

Files:

Create: package.json
Create: .gitignore
Create: .env.example
Create: test/smoke.test.js

Interfaces:

Produces: npm test runs node --test test/; npm start runs node server.js (server.js created in Task 8).
Step 1: Create package.json

{
  "name": "korting",
  "version": "1.0.0",
  "private": true,
  "description": "Dagelijkse prijstracker die mailt bij prijswijzigingen",
  "main": "server.js",
  "scripts": {
    "start": "node server.js",
    "test": "node --test test/"
  },
  "dependencies": {
    "better-sqlite3": "^11.3.0",
    "cheerio": "^1.0.0",
    "express": "^4.19.2",
    "express-basic-auth": "^1.2.1",
    "node-cron": "^3.0.3",
    "nodemailer": "^6.9.14"
  },
  "devDependencies": {
    "supertest": "^7.0.0"
  }
}

Step 2: Install dependencies

Run: npm install Expected: completes with no errors, creates node_modules/ and package-lock.json.

Step 3: Create .gitignore

node_modules/
data/
.env

Step 4: Create .env.example

PORT=3000
DB_PATH=./data/korting.db
BASIC_AUTH_USER=admin
BASIC_AUTH_PASS=change-me
SMTP_HOST=mail.sjoerd.app
SMTP_PORT=587
SMTP_USER=
SMTP_PASS=
SMTP_FROM=korting@sjoerd.app
NOTIFY_EMAIL=openai@sjoerddev.com

Step 5: Create test/smoke.test.js

const test = require('node:test');
const assert = require('node:assert/strict');

test('smoke test runs', () => {
  assert.ok(true);
});

Step 6: Run the test suite

Run: npm test Expected: PASS (1 test)

Step 7: Commit

git add package.json package-lock.json .gitignore .env.example test/smoke.test.js
git commit -m "chore: project scaffolding"

Task 2: Database module

Files:

Create: src/db.js
Test: test/db.test.js

Interfaces:

Produces:
- openDb(dbPath: string) -> Database — creates parent dir (unless :memory:), ensures schema, returns a better-sqlite3 instance.
- listProducts(db) -> Array<Product> ordered by created_at ascending.
- getProduct(db, id) -> Product | undefined
- addProduct(db, { name, url }) -> Product (sets created_at, last_price: null, last_checked_at: null)
- deleteProduct(db, id) -> void
- recordCheckSuccess(db, id, { price, checkedAt }) -> void — sets last_price, last_checked_at, last_check_status = 'ok'
- recordCheckFailure(db, id, status: string) -> void — sets only last_check_status
- Product shape: { id, name, url, last_price, last_checked_at, last_check_status, created_at }
Step 1: Write the failing tests

Create test/db.test.js:

const test = require('node:test');
const assert = require('node:assert/strict');
const {
  openDb, listProducts, getProduct, addProduct, deleteProduct,
  recordCheckSuccess, recordCheckFailure,
} = require('../src/db');

function freshDb() {
  return openDb(':memory:');
}

test('addProduct inserts a row with defaults and returns it', () => {
  const db = freshDb();
  const product = addProduct(db, { name: 'Voorbeeld', url: 'https://example.com' });
  assert.equal(product.name, 'Voorbeeld');
  assert.equal(product.url, 'https://example.com');
  assert.equal(product.last_price, null);
  assert.equal(product.last_checked_at, null);
  assert.ok(product.created_at);
  assert.ok(product.id);
});

test('listProducts returns all inserted products in insertion order', () => {
  const db = freshDb();
  addProduct(db, { name: 'A', url: 'https://a' });
  addProduct(db, { name: 'B', url: 'https://b' });
  const products = listProducts(db);
  assert.equal(products.length, 2);
  assert.equal(products[0].name, 'A');
  assert.equal(products[1].name, 'B');
});

test('getProduct returns undefined for an unknown id', () => {
  const db = freshDb();
  assert.equal(getProduct(db, 999), undefined);
});

test('deleteProduct removes the row', () => {
  const db = freshDb();
  const product = addProduct(db, { name: 'A', url: 'https://a' });
  deleteProduct(db, product.id);
  assert.equal(getProduct(db, product.id), undefined);
});

test('recordCheckSuccess updates price, timestamp and status', () => {
  const db = freshDb();
  const product = addProduct(db, { name: 'A', url: 'https://a' });
  recordCheckSuccess(db, product.id, { price: 12.5, checkedAt: '2026-09-05T08:00:00.000Z' });
  const updated = getProduct(db, product.id);
  assert.equal(updated.last_price, 12.5);
  assert.equal(updated.last_checked_at, '2026-09-05T08:00:00.000Z');
  assert.equal(updated.last_check_status, 'ok');
});

test('recordCheckFailure only updates the status, leaving price and timestamp untouched', () => {
  const db = freshDb();
  const product = addProduct(db, { name: 'A', url: 'https://a' });
  recordCheckSuccess(db, product.id, { price: 12.5, checkedAt: '2026-09-05T08:00:00.000Z' });
  recordCheckFailure(db, product.id, 'fetch_error');
  const updated = getProduct(db, product.id);
  assert.equal(updated.last_price, 12.5);
  assert.equal(updated.last_checked_at, '2026-09-05T08:00:00.000Z');
  assert.equal(updated.last_check_status, 'fetch_error');
});

Step 2: Run tests to verify they fail

Run: npm test Expected: FAIL with "Cannot find module '../src/db'"

Step 3: Write src/db.js

const Database = require('better-sqlite3');
const fs = require('node:fs');
const path = require('node:path');

const SCHEMA = `
CREATE TABLE IF NOT EXISTS products (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  name TEXT NOT NULL,
  url TEXT NOT NULL,
  last_price REAL,
  last_checked_at TEXT,
  last_check_status TEXT,
  created_at TEXT NOT NULL
)`;

function openDb(dbPath) {
  if (dbPath !== ':memory:') {
    fs.mkdirSync(path.dirname(dbPath), { recursive: true });
  }
  const db = new Database(dbPath);
  db.exec(SCHEMA);
  return db;
}

function listProducts(db) {
  return db.prepare('SELECT * FROM products ORDER BY created_at ASC').all();
}

function getProduct(db, id) {
  return db.prepare('SELECT * FROM products WHERE id = ?').get(id);
}

function addProduct(db, { name, url }) {
  const createdAt = new Date().toISOString();
  const info = db.prepare(
    'INSERT INTO products (name, url, created_at) VALUES (?, ?, ?)'
  ).run(name, url, createdAt);
  return getProduct(db, info.lastInsertRowid);
}

function deleteProduct(db, id) {
  db.prepare('DELETE FROM products WHERE id = ?').run(id);
}

function recordCheckSuccess(db, id, { price, checkedAt }) {
  db.prepare(
    'UPDATE products SET last_price = ?, last_checked_at = ?, last_check_status = ? WHERE id = ?'
  ).run(price, checkedAt, 'ok', id);
}

function recordCheckFailure(db, id, status) {
  db.prepare('UPDATE products SET last_check_status = ? WHERE id = ?').run(status, id);
}

module.exports = {
  openDb, listProducts, getProduct, addProduct, deleteProduct,
  recordCheckSuccess, recordCheckFailure,
};

Step 4: Run tests to verify they pass

Run: npm test Expected: PASS (all db.test.js tests + smoke test)

Step 5: Commit

git add src/db.js test/db.test.js
git commit -m "feat: add SQLite-backed products store"

Task 3: Scraper module

Files:

Create: src/scraper.js
Create: test/fixtures/json-ld.html
Create: test/fixtures/meta-tag.html
Create: test/fixtures/regex-fallback.html
Create: test/fixtures/no-price.html
Test: test/scraper.test.js

Interfaces:

Produces:
- extractPrice(html: string) -> { price: number | null, method: 'json-ld' | 'meta' | 'regex' | null }
- fetchAndExtractPrice(url: string, { fetchImpl? }) -> Promise<{ price: number | null, method: string | null, error: Error | null }> — fetchImpl defaults to the global fetch; never throws, errors are returned on the error field.
Step 1: Create fixture files

test/fixtures/json-ld.html:

<html><head>
<script type="application/ld+json">
{"@context":"https://schema.org","@type":"Product","name":"Voorbeeld product","offers":{"@type":"Offer","price":"19.99","priceCurrency":"EUR"}}
</script>
</head><body><p>Voorbeeld product</p></body></html>

test/fixtures/meta-tag.html:

<html><head>
<meta property="og:price:amount" content="24.50" />
</head><body><p>Voorbeeld product</p></body></html>

test/fixtures/regex-fallback.html:

<html><body><p>Onze prijs: &euro;14,95 vandaag</p></body></html>

test/fixtures/no-price.html:

<html><body><p>Geen prijs op deze pagina.</p></body></html>

Step 2: Write the failing tests

Create test/scraper.test.js:

const test = require('node:test');
const assert = require('node:assert/strict');
const fs = require('node:fs');
const path = require('node:path');
const { extractPrice, fetchAndExtractPrice } = require('../src/scraper');

function loadFixture(name) {
  return fs.readFileSync(path.join(__dirname, 'fixtures', name), 'utf8');
}

test('extractPrice reads price from JSON-LD Product offers', () => {
  const result = extractPrice(loadFixture('json-ld.html'));
  assert.deepEqual(result, { price: 19.99, method: 'json-ld' });
});

test('extractPrice reads price from og:price:amount meta tag', () => {
  const result = extractPrice(loadFixture('meta-tag.html'));
  assert.deepEqual(result, { price: 24.50, method: 'meta' });
});

test('extractPrice falls back to euro-amount regex in page text', () => {
  const result = extractPrice(loadFixture('regex-fallback.html'));
  assert.deepEqual(result, { price: 14.95, method: 'regex' });
});

test('extractPrice returns null price when nothing matches', () => {
  const result = extractPrice(loadFixture('no-price.html'));
  assert.deepEqual(result, { price: null, method: null });
});

test('fetchAndExtractPrice returns error when fetch rejects', async () => {
  const fetchImpl = async () => { throw new Error('network down'); };
  const result = await fetchAndExtractPrice('https://example.invalid', { fetchImpl });
  assert.equal(result.price, null);
  assert.ok(result.error instanceof Error);
});

test('fetchAndExtractPrice returns error on non-OK response', async () => {
  const fetchImpl = async () => ({ ok: false, status: 404, text: async () => '' });
  const result = await fetchAndExtractPrice('https://example.invalid', { fetchImpl });
  assert.equal(result.price, null);
  assert.ok(result.error instanceof Error);
});

test('fetchAndExtractPrice extracts price from fetched HTML', async () => {
  const html = loadFixture('meta-tag.html');
  const fetchImpl = async () => ({ ok: true, status: 200, text: async () => html });
  const result = await fetchAndExtractPrice('https://example.invalid', { fetchImpl });
  assert.deepEqual(result, { price: 24.50, method: 'meta', error: null });
});

Step 3: Run tests to verify they fail

Run: npm test Expected: FAIL with "Cannot find module '../src/scraper'"

Step 4: Write src/scraper.js

const cheerio = require('cheerio');

function roundMoney(n) {
  return Math.round(n * 100) / 100;
}

function parsePriceValue(raw) {
  if (raw == null) return null;
  const cleaned = String(raw).trim().replace(',', '.');
  const value = parseFloat(cleaned);
  return Number.isFinite(value) ? roundMoney(value) : null;
}

function searchJsonLdNode(node) {
  if (Array.isArray(node)) {
    for (const item of node) {
      const price = searchJsonLdNode(item);
      if (price != null) return price;
    }
    return null;
  }
  if (node && typeof node === 'object') {
    const type = node['@type'];
    const isProduct = type === 'Product' || (Array.isArray(type) && type.includes('Product'));
    if (isProduct && node.offers) {
      const offers = Array.isArray(node.offers) ? node.offers[0] : node.offers;
      const price = parsePriceValue(offers && offers.price);
      if (price != null) return price;
    }
    if (Array.isArray(node['@graph'])) {
      const price = searchJsonLdNode(node['@graph']);
      if (price != null) return price;
    }
  }
  return null;
}

function findPriceInJsonLd($) {
  const scripts = $('script[type="application/ld+json"]').toArray();
  for (const el of scripts) {
    let data;
    try {
      data = JSON.parse($(el).contents().text());
    } catch {
      continue;
    }
    const price = searchJsonLdNode(data);
    if (price != null) return price;
  }
  return null;
}

function findPriceInMeta($) {
  const selectors = ['meta[property="product:price:amount"]', 'meta[property="og:price:amount"]'];
  for (const selector of selectors) {
    const price = parsePriceValue($(selector).attr('content'));
    if (price != null) return price;
  }
  return null;
}

function findPriceInText($) {
  const text = $('body').text();
  const match = text.match(/€\s?(\d+[.,]\d{2})/);
  return match ? parsePriceValue(match[1]) : null;
}

function extractPrice(html) {
  const $ = cheerio.load(html);
  let price = findPriceInJsonLd($);
  if (price != null) return { price, method: 'json-ld' };
  price = findPriceInMeta($);
  if (price != null) return { price, method: 'meta' };
  price = findPriceInText($);
  if (price != null) return { price, method: 'regex' };
  return { price: null, method: null };
}

async function fetchAndExtractPrice(url, { fetchImpl = fetch } = {}) {
  let response;
  try {
    response = await fetchImpl(url);
  } catch (err) {
    return { price: null, method: null, error: err };
  }
  if (!response.ok) {
    return { price: null, method: null, error: new Error(`HTTP ${response.status}`) };
  }
  const html = await response.text();
  const { price, method } = extractPrice(html);
  return { price, method, error: null };
}

module.exports = { extractPrice, fetchAndExtractPrice, parsePriceValue };

Step 5: Run tests to verify they pass

Run: npm test Expected: PASS (all scraper.test.js tests + previous tests)

Step 6: Commit

git add src/scraper.js test/scraper.test.js test/fixtures
git commit -m "feat: add price scraper with JSON-LD/meta/regex fallback"

Task 4: Mailer module

Files:

Create: src/mailer.js
Test: test/mailer.test.js

Interfaces:

Produces:
- createTransport(env = process.env) -> Transport — builds a nodemailer transport from SMTP_HOST/SMTP_PORT/SMTP_USER/SMTP_PASS.
- sendPriceChangeEmail(transport, { from, to, name, url, oldPrice, newPrice }) -> Promise<void> — throws/rejects if the transport's sendMail rejects.
- formatPrice(value: number) -> string — e.g. 19.9 -> "19,90".
Step 1: Write the failing tests

Create test/mailer.test.js:

const test = require('node:test');
const assert = require('node:assert/strict');
const { sendPriceChangeEmail, formatPrice, createTransport } = require('../src/mailer');

test('formatPrice formats with comma decimal separator', () => {
  assert.equal(formatPrice(19.9), '19,90');
  assert.equal(formatPrice(24.5), '24,50');
});

test('sendPriceChangeEmail sends mail with expected fields', async () => {
  const calls = [];
  const fakeTransport = { sendMail: async (opts) => { calls.push(opts); } };
  await sendPriceChangeEmail(fakeTransport, {
    from: 'korting@sjoerd.app',
    to: 'me@example.com',
    name: 'Voorbeeld product',
    url: 'https://shop.example/product',
    oldPrice: 19.99,
    newPrice: 17.5,
  });
  assert.equal(calls.length, 1);
  assert.equal(calls[0].from, 'korting@sjoerd.app');
  assert.equal(calls[0].to, 'me@example.com');
  assert.equal(calls[0].subject, 'Prijs gewijzigd: Voorbeeld product');
  assert.match(calls[0].text, /19,99/);
  assert.match(calls[0].text, /17,50/);
  assert.match(calls[0].text, /https:\/\/shop\.example\/product/);
});

test('sendPriceChangeEmail propagates transport errors', async () => {
  const fakeTransport = { sendMail: async () => { throw new Error('smtp down'); } };
  await assert.rejects(
    () => sendPriceChangeEmail(fakeTransport, {
      from: 'a@b.com', to: 'c@d.com', name: 'X', url: 'https://x', oldPrice: 1, newPrice: 2,
    }),
    /smtp down/
  );
});

test('createTransport builds a transport from env vars', () => {
  const transport = createTransport({ SMTP_HOST: 'mail.sjoerd.app', SMTP_PORT: '587', SMTP_USER: 'u', SMTP_PASS: 'p' });
  assert.equal(typeof transport.sendMail, 'function');
});

Step 2: Run tests to verify they fail

Run: npm test Expected: FAIL with "Cannot find module '../src/mailer'"

Step 3: Write src/mailer.js

const nodemailer = require('nodemailer');

function createTransport(env = process.env) {
  return nodemailer.createTransport({
    host: env.SMTP_HOST,
    port: Number(env.SMTP_PORT || 587),
    secure: Number(env.SMTP_PORT) === 465,
    auth: env.SMTP_USER ? { user: env.SMTP_USER, pass: env.SMTP_PASS } : undefined,
  });
}

function formatPrice(value) {
  return value.toFixed(2).replace('.', ',');
}

async function sendPriceChangeEmail(transport, { from, to, name, url, oldPrice, newPrice }) {
  await transport.sendMail({
    from,
    to,
    subject: `Prijs gewijzigd: ${name}`,
    text: [
      `De prijs van "${name}" is gewijzigd.`,
      `Oude prijs: €${formatPrice(oldPrice)}`,
      `Nieuwe prijs: €${formatPrice(newPrice)}`,
      `Link: ${url}`,
    ].join('\n'),
  });
}

module.exports = { createTransport, sendPriceChangeEmail, formatPrice };

Step 4: Run tests to verify they pass

Run: npm test Expected: PASS (all mailer.test.js tests + previous tests)

Step 5: Commit

git add src/mailer.js test/mailer.test.js
git commit -m "feat: add SMTP mailer for price-change notifications"

Task 5: Price-check core logic

Files:

Create: src/checkProducts.js
Test: test/checkProducts.test.js

Interfaces:

Consumes (via dependency injection, no direct import): a fetchAndExtractPrice-shaped function (Task 3), a sendMail(payload) -> Promise<void>-shaped function (built from Task 4's sendPriceChangeEmail), recordCheckSuccess/recordCheckFailure-shaped functions (Task 2), and a now() -> string clock function.
Produces:
- checkProduct(product: Product, deps) -> Promise<{ status: 'ok'|'fetch_error'|'not_found'|'mail_error', changed?: boolean }>
- checkAllProducts(products: Product[], deps) -> Promise<Array<{ productId, status, changed?, error? }>> — never throws; a single product's failure is caught and reported as { status: 'error', error }.
Step 1: Write the failing tests

Create test/checkProducts.test.js:

const test = require('node:test');
const assert = require('node:assert/strict');
const { checkProduct, checkAllProducts } = require('../src/checkProducts');

function makeDeps(overrides = {}) {
  const failures = [];
  const successes = [];
  const mailCalls = [];
  const deps = {
    fetchAndExtractPrice: async () => ({ price: 10, method: 'meta', error: null }),
    sendMail: async (payload) => { mailCalls.push(payload); },
    recordCheckFailure: (id, status) => failures.push({ id, status }),
    recordCheckSuccess: (id, data) => successes.push({ id, ...data }),
    now: () => '2026-09-05T08:00:00.000Z',
    ...overrides,
  };
  return { deps, failures, successes, mailCalls };
}

test('checkProduct stores baseline price without sending mail', async () => {
  const { deps, successes, mailCalls } = makeDeps();
  const product = { id: 1, name: 'X', url: 'https://x', last_price: null };
  const result = await checkProduct(product, deps);
  assert.equal(result.status, 'ok');
  assert.equal(result.changed, false);
  assert.equal(mailCalls.length, 0);
  assert.deepEqual(successes, [{ id: 1, price: 10, checkedAt: '2026-09-05T08:00:00.000Z' }]);
});

test('checkProduct sends mail and updates price when price changed', async () => {
  const { deps, successes, mailCalls } = makeDeps();
  const product = { id: 2, name: 'X', url: 'https://x', last_price: 15 };
  const result = await checkProduct(product, deps);
  assert.equal(result.status, 'ok');
  assert.equal(result.changed, true);
  assert.equal(mailCalls.length, 1);
  assert.deepEqual(mailCalls[0], { name: 'X', url: 'https://x', oldPrice: 15, newPrice: 10 });
  assert.equal(successes.length, 1);
});

test('checkProduct does not send mail or update when price unchanged', async () => {
  const { deps, successes, mailCalls } = makeDeps();
  const product = { id: 3, name: 'X', url: 'https://x', last_price: 10 };
  const result = await checkProduct(product, deps);
  assert.equal(result.status, 'ok');
  assert.equal(result.changed, false);
  assert.equal(mailCalls.length, 0);
  assert.equal(successes.length, 1);
});

test('checkProduct records fetch_error and does not touch price on network failure', async () => {
  const { deps, failures, successes } = makeDeps({
    fetchAndExtractPrice: async () => ({ price: null, method: null, error: new Error('down') }),
  });
  const product = { id: 4, name: 'X', url: 'https://x', last_price: 10 };
  const result = await checkProduct(product, deps);
  assert.equal(result.status, 'fetch_error');
  assert.deepEqual(failures, [{ id: 4, status: 'fetch_error' }]);
  assert.equal(successes.length, 0);
});

test('checkProduct records not_found when no price could be extracted', async () => {
  const { deps, failures, successes } = makeDeps({
    fetchAndExtractPrice: async () => ({ price: null, method: null, error: null }),
  });
  const product = { id: 5, name: 'X', url: 'https://x', last_price: 10 };
  const result = await checkProduct(product, deps);
  assert.equal(result.status, 'not_found');
  assert.deepEqual(failures, [{ id: 5, status: 'not_found' }]);
  assert.equal(successes.length, 0);
});

test('checkProduct records mail_error and keeps old price when sending mail fails', async () => {
  const { deps, failures, successes } = makeDeps({
    sendMail: async () => { throw new Error('smtp down'); },
  });
  const product = { id: 6, name: 'X', url: 'https://x', last_price: 15 };
  const result = await checkProduct(product, deps);
  assert.equal(result.status, 'mail_error');
  assert.deepEqual(failures, [{ id: 6, status: 'mail_error' }]);
  assert.equal(successes.length, 0);
});

test('checkAllProducts processes every product independently, isolating failures', async () => {
  const { deps } = makeDeps();
  let calls = 0;
  const throwingDeps = {
    ...deps,
    fetchAndExtractPrice: async () => {
      calls += 1;
      if (calls === 1) throw new Error('boom');
      return { price: 10, method: 'meta', error: null };
    },
  };
  const products = [
    { id: 1, name: 'A', url: 'https://a', last_price: null },
    { id: 2, name: 'B', url: 'https://b', last_price: null },
  ];
  const results = await checkAllProducts(products, throwingDeps);
  assert.equal(results.length, 2);
  assert.equal(results[0].status, 'error');
  assert.equal(results[1].status, 'ok');
});

Step 2: Run tests to verify they fail

Run: npm test Expected: FAIL with "Cannot find module '../src/checkProducts'"

Step 3: Write src/checkProducts.js

async function checkProduct(product, deps) {
  const { fetchAndExtractPrice, sendMail, recordCheckSuccess, recordCheckFailure, now } = deps;
  const result = await fetchAndExtractPrice(product.url);

  if (result.error) {
    recordCheckFailure(product.id, 'fetch_error');
    return { status: 'fetch_error' };
  }
  if (result.price == null) {
    recordCheckFailure(product.id, 'not_found');
    return { status: 'not_found' };
  }

  const isBaseline = product.last_price == null;
  const changed = !isBaseline && result.price !== product.last_price;

  if (changed) {
    try {
      await sendMail({
        name: product.name,
        url: product.url,
        oldPrice: product.last_price,
        newPrice: result.price,
      });
    } catch {
      recordCheckFailure(product.id, 'mail_error');
      return { status: 'mail_error' };
    }
  }

  recordCheckSuccess(product.id, { price: result.price, checkedAt: now() });
  return { status: 'ok', changed };
}

async function checkAllProducts(products, deps) {
  const results = [];
  for (const product of products) {
    try {
      const result = await checkProduct(product, deps);
      results.push({ productId: product.id, ...result });
    } catch (err) {
      results.push({ productId: product.id, status: 'error', error: err });
    }
  }
  return results;
}

module.exports = { checkProduct, checkAllProducts };

Step 4: Run tests to verify they pass

Run: npm test Expected: PASS (all checkProducts.test.js tests + previous tests)

Step 5: Commit

git add src/checkProducts.js test/checkProducts.test.js
git commit -m "feat: add price-check orchestration with retry-safe error handling"

Task 6: Scheduler

Files:

Create: src/scheduler.js
Test: test/scheduler.test.js

Interfaces:

Produces:
- start(runCheck: () => Promise<any>, { cron? }) -> ScheduledTask — schedules runCheck with cron expression '0 8 * * *' and { timezone: 'Europe/Amsterdam' }; cron defaults to the real node-cron module and is injectable for tests.
- SCHEDULE (string constant '0 8 * * *'), TIMEZONE (string constant 'Europe/Amsterdam').
Step 1: Write the failing test

Create test/scheduler.test.js:

const test = require('node:test');
const assert = require('node:assert/strict');
const { start, SCHEDULE, TIMEZONE } = require('../src/scheduler');

test('start schedules the daily check with the expected cron expression and timezone', () => {
  const calls = [];
  const fakeCron = {
    schedule: (expression, fn, options) => {
      calls.push({ expression, fn, options });
      return { stop: () => {} };
    },
  };
  const runCheck = async () => {};
  start(runCheck, { cron: fakeCron });
  assert.equal(calls.length, 1);
  assert.equal(calls[0].expression, SCHEDULE);
  assert.equal(SCHEDULE, '0 8 * * *');
  assert.equal(calls[0].options.timezone, TIMEZONE);
  assert.equal(TIMEZONE, 'Europe/Amsterdam');
  assert.equal(calls[0].fn, runCheck);
});

Step 2: Run test to verify it fails

Run: npm test Expected: FAIL with "Cannot find module '../src/scheduler'"

Step 3: Write src/scheduler.js

const cronDefault = require('node-cron');

const SCHEDULE = '0 8 * * *';
const TIMEZONE = 'Europe/Amsterdam';

function start(runCheck, { cron = cronDefault } = {}) {
  return cron.schedule(SCHEDULE, runCheck, { timezone: TIMEZONE });
}

module.exports = { start, SCHEDULE, TIMEZONE };

Step 4: Run test to verify it passes

Run: npm test Expected: PASS (all scheduler.test.js tests + previous tests)

Step 5: Commit

git add src/scheduler.js test/scheduler.test.js
git commit -m "feat: add daily cron scheduler for price checks"

Task 7: Views + Express app

Files:

Create: src/views.js
Create: src/app.js
Test: test/app.test.js

Interfaces:

Consumes: nothing directly imported from earlier tasks — createApp(deps) takes injected listProducts, addProduct, deleteProduct, checkAllProducts, authUser, authPass functions/values, matching the shapes produced by Tasks 2 and 5 (wired together in Task 8).
Produces:
- renderIndex({ products: Product[], message?: string }) -> string (full HTML document)
- createApp(deps) -> express.Application with routes: GET /, POST /products, POST /products/:id/delete, POST /check-now — all behind basic auth.
Step 1: Write the failing tests

Create test/app.test.js:

const test = require('node:test');
const assert = require('node:assert/strict');
const request = require('supertest');
const { createApp } = require('../src/app');

function makeDeps(overrides = {}) {
  let products = [];
  let nextId = 1;
  return {
    listProducts: () => products,
    addProduct: ({ name, url }) => {
      const product = { id: nextId++, name, url, last_price: null, last_checked_at: null, last_check_status: null };
      products.push(product);
      return product;
    },
    deleteProduct: (id) => { products = products.filter((p) => p.id !== id); },
    checkAllProducts: async () => [],
    authUser: 'admin',
    authPass: 'secret',
    ...overrides,
  };
}

test('rejects requests without basic auth credentials', async () => {
  const app = createApp(makeDeps());
  const res = await request(app).get('/');
  assert.equal(res.status, 401);
});

test('lists products for authenticated requests', async () => {
  const deps = makeDeps();
  deps.addProduct({ name: 'Voorbeeld', url: 'https://example.com' });
  const app = createApp(deps);
  const res = await request(app).get('/').auth('admin', 'secret');
  assert.equal(res.status, 200);
  assert.match(res.text, /Voorbeeld/);
});

test('adding a product redirects to the list', async () => {
  const deps = makeDeps();
  const app = createApp(deps);
  const res = await request(app)
    .post('/products')
    .auth('admin', 'secret')
    .type('form')
    .send({ name: 'Nieuw product', url: 'https://shop.example/p' });
  assert.equal(res.status, 302);
  assert.equal(res.headers.location, '/');
  const list = await request(app).get('/').auth('admin', 'secret');
  assert.match(list.text, /Nieuw product/);
});

test('deleting a product removes it from the list', async () => {
  const deps = makeDeps();
  const product = deps.addProduct({ name: 'Weg ermee', url: 'https://shop.example/p2' });
  const app = createApp(deps);
  const res = await request(app).post(`/products/${product.id}/delete`).auth('admin', 'secret');
  assert.equal(res.status, 302);
  const list = await request(app).get('/').auth('admin', 'secret');
  assert.doesNotMatch(list.text, /Weg ermee/);
});

test('check-now triggers checkAllProducts and redirects with confirmation', async () => {
  let called = false;
  const deps = makeDeps({ checkAllProducts: async () => { called = true; return []; } });
  const app = createApp(deps);
  const res = await request(app).post('/check-now').auth('admin', 'secret');
  assert.equal(res.status, 302);
  assert.equal(res.headers.location, '/?checked=1');
  assert.equal(called, true);
});

Step 2: Run tests to verify they fail

Run: npm test Expected: FAIL with "Cannot find module '../src/app'"

Step 3: Write src/views.js

function escapeHtml(str) {
  return String(str).replace(/[&<>"']/g, (c) => ({
    '&': '&amp;', '<': '&lt;', '>': '&gt;', '"': '&quot;', "'": '&#39;',
  }[c]));
}

function formatPrice(price) {
  return price == null ? '—' : `€${Number(price).toFixed(2).replace('.', ',')}`;
}

function renderProductRow(product) {
  return `
    <tr>
      <td>${escapeHtml(product.name)}</td>
      <td><a href="${escapeHtml(product.url)}" target="_blank" rel="noopener">bekijken</a></td>
      <td>${formatPrice(product.last_price)}</td>
      <td>${product.last_checked_at ? escapeHtml(product.last_checked_at) : '—'}</td>
      <td>${escapeHtml(product.last_check_status || '—')}</td>
      <td>
        <form method="post" action="/products/${product.id}/delete" onsubmit="return confirm('Product verwijderen?');">
          <button type="submit">Verwijder</button>
        </form>
      </td>
    </tr>`;
}

function renderIndex({ products, message }) {
  return `<!doctype html>
<html lang="nl">
<head>
  <meta charset="utf-8">
  <title>Korting — prijstracker</title>
  <style>
    body { font-family: system-ui, sans-serif; max-width: 800px; margin: 2rem auto; padding: 0 1rem; }
    table { width: 100%; border-collapse: collapse; margin-top: 1rem; }
    th, td { text-align: left; padding: 0.5rem; border-bottom: 1px solid #ddd; }
    .message { background: #eef; padding: 0.5rem 1rem; border-radius: 4px; }
  </style>
</head>
<body>
  <h1>Korting — prijstracker</h1>
  ${message ? `<p class="message">${escapeHtml(message)}</p>` : ''}
  <form method="post" action="/products">
    <input name="name" placeholder="Naam" required>
    <input name="url" type="url" placeholder="https://..." required>
    <button type="submit">Toevoegen</button>
  </form>
  <form method="post" action="/check-now">
    <button type="submit">Nu controleren</button>
  </form>
  <table>
    <thead>
      <tr><th>Naam</th><th>URL</th><th>Prijs</th><th>Laatst gecontroleerd</th><th>Status</th><th></th></tr>
    </thead>
    <tbody>
      ${products.map(renderProductRow).join('')}
    </tbody>
  </table>
</body>
</html>`;
}

module.exports = { renderIndex, escapeHtml, formatPrice };

Step 4: Write src/app.js

const express = require('express');
const basicAuth = require('express-basic-auth');
const { renderIndex } = require('./views');

function createApp(deps) {
  const {
    listProducts, addProduct, deleteProduct,
    checkAllProducts, authUser, authPass,
  } = deps;

  const app = express();
  app.use(express.urlencoded({ extended: false }));
  app.use(basicAuth({ users: { [authUser]: authPass }, challenge: true }));

  app.get('/', (req, res) => {
    const products = listProducts();
    const message = req.query.checked ? 'Controle uitgevoerd.' : undefined;
    res.send(renderIndex({ products, message }));
  });

  app.post('/products', (req, res) => {
    const { name, url } = req.body;
    if (name && url) {
      addProduct({ name, url });
    }
    res.redirect('/');
  });

  app.post('/products/:id/delete', (req, res) => {
    deleteProduct(Number(req.params.id));
    res.redirect('/');
  });

  app.post('/check-now', async (req, res) => {
    const products = listProducts();
    await checkAllProducts(products);
    res.redirect('/?checked=1');
  });

  return app;
}

module.exports = { createApp };

Step 5: Run tests to verify they pass

Run: npm test Expected: PASS (all app.test.js tests + previous tests)

Step 6: Commit

git add src/views.js src/app.js test/app.test.js
git commit -m "feat: add web UI for managing products"

Task 8: Composition root (server entry point)

Files:

Create: server.js

Interfaces:

Consumes: openDb, listProducts, addProduct, deleteProduct, recordCheckSuccess, recordCheckFailure (Task 2); fetchAndExtractPrice (Task 3); createTransport, sendPriceChangeEmail (Task 4); checkAllProducts (Task 5); start as startScheduler (Task 6); createApp (Task 7).
Produces: the running HTTP server (no further consumers — this is the top of the dependency graph).

No new automated tests here — this file only wires together already-tested pieces. It is verified with a manual smoke check.

Step 1: Write server.js

const path = require('node:path');
const {
  openDb, listProducts, addProduct, deleteProduct,
  recordCheckSuccess, recordCheckFailure,
} = require('./src/db');
const { fetchAndExtractPrice } = require('./src/scraper');
const { createTransport, sendPriceChangeEmail } = require('./src/mailer');
const { checkAllProducts } = require('./src/checkProducts');
const { start: startScheduler } = require('./src/scheduler');
const { createApp } = require('./src/app');

const PORT = process.env.PORT || 3000;
const DB_PATH = process.env.DB_PATH || path.join(__dirname, 'data', 'korting.db');

const db = openDb(DB_PATH);
const transport = createTransport(process.env);

function runCheck() {
  const products = listProducts(db);
  const deps = {
    fetchAndExtractPrice,
    sendMail: (payload) => sendPriceChangeEmail(transport, {
      from: process.env.SMTP_FROM,
      to: process.env.NOTIFY_EMAIL,
      ...payload,
    }),
    recordCheckFailure: (id, status) => recordCheckFailure(db, id, status),
    recordCheckSuccess: (id, data) => recordCheckSuccess(db, id, data),
    now: () => new Date().toISOString(),
  };
  return checkAllProducts(products, deps);
}

startScheduler(runCheck);

const app = createApp({
  listProducts: () => listProducts(db),
  addProduct: (data) => addProduct(db, data),
  deleteProduct: (id) => deleteProduct(db, id),
  checkAllProducts: runCheck,
  authUser: process.env.BASIC_AUTH_USER,
  authPass: process.env.BASIC_AUTH_PASS,
});

app.listen(PORT, () => {
  console.log(`Korting draait op poort ${PORT}`);
});

Step 2: Manual smoke check

Run:

cp .env.example .env
# edit .env: fill in a real BASIC_AUTH_USER/PASS at minimum
set -a; source .env; set +a
node server.js

In another terminal:

curl -u "$BASIC_AUTH_USER:$BASIC_AUTH_PASS" http://localhost:3000/
curl -u "$BASIC_AUTH_USER:$BASIC_AUTH_PASS" -X POST http://localhost:3000/products \
  -d "name=Testproduct" -d "url=https://example.com"
curl -u "$BASIC_AUTH_USER:$BASIC_AUTH_PASS" http://localhost:3000/

Expected: first curl returns the HTML page; second redirects (302); third shows "Testproduct" in the table. Stop the server with Ctrl-C.

Step 3: Commit

git add server.js
git commit -m "feat: wire modules together into the running server"

Task 9: Dockerfile

Files:

Create: Dockerfile
Create: .dockerignore

Interfaces:

Produces: a Docker image that runs node server.js on $PORT (default 3000), expecting /data to be mounted as a persistent volume when DB_PATH=/data/korting.db is set.
Step 1: Write Dockerfile

better-sqlite3 needs a native build step, so this uses a two-stage build: the first stage has build tools to compile it, the second stage is the slim runtime image.

FROM node:22-slim AS build
RUN apt-get update && apt-get install -y --no-install-recommends python3 build-essential \
    && rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci --omit=dev
COPY . .

FROM node:22-slim
WORKDIR /app
COPY --from=build /app .
ENV NODE_ENV=production
EXPOSE 3000
CMD ["node", "server.js"]

Step 2: Write .dockerignore

node_modules
data
.env
.git
docs
test

Step 3: Manual build/run verification

Run:

docker build -t korting .
docker run --rm -p 3000:3000 \
  -e BASIC_AUTH_USER=admin -e BASIC_AUTH_PASS=test \
  -e SMTP_HOST=mail.sjoerd.app -e SMTP_PORT=587 \
  -e SMTP_FROM=korting@sjoerd.app -e NOTIFY_EMAIL=test@example.com \
  -e DB_PATH=/data/korting.db -v korting-data:/data \
  korting

In another terminal: curl -u admin:test http://localhost:3000/ should return the HTML page. Stop with Ctrl-C.

Step 4: Commit

git add Dockerfile .dockerignore
git commit -m "chore: add Dockerfile for deployment"

Task 10: README

Files:

Create: README.md
Step 1: Write README.md

# Korting — prijstracker

Houdt een lijst productpagina's dagelijks in de gaten en mailt wanneer
de prijs verandert.

## Lokaal draaien

\`\`\`bash
npm install
cp .env.example .env   # env vars invullen
npm test
npm start
\`\`\`

De UI staat achter basic auth (`BASIC_AUTH_USER` / `BASIC_AUTH_PASS`) op
`http://localhost:3000`.

## Environment variables

| Variabele | Omschrijving |
|---|---|
| `PORT` | HTTP-poort (default `3000`) |
| `DB_PATH` | Pad naar het SQLite-bestand (default `./data/korting.db`) |
| `BASIC_AUTH_USER` / `BASIC_AUTH_PASS` | Login voor de web-UI |
| `SMTP_HOST` / `SMTP_PORT` / `SMTP_USER` / `SMTP_PASS` | SMTP-server (Postal op `mail.sjoerd.app`) |
| `SMTP_FROM` | Afzenderadres van de notificatiemails |
| `NOTIFY_EMAIL` | Ontvangeradres van de notificatiemails |

## Hoe het werkt

Elke dag om 08:00 (Europe/Amsterdam) doorloopt de app alle producten,
haalt de huidige prijs op (JSON-LD → meta tags → regex-fallback), en
stuurt een email als de prijs is veranderd sinds de vorige geslaagde
check. Een mislukte poging (pagina onbereikbaar, geen prijs gevonden,
of de mail kon niet verstuurd worden) verandert de opgeslagen prijs
niet, zodat de volgende dag opnieuw wordt geprobeerd. Via de "Nu
controleren"-knop in de UI kan dit ook handmatig getriggerd worden.

Step 2: Commit

git add README.md
git commit -m "docs: add README"

After this plan

Deployment (Gitea repo + Coolify application, domain korting.sjoerd.app, persistent volume, env vars, auto-deploy on push) is handled separately via the deploying-to-coolify skill — it is an operational step, not a coding task.

41 KiB Raw Blame History

Korting prijstracker Implementation Plan

Global Constraints

Task 1: Project scaffolding

Task 2: Database module

Task 3: Scraper module

Task 4: Mailer module

Task 5: Price-check core logic

Task 6: Scheduler

Task 7: Views + Express app

Task 8: Composition root (server entry point)

Task 9: Dockerfile

Task 10: README

After this plan

41 KiB

Raw Blame History