Compare commits

..

10 Commits

Author SHA1 Message Date
Sjoerd de Vries 7f378115b5 fix: guard the async /check-now handler against crashing (finding 8)
Express 4 geeft een afgewezen promise uit een async handler niet door aan
de error-handling; met Node 22 crashte daardoor het hele proces. De
handler vangt de fout nu zelf af, logt hem en antwoordt met een 500.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013gwc3aCiyQxmCWQke8RvEz
2026-09-05 15:53:20 +02:00
Sjoerd de Vries d07cda48c6 chore: default DB_PATH to the mounted volume in Docker (finding 5)
Zonder ENV DB_PATH schreef een vergeten env var stilletjes naar de
container-filesystem, waardoor alle data bij de volgende redeploy weg
was. VOLUME /data documenteert het verwachte mountpoint.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013gwc3aCiyQxmCWQke8RvEz
2026-09-05 15:53:08 +02:00
Sjoerd de Vries fb3fc62b4b feat: log failures, validate env at startup, load .env (findings 3, 4, 7)
Finding 3: fetch_error, not_found, mail_error and de onverwachte-fout
branch loggen nu naam/url plus reden naar stderr; runCheck logt na elke
run een regel met de tellingen per status, zodat een cron-run zichtbaar
is in de container-logs. De recordCheckSuccess/recordCheckFailure-condities
en alle return-waarden zijn ongewijzigd.

Finding 4: server.js controleert bij het opstarten dat BASIC_AUTH_USER,
BASIC_AUTH_PASS, SMTP_FROM en NOTIFY_EMAIL gezet zijn en stopt anders met
exit code 1 in plaats van later opake 500's te geven.

Finding 7: dotenv toegevoegd en als eerste regel in server.js geladen,
zodat het gedocumenteerde `cp .env.example .env && npm start` echt werkt.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013gwc3aCiyQxmCWQke8RvEz
2026-09-05 15:52:57 +02:00
Sjoerd de Vries 52c9947267 fix: harden price scraping (findings 1, 2, 6)
Finding 1: parsePriceValue mishandled thousands separators — "1.299,00"
and "1,299.00" both parsed to 1.3. Both the meta-tag path and the
text-regex path now funnel through a single normalizeAmount() helper
that picks the last separator as the decimal one, and the text regex
also matches thousands-grouped amounts.

Finding 2: the text fallback read <script>/<style> contents via
$('body').text(); those elements are now stripped from a clone first.

Finding 6: the default fetch now uses a 15s AbortSignal.timeout and a
browser-like User-Agent, so one unresponsive host cannot stall the
whole sequential run. The fetchImpl injection point is unchanged.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013gwc3aCiyQxmCWQke8RvEz
2026-09-05 15:49:56 +02:00
Sjoerd de Vries 862699d35b docs: add README 2026-09-05 15:40:08 +02:00
Sjoerd de Vries 195dff6fe3 chore: add Dockerfile for deployment 2026-09-05 15:28:31 +02:00
Sjoerd de Vries 5b6d672506 feat: wire modules together into the running server
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013gwc3aCiyQxmCWQke8RvEz
2026-09-05 15:21:15 +02:00
Sjoerd de Vries f73ea41a6c feat: add web UI for managing products
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013gwc3aCiyQxmCWQke8RvEz
2026-09-05 15:17:42 +02:00
Sjoerd de Vries 3b0e824c5b feat: add daily cron scheduler for price checks
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-05 15:15:09 +02:00
Sjoerd de Vries 68cb8d0c08 feat: add price-check orchestration with retry-safe error handling
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013gwc3aCiyQxmCWQke8RvEz
2026-09-05 15:12:16 +02:00
18 changed files with 647 additions and 7 deletions
+6
View File
@@ -0,0 +1,6 @@
node_modules
data
.env
.git
docs
test
+19
View File
@@ -0,0 +1,19 @@
FROM node:22-slim AS build
RUN apt-get update && apt-get install -y --no-install-recommends python3 build-essential \
&& rm -rf /var/lib/apt/lists/*
WORKDIR /app
COPY package.json package-lock.json ./
RUN npm ci --omit=dev
COPY . .
FROM node:22-slim
WORKDIR /app
COPY --from=build /app .
ENV NODE_ENV=production
# De SQLite-database hoort op het persistente volume te staan; zonder deze
# default zou een vergeten DB_PATH stilletjes naar de container-fs schrijven
# en bij elke redeploy alle data verliezen.
ENV DB_PATH=/data/korting.db
VOLUME /data
EXPOSE 3000
CMD ["node", "server.js"]
+41
View File
@@ -0,0 +1,41 @@
# Korting — prijstracker
Houdt een lijst productpagina's dagelijks in de gaten en mailt wanneer
de prijs verandert.
## Lokaal draaien
```bash
npm install
cp .env.example .env # env vars invullen
npm test
npm start
```
De app leest `.env` in via `dotenv`. `BASIC_AUTH_USER`, `BASIC_AUTH_PASS`,
`SMTP_FROM` en `NOTIFY_EMAIL` zijn verplicht; ontbreken ze, dan stopt de
app bij het opstarten met een melding.
De UI staat achter basic auth (`BASIC_AUTH_USER` / `BASIC_AUTH_PASS`) op
`http://localhost:3000`.
## Environment variables
| Variabele | Omschrijving |
|---|---|
| `PORT` | HTTP-poort (default `3000`) |
| `DB_PATH` | Pad naar het SQLite-bestand (default `./data/korting.db`) |
| `BASIC_AUTH_USER` / `BASIC_AUTH_PASS` | Login voor de web-UI |
| `SMTP_HOST` / `SMTP_PORT` / `SMTP_USER` / `SMTP_PASS` | SMTP-server (Postal op `mail.sjoerd.app`) |
| `SMTP_FROM` | Afzenderadres van de notificatiemails |
| `NOTIFY_EMAIL` | Ontvangeradres van de notificatiemails |
## Hoe het werkt
Elke dag om 08:00 (Europe/Amsterdam) doorloopt de app alle producten,
haalt de huidige prijs op (JSON-LD → meta tags → regex-fallback), en
stuurt een email als de prijs is veranderd sinds de vorige geslaagde
check. Een mislukte poging (pagina onbereikbaar, geen prijs gevonden,
of de mail kon niet verstuurd worden) verandert de opgeslagen prijs
niet, zodat de volgende dag opnieuw wordt geprobeerd. Via de "Nu
controleren"-knop in de UI kan dit ook handmatig getriggerd worden.
+13
View File
@@ -10,6 +10,7 @@
"dependencies": {
"better-sqlite3": "^11.3.0",
"cheerio": "^1.0.0",
"dotenv": "^17.4.2",
"express": "^4.19.2",
"express-basic-auth": "^1.2.1",
"node-cron": "^3.0.3",
@@ -527,6 +528,18 @@
"url": "https://github.com/fb55/domutils?sponsor=1"
}
},
"node_modules/dotenv": {
"version": "17.4.2",
"resolved": "https://registry.npmjs.org/dotenv/-/dotenv-17.4.2.tgz",
"integrity": "sha512-nI4U3TottKAcAD9LLud4Cb7b2QztQMUEfHbvhTH09bqXTxnSie8WnjPALV/WMCrJZ6UV/qHJ6L03OqO3LcdYZw==",
"license": "BSD-2-Clause",
"engines": {
"node": ">=12"
},
"funding": {
"url": "https://dotenvx.com"
}
},
"node_modules/dunder-proto": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/dunder-proto/-/dunder-proto-1.0.1.tgz",
+1
View File
@@ -11,6 +11,7 @@
"dependencies": {
"better-sqlite3": "^11.3.0",
"cheerio": "^1.0.0",
"dotenv": "^17.4.2",
"express": "^4.19.2",
"express-basic-auth": "^1.2.1",
"node-cron": "^3.0.3",
+71
View File
@@ -0,0 +1,71 @@
// Leest een .env-bestand in voor lokale ontwikkeling; in productie komen de
// env vars van Coolify. quiet houdt de opstartlogging schoon.
require('dotenv').config({ quiet: true });
const path = require('node:path');
const {
openDb, listProducts, addProduct, deleteProduct,
recordCheckSuccess, recordCheckFailure,
} = require('./src/db');
const { fetchAndExtractPrice } = require('./src/scraper');
const { createTransport, sendPriceChangeEmail } = require('./src/mailer');
const { checkAllProducts } = require('./src/checkProducts');
const { start: startScheduler } = require('./src/scheduler');
const { createApp } = require('./src/app');
const REQUIRED_ENV = ['BASIC_AUTH_USER', 'BASIC_AUTH_PASS', 'SMTP_FROM', 'NOTIFY_EMAIL'];
const missing = REQUIRED_ENV.filter((name) => !process.env[name]);
if (missing.length > 0) {
console.error(`Ontbrekende verplichte environment variables: ${missing.join(', ')}`);
console.error('Zet ze (bijvoorbeeld in .env, zie .env.example) en start opnieuw.');
process.exit(1);
}
const PORT = process.env.PORT || 3000;
const DB_PATH = process.env.DB_PATH || path.join(__dirname, 'data', 'korting.db');
const db = openDb(DB_PATH);
const transport = createTransport(process.env);
function summarize(results) {
const counts = {};
for (const result of results) {
counts[result.status] = (counts[result.status] || 0) + 1;
}
const parts = Object.entries(counts).map(([status, count]) => `${status}=${count}`);
return parts.length > 0 ? parts.join(' ') : 'geen producten';
}
function runCheck() {
const products = listProducts(db);
const deps = {
fetchAndExtractPrice,
sendMail: (payload) => sendPriceChangeEmail(transport, {
from: process.env.SMTP_FROM,
to: process.env.NOTIFY_EMAIL,
...payload,
}),
recordCheckFailure: (id, status) => recordCheckFailure(db, id, status),
recordCheckSuccess: (id, data) => recordCheckSuccess(db, id, data),
now: () => new Date().toISOString(),
};
return checkAllProducts(products, deps).then((results) => {
console.log(`[check] ${products.length} product(en) gecontroleerd: ${summarize(results)}`);
return results;
});
}
startScheduler(runCheck);
const app = createApp({
listProducts: () => listProducts(db),
addProduct: (data) => addProduct(db, data),
deleteProduct: (id) => deleteProduct(db, id),
checkAllProducts: runCheck,
authUser: process.env.BASIC_AUTH_USER,
authPass: process.env.BASIC_AUTH_PASS,
});
app.listen(PORT, () => {
console.log(`Korting draait op poort ${PORT}`);
});
+50
View File
@@ -0,0 +1,50 @@
const express = require('express');
const basicAuth = require('express-basic-auth');
const { renderIndex } = require('./views');
function createApp(deps) {
const {
listProducts, addProduct, deleteProduct,
checkAllProducts, authUser, authPass,
} = deps;
const app = express();
app.use(express.urlencoded({ extended: false }));
app.use(basicAuth({ users: { [authUser]: authPass }, challenge: true }));
app.get('/', (req, res) => {
const products = listProducts();
const message = req.query.checked ? 'Controle uitgevoerd.' : undefined;
res.send(renderIndex({ products, message }));
});
app.post('/products', (req, res) => {
const { name, url } = req.body;
if (name && url) {
addProduct({ name, url });
}
res.redirect('/');
});
app.post('/products/:id/delete', (req, res) => {
deleteProduct(Number(req.params.id));
res.redirect('/');
});
// Express 4 vangt een afgewezen promise uit een async handler niet af, en
// Node 22 laat het proces daarop crashen — dus zelf afvangen.
app.post('/check-now', async (req, res) => {
try {
const products = listProducts();
await checkAllProducts(products);
res.redirect('/?checked=1');
} catch (err) {
console.error(`[check-now] mislukt: ${err && err.message}`);
res.status(500).send('Er ging iets mis bij het controleren van de prijzen.');
}
});
return app;
}
module.exports = { createApp };
+52
View File
@@ -0,0 +1,52 @@
async function checkProduct(product, deps) {
const { fetchAndExtractPrice, sendMail, recordCheckSuccess, recordCheckFailure, now } = deps;
const result = await fetchAndExtractPrice(product.url);
if (result.error) {
console.error(`[check] fetch_error "${product.name}" (${product.url}): ${result.error.message}`);
recordCheckFailure(product.id, 'fetch_error');
return { status: 'fetch_error' };
}
if (result.price == null) {
console.error(`[check] not_found "${product.name}" (${product.url}): geen prijs gevonden op de pagina`);
recordCheckFailure(product.id, 'not_found');
return { status: 'not_found' };
}
const isBaseline = product.last_price == null;
const changed = !isBaseline && result.price !== product.last_price;
if (changed) {
try {
await sendMail({
name: product.name,
url: product.url,
oldPrice: product.last_price,
newPrice: result.price,
});
} catch (err) {
console.error(`[check] mail_error "${product.name}" (${product.url}): ${err && err.message}`);
recordCheckFailure(product.id, 'mail_error');
return { status: 'mail_error' };
}
}
recordCheckSuccess(product.id, { price: result.price, checkedAt: now() });
return { status: 'ok', changed };
}
async function checkAllProducts(products, deps) {
const results = [];
for (const product of products) {
try {
const result = await checkProduct(product, deps);
results.push({ productId: product.id, ...result });
} catch (err) {
console.error(`[check] error "${product.name}" (${product.url}): ${err && err.message}`);
results.push({ productId: product.id, status: 'error', error: err });
}
}
return results;
}
module.exports = { checkProduct, checkAllProducts };
+10
View File
@@ -0,0 +1,10 @@
const cronDefault = require('node-cron');
const SCHEDULE = '0 8 * * *';
const TIMEZONE = 'Europe/Amsterdam';
function start(runCheck, { cron = cronDefault } = {}) {
return cron.schedule(SCHEDULE, runCheck, { timezone: TIMEZONE });
}
module.exports = { start, SCHEDULE, TIMEZONE };
+44 -6
View File
@@ -4,10 +4,32 @@ function roundMoney(n) {
return Math.round(n * 100) / 100;
}
function parsePriceValue(raw) {
// Turns a human-written amount into a plain JS-parsable number string.
// When both separators occur, the last one is the decimal separator
// ("1.299,00" and "1,299.00" both mean 1299.00); a lone comma is decimal
// only when exactly two digits follow it, otherwise it groups thousands.
function normalizeAmount(raw) {
if (raw == null) return null;
const cleaned = String(raw).trim().replace(',', '.');
const value = parseFloat(cleaned);
const match = String(raw).trim().match(/-?\d[\d.,]*/);
if (!match) return null;
const str = match[0].replace(/[.,]+$/, '');
const lastDot = str.lastIndexOf('.');
const lastComma = str.lastIndexOf(',');
if (lastDot !== -1 && lastComma !== -1) {
return lastComma > lastDot
? str.replace(/\./g, '').replace(',', '.')
: str.replace(/,/g, '');
}
if (lastComma !== -1) {
return /,\d{2}$/.test(str) ? str.replace(',', '.') : str.replace(/,/g, '');
}
return str;
}
function parsePriceValue(raw) {
const normalized = normalizeAmount(raw);
if (normalized == null) return null;
const value = parseFloat(normalized);
return Number.isFinite(value) ? roundMoney(value) : null;
}
@@ -59,9 +81,14 @@ function findPriceInMeta($) {
return null;
}
// Thousands-grouped amounts first, then a plain amount, so "€1.299,00"
// is not truncated to "€1.29".
const TEXT_PRICE_RE = /€\s?(\d{1,3}(?:[.,]\d{3})+[.,]\d{2}|\d+[.,]\d{2})/;
function findPriceInText($) {
const text = $('body').text();
const match = text.match(/€\s?(\d+[.,]\d{2})/);
const text = $('body').clone().find('script, style, noscript').remove().end()
.text();
const match = text.match(TEXT_PRICE_RE);
return match ? parsePriceValue(match[1]) : null;
}
@@ -76,7 +103,18 @@ function extractPrice(html) {
return { price: null, method: null };
}
async function fetchAndExtractPrice(url, { fetchImpl = fetch } = {}) {
const FETCH_TIMEOUT_MS = 15000;
const USER_AGENT = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 '
+ '(KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36';
function defaultFetch(url) {
return fetch(url, {
signal: AbortSignal.timeout(FETCH_TIMEOUT_MS),
headers: { 'User-Agent': USER_AGENT },
});
}
async function fetchAndExtractPrice(url, { fetchImpl = defaultFetch } = {}) {
let response;
try {
response = await fetchImpl(url);
+63
View File
@@ -0,0 +1,63 @@
function escapeHtml(str) {
return String(str).replace(/[&<>"']/g, (c) => ({
'&': '&amp;', '<': '&lt;', '>': '&gt;', '"': '&quot;', "'": '&#39;',
}[c]));
}
function formatPrice(price) {
return price == null ? '—' : `${Number(price).toFixed(2).replace('.', ',')}`;
}
function renderProductRow(product) {
return `
<tr>
<td>${escapeHtml(product.name)}</td>
<td><a href="${escapeHtml(product.url)}" target="_blank" rel="noopener">bekijken</a></td>
<td>${formatPrice(product.last_price)}</td>
<td>${product.last_checked_at ? escapeHtml(product.last_checked_at) : '—'}</td>
<td>${escapeHtml(product.last_check_status || '—')}</td>
<td>
<form method="post" action="/products/${product.id}/delete" onsubmit="return confirm('Product verwijderen?');">
<button type="submit">Verwijder</button>
</form>
</td>
</tr>`;
}
function renderIndex({ products, message }) {
return `<!doctype html>
<html lang="nl">
<head>
<meta charset="utf-8">
<title>Korting — prijstracker</title>
<style>
body { font-family: system-ui, sans-serif; max-width: 800px; margin: 2rem auto; padding: 0 1rem; }
table { width: 100%; border-collapse: collapse; margin-top: 1rem; }
th, td { text-align: left; padding: 0.5rem; border-bottom: 1px solid #ddd; }
.message { background: #eef; padding: 0.5rem 1rem; border-radius: 4px; }
</style>
</head>
<body>
<h1>Korting — prijstracker</h1>
${message ? `<p class="message">${escapeHtml(message)}</p>` : ''}
<form method="post" action="/products">
<input name="name" placeholder="Naam" required>
<input name="url" type="url" placeholder="https://..." required>
<button type="submit">Toevoegen</button>
</form>
<form method="post" action="/check-now">
<button type="submit">Nu controleren</button>
</form>
<table>
<thead>
<tr><th>Naam</th><th>URL</th><th>Prijs</th><th>Laatst gecontroleerd</th><th>Status</th><th></th></tr>
</thead>
<tbody>
${products.map(renderProductRow).join('')}
</tbody>
</table>
</body>
</html>`;
}
module.exports = { renderIndex, escapeHtml, formatPrice };
+87
View File
@@ -0,0 +1,87 @@
const test = require('node:test');
const assert = require('node:assert/strict');
const request = require('supertest');
const { createApp } = require('../src/app');
function makeDeps(overrides = {}) {
let products = [];
let nextId = 1;
return {
listProducts: () => products,
addProduct: ({ name, url }) => {
const product = { id: nextId++, name, url, last_price: null, last_checked_at: null, last_check_status: null };
products.push(product);
return product;
},
deleteProduct: (id) => { products = products.filter((p) => p.id !== id); },
checkAllProducts: async () => [],
authUser: 'admin',
authPass: 'secret',
...overrides,
};
}
test('rejects requests without basic auth credentials', async () => {
const app = createApp(makeDeps());
const res = await request(app).get('/');
assert.equal(res.status, 401);
});
test('lists products for authenticated requests', async () => {
const deps = makeDeps();
deps.addProduct({ name: 'Voorbeeld', url: 'https://example.com' });
const app = createApp(deps);
const res = await request(app).get('/').auth('admin', 'secret');
assert.equal(res.status, 200);
assert.match(res.text, /Voorbeeld/);
});
test('adding a product redirects to the list', async () => {
const deps = makeDeps();
const app = createApp(deps);
const res = await request(app)
.post('/products')
.auth('admin', 'secret')
.type('form')
.send({ name: 'Nieuw product', url: 'https://shop.example/p' });
assert.equal(res.status, 302);
assert.equal(res.headers.location, '/');
const list = await request(app).get('/').auth('admin', 'secret');
assert.match(list.text, /Nieuw product/);
});
test('deleting a product removes it from the list', async () => {
const deps = makeDeps();
const product = deps.addProduct({ name: 'Weg ermee', url: 'https://shop.example/p2' });
const app = createApp(deps);
const res = await request(app).post(`/products/${product.id}/delete`).auth('admin', 'secret');
assert.equal(res.status, 302);
const list = await request(app).get('/').auth('admin', 'secret');
assert.doesNotMatch(list.text, /Weg ermee/);
});
test('check-now triggers checkAllProducts and redirects with confirmation', async () => {
let called = false;
const deps = makeDeps({ checkAllProducts: async () => { called = true; return []; } });
const app = createApp(deps);
const res = await request(app).post('/check-now').auth('admin', 'secret');
assert.equal(res.status, 302);
assert.equal(res.headers.location, '/?checked=1');
assert.equal(called, true);
});
test('check-now answers 500 when the check fails, without crashing the process', async (t) => {
t.mock.method(console, 'error', () => {});
const deps = makeDeps({
checkAllProducts: async () => { throw new Error('db weg'); },
});
deps.addProduct({ name: 'Nog steeds hier', url: 'https://shop.example/p3' });
const app = createApp(deps);
const res = await request(app).post('/check-now').auth('admin', 'secret');
assert.equal(res.status, 500);
assert.match(res.text, /Er ging iets mis/);
// Same process, same app instance: proves nothing crashed.
const list = await request(app).get('/').auth('admin', 'secret');
assert.equal(list.status, 200);
assert.match(list.text, /Nog steeds hier/);
});
+121
View File
@@ -0,0 +1,121 @@
const test = require('node:test');
const assert = require('node:assert/strict');
const { checkProduct, checkAllProducts } = require('../src/checkProducts');
function makeDeps(overrides = {}) {
const failures = [];
const successes = [];
const mailCalls = [];
const deps = {
fetchAndExtractPrice: async () => ({ price: 10, method: 'meta', error: null }),
sendMail: async (payload) => { mailCalls.push(payload); },
recordCheckFailure: (id, status) => failures.push({ id, status }),
recordCheckSuccess: (id, data) => successes.push({ id, ...data }),
now: () => '2026-09-05T08:00:00.000Z',
...overrides,
};
return { deps, failures, successes, mailCalls };
}
// Failure paths log to stderr; capture it so the test output stays clean
// while still proving the logging happened.
function captureErrorLog(t) {
const logged = [];
t.mock.method(console, 'error', (...args) => { logged.push(args.join(' ')); });
return logged;
}
test('checkProduct stores baseline price without sending mail', async () => {
const { deps, successes, mailCalls } = makeDeps();
const product = { id: 1, name: 'X', url: 'https://x', last_price: null };
const result = await checkProduct(product, deps);
assert.equal(result.status, 'ok');
assert.equal(result.changed, false);
assert.equal(mailCalls.length, 0);
assert.deepEqual(successes, [{ id: 1, price: 10, checkedAt: '2026-09-05T08:00:00.000Z' }]);
});
test('checkProduct sends mail and updates price when price changed', async () => {
const { deps, successes, mailCalls } = makeDeps();
const product = { id: 2, name: 'X', url: 'https://x', last_price: 15 };
const result = await checkProduct(product, deps);
assert.equal(result.status, 'ok');
assert.equal(result.changed, true);
assert.equal(mailCalls.length, 1);
assert.deepEqual(mailCalls[0], { name: 'X', url: 'https://x', oldPrice: 15, newPrice: 10 });
assert.equal(successes.length, 1);
});
test('checkProduct does not send mail or update when price unchanged', async () => {
const { deps, successes, mailCalls } = makeDeps();
const product = { id: 3, name: 'X', url: 'https://x', last_price: 10 };
const result = await checkProduct(product, deps);
assert.equal(result.status, 'ok');
assert.equal(result.changed, false);
assert.equal(mailCalls.length, 0);
assert.equal(successes.length, 1);
});
test('checkProduct records fetch_error and does not touch price on network failure', async (t) => {
const logged = captureErrorLog(t);
const { deps, failures, successes } = makeDeps({
fetchAndExtractPrice: async () => ({ price: null, method: null, error: new Error('down') }),
});
const product = { id: 4, name: 'X', url: 'https://x', last_price: 10 };
const result = await checkProduct(product, deps);
assert.equal(result.status, 'fetch_error');
assert.deepEqual(failures, [{ id: 4, status: 'fetch_error' }]);
assert.equal(successes.length, 0);
assert.equal(logged.length, 1);
assert.match(logged[0], /fetch_error.*https:\/\/x.*down/);
});
test('checkProduct records not_found when no price could be extracted', async (t) => {
const logged = captureErrorLog(t);
const { deps, failures, successes } = makeDeps({
fetchAndExtractPrice: async () => ({ price: null, method: null, error: null }),
});
const product = { id: 5, name: 'X', url: 'https://x', last_price: 10 };
const result = await checkProduct(product, deps);
assert.equal(result.status, 'not_found');
assert.deepEqual(failures, [{ id: 5, status: 'not_found' }]);
assert.equal(successes.length, 0);
assert.equal(logged.length, 1);
assert.match(logged[0], /not_found.*https:\/\/x/);
});
test('checkProduct records mail_error and keeps old price when sending mail fails', async (t) => {
const logged = captureErrorLog(t);
const { deps, failures, successes } = makeDeps({
sendMail: async () => { throw new Error('smtp down'); },
});
const product = { id: 6, name: 'X', url: 'https://x', last_price: 15 };
const result = await checkProduct(product, deps);
assert.equal(result.status, 'mail_error');
assert.deepEqual(failures, [{ id: 6, status: 'mail_error' }]);
assert.equal(successes.length, 0);
assert.equal(logged.length, 1);
assert.match(logged[0], /mail_error.*https:\/\/x.*smtp down/);
});
test('checkAllProducts processes every product independently, isolating failures', async (t) => {
captureErrorLog(t);
const { deps } = makeDeps();
let calls = 0;
const throwingDeps = {
...deps,
fetchAndExtractPrice: async () => {
calls += 1;
if (calls === 1) throw new Error('boom');
return { price: 10, method: 'meta', error: null };
},
};
const products = [
{ id: 1, name: 'A', url: 'https://a', last_price: null },
{ id: 2, name: 'B', url: 'https://b', last_price: null },
];
const results = await checkAllProducts(products, throwingDeps);
assert.equal(results.length, 2);
assert.equal(results[0].status, 'error');
assert.equal(results[1].status, 'ok');
});
+5
View File
@@ -0,0 +1,5 @@
<html><body>
<script>var config = {"fallbackPrice": "€9,99"};</script>
<style>.price:after { content: "€1,11"; }</style>
<p>Onze prijs: &euro;49,95 vandaag</p>
</body></html>
+3
View File
@@ -0,0 +1,3 @@
<html><head>
<meta property="product:price:amount" content="1.299,00" />
</head><body><p>Duur product</p></body></html>
+1
View File
@@ -0,0 +1 @@
<html><body><p>Nu voor &euro;1.299,00 incl. btw</p></body></html>
+21
View File
@@ -0,0 +1,21 @@
const test = require('node:test');
const assert = require('node:assert/strict');
const { start, SCHEDULE, TIMEZONE } = require('../src/scheduler');
test('start schedules the daily check with the expected cron expression and timezone', () => {
const calls = [];
const fakeCron = {
schedule: (expression, fn, options) => {
calls.push({ expression, fn, options });
return { stop: () => {} };
},
};
const runCheck = async () => {};
start(runCheck, { cron: fakeCron });
assert.equal(calls.length, 1);
assert.equal(calls[0].expression, SCHEDULE);
assert.equal(SCHEDULE, '0 8 * * *');
assert.equal(calls[0].options.timezone, TIMEZONE);
assert.equal(TIMEZONE, 'Europe/Amsterdam');
assert.equal(calls[0].fn, runCheck);
});
+39 -1
View File
@@ -2,7 +2,7 @@ const test = require('node:test');
const assert = require('node:assert/strict');
const fs = require('node:fs');
const path = require('node:path');
const { extractPrice, fetchAndExtractPrice } = require('../src/scraper');
const { extractPrice, fetchAndExtractPrice, parsePriceValue } = require('../src/scraper');
function loadFixture(name) {
return fs.readFileSync(path.join(__dirname, 'fixtures', name), 'utf8');
@@ -28,6 +28,31 @@ test('extractPrice returns null price when nothing matches', () => {
assert.deepEqual(result, { price: null, method: null });
});
test('parsePriceValue handles both thousands-separator conventions', () => {
assert.equal(parsePriceValue('1.299,00'), 1299.00);
assert.equal(parsePriceValue('1,299.00'), 1299.00);
assert.equal(parsePriceValue('19,99'), 19.99);
assert.equal(parsePriceValue('19.99'), 19.99);
assert.equal(parsePriceValue('1.234.567,89'), 1234567.89);
assert.equal(parsePriceValue(null), null);
assert.equal(parsePriceValue('geen prijs'), null);
});
test('extractPrice reads a thousands-grouped price from a meta tag', () => {
const result = extractPrice(loadFixture('thousands-meta.html'));
assert.deepEqual(result, { price: 1299.00, method: 'meta' });
});
test('extractPrice reads a thousands-grouped price from page text', () => {
const result = extractPrice(loadFixture('thousands-regex.html'));
assert.deepEqual(result, { price: 1299.00, method: 'regex' });
});
test('extractPrice ignores euro amounts inside script and style tags', () => {
const result = extractPrice(loadFixture('script-decoy.html'));
assert.deepEqual(result, { price: 49.95, method: 'regex' });
});
test('fetchAndExtractPrice returns error when fetch rejects', async () => {
const fetchImpl = async () => { throw new Error('network down'); };
const result = await fetchAndExtractPrice('https://example.invalid', { fetchImpl });
@@ -42,6 +67,19 @@ test('fetchAndExtractPrice returns error on non-OK response', async () => {
assert.ok(result.error instanceof Error);
});
test('fetchAndExtractPrice returns error when the request times out', async () => {
const fetchImpl = async () => {
const err = new Error('The operation was aborted due to timeout');
err.name = 'TimeoutError';
throw err;
};
const result = await fetchAndExtractPrice('https://example.invalid', { fetchImpl });
assert.deepEqual(result.price, null);
assert.equal(result.method, null);
assert.ok(result.error instanceof Error);
assert.equal(result.error.name, 'TimeoutError');
});
test('fetchAndExtractPrice extracts price from fetched HTML', async () => {
const html = loadFixture('meta-tag.html');
const fetchImpl = async () => ({ ok: true, status: 200, text: async () => html });