Commit graph

81 commits

Author SHA1 Message Date
tgiachi
e4e08cd25e test(scale): measure big lists and big archives, document the numbers
- Add make scale: a 10000-member fan-out through the queue and a 50000-message import with search latencies
- Report the figures in the README
2026-11-09 20:32:09 +01:00
tgiachi
e7e1afbde9 fix: approve a held message once, however many approvals race
- Release removes the held message and queues its deliveries in one transaction
- Of several simultaneous approvals one wins, the others get 404
- Extract the thread-root computation and test its linearity without the database (fast under -race)
2026-11-09 20:29:42 +01:00
tgiachi
fd113c4ac2 fix(security): bound the moderation queue
- Cap the held queue at 500 per list and drop messages nobody moderated for 30 days
- Answer 452 when it is full, so a legitimate sender retries and a spam run goes away
2026-11-09 20:26:33 +01:00
tgiachi
bad3c9bc10 fix(security): cap confirmation mail per address and in total 2026-11-09 20:25:53 +01:00
tgiachi
625d487eca fix(security): API token strength and brute-force lockout
- Refuse an API token under 24 characters or with whitespace at startup, without echoing it
- Lock an address out for 5 minutes after 10 wrong or missing tokens in 5 minutes (429 with Retry-After)
- Ignore X-Forwarded-For for the lockout, forget idle addresses
- Document both
2026-11-09 20:25:05 +01:00
tgiachi
ba627a50b8 fix(security): discard mail loops and automatic replies
- Drop posts that carry the list's own List-Id, Auto-Submitted other than no, or Precedence: auto_reply
- Acknowledge them to the MTA instead of refusing, so no bounce starts another loop
- Never archive, hold or queue a discarded message
2026-11-09 20:24:16 +01:00
tgiachi
eb9e915805 fix(security): hold posts whose From header does not match the sender
- Judge membership on the envelope sender and require the From header to name the same address
- Hold mismatches (and missing, unparsable or multiple From addresses) for the admin instead of delivering
- Record why a message was held: new held.reason column, migrated, shown in the API, CLI and UI
- Accept an empty message in the held queue
- Document who may post
2026-11-09 20:23:07 +01:00
tgiachi
064e8beed8 fix(security): LMTP allow list and a warning for an open LMTP port
- Add lmtp.allow: clients outside the list are dropped before the greeting (TCP only)
- Validate the list at load, naming the bad entry
- Warn at startup when LMTP is reachable beyond the machine and no list is set
- Stop publishing the LMTP port in the README and the example config; add Security notes
2026-11-09 20:20:35 +01:00
tgiachi
7ba07501c3 chore: Go 1.27.3, GPL-3.0-or-later licence, lint and vulnerability checks
- Require Go 1.27.3: fixes the standard library vulnerabilities govulncheck reported
- Add the GPLv3 text and the licence note
- Move shadcn to devDependencies: npm audit finds nothing in production dependencies
- Add make lint (staticcheck) to go-ci and a separate vuln job in CI
- Fix the staticcheck warning in a test
2026-11-09 20:18:57 +01:00
squidd
9d31549f50 Merge pull request 'DKIM signing of outgoing mail' (#25) from feat/dkim into main 2026-11-09 20:06:42 +01:00
tgiachi
849a85ec7d feat: end-to-end DKIM check through a real Postfix, docs and example config
- Check that every message the list delivered through Postfix is signed with the list domain and still verifies, and that edits break it
- Run dkim keygen, record and the no-overwrite rule inside the read-only container
- Document DKIM, SPF and DMARC in the README and show the dkim block in the example config
2026-11-09 20:04:32 +01:00
tgiachi
e7cc9cacfb test: close gaps found by mutation checks on DKIM
- Cover headers outside the signed list changing in transit, quoted local parts with @, signing errors and lowercase domains
- Ignore *.pem so a private key is never committed
2026-11-09 20:02:54 +01:00
tgiachi
1e223f1ad1 feat(cli): dkim keygen, record and check
- Create a signing key (mode 0600, never overwritten without --force) and print the DNS record and the config snippet
- Print the record of an existing key, split in DNS-sized strings
- Check that DNS publishes the key that matches the private key file
- Work offline of the API: no token needed
2026-11-09 20:01:36 +01:00
tgiachi
45303b7143 feat(dkimsign): key generation, DNS record formatting and parsing helpers 2026-11-09 20:00:47 +01:00
tgiachi
dca9117a2e feat(app): sign outgoing mail with the configured DKIM keys
- Load the keys at startup and refuse to start on a broken one
- Sign every outgoing message, confirmation mail included, with the key of its From domain
- Send unsigned when the domain has no key
2026-11-09 19:59:57 +01:00
tgiachi
aa55cd3c17 feat(queue): optional signing hook applied to the final message 2026-11-09 19:58:47 +01:00
tgiachi
be413a75e0 feat(config): dkim keys per domain, validated at load 2026-11-09 19:58:24 +01:00
tgiachi
141c5d98e9 feat(dkimsign): sign messages per From domain with RSA or Ed25519 keys
- Add Signer choosing the key by the From address domain: exact match, then parent domains
- Sign list headers too, relaxed/relaxed sha256, never a body length limit
- Load PKCS#1 and PKCS#8 PEM keys, reject unusable files without echoing key material
- Leave messages without a matching key or a usable From untouched
2026-11-09 19:58:07 +01:00
squidd
fb313736bb Merge pull request 'Import archives from mbox files (Mailman migration)' (#23) from feat/archive-import into main 2026-11-09 19:53:57 +01:00
tgiachi
b777acc154 feat: end-to-end test and docs for the archive import
- Import a Mailman-style mbox through the CLI inside the container: encoded headers, multipart, escaped From lines, threading, idempotence
- Check accent-insensitive search and the public page hiding addresses
- Document the migration from Mailman in the README and the endpoint in OpenAPI
2026-11-09 19:52:14 +01:00
tgiachi
a170204ecb test: close gaps found by mutation checks on the archive import
- Cover the CRLF separator line, cycle roots entered from outside, linear rethreading of long chains
- Cover a read error mid-import, the "and N more" lines, and the mbox content type
- Cover clearing the previous result and the capped error list in the UI
2026-11-09 19:50:40 +01:00
tgiachi
d8cc5ed68a feat(admin): import an mbox from the Archive tab
- Add upload of an mbox file with a summary of imported, already present and failed messages
- List the first failing messages and refresh the archive
- Stream the file as the raw request body
2026-11-09 19:45:48 +01:00
tgiachi
cc47784fdb feat(cli): archive import from a file or stdin
- Add parsimony archive import <list> <file|->, streaming with no overall timeout
- Print imported, already present and failed counts; report failing messages and exit 1
- Support --json
2026-11-09 17:15:43 +01:00
tgiachi
cd596e1810 feat(api): import an mbox into a list's archive
- Add POST /api/v1/lists/{id}/archive/import, streaming the body
- Return a summary of imported, duplicate and failed messages; failures do not fail the request
- Import regardless of the archive policy
2026-11-09 17:15:05 +01:00
tgiachi
45a61ef294 feat(archive): import messages from an mbox
- Add ImportMbox: batches of 200, idempotent by Message-ID, threads rebuilt at the end
- Date fallbacks: Date header, then the From line, then the import time
- Report failed and oversized messages without stopping, keep progress on a read error
- Import regardless of the list's archive policy
2026-11-09 17:14:46 +01:00
tgiachi
ba6c37bc21 feat(mbox,store): mbox reader, batch archiving and thread rebuilding
- Add internal/mbox: streaming reader with strict From_ detection, mboxrd unescaping, size cap and envelope dates
- Add Store.ArchiveBatch: many messages in one transaction
- Add Store.Rethread: rebuild threads from In-Reply-To whatever the insertion order, tolerating cycles
2026-11-09 17:13:56 +01:00
squidd
3b7e8c0d9a Merge pull request 'Continuous integration on Codeberg' (#21) from chore/ci into main 2026-11-09 17:06:45 +01:00
tgiachi
64b7614db1 docs: describe the CI workflows and make ci 2026-11-09 17:04:40 +01:00
tgiachi
65528830d3 ci: Forgejo Actions workflows and make targets
- Add make fmt-check, go-ci, admin-ci and ci so CI and a local run do the same thing
- Add a ci workflow: Go (gofmt, vet, race tests) and the admin UI (tests, type check, build)
- Add a manual e2e workflow for runners that can run Docker
2026-11-09 17:03:17 +01:00
squidd
11d6e21f41 Merge pull request 'Command-line client for administration' (#19) from feat/cli into main 2026-11-09 16:59:05 +01:00
tgiachi
2d8b0ccce6 feat(cli): command-line client for administration
- Add parsimony list, member, held, archive and queue subcommands over the REST API
- Read the token only from PARSIMONY_API_TOKEN; take the URL from --url or PARSIMONY_URL
- Print tables by default and JSON with --json
- Add member import from a file or stdin, reporting failing lines and exiting non-zero
- Require --yes for destructive commands
- Add parsimony serve; keep the old -config invocation working
- Run the CLI inside the container in the end-to-end test
- Test every command against the real API handler; mutation checks over the CLI
2026-11-09 16:57:25 +01:00
squidd
77fcdb3bbe Merge pull request 'End-to-end test with a real Postfix' (#17) from test/e2e-postfix into main 2026-11-09 16:48:22 +01:00
tgiachi
9992a50ab9 test(e2e): run the whole path through a real Postfix in Docker
- Add a Postfix test image: relay domain handed to parsimony over LMTP, members in maildirs, unknown users rejected with 5.1.1
- Check list headers, VERP return path and per-recipient unsubscribe links on real delivered mail
- Check a real 5.1.1 disabling a member after three hard bounces, and a real LMTP 550 for non-members
- Check join and leave by email, moderation, the public archive and one-click unsubscribe
- Add make e2e behind the e2e build tag
2026-11-09 16:46:55 +01:00
tgiachi
ccb2940d6e build(docker): run as a non-root user with a read-only root filesystem
- Use the distroless nonroot image and USER 65532
- Make /data the only writable path, owned by the unprivileged user
- Read the config from /etc/parsimony/parsimony.yaml, mounted read-only
- Expose the three listener ports
- Document the run command in the README
2026-11-09 16:11:55 +01:00
squidd
eed5d857b3 Merge pull request 'Milestone 5: persistent outbound queue with retries' (#14) from feat/m5-outbound-queue into main 2026-11-09 16:07:35 +01:00
tgiachi
6461e77a95 feat: document the queue, close mutation gaps, plain relay errors
- Document delivery semantics, retries and the queue API in the README and OpenAPI
- Add relay.concurrency to the example config
- Test that a relay problem never counts against a member, that the longest-waiting delivery goes first, and the error and limit details
- Show the plain relay error when there is a single batch
2026-11-09 16:04:45 +01:00
tgiachi
f3c75dc97d feat(admin): outbound queue page
- Add queue totals, delivery table with errors and next attempt
- Refresh on demand and every 5 seconds
- Add confirmed discard of a stuck delivery
- Link the page from the header
2026-11-09 15:55:52 +01:00
tgiachi
98bbcf6b68 feat(api): inspect and discard the outbound queue
- Add GET /api/v1/queue with totals, oldest age and up to 100 deliveries, failing ones first
- Add DELETE /api/v1/queue/{id} to discard a stuck delivery
- Never expose message contents
2026-11-09 15:54:13 +01:00
tgiachi
6c95184adc feat: deliver through the persistent queue
- Router.Deliver queues one entry per recipient atomically; posts are accepted once queued
- LMTP answers 451 only when the database fails, never because the relay is down
- Confirmation mail uses the same queue
- Wire the worker into the daemon: relay.concurrency config, wake on enqueue, interrupt on Close
- Make SMTPRelay honour the context deadline and cancellation on a silent relay
- Test restart survival, relay outage with retry, and Close with a hung relay
2026-11-09 15:53:47 +01:00
tgiachi
b207312cef test(queue): do not depend on the order of concurrent sends 2026-11-09 15:47:46 +01:00
tgiachi
e6713962f9 feat(queue): background worker with retries, backoff and bounded concurrency
- Add Worker.RunOnce, Run and Wake over the durable outbox
- Send prefix plus shared message per recipient with a per-send timeout
- Retry temporary failures after 1m, 5m, 15m, 1h, 3h, then every 6h; give up after 3 days
- Drop permanent failures; count a hard bounce only for bad-mailbox replies
- Leave interrupted sends queued on shutdown
- Share the bounce threshold between router and worker
2026-11-09 15:47:14 +01:00
tgiachi
ecbab4d18c feat(outbound): classify relay errors as permanent or bad mailbox
- Add IsPermanent for SMTP 5xx and PermanentError
- Add IsBadMailbox for enhanced status 5.1.x only, so a misconfigured relay says nothing about recipients
2026-11-09 15:46:27 +01:00
tgiachi
092cf4c350 feat(store): durable outbox with shared bodies and retry scheduling
- Add outbox_msgs and outbox tables
- Enqueue every recipient of a post in one transaction
- Add due, list, complete, retry and stats queries
- Delete the shared body with the last recipient
2026-11-09 15:45:19 +01:00
squidd
16124bbaf3 Merge pull request 'Milestone 4: admin web UI (Vite, React, shadcn/ui)' (#12) from feat/m4-admin-ui into main 2026-11-09 15:42:37 +01:00
tgiachi
a0a083273c feat(admin): build tooling, Docker stage and docs; close test gaps
- Add Makefile (admin, build, test) and a node stage in the Dockerfile
- Add .dockerignore so host node_modules and builds stay out of the image
- Document the admin UI in the README
- Cover mutation survivors: token cleanup on non-401 failures, cache dropped on 401 and sign out, member filter by name, archive invalidated after approval, page offset
2026-11-09 15:42:07 +01:00
tgiachi
b3a85a55ea feat(admin): archive tab with search, paging and message viewer
- Add paginated archive table with full-text search
- Add message viewer that renders the body as plain text
- Add inline-confirmed deletion
- Warn when archiving is off
2026-11-09 15:37:39 +01:00
tgiachi
c5f158b664 feat(admin): held messages tab
- Add held queue table with approve and confirmed discard
- Explain how messages get held when posting is not moderated
- Refresh the tab badge and the archive after approving
- Add byte and date formatting helpers
2026-11-09 15:36:47 +01:00
tgiachi
75a32386b2 feat(admin): members tab
- Add member table with status, bounce count and client-side filter
- Add add-member form, re-enable for disabled members and confirmed removal
2026-11-09 15:35:43 +01:00
tgiachi
1a7d16f8a9 test(admin): wait for the modal cleanup after deleting a list 2026-11-09 15:34:46 +01:00
tgiachi
f9e5b142c2 feat(admin): list page with tabs, settings and deletion
- Add list header, tab navigation driven by the URL and held-queue badge
- Add settings form that sends only changed fields
- Add confirmed list deletion with error handling in the dialog
- Add labelled select field wrapper for shadcn Select
2026-11-09 15:33:47 +01:00