Commit graph parsimony/internal/router
Author SHA1 Message Date
tgiachi
93a3edc58f chore: move the module to git.kmalloc.org and track cmd/parsimony
All checks were successful
ci / admin (push) Successful in 37s
ci / vuln (push) Successful in 48s
ci / go (push) Successful in 3m9s
- Rename the Go module from codeberg.org/squidd/parsimony to
  git.kmalloc.org/squid/parsimony in go.mod and every import.
- Fix .gitignore: the bare `parsimony` pattern also ignored the cmd/parsimony
  folder, so the main package was never committed and a clean checkout could
  not be built. Anchor it to the root (`/parsimony`) and add cmd/parsimony.
2026-11-16 09:40:48 +01:00
tgiachi
c42ae9a276 test: share one SetPolicies-based helper for the newsletter priority tests
- Rename the priority helper to priorityEnv and set the publishers policy through SetPolicies instead of raw SQL
- Fixes the duplicate newsletterEnv after merging the publishers and posts branches
2026-11-10 13:02:58 +01:00
tgiachi
6a819aaeaa Merge branch 'feat/nl-web-signup' into feat/newsletter 2026-11-10 13:02:41 +01:00
tgiachi
34f461bb6b Merge branch 'feat/nl-posts-stats' into feat/newsletter 2026-11-10 13:02:41 +01:00
tgiachi
0dc9c03bae feat(router): send newsletter deliveries with priority 1
- Deliver and Release (through outbound) queue the deliveries of a list
  whose post policy is publishers with priority 1, everything else 0;
  confirmation mail stays 0
- tests for Deliver, Release, confirmation mail and the recorded post
2026-11-10 12:39:34 +01:00
tgiachi
d4f706523f feat(router): the publishers post policy refuses non-publishers
- Route on a publishers list lets through a member flagged publisher, after the
  unchanged discard and loop checks, and applies the usual From header check
- anyone else gets the new ErrNotPublisher: never held, archived or queued
- ingest answers ErrNotPublisher with 550 5.7.1 only the publishers of this list may post
- tests: publisher ok, plain member and stranger refused, disabled publisher posts,
  identity mismatch held and released, discard rules first, other policies unchanged
2026-11-10 12:37:00 +01:00
tgiachi
daea68f69d feat(router): RequestJoin for callers that are not an LMTP session
- RequestJoin(listAddress, addr) reuses join: the same open-policy check, the
  silence for active members, the confirmation mail and the store rate limits
- it returns nil for an unknown or closed list, a member and a rate limited
  request, and an error only for internal failures
- tests: confirmation mail and confirming it, nothing revealed, a bounce
  disabled member can rejoin, a database failure is reported
2026-11-10 12:35:17 +01:00
tgiachi
b4f45035bb Merge branch 'fix/hygiene-cli-ui' into fix/fable-mediums
# Conflicts:
#	internal/router/m2_test.go
2026-11-10 12:01:40 +01:00
tgiachi
b2f447f875 Merge branch 'fix/input-validation' into fix/fable-mediums 2026-11-10 12:01:07 +01:00
tgiachi
5af20f5891 Merge branch 'fix/dup-and-shutdown' into fix/fable-mediums 2026-11-10 12:01:06 +01:00
tgiachi
dd0096093b test: pin the duplicate guard and the graceful shutdown
- router: an approved post is remembered, so the MTA's copy is refused
- store: a post with no recipients stores no message body
- app: an API request in flight during Close gets its answer
- app: finished LMTP sessions leave the connection tracker
- router: drop a TrimSpace on the Message-ID that net/mail already does
2026-11-10 11:58:07 +01:00
tgiachi
f7ce18c3d4 test: close the gaps found by mutation checks
- api: empty names, '{' in a name, 253/254-character domains, title trimming and the name fallback
- config: an empty query or fragment in base_url
- router: leaving works on a list closed to new members
- store: an expired token is removed when presented
2026-11-10 11:57:49 +01:00
tgiachi
19c7dbb9c2 Merge branch 'fix/bounce-accounting' into fix/fable-mediums 2026-11-10 11:56:17 +01:00
tgiachi
555578bac5 fix: shorten the confirmation token to 20 hex characters
name(<=30) + "+confirm-" + 20 = 59 characters, inside the 64 of a local part
(a 30-character name with the old 32-digit token made 71).

- store: CreatePending makes an 80-bit token
- tests: a 30-character list name keeps the confirm and the VERP bounce address (12-digit id) within 64,
  and the shorter token still confirms
2026-11-10 11:54:28 +01:00
tgiachi
9a64146fce fix(router): deliver automated posts, stop robots from confirming subscriptions
- discardReason drops only the own-List-Id loop, Auto-Submitted: auto-replied and Precedence: auto_reply; auto-generated, auto-notified and unknown values take the normal path (membership, From, moderation)
- join, leave and confirm commands ignore (250, token kept) a null sender, any Auto-Submitted other than no, Precedence auto_reply/bulk/junk and the list's own List-Id
- tests: loop_test, new auto_test, ingest LMTP test
- README: loops and robots paragraph in the security notes
2026-11-10 11:54:08 +01:00
tgiachi
cfcfd42110 fix(router,store): consume a confirmation token only when its action happened
- store: ConfirmPending claims the row (DELETE ... RETURNING), applies join or leave and commits in one
  transaction; a refusal from check, a database error, a token of another list or a rollback leaves the
  token usable; concurrent confirmations of one token apply exactly once
- store: CreatePending purges pending rows older than the 72 h TTL
- store: IsActiveMember
- router: confirm uses it, the list policy check runs before anything is consumed
- router: +join from a member disabled by bounces sends the confirmation; confirming re-enables it
- tests: purge, refusal, forced database error, other list, expiry, 8 concurrent confirms, whole rejoin flow
- cli test: the create error text changed with the new name rule
2026-11-10 11:54:05 +01:00
tgiachi
79fe51b206 fix(router): drop incoming List-Help/Archive/Owner/Subscribe, DKIM-Signature, Authentication-Results and ARC-*
- Add the four list-management headers to the list that is already stripped on the way in.
- Drop the author's DKIM-Signature (broken by our header and From changes) and any
  Authentication-Results / ARC-* header from outside.
- Tests: every dropped header, repeated ARC headers, and similarly named headers kept.
2026-11-10 11:52:30 +01:00
tgiachi
cd31d919a9 test(router): pin the edges of DSN parsing
- message/global-delivery-status (RFC 6533) counts like delivery-status.
- Only the first delivery-status part is read.
- A malformed delivery-status part does not count.
- Drop the redundant empty-boundary check: the multipart reader already
  fails on it.
2026-11-10 11:52:25 +01:00
tgiachi
bd4258ebac feat(router): queue a post to a list only once per hour
When the reply to a delivered post is lost, the MTA sends the post again
and every member got it twice. The list now remembers the posts it queued.

- store: new delivered table (list id, post id, time), created with
  CREATE TABLE IF NOT EXISTS and dropped with its list
- store: EnqueuePost records the post and queues it in one transaction,
  returns ErrDuplicate when the list queued it in the last hour, and
  purges older records
- store: EnqueueFromHeld records the post too, without refusing it: an
  approval always delivers
- router: Deliver uses EnqueuePost and logs a duplicate; the post id is
  the Message-ID, or a SHA-256 of the message when it has none
- router: Release passes the post id to EnqueueFromHeld
- tests for duplicates, other ids and lists, expiry on the store clock,
  atomicity, posts without Message-ID and approvals
2026-11-10 11:52:25 +01:00
tgiachi
087bc92d41 fix(router): RFC 2047 encode a non-ASCII List-Id phrase
- headers: the title in List-Id goes through mime.QEncoding, the <id> part stays ASCII
- test: UTF-8 and 200-character titles read back through net/mail and the word decoder
2026-11-10 11:51:05 +01:00
tgiachi
fddf14e285 fix(router): count only failed recipient-unknown DSNs as hard bounces
- Parse the bounce as an RFC 3464 report: multipart/report with a
  message/delivery-status (or message/global-delivery-status) part.
- Count a hard bounce only for a per-recipient block with Action: failed
  and a status of 5.1.1, 5.1.2, 5.1.3, 5.1.6 or 5.1.10.
- Policy and content rejections (5.7.x), full mailboxes (5.2.x), delays,
  4.x.x and Status lines in plain text or quoted text no longer count.
- Tests with real-looking DSNs for each class, quoted text and multiple
  recipient blocks; the VERP token check is unchanged.
2026-11-10 11:50:01 +01:00
tgiachi
1e46be79b0 fix(ingest): answer 552 for a message too big to hold, with tests 2026-11-10 11:39:41 +01:00
tgiachi
e7e1afbde9 fix: approve a held message once, however many approvals race
- Release removes the held message and queues its deliveries in one transaction
- Of several simultaneous approvals one wins, the others get 404
- Extract the thread-root computation and test its linearity without the database (fast under -race)
2026-11-09 20:29:42 +01:00
tgiachi
fd113c4ac2 fix(security): bound the moderation queue
- Cap the held queue at 500 per list and drop messages nobody moderated for 30 days
- Answer 452 when it is full, so a legitimate sender retries and a spam run goes away
2026-11-09 20:26:33 +01:00
tgiachi
ba627a50b8 fix(security): discard mail loops and automatic replies
- Drop posts that carry the list's own List-Id, Auto-Submitted other than no, or Precedence: auto_reply
- Acknowledge them to the MTA instead of refusing, so no bounce starts another loop
- Never archive, hold or queue a discarded message
2026-11-09 20:24:16 +01:00
tgiachi
eb9e915805 fix(security): hold posts whose From header does not match the sender
- Judge membership on the envelope sender and require the From header to name the same address
- Hold mismatches (and missing, unparsable or multiple From addresses) for the admin instead of delivering
- Record why a message was held: new held.reason column, migrated, shown in the API, CLI and UI
- Accept an empty message in the held queue
- Document who may post
2026-11-09 20:23:07 +01:00
tgiachi
6c95184adc feat: deliver through the persistent queue
- Router.Deliver queues one entry per recipient atomically; posts are accepted once queued
- LMTP answers 451 only when the database fails, never because the relay is down
- Confirmation mail uses the same queue
- Wire the worker into the daemon: relay.concurrency config, wake on enqueue, interrupt on Close
- Make SMTPRelay honour the context deadline and cancellation on a silent relay
- Test restart survival, relay outage with retry, and Close with a hung relay
2026-11-09 15:53:47 +01:00
tgiachi
e6713962f9 feat(queue): background worker with retries, backoff and bounded concurrency
- Add Worker.RunOnce, Run and Wake over the durable outbox
- Send prefix plus shared message per recipient with a per-send timeout
- Retry temporary failures after 1m, 5m, 15m, 1h, 3h, then every 6h; give up after 3 days
- Drop permanent failures; count a hard bounce only for bad-mailbox replies
- Leave interrupted sends queued on shutdown
- Share the bounce threshold between router and worker
2026-11-09 15:47:14 +01:00
tgiachi
8f64ac671d feat: serve the public web from the daemon and document the archive
- Add web.listen config (default 127.0.0.1:8081, empty disables)
- Start the public web in app, close it with the rest
- Add end-to-end test: archive stays private, becomes public, hides addresses, one-click unsubscribe works
- Cover mutation survivors: attachments, first text part, multibyte cap, member id 0, page clamp, API limit cap, closing connections
- Update OpenAPI, README and example config
2026-11-09 15:22:50 +01:00
tgiachi
8253dd608e feat(router): archive accepted posts and add per-recipient unsubscribe links
- Archive the original message on delivery and on release, never when held or rejected
- Log archive failures instead of blocking delivery
- Add List-Unsubscribe per recipient: signed one-click URL and the +leave address
- Add signed unsubscribe tokens bound to their purpose
- Fix the mailto unsubscribe address, +unsubscribe was not a known command
2026-11-09 15:20:01 +01:00
tgiachi
d8275aaa1f feat(store,api): archive with full-text search, threading and archive policy
- Add archive table with FTS5 index kept in sync by triggers
- Deduplicate by Message-ID per list so MTA retries do not repeat posts
- Thread messages through In-Reply-To within the same list
- Quote every search word so user input is never FTS syntax
- Add archive_policy (public, private, off) with migration
- Add API to list, search, read and delete archived messages
2026-11-09 15:17:13 +01:00
tgiachi
d6a0f6e5a9 test: close gaps found by mutation checks
- Cover the default relay batch size
- Cover partial and total relay failure in Router.Send
- Cover stripping of spoofed List-* headers without a base URL
- Cover the end of the confirmation rate-limit window
2026-11-09 15:09:23 +01:00
tgiachi
91c01bcb83 feat(router): moderation, confirmation commands and VERP bounces
- Hold posts on moderated lists and release them on approval
- Add +join, +leave and +confirm-TOKEN command addresses
- Send one message per recipient with a signed bounce envelope sender
- Disable members after 3 hard bounces and skip them on delivery
- Reject forged bounce addresses and tokens from other lists
2026-11-09 15:04:26 +01:00
tgiachi
d8f31d9b3e feat(router): rewrite headers for list delivery
- Add List-Id, List-Post, List-Unsubscribe and Precedence headers
- Add one-click unsubscribe (RFC 8058) when a base URL is configured
- Strip incoming List-* headers to prevent spoofing
- Rewrite From as "Name via List" and keep the original in Reply-To
- Copy the message body byte for byte
2026-11-09 14:50:56 +01:00
tgiachi
eb9e992f4a feat(router): accept posts only from list members
- Add Router.Route resolving list, checking sender membership
- Return ErrUnknownList and ErrNotMember
- Return subscriber addresses as recipients
2026-11-09 14:47:44 +01:00