Commit graph

147 commits

Author SHA1 Message Date
tgiachi
ee1f837a20 feat(admin): a Sent tab with the posts of a list and their progress
- route /lists/:id/sent and a Sent tab on the list page
- a table of posts (subject, date, sent of recipients with a progress
  bar, recipients, pending, failed, dropped) and an empty state; a post
  with no recipients shows no bar
- refreshes every 5 seconds while any post has pending deliveries, and
  stops when none has
- api.posts.list, the Post type and the fake API route; tests
2026-11-10 12:43:55 +01:00
tgiachi
7e43117c74 feat(cli): parsimony posts ls
- posts ls <list> [--limit N] prints ID, DATE, SUBJECT, RECIPIENTS, SENT,
  PENDING, FAILED and DROPPED, newest first, "No posts yet." when empty,
  and the API's JSON with --json
- cells go through the same control-character escaping as the other
  tables; tests for the table, the limit, JSON, errors and escaping
2026-11-10 12:42:15 +01:00
tgiachi
2f2ee71f92 feat(api): list the posts of a list with their delivery counters
- GET /api/v1/lists/{id}/posts?limit= returns id, message_id, subject,
  created_at, recipients, sent, failed, dropped and pending, newest
  first; 404 for an unknown list, 400 for a bad limit, token required
- discarding a queued delivery counts it as dropped (store.DropOutbox)
- openapi: the new path and the discard note
2026-11-10 12:40:55 +01:00
tgiachi
0dc9c03bae feat(router): send newsletter deliveries with priority 1
- Deliver and Release (through outbound) queue the deliveries of a list
  whose post policy is publishers with priority 1, everything else 0;
  confirmation mail stays 0
- tests for Deliver, Release, confirmation mail and the recorded post
2026-11-10 12:39:34 +01:00
tgiachi
51e2bf683e feat(queue): count sent, failed and dropped deliveries of a post
- a delivery the relay accepted completes as sent, one given up as
  failed, one to a missing mailbox (or with a missing body) is dropped
  through store.DropOutbox; a retry changes nothing
- tests: each outcome of a post, giving up after MaxAge, a missing body,
  and discussion mail sent before a bigger newsletter queued earlier
2026-11-10 12:38:32 +01:00
tgiachi
2abd435494 feat(store): record posts and count their deliveries
- posts table: one row per post queued to a list (message id, subject,
  time, recipients, sent, failed, dropped), deleted with the list and
  forgotten after 400 days when a new post is recorded
- EnqueuePost and EnqueueFromHeld record the post in the same
  transaction and stamp every delivery with its id; a refused duplicate
  records nothing; other mail keeps post_id 0
- CompleteOutbox counts sent, the new DropOutbox counts dropped and
  FailOutbox counts failed, each in the transaction that changes the
  delivery; a delivery that already failed is never counted twice
- outbox priority column; DueOutbox orders by priority, next_at, id
- ListPosts newest first (default 50, at most 200) with Pending derived
- additive migration of the outbox columns, tested on an old database
2026-11-10 12:37:52 +01:00
tgiachi
e772777a2a chore: name the newsletter post policy 2026-11-10 12:33:03 +01:00
squidd
83eae529a4 Merge pull request 'Screenshots in the README' (#37) from docs/screenshots into main 2026-11-10 12:30:40 +01:00
tgiachi
0ea96c28d9 docs: screenshots of the admin UI and the public archive in the README
- Add five screenshots taken from a demo instance: lists, moderation, archive, queue and a public thread
- Cropped to their content and optimised (about 740 kB in total)
2026-11-10 12:29:54 +01:00
squidd
dac928144a Merge pull request 'A logo for Parsimony' (#35) from feat/logo into main 2026-11-10 12:16:44 +01:00
tgiachi
1dddf14cfd feat: the logo in the admin UI, the README and as PNG exports
- Show the wordmark on the login page and the mark in the header; set the page title to Parsimony
- Put the logo at the top of the README, with a dark-mode variant
- Add PNG exports (a 512 px mark for avatars, a 1200 px wordmark), both transparent
- Say that the logo is under the same licence
2026-11-10 12:14:50 +01:00
tgiachi
ec8ca567ff feat(web): logo and favicon on the public pages
- Show the mark on every page, linking to the index, inline so the CSP needs no image source
- Add the favicon as a data: URI and allow img-src data: only
2026-11-10 12:13:53 +01:00
tgiachi
5f5deb3e30 feat(assets): the Parsimony logo
- Add the mark (an envelope in a rounded square) and the wordmark, drawn with monoline strokes so no font is needed
- Add light and dark versions, and the generator script
- Embed them in a Go package and ship the same files with the admin UI
- Test that the SVGs are well formed, need nothing from outside, stay in sync with the admin copies and have enough contrast
2026-11-10 12:13:17 +01:00
squidd
6f745e9b25 Merge pull request 'Fix the medium and low findings of the global review' (#33) from fix/fable-mediums into main 2026-11-10 12:04:26 +01:00
tgiachi
e8c359add5 test(cli): drop an unused assignment flagged by staticcheck 2026-11-10 12:01:54 +01:00
tgiachi
b4f45035bb Merge branch 'fix/hygiene-cli-ui' into fix/fable-mediums
# Conflicts:
#	internal/router/m2_test.go
2026-11-10 12:01:40 +01:00
tgiachi
b2f447f875 Merge branch 'fix/input-validation' into fix/fable-mediums 2026-11-10 12:01:07 +01:00
tgiachi
5af20f5891 Merge branch 'fix/dup-and-shutdown' into fix/fable-mediums 2026-11-10 12:01:06 +01:00
tgiachi
266bcba354 test(app): use an HTTP client without keep-alive
- the transport could dial a connection and never use it; the HTTP
  server's graceful shutdown waits 5 s for such a connection, which made
  the Close tests flaky under -race
- the call and fetch helpers now open one connection per request
2026-11-10 12:00:04 +01:00
tgiachi
d957be3e84 fix(admin): the Held empty state names both reasons a message is held
- The hint said mail is held only on moderated lists; it is also held when the From
  header does not match the sender. It now says both, and shows for every posting policy.
- Test covers members and moderated lists and the hint staying away when messages are held.
2026-11-10 11:58:59 +01:00
tgiachi
d435f80297 docs: describe the duplicate guard and the graceful stop
- replace the sentence on at-least-once delivery with what is guarded
  now (a post queued once per list per hour, approvals always delivered)
- describe the 30 s graceful stop of LMTP and HTTP
- state what remains: at-least-once across the relay, and copies that
  arrive after an hour
2026-11-10 11:58:21 +01:00
tgiachi
dd0096093b test: pin the duplicate guard and the graceful shutdown
- router: an approved post is remembered, so the MTA's copy is refused
- store: a post with no recipients stores no message body
- app: an API request in flight during Close gets its answer
- app: finished LMTP sessions leave the connection tracker
- router: drop a TrimSpace on the Message-ID that net/mail already does
2026-11-10 11:58:07 +01:00
tgiachi
eccd82af69 feat(cli): escape control characters from mail, refuse the token over plain http
- printable() shows C0, DEL and C1 control characters (except tab and newline; CRLF counts
  as a newline) as \xNN / \uNNNN, applied to every table cell, archive show, member
  listings, import reports and error text, so an escape sequence in a subject cannot
  drive the terminal.
- The client refuses to send the bearer token over http:// to a non-loopback host unless
  --insecure is given: usage error, exit 2. localhost, 127.0.0.0/8 and ::1 stay allowed.
  Offline commands (dkim) never build the client, so they are not affected.
- Tests: the helper, ESC sequences in archive ls/show, held ls, queue ls, list/member names,
  API errors and import reports; the URL check by flag and PARSIMONY_URL. README updated.
2026-11-10 11:57:58 +01:00
tgiachi
f7ce18c3d4 test: close the gaps found by mutation checks
- api: empty names, '{' in a name, 253/254-character domains, title trimming and the name fallback
- config: an empty query or fragment in base_url
- router: leaving works on a list closed to new members
- store: an expired token is removed when presented
2026-11-10 11:57:49 +01:00
tgiachi
8151204a3d Merge branch 'fix/limits-timeouts' into fix/fable-mediums 2026-11-10 11:56:17 +01:00
tgiachi
19c7dbb9c2 Merge branch 'fix/bounce-accounting' into fix/fable-mediums 2026-11-10 11:56:17 +01:00
tgiachi
13e3d16723 fix(app): let LMTP sessions and HTTP requests finish on Close
Close cut the LMTP connections while their handlers still ran, so a post
committed to the queue could lose its 250 and be sent again by the MTA.

- Close stops accepting, waits for the LMTP Serve loop to return, then
  calls smtp.Server.Shutdown and http.Server.Shutdown with one shared
  deadline (shutdownTimeout, 30 s)
- when the deadline passes, the LMTP connections still open are cut
  through a tracking listener (smtp.Server.Close is a no-op once Shutdown
  began) and the HTTP servers are closed
- the queue worker and the store stop only after that
- tests: a post in flight during Close is queued and gets its 250; a hung
  client is cut after the grace period; the open-connections test uses a
  short grace period
2026-11-10 11:55:45 +01:00
tgiachi
9ea8c4c71a test(archive): give the pathological HTML check headroom under -race
- it is linear (about 1.5 s per MiB under -race); a quadratic one would
  take minutes, so 15 s still catches it
2026-11-10 11:55:14 +01:00
tgiachi
53ca30ad61 docs: list name, domain and title rules, confirmation behaviour
- openapi: name, domain and title constraints of POST /lists, title of PATCH
- README: the rules, the RFC 2047 List-Id, single-use and re-usable confirmation, rejoin after bounces
2026-11-10 11:54:52 +01:00
tgiachi
38552f8025 fix(app): add read, write and idle timeouts to the HTTP servers
- the API + admin UI server and the public web server get ReadTimeout 30s,
  WriteTimeout 60s and IdleTimeout 120s next to ReadHeaderTimeout
- the archive import extends its own read and write deadlines to 30
  minutes with http.ResponseController, so a big mbox is not cut
- tests: a stalled body and an idle connection are cut on both servers,
  an import longer than the default timeouts succeeds, fields are pinned
- README: document the limits and timeouts
2026-11-10 11:54:50 +01:00
tgiachi
555578bac5 fix: shorten the confirmation token to 20 hex characters
name(<=30) + "+confirm-" + 20 = 59 characters, inside the 64 of a local part
(a 30-character name with the old 32-digit token made 71).

- store: CreatePending makes an 80-bit token
- tests: a 30-character list name keeps the confirm and the VERP bounce address (12-digit id) within 64,
  and the shorter token still confirms
2026-11-10 11:54:28 +01:00
tgiachi
9a64146fce fix(router): deliver automated posts, stop robots from confirming subscriptions
- discardReason drops only the own-List-Id loop, Auto-Submitted: auto-replied and Precedence: auto_reply; auto-generated, auto-notified and unknown values take the normal path (membership, From, moderation)
- join, leave and confirm commands ignore (250, token kept) a null sender, any Auto-Submitted other than no, Precedence auto_reply/bulk/junk and the list's own List-Id
- tests: loop_test, new auto_test, ingest LMTP test
- README: loops and robots paragraph in the security notes
2026-11-10 11:54:08 +01:00
tgiachi
cfcfd42110 fix(router,store): consume a confirmation token only when its action happened
- store: ConfirmPending claims the row (DELETE ... RETURNING), applies join or leave and commits in one
  transaction; a refusal from check, a database error, a token of another list or a rollback leaves the
  token usable; concurrent confirmations of one token apply exactly once
- store: CreatePending purges pending rows older than the 72 h TTL
- store: IsActiveMember
- router: confirm uses it, the list policy check runs before anything is consumed
- router: +join from a member disabled by bounces sends the confirmation; confirming re-enables it
- tests: purge, refusal, forced database error, other list, expiry, 8 concurrent confirms, whole rejoin flow
- cli test: the create error text changed with the new name rule
2026-11-10 11:54:05 +01:00
tgiachi
9968c3b70f fix(mbox): stop buffering a line beyond the size limit
- readLine now holds at most max(limit+1, 1024) bytes of a line and reports its full
  length; the rest is read and thrown away, so a single 25 MiB line is no longer read whole.
- The message size is counted from the reported line lengths (after restoring >From),
  so the limit is unchanged for normal input.
- Tests: a line over the limit mid-file, at the end of the input and before the first
  separator, long lines under the limit, a tiny limit, exact boundary, readLine itself.
2026-11-10 11:53:32 +01:00
tgiachi
1e8d60b72c fix(store): run the v1 members migration in one transaction
- store: migrateMembersV1 renames, creates, copies and drops inside a transaction
- test: a copy that fails on an orphan row leaves the old members table with its rows and no members_v1
2026-11-10 11:52:42 +01:00
tgiachi
79fe51b206 fix(router): drop incoming List-Help/Archive/Owner/Subscribe, DKIM-Signature, Authentication-Results and ARC-*
- Add the four list-management headers to the list that is already stripped on the way in.
- Drop the author's DKIM-Signature (broken by our header and From changes) and any
  Authentication-Results / ARC-* header from outside.
- Tests: every dropped header, repeated ARC headers, and similarly named headers kept.
2026-11-10 11:52:30 +01:00
tgiachi
cd31d919a9 test(router): pin the edges of DSN parsing
- message/global-delivery-status (RFC 6533) counts like delivery-status.
- Only the first delivery-status part is read.
- A malformed delivery-status part does not count.
- Drop the redundant empty-boundary check: the multipart reader already
  fails on it.
2026-11-10 11:52:25 +01:00
tgiachi
6a68f316ba fix(ingest): acknowledge a duplicate post with 250
- map router.ErrDuplicate to 250, like a discarded post: the post was
  already queued, so the MTA must stop retrying
- test that a post delivered twice over LMTP is queued once
2026-11-10 11:52:25 +01:00
tgiachi
bd4258ebac feat(router): queue a post to a list only once per hour
When the reply to a delivered post is lost, the MTA sends the post again
and every member got it twice. The list now remembers the posts it queued.

- store: new delivered table (list id, post id, time), created with
  CREATE TABLE IF NOT EXISTS and dropped with its list
- store: EnqueuePost records the post and queues it in one transaction,
  returns ErrDuplicate when the list queued it in the last hour, and
  purges older records
- store: EnqueueFromHeld records the post too, without refusing it: an
  approval always delivers
- router: Deliver uses EnqueuePost and logs a duplicate; the post id is
  the Message-ID, or a SHA-256 of the message when it has none
- router: Release passes the post id to EnqueueFromHeld
- tests for duplicates, other ids and lists, expiry on the store clock,
  atomicity, posts without Message-ID and approvals
2026-11-10 11:52:25 +01:00
tgiachi
f340078aed fix(config): check base_url at load
- config: empty is fine; otherwise https with a host, http only for localhost/127.0.0.1/::1, no
  userinfo, query, fragment or whitespace; the error names base_url
- test: accepted and refused URLs
- docs: README states the rule
2026-11-10 11:51:38 +01:00
tgiachi
d9012cc13a perf(queue): purge old failures at most every 5 minutes and read each body once per run
- RunOnce purged failed deliveries (a full scan) on every batch of 50: the Worker now
  does it at most once every 5 minutes, by the store clock; a failed purge is retried
  on the next run.
- Every recipient re-read the whole message from SQLite: a per-run map msgID -> body,
  filled once per message even under concurrency, is dropped when the run ends.
- Tests count purges across batches and after the interval, and body reads per message
  (including a missing body) through unexported hooks.
2026-11-10 11:51:26 +01:00
tgiachi
8317e44338 docs: describe which bounce reports count and the 30-day window
- README: only RFC 3464 reports with Action: failed and a
  recipient-unknown code count; bounces count for 30 days.
- openapi: Member.disabled and bounce_count say the count is recent.
2026-11-10 11:51:17 +01:00
tgiachi
0a21e998e7 fix(archive): bound the multipart walk and distrust extreme dates
- walk at most 10 nested multipart levels and 100 parts, then use what
  was found: the post is still delivered, the archive stores less
- read a text part only while it can still become the body, and stop at
  the first non-blank plain part, so memory stays near MaxBody
- a Date more than 24 hours after the time of parsing becomes that time;
  a Date before 1990 is treated as missing
- tests: 20000-level nesting finishes fast, exact limits, allocation
  bound, pathological HTML, date clamps
2026-11-10 11:51:13 +01:00
tgiachi
087bc92d41 fix(router): RFC 2047 encode a non-ASCII List-Id phrase
- headers: the title in List-Id goes through mime.QEncoding, the <id> part stays ASCII
- test: UTF-8 and 200-character titles read back through net/mail and the word decoder
2026-11-10 11:51:05 +01:00
tgiachi
ca2a1d63d7 fix(store): count only hard bounces of the last 30 days
- Add members.bounce_at (unix time of the last counted bounce) with an
  additive migration; existing counts get 0, so they are treated as old.
- RecordHardBounce restarts the count at 1 when the last bounce is older
  than BounceWindow (30 days); a disabled member stays disabled.
- Tests for the window boundary, the restart and the migration of an
  old database.
2026-11-10 11:50:49 +01:00
tgiachi
b8fcbf9acf fix(api): validate list name, domain and title on create
createList accepted a title with CR/LF (every later post to the list then
failed to rewrite and got 451) and names with '/', '<', '>' or tabs.

- name: lowercased, 1 to 30 characters of a-z 0-9 . _ -, no leading or trailing dot, no ".."
- domain: lowercased ASCII host name, labels of 1 to 63 characters
- title: non-empty single line, at most 200 characters, no control characters
- updateList uses the same title rule (it also refuses tabs and long titles now)
- tests: refused names, domains and titles, the accepted limits, nothing created for a bad title
2026-11-10 11:50:40 +01:00
tgiachi
fddf14e285 fix(router): count only failed recipient-unknown DSNs as hard bounces
- Parse the bounce as an RFC 3464 report: multipart/report with a
  message/delivery-status (or message/global-delivery-status) part.
- Count a hard bounce only for a per-recipient block with Action: failed
  and a status of 5.1.1, 5.1.2, 5.1.3, 5.1.6 or 5.1.10.
- Policy and content rejections (5.7.x), full mailboxes (5.2.x), delays,
  4.x.x and Status lines in plain text or quoted text no longer count.
- Tests with real-looking DSNs for each class, quoted text and multiple
  recipient blocks; the VERP token check is unchanged.
2026-11-10 11:50:01 +01:00
tgiachi
28b73fa24a test(api): every route but health answers 401 without the token
routes() is now the single table New registers, so a route added later is
covered by TestEveryRouteNeedsTheToken without editing it.

- api: declare the routes in one table (routes) and register from it
- api: table test walks it, 401 without a token, not 401 with one
2026-11-10 11:49:32 +01:00
tgiachi
d43c47ea08 fix(store): put the pragmas in the DSN so every connection has them
- foreign_keys, busy_timeout(5000) and WAL now come from the DSN, not from
  an Exec on one connection
- test: every connection of a 2-connection pool, in memory and on disk
2026-11-10 11:49:00 +01:00
squidd
b09ee2e419 Merge pull request 'Fix the two High findings of the global review' (#31) from fix/review-high into main 2026-11-10 11:44:02 +01:00