Commit graph parsimony/internal
Author SHA1 Message Date
tgiachi
b2f447f875 Merge branch 'fix/input-validation' into fix/fable-mediums 2026-11-10 12:01:07 +01:00
tgiachi
5af20f5891 Merge branch 'fix/dup-and-shutdown' into fix/fable-mediums 2026-11-10 12:01:06 +01:00
tgiachi
266bcba354 test(app): use an HTTP client without keep-alive
- the transport could dial a connection and never use it; the HTTP
  server's graceful shutdown waits 5 s for such a connection, which made
  the Close tests flaky under -race
- the call and fetch helpers now open one connection per request
2026-11-10 12:00:04 +01:00
tgiachi
dd0096093b test: pin the duplicate guard and the graceful shutdown
- router: an approved post is remembered, so the MTA's copy is refused
- store: a post with no recipients stores no message body
- app: an API request in flight during Close gets its answer
- app: finished LMTP sessions leave the connection tracker
- router: drop a TrimSpace on the Message-ID that net/mail already does
2026-11-10 11:58:07 +01:00
tgiachi
f7ce18c3d4 test: close the gaps found by mutation checks
- api: empty names, '{' in a name, 253/254-character domains, title trimming and the name fallback
- config: an empty query or fragment in base_url
- router: leaving works on a list closed to new members
- store: an expired token is removed when presented
2026-11-10 11:57:49 +01:00
tgiachi
8151204a3d Merge branch 'fix/limits-timeouts' into fix/fable-mediums 2026-11-10 11:56:17 +01:00
tgiachi
19c7dbb9c2 Merge branch 'fix/bounce-accounting' into fix/fable-mediums 2026-11-10 11:56:17 +01:00
tgiachi
13e3d16723 fix(app): let LMTP sessions and HTTP requests finish on Close
Close cut the LMTP connections while their handlers still ran, so a post
committed to the queue could lose its 250 and be sent again by the MTA.

- Close stops accepting, waits for the LMTP Serve loop to return, then
  calls smtp.Server.Shutdown and http.Server.Shutdown with one shared
  deadline (shutdownTimeout, 30 s)
- when the deadline passes, the LMTP connections still open are cut
  through a tracking listener (smtp.Server.Close is a no-op once Shutdown
  began) and the HTTP servers are closed
- the queue worker and the store stop only after that
- tests: a post in flight during Close is queued and gets its 250; a hung
  client is cut after the grace period; the open-connections test uses a
  short grace period
2026-11-10 11:55:45 +01:00
tgiachi
9ea8c4c71a test(archive): give the pathological HTML check headroom under -race
- it is linear (about 1.5 s per MiB under -race); a quadratic one would
  take minutes, so 15 s still catches it
2026-11-10 11:55:14 +01:00
tgiachi
38552f8025 fix(app): add read, write and idle timeouts to the HTTP servers
- the API + admin UI server and the public web server get ReadTimeout 30s,
  WriteTimeout 60s and IdleTimeout 120s next to ReadHeaderTimeout
- the archive import extends its own read and write deadlines to 30
  minutes with http.ResponseController, so a big mbox is not cut
- tests: a stalled body and an idle connection are cut on both servers,
  an import longer than the default timeouts succeeds, fields are pinned
- README: document the limits and timeouts
2026-11-10 11:54:50 +01:00
tgiachi
555578bac5 fix: shorten the confirmation token to 20 hex characters
name(<=30) + "+confirm-" + 20 = 59 characters, inside the 64 of a local part
(a 30-character name with the old 32-digit token made 71).

- store: CreatePending makes an 80-bit token
- tests: a 30-character list name keeps the confirm and the VERP bounce address (12-digit id) within 64,
  and the shorter token still confirms
2026-11-10 11:54:28 +01:00
tgiachi
9a64146fce fix(router): deliver automated posts, stop robots from confirming subscriptions
- discardReason drops only the own-List-Id loop, Auto-Submitted: auto-replied and Precedence: auto_reply; auto-generated, auto-notified and unknown values take the normal path (membership, From, moderation)
- join, leave and confirm commands ignore (250, token kept) a null sender, any Auto-Submitted other than no, Precedence auto_reply/bulk/junk and the list's own List-Id
- tests: loop_test, new auto_test, ingest LMTP test
- README: loops and robots paragraph in the security notes
2026-11-10 11:54:08 +01:00
tgiachi
cfcfd42110 fix(router,store): consume a confirmation token only when its action happened
- store: ConfirmPending claims the row (DELETE ... RETURNING), applies join or leave and commits in one
  transaction; a refusal from check, a database error, a token of another list or a rollback leaves the
  token usable; concurrent confirmations of one token apply exactly once
- store: CreatePending purges pending rows older than the 72 h TTL
- store: IsActiveMember
- router: confirm uses it, the list policy check runs before anything is consumed
- router: +join from a member disabled by bounces sends the confirmation; confirming re-enables it
- tests: purge, refusal, forced database error, other list, expiry, 8 concurrent confirms, whole rejoin flow
- cli test: the create error text changed with the new name rule
2026-11-10 11:54:05 +01:00
tgiachi
1e8d60b72c fix(store): run the v1 members migration in one transaction
- store: migrateMembersV1 renames, creates, copies and drops inside a transaction
- test: a copy that fails on an orphan row leaves the old members table with its rows and no members_v1
2026-11-10 11:52:42 +01:00
tgiachi
cd31d919a9 test(router): pin the edges of DSN parsing
- message/global-delivery-status (RFC 6533) counts like delivery-status.
- Only the first delivery-status part is read.
- A malformed delivery-status part does not count.
- Drop the redundant empty-boundary check: the multipart reader already
  fails on it.
2026-11-10 11:52:25 +01:00
tgiachi
6a68f316ba fix(ingest): acknowledge a duplicate post with 250
- map router.ErrDuplicate to 250, like a discarded post: the post was
  already queued, so the MTA must stop retrying
- test that a post delivered twice over LMTP is queued once
2026-11-10 11:52:25 +01:00
tgiachi
bd4258ebac feat(router): queue a post to a list only once per hour
When the reply to a delivered post is lost, the MTA sends the post again
and every member got it twice. The list now remembers the posts it queued.

- store: new delivered table (list id, post id, time), created with
  CREATE TABLE IF NOT EXISTS and dropped with its list
- store: EnqueuePost records the post and queues it in one transaction,
  returns ErrDuplicate when the list queued it in the last hour, and
  purges older records
- store: EnqueueFromHeld records the post too, without refusing it: an
  approval always delivers
- router: Deliver uses EnqueuePost and logs a duplicate; the post id is
  the Message-ID, or a SHA-256 of the message when it has none
- router: Release passes the post id to EnqueueFromHeld
- tests for duplicates, other ids and lists, expiry on the store clock,
  atomicity, posts without Message-ID and approvals
2026-11-10 11:52:25 +01:00
tgiachi
f340078aed fix(config): check base_url at load
- config: empty is fine; otherwise https with a host, http only for localhost/127.0.0.1/::1, no
  userinfo, query, fragment or whitespace; the error names base_url
- test: accepted and refused URLs
- docs: README states the rule
2026-11-10 11:51:38 +01:00
tgiachi
0a21e998e7 fix(archive): bound the multipart walk and distrust extreme dates
- walk at most 10 nested multipart levels and 100 parts, then use what
  was found: the post is still delivered, the archive stores less
- read a text part only while it can still become the body, and stop at
  the first non-blank plain part, so memory stays near MaxBody
- a Date more than 24 hours after the time of parsing becomes that time;
  a Date before 1990 is treated as missing
- tests: 20000-level nesting finishes fast, exact limits, allocation
  bound, pathological HTML, date clamps
2026-11-10 11:51:13 +01:00
tgiachi
087bc92d41 fix(router): RFC 2047 encode a non-ASCII List-Id phrase
- headers: the title in List-Id goes through mime.QEncoding, the <id> part stays ASCII
- test: UTF-8 and 200-character titles read back through net/mail and the word decoder
2026-11-10 11:51:05 +01:00
tgiachi
ca2a1d63d7 fix(store): count only hard bounces of the last 30 days
- Add members.bounce_at (unix time of the last counted bounce) with an
  additive migration; existing counts get 0, so they are treated as old.
- RecordHardBounce restarts the count at 1 when the last bounce is older
  than BounceWindow (30 days); a disabled member stays disabled.
- Tests for the window boundary, the restart and the migration of an
  old database.
2026-11-10 11:50:49 +01:00
tgiachi
b8fcbf9acf fix(api): validate list name, domain and title on create
createList accepted a title with CR/LF (every later post to the list then
failed to rewrite and got 451) and names with '/', '<', '>' or tabs.

- name: lowercased, 1 to 30 characters of a-z 0-9 . _ -, no leading or trailing dot, no ".."
- domain: lowercased ASCII host name, labels of 1 to 63 characters
- title: non-empty single line, at most 200 characters, no control characters
- updateList uses the same title rule (it also refuses tabs and long titles now)
- tests: refused names, domains and titles, the accepted limits, nothing created for a bad title
2026-11-10 11:50:40 +01:00
tgiachi
fddf14e285 fix(router): count only failed recipient-unknown DSNs as hard bounces
- Parse the bounce as an RFC 3464 report: multipart/report with a
  message/delivery-status (or message/global-delivery-status) part.
- Count a hard bounce only for a per-recipient block with Action: failed
  and a status of 5.1.1, 5.1.2, 5.1.3, 5.1.6 or 5.1.10.
- Policy and content rejections (5.7.x), full mailboxes (5.2.x), delays,
  4.x.x and Status lines in plain text or quoted text no longer count.
- Tests with real-looking DSNs for each class, quoted text and multiple
  recipient blocks; the VERP token check is unchanged.
2026-11-10 11:50:01 +01:00
tgiachi
28b73fa24a test(api): every route but health answers 401 without the token
routes() is now the single table New registers, so a route added later is
covered by TestEveryRouteNeedsTheToken without editing it.

- api: declare the routes in one table (routes) and register from it
- api: table test walks it, 401 without a token, not 401 with one
2026-11-10 11:49:32 +01:00
tgiachi
d43c47ea08 fix(store): put the pragmas in the DSN so every connection has them
- foreign_keys, busy_timeout(5000) and WAL now come from the DSN, not from
  an Exec on one connection
- test: every connection of a 2-connection pool, in memory and on disk
2026-11-10 11:49:00 +01:00
tgiachi
beff2579c2 test: pin the SMTP command named in a relay error, SMTPUTF8 and thread order
- Name the refused command in the error (MAIL FROM, RCPT TO, DATA) so the queue tells a refused sender from a refused recipient
- Test a refusal of the DATA command itself, SMTPUTF8 for non-ASCII addresses and oldest-first threads
2026-11-10 11:42:19 +01:00
tgiachi
bcca031ea3 feat: show who really wrote a held message in the CLI and the UI
- Add SENDER and FROM columns to held ls, flagging a From header that differs from the sender
- Show the From header and a badge in the Held tab; keep the envelope sender visible
2026-11-10 11:40:31 +01:00
tgiachi
1e46be79b0 fix(ingest): answer 552 for a message too big to hold, with tests 2026-11-10 11:39:41 +01:00
tgiachi
94b5a58e93 fix(store): moderation listings never read message bodies, bound the held bytes
- Store subject, From header and size when a message is held; ListHeld returns the summary only
- Cap one held message at 10 MiB (ErrHeldTooLarge) and the bytes held per list at 200 MiB (ErrHeldFull)
- Summarise the messages already held when migrating an existing database, one at a time
- Show the From header and whether it differs from the envelope sender in the held API
- Thread listings no longer load the bodies of the thread
2026-11-10 11:39:21 +01:00
tgiachi
c1c8211c1b fix(outbound): count a missing mailbox only when RCPT TO says so
- Issue MAIL, RCPT and DATA one by one so a failure knows which command it answers
- Treat 5.1.1, 5.1.2, 5.1.3, 5.1.6 and 5.1.10 as 'no such mailbox' only in answer to RCPT TO
- A refused sender (5.1.x answering MAIL FROM) or a refusal after DATA keeps the delivery as failed and never strikes members
- Ignore a QUIT error once the relay accepted the message: it caused duplicate deliveries
2026-11-10 11:37:45 +01:00
tgiachi
cc9fbc6b24 test: pin the relay settings that reach the SMTP client
- Check the EHLO name, its host-name default, server_name and the password end to end
- Pin the wording and codes of the authentication error
2026-11-10 11:11:50 +01:00
tgiachi
a6601caed7 feat(app): send through the configured relay with TLS and authentication
- Read the relay password and CA file at startup; refuse to start on an unusable one
- Pass TLS mode, server name, CA pool, credentials and EHLO name to the SMTP relay
- Default the EHLO name to the host name
- Test an authenticating STARTTLS relay, a wrong password (mail stays queued) and the startup checks
- Document external relays in the README and the example config
2026-11-10 11:05:24 +01:00
tgiachi
b4e8644757 feat(config): relay TLS, credentials and CA settings with validation
- Add relay.tls, server_name, ca_file, helo, username and password_file
- Refuse credentials over a clear connection to a remote relay, naming the setting to change
- Read the password and CA bundle from files; an empty or missing file is an error
- Check that relay.addr is host:port
2026-11-10 11:04:41 +01:00
tgiachi
5dadf46dc8 feat(outbound): TLS, STARTTLS and SMTP AUTH towards an external relay
- Add relay TLS modes: none (default), starttls (required, never downgraded) and implicit tls
- Verify the certificate (system roots, optional CA pool and server name), TLS 1.2 minimum
- Add AUTH PLAIN, refused over a clear connection to a remote relay
- Report a wrong password or a relay that wants authentication as a temporary error, so mail waits instead of failing for good
- Add a configurable EHLO name, validated against line breaks
- Arm the context watcher before EHLO so a silent relay cannot hang a send
2026-11-10 11:03:43 +01:00
tgiachi
7005edcb9b test: pin the documented limits and close gaps found by mutation checks
- Test the lockout, the confirmation cap and Retry-After with literal numbers, not the constants
- Test that failed deliveries are listed before the most retried ones
2026-11-09 20:43:41 +01:00
tgiachi
afd067688c feat: show failed deliveries in the API, CLI and UI 2026-11-09 20:34:37 +01:00
tgiachi
bb0858ef87 fix: keep refused deliveries visible instead of dropping them silently
- Add failed deliveries: a permanent refusal by the relay (policy, size, line length) or the 3-day give-up marks the delivery failed, with the reason, and never retries it
- Keep failed deliveries listed for a week, then purge them and their stored message
- A non-existent mailbox still drops the delivery and counts a hard bounce
- Migrate the outbox table
2026-11-09 20:33:27 +01:00
tgiachi
e7e1afbde9 fix: approve a held message once, however many approvals race
- Release removes the held message and queues its deliveries in one transaction
- Of several simultaneous approvals one wins, the others get 404
- Extract the thread-root computation and test its linearity without the database (fast under -race)
2026-11-09 20:29:42 +01:00
tgiachi
fd113c4ac2 fix(security): bound the moderation queue
- Cap the held queue at 500 per list and drop messages nobody moderated for 30 days
- Answer 452 when it is full, so a legitimate sender retries and a spam run goes away
2026-11-09 20:26:33 +01:00
tgiachi
bad3c9bc10 fix(security): cap confirmation mail per address and in total 2026-11-09 20:25:53 +01:00
tgiachi
625d487eca fix(security): API token strength and brute-force lockout
- Refuse an API token under 24 characters or with whitespace at startup, without echoing it
- Lock an address out for 5 minutes after 10 wrong or missing tokens in 5 minutes (429 with Retry-After)
- Ignore X-Forwarded-For for the lockout, forget idle addresses
- Document both
2026-11-09 20:25:05 +01:00
tgiachi
ba627a50b8 fix(security): discard mail loops and automatic replies
- Drop posts that carry the list's own List-Id, Auto-Submitted other than no, or Precedence: auto_reply
- Acknowledge them to the MTA instead of refusing, so no bounce starts another loop
- Never archive, hold or queue a discarded message
2026-11-09 20:24:16 +01:00
tgiachi
eb9e915805 fix(security): hold posts whose From header does not match the sender
- Judge membership on the envelope sender and require the From header to name the same address
- Hold mismatches (and missing, unparsable or multiple From addresses) for the admin instead of delivering
- Record why a message was held: new held.reason column, migrated, shown in the API, CLI and UI
- Accept an empty message in the held queue
- Document who may post
2026-11-09 20:23:07 +01:00
tgiachi
064e8beed8 fix(security): LMTP allow list and a warning for an open LMTP port
- Add lmtp.allow: clients outside the list are dropped before the greeting (TCP only)
- Validate the list at load, naming the bad entry
- Warn at startup when LMTP is reachable beyond the machine and no list is set
- Stop publishing the LMTP port in the README and the example config; add Security notes
2026-11-09 20:20:35 +01:00
tgiachi
7ba07501c3 chore: Go 1.27.3, GPL-3.0-or-later licence, lint and vulnerability checks
- Require Go 1.27.3: fixes the standard library vulnerabilities govulncheck reported
- Add the GPLv3 text and the licence note
- Move shadcn to devDependencies: npm audit finds nothing in production dependencies
- Add make lint (staticcheck) to go-ci and a separate vuln job in CI
- Fix the staticcheck warning in a test
2026-11-09 20:18:57 +01:00
tgiachi
e7cc9cacfb test: close gaps found by mutation checks on DKIM
- Cover headers outside the signed list changing in transit, quoted local parts with @, signing errors and lowercase domains
- Ignore *.pem so a private key is never committed
2026-11-09 20:02:54 +01:00
tgiachi
1e223f1ad1 feat(cli): dkim keygen, record and check
- Create a signing key (mode 0600, never overwritten without --force) and print the DNS record and the config snippet
- Print the record of an existing key, split in DNS-sized strings
- Check that DNS publishes the key that matches the private key file
- Work offline of the API: no token needed
2026-11-09 20:01:36 +01:00
tgiachi
45303b7143 feat(dkimsign): key generation, DNS record formatting and parsing helpers 2026-11-09 20:00:47 +01:00
tgiachi
dca9117a2e feat(app): sign outgoing mail with the configured DKIM keys
- Load the keys at startup and refuse to start on a broken one
- Sign every outgoing message, confirmation mail included, with the key of its From domain
- Send unsigned when the domain has no key
2026-11-09 19:59:57 +01:00
tgiachi
aa55cd3c17 feat(queue): optional signing hook applied to the final message 2026-11-09 19:58:47 +01:00