Commit graph parsimony/internal
Author SHA1 Message Date
tgiachi
38552f8025 fix(app): add read, write and idle timeouts to the HTTP servers
- the API + admin UI server and the public web server get ReadTimeout 30s,
  WriteTimeout 60s and IdleTimeout 120s next to ReadHeaderTimeout
- the archive import extends its own read and write deadlines to 30
  minutes with http.ResponseController, so a big mbox is not cut
- tests: a stalled body and an idle connection are cut on both servers,
  an import longer than the default timeouts succeeds, fields are pinned
- README: document the limits and timeouts
2026-11-10 11:54:50 +01:00
tgiachi
0a21e998e7 fix(archive): bound the multipart walk and distrust extreme dates
- walk at most 10 nested multipart levels and 100 parts, then use what
  was found: the post is still delivered, the archive stores less
- read a text part only while it can still become the body, and stop at
  the first non-blank plain part, so memory stays near MaxBody
- a Date more than 24 hours after the time of parsing becomes that time;
  a Date before 1990 is treated as missing
- tests: 20000-level nesting finishes fast, exact limits, allocation
  bound, pathological HTML, date clamps
2026-11-10 11:51:13 +01:00
tgiachi
d43c47ea08 fix(store): put the pragmas in the DSN so every connection has them
- foreign_keys, busy_timeout(5000) and WAL now come from the DSN, not from
  an Exec on one connection
- test: every connection of a 2-connection pool, in memory and on disk
2026-11-10 11:49:00 +01:00
tgiachi
beff2579c2 test: pin the SMTP command named in a relay error, SMTPUTF8 and thread order
- Name the refused command in the error (MAIL FROM, RCPT TO, DATA) so the queue tells a refused sender from a refused recipient
- Test a refusal of the DATA command itself, SMTPUTF8 for non-ASCII addresses and oldest-first threads
2026-11-10 11:42:19 +01:00
tgiachi
bcca031ea3 feat: show who really wrote a held message in the CLI and the UI
- Add SENDER and FROM columns to held ls, flagging a From header that differs from the sender
- Show the From header and a badge in the Held tab; keep the envelope sender visible
2026-11-10 11:40:31 +01:00
tgiachi
1e46be79b0 fix(ingest): answer 552 for a message too big to hold, with tests 2026-11-10 11:39:41 +01:00
tgiachi
94b5a58e93 fix(store): moderation listings never read message bodies, bound the held bytes
- Store subject, From header and size when a message is held; ListHeld returns the summary only
- Cap one held message at 10 MiB (ErrHeldTooLarge) and the bytes held per list at 200 MiB (ErrHeldFull)
- Summarise the messages already held when migrating an existing database, one at a time
- Show the From header and whether it differs from the envelope sender in the held API
- Thread listings no longer load the bodies of the thread
2026-11-10 11:39:21 +01:00
tgiachi
c1c8211c1b fix(outbound): count a missing mailbox only when RCPT TO says so
- Issue MAIL, RCPT and DATA one by one so a failure knows which command it answers
- Treat 5.1.1, 5.1.2, 5.1.3, 5.1.6 and 5.1.10 as 'no such mailbox' only in answer to RCPT TO
- A refused sender (5.1.x answering MAIL FROM) or a refusal after DATA keeps the delivery as failed and never strikes members
- Ignore a QUIT error once the relay accepted the message: it caused duplicate deliveries
2026-11-10 11:37:45 +01:00
tgiachi
cc9fbc6b24 test: pin the relay settings that reach the SMTP client
- Check the EHLO name, its host-name default, server_name and the password end to end
- Pin the wording and codes of the authentication error
2026-11-10 11:11:50 +01:00
tgiachi
a6601caed7 feat(app): send through the configured relay with TLS and authentication
- Read the relay password and CA file at startup; refuse to start on an unusable one
- Pass TLS mode, server name, CA pool, credentials and EHLO name to the SMTP relay
- Default the EHLO name to the host name
- Test an authenticating STARTTLS relay, a wrong password (mail stays queued) and the startup checks
- Document external relays in the README and the example config
2026-11-10 11:05:24 +01:00
tgiachi
b4e8644757 feat(config): relay TLS, credentials and CA settings with validation
- Add relay.tls, server_name, ca_file, helo, username and password_file
- Refuse credentials over a clear connection to a remote relay, naming the setting to change
- Read the password and CA bundle from files; an empty or missing file is an error
- Check that relay.addr is host:port
2026-11-10 11:04:41 +01:00
tgiachi
5dadf46dc8 feat(outbound): TLS, STARTTLS and SMTP AUTH towards an external relay
- Add relay TLS modes: none (default), starttls (required, never downgraded) and implicit tls
- Verify the certificate (system roots, optional CA pool and server name), TLS 1.2 minimum
- Add AUTH PLAIN, refused over a clear connection to a remote relay
- Report a wrong password or a relay that wants authentication as a temporary error, so mail waits instead of failing for good
- Add a configurable EHLO name, validated against line breaks
- Arm the context watcher before EHLO so a silent relay cannot hang a send
2026-11-10 11:03:43 +01:00
tgiachi
7005edcb9b test: pin the documented limits and close gaps found by mutation checks
- Test the lockout, the confirmation cap and Retry-After with literal numbers, not the constants
- Test that failed deliveries are listed before the most retried ones
2026-11-09 20:43:41 +01:00
tgiachi
afd067688c feat: show failed deliveries in the API, CLI and UI 2026-11-09 20:34:37 +01:00
tgiachi
bb0858ef87 fix: keep refused deliveries visible instead of dropping them silently
- Add failed deliveries: a permanent refusal by the relay (policy, size, line length) or the 3-day give-up marks the delivery failed, with the reason, and never retries it
- Keep failed deliveries listed for a week, then purge them and their stored message
- A non-existent mailbox still drops the delivery and counts a hard bounce
- Migrate the outbox table
2026-11-09 20:33:27 +01:00
tgiachi
e7e1afbde9 fix: approve a held message once, however many approvals race
- Release removes the held message and queues its deliveries in one transaction
- Of several simultaneous approvals one wins, the others get 404
- Extract the thread-root computation and test its linearity without the database (fast under -race)
2026-11-09 20:29:42 +01:00
tgiachi
fd113c4ac2 fix(security): bound the moderation queue
- Cap the held queue at 500 per list and drop messages nobody moderated for 30 days
- Answer 452 when it is full, so a legitimate sender retries and a spam run goes away
2026-11-09 20:26:33 +01:00
tgiachi
bad3c9bc10 fix(security): cap confirmation mail per address and in total 2026-11-09 20:25:53 +01:00
tgiachi
625d487eca fix(security): API token strength and brute-force lockout
- Refuse an API token under 24 characters or with whitespace at startup, without echoing it
- Lock an address out for 5 minutes after 10 wrong or missing tokens in 5 minutes (429 with Retry-After)
- Ignore X-Forwarded-For for the lockout, forget idle addresses
- Document both
2026-11-09 20:25:05 +01:00
tgiachi
ba627a50b8 fix(security): discard mail loops and automatic replies
- Drop posts that carry the list's own List-Id, Auto-Submitted other than no, or Precedence: auto_reply
- Acknowledge them to the MTA instead of refusing, so no bounce starts another loop
- Never archive, hold or queue a discarded message
2026-11-09 20:24:16 +01:00
tgiachi
eb9e915805 fix(security): hold posts whose From header does not match the sender
- Judge membership on the envelope sender and require the From header to name the same address
- Hold mismatches (and missing, unparsable or multiple From addresses) for the admin instead of delivering
- Record why a message was held: new held.reason column, migrated, shown in the API, CLI and UI
- Accept an empty message in the held queue
- Document who may post
2026-11-09 20:23:07 +01:00
tgiachi
064e8beed8 fix(security): LMTP allow list and a warning for an open LMTP port
- Add lmtp.allow: clients outside the list are dropped before the greeting (TCP only)
- Validate the list at load, naming the bad entry
- Warn at startup when LMTP is reachable beyond the machine and no list is set
- Stop publishing the LMTP port in the README and the example config; add Security notes
2026-11-09 20:20:35 +01:00
tgiachi
7ba07501c3 chore: Go 1.27.3, GPL-3.0-or-later licence, lint and vulnerability checks
- Require Go 1.27.3: fixes the standard library vulnerabilities govulncheck reported
- Add the GPLv3 text and the licence note
- Move shadcn to devDependencies: npm audit finds nothing in production dependencies
- Add make lint (staticcheck) to go-ci and a separate vuln job in CI
- Fix the staticcheck warning in a test
2026-11-09 20:18:57 +01:00
tgiachi
e7cc9cacfb test: close gaps found by mutation checks on DKIM
- Cover headers outside the signed list changing in transit, quoted local parts with @, signing errors and lowercase domains
- Ignore *.pem so a private key is never committed
2026-11-09 20:02:54 +01:00
tgiachi
1e223f1ad1 feat(cli): dkim keygen, record and check
- Create a signing key (mode 0600, never overwritten without --force) and print the DNS record and the config snippet
- Print the record of an existing key, split in DNS-sized strings
- Check that DNS publishes the key that matches the private key file
- Work offline of the API: no token needed
2026-11-09 20:01:36 +01:00
tgiachi
45303b7143 feat(dkimsign): key generation, DNS record formatting and parsing helpers 2026-11-09 20:00:47 +01:00
tgiachi
dca9117a2e feat(app): sign outgoing mail with the configured DKIM keys
- Load the keys at startup and refuse to start on a broken one
- Sign every outgoing message, confirmation mail included, with the key of its From domain
- Send unsigned when the domain has no key
2026-11-09 19:59:57 +01:00
tgiachi
aa55cd3c17 feat(queue): optional signing hook applied to the final message 2026-11-09 19:58:47 +01:00
tgiachi
be413a75e0 feat(config): dkim keys per domain, validated at load 2026-11-09 19:58:24 +01:00
tgiachi
141c5d98e9 feat(dkimsign): sign messages per From domain with RSA or Ed25519 keys
- Add Signer choosing the key by the From address domain: exact match, then parent domains
- Sign list headers too, relaxed/relaxed sha256, never a body length limit
- Load PKCS#1 and PKCS#8 PEM keys, reject unusable files without echoing key material
- Leave messages without a matching key or a usable From untouched
2026-11-09 19:58:07 +01:00
tgiachi
a170204ecb test: close gaps found by mutation checks on the archive import
- Cover the CRLF separator line, cycle roots entered from outside, linear rethreading of long chains
- Cover a read error mid-import, the "and N more" lines, and the mbox content type
- Cover clearing the previous result and the capped error list in the UI
2026-11-09 19:50:40 +01:00
tgiachi
cc47784fdb feat(cli): archive import from a file or stdin
- Add parsimony archive import <list> <file|->, streaming with no overall timeout
- Print imported, already present and failed counts; report failing messages and exit 1
- Support --json
2026-11-09 17:15:43 +01:00
tgiachi
cd596e1810 feat(api): import an mbox into a list's archive
- Add POST /api/v1/lists/{id}/archive/import, streaming the body
- Return a summary of imported, duplicate and failed messages; failures do not fail the request
- Import regardless of the archive policy
2026-11-09 17:15:05 +01:00
tgiachi
45a61ef294 feat(archive): import messages from an mbox
- Add ImportMbox: batches of 200, idempotent by Message-ID, threads rebuilt at the end
- Date fallbacks: Date header, then the From line, then the import time
- Report failed and oversized messages without stopping, keep progress on a read error
- Import regardless of the list's archive policy
2026-11-09 17:14:46 +01:00
tgiachi
ba6c37bc21 feat(mbox,store): mbox reader, batch archiving and thread rebuilding
- Add internal/mbox: streaming reader with strict From_ detection, mboxrd unescaping, size cap and envelope dates
- Add Store.ArchiveBatch: many messages in one transaction
- Add Store.Rethread: rebuild threads from In-Reply-To whatever the insertion order, tolerating cycles
2026-11-09 17:13:56 +01:00
tgiachi
2d8b0ccce6 feat(cli): command-line client for administration
- Add parsimony list, member, held, archive and queue subcommands over the REST API
- Read the token only from PARSIMONY_API_TOKEN; take the URL from --url or PARSIMONY_URL
- Print tables by default and JSON with --json
- Add member import from a file or stdin, reporting failing lines and exiting non-zero
- Require --yes for destructive commands
- Add parsimony serve; keep the old -config invocation working
- Run the CLI inside the container in the end-to-end test
- Test every command against the real API handler; mutation checks over the CLI
2026-11-09 16:57:25 +01:00
tgiachi
6461e77a95 feat: document the queue, close mutation gaps, plain relay errors
- Document delivery semantics, retries and the queue API in the README and OpenAPI
- Add relay.concurrency to the example config
- Test that a relay problem never counts against a member, that the longest-waiting delivery goes first, and the error and limit details
- Show the plain relay error when there is a single batch
2026-11-09 16:04:45 +01:00
tgiachi
98bbcf6b68 feat(api): inspect and discard the outbound queue
- Add GET /api/v1/queue with totals, oldest age and up to 100 deliveries, failing ones first
- Add DELETE /api/v1/queue/{id} to discard a stuck delivery
- Never expose message contents
2026-11-09 15:54:13 +01:00
tgiachi
6c95184adc feat: deliver through the persistent queue
- Router.Deliver queues one entry per recipient atomically; posts are accepted once queued
- LMTP answers 451 only when the database fails, never because the relay is down
- Confirmation mail uses the same queue
- Wire the worker into the daemon: relay.concurrency config, wake on enqueue, interrupt on Close
- Make SMTPRelay honour the context deadline and cancellation on a silent relay
- Test restart survival, relay outage with retry, and Close with a hung relay
2026-11-09 15:53:47 +01:00
tgiachi
b207312cef test(queue): do not depend on the order of concurrent sends 2026-11-09 15:47:46 +01:00
tgiachi
e6713962f9 feat(queue): background worker with retries, backoff and bounded concurrency
- Add Worker.RunOnce, Run and Wake over the durable outbox
- Send prefix plus shared message per recipient with a per-send timeout
- Retry temporary failures after 1m, 5m, 15m, 1h, 3h, then every 6h; give up after 3 days
- Drop permanent failures; count a hard bounce only for bad-mailbox replies
- Leave interrupted sends queued on shutdown
- Share the bounce threshold between router and worker
2026-11-09 15:47:14 +01:00
tgiachi
ecbab4d18c feat(outbound): classify relay errors as permanent or bad mailbox
- Add IsPermanent for SMTP 5xx and PermanentError
- Add IsBadMailbox for enhanced status 5.1.x only, so a misconfigured relay says nothing about recipients
2026-11-09 15:46:27 +01:00
tgiachi
092cf4c350 feat(store): durable outbox with shared bodies and retry scheduling
- Add outbox_msgs and outbox tables
- Enqueue every recipient of a post in one transaction
- Add due, list, complete, retry and stats queries
- Delete the shared body with the last recipient
2026-11-09 15:45:19 +01:00
tgiachi
5a0dd7f6d6 feat(adminui): embed the admin SPA and serve it next to the API
- Add internal/adminui with a SPA handler: index fallback for client routes, immutable caching for hashed assets, 404 for missing assets, security headers
- Answer 503 with a hint when the UI has not been built, so go build always works
- Mount the UI at / and the API at /api/ on the same listener, no CORS needed
- Scaffold admin/ with Vite, React, TypeScript, Tailwind v4, shadcn/ui and Vitest
2026-11-09 15:29:12 +01:00
tgiachi
8f64ac671d feat: serve the public web from the daemon and document the archive
- Add web.listen config (default 127.0.0.1:8081, empty disables)
- Start the public web in app, close it with the rest
- Add end-to-end test: archive stays private, becomes public, hides addresses, one-click unsubscribe works
- Cover mutation survivors: attachments, first text part, multibyte cap, member id 0, page clamp, API limit cap, closing connections
- Update OpenAPI, README and example config
2026-11-09 15:22:50 +01:00
tgiachi
19c845caab feat(web): public archive pages and one-click unsubscribe
- Add list index, paginated archive with search, and message pages with thread links
- Show only lists whose archive policy is public; hide others and foreign messages with 404
- Escape all content and obfuscate email addresses in subjects, senders and bodies
- Add GET confirm page and POST one-click unsubscribe with signed tokens
- Send CSP, nosniff, no-referrer and no-store headers
2026-11-09 15:20:45 +01:00
tgiachi
8253dd608e feat(router): archive accepted posts and add per-recipient unsubscribe links
- Archive the original message on delivery and on release, never when held or rejected
- Log archive failures instead of blocking delivery
- Add List-Unsubscribe per recipient: signed one-click URL and the +leave address
- Add signed unsubscribe tokens bound to their purpose
- Fix the mailto unsubscribe address, +unsubscribe was not a known command
2026-11-09 15:20:01 +01:00
tgiachi
4ac967e001 feat(archive): parse messages into archivable entries
- Decode subject, sender name, date, Message-ID and In-Reply-To
- Pick the plain text body from nested multipart, fall back to stripped HTML
- Decode transfer encodings and charsets, repair invalid UTF-8, cap at 1 MiB
- Synthesize a stable Message-ID when missing
- Never expose a sender address domain; add email obfuscation for public pages
2026-11-09 15:18:23 +01:00
tgiachi
d8275aaa1f feat(store,api): archive with full-text search, threading and archive policy
- Add archive table with FTS5 index kept in sync by triggers
- Deduplicate by Message-ID per list so MTA retries do not repeat posts
- Thread messages through In-Reply-To within the same list
- Quote every search word so user input is never FTS syntax
- Add archive_policy (public, private, off) with migration
- Add API to list, search, read and delete archived messages
2026-11-09 15:17:13 +01:00
tgiachi
7fc1bed7e9 fix(app): do not log closed listener errors on shutdown 2026-11-09 15:13:35 +01:00