Commit graph

170 commits

Author SHA1 Message Date
squidd
bc1ea2e7c3 Merge pull request 'End-to-end test for newsletters' (#41) from test/e2e-newsletter into main 2026-11-10 14:49:25 +01:00
tgiachi
2030b39270 test: end-to-end newsletter through a real Postfix
- A subscriber's post is refused with a real LMTP 550 and nothing is held or delivered
- The publisher's post reaches subscribers with list headers; the post stats add up
- The public subscribe form shows the neutral page and sends one confirmation mail
2026-11-10 14:48:59 +01:00
squidd
9a51b39eef Merge pull request 'Newsletter mode' (#39) from feat/newsletter into main 2026-11-10 13:08:16 +01:00
tgiachi
275b269977 docs: Newsletters section in the README
- Publishers policy, sent-post statistics, priority, rate limit and the public subscribe form
- State what is not included: scheduled sends and open or click tracking
2026-11-10 13:07:16 +01:00
tgiachi
c42ae9a276 test: share one SetPolicies-based helper for the newsletter priority tests
- Rename the priority helper to priorityEnv and set the publishers policy through SetPolicies instead of raw SQL
- Fixes the duplicate newsletterEnv after merging the publishers and posts branches
2026-11-10 13:02:58 +01:00
tgiachi
6a819aaeaa Merge branch 'feat/nl-web-signup' into feat/newsletter 2026-11-10 13:02:41 +01:00
tgiachi
3d25f73d9b Merge branch 'feat/nl-rate-limit' into feat/newsletter 2026-11-10 13:02:41 +01:00
tgiachi
34f461bb6b Merge branch 'feat/nl-posts-stats' into feat/newsletter 2026-11-10 13:02:41 +01:00
tgiachi
c853ae7e61 test(queue): cover a pacer slot given up while a later caller holds its own
- a caller that gives up must not release a slot when another caller reserved after it
- a delivery that did not get its slot is not counted in the run's stats
2026-11-10 12:55:26 +01:00
tgiachi
47dc118b98 test: tighten the posts tests found weak by mutation
- ListPosts: a limit of 201 is capped to 200
- posts ls: distinct pending, failed and dropped counts, so a swapped
  column is caught
2026-11-10 12:47:17 +01:00
tgiachi
5ca091ccbf test(web): trimming, Retry-After rounding and the exact edges of the limit window
- the typed address is trimmed before it is used
- Retry-After rounds up to whole seconds
- the hour ends after exactly an hour, and the sweep forgets such an address
- found by mutation testing: these edges survived the first tests
2026-11-10 12:45:49 +01:00
tgiachi
ee1f837a20 feat(admin): a Sent tab with the posts of a list and their progress
- route /lists/:id/sent and a Sent tab on the list page
- a table of posts (subject, date, sent of recipients with a progress
  bar, recipients, pending, failed, dropped) and an empty state; a post
  with no recipients shows no bar
- refreshes every 5 seconds while any post has pending deliveries, and
  stops when none has
- api.posts.list, the Post type and the fake API route; tests
2026-11-10 12:43:55 +01:00
tgiachi
7e43117c74 feat(cli): parsimony posts ls
- posts ls <list> [--limit N] prints ID, DATE, SUBJECT, RECIPIENTS, SENT,
  PENDING, FAILED and DROPPED, newest first, "No posts yet." when empty,
  and the API's JSON with --json
- cells go through the same control-character escaping as the other
  tables; tests for the table, the limit, JSON, errors and escaping
2026-11-10 12:42:15 +01:00
tgiachi
10f593e9bd feat(admin): publishers in the Settings and Members tabs
- Settings: the Posting select offers Publishers only (newsletter)
- Members: a Can publish checkbox in the add form (unchecked sends no flag, so
  adding an existing publisher again keeps it), a Publisher badge in the table,
  and a Make publisher / Make subscriber row action
- api client: publisher on Member, the publishers post policy, setPublisher
- the fake API keeps the flag on re-add and answers the member PATCH
- tests for each
2026-11-10 12:41:44 +01:00
tgiachi
2f2ee71f92 feat(api): list the posts of a list with their delivery counters
- GET /api/v1/lists/{id}/posts?limit= returns id, message_id, subject,
  created_at, recipients, sent, failed, dropped and pending, newest
  first; 404 for an unknown list, 400 for a bad limit, token required
- discarding a queued delivery counts it as dropped (store.DropOutbox)
- openapi: the new path and the discard note
2026-11-10 12:40:55 +01:00
tgiachi
e850547b3d feat(cli): publishers in member add, ls, set and import
- member add --publisher makes a publisher, --publisher=false demotes; without
  the flag an existing member keeps its role
- member ls has a ROLE column: publisher or subscriber
- new member set <list> <email> --publisher=true|false (usage error without the flag)
- member import reads an optional third column, the keyword publisher (any case);
  anything else there fails that line; an empty third column is ignored
- list set --post publishers; help text updated
- the old import test now uses the publisher keyword in its third column
2026-11-10 12:40:10 +01:00
tgiachi
0dc9c03bae feat(router): send newsletter deliveries with priority 1
- Deliver and Release (through outbound) queue the deliveries of a list
  whose post policy is publishers with priority 1, everything else 0;
  confirmation mail stays 0
- tests for Deliver, Release, confirmation mail and the recorded post
2026-11-10 12:39:34 +01:00
tgiachi
0bbfd43e30 feat(api): publisher flag on members and the publishers post policy
- the member DTO carries publisher
- POST /lists/{id}/members takes an optional publisher: absent leaves the flag
  of an existing member alone, true or false sets it
- PATCH /lists/{id}/members/{address} with {"publisher": bool}: 404 for a
  non-member, 400 when the field is missing
- the list PATCH accepts post_policy publishers; the validation message says so
- openapi updated
2026-11-10 12:38:35 +01:00
tgiachi
51e2bf683e feat(queue): count sent, failed and dropped deliveries of a post
- a delivery the relay accepted completes as sent, one given up as
  failed, one to a missing mailbox (or with a missing body) is dropped
  through store.DropOutbox; a retry changes nothing
- tests: each outcome of a post, giving up after MaxAge, a missing body,
  and discussion mail sent before a bigger newsletter queued earlier
2026-11-10 12:38:32 +01:00
tgiachi
922e2dc500 feat(web): public subscribe form for lists with a public archive and open joins
- GET /subscribe/{address}: email field, an off-screen honeypot named website,
  a Subscribe button; 404 (the usual page) for a private, closed or unknown list
- POST /subscribe/{address}: one plain ASCII address of at most 254 characters,
  else the form again (400) with the escaped typed value; a filled honeypot gets
  the success page and nothing happens; otherwise Router.RequestJoin and always
  the same neutral page, so a member, a rate limited address or an unknown
  one cannot be told apart; the body is limited to 4 KiB (413)
- at most 10 POSTs per hour per TCP peer address (X-Forwarded-For is ignored),
  then 429 with Retry-After; idle addresses are swept; the clock is injectable
- a Subscribe link on the archive page of a list that qualifies, nothing else
- no-store on the subscribe pages; the CSP is unchanged
- tests for each of these, including escaping, enumeration, the limit and its expiry
2026-11-10 12:38:02 +01:00
tgiachi
2abd435494 feat(store): record posts and count their deliveries
- posts table: one row per post queued to a list (message id, subject,
  time, recipients, sent, failed, dropped), deleted with the list and
  forgotten after 400 days when a new post is recorded
- EnqueuePost and EnqueueFromHeld record the post in the same
  transaction and stamp every delivery with its id; a refused duplicate
  records nothing; other mail keeps post_id 0
- CompleteOutbox counts sent, the new DropOutbox counts dropped and
  FailOutbox counts failed, each in the transaction that changes the
  delivery; a delivery that already failed is never counted twice
- outbox priority column; DueOutbox orders by priority, next_at, id
- ListPosts newest first (default 50, at most 200) with Pending derived
- additive migration of the outbox columns, tested on an old database
2026-11-10 12:37:52 +01:00
tgiachi
adeae3f451 feat(app): pass relay.rate_per_second to the queue worker
- one line of wiring, a commented example in parsimony.example.yaml
- tests: with a rate of 20/s eight deliveries to a sink take at least 350 ms, without a rate they are not paced
2026-11-10 12:37:15 +01:00
tgiachi
8ac4943e49 feat(config): relay.rate_per_second
- float, default 0 (unlimited); negative, NaN, infinite or non-numeric values are refused with an error naming the key
- tests with 14, 0.5, 0, 1e2 and -1, .nan, .inf, nan, fast, lists
2026-11-10 12:37:15 +01:00
tgiachi
67ba07b642 feat(queue): Worker.Rate paces every send attempt
- Rate (sends per second, 0 = unlimited) builds one pacer, used right before Sender.Send
- the wait comes before the send timeout starts, so waiting does not eat into it
- a cancelled wait leaves the delivery queued, with no attempt recorded
- tests: 5 deliveries at 2/s take 4 intervals of fake time, rate 0 never waits, retries share the pacer,
  cancellation while waiting, and one short real-time test (20 sends at 200/s)
2026-11-10 12:37:15 +01:00
tgiachi
b125c61fcf feat(queue): a pacer that spaces outgoing sends
- newPacer(perSecond, now, sleep): strictly spaced slots, no burst, safe for concurrent callers
- Wait reserves the next slot under a mutex and sleeps outside it
- a cancelled caller gives its slot back when it is the last one; a cancelled context returns at once
- a rate that is zero, negative, NaN or infinite means no pacer; a very low rate is capped at one interval of a year
- tests with an injected clock and sleep
2026-11-10 12:37:15 +01:00
tgiachi
b6f776530f test(ingest): match the refusal text of a non-publisher by suffix
- the client sees the recipient in front of the server message
2026-11-10 12:37:13 +01:00
tgiachi
d4f706523f feat(router): the publishers post policy refuses non-publishers
- Route on a publishers list lets through a member flagged publisher, after the
  unchanged discard and loop checks, and applies the usual From header check
- anyone else gets the new ErrNotPublisher: never held, archived or queued
- ingest answers ErrNotPublisher with 550 5.7.1 only the publishers of this list may post
- tests: publisher ok, plain member and stranger refused, disabled publisher posts,
  identity mismatch held and released, discard rules first, other policies unchanged
2026-11-10 12:37:00 +01:00
tgiachi
5bed75864f feat(store): publisher flag on members and the publishers post policy
- members.publisher column, added to old databases by an additive migration
- Member.Publisher, SetPublisher (ErrNotFound for a non-member) and IsPublisher
- a publisher disabled by bounces is still a publisher
- AddMember keeps the publisher flag of an existing member
- SetPolicies accepts the publishers post policy
2026-11-10 12:35:58 +01:00
tgiachi
daea68f69d feat(router): RequestJoin for callers that are not an LMTP session
- RequestJoin(listAddress, addr) reuses join: the same open-policy check, the
  silence for active members, the confirmation mail and the store rate limits
- it returns nil for an unknown or closed list, a member and a rate limited
  request, and an error only for internal failures
- tests: confirmation mail and confirming it, nothing revealed, a bounce
  disabled member can rejoin, a database failure is reported
2026-11-10 12:35:17 +01:00
tgiachi
e772777a2a chore: name the newsletter post policy 2026-11-10 12:33:03 +01:00
squidd
83eae529a4 Merge pull request 'Screenshots in the README' (#37) from docs/screenshots into main 2026-11-10 12:30:40 +01:00
tgiachi
0ea96c28d9 docs: screenshots of the admin UI and the public archive in the README
- Add five screenshots taken from a demo instance: lists, moderation, archive, queue and a public thread
- Cropped to their content and optimised (about 740 kB in total)
2026-11-10 12:29:54 +01:00
squidd
dac928144a Merge pull request 'A logo for Parsimony' (#35) from feat/logo into main 2026-11-10 12:16:44 +01:00
tgiachi
1dddf14cfd feat: the logo in the admin UI, the README and as PNG exports
- Show the wordmark on the login page and the mark in the header; set the page title to Parsimony
- Put the logo at the top of the README, with a dark-mode variant
- Add PNG exports (a 512 px mark for avatars, a 1200 px wordmark), both transparent
- Say that the logo is under the same licence
2026-11-10 12:14:50 +01:00
tgiachi
ec8ca567ff feat(web): logo and favicon on the public pages
- Show the mark on every page, linking to the index, inline so the CSP needs no image source
- Add the favicon as a data: URI and allow img-src data: only
2026-11-10 12:13:53 +01:00
tgiachi
5f5deb3e30 feat(assets): the Parsimony logo
- Add the mark (an envelope in a rounded square) and the wordmark, drawn with monoline strokes so no font is needed
- Add light and dark versions, and the generator script
- Embed them in a Go package and ship the same files with the admin UI
- Test that the SVGs are well formed, need nothing from outside, stay in sync with the admin copies and have enough contrast
2026-11-10 12:13:17 +01:00
squidd
6f745e9b25 Merge pull request 'Fix the medium and low findings of the global review' (#33) from fix/fable-mediums into main 2026-11-10 12:04:26 +01:00
tgiachi
e8c359add5 test(cli): drop an unused assignment flagged by staticcheck 2026-11-10 12:01:54 +01:00
tgiachi
b4f45035bb Merge branch 'fix/hygiene-cli-ui' into fix/fable-mediums
# Conflicts:
#	internal/router/m2_test.go
2026-11-10 12:01:40 +01:00
tgiachi
b2f447f875 Merge branch 'fix/input-validation' into fix/fable-mediums 2026-11-10 12:01:07 +01:00
tgiachi
5af20f5891 Merge branch 'fix/dup-and-shutdown' into fix/fable-mediums 2026-11-10 12:01:06 +01:00
tgiachi
266bcba354 test(app): use an HTTP client without keep-alive
- the transport could dial a connection and never use it; the HTTP
  server's graceful shutdown waits 5 s for such a connection, which made
  the Close tests flaky under -race
- the call and fetch helpers now open one connection per request
2026-11-10 12:00:04 +01:00
tgiachi
d957be3e84 fix(admin): the Held empty state names both reasons a message is held
- The hint said mail is held only on moderated lists; it is also held when the From
  header does not match the sender. It now says both, and shows for every posting policy.
- Test covers members and moderated lists and the hint staying away when messages are held.
2026-11-10 11:58:59 +01:00
tgiachi
d435f80297 docs: describe the duplicate guard and the graceful stop
- replace the sentence on at-least-once delivery with what is guarded
  now (a post queued once per list per hour, approvals always delivered)
- describe the 30 s graceful stop of LMTP and HTTP
- state what remains: at-least-once across the relay, and copies that
  arrive after an hour
2026-11-10 11:58:21 +01:00
tgiachi
dd0096093b test: pin the duplicate guard and the graceful shutdown
- router: an approved post is remembered, so the MTA's copy is refused
- store: a post with no recipients stores no message body
- app: an API request in flight during Close gets its answer
- app: finished LMTP sessions leave the connection tracker
- router: drop a TrimSpace on the Message-ID that net/mail already does
2026-11-10 11:58:07 +01:00
tgiachi
eccd82af69 feat(cli): escape control characters from mail, refuse the token over plain http
- printable() shows C0, DEL and C1 control characters (except tab and newline; CRLF counts
  as a newline) as \xNN / \uNNNN, applied to every table cell, archive show, member
  listings, import reports and error text, so an escape sequence in a subject cannot
  drive the terminal.
- The client refuses to send the bearer token over http:// to a non-loopback host unless
  --insecure is given: usage error, exit 2. localhost, 127.0.0.0/8 and ::1 stay allowed.
  Offline commands (dkim) never build the client, so they are not affected.
- Tests: the helper, ESC sequences in archive ls/show, held ls, queue ls, list/member names,
  API errors and import reports; the URL check by flag and PARSIMONY_URL. README updated.
2026-11-10 11:57:58 +01:00
tgiachi
f7ce18c3d4 test: close the gaps found by mutation checks
- api: empty names, '{' in a name, 253/254-character domains, title trimming and the name fallback
- config: an empty query or fragment in base_url
- router: leaving works on a list closed to new members
- store: an expired token is removed when presented
2026-11-10 11:57:49 +01:00
tgiachi
8151204a3d Merge branch 'fix/limits-timeouts' into fix/fable-mediums 2026-11-10 11:56:17 +01:00
tgiachi
19c7dbb9c2 Merge branch 'fix/bounce-accounting' into fix/fable-mediums 2026-11-10 11:56:17 +01:00
tgiachi
13e3d16723 fix(app): let LMTP sessions and HTTP requests finish on Close
Close cut the LMTP connections while their handlers still ran, so a post
committed to the queue could lose its 250 and be sent again by the MTA.

- Close stops accepting, waits for the LMTP Serve loop to return, then
  calls smtp.Server.Shutdown and http.Server.Shutdown with one shared
  deadline (shutdownTimeout, 30 s)
- when the deadline passes, the LMTP connections still open are cut
  through a tracking listener (smtp.Server.Close is a no-op once Shutdown
  began) and the HTTP servers are closed
- the queue worker and the store stop only after that
- tests: a post in flight during Close is queued and gets its 250; a hung
  client is cut after the grace period; the open-connections test uses a
  short grace period
2026-11-10 11:55:45 +01:00