- Rename the Go module from codeberg.org/squidd/parsimony to
git.kmalloc.org/squid/parsimony in go.mod and every import.
- Fix .gitignore: the bare `parsimony` pattern also ignored the cmd/parsimony
folder, so the main package was never committed and a clean checkout could
not be built. Anchor it to the root (`/parsimony`) and add cmd/parsimony.
Worker.Timeout defaulted to one minute over the entire SMTP
transaction, cutting relays that are slow to accept a message and
causing duplicate deliveries. The default is now no total deadline:
SMTPRelay bounds each phase itself. A positive Timeout still applies.
A relay can take minutes to answer the final dot (content filter,
large message, slow link). One deadline over the whole transaction
cut it after the message had been accepted, so the delivery was
retried and sent again.
The connection now moves its deadline as the transaction advances:
setup (dial, greeting, TLS, AUTH, MAIL, RCPT, DATA) has
SMTPRelay.Timeout, 60 s by default; the upload has 60 s plus the size
at 32 KiB/s, at most 10 minutes; the reply to the final dot has
10 minutes (RFC 5321 4.5.3.2.6). go-smtp sets and clears its own
per-command deadlines, so a wrapper clips them to the current phase.
Context cancellation still closes the connection at once. A timeout
in any phase is temporary; after the final dot the error says the
outcome is unknown.
a@127.0.0.1 has dots and digit labels, so it passed the domain check
and named the daemon's own host without brackets. No top-level domain
is all digits: a domain whose last label is all digits is refused.
mail.ParseAddress already refuses empty labels such as x..com, so the
form-level tests could not see the empty-label check in publicDomain.
A direct table test pins the rule on its own.
When the global cap of unconfirmed requests (300 an hour) was reached,
a subscribe request silently got no mail while the visitor saw "Check
your email"; a newsletter launch can pass 300 signups an hour.
- limits.pending_per_hour (default 300, must be a whole number >= 1)
is validated at load and set on the store by the app.
- When the global cap refuses a request the store logs a warning naming
the setting, at most once a minute. The visitor still gets the
neutral page, so nothing about members is revealed. The per-address
cap is unchanged and is not logged.
- parsimony.example.yaml documents the setting.
root@localhost, a@b and a@[127.0.0.1] passed mail.ParseAddress, so the
public form could make the daemon mail itself or its own network.
The domain of a typed address must now contain a dot, not be a domain
literal, be made of non-empty labels of letters, digits and hyphens, and
not be localhost or end in .localhost, .local, .internal or .invalid.
.test and .example stay accepted (the end-to-end tests use them).
Behind a reverse proxy every visitor shared the proxy's address, so the
whole site got 10 signups an hour and one person could block everyone.
Over IPv6 each address of a /64 was a fresh key, so the limit did not
bind and the map grew by one entry per address.
- web.trusted_proxies (addresses or CIDR ranges, default empty) is
validated at load and passed to the web handler. X-Forwarded-For is
read only when the TCP peer is a trusted proxy; the client is then the
right-most entry that is not itself a trusted proxy. An empty or
malformed header falls back to the peer. With no trusted proxy the
behaviour is unchanged: the header is ignored.
- IPv6 clients are keyed by their /64, IPv4 clients by address.
- The limiter remembers at most 100000 clients. When full it sweeps
expired entries first and otherwise answers 429 to new clients until
the oldest window ends, instead of growing.
The two columns were added in one non-transactional Exec guarded by
post_id alone. A crash between the two statements left a database
with post_id but no priority, and every later start failed at
CREATE INDEX outbox_due_prio. Each column is now checked and added on
its own, as the members columns are.
- Rename the priority helper to priorityEnv and set the publishers policy through SetPolicies instead of raw SQL
- Fixes the duplicate newsletterEnv after merging the publishers and posts branches
- a caller that gives up must not release a slot when another caller reserved after it
- a delivery that did not get its slot is not counted in the run's stats
- the typed address is trimmed before it is used
- Retry-After rounds up to whole seconds
- the hour ends after exactly an hour, and the sweep forgets such an address
- found by mutation testing: these edges survived the first tests
- posts ls <list> [--limit N] prints ID, DATE, SUBJECT, RECIPIENTS, SENT,
PENDING, FAILED and DROPPED, newest first, "No posts yet." when empty,
and the API's JSON with --json
- cells go through the same control-character escaping as the other
tables; tests for the table, the limit, JSON, errors and escaping
- GET /api/v1/lists/{id}/posts?limit= returns id, message_id, subject,
created_at, recipients, sent, failed, dropped and pending, newest
first; 404 for an unknown list, 400 for a bad limit, token required
- discarding a queued delivery counts it as dropped (store.DropOutbox)
- openapi: the new path and the discard note
- member add --publisher makes a publisher, --publisher=false demotes; without
the flag an existing member keeps its role
- member ls has a ROLE column: publisher or subscriber
- new member set <list> <email> --publisher=true|false (usage error without the flag)
- member import reads an optional third column, the keyword publisher (any case);
anything else there fails that line; an empty third column is ignored
- list set --post publishers; help text updated
- the old import test now uses the publisher keyword in its third column
- Deliver and Release (through outbound) queue the deliveries of a list
whose post policy is publishers with priority 1, everything else 0;
confirmation mail stays 0
- tests for Deliver, Release, confirmation mail and the recorded post
- the member DTO carries publisher
- POST /lists/{id}/members takes an optional publisher: absent leaves the flag
of an existing member alone, true or false sets it
- PATCH /lists/{id}/members/{address} with {"publisher": bool}: 404 for a
non-member, 400 when the field is missing
- the list PATCH accepts post_policy publishers; the validation message says so
- openapi updated
- a delivery the relay accepted completes as sent, one given up as
failed, one to a missing mailbox (or with a missing body) is dropped
through store.DropOutbox; a retry changes nothing
- tests: each outcome of a post, giving up after MaxAge, a missing body,
and discussion mail sent before a bigger newsletter queued earlier
- GET /subscribe/{address}: email field, an off-screen honeypot named website,
a Subscribe button; 404 (the usual page) for a private, closed or unknown list
- POST /subscribe/{address}: one plain ASCII address of at most 254 characters,
else the form again (400) with the escaped typed value; a filled honeypot gets
the success page and nothing happens; otherwise Router.RequestJoin and always
the same neutral page, so a member, a rate limited address or an unknown
one cannot be told apart; the body is limited to 4 KiB (413)
- at most 10 POSTs per hour per TCP peer address (X-Forwarded-For is ignored),
then 429 with Retry-After; idle addresses are swept; the clock is injectable
- a Subscribe link on the archive page of a list that qualifies, nothing else
- no-store on the subscribe pages; the CSP is unchanged
- tests for each of these, including escaping, enumeration, the limit and its expiry
- posts table: one row per post queued to a list (message id, subject,
time, recipients, sent, failed, dropped), deleted with the list and
forgotten after 400 days when a new post is recorded
- EnqueuePost and EnqueueFromHeld record the post in the same
transaction and stamp every delivery with its id; a refused duplicate
records nothing; other mail keeps post_id 0
- CompleteOutbox counts sent, the new DropOutbox counts dropped and
FailOutbox counts failed, each in the transaction that changes the
delivery; a delivery that already failed is never counted twice
- outbox priority column; DueOutbox orders by priority, next_at, id
- ListPosts newest first (default 50, at most 200) with Pending derived
- additive migration of the outbox columns, tested on an old database
- one line of wiring, a commented example in parsimony.example.yaml
- tests: with a rate of 20/s eight deliveries to a sink take at least 350 ms, without a rate they are not paced
- float, default 0 (unlimited); negative, NaN, infinite or non-numeric values are refused with an error naming the key
- tests with 14, 0.5, 0, 1e2 and -1, .nan, .inf, nan, fast, lists
- Rate (sends per second, 0 = unlimited) builds one pacer, used right before Sender.Send
- the wait comes before the send timeout starts, so waiting does not eat into it
- a cancelled wait leaves the delivery queued, with no attempt recorded
- tests: 5 deliveries at 2/s take 4 intervals of fake time, rate 0 never waits, retries share the pacer,
cancellation while waiting, and one short real-time test (20 sends at 200/s)
- newPacer(perSecond, now, sleep): strictly spaced slots, no burst, safe for concurrent callers
- Wait reserves the next slot under a mutex and sleeps outside it
- a cancelled caller gives its slot back when it is the last one; a cancelled context returns at once
- a rate that is zero, negative, NaN or infinite means no pacer; a very low rate is capped at one interval of a year
- tests with an injected clock and sleep
- Route on a publishers list lets through a member flagged publisher, after the
unchanged discard and loop checks, and applies the usual From header check
- anyone else gets the new ErrNotPublisher: never held, archived or queued
- ingest answers ErrNotPublisher with 550 5.7.1 only the publishers of this list may post
- tests: publisher ok, plain member and stranger refused, disabled publisher posts,
identity mismatch held and released, discard rules first, other policies unchanged
- members.publisher column, added to old databases by an additive migration
- Member.Publisher, SetPublisher (ErrNotFound for a non-member) and IsPublisher
- a publisher disabled by bounces is still a publisher
- AddMember keeps the publisher flag of an existing member
- SetPolicies accepts the publishers post policy
- RequestJoin(listAddress, addr) reuses join: the same open-policy check, the
silence for active members, the confirmation mail and the store rate limits
- it returns nil for an unknown or closed list, a member and a rate limited
request, and an error only for internal failures
- tests: confirmation mail and confirming it, nothing revealed, a bounce
disabled member can rejoin, a database failure is reported
- Show the mark on every page, linking to the index, inline so the CSP needs no image source
- Add the favicon as a data: URI and allow img-src data: only
- the transport could dial a connection and never use it; the HTTP
server's graceful shutdown waits 5 s for such a connection, which made
the Close tests flaky under -race
- the call and fetch helpers now open one connection per request
- router: an approved post is remembered, so the MTA's copy is refused
- store: a post with no recipients stores no message body
- app: an API request in flight during Close gets its answer
- app: finished LMTP sessions leave the connection tracker
- router: drop a TrimSpace on the Message-ID that net/mail already does
- printable() shows C0, DEL and C1 control characters (except tab and newline; CRLF counts
as a newline) as \xNN / \uNNNN, applied to every table cell, archive show, member
listings, import reports and error text, so an escape sequence in a subject cannot
drive the terminal.
- The client refuses to send the bearer token over http:// to a non-loopback host unless
--insecure is given: usage error, exit 2. localhost, 127.0.0.0/8 and ::1 stay allowed.
Offline commands (dkim) never build the client, so they are not affected.
- Tests: the helper, ESC sequences in archive ls/show, held ls, queue ls, list/member names,
API errors and import reports; the URL check by flag and PARSIMONY_URL. README updated.
- api: empty names, '{' in a name, 253/254-character domains, title trimming and the name fallback
- config: an empty query or fragment in base_url
- router: leaving works on a list closed to new members
- store: an expired token is removed when presented
Close cut the LMTP connections while their handlers still ran, so a post
committed to the queue could lose its 250 and be sent again by the MTA.
- Close stops accepting, waits for the LMTP Serve loop to return, then
calls smtp.Server.Shutdown and http.Server.Shutdown with one shared
deadline (shutdownTimeout, 30 s)
- when the deadline passes, the LMTP connections still open are cut
through a tracking listener (smtp.Server.Close is a no-op once Shutdown
began) and the HTTP servers are closed
- the queue worker and the store stop only after that
- tests: a post in flight during Close is queued and gets its 250; a hung
client is cut after the grace period; the open-connections test uses a
short grace period
- the API + admin UI server and the public web server get ReadTimeout 30s,
WriteTimeout 60s and IdleTimeout 120s next to ReadHeaderTimeout
- the archive import extends its own read and write deadlines to 30
minutes with http.ResponseController, so a big mbox is not cut
- tests: a stalled body and an idle connection are cut on both servers,
an import longer than the default timeouts succeeds, fields are pinned
- README: document the limits and timeouts