# parsimony
A modern, single-binary mailing list manager: an evolution of Mailman.
- Postfix delivers to parsimony over LMTP; parsimony relays through SMTP.
- Discussion lists with member-only posting, `List-*` headers, one-click unsubscribe
and DMARC-safe `From` rewriting.
- DKIM signing of outgoing mail with one key per list domain, so DMARC holds after the From rewrite.
- Durable outbound queue: a post is accepted once it is stored, one delivery per recipient,
and a background worker sends it with retries (1m, 5m, 15m, 1h, 3h, then every 6h, for up to
3 days). Relay outages and restarts lose nothing.
- Self-service subscribe and unsubscribe by email with double opt-in:
send to `list+join@domain` or `list+leave@domain`, then reply to the confirmation. A reply works once, a request
expires after 72 hours, and a confirmation that could not be applied (closed list, database error) can be replied to again.
A member disabled by bounces can rejoin the same way.
- Moderated lists: posts wait in a queue that the API can approve or discard.
- VERP bounces: every recipient gets a signed return address; 3 hard bounces within 30 days disable a member.
- Archive with full-text search (SQLite FTS5) and threading. Each list has an
`archive_policy`: `private` (default, API only), `public` (web pages) or `off`.
Public pages hide email addresses.
- Per-recipient one-click unsubscribe links (RFC 8058) with signed tokens.
- SQLite storage, one YAML config file.
- Command-line client for administration and scripts (`parsimony list|member|held|archive|queue`).
- Admin REST API under `/api/v1` (see `docs/openapi.yaml`).
- Admin web UI (Vite, React, shadcn/ui) served by the API listener at `/`: lists, settings,
members, held queue and archive.
## Run
```
cp parsimony.example.yaml parsimony.yaml
PARSIMONY_API_TOKEN= go run ./cmd/parsimony serve -config parsimony.yaml
```
The API is disabled when `PARSIMONY_API_TOKEN` is not set.
Three listeners, each configurable:
| Listener | Default | Purpose |
|----------|---------|---------|
| `lmtp` | `127.0.0.1:2424` | mail from Postfix; **no authentication**, see Security notes |
| `api` | `127.0.0.1:8080` | admin REST API, bearer token |
| `web` | `127.0.0.1:8081` | public archive and unsubscribe pages (`""` disables) |
Put the `web` listener behind a reverse proxy with TLS and set `base_url` to its public URL:
the unsubscribe links in every message use it. It must be an `https://` URL (RFC 8058 one-click unsubscribe needs
it; `http://` is accepted only for `localhost`, `127.0.0.1` and `::1`) with a host and no credentials, query or fragment;
the daemon refuses to start otherwise. Leave it empty to turn the one-click links off. Keep the `api` listener private.
```
curl -H "Authorization: Bearer $PARSIMONY_API_TOKEN" -d '{"name":"dev","domain":"example.org","title":"Dev"}' localhost:8080/api/v1/lists
```
A list `name` is 1 to 30 characters of `a-z`, `0-9`, `.`, `_` and `-` (upper case is lowered, no dot at either end), the `domain`
a host name, and the `title` one line of at most 200 characters without control characters; anything else is a 400.
The title is encoded (RFC 2047) in the `List-Id` header when it is not ASCII.
## Command line
The same binary is an administration client. It talks to the REST API, so it works against a local or remote daemon.
```
export PARSIMONY_API_TOKEN=... # the token is never a flag
export PARSIMONY_URL=http://127.0.0.1:8080 # default; --url overrides
parsimony list add dev@example.org --title "Developers"
parsimony list set dev@example.org --post moderated --archive public
parsimony member add dev@example.org alice@example.com --name Alice
parsimony member import dev@example.org subscribers.txt # "email[,name]" per line, "-" reads stdin
parsimony held ls dev@example.org
parsimony held approve 12
parsimony archive ls dev@example.org --q "deploy"
parsimony archive import dev@example.org dev.mbox # an mbox file or "-" for stdin
parsimony queue ls
parsimony --json list ls # JSON for scripts
parsimony help
```
`` is an address or a numeric id. Destructive commands (`list rm`, `archive rm`) need `--yes`.
Exit status: 0 success, 1 failure (including lines that failed in `member import`), 2 usage error.
The client refuses to send the token over plain `http://` to any host other than `localhost`, `127.0.0.1` or `::1`
(usage error, exit 2): put the API behind HTTPS, or pass `--insecure` if you really mean it.
Subjects, senders, message bodies and error texts that come from mail are printed with control characters
(escape sequences) shown as `\x1b`-style text instead of being sent to the terminal; `--json` output is already escaped.
`parsimony serve -config FILE` runs the daemon; the old `parsimony -config FILE` form still works.
In the container: `docker exec parsimony /parsimony list ls`.
## Migrating from Mailman
Both Mailman 2 and Mailman 3 (HyperKitty) can export a list's history as an mbox file.
```
parsimony list add dev@example.org --title "Developers"
parsimony member import dev@example.org subscribers.txt # "email[,name]" per line
parsimony archive import dev@example.org dev.mbox
```
The import reads standard mbox (a message starts at a `From addr Tue Jan 2 15:04:05 2024` line, `>From ` escapes are
restored), decodes encoded headers, charsets and multipart bodies, and stores the plain text. It is safe to repeat: a
message already in the archive (same `Message-ID`) is skipped. Threads are rebuilt from `In-Reply-To` whatever the order
of the messages. Messages that cannot be read are listed and counted, and the exit status is 1; the others are kept.
It works whatever the list's archive policy is (the policy only governs live mail). The admin UI has the same import on
the Archive tab. Search is accent-insensitive: `caffe` finds "Caffè".
## Outgoing mail: DKIM, SPF and DMARC
Parsimony rewrites `From` to the list address, so a signature of the original author's domain is of no use and
receivers that enforce DMARC need a signature of the **list domain**.
```
parsimony dkim keygen lists.example.org --selector mail --out /etc/parsimony/dkim-lists.pem
```
prints the DNS TXT record to publish and the config to add (the private key is written with mode 0600 and the
command never overwrites an existing key without `--force`):
```yaml
dkim:
- domain: lists.example.org
selector: mail
key_file: /etc/parsimony/dkim-lists.pem
```
Once the record is in DNS, `parsimony dkim check lists.example.org --key FILE --selector mail` looks it up and compares it
with the key. Both commands work offline of the API and need no token.
- A message is signed with the key of the domain in its From address (exact match, else a parent domain), at send time, so
the per-recipient `List-Unsubscribe` headers are covered. Relaxed/relaxed, SHA-256, no body length limit.
Signed: From, To, Cc, Subject, Date, Message-ID, In-Reply-To, References, Reply-To, MIME-Version, Content-Type,
Content-Transfer-Encoding and the `List-*` headers. A relay may add `Received` or rewrite other headers.
- A list on a domain without a key is sent unsigned. A broken key file stops the daemon at startup. If signing fails at
send time the error is logged and the message goes out unsigned: not delivering is worse than a missed signature.
- RSA (2048 bits recommended; PKCS#1 or PKCS#8 PEM) and Ed25519 keys are supported. Some receivers do not verify Ed25519 yet.
- Also publish SPF for the machine that relays your mail (for example `v=spf1 ip4:203.0.113.10 -all` on the list domain)
and a DMARC record (start with `_dmarc.lists.example.org TXT "v=DMARC1; p=none; rua=mailto:postmaster@example.org"`,
then tighten the policy once the reports look clean).
## Docker
The image runs as an unprivileged user (uid 65532) and works with a read-only root filesystem.
Config is mounted read-only; the only writable path is the `/data` volume (the database).
```
docker build -t parsimony .
docker run -d --name parsimony --read-only --cap-drop ALL \
-v parsimony-data:/data \
-v $PWD/parsimony.yaml:/etc/parsimony/parsimony.yaml:ro \
-e PARSIMONY_API_TOKEN=... -p 127.0.0.1:8080:8080 -p 8081:8081 parsimony
```
Use `db: /data/parsimony.db` in the config. Do not publish the LMTP port (2424): Postfix reaches it over a Docker network
shared with the container, and `lmtp.allow` should list that network's range (see Security notes). A new named volume gets the right ownership by itself;
for a bind mount, make the host directory writable by uid 65532.
## Sending through an external relay
By default parsimony hands mail to an SMTP server on the same machine (`relay.addr: 127.0.0.1:25`). It can also send through
an external relay such as Amazon SES, Mailgun or a company smarthost:
```yaml
relay:
addr: email-smtp.eu-west-1.amazonaws.com:587
tls: starttls # none (default), starttls or tls (port 465)
username: AKIAEXAMPLE
password_file: /run/secrets/relay-password # the password is never in the config
helo: lists.example.org # EHLO name; this machine's host name by default
# ca_file: /etc/parsimony/private-ca.pem # to trust a private CA
# server_name: smtp.example.com # when the certificate is for another name than the host in addr
```
- **TLS is required, never silently skipped.** With `starttls` a relay that does not offer it makes the send fail; the
certificate is always verified (TLS 1.2 or later; system roots, or `ca_file`).
- **Credentials only travel encrypted.** SMTP AUTH PLAIN is refused over a clear connection to a remote relay, at load
time and again at send time. A local relay (127.0.0.1, `::1`, `localhost`) may be used without TLS.
- **A wrong password does not lose mail.** A refused login (or a relay that wants authentication) is treated as a temporary
error: the deliveries stay queued, retry with backoff, and show the reason in the queue (without the password). Fix the
password and restart; the mail goes out. A missing or empty password file, or an unreadable CA file, stops the daemon at
startup.
- With `starttls`, the very first EHLO before the upgrade says `localhost` (the SMTP library offers no way to set it
earlier); the configured `helo` is used right after, over TLS.
- Keep SPF and DKIM right for the path your mail takes (for SES, `include:amazonses.com` in SPF), and check that the provider
accepts the list's per-recipient return addresses (`list+bounce-ID-MAC@domain`) as envelope sender. Bounces come back
to those addresses, so they reach parsimony only if the domain's MX delivers them to its LMTP.
## Delivery and the queue
LMTP answers `250` when the post is in the queue, so a down relay never bounces mail back to Postfix.
It answers `451` only when the database itself fails.
- Temporary relay errors (4xx, timeouts, connection failures) are retried with backoff.
- A permanent rejection of the message by the relay (5xx: policy, size, line length...) is never retried, but the
delivery is not made to vanish: it is kept as **failed**, with the reason, for a week (queue API, `parsimony queue ls`,
the Queue page; discard it to clear it). The same happens when a delivery is still failing after 3 days.
- A reply to `RCPT TO` that says the address does not exist (`5.1.1`, `5.1.2`, `5.1.3`, `5.1.6`, `5.1.10`) drops the
delivery and counts as a hard bounce for the member (3 disable it). Nothing else counts: not "relay access denied", not a
refused sender (`5.1.7`, `5.1.8`, or any code answering `MAIL FROM`), not a refusal after `DATA`. Those deliveries are kept
as failed.
- A bounce that comes back later to the member's return address counts only if it is a real delivery status report
(RFC 3464: `multipart/report` with a `message/delivery-status` part) with `Action: failed` and one of the same
codes. A spam or policy rejection (`5.7.x`), a full mailbox (`5.2.x`), a delay, a `4.x.x`, or a `Status:` line in a
plain or quoted message does not count.
- Hard bounces count for 30 days: when the last one is older, the next one starts again from 1. A disabled member
stays disabled until it is added again.
- A relay that accepted the message is never asked again because `QUIT` went wrong.
- The same post is queued once. If the `250` for a post never reaches Postfix, Postfix delivers the post again; a list
remembers each post it queued for an hour (by `Message-ID`, or by a hash of the whole message when it has none),
answers `250` to a copy and does not queue it again (the log says so). An approved held post is always delivered and is
remembered too, so a copy that arrives later is not.
- On stop, the daemon refuses new connections and lets the LMTP sessions and API or web requests in flight finish, so a
post being received gets its `250`. After 30 seconds it cuts whatever is still open.
- What remains is at-least-once delivery across the relay: a crash between the relay accepting a message and the queue
entry being removed can send one message to one recipient twice. A copy of a post that arrives more than an hour later
is also delivered again.
- `GET /api/v1/queue` and the Queue page show what is pending or failing, and a stuck delivery can be discarded.
## Admin UI
The UI is a single-page app in `admin/`, built into `internal/adminui/dist` and embedded in the binary.
It is served on the API listener together with `/api`, so it needs no CORS, and only when
`PARSIMONY_API_TOKEN` is set. You sign in with that token (kept in `sessionStorage`, gone when the tab closes).
```
make admin # npm ci && npm run build
make build # go build, embeds the UI
make test # go test + vitest
make ci # what CI runs: gofmt, vet, race tests, frontend tests and build
make e2e # end-to-end with a real Postfix in Docker (about 2 minutes, mostly the image build)
```
Without `make admin` the binary still builds and the UI answers 503 with a hint.
For frontend work run the daemon, then `cd admin && npm run dev` (Vite proxies `/api` to `127.0.0.1:8080`).
## Continuous integration
`.forgejo/workflows/ci.yaml` runs on Forgejo Actions (Codeberg) for pushes to `main` and for pull requests:
a Go job (`make go-ci`: gofmt, `go vet`, tests with the race detector) and an admin UI job (`make admin-ci`:
`npm ci`, tests, type check, build). Enable Actions in the repository settings and provide a runner with the
`docker` label (change `runs-on` if yours is named differently). `make ci` does the same locally.
`.forgejo/workflows/e2e.yaml` runs the Postfix end-to-end test on demand, because it needs a runner that can run Docker.
## End-to-end test
`make e2e` starts a real Postfix and parsimony in Docker and walks the whole path: Postfix hands list mail to
parsimony over LMTP, parsimony queues it and relays the copies back through Postfix, which delivers them to
maildirs. It checks the list headers and VERP return path, a real `550 5.1.1` from Postfix disabling a member after
three hard bounces, a real LMTP `550` for a non-member, join and leave by email, moderation, the public archive
and one-click unsubscribe. The files are in `e2e/` and behind the `e2e` build tag, so `go test ./...` stays fast.
## Postfix
Deliver the list domains to parsimony over LMTP and keep the `+` extensions intact, for example:
```
transport_maps = hash:/etc/postfix/transport
# /etc/postfix/transport
lists.example.org lmtp:inet:127.0.0.1:2424
```
## Security notes
- **LMTP has no authentication.** Whoever can connect to it can send mail as any member. Keep it on `127.0.0.1`, on a
private network, or on a unix socket, and list the clients that may connect:
```yaml
lmtp:
addr: 0.0.0.0:2424
allow: [172.16.0.0/12] # CIDR ranges or single addresses; other clients are dropped before the greeting
```
The daemon logs a warning at startup when LMTP is reachable beyond the machine and `lmtp.allow` is empty.
`allow` does not apply to unix sockets: use the permissions of the socket file.
- **Who is posting.** A post is accepted when the SMTP sender is a member *and* the `From` header names the same
address. The SMTP sender can be forged by anyone who can send mail to your MX, and the archive, the rewritten `From`
and `Reply-To` come from the header; so a post whose header names someone else (or has no usable `From`, or several
addresses) is held for moderation instead of delivered, with the reason shown to the admin (API, CLI, UI). Let Postfix
reject forged senders too (SPF, DKIM, DMARC checks on inbound mail).
- **Loops and robots.** Mail that carries the list's own `List-Id` (a loop) or is a genuine auto-reply
(`Auto-Submitted: auto-replied`, `Precedence: auto_reply`) is answered `250` and dropped without a trace, so the
sender's server never bounces it. Other machine-generated mail (`Auto-Submitted: auto-generated`, cron, CI and
monitoring posts) is a normal post: the membership, `From` and moderation checks apply. The `join`, `leave` and
`confirm` addresses ignore (`250`, nothing happens, the token is kept) a mail with a null sender, any
`Auto-Submitted` other than `no`, a `Precedence` of `auto_reply`, `bulk` or `junk`, or the list's own `List-Id`.
This stops an auto-responder from confirming a subscription someone forged in your name.
- **The moderation queue is bounded.** A list holds at most 500 messages and 200 MiB, and one held message at most 10 MiB; a
full queue answers `452` (the sender retries later), a message over 10 MiB `552`. Messages nobody moderated for 30 days
are dropped. Listings (API, CLI, UI) show a summary and never read the bodies. The held view shows the From header
readers would see next to the real sender and flags a difference.
- **Hostile messages and slow clients.** The archive reads at most 10 levels of nested multipart and 100 parts per
message, then uses what it found (the post is still delivered). A `Date` header more than 24 hours ahead is replaced
by the arrival time, and one before 1990 is ignored, so a message cannot pin itself to the top of the archive. The
HTTP servers (API + admin UI, and the public web) cut a request that takes more than 30 s to read, a response that
takes more than 60 s to write, and a connection idle for 120 s; the archive import is the exception and may stream
for up to 30 minutes.
- **API token.** `parsimony serve` refuses a token shorter than 24 characters or containing whitespace
(`openssl rand -hex 24` makes a good one). After 10 wrong or missing tokens within 5 minutes an address is
locked out for 5 minutes (HTTP 429, valid token included). The address is the TCP peer: behind a reverse proxy
every client shares the proxy's address, so rate-limit at the proxy as well.
- The API and the admin UI use one bearer token and no TLS: put them behind a reverse proxy with TLS and keep the `api`
listener private.
## Scale
Measured on one machine with `make scale` (SQLite file on a local disk, relay on localhost; your numbers will differ):
| | |
|---|---|
| Queue a post for a 10 001-member list (one transaction) | 0.12 s |
| Send it through a local relay, 4 deliveries at a time | about 460 messages/s (22 s for the whole list) |
| Import a 50 000-message mbox (14 MiB) | 2.9 s, about 17 000 messages/s |
| Search a 50 000-message archive (one page of 25) | 6 to 15 ms |
| Open page 1 000 of the archive, read a thread | about 4 ms, about 9 ms |
| Subscribe members one at a time (`member add`, no batching) | about 1.9 ms each at the store, plus one HTTP request |
Sending is bounded by the SMTP round trips to your relay: raise `relay.concurrency` for big lists.
## Licence
Parsimony is free software, released under the GNU General Public License, version 3 or (at your option) any later
version. See [LICENSE](LICENSE). The logo in `assets/` (generated by `assets/make-logo.py`) is released under the same licence. The components in `admin/src/components/ui` come from shadcn/ui (MIT); the Go
dependencies (go-smtp, go-message, go-msgauth, modernc.org/sqlite, yaml.v3) use MIT, BSD or Apache 2.0 licences, which are
compatible with the GPL.