A friend's server can now read what an admin chose to share.
- A library has a shared flag, off by default (migration 0024), set with
PATCH /libraries/{id}. Changing it never waits for a scan.
- Requests between friends are signed with each server's Ed25519 key: the
method, the path and query from /api/v1/peer on, a date, a nonce and the
hash of the empty body. A date more than two minutes off, a nonce used
before, an unknown or removed key and a bad signature all get the same
401. A friend still waiting for approval gets 403, and while federation is
off the routes are missing.
- Routes under /api/v1/peer: info, artists, albums, tracks, covers and
streaming with Range. The catalogue, the counts and the format come from
the shared libraries only, and the file served is the one in a shared
library. No path, library id, listen, taste, user or provider data is part
of an answer.
- A friend never makes this server build a cover, read an audio file or ask
a provider: only covers already built from the audio files are served.
- Per-friend limits, set by the admin: parallel streams (default 3) and KiB
a second (default 4096, 0 for none), shared by all the streams of a friend.
- A log of the last requests friends made (route, status, bytes; never the
query), kept for 7 days, in GET /federation/requests.
- POST /federation/friends/{id}/check lets the side that pasted an invite
find out that the other admin approved; a signed call from a friend this
server is waiting for makes it active too.
- Admin: a Shared with friends switch on each library, which asks first, and
limits, recent requests and a Check button in the Federation section, in
English and Italian.
- OpenAPI paths and schemas, contract test, two real servers checked end to
end with tagged audio files.
412 lines
63 KiB
Markdown
412 lines
63 KiB
Markdown
# Sonda
|
|
|
|
Sonda is a privacy-first, self-hosted music server written in Go. It streams the music you already own, from your own disks, and keeps everything it knows about you on your own machine.
|
|
|
|
It is a single executable, one database (SQLite by default, PostgreSQL if you prefer) and the folders that hold your music. There is no Redis, no message broker and no cloud account.
|
|
|
|
> **Status: early.** The local-playback part works end to end: accounts, libraries, scanning, search, and streaming with HTTP Range. Suggestions of new artists work too (Discover). Radio queues and OpenSubsonic compatibility are not built yet, and the web client in `web/`, which `sondad` serves, is young (see [What is not there yet](#what-is-not-there-yet) and [Web app](#web-app)). The API may still change.
|
|
|
|
## What works today
|
|
|
|
- **Accounts.** An account is an email address and a password. The first user to register becomes the admin and registration then closes; admins create the other users. The address only identifies the account: Sonda sends no mail, does not check that the address exists, and keeps it on your server. It is trimmed and lowercased, and unique. Accounts made before addresses were asked for keep the plain name they had as their login. Short-lived access tokens (JWT) plus rotating refresh tokens, so a stolen refresh token is detected.
|
|
- **Libraries.** A library is a name and a folder, managed through the API. Sonda only reads your music folders: it never writes or renames a file, and it deletes one only when an admin asks to remove the extra copy of a song and you turned on `library.allow_delete` (see **Duplicates**), which is off by default.
|
|
- **Scanner.** Reads MP3 and FLAC tags and audio properties with several workers in parallel. Rescans are incremental, renamed files keep their identity, vanished files are flagged (never deleted), and a library whose folder looks empty or unreadable (an unmounted NAS) is left alone instead of being wiped. Audio in other formats (m4a, ogg, opus, wav, wma and similar) is not read yet, but it is counted as `unsupported` in the scan result instead of disappearing silently. NAS housekeeping folders (`@Recycle`, `#recycle`, `@eaDir`, `.@__thumb`, `$RECYCLE.BIN`) are skipped.
|
|
- **Catalog.** Browse and search artists, albums and tracks, with paging, by library, artist or album. Only tracks you can actually play are listed.
|
|
- **Duplicates.** `GET /library/duplicates` (admin) lists the songs that are in more than one file: the same set of main artists, the same title and durations within 3 seconds, possibly on different albums. It is worked out from the index, touches no file, and returns no file path. Each group proposes the copy to keep (lossless first, then the higher bitrate, then the bigger file, then the oldest). A background job (`POST /library/duplicates/check`, like the other bulk jobs) reads the files of the groups and keeps their SHA-256, so a group can be called `identical` (byte for byte). `POST /library/duplicates/clean` **deletes the extra copies of one song from the disk**, and it is the one thing Sonda does that writes in your music folders, so it is off until you set `library.allow_delete` (or `SONDA_ALLOW_DELETE=true`) and mount the folders writable. It deletes only when an admin asks, never by itself, and the server checks again at that moment that no scan is running, that the tracks are copies of one song, and that every file is still the one the last scan saw (same size and date, a regular file, not a link, inside its library): so the last copy of a song is never deleted, a file that changed is not deleted, and every file is checked before the first is deleted. The listens and the playlist items of a deleted copy move to the one that stays, so your taste and your playlists are not lost. Folders are never removed, and nothing is sent anywhere.
|
|
- **Streaming.** `GET` and `HEAD` with `Range` support, so players can seek. Opening or prefetching a stream does not record a listen.
|
|
- **API documentation.** The full API is described in OpenAPI and browsable with Swagger UI, served by Sonda itself. The UI is embedded in the binary, so opening it contacts no other server.
|
|
- **Listening history.** Clients report explicit `play`, `complete` and `skip` events. An event counts as a listen when it is `complete`, or when it was played for at least 20 seconds (`listening.min_listen_seconds`); the rule is applied when the history is read, so changing it re-reads the past. You can read your own history, with a `counted` flag on every event. It stays in your database and is never sent anywhere.
|
|
- **Your taste.** `GET /me/taste` shows what Sonda has learned from your own listening: the artists you like and the kinds of music (tags) they have in common. A complete listen counts for more than a short one, a skip counts against the artist, and older listens weigh less (they halve every 90 days, `taste.half_life_days`; only the last ten half-lives are read). It is worked out on every request from data already in your database; nothing is sent to any provider.
|
|
- **Discover.** `GET /discover` suggests artists you may like, in your library or not, each with a score from 0 to 1 and the parts it is made of: how close it is to the artists you like (affinity), how close its tags are to yours (genre), what your listening says about it (preference) and a bonus for what you have not played (discovery). The weights are in `[scoring]` in the config. A part with no data is left out and the rest rescaled, and every answer says which seeds led to each suggestion. It starts from the artists you name with `?seed=`, or, only if you turned on `discover_from_listening`, from the artists you play most. It needs Last.fm to be switched on to ask for anything new: when Last.fm is off and a seed's similar artists are not stored yet, the answer says `disabled` (what was stored earlier is still used). See [External providers](#external-providers). A listener has one Discover at a time (a second one answers `429` with `Retry-After`), and at most three run at once on the server, so the providers are never lined up with work: one that waits more than 30 seconds for its turn answers `429` too, and one that stops waiting does none.
|
|
- **Artist descriptions.** `GET /artists/{id}/description` shows what Wikipedia says about an artist: a short text, with the page it comes from, a link to it and the licence (CC BY-SA 4.0), which you must show next to it. With the `wikipedia` provider and the `musicbrainz` one switched on (the link to Wikidata comes from MusicBrainz), Sonda follows ids and never a name: the artist's MusicBrainz id, then the Wikidata item that MusicBrainz links to it, then the Wikipedia page that Wikidata names in the first of your languages (`[wikipedia] languages`, the code of that Wikipedia, for example `["it", "en"]`) that has one. An artist without a MusicBrainz id has no description. It is kept for 30 days and removed with the provider's data. To read the descriptions of the whole library in one go, an admin starts a background job like a scan: `POST /library/descriptions` (follow it with `GET /library/descriptions`, stop it with `DELETE /library/descriptions`; `?refresh=true` reads every artist again). It only goes through artists that have a MusicBrainz id, never replaces a description by a failure, and ends at once, saying so, when a provider it needs is off. See [External providers](#external-providers).
|
|
- **Artist photos.** `GET /artists/{id}/image` serves a free picture of an artist from Wikimedia Commons, and `GET /artists/{id}/image/credit` says whose it is and under which licence, which you must show next to it. Only free licences are kept (public domain, CC0, CC BY, CC BY-SA): non-commercial, no-derivatives, fair-use and unknown ones are refused, because Sonda stores a copy (`?size=small` is 300 pixels, `large` 1000) in the data folder. With the `commons` provider and the `musicbrainz` one switched on, Sonda follows ids and never a name: the artist's MusicBrainz id, the Wikidata item MusicBrainz links to it, the picture Wikidata names for it, and Commons for its licence. Many artists will have none. Kept for 30 days (a week when there is none) and removed, files included, with the provider's data. To look for the pictures of the whole library in one go, an admin starts a background job like a scan: `POST /library/artist-images` (follow it with `GET /library/artist-images`, stop it with `DELETE /library/artist-images`; `?refresh=true` searches every artist again). It only goes through artists that have a MusicBrainz id, never replaces a picture by a failure, and ends at once, saying so, when a provider it needs is off. See [External providers](#external-providers).
|
|
- **Album covers.** `GET /albums/{id}/cover` serves the cover of an album from your own files: the picture embedded in them, or else a `cover`, `folder`, `front`, `album` or `albumart` image in the album's folder. It is built the first time it is asked for (a few at a time), scaled (`?size=small` is 300 pixels, `large` 1000) and kept in the data folder (`storage.data_dir`, you can delete it: covers are built again). A picture you replace in a folder is noticed within a week, or at once after a full rescan. When your files have no picture at all and an admin switched on the `coverartarchive` provider, the cover is asked from the Cover Art Archive by the MusicBrainz release group id that the tags of the album's files carry (`MUSICBRAINZ_RELEASEGROUPID`; run a full rescan to read it for music you already indexed); your own files always come first, and only that id leaves the server. An album whose files have no id is looked up in MusicBrainz by its artist and title when the `musicbrainz` provider is on too (see the list of what leaves your server). For an album split in `CD1`/`Disc 2` folders, the picture in the folder above is used. Nothing is sent anywhere unless you switched on the `coverartarchive` provider (see [External providers](#external-providers)). It also takes `?access_token=`, so an `<img>` element can use it. To build the covers of the whole library in one go instead of one album at a time, an admin starts a background job like a scan: `POST /library/covers` (follow it with `GET /library/covers`, stop it with `DELETE /library/covers`; `?refresh=true` searches every album again; it never replaces a cover by "no cover" because a disk was not mounted, and keeps the covers that came from the archive if you switched it off).
|
|
- **Mixes.** `GET /mixes` lists playlists that Sonda makes for you, like the mixes of a streaming service, from what you listen to and from the genres of your music: one mix for each genre of your library that has at least 30 songs by 4 artists (the genres you play most first), and, once you have some listening history, *Your favourites* and *Rediscover* (artists you like and have not played for a month); *New to you* is the songs you never played. `GET /mixes/{id}` gives a mix of up to 50 songs, at most 3 by one artist and never two of the same artist in a row. They are picked by weighted chance: the artists you play weigh more and songs you skipped lately or artists you avoid are left out. A mix stays the same for a day, and `?variant=2` gives another; `POST /mixes/{id}/playlist` keeps the one you are looking at as a playlist of your own. A mix is worked out when you ask for it, from your own history and the genre tags of your files: nothing is stored and nothing is sent to a provider. The id of a genre mix has the genre in it (`genre:hard rock`), so it is percent-encoded in the path. Each mix says which albums stand for it (`album_ids`: up to 4 albums that have a cover, by different artists, with most of the songs the mix draws from), for a picture made of their covers; the covers are the ones the library already has, so a mix has a picture once the covers of its albums were looked up (browsing the albums, or the covers job), and none is fetched for it.
|
|
- **Smart playlists.** A smart playlist is a set of rules instead of a list of songs, worked out again every time you open it, so new songs and your listening change it by themselves. `POST /smart-playlists` keeps a name, the rules, an order (`random`, `most_played`, `least_played`, `recently_added` or `title`) and a limit (50 by default, 500 at most); `GET /smart-playlists/{id}/tracks` gives the songs now. Every rule that is set must hold: genres, artists and albums (is, or is not), file formats and a minimum bitrate (about the same file), added in the last N days (by the date of the file), never played, played or not played in the last N days, played at least N times (optionally counting only the plays of the last N days, which is what makes the songs in rotation lately and the top of the year), skipped or not skipped lately, only artists you like or without the ones you avoid, and the length of the song. `POST /smart-playlists/preview` tries rules without keeping them, and `POST /smart-playlists/{id}/playlist` keeps what you see as a normal playlist (a copy). Only the rules are stored; the songs come from the files and from your own history, nothing is sent to a provider, and a smart playlist is yours alone, and `GET /genres` lists the genres to choose from. A random order stays the same for a day, and `?variant=2` gives another. `GET /smart-playlists/{id}/covers` and `GET /smart-playlists/suggested/{key}/covers` say which albums stand for one (up to 4 that have a cover, in the order of the songs it gives now), for its picture; they are apart from the songs so that the list of smart playlists shows at once and each card fills in its cover when it arrives. Some are there without anybody making them: `GET /smart-playlists/suggested` lists a fixed set (`recently-added`, `on-rotation`, `top-year`, `most-played`, `never-played`, `forgotten`, `high-quality`, `long-tracks`), the same rules for everyone and worked out on your own library and history, nothing stored; `GET /smart-playlists/suggested/{key}/tracks` gives their songs and `POST /smart-playlists/suggested/{key}/playlist` keeps them as a playlist, and `on-rotation` is the songs you played most in the last 30 days and `top-year` the same over a year, so they change as you listen; a copy of their rules becomes a smart playlist of your own with `POST /smart-playlists`. The year of a song is not in the index yet, so there is no rule about it. A user can have 200.
|
|
- **Playlists.** `POST /playlists` makes a playlist (a name and an optional description) and `GET /playlists` lists yours, the one changed last first. `POST /playlists/{id}/tracks` adds 1 to 500 tracks at the end or at a `position`, `GET /playlists/{id}/tracks` lists them in order, `PATCH /playlists/{id}/tracks/{item}` moves one and `DELETE` takes one out; `PATCH /playlists/{id}` renames and `DELETE /playlists/{id}` deletes (the tracks stay in the library). A track can be in a playlist more than once, which is why each place has its own item id. A playlist is yours alone: someone else's answers `playlist_not_found`, as one that does not exist. A track whose files were deleted or moved away stays in the playlist and shows `available: false`; it plays again when the files come back and a scan finds them. Each playlist says which albums stand for it (`album_ids`: up to 4 albums that have a cover, in the order they first appear in it, each once), for a picture made of their covers; nothing is fetched for it. A user can have 1000 playlists of up to 10000 tracks. Nothing is shared and nothing leaves your server.
|
|
- **Wishlist and "not interested".** `PUT /wishlist/{artist}` puts an artist on your wishlist (`GET /wishlist` lists it, `DELETE` takes it off), and `POST /discover/{artist}/dismiss` hides an artist from your Discover for good (`DELETE` brings it back, `GET /discover/dismissed` lists them). An artist is wished or dismissed, never both, and both lists are yours alone. Discover marks the wished artists it shows (`wishlisted`) and never suggests a dismissed one. The wishlist is a list for you and sends nothing anywhere; Sonda downloads nothing, but an admin can send an artist that Discover showed to their own Lidarr (see [External providers](#external-providers)), which works for an artist that has a MusicBrainz id and that Sonda knows of from a list or from your files. An artist that only Last.fm named is kept while you have it marked (your lists still show it, but other users cannot see that it exists), and goes with the rest of Last.fm's data if an admin removes Last.fm's data. Removing MusicBrainz's data does not touch it.
|
|
- **Your own settings.** `GET` and `PATCH /me/settings` hold choices that belong to you, not to the server. Everyone starts private: `discover_from_listening` is off, so this setting never sends your listening to a provider. When you turn it on, Discover may start from the artists you listen to most, and their names are sent to Last.fm (only if an admin switched it on). What Last.fm answers is kept in the shared database like any other lookup, so another local user can see, from `retrieved_at` of `GET /artists/{id}/similar`, that someone looked up an artist, but not who.
|
|
- **Tags.** `GET /artists/{id}/tags` shows what Last.fm and MusicBrainz say kind of music an artist plays, each with its own weights, and the genres of the artist's own files, read offline. See [External providers](#external-providers).
|
|
- **Similar artists.** With Last.fm switched on, `GET /artists/{id}/similar` lists artists similar to an artist, and says which of them you already have. See [External providers](#external-providers). Without it, Sonda makes no outbound network requests at all.
|
|
- **Friends (federation).** Two Sonda servers can become friends, and the listeners of one can read what the other chose to share. `PUT /federation` (admin) turns federation on, which is off by default, and gives this server a name and a public address (`https`; plain `http` only on a network of your own); the server then makes its own Ed25519 key, kept in the database and never returned. `POST /federation/invites` makes an invite that works once and expires in 24 hours (only a hash of its code is kept). The other admin uses it with `POST /federation/friends`: that server is asked to accept this one, the key it answers with must be the one named in the invite, and the friend waits as `outgoing` until the first admin approves it with `POST /federation/friends/{id}/approve` (`POST /federation/friends/{id}/check` lets the waiting side find out). Either side can remove a friend with `DELETE /federation/friends/{id}`. Nothing is shared until an admin turns on `shared` for a library (`PATCH /libraries/{id}`). A friend's server then reads, with requests signed by its key (the method, the path and query, a date, a nonce and the hash of the body), the routes under `/peer`: the artists, albums and tracks that have a file in a shared library, their covers once they are built, and the audio itself with `Range`. A friend never sees a path, a library id, a listen, a taste or a user, never makes this server build a cover or ask a provider, and has a few streams at a time and a number of KiB a second to share (`max_streams`, `max_kib_per_sec`); the admin sees the last requests it made (`GET /federation/requests`, the route and the size, never what was searched for). Sharing a library lets friends play it and, in practice, copy its files. Browsing a friend's music from the web app is not there yet. The admin area has a Federation section for all of it.
|
|
|
|
## What is not there yet
|
|
|
|
Browsing and playing a friend's libraries from the web app (federation), radio queues, OpenSubsonic compatibility, transcoding, and the ListenBrainz integration. The web client is built into the binary (see [Web app](#web-app)), and its admin area covers everything the API can do for an admin: libraries and scanning, users, the external providers and the bulk jobs.
|
|
|
|
## Requirements
|
|
|
|
- Go 1.27 or newer to build. The binary needs nothing else: SQLite and the tag reader are compiled in, with no CGO and no system libraries.
|
|
- `curl` and [`jq`](https://jqlang.github.io/jq/) for the demo below.
|
|
- Optional: `ffmpeg`, only to generate a few test tones for the demo.
|
|
|
|
## Build
|
|
|
|
```bash
|
|
make build # the web app (needs Node), then bin/sondad with the app inside
|
|
```
|
|
|
|
`make build` runs `npm ci` and `npm run build` in `web/`, which writes the app to `internal/webui/dist`, and then builds `sondad` with the tag `webui`, which embeds it: one program that serves the app on `/` and the API on `/api`. You can do the same by hand: `(cd web && npm ci && npm run build) && go build -tags webui -o bin/sondad ./cmd/sondad`.
|
|
|
|
A plain `go build -o bin/sondad ./cmd/sondad` needs no Node and gives the API only; `/` then shows a page that says how to build the app. `make test` runs the Go tests and the web tests.
|
|
|
|
### Docker
|
|
|
|
The image holds `sondad` with the web app inside, and nothing else: it builds the app itself, so you need no Node on your machine. It runs as an ordinary user, keeps the database and the files Sonda makes under `/data`, listens on port 8080 inside the container, and does not start without `SONDA_JWT_SECRET`.
|
|
|
|
```bash
|
|
docker build -t sonda .
|
|
(umask 077; echo "SONDA_JWT_SECRET=$(openssl rand -hex 32)" > sonda.env) # once: keep this file
|
|
docker run -d --name sonda --restart unless-stopped --stop-timeout 15 \
|
|
-p 127.0.0.1:8080:8080 \
|
|
--env-file sonda.env \
|
|
-v sonda-data:/data \
|
|
-v "$HOME/Music":/music:ro \
|
|
sonda
|
|
```
|
|
|
|
Mount your music read-only wherever you like (only if you want Sonda to delete duplicate files, mount it writable and set `SONDA_ALLOW_DELETE=true`: see **Duplicates**), then add the library through the API with the path *inside* the container, `/music` above (not the folder of your host): the [Demo](#demo) works the same way with `BASE=http://127.0.0.1:8080/api/v1`, except that step 4 creates the library with `"path":"/music"`. Keep `sonda.env` and pass it every time you run a new container, for example after an upgrade: a new secret signs out everybody. The container's health check asks the port in `SONDA_LISTEN`, so it works for `0.0.0.0`, `[::]` and a port alone, but not if you bind the server to one interface address. Published like this the server answers only on this machine; to serve your network, publish the port without `127.0.0.1:` and put a reverse proxy with https in front. The other settings are the environment variables of the [table below](#configure) (`SONDA_DB_DRIVER` and `SONDA_DB_DSN` for PostgreSQL); the image sets `SONDA_LISTEN`, `SONDA_DB_PATH` and `SONDA_DATA_DIR` for the volume.
|
|
|
|
To keep the data in a folder of the host instead of a volume, run the container as the owner of that folder, or it cannot write there: add `--user "$(id -u):$(id -g)"` and `-v /path/to/sonda-data:/data`. Back up `sonda.db` (and treat the backup like a secret: it holds the API keys of the providers); the rest of `/data` is built from your music.
|
|
|
|
`compose.yaml` is the same as an example: put `SONDA_JWT_SECRET` and `MUSIC_DIR` (and `SONDA_PORT` if 8080 is taken) in a `.env` file next to it and run `docker compose up -d --build`. The image has a health check on `/api/v1/health`, which is logged at debug level when it succeeds.
|
|
|
|
If you prefer a config file to environment variables (it can hold the secret and the other settings, so a Docker or Compose secret can provide it), put a `sonda.toml` in `/data` or pass `--config /run/secrets/sonda.toml`; the variables the image sets still win over the file.
|
|
|
|
## Configure
|
|
|
|
Sonda reads `./sonda.toml`, or the file given with `--config`. Every key has a default except the secret.
|
|
|
|
```bash
|
|
cp sonda.example.toml sonda.toml
|
|
# Put a random secret in it (32 characters or more). Edit the file by hand if your sed differs.
|
|
sed -i "s|^jwt_secret = .*|jwt_secret = \"$(openssl rand -hex 32)\"|" sonda.toml
|
|
```
|
|
|
|
`sonda.toml` is git-ignored because it holds the secret. Keep it readable only by the user that runs Sonda (`chmod 600 sonda.toml`).
|
|
|
|
Precedence is: built-in defaults, then the file, then environment variables. An unknown key in the file is an error, so a typo cannot be ignored silently.
|
|
|
|
| Key | Environment variable | Default | Meaning |
|
|
|---|---|---|---|
|
|
| `server.listen` | `SONDA_LISTEN` | `127.0.0.1:8080` | Address to listen on. Localhost by default. |
|
|
| `library.scan_workers` | | `4` | Files whose tags are read in parallel during a scan. |
|
|
| `library.allow_delete` | `SONDA_ALLOW_DELETE` | `false` | Lets an admin delete the extra copy of a song from the music folders (see **Duplicates**). It is the one thing that makes Sonda write there, so it needs the folders mounted writable. A value that is not `true` or `false` is an error. |
|
|
| `database.driver` | `SONDA_DB_DRIVER` | `sqlite` | `sqlite` or `postgres`. |
|
|
| `database.path` | `SONDA_DB_PATH` | `sonda.db` | SQLite file (driver `sqlite`). |
|
|
| `database.dsn` | `SONDA_DB_DSN` | | Connection string (driver `postgres`). |
|
|
| `log.env` | `SONDA_ENV` | `production` | `production` logs JSON; `dev` logs readable text. |
|
|
| `auth.jwt_secret` | `SONDA_JWT_SECRET` | *(required)* | Signs access tokens. At least 32 characters. |
|
|
| `auth.access_ttl_minutes` | | `15` | Lifetime of an access token. |
|
|
| `auth.refresh_ttl_hours` | | `720` | Lifetime of a refresh token (30 days). |
|
|
| `listening.min_listen_seconds` | | `20` | A play counts as a listen after this long (1 to 3600), or when it is `complete`. |
|
|
| `taste.half_life_days` | | `90` | After this many days a listen weighs half as much when learning your taste (1 to 3650). |
|
|
| `wikipedia.languages` | | `["en"]` | Languages for artist descriptions, best first: one to five codes of two or three lower case letters. |
|
|
| `storage.data_dir` | `SONDA_DATA_DIR` | `data` next to the SQLite database (`data` in the working directory with PostgreSQL) | Where Sonda keeps the files it makes (covers). They are built from your music, so they need no backup. |
|
|
| `scoring.weight_affinity`, `weight_genre`, `weight_preference`, `weight_discovery` | | `0.40`, `0.25`, `0.25`, `0.10` | The weights of the parts of a Discover score (0 to 10 each, not all zero). Only their proportions matter. |
|
|
|
|
Libraries and the external providers are not in the config: you set them up through the API (see the demo and [External providers](#external-providers)).
|
|
|
|
### Using PostgreSQL
|
|
|
|
```toml
|
|
[database]
|
|
driver = "postgres"
|
|
dsn = "postgres://user:password@localhost:5432/sonda?sslmode=disable"
|
|
```
|
|
|
|
Sonda creates and migrates its tables on start. The storage tests run against both databases.
|
|
|
|
## Demo
|
|
|
|
This takes you from an empty database to streaming a track. Run the server in one terminal and the commands in another.
|
|
|
|
**1. Get some music.** Point `MUSIC` at a folder of MP3 or FLAC files. The path must be absolute. If you have none, generate two 30-second test tones:
|
|
|
|
```bash
|
|
export MUSIC="$HOME/Music"
|
|
mkdir -p "$MUSIC/demo"
|
|
ffmpeg -loglevel error -y -f lavfi -i "sine=frequency=440:duration=30" -metadata title="Tone A" -metadata artist="Demo Artist" -metadata album="Demo Album" -metadata track=1 "$MUSIC/demo/a.mp3"
|
|
ffmpeg -loglevel error -y -f lavfi -i "sine=frequency=660:duration=30" -metadata title="Tone B" -metadata artist="Demo Artist" -metadata album="Demo Album" -metadata track=2 "$MUSIC/demo/b.flac"
|
|
```
|
|
|
|
**2. Start the server** (after the Build and Configure steps above):
|
|
|
|
```bash
|
|
./bin/sondad
|
|
```
|
|
|
|
**3. Check it is up, create the first user, and log in:**
|
|
|
|
```bash
|
|
BASE=http://127.0.0.1:8080/api/v1
|
|
|
|
curl -s $BASE/health
|
|
|
|
# The first user becomes the admin; registration is then closed.
|
|
curl -s -X POST $BASE/auth/register -d '{"email":"alice@example.org","password":"correct horse battery"}'
|
|
|
|
LOGIN=$(curl -s -X POST $BASE/auth/login -d '{"email":"alice@example.org","password":"correct horse battery"}')
|
|
TOKEN=$(echo "$LOGIN" | jq -r .access_token)
|
|
REFRESH=$(echo "$LOGIN" | jq -r .refresh_token)
|
|
AUTH="Authorization: Bearer $TOKEN"
|
|
```
|
|
|
|
**4. Add a library and scan it:**
|
|
|
|
```bash
|
|
curl -s -X POST $BASE/libraries -H "$AUTH" -d "{\"name\":\"Music\",\"path\":\"$MUSIC\"}" | jq
|
|
|
|
curl -s -X POST $BASE/library/scan -H "$AUTH"
|
|
until [ "$(curl -s $BASE/library/scan -H "$AUTH" | jq .running)" = false ]; do sleep 1; done
|
|
curl -s $BASE/library/scan -H "$AUTH" | jq .last
|
|
```
|
|
|
|
`found`, `added`, `unchanged`, `renamed`, `missing`, `failed` and `unsupported` tell you what the scan did. Run it again and everything is `unchanged`.
|
|
|
|
**5. Browse and search:**
|
|
|
|
```bash
|
|
curl -s "$BASE/artists?limit=5" -H "$AUTH" | jq
|
|
curl -s "$BASE/albums" -H "$AUTH" | jq '.items[] | {id, title, artist: .artist.name, tracks: .track_count}'
|
|
curl -s "$BASE/tracks?q=tone" -H "$AUTH" | jq '.items[] | {id, title, artists: [.artists[].name], format}'
|
|
```
|
|
|
|
**6. Stream a track:**
|
|
|
|
```bash
|
|
TRACK=$(curl -s "$BASE/tracks?limit=1" -H "$AUTH" | jq -r '.items[0].id')
|
|
|
|
# Ask for the first 100 bytes: the answer is 206 Partial Content with a Content-Range header.
|
|
curl -s -D - -o /dev/null -H "$AUTH" -H 'Range: bytes=0-99' $BASE/tracks/$TRACK/stream
|
|
|
|
# The whole file.
|
|
curl -s -o track.audio -H "$AUTH" $BASE/tracks/$TRACK/stream
|
|
```
|
|
|
|
A browser `<audio>` element cannot send headers, so the stream URL also accepts the access token in the query string: `$BASE/tracks/$TRACK/stream?access_token=$TOKEN`. That is the only route that does, and the query string is never logged.
|
|
|
|
**7. Report a listen.** Streaming never counts as a listen. The player reports one event per play, when the play ends or is abandoned, with the time actually played. Use one idempotency key per play: a retry with the same key is harmless, but a key that was already used is ignored even if the event differs, so an event cannot be upgraded later. An event counts as a listen when it is `complete` or lasted at least the configured minimum (20 seconds by default); a shorter play is stored too, but does not count:
|
|
|
|
```bash
|
|
curl -s -X POST $BASE/listens -H "$AUTH" -d "{\"track_id\":$TRACK,\"event_type\":\"play\",\"duration_played_ms\":25000,\"idempotency_key\":\"demo-1\"}"
|
|
curl -s -X POST $BASE/listens -H "$AUTH" -d "{\"track_id\":$TRACK,\"event_type\":\"play\",\"duration_played_ms\":3000,\"idempotency_key\":\"demo-2\"}"
|
|
|
|
# Your history, newest first. Add ?counted=true to see only what counts.
|
|
curl -s "$BASE/listens" -H "$AUTH" | jq '.items[] | {track: .track.title, event_type, duration_played_ms, counted}'
|
|
```
|
|
|
|
The answer to each report says whether it counts. A reported play time far beyond the length of the track (more than twice its length plus 10 seconds) is rejected.
|
|
|
|
**8. Refresh the session and log out:**
|
|
|
|
```bash
|
|
NEW=$(curl -s -X POST $BASE/auth/refresh -d "{\"refresh_token\":\"$REFRESH\"}")
|
|
TOKEN=$(echo "$NEW" | jq -r .access_token)
|
|
REFRESH=$(echo "$NEW" | jq -r .refresh_token)
|
|
AUTH="Authorization: Bearer $TOKEN"
|
|
|
|
curl -s -X POST $BASE/auth/logout -d "{\"refresh_token\":\"$REFRESH\"}"
|
|
```
|
|
|
|
Each refresh token works once. Presenting a spent one again, more than ten seconds after it was used, ends every session of that user.
|
|
|
|
**9. Add another user** (admins only):
|
|
|
|
```bash
|
|
TOKEN=$(curl -s -X POST $BASE/auth/login -d '{"email":"alice@example.org","password":"correct horse battery"}' | jq -r .access_token)
|
|
AUTH="Authorization: Bearer $TOKEN"
|
|
curl -s -X POST $BASE/users -H "$AUTH" -d '{"email":"bob@example.org","password":"another long password"}' | jq
|
|
```
|
|
|
|
## External providers
|
|
|
|
Sonda can ask Last.fm, MusicBrainz, Wikipedia, Wikimedia Commons and the Cover Art Archive about artists and albums, and can tell your own Lidarr which artist to add. This is **off by default**: until an admin turns a provider on, nothing leaves your server. Today Last.fm is used for similar artists and tags, MusicBrainz for tags and for the link to Wikidata, Wikipedia (with Wikidata) for descriptions of artists, Wikimedia Commons (with Wikidata) for free photos of artists, and the Cover Art Archive for the cover of an album that has none in your files.
|
|
|
|
- **Last.fm** needs an API key. Create one for your own use at `last.fm/api` (it needs a Last.fm account). No user name or password is involved and the "shared secret" is not needed. Last.fm data is for non-commercial use only, and you should credit Last.fm when you show it.
|
|
- **MusicBrainz** needs no key, but it asks every client to identify itself with a contact (a URL or an e-mail address) in the `User-Agent`. By default Sonda uses the address of the project, so no personal address is sent unless you choose to.
|
|
|
|
Set them up with `PUT /api/v1/providers/{name}` (admin), for example:
|
|
|
|
```bash
|
|
curl -s -X PUT $BASE/providers/lastfm -H "$AUTH" -d '{"api_key":"<your 32 character key>"}'
|
|
curl -s -X PUT $BASE/providers/lastfm -H "$AUTH" -d '{"enabled":true}'
|
|
curl -s $BASE/providers -H "$AUTH" | jq
|
|
curl -s -X POST $BASE/providers/lastfm/test -H "$AUTH" | jq # does the key work?
|
|
curl -s -X PUT $BASE/providers/musicbrainz -H "$AUTH" -d '{"enabled":true,"contact":"https://example.org/me"}'
|
|
```
|
|
|
|
**Lidarr** is not a source of data but a server of your own that looks for and downloads music. Sonda never downloads, searches for or chooses music: with the `lidarr` provider an admin can send an artist, by its MusicBrainz id and never by name, to *your* Lidarr, which then does what you set it up to do. Nothing is sent by itself, and only an admin sends. It needs the address of your Lidarr and its API key (Settings, General), then you pick, once, from what Lidarr offers:
|
|
|
|
```sh
|
|
curl -s -X PUT $BASE/providers/lidarr -H "$AUTH" -d '{"url":"http://orion:8686","api_key":"<the 32 character key of Lidarr>"}'
|
|
curl -s -X POST $BASE/providers/lidarr/test -H "$AUTH" | jq # does the key work?
|
|
curl -s $BASE/providers/lidarr/options -H "$AUTH" | jq # profiles and root folders to choose from
|
|
curl -s -X PUT $BASE/providers/lidarr -H "$AUTH" -d '{"lidarr":{"quality_profile_id":1,"metadata_profile_id":1,"root_folder_path":"/music","monitor":"all","search_missing":true}}'
|
|
curl -s -X PUT $BASE/providers/lidarr -H "$AUTH" -d '{"enabled":true}'
|
|
```
|
|
|
|
Then an admin sends an artist with one request, for any artist that has a MusicBrainz id, **including one that Sonda only learned of from Discover and that is not in your library**, which is the point: asking for music you do not have. Anyone signed in can see whether Lidarr has it:
|
|
|
|
```sh
|
|
curl -s -X POST $BASE/artists/<id>/lidarr -H "$AUTH" | jq # {"status":"added"} or {"status":"already_added"}
|
|
curl -s $BASE/artists/<id>/lidarr -H "$AUTH" | jq # {"provider_status":"ok","in_lidarr":true,"monitored":true}
|
|
```
|
|
|
|
Sending is explicit, for one artist, never automatic, and can be repeated: an artist that Lidarr already has changes nothing. If Lidarr does not answer in time the artist may have been added all the same, so read it before sending again.
|
|
|
|
The address can have a path if Lidarr runs under a URL base, and it cannot carry a user, a password or a query: the key is sent in a header, never in the address, and Sonda never follows a redirect, so it cannot go anywhere else (if your `http://` address redirects to `https://`, set the `https://` one) and never goes through a proxy named by the environment, since Lidarr is on your own network. `lidarr` replaces all its options at once; what it leaves out is `monitor` `all` and `search_missing` `true`. The folder of an artist that Lidarr downloads shows up in Lidarr's root folder only with the first import: add that folder as a library of Sonda yourself, and a scan finds the new files.
|
|
|
|
The key is stored in the database, in clear text, and is never returned by the API or written to the log: the list only says `has_api_key`. Because the database holds it, treat database backups like secrets. Removing the key turns the provider off, and each provider describes in the list exactly what it sends.
|
|
|
|
**Claude (Anthropic)** is the one provider that names things: with it on, a playlist, a mix or a smart playlist can get a name and a line of description that fit its songs. An admin turns it on with an Anthropic API key (`PUT /providers/anthropic`, the key is write-only, and `POST /providers/anthropic/test` reads one model, which costs nothing and sends no text). `POST /playlists/{id}/suggest-name`, `POST /mixes/{id}/suggest-name` and `POST /smart-playlists/{id}/suggest-name` answer with a name, a description and a `source`. They are a pipeline: the facts about the list (up to 5 genres, up to 8 artist names, how many songs, for a smart playlist a few words about its rules, and the language) go to a namer. With the provider off, or when Claude fails or declines, the local namer builds a name from the genres and the artists on this server, and nothing leaves it. When Claude answers, the name is kept (under a hash of the facts, which holds no artist name), so the same facts are not asked, or paid, twice. A suggestion changes nothing: the client decides what to do with it, and the web app offers it where a playlist is renamed. With the provider on, names are also made by themselves: a mix, a smart playlist or a suggested smart playlist that is kept as a playlist with `auto_name` (the web app always sends it) gets Claude's name and line of description, with `name` as the fallback when Claude is off, fails or declines; a smart playlist can be made with no name at all and Claude names it from the songs its rules give (without Claude that is `invalid_input`); and `GET /naming` says whether names are automatic, so that the editor does not ask for one. Every question to Claude is counted (a made name, a name taken from the kept ones, a failure with its reason, and the tokens that were billed, also for a failure that was billed) and nothing of what was sent is kept; `GET /providers/anthropic/usage?days=30` (admin) gives the totals, a row for each day and an **estimate** of the cost in dollars, worked out from the tokens and the price of the model (the invoice of Anthropic is the truth), and the Costs section of the admin area shows it for 7, 30 or 90 days; records older than a year are removed. The model is `claude-opus-5-5`, at low effort; the key is never taken from the environment of the server, and a redirect is never followed.
|
|
|
|
**What leaves your server, when a provider is on:** to Anthropic, only when someone asks for a name, up to 5 genres and 8 artist names of that list, how many songs it has, a few words about the rules of a smart playlist, the language, your API key, your IP address and a User-Agent (never your users, your listens, track titles or file paths); to Last.fm, the artist name (or MusicBrainz id) being looked up, your API key, your IP address and the `User-Agent`; to MusicBrainz, the MusicBrainz id of the artist (and, only for the covers of albums that have no id, the artist's name and the album's title: see **Covers**), your IP address and a `User-Agent` that carries your contact. Never your users, your listens or file paths. Requests only happen when someone asks for them, never as a bulk pass over your library (the exceptions are the three jobs that an admin starts on purpose, each of them one request that goes through the whole library, one item after the other, spaced out, and each stops when its providers keep failing: `POST /library/covers`, with the Cover Art Archive on, sends the release group id of every album that has no picture in its files (and, for the albums whose files have no id, with MusicBrainz on, the artist and the title, one request a second); `POST /library/artist-images`, with Commons and MusicBrainz on, sends the MusicBrainz id of every artist that has one, to MusicBrainz, Wikidata and Commons; `POST /library/descriptions`, with Wikipedia and MusicBrainz on, sends it to MusicBrainz, Wikidata and Wikipedia), and they are spaced out, retried with backoff, and stopped by a circuit breaker if a provider is down. When MusicBrainz answers that an id was merged into another, Sonda follows that one redirect, and only to an artist or a release group of musicbrainz.org over https; the new id is used for that request and is not stored, and any other redirect is refused. **Descriptions** (the Wikipedia provider, which also needs MusicBrainz on) send the MusicBrainz id to MusicBrainz, the Wikidata item to Wikidata and the title of the page to Wikipedia, with your IP address and a `User-Agent` that carries a contact (the one set for each provider; Wikimedia refuses requests without one), and never a name. **Photos** (the Commons provider, which also needs MusicBrainz on) send the MusicBrainz id to MusicBrainz, the Wikidata item to Wikidata, the name of the picture to Commons and the address of a copy to Wikimedia, with your IP address and a `User-Agent` with a contact, and never a name. **Covers** (the Cover Art Archive provider) send the MusicBrainz release group id of an album, read from the tags of its own files, and only when the files have no picture. When the files have no id either and the MusicBrainz provider is on too, the name of the album's artist and its title go to MusicBrainz, which is asked for the release group: a result is taken only when MusicBrainz scores it 95 or more and its title and artist are the ones asked for (compared without case, accents or punctuation), and anything doubtful is no cover. A title that ends with a note about the disc or the edition (`(disc 2)`, `(bonus disc)`, `(Deluxe Edition)`) is asked as it is first and, if MusicBrainz is not sure, once more without the note, because a release group is named after the album and not after a disc. The id that comes back is used for that request and is not stored: it is not what your files say, and it is never used to read an artist or to send anything to Lidarr; the redirects of the archive are followed only to archive.org, over https. Its covers have no licence of their own: Sonda keeps a copy for your library and never treats it as free. **Discover** asks Last.fm for the similar artists of each seed (the seed's name or MusicBrainz id) and, for the best suggestions that have no tags yet, for the tags of that suggestion (its name); never MusicBrainz. Seeds you name are sent when you ask. Seeds taken from your listening, which are the names of the artists you play most, are sent only if you turned on `discover_from_listening` yourself; otherwise Discover refuses to run without a `seed`. **Lidarr** (your own server, at the address an admin set) gets the MusicBrainz id of an artist an admin sends, or of an artist someone opens `GET /artists/{id}/lidarr` for, and the API key, nothing else; asking it what it offers and testing the key send only the key. Never a name, your users, your listens or your files. What Lidarr then does is up to its own indexers and download clients. Besides the three jobs, the other exception to "only when someone asks" is `POST /providers/{name}/test`, which an admin sends to check a key (or, for Lidarr, the address and the key); it sends nothing but the key.
|
|
|
|
**Similar artists.** `GET /api/v1/artists/{id}/similar` asks Last.fm for the artists similar to an artist (one of yours, or one Last.fm named earlier) and keeps the answer in your database: 7 days, or 1 day when Last.fm does not know the artist. Until then a second look costs nothing and sends nothing, and if Last.fm is down you still get the last answer, marked `stale`. If the provider is off the answer says `disabled` instead of failing. Each similar artist says whether you have it (`in_library`); the ones you do not have are stored as artists without files. They can be opened with `GET /artists/{id}` but are never part of your artist lists, and if you later add their music, the scan finds them there. Last.fm's `score` is relative to each answer (the best one is usually 1), so compare artists only inside one answer.
|
|
|
|
**Tags.** `GET /api/v1/artists/{id}/tags` asks each provider that is on, at the same time, and keeps the answers with the same rules (7 days, 1 for an unknown artist, old data if a provider fails). Each provider has its own entry with its own weights: Last.fm's are relative to the artist (1 to 100), MusicBrainz's are counts of votes, and they are never mixed. MusicBrainz also marks the tags it lists as genres. Its tags and genres are CC BY-NC-SA, so show the `attribution` that comes with them.
|
|
|
|
**Genres from your own files.** The scan also reads the genre tag of each file (several genres separated by `;`, at most five per file) and `GET /artists/{id}/tags` adds a `files` entry: the genres of the playable tracks the artist is credited on, each weighted by how many tracks have it. It needs no provider, asks nobody, and is always there.
|
|
|
|
**MusicBrainz ids come from your files, never from a guess.** If your files carry MusicBrainz artist ids in their tags (taggers like Picard write them), the scan reads them, offline, and keeps them on your artists, but only when each id can be paired with its artist for certain. MusicBrainz is asked only about artists whose id is known; for the others its entry says `no_id`. Sonda never searches MusicBrainz for an artist by name, because a search returns look-alikes and picking one would be a guess; the one search there is is for the cover of an album that has no id (see **Covers** below), and it takes a result only when it is sure. A normal scan reads a file's tags when the file is new or changed. To pick up ids and genres for music that is already indexed, scan with `full=true` (`POST /library/scan?full=true`, or `POST /libraries/{id}/scan?full=true`): every file is read again, keeps its identity, and your listening history is untouched.
|
|
|
|
**Taking it back.** `DELETE /api/v1/providers/{name}/data` (admin) removes everything a provider gave: its similar artists, its tags and the records of when it was asked, and then the artists that only that data named and that you have no files for. Last.fm's terms ask for this when an agreement ends. Your own artists, music and listens stay, and the provider is not switched off. An artist that only that provider's lists named goes too, even if someone had it on their wishlist or dismissed. An artist that no list names any more is also removed when the list that named it is replaced, so they do not pile up.
|
|
|
|
The artist is sent to Last.fm by its MusicBrainz id when your files gave one (if Last.fm does not know the id, by name), otherwise by its name exactly as it is written in your library, and Sonda never lets Last.fm correct it into another artist. Names and ids from Last.fm are never written onto your own artists.
|
|
|
|
## The API at a glance
|
|
|
|
Everything lives under `/api/v1`. Bodies are JSON. Errors always look like `{"error":{"code":"...","message":"..."}}`.
|
|
|
|
**Interactive documentation:** open `http://127.0.0.1:8080/api/docs/` in a browser. Use *Authorize* with the `access_token` from the login to try the protected operations. The OpenAPI file itself is at `/api/v1/openapi.yaml`. Both are public, since they describe the API and contain no data.
|
|
|
|
| Who | Method and path | What it does |
|
|
|---|---|---|
|
|
| anyone | `GET /health` | Liveness check. |
|
|
| anyone | `POST /auth/register` | Create the first user (admin). Closed afterwards. |
|
|
| anyone | `POST /auth/login`, `/auth/refresh`, `/auth/logout` | Open, renew and end a session. |
|
|
| signed in | `GET /auth/me`, `POST /auth/password` | Your account; change your password (signs you out everywhere). |
|
|
| signed in | `GET /artists`, `/albums`, `/tracks` and `/…/{id}` | Browse and search. Query: `q`, `limit` (default 50, max 200), `offset`, `library_id`, `artist_id`, `album_id`. |
|
|
| signed in | `GET /artists/{id}/tags` | What Last.fm and MusicBrainz say about the artist's kind of music, from each provider that is on. |
|
|
| signed in | `GET`, `HEAD /artists/{id}/image`, `GET /artists/{id}/image/credit` | A free picture of the artist from Wikimedia Commons (query `size=small\|large`, also `?access_token=`) and its credit. |
|
|
| signed in | `GET /artists/{id}/description` | What Wikipedia says about the artist, with its page and licence, from the Wikipedia provider when an admin switched it on. |
|
|
| signed in | `GET /artists/{id}/similar` | Artists similar to one, from Last.fm when an admin switched it on. Query: `limit` (default 20, max 50). |
|
|
| signed in | `GET`, `HEAD /tracks/{id}/stream` | Stream the original file. Also accepts `?access_token=`. |
|
|
| signed in | `POST /listens` | Record a `play`, `complete` or `skip`; the answer says whether it counts as a listen. |
|
|
| signed in | `GET /listens` | Your own history, newest first, with `counted` per event. Query: `limit`, `offset`, `counted=true`. |
|
|
| signed in | `GET /discover` | Artists you may like, with a score and its parts. Query: `seed` (an artist id, up to 5 times), `limit` (default 20, max 30), `in_library=true\|false`. |
|
|
| signed in | `GET`, `HEAD /albums/{id}/cover` | The cover of an album, a JPEG made from your own files. Query: `size=small\|large`. Also accepts `?access_token=`. |
|
|
| signed in | `GET /mixes`, `GET /mixes/{id}`, `POST /mixes/{id}/playlist` | Your mixes: list them, read one (query `variant`), keep one as a playlist. Made for you alone, from your history and your genres; nothing is stored. |
|
|
| signed in | `GET /genres` | The genres of the library with their number of songs, the biggest first, spellings merged: what the rules of a smart playlist choose from. |
|
|
| signed in | `GET /smart-playlists`, `POST /smart-playlists`, `GET`, `PATCH`, `DELETE /smart-playlists/{id}` | Your smart playlists: list them (query `limit`, `offset`), make one from rules, read, change or delete one. Never anybody else's. |
|
|
| signed in | `GET /naming` | Whether names are made by themselves (`auto`): true when the `anthropic` provider is on. |
|
|
| signed in | `POST /playlists/{id}/suggest-name`, `POST /mixes/{id}/suggest-name`, `POST /smart-playlists/{id}/suggest-name` | A name and a line of description for a list of songs (body: `lang`, `variant`): Claude's when an admin turned the `anthropic` provider on, a local one otherwise (`source`). |
|
|
| signed in | `GET /smart-playlists/suggested`, `GET /smart-playlists/suggested/{key}/tracks`, `POST /smart-playlists/suggested/{key}/playlist` | The smart playlists Sonda makes for everybody (a fixed set), their songs for you now (query `variant`), and one kept as a normal playlist. |
|
|
| signed in | `GET /smart-playlists/{id}/tracks`, `POST /smart-playlists/preview`, `POST /smart-playlists/{id}/playlist` | The songs the rules give now (query `variant`), a preview of rules that are not kept, and a smart playlist kept as a normal playlist. |
|
|
| signed in | `GET /playlists`, `POST /playlists`, `GET`, `PATCH`, `DELETE /playlists/{id}` | Your playlists: list them (query `limit`, `offset`), make one, read, rename or delete one. Never anybody else's. |
|
|
| signed in | `GET /playlists/{id}/tracks`, `POST /playlists/{id}/tracks`, `PATCH`, `DELETE /playlists/{id}/tracks/{item}` | The tracks of a playlist: list them in order, add some (at the end or at a `position`), move or remove an item. |
|
|
| signed in | `GET /wishlist`, `PUT /wishlist/{id}`, `DELETE /wishlist/{id}` | Your wishlist: list it (query `limit`, `offset`), add an artist, take one off. |
|
|
| signed in | `GET /discover/dismissed`, `POST /discover/{id}/dismiss`, `DELETE /discover/{id}/dismiss` | The artists you are not interested in: list them, dismiss one, bring it back. |
|
|
| signed in | `GET /me/taste` | The artists and tags Sonda learned you like, from your own listening. Query: `limit` (default 20, max 50). |
|
|
| signed in | `GET /me/settings`, `PATCH /me/settings` | Your own settings (today: `discover_from_listening`, off by default). |
|
|
| signed in | `GET /library/covers`, `GET /library/artist-images`, `GET /library/descriptions` | Progress of the job that builds the covers, looks for the artist pictures or for the artist descriptions, and how the last one ended. |
|
|
| signed in | `GET /library/scan` | Scan status and the last result. |
|
|
| admin | `GET`, `POST /libraries`; `PATCH`, `DELETE /libraries/{id}` | Manage libraries. Deleting one removes its index entries, never your files. |
|
|
| admin | `POST`, `DELETE /library/covers`; `POST`, `DELETE /library/artist-images`; `POST`, `DELETE /library/descriptions` | Start, in the background, a job that builds the cover of every album, looks for the picture of every artist or for the description of every artist (`?refresh=true` for all), or stop it. One of each at a time. |
|
|
| admin | `POST /library/scan`, `POST /libraries/{id}/scan` | Scan everything, or one library. `?full=true` reads every file again. |
|
|
| admin | `GET /library/duplicates`; `POST`, `GET`, `DELETE /library/duplicates/check`; `POST /library/duplicates/clean` | Find the songs that are in more than one file, read their files to tell the identical ones, and delete the extra copies. See **Duplicates**. |
|
|
| admin | `GET /providers`, `PUT /providers/{name}`, `POST /providers/{name}/test`, `DELETE /providers/{name}/data` | See, set up and check the providers (Last.fm, MusicBrainz, Wikipedia, Commons, the Cover Art Archive, Lidarr), and remove what they gave. The API key is write-only. |
|
|
| admin | `GET /providers/anthropic/usage` | What the Claude provider costs over the last days (query `days`, 1 to 365): totals, a row for each day with calls and an estimate in dollars. |
|
|
| admin | `GET /providers/lidarr/options` | The profiles and root folders that Lidarr offers, to choose the ones artists are sent with. |
|
|
| admin | `POST /artists/{id}/lidarr` | Send an artist to your own Lidarr, by its MusicBrainz id, which then looks for the music. Never by name, never by itself. |
|
|
| signed in | `GET /artists/{id}/lidarr` | Whether Lidarr has the artist and monitors it. |
|
|
| admin | `GET`, `POST /users`; `PATCH`, `DELETE /users/{id}`; `PUT /users/{id}/password` | Manage users and the admin role. The last admin cannot be demoted or deleted. |
|
|
|
|
A library path must be an absolute, existing folder, never the filesystem root, and never inside or around another library. Search is a case- and accent-insensitive substring match: `bjork`, `BJORK` and `BJÖRK` all find `Björk`, and `strasse` finds `Straße`. Accents still tell artists apart: `Beyoncé` and `Beyonce` stay two artists, and a search for either finds both.
|
|
|
|
## Security notes
|
|
|
|
- Sonda listens on localhost only, unless you change `server.listen`. To use it from other machines, put it behind a reverse proxy that terminates HTTPS.
|
|
- Passwords are hashed with Argon2id. Refresh tokens are stored only as hashes. An access token is checked against the database on every request, so it stops working as soon as its account is deleted, even if the same user id is given to a new account.
|
|
- Music folders are read-only to Sonda, except for one thing that is off until you turn it on: deleting the extra copy of a song (`library.allow_delete`). Streaming and that deletion open files through `os.Root`, so a path can never leave its library, even if a symlink is swapped in later.
|
|
- File paths are never returned by the catalog, and audio file paths are not logged.
|
|
- Provider API keys are write-only through the API and never logged. They sit in the database, so protect your backups.
|
|
|
|
## Back up
|
|
|
|
Back up the database and `sonda.toml`. Your music stays wherever you keep it.
|
|
|
|
- **SQLite:** `sqlite3 sonda.db ".backup sonda-backup.db"` works while Sonda runs. Or stop Sonda and copy `sonda.db` (and `sonda.db-wal` if it exists).
|
|
- **PostgreSQL:** `pg_dump` the database.
|
|
- `sonda.toml` holds the secret that signs tokens; treat the copy like a password.
|
|
|
|
## Web app
|
|
|
|
A web client lives in `web/` (React, Vite and shadcn/ui). It talks only to the Sonda server: no CDN, no external fonts, no analytics. With the app built into the binary (`make build`, or the Docker image) `sondad` serves it: open the address of the server, for example `http://127.0.0.1:8080`, and sign in. The page carries a strict Content-Security-Policy (only resources of the server itself, no inline scripts), cannot be framed and sends no referrer, and a link inside the app (`/admin/users`) works when you reload it.
|
|
|
|
To work on the interface itself, run the Vite dev server next to a running `sondad` (set `SONDA_URL` to where it listens: the default is `http://127.0.0.1:8080`):
|
|
|
|
```bash
|
|
cd web
|
|
npm install
|
|
SONDA_URL=http://127.0.0.1:8080 npm run dev # open http://localhost:5173, /api is proxied to Sonda
|
|
```
|
|
|
|
What it does today: sign in (and create the first account of a new server), browse artists, albums and songs, search, and play with a queue, shuffle, repeat, media keys, a volume slider with mute (remembered in the browser) and a full-screen player. Artist pages show the Wikipedia text, similar artists and tags when those providers are on, each with its credit, and the photo credit next to the photo. It reports one listen per play. Playlists have a page of their own, shown as cards with a collage of their covers (as the mixes and the smart playlists have): make one, put a song in it with the button next to any song (or make a new playlist right there), play it, rename it, delete it, move a song up or down and take one out; a song whose files are gone is shown as not available and skipped when the playlist plays. Mixes have a page too: a card for each mix, with a collage of the covers of its albums (the ones from your listening first, then one for each genre), and on a mix its songs, Play, Shuffle again (another set of songs) and Save as playlist, which keeps what you see as a playlist of your own. Where a playlist is renamed, and where a smart playlist is edited, a Suggest a name button asks for a name. Smart playlists have their own page too, which begins with the ones Sonda makes for you (ready to play, to keep as a playlist, or to make your own with a click), and then yours: make one with an editor of the rules (genres, format and bitrate, when it was added, your listening and taste, the length, the order and the limit) and a live preview of the songs it gives, play it, shuffle it again (for the random order), keep what you see as a playlist, edit it or delete it; rules by artist and album are in the API only, and an edit keeps them. Discover suggests artists from the ones you pick, or from your listening only after you allow it (a dialog says that their names go to Last.fm), explains each score, and keeps your wishlist and the artists you are not interested in. A suggested artist you do not have shows a letter, not a photo, because a photo is a lookup at the providers: a "Show photo" button on the card asks for that one artist when you press it (the card says it goes to Wikimedia Commons by MusicBrainz id, and says so when there is no free photo). An admin who set up the `lidarr` provider gets a "Request on Lidarr" button on an artist's page and on the Discover cards: it asks first, then sends that one artist to their own Lidarr, which does the downloading (the web app downloads nothing; the page of an artist reads what Lidarr has, the cards ask nothing until you send). An admin also gets an Admin area to add, rename, move and remove libraries and to start a scan (of one library or all, optionally reading every file again) and follow it, and to manage accounts: add one by email, give or take away the admin role, reset a password (which ends that person's sessions) and delete an account. The last admin cannot be changed, and your own account has no buttons there. Providers have a card each that says what the provider is sent: turning one on asks first, the API key is write-only (never shown, and gone from the page once saved), and there are a test of the key, the contact or address, removing what a provider gave, and, for Lidarr, the profiles and root folder to use when an artist is added. The bulk jobs (album covers, artist pictures, artist descriptions) each have a card that says what the job sends and to whom: starting one asks first and says it goes through the whole library, reading everything again is a separate choice, a job whose providers are off cannot be started, and the page follows its progress and can stop it. The web app is a PWA: it has a manifest and a service worker, so a browser can install it (Settings has an Install button where the browser lets a page ask) and it opens at once from its own icon, and shows its own page when the server cannot be reached. The worker keeps only the files of the app (the page, its scripts and styles, the manifest and the icons) and never anything from `/api/`: the catalog, the covers, the songs and what a person did are private and are not kept. A new version waits until you press Reload, so nothing changes under a song. It needs https, or localhost. The menu on the left folds to a narrow bar with the icons of the pages (a button in it, remembered in the browser; on a phone it stays a row of links). There are six themes (Sonda, Tokyo Night, Gruvbox dark, Gruvbox light, Dracula and Neon) and two languages (English and Italian), both chosen in Settings.
|
|
|
|
More commands, all run in `web/`: `npm test`, `npm run build`, and `npm run gen:api`, which rebuilds `src/api/schema.d.ts` from the OpenAPI spec (it fetches the generator with `npx`, so it needs the network once). `npm run build` writes to `internal/webui/dist` (git-ignored), not to `web/dist`.
|
|
|
|
## Development
|
|
|
|
```bash
|
|
go vet ./...
|
|
go test -race ./...
|
|
```
|
|
|
|
The default test run needs no network and uses SQLite only. To run the same storage tests against PostgreSQL as well, start a throwaway server and point the tests at it:
|
|
|
|
```bash
|
|
docker run -d --rm --name sonda-test-pg -e POSTGRES_PASSWORD=sonda -e POSTGRES_DB=sonda -p 127.0.0.1:55432:5432 postgres:17-alpine
|
|
SONDA_TEST_POSTGRES_DSN='postgres://postgres:sonda@127.0.0.1:55432/sonda?sslmode=disable' go test -race ./...
|
|
docker rm -f sonda-test-pg
|
|
```
|
|
|
|
Tests that read real audio need `ffmpeg` and are skipped when it is missing.
|
|
|
|
Layout:
|
|
|
|
```
|
|
cmd/sondad/ the server: wiring, config, logging, shutdown
|
|
internal/api/http/ the REST API (thin handlers) and the docs routes
|
|
internal/api/openapi/ the OpenAPI spec, embedded in the binary
|
|
internal/auth/ users, roles, passwords, access and refresh tokens
|
|
internal/catalog/ browsing and searching artists, albums, tracks
|
|
internal/library/ libraries, the scanner, the event hub, the tag reader
|
|
internal/about/ descriptions of artists, from Wikipedia through MusicBrainz and Wikidata
|
|
internal/discovery/ similar artists: what the providers said, kept and refreshed
|
|
internal/providers/ external providers: settings, the outbound HTTP client, the clients of the providers
|
|
internal/naming/ names for lists of songs: a local namer, and Claude when it is on
|
|
internal/listening/ listen events
|
|
internal/taste/ what the listener likes, learned from the listening history
|
|
internal/discover/ the score of an artist you may like, and the suggestions
|
|
internal/feedback/ the wishlist and the artists a user is not interested in
|
|
internal/playlists/ the playlists of each user
|
|
internal/mixes/ the mixes made for a listener, by genre and by listening
|
|
internal/smart/ smart playlists: rules that are worked out into songs
|
|
internal/images/ covers made from the pictures in the user's own files
|
|
internal/usersettings/ the settings that belong to one user
|
|
internal/storage/ SQLite and PostgreSQL through bun, the repositories
|
|
internal/streaming/ opening a track's file safely
|
|
migrations/ SQL migrations, one folder per database
|
|
web/ the web client (React, Vite, shadcn/ui)
|
|
Dockerfile, compose.yaml the image of sondad, and an example for Docker Compose
|
|
```
|
|
|
|
The API description is `internal/api/openapi/openapi.yaml`, written by hand and embedded in the binary. Two tests keep it in step with the code: one fails when a route and the spec disagree in either direction, and a contract test sends real requests through every operation and validates each response (status, headers, JSON body) against the spec. Adding or changing an endpoint therefore means updating the spec in the same change. Response objects are closed (`additionalProperties: false`), so a field added in the code and forgotten in the spec fails the build.
|
|
|
|
Migrations are plain SQL files in `migrations/sqlite` and `migrations/postgres`. They must describe the same schema, and a test checks that both folders hold the same files. Add a new numbered file to both whenever the schema changes.
|
|
|
|
## Third-party code
|
|
|
|
Audio tags are read with `go.senan.xyz/taglib`, which embeds [TagLib](https://github.com/taglib/taglib) compiled to WebAssembly and is licensed under the LGPL 2.1. The Swagger UI assets come from `github.com/swaggest/swgui` (Apache-2.0), which embeds [Swagger UI](https://github.com/swagger-api/swagger-ui) (Apache-2.0). The other dependencies are listed in `go.mod`.
|