The README had grown to about 500 lines. It now holds the intro, a quick start and an index, and the rest lives in docs/. - docs: add docs/README.md as the index of all pages - docs: move install, configuration, demo, providers, OpenSubsonic, API, security and backups, web app, Android app and development/releases to their own pages, text unchanged - docs: split the feature list into five pages under docs/features/ - docs: rewrite internal anchors as links between the new pages - chore: add docs to .dockerignore, next to README.md
15 KiB
External providers
The commands below use $BASE and $AUTH from the demo.
Sonda can ask Last.fm, MusicBrainz, Wikipedia, Wikimedia Commons and the Cover Art Archive about artists and albums, and can tell your own Lidarr which artist to add. This is off by default: until an admin turns a provider on, nothing leaves your server. Today Last.fm is used for similar artists and tags, MusicBrainz for tags and for the link to Wikidata, Wikipedia (with Wikidata) for descriptions of artists, Wikimedia Commons (with Wikidata) for free photos of artists, and the Cover Art Archive for the cover of an album that has none in your files.
- Last.fm needs an API key. Create one for your own use at
last.fm/api(it needs a Last.fm account). No user name or password is involved and the "shared secret" is not needed. Last.fm data is for non-commercial use only, and you should credit Last.fm when you show it. - MusicBrainz needs no key, but it asks every client to identify itself with a contact (a URL or an email address) in the
User-Agent. By default Sonda uses the address of the project, so no personal address is sent unless you choose to.
Set them up with PUT /api/v1/providers/{name} (admin), for example:
curl -s -X PUT $BASE/providers/lastfm -H "$AUTH" -d '{"api_key":"<your 32 character key>"}'
curl -s -X PUT $BASE/providers/lastfm -H "$AUTH" -d '{"enabled":true}'
curl -s $BASE/providers -H "$AUTH" | jq
curl -s -X POST $BASE/providers/lastfm/test -H "$AUTH" | jq # does the key work?
curl -s -X PUT $BASE/providers/musicbrainz -H "$AUTH" -d '{"enabled":true,"contact":"https://example.org/me"}'
Lidarr is not a source of data but a server of your own that looks for and downloads music. Sonda never downloads, searches for or chooses music: with the lidarr provider an admin can send an artist, by its MusicBrainz id and never by name, to your Lidarr, which then does what you set it up to do. Nothing is sent by itself, and only an admin sends. It needs the address of your Lidarr and its API key (Settings, General), then you pick, once, from what Lidarr offers:
curl -s -X PUT $BASE/providers/lidarr -H "$AUTH" -d '{"url":"http://orion:8686","api_key":"<the 32 character key of Lidarr>"}'
curl -s -X POST $BASE/providers/lidarr/test -H "$AUTH" | jq # does the key work?
curl -s $BASE/providers/lidarr/options -H "$AUTH" | jq # profiles and root folders to choose from
curl -s -X PUT $BASE/providers/lidarr -H "$AUTH" -d '{"lidarr":{"quality_profile_id":1,"metadata_profile_id":1,"root_folder_path":"/music","monitor":"all","search_missing":true}}'
curl -s -X PUT $BASE/providers/lidarr -H "$AUTH" -d '{"enabled":true}'
Then an admin sends an artist with one request, for any artist that has a MusicBrainz id, including one that Sonda only learned of from Discover and that is not in your library, which is the point: asking for music you do not have. Anyone signed in can see whether Lidarr has it:
curl -s -X POST $BASE/artists/<id>/lidarr -H "$AUTH" | jq # {"status":"added"} or {"status":"already_added"}
curl -s $BASE/artists/<id>/lidarr -H "$AUTH" | jq # {"provider_status":"ok","in_lidarr":true,"monitored":true}
Sending is explicit, for one artist, never automatic, and can be repeated: an artist that Lidarr already has changes nothing. If Lidarr does not answer in time the artist may have been added all the same, so read it before sending again.
The address can have a path if Lidarr runs under a URL base, and it cannot carry a user, a password or a query: the key is sent in a header, never in the address, and Sonda never follows a redirect, so it cannot go anywhere else (if your http:// address redirects to https://, set the https:// one) and never goes through a proxy named by the environment, since Lidarr is on your own network. lidarr replaces all its options at once; what it leaves out is monitor all and search_missing true. The folder of an artist that Lidarr downloads shows up in Lidarr's root folder only with the first import: add that folder as a library of Sonda yourself, and a scan finds the new files.
The key is stored in the database, in clear text, and is never returned by the API or written to the log: the list only says has_api_key. Because the database holds it, treat database backups like secrets. Removing the key turns the provider off, and each provider describes in the list exactly what it sends.
Claude (Anthropic) is the one provider that names things: with it on, a playlist, a mix or a smart playlist can get a name and a line of description that fit its songs. An admin turns it on with an Anthropic API key (PUT /providers/anthropic, the key is write-only, and POST /providers/anthropic/test reads one model, which costs nothing and sends no text). POST /playlists/{id}/suggest-name, POST /mixes/{id}/suggest-name and POST /smart-playlists/{id}/suggest-name answer with a name, a description and a source. They are a pipeline: the facts about the list (up to 5 genres, up to 8 artist names, how many songs, for a smart playlist a few words about its rules, and the language) go to a namer. With the provider off, or when Claude fails or declines, the local namer builds a name from the genres and the artists on this server, and nothing leaves it. When Claude answers, the name is kept (under a hash of the facts, which holds no artist name), so the same facts are not asked, or paid, twice. A suggestion changes nothing: the client decides what to do with it, and the web app offers it where a playlist is renamed. With the provider on, names are also made by themselves: a mix, a smart playlist or a suggested smart playlist that is kept as a playlist with auto_name (the web app always sends it) gets Claude's name and line of description, with name as the fallback when Claude is off, fails or declines; a smart playlist can be made with no name at all and Claude names it from the songs its rules give (without Claude that is invalid_input); and GET /naming says whether names are automatic, so that the editor does not ask for one. Every question to Claude is counted (a made name, a name taken from the kept ones, a failure with its reason, and the tokens that were billed, also for a failure that was billed) and nothing of what was sent is kept; GET /providers/anthropic/usage?days=30 (admin) gives the totals, a row for each day and an estimate of the cost in dollars, worked out from the tokens and the price of the model (the invoice of Anthropic is the truth), and the Costs section of the admin area shows it for 7, 30 or 90 days; records older than a year are removed. The model is claude-opus-5-5, at low effort; the key is never taken from the environment of the server, and a redirect is never followed.
What leaves your server, when a provider is on: to Anthropic, only when someone asks for a name, up to 5 genres and 8 artist names of that list, how many songs it has, a few words about the rules of a smart playlist, the language, your API key, your IP address and a User-Agent (never your users, your listens, song titles or file paths); to Last.fm, the artist name (or MusicBrainz id) being looked up, your API key, your IP address and the User-Agent; to MusicBrainz, the MusicBrainz id of the artist (and, only for the covers of albums that have no id, the artist's name and the album's title: see Covers), your IP address and a User-Agent that carries your contact. Never your users, your listens or file paths. Requests only happen when someone asks for them, never as a bulk pass over your library (the exceptions are the three jobs that an admin starts on purpose, each of them one request that goes through the whole library, one item after the other, spaced out, and each stops when its providers keep failing: POST /library/covers, with the Cover Art Archive on, sends the release group id of every album that has no picture in its files (and, for the albums whose files have no id, with MusicBrainz on, the artist and the title, one request a second); POST /library/artist-images, with Commons and MusicBrainz on, sends the MusicBrainz id of every artist that has one, to MusicBrainz, Wikidata and Commons; POST /library/descriptions, with Wikipedia and MusicBrainz on, sends it to MusicBrainz, Wikidata and Wikipedia), and they are spaced out, retried with backoff, and stopped by a circuit breaker if a provider is down. When MusicBrainz answers that an id was merged into another, Sonda follows that one redirect, and only to an artist or a release group of musicbrainz.org over https; the new id is used for that request and is not stored, and any other redirect is refused. Descriptions (the Wikipedia provider, which also needs MusicBrainz on) send the MusicBrainz id to MusicBrainz, the Wikidata item to Wikidata and the title of the page to Wikipedia, with your IP address and a User-Agent that carries a contact (the one set for each provider; Wikimedia refuses requests without one), and never a name. Photos (the Commons provider, which also needs MusicBrainz on) send the MusicBrainz id to MusicBrainz, the Wikidata item to Wikidata, the name of the picture to Commons and the address of a copy to Wikimedia, with your IP address and a User-Agent with a contact, and never a name. Covers (the Cover Art Archive provider) send the MusicBrainz release group id of an album, read from the tags of its own files, and only when the files have no picture. When the files have no id either and the MusicBrainz provider is on too, the name of the album's artist and its title go to MusicBrainz, which is asked for the release group: a result is taken only when MusicBrainz scores it 95 or more and its title and artist are the ones asked for (compared without case, accents or punctuation), and anything doubtful is no cover. A title that ends with a note about the disc or the edition ((disc 2), (bonus disc), (Deluxe Edition)) is asked as it is first and, if MusicBrainz is not sure, once more without the note, because a release group is named after the album and not after a disc. The id that comes back is used for that request and is not stored: it is not what your files say, and it is never used to read an artist or to send anything to Lidarr; the redirects of the archive are followed only to archive.org, over https. Its covers have no licence of their own: Sonda keeps a copy for your library and never treats it as free. Discover asks Last.fm for the similar artists of each seed (the seed's name or MusicBrainz id) and, for the best suggestions that have no tags yet, for the tags of that suggestion (its name); never MusicBrainz. Seeds you name are sent when you ask. Seeds taken from your listening, which are the names of the artists you play most, are sent only if you turned on discover_from_listening yourself; otherwise Discover refuses to run without a seed. Lidarr (your own server, at the address an admin set) gets the MusicBrainz id of an artist an admin sends, or of an artist someone opens GET /artists/{id}/lidarr for, and the API key, nothing else; asking it what it offers and testing the key send only the key. Never a name, your users, your listens or your files. What Lidarr then does is up to its own indexers and download clients. Besides the three jobs, the other exception to "only when someone asks" is POST /providers/{name}/test, which an admin sends to check a key (or, for Lidarr, the address and the key); it sends nothing but the key.
Similar artists. GET /api/v1/artists/{id}/similar asks Last.fm for the artists similar to an artist (one of yours, or one Last.fm named earlier) and keeps the answer in your database: 7 days, or 1 day when Last.fm does not know the artist. Until then a second look costs nothing and sends nothing, and if Last.fm is down you still get the last answer, marked stale. If the provider is off the answer says disabled instead of failing. Each similar artist says whether you have it (in_library); the ones you do not have are stored as artists without files. They can be opened with GET /artists/{id} but are never part of your artist lists, and if you later add their music, the scan finds them there. Last.fm's score is relative to each answer (the best one is usually 1), so compare artists only inside one answer.
Tags. GET /api/v1/artists/{id}/tags asks each provider that is on, at the same time, and keeps the answers with the same rules (7 days, 1 for an unknown artist, old data if a provider fails). Each provider has its own entry with its own weights: Last.fm's are relative to the artist (1 to 100), MusicBrainz's are counts of votes, and they are never mixed. MusicBrainz also marks the tags it lists as genres. Its tags and genres are CC BY-NC-SA, so show the attribution that comes with them.
Genres from your own files. The scan also reads the genre tag of each file (several genres separated by ;, at most five per file) and GET /artists/{id}/tags adds a files entry: the genres of the playable songs the artist is credited on, each weighted by how many songs have it. It needs no provider, asks nobody, and is always there.
MusicBrainz ids come from your files, never from a guess. If your files carry MusicBrainz artist ids in their tags (taggers like Picard write them), the scan reads them, offline, and keeps them on your artists, but only when each id can be paired with its artist for certain. MusicBrainz is asked only about artists whose id is known; for the others its entry says no_id. Sonda never searches MusicBrainz for an artist by name, because a search returns look-alikes and picking one would be a guess; the one search there is is for the cover of an album that has no id (see Covers below), and it takes a result only when it is sure. A normal scan reads a file's tags when the file is new or changed. To pick up ids and genres for music that is already indexed, scan with full=true (POST /library/scan?full=true, or POST /libraries/{id}/scan?full=true): every file is read again, keeps its identity, and your listening history is untouched.
Taking it back. DELETE /api/v1/providers/{name}/data (admin) removes everything a provider gave: its similar artists, its tags and the records of when it was asked, and then the artists that only that data named and that you have no files for. Last.fm's terms ask for this when an agreement ends. Your own artists, music and listens stay, and the provider is not turned off. An artist that only that provider's lists named goes too, even if someone had it on their wishlist or dismissed. An artist that no list names any more is also removed when the list that named it is replaced, so they do not pile up.
The artist is sent to Last.fm by its MusicBrainz id when your files gave one (if Last.fm does not know the id, by name), otherwise by its name exactly as it is written in your library, and Sonda never lets Last.fm correct it into another artist. Names and ids from Last.fm are never written onto your own artists.