A new setting, radio_in_apps, off until the person turns it on (Settings, Apps and
devices, or PATCH /me/settings). With it off, the list of playlists of an OpenSubsonic
app is only the playlists of the person, and a station asked for by id is not found.
With it on, the stations come as playlists called Radio · Rock.
- migration 0036: user_settings.radio_in_apps
- usersettings, the settings API and the OpenAPI spec have the new field; the PATCH
now keeps each setting apart (naming one no longer sets the other)
- Settings has a switch
Stations made from the music, random and without end, with nothing to set up: the
mixes the listener has (genres, favourites, rediscover, new to you), everything,
one for each decade that has 30 songs by 4 artists, and the radio of an artist (their
songs, four times as likely, and the songs of the two genres most of theirs have).
Every time a station is asked for it picks other songs, with the same weights as the
mixes (taste, hearts, nothing skipped lately).
- mixes.Service.Radios and Radio; Filter has All and years, the store has Decades
- GET /radio and GET /radio/{id}, in the OpenAPI spec
- OpenSubsonic: each station is a playlist (rd-<id>, "Radio · Rock") that has other
songs every time it is opened, for apps that cannot play a radio that never ends
A heart adds to what the listening says of an artist: a favourite artist counts
as much as three complete listens, a favourite album 1.5 for its album artist and
a favourite song 1 for each of its main artists, at most 6 for one artist, without
fading with time. An artist hearted but never played is liked, with no listens.
That feeds Discover, the mixes and the smart playlists that use the taste, and
GET /me/taste.
In the mixes of what the listener likes (genre mixes, Your favourites,
Rediscover) a hearted song is wanted 1.5 times as much and a song on a hearted
album 1.25 times. New to you is not changed.
A person can mark songs, albums and artists as favourites. They are theirs
alone: never shown to a friend, never sent anywhere. Only what can be played is
listed.
- migration 0035 favorites, internal/favorites, storage.FavoritesRepo
- PUT/DELETE /me/favorites/{kind}/{id}, GET /me/favorites/{kind} and
GET /me/favorites/ids, in the OpenAPI spec
- catalog.Filter IDs now also reads artists by id
- OpenSubsonic: star, unstar, getStarred2, the starred date on songs, albums and
artists, and getAlbumList2 of type starred
Apps can play and browse: stream and download send the original file (Range,
HEAD), getCoverArt serves the cover of an album, an artist or the album of a
song (300 pixels or less is the small one), getAlbumList2 knows every kind of
list, getRandomSongs and getSongsByGenre list songs, search3 finds artists,
albums and songs, and the starred calls answer with nothing so apps do not break.
- catalog.Filter has IDs to read albums and songs by id
- storage.SubsonicRepo answers the album and song lists (random, newest,
frequent, recent, by name, by artist, by year, by genre)
/rest speaks the OpenSubsonic API (Subsonic 1.16.1) so that other apps can browse
the music of a server. A call logs in with the password of a device: an apiKey,
Subsonic's token (u, t, s) or the password itself (u, p). The account password is
never accepted there.
- devices: ByKey, ByToken, ByPassword and the note of last use
- opensubsonic package: JSON and XML answers, ping, getLicense,
getOpenSubsonicExtensions, tokenInfo, getMusicFolders, getArtists, getArtist,
getAlbum, getSong, getGenres
- storage.SubsonicRepo: year, genres, size, bitrate, suffix and added date
A person can make a password for each app that connects to their server, in
Settings, Apps and devices. The secret is shown once, can be taken back, and
is for the music API of other apps only: it cannot sign in or change the
account. Secrets are kept sealed (AES-256-GCM, key derived from the jwt secret)
because token login needs them in clear, and a hash finds a device by its key.
- migration 0034 device_passwords (sqlite, postgres)
- internal/devices service, storage repo
- GET/POST /me/devices, DELETE /me/devices/{id}, in the OpenAPI spec
- Settings section with copy and remove
- GET /stats/listening has genre_timeline: the share of the five biggest genres, of the smaller ones and of the songs with no genre, for each year, or for each month when all the listens are in one year, in the listener's time zone; years and months with no listen are empty
- web: a stacked column chart on the Statistics page, coloured in tints of the colour of this server so it follows every theme, with a legend; left out when there is only one column
- README: the new field
- GET /stats/library: the songs that can be played with their artists, albums, length and size, the files by format, the share of each genre and decade
- GET /stats/listening: listens, time listened, songs and artists, listens by year, hour and weekday in the listener's time zone, top artists and songs, the share of each genre, for all time or a period
- a song with several genres is split between them, so the shares add up
- web: a Statistics page in the Your listening group, with bars made of CSS and four periods
- the MP3 files, which the scanner calls mpeg, are shown as MP3
- README: the two endpoints
Migrating hosting from Codeberg to git.kmalloc.org. Renames the
module path in go.mod, every internal import, and the OCI image
source label in the Dockerfile. Build and full test suite verified
green after the rename.
Sonda kept no record of what an admin sent to Lidarr. Each send is now kept
(migration 0033, lidarr_requests), once for each artist, with the first date.
GET /lidarr/requests lists them newest first and says for each whether its
music is in the library: it is noticed when the list is read, by the artist
that was sent or by an artist of the same name that a scan made, and the day
it first was is kept.
The web app has a Requests page for admins, under Discover in the menu and on
Home on a phone, with the waiting and the arrived artists apart.
GET /listens/months counts the listens that count in each calendar month,
newest first, in the time zone the client sends (the zone database is built
into the binary). GET /listens takes from and until to read one month, and
each listen now names its artists.
The web app has a History page: a button for each year with its listens, the
months of the year as bars, and the listens of the chosen month a day at a
time, with the choice kept in the address.
A new model (internal/affinity) splits my listening into sittings (plays less
than 30 minutes apart) and links the artists that are often played in the same
sitting. It is built from my listening events and from the plays of artists I
have no music of that an import kept, on my server, with no provider, and is
built again when plays are added (migration 0032 keeps it as a cache).
GET /me/suggestions gives two lists, each artist with the artists it goes
with: the ones I have and did not play for two months, and the ones I played
at least three times and have no music of. It needs discover_from_listening.
POST /me/suggestions/wishlist puts an artist I played on my wishlist by name.
Discover has a "From your listening" tab for both lists.
The plays of songs that are not in the library were only counted. Now, when
such a play counts as listened (not a skip, long enough), its artist and its
time are kept in a new table (migration 0031, outside_plays), so a model can
later learn which artists are played together, also those not in the library.
No title, album, IP address, country or platform is stored. The same play is
kept once however many times the export is imported, and the report says how
many were kept (history.kept_outside).
The report of an import used to be shown once in Settings and lost. Each
import is now stored (migration 0030, spotify_imports) as the report in
JSON, and GET /me/import/spotify lists the past ones, newest first.
POST /me/import/spotify answers with the same shape plus id and created_at.
The web app gets an Import page (menu, More tab) with the form and one card
for each past import: plays read, added, already there, not in the library,
the playlists it made and the artists that are missing. Settings only links
to it. An import that finds nothing is not kept, and a report that cannot be
stored is still returned to the person who asked.
I want my listening history and my playlists from Spotify in Sonda, so that Mixes, Discover and the smart playlists know what I listened to before.
- POST /me/import/spotify takes the zip of a Spotify data export, or the JSON files in it (extended streaming history, the older StreamingHistory files, the playlists), as a multipart form. A request may be up to 512 MB; the zip is limited in how much it may inflate and how many files it may hold, and only the history and playlist files are opened.
- Everything is read on the server and nothing is sent anywhere. The files are not kept, and of each play only the song, artist, album, time and how it ended are read: never the IP addresses, the country or the platform.
- A play is matched to a song of the library by artist and title (no case, accents or punctuation; a "Remastered" note on either side; any credited artist; a compilation by its album). A match is recorded as a listen at the time of the play: complete, skip or play. Nothing is posted to Mastodon.
- Importing the same files again records nothing twice, and the same play in the two shapes of history file is counted once. One import per person at a time.
- A playlist becomes a playlist of mine with the songs that were found; one with a name I already have is left alone, and one with none of its songs found is not made.
- The report counts what was added, what was already there and what is not in the library, and lists the missing artists, most played first.
- Settings gets an Import from Spotify section; the API client can send files.
- English and Italian.
I want songs from different albums to play at about the same loudness without touching the volume.
- The scan reads REPLAYGAIN_TRACK_GAIN, REPLAYGAIN_ALBUM_GAIN and the two peaks from each file, offline, and keeps them on the file (migration 0029, SQLite and PostgreSQL). A tag that makes no sense (not a number, more than 60 dB) is left out. Files indexed before have none until a full rescan.
- A track in the API carries them as replay_gain (null when its first playable file says nothing).
- The web player levels each song with them: the sound passes through a gain node, so a quiet song can be raised too, but never so much that its peak would clip (and never more than 12 dB when there is no peak). Without Web Audio the element is turned down instead.
- A new Playback section in Settings chooses song by song (default), album by album or off; the choice stays on the device.
- README: the ReplayGain entry, and the old line that said the year was not in the index is removed.
- English and Italian.
I can now see on Home what I added to the library lately.
- GET /albums takes sort=recent: the albums added last first, by the date of the newest file of the album that is not missing, then the newest album. The default stays by title and any other value is invalid_input.
- The albums shelf on Home is Recently added and asks for that order; See all leads to /albums?sort=recent, which the albums page understands.
- English and Italian.
I can now ask for songs from a year to a year, for example the eighties.
- The year is read from the tags of each file (the original date first, then the date) and kept on the track: migration 0028 adds tracks.year (SQLite and PostgreSQL).
- Two new rules, min_year and max_year, both included and checked (1000 to 2999, the first not after the second). The index answers them. A song with no year never matches.
- Files indexed before have no year until a full rescan reads their tags again; the editor and the API description say so.
- The smart playlist editor gets From year and Up to year, and the rules are described in words.
- English and Italian.
When the instance refuses the token on a post (it was revoked or expired), the account now remembers it: GET /me/mastodon says problem: token_refused and nothing is posted until I connect again with a new token.
- migration 0027 adds mastodon_accounts.problem (SQLite and PostgreSQL).
- Only a refused token marks the account; an instance that is down does not.
- Saving the account again clears the problem.
- Settings shows the warning and the form with the instance already filled in, next to Disconnect.
- English and Italian.
In Settings a person connects their Mastodon account: the address of their instance and an access token that can only write posts. Nothing is connected by default, and disconnecting forgets the token.
- When a track of this server counts as a listen, it is posted as an unlisted status: title, artists, album and #NowPlaying, with no link. Music of a friend is never reported as a listen, so it is never posted. At most one post a minute for each person, in the background, with a bounded queue.
- PUT, GET and DELETE /me/mastodon. The token is never sent back.
- The address of the instance is typed by a person, so it must be a domain name on https, and the client never connects to a private, loopback or link-local address (checked on the address it really dials, so a name that points inside is no way in), never follows a redirect, ignores proxy settings and reads at most a megabyte.
- New table mastodon_accounts (SQLite and PostgreSQL) that goes with the account.
- README: the feature, and the credit for butterchurn and its presets.
The seek bar of the player is the shape of the track, with the part already
heard in colour, as in the approved mockups.
- GET /tracks/{id}/peaks gives 120 levels from 0 to 255 (the loudest part is
255 whatever the volume). The server finds them by decoding the file, MP3 with
go-mp3 and FLAC with eaburns/flac, both in pure Go. It takes a few seconds, so
the first request starts the work in the background and answers 202 with
Retry-After. It is done once for each file (migration 0025), worked out again
if the size or the date of the file change, one track at a time so that a NAS
is not kept busy, and a file that cannot be decoded is remembered as such. A
track deleted while its file is being decoded is not an error.
- The seek bar of the player bar and of the full-screen player draws the shape
and asks again every few seconds until it is there. Until then, and for a
friend's track, which has none, the bar is flat. The heard part is teal, or
the friend colour. It is still a native range input, 44 pixels high in the
bar.
- OpenAPI path and schema, contract test, tests with real MP3 and FLAC files,
and a real server checked: a 4-minute MP3 is ready in about a second and a
half, a 12 MB FLAC in three, and asking again takes a millisecond.
A friend's server can now read what an admin chose to share.
- A library has a shared flag, off by default (migration 0024), set with
PATCH /libraries/{id}. Changing it never waits for a scan.
- Requests between friends are signed with each server's Ed25519 key: the
method, the path and query from /api/v1/peer on, a date, a nonce and the
hash of the empty body. A date more than two minutes off, a nonce used
before, an unknown or removed key and a bad signature all get the same
401. A friend still waiting for approval gets 403, and while federation is
off the routes are missing.
- Routes under /api/v1/peer: info, artists, albums, tracks, covers and
streaming with Range. The catalogue, the counts and the format come from
the shared libraries only, and the file served is the one in a shared
library. No path, library id, listen, taste, user or provider data is part
of an answer.
- A friend never makes this server build a cover, read an audio file or ask
a provider: only covers already built from the audio files are served.
- Per-friend limits, set by the admin: parallel streams (default 3) and KiB
a second (default 4096, 0 for none), shared by all the streams of a friend.
- A log of the last requests friends made (route, status, bytes; never the
query), kept for 7 days, in GET /federation/requests.
- POST /federation/friends/{id}/check lets the side that pasted an invite
find out that the other admin approved; a signed call from a friend this
server is waiting for makes it active too.
- Admin: a Shared with friends switch on each library, which asks first, and
limits, recent requests and a Check button in the Federation section, in
English and Italian.
- OpenAPI paths and schemas, contract test, two real servers checked end to
end with tagged audio files.
Two servers can become friends. Federation is off by default; turning it
on makes an Ed25519 key pair, kept in the database and never returned.
- Admin routes: GET and PUT /federation (on or off, name, public address),
POST /federation/invites, POST /federation/friends (use an invite),
POST /federation/friends/{id}/approve and DELETE /federation/friends/{id}.
- POST /api/v1/peer/pair, the one route without a login: the code of an
invite is the proof. An invite works once and expires in 24 hours; only
a hash of its code is stored. At most 10 attempts a minute.
- The server that uses an invite checks that the key it gets back is the
one named in the invite. A friend is kept as incoming or outgoing until
both admins have agreed.
- https only, except plain http on a network of one's own (private,
loopback or link-local address, or a local name).
- A client of its own for friends, with no redirects, no proxy from the
environment, a size limit and a time limit; the providers' client stays
as it is.
- Migration 0023 for SQLite and PostgreSQL, OpenAPI paths and schemas,
contract test with two real servers.
- Admin > Federation in the web app, in English and Italian, and a
section in the README.
- GET /smart-playlists/{id}/covers and GET /smart-playlists/suggested/{key}/covers say which albums stand for a smart playlist: up to 4 albums that have a cover, in the order of the songs the rules give now (for the variant asked), each once. They are worked out apart from the songs, and the answer may be kept for a few minutes.
- The cards of the Smart playlists page are a grid with the collage. The list shows at once and each card asks for its own covers after that, showing the rings until they arrive or when they cannot be read.
- The pages of a smart playlist and of a suggested one use the same covers for the variant on screen, so only albums that have a cover are shown.
- A playlist says which albums stand for it (album_ids): up to 4 albums that have a cover, in the order they first appear in it, each once, read for a whole page of playlists in one query. An empty one has an empty list.
- The Playlists page is a grid of cards with the collage, and the page of a playlist shows it in its header. The pages of a smart playlist and of a suggested one show a collage of the albums of their first songs.
- Nothing is fetched for it: the covers are the ones the library already has.
- Every question asked of Claude is recorded: when, the kind of list, the model, the tokens read and written (also for an answer that was no name but was billed) and how it ended: a name made, a name taken from the kept ones, or a failure and why. Nothing of the facts that were sent is recorded. Records older than a year are removed (migration 0022).
- GET /providers/anthropic/usage (admin) gives the totals of the last days (1 to 365, 30 by default), a row for each day with calls and an estimate of the cost in dollars, worked out from the tokens and the price of the model. It says it is an estimate and flags calls to a model whose price is not known.
- A Costs section in the admin area shows the last 7, 30 or 90 days.
- A smart playlist rule, plays_within_days, that counts only the plays of the last N days where plays are counted: the minimum of plays and the most and least played orders. The other listening rules are not changed.
- Two more suggested smart playlists on it: on-rotation (the most played of the last 30 days, at least twice) and top-year (the same over 365 days). most-played stays as the plays of all time.
- The editor has the new rule and the page says it in words. Texts in English and Italian.
- A pipeline over the facts about a list of songs (up to 5 genres, up to 8 artist names, how many songs, a few words about the rules of a smart playlist, the language). A local namer builds a name from the genres and the artists on this server and is always there; Claude is asked first only when an admin turned the new anthropic provider on with an API key. A failure or a refusal gives the local name. What Claude answers is kept under a hash of the facts, so the same question is not asked or paid twice.
- The anthropic provider: a write-only key (kept as typed, it is case-sensitive), a test that reads one model and sends no text, and a plain statement of what is sent. It uses the official Go SDK with nothing taken from the environment of the server (no key, profile, base URL or extra headers), no redirects, no system headers, and an answer that is cleaned and bounded.
- POST /playlists/{id}/suggest-name, POST /mixes/{id}/suggest-name and POST /smart-playlists/{id}/suggest-name answer with a name, a description and where it came from.
- The server says which albums stand for a mix (album_ids): up to 4 albums that have a cover, by different artists, with most of the songs the mix draws from. The list gives them for every mix and a mix gives its own.
- The Mixes page shows the collage on each card (a 2 by 2 collage with four covers, the first one alone with fewer, the rings with none), and the page of a mix shows it in its header.
- Nothing is fetched for it: a mix has a picture once the covers of its albums were looked up.
A smart playlist is a set of rules, an order and a limit, kept under a name and worked out again every time it is opened.
- Rules (all that are set must hold): genres, artists and albums (is, or is not), file formats and a minimum bitrate of the same file, added in the last N days, never played, played or not played in the last N days, played at least N times, skipped or not skipped lately, only liked artists or without the avoided ones, and the length of the song.
- Orders: random (the same all day, a variant gives another), most played, least played, recently added, by title. A limit of 50 by default, 500 at most.
- GET/POST /smart-playlists, GET/PATCH/DELETE /smart-playlists/{id}, GET /smart-playlists/{id}/tracks, POST /smart-playlists/preview to try rules without keeping them, and POST /smart-playlists/{id}/playlist to keep what is shown as a normal playlist.
- Only the rules are stored (migration 0020); the songs come from the files and from the owner's own history and taste. Nothing is sent to a provider, and a smart playlist is its owner's alone. A user can have 200.
- Mixes and smart playlists share the way a list of songs becomes a playlist.
GET /mixes lists the mixes a listener has, GET /mixes/{id}?variant= gives one with its
songs, and POST /mixes/{id}/playlist keeps one as a playlist of their own.
- Genre mixes: one for each genre with at least 30 songs by 4 artists, the genres the
listener plays most first. Spellings that differ by case or punctuation are one
genre (jazz+funk, jazz-funk); other and unknown are not genres.
- Made for the listener: Your favourites, Rediscover (liked artists not played for a
month; both need history) and New to you (songs never played).
- A mix has up to 50 songs, at most 3 by one artist, never two of an artist in a row,
picked by weighted chance (artist preference, never-played, less if played lately;
skipped lately and avoided artists are out). The seed is user, mix, day and variant,
so a mix is steady for a day and another is one variant away.
- Worked out when asked, from the listener's own history and the genre tags; nothing is
stored and nothing leaves the server.
internal/mixes holds the selection and the service, internal/storage/mixes.go the
queries (SQLite and PostgreSQL). Spec, contract test, README and web types follow;
the web app comes separately.
Closes#95
playlist_items.track_id is ON DELETE CASCADE (added by the playlists work on main), so deleting a
duplicate would have taken the song out of every user's playlists, as it would have done with the
listens. RemoveCopy now repoints the items first, in the same transaction.
Also merges main (playlists) into the branch.
Each user can make playlists: a name (1 to 200 characters), an optional
description and an ordered list of tracks, where a track may appear more than once
(each place has its own item id). The API makes, lists, renames and deletes
playlists, and adds tracks (at the end or at a position), lists them, moves one and
removes one; positions count from 0 without gaps.
A playlist is its owner's alone: somebody else's answers playlist_not_found (404),
as one that does not exist. A track whose files were deleted or moved stays in the
playlist and is listed with available false. Deleting a user deletes their
playlists. Limits: 1,000 playlists per user, 10,000 tracks per playlist, 500 tracks
per request.
Changes of one playlist are serialised by locking its row, so two requests at once
keep the positions unique on PostgreSQL too. Migration 0019, OpenAPI spec, contract
test, README and the web API types follow. The web app comes separately.
Closes#89
Candidates are the playable files whose title another file shares. A fingerprint is kept only for the file that was read (same size and date), and a scan that finds the file changed forgets it. Removing a copy moves its listens to the track that stays, because they would be deleted with the track, then removes the track with its file, credits and genres. Tested on SQLite and PostgreSQL.
The covers asked the Cover Art Archive only for albums whose files carried a
release group id. In a library tagged without MusicBrainz, 3,321 of 3,433 albums
have none, so almost no cover could come from the archive.
An album with no id is now looked up in MusicBrainz by the name of its artist and
its title (musicbrainz.Client.FindReleaseGroup), when the musicbrainz provider is
on too. A result is taken only if MusicBrainz scores it 95 or more and its title
and one name of its artist credit equal the ones asked for after folding case,
accents and punctuation; the first such result wins and anything doubtful is no
cover. The id is used for that request and is not stored on the album.
Artists are still never looked up by name. The MusicBrainz and Cover Art Archive
provider texts, the README and the API spec say what is sent.
On 47 albums sampled from a real library, 23 got a cover (all checked as the right
ones by eye or by exact title and artist); a title like 'Live (disc 2)' finds nothing.
Closes#83
Registration, login and the admin's create-user call take `email` instead of
`username`, and the user object returns `email`. The address is checked for its
shape (a plain address, no display name, at most 254 bytes), trimmed, lowercased
and unique. Sonda sends no mail and does not verify it.
- Migration 0018 renames the users column (and the PostgreSQL index); values are
kept, so an account made before keeps its plain name and still logs in with it
(login does not check the shape).
- Error code username_taken becomes email_taken; the JWT claim is now `email`,
so access tokens issued before the upgrade are refused once and clients refresh.
- OpenAPI spec, README, the web login form (type email), strings (en, it) and
the generated API types follow.
Closes#65
The refresh at start fixed each stale row with its own UPDATE. On PostgreSQL that
is one round trip per row: 100,000 stale rows took 5.3 s on localhost, and more
over a network. The rows now go 500 at a time in one UPDATE ... FROM (VALUES ...):
1.9 s for the same rows. SQLite keeps one UPDATE per row, which is faster there
(no round trip to save). A run with nothing to fix is unchanged (about 0.1 s per
100,000 rows).
A test with 1,250 stale rows covers more than two batches on both databases.
Lidarr is a server of the user's own that looks for and downloads music. Sonda
never downloads; this is the first piece of letting an admin send it an artist.
- The `lidarr` provider, off by default: an address, an API key (32 hexadecimal
characters, never returned) and, chosen once, a quality profile, a metadata
profile, a root folder, the monitor option and whether to search for missing
albums. It cannot be enabled without an address and a key.
- The address is checked: http or https, no user and password, no query, no
fragment, no closing slash.
- A small client of its own, apart from the outbound one (which only makes plain
GETs): the key travels in a header, no redirect is ever followed, answers and
time are bounded, and the address and key are read for every request.
Status, options, artist by MusicBrainz id and add (lookup, then the add that
Lidarr's own screen sends).
- `GET /providers/lidarr/options` lists the profiles and root folders; the test of
the provider checks the address and the key (`no_address` when it has none).
- Migration 0017 adds the address and the options to the provider settings.
- OpenAPI, contract test and README say what is sent and where.
- A search text that has nothing left once its accents are taken off matches
nothing instead of everything.
- The search key folds compatibility forms (the ligature "fi", full-width
letters) and the Greek final sigma.
- The start-up refresh of the keys logs when it starts and how long it took, since
it can take a while on a big library the first time.
- More tests: a retitled track is found by its new title, a refresh fixes both
keys and keeps "Beyoncé" and "Beyonce" apart, more cases for the key.
- A search key apart from the key that identifies artists, albums and tracks:
lower case, accents taken off (Latin, Greek and Cyrillic marks only, so kana
and Hangul stay intact), and letters like ß, æ, ø, ł written plain.
- GET /artists, /albums and /tracks look in it, so "bjork" finds "Björk" and
"strasse" finds "Straße".
- "Beyoncé" and "Beyonce" stay two artists: the identity key keeps accents.
- Migration 0016 adds the column on SQLite and PostgreSQL; RefreshMatchKeys
fills it for existing rows at start.
- golang.org/x/text is now a direct dependency.
Add one engine, internal/jobs, that runs the jobs which go through the whole
library: start, stop, progress, one at a time, give up after five failures in
a row, end at once when a provider is off.
- The covers job now runs on the engine; its API is unchanged.
- POST/GET/DELETE /library/artist-images looks for the picture of every
artist of the library that has a MusicBrainz id.
- POST/GET/DELETE /library/descriptions does the same for the Wikipedia
description.
- ?refresh=true searches everything again and never replaces a stored result
by a failure; artists without an id are never touched, nothing is looked up
by name.
- images.Service.EnsureArtistImage and about.Service.Ensure do the work for one
artist; the lists come from ImageRepo and DiscoveryRepo.
- jobs.Runner binds a job to the application context, replacing the two
hand-written controls.
- OpenAPI, contract test and README describe the new endpoints and say that
they are bulk requests.
- POST /library/covers (admin) starts, in the background and one at a time like a scan,
a job that builds the cover of every album that has something to play and no cover that
is still good; GET /library/covers shows the progress and how the last job ended;
DELETE /library/covers stops it. ?refresh=true searches every album again.
- For each album it does what opening its cover does: the user's own files first, then the
Cover Art Archive if the provider is on. It gives up, and says why, when the provider
fails for five albums in a row; an album that vanished is skipped, not a failure.
- This is the one request that goes through the whole library, and the README and the
provider text say so: with the archive on it sends the release group id of each album with
no picture in its files, one after the other, spaced out.
- NewRouter now refuses to build when a service was not wired, naming it, instead of
panicking at the first request (the wiring of this job in main had been missed, which no
test could see).
- The outbound spacing test measures where the request leaves the client, so a busy
machine cannot make it fail.
- Spec, contract and HTTP tests, README updated.
Closes#45
- The covers that come from the Cover Art Archive now live in a folder of their own
(images/album-remote). Taking the provider's data back deletes that folder, so nothing
from the archive is left whatever became of its records: a rescan makes a record stale
and a later search that finds nothing replaces it, and neither of those used to remove
the files. A cover that moves between the user's files and the archive, or goes away,
no longer leaves the old files behind.
- A redirect to a place that is not allowed is an outage (502, not remembered, an older
cover served), not "no cover" for a day for every album. A picture over the limit is
still "no cover".
- A track that moves to another album (a fixed title) leaves the first with no track: it
gives up its release group id, so the album the track went to can take it. An album
that has an id keeps it; an id another album owns is not given twice.
- Waiting for a local slot is no longer reported as the provider being unreachable, and a
build whose reads were purged three times is not served with files that were never written.
- README and spec no longer say covers never contact anybody.
- Tests: the new folder and the purge after a rescan, files left by a replaced cover, the
local wait, repeated purges, a refused redirect, the update path with a taken id and the
emptied album.
- The scan reads the MusicBrainz release group id of an album from the tags of its
files (MUSICBRAINZ_RELEASEGROUPID, a vorbis comment or an ID3 frame) and stores it in
albums.mbid: it is taken only when the file has an album and exactly one valid id, an
album keeps the one it has, and an id that another album owns is not given twice. A full
rescan reads it for music that was already indexed.
- New provider "coverartarchive" (off by default, no key, contact in the User-Agent). When
an album has no picture in the user's own files, which always come first, its cover is
asked from the archive by that id and never by a title: only the id leaves the server.
The same GET /albums/{id}/cover; the record keeps the source. An outage is a 502
provider_unavailable and is not remembered, and an older cover is served in its place.
- The archive answers with three redirects (coverartarchive.org, archive.org, a data node
whose name changes). The outbound client can now follow redirects, but only the ones the
caller allows, up to a limit; here that is https on archive.org and its subdomains.
- A purge counter now guards the covers too: a read that started before the provider's data
was taken back is not kept. Removing the provider's data forgets the covers that came from
it and deletes their files, and leaves the ones from the user's files.
- Spec, contract and HTTP tests, README updated; the facts of the real archive are written down.
Closes#43
- GET and HEAD /artists/{id}/image serve a free picture of an artist as a JPEG
(size=small 300 px or large 1000 px, token in the query allowed for an <img>), and
GET /artists/{id}/image/credit gives the author, the licence and the page of the file,
to be shown next to it.
- New provider "commons" (off by default, no key, contact in the User-Agent), which also
needs MusicBrainz on. The way to the picture uses ids only, never a name: the artist's
MusicBrainz id, the Wikidata item MusicBrainz links to it, the file Wikidata names for
it (P18), Commons for its licence, and a 960 px copy from a Wikimedia picture host.
- Only free licences: public domain, CC0, CC BY and CC BY-SA. Non-commercial,
no-derivatives, GFDL alone, fair use and unknown licences are refused, since a copy is
kept. The author is plain text, the licence address is kept only on creativecommons.org,
and the download only from https thumb.wikimedia.org or upload.wikimedia.org.
- Built on the first request, kept in the data folder: 30 days, or a week when there is no
free picture; an older copy is served when Commons cannot be reached; a failure is not
remembered. Credit changes change the ETag.
- Removing the provider's data forgets the records and deletes the files; a read that started
before it is not saved over it.
- images table gets the credit (migration 0015); the purge reports images.
- Spec, contract test, README updated; the facts of the real APIs are written down.
Closes#41
- GET /artists/{id}/description shows what Wikipedia says about an artist: a short
text with its page, a link and the licence (CC BY-SA 4.0).
- New provider "wikipedia" (off by default, no key, contact in the User-Agent, which
Wikimedia requires). The way to the text is a chain of ids, never a name: the
artist's MusicBrainz id, the Wikidata item MusicBrainz links to it, the Wikipedia
page Wikidata names in the first configured language that has one. A page is used
only if it is an article about that same item.
- [wikipedia] languages in the config (one to five codes of two or three lower case
letters, checked before they are sent: an unknown site makes Wikidata answer with
every language).
- Stored 30 days (a day when there was nothing to find), served stale if Wikipedia
fails, removed with the provider's data (the purge now reports descriptions).
- MusicBrainz client: WikidataID. Wikipedia client for Wikidata and the summary,
which also reads Wikidata's errors that come with HTTP 200.
- artist_descriptions table (migration 0014, SQLite and PostgreSQL).
- Spec, contract test, README and example config updated.
Closes#39
- At most two covers are built at the same time: a page of albums asked for at once
could decode dozens of large pictures together.
- A change to an album's files marks its record stale and moves a generation instead
of deleting the row. A build that started before the change is not saved over it
and starts again, so a "no cover" found too early cannot win for a day.
- A cover that was found is checked again after a week, so a picture replaced in a
folder is noticed.
- For CD1/Disc 2 folders the picture in the folder above is used. Folder pictures
are read one at a time, and the pixel count is computed in 64 bits.
- Stored files are synced before they get their name, an empty one counts as
missing, and old temporary files from a crash are removed.
- Files without track or disc numbers sort the same on both databases.
- storage.data_dir defaults next to the SQLite database, so a service started from
another folder does not need a writable working directory.
- Tests: a listener who leaves, a change during a build, the build limit, a named
pipe, an embedded picture of a file outside the library that exists, builds that
really overlap.
- GET and HEAD /albums/{id}/cover serve a JPEG made from the picture embedded in
the album's files, or else a cover, folder, front, album or albumart image
(jpg, jpeg, png) in the album's folder. ?size=small (300 px) or large (1000 px).
The token can be in the query, so an <img> element can use it.
- Built the first time it is asked for, scaled, and kept in storage.data_dir
(SONDA_DATA_DIR, default "data"); an album with no cover is remembered for a day.
A new or changed file of the album, or a full rescan, clears the record.
- Nothing is sent anywhere. Pictures are untrusted: files are opened inside the
library only (links are not followed), over 20 MB or 16 million pixels are
ignored, and what is served is always a JPEG made here. Stored files are 0600.
- images table (migration 0013, SQLite and PostgreSQL).
- Spec, contract test, README and example config updated.
Closes#37