Commit graph sonda/migrations/sqlite
Author SHA1 Message Date
tgiachi
7bdbc9e160 fix(images): limit parallel builds, no stale saves, safer files
- At most two covers are built at the same time: a page of albums asked for at once
  could decode dozens of large pictures together.
- A change to an album's files marks its record stale and moves a generation instead
  of deleting the row. A build that started before the change is not saved over it
  and starts again, so a "no cover" found too early cannot win for a day.
- A cover that was found is checked again after a week, so a picture replaced in a
  folder is noticed.
- For CD1/Disc 2 folders the picture in the folder above is used. Folder pictures
  are read one at a time, and the pixel count is computed in 64 bits.
- Stored files are synced before they get their name, an empty one counts as
  missing, and old temporary files from a crash are removed.
- Files without track or disc numbers sort the same on both databases.
- storage.data_dir defaults next to the SQLite database, so a service started from
  another folder does not need a writable working directory.
- Tests: a listener who leaves, a change during a build, the build limit, a named
  pipe, an embedded picture of a file outside the library that exists, builds that
  really overlap.
2026-11-11 21:03:26 +01:00
tgiachi
0e121ead8e feat(images): album covers from my own files
- GET and HEAD /albums/{id}/cover serve a JPEG made from the picture embedded in
  the album's files, or else a cover, folder, front, album or albumart image
  (jpg, jpeg, png) in the album's folder. ?size=small (300 px) or large (1000 px).
  The token can be in the query, so an <img> element can use it.
- Built the first time it is asked for, scaled, and kept in storage.data_dir
  (SONDA_DATA_DIR, default "data"); an album with no cover is remembered for a day.
  A new or changed file of the album, or a full rescan, clears the record.
- Nothing is sent anywhere. Pictures are untrusted: files are opened inside the
  library only (links are not followed), over 20 MB or 16 million pixels are
  ignored, and what is served is always a JPEG made here. Stored files are 0600.
- images table (migration 0013, SQLite and PostgreSQL).
- Spec, contract test, README and example config updated.

Closes #37
2026-11-11 20:52:36 +01:00
tgiachi
a6dbf60ac8 feat(feedback): a wishlist and "not interested"
- PUT/DELETE /wishlist/{artist} and GET /wishlist: the artists I want.
- POST/DELETE /discover/{artist}/dismiss and GET /discover/dismissed: the
  artists I am not interested in. An artist is wished or dismissed, never both,
  and both lists are mine alone.
- Discover never suggests a dismissed artist, does not use one as a seed taken
  from my listening, and marks the wished ones it shows (wishlisted).
- New artist_feedback table (migration 0012, SQLite and PostgreSQL), removed
  with the account.
- An artist I marked is not cleaned up when the list that named it is replaced,
  and goes when its last mark is taken off and nobody names it. Taking a
  provider's data back still removes what only it named, marks included.
- Spec, contract test and README updated.

Closes #35
2026-11-11 20:11:18 +01:00
tgiachi
e0592c0260 feat(settings): my own settings, starting with Discover from my listening
- GET and PATCH /me/settings: settings that belong to the signed-in user.
- First setting, discover_from_listening, is off by default. When on, Discover
  may start from the artists I listen to most, which sends their names to
  Last.fm.
- New user_settings table (migration 0011, SQLite and PostgreSQL), removed with
  the account. A user with no row has the private defaults.
- A misspelt setting is rejected instead of being ignored.
- Spec, contract test and README updated.

Refs #29
2026-11-11 19:16:26 +01:00
tgiachi
1857bb9c53 feat(discovery): remove provider data, genres from my files, full rescans
- feat: DELETE /api/v1/providers/{name}/data (admin) removes everything a
  provider gave: its lists of similar artists, its tags and the records of
  when it was asked, then the artists that only that data named and that
  nobody has files for. Last.fm's terms ask for this when an agreement ends.
  The user's own artists, music, listens and settings stay, the provider is not
  switched off, and a second call removes nothing
- feat: a discovered artist that no list names any more is removed when the
  list that named it is replaced, together with its tags, so they do not pile
  up. Real artists, and any artist that another list still names, are kept
- feat: the scan reads the GENRE tag of each file, offline (several genres
  separated by ";", lower case, at most five per file), and
  GET /artists/{id}/tags gets a third entry, files: the genres of the artist's
  own playable tracks, each weighted by how many tracks have it. It asks nobody
  and is always there
- feat: POST /library/scan and POST /libraries/{id}/scan take full=true to read
  every file again, not only the new or changed ones, so MusicBrainz ids and
  genres reach music that was indexed before. Files keep their identity, so
  listening history is kept. Anything but true or false is a 400
- test: the plan of a full scan, the scanner and the manager; the purge on both
  databases, checked by removing the rule that spares real artists; the
  cleanup of replaced lists; genres per file and per artist
- docs: README, OpenAPI and the todo list updated
2026-11-11 17:27:29 +01:00
tgiachi
07ea997cd0 feat(discovery): artist tags from Last.fm and MusicBrainz, and ids from my files
- feat: GET /api/v1/artists/{id}/tags lists, for each provider that is on, the
  tags it gives to an artist with its own weights, kept as they came: Last.fm's
  are relative to the artist, MusicBrainz's are counts of votes, and they are
  never mixed. MusicBrainz also marks the tags it lists as genres. The same
  rules as similar artists apply: asked only on demand, kept 7 days (1 for an
  unknown artist), old data served when a provider fails, and one call for
  simultaneous requests. A provider that is off, failing or lacking an id does
  not hide the other
- feat: a thin MusicBrainz client that reads an artist's tags and genres by its
  id. It never searches by name, because a search returns look-alikes and
  picking one would be a guess. The id is checked before it reaches the path,
  and errors never repeat the provider's text
- feat: the User-Agent can change while the server runs, so the contact an
  admin sets for MusicBrainz applies to the next request; if the settings
  cannot be read it falls back to the project address
- feat: Last.fm top tags, by id when known and by name otherwise, never
  autocorrected
- feat: the scan reads MusicBrainz artist ids from the file tags, offline, and
  keeps them on the artists. An id is only used when it can be paired with its
  artist for certain: the same number of ids as artists, all valid. The same id
  is the same artist whatever the spelling, an artist without an id takes the
  one its file gives, and another id under the same name is another artist. A
  file that confirms the id a provider gave to a discovered artist keeps it
- feat: POST /providers/musicbrainz/test checks that the service answers
- test: what leaves the server for MusicBrainz is checked from the other end:
  the id of the artist and the admin's contact, nothing from the library;
  removing the safeguards on ids and pairing makes tests fail
- docs: README, OpenAPI and the provider text updated
2026-11-11 17:01:43 +01:00
tgiachi
b769ad7ac5 fix(discovery): concurrent writes, adopted artists and a client that leaves
- fix: on SQLite a transaction that read and then wrote failed at once with
  "database is locked" when another writer committed in between, because the
  busy timeout does not apply there. Transactions now start with BEGIN
  IMMEDIATE. Before this change a scan running while someone opened similar
  artists made files fail, and parallel requests stored nothing
- fix: on PostgreSQL two requests naming the same new artist could fail on the
  unique MusicBrainz id or leave two rows with one name. Creating artists, in
  the discovery store and in the scanner, now takes turns through an advisory
  lock and looks again once it has the turn
- fix: music arriving for an artist that a provider only named used to keep the
  provider's spelling and its unchecked MusicBrainz id, which later lookups
  would then send instead of the name. Such artists are now marked
  (artists.discovered); the first scan that finds them renames them to the
  spelling of the file tags, drops the id and clears the mark
- fix: a panic in the shared provider call is turned into a failure; the
  singleflight group would have re-raised it where nothing can recover
- fix: a client that leaves while Last.fm is being asked no longer gets a 500
  and an error in the log
- fix: no page can ask for more than is stored (limit tops out at 50), and the
  docs no longer say the best score is always 1
- test: concurrent saves plus a scan on both databases, checked by removing each
  fix; adoption keeps neither the name nor the id; the panic; the client that
  leaves
2026-11-11 16:47:33 +01:00
tgiachi
aafb2dd906 feat(discovery): similar artists from Last.fm
- feat: GET /api/v1/artists/{id}/similar lists the artists Last.fm finds similar
  to one in the library, best first, each with the provider's own score and
  rank and a flag saying whether it is in the library. It answers 200 when the
  provider is off or failing, with provider_status, stale and retrieved_at
- feat: a thin Last.fm client on top of the outbound client. It always sends
  autocorrect=0 so a misspelled local name cannot turn into another artist,
  prefers the MusicBrainz id when known, always sets a limit, and reads the
  error field even when the status is 200. Errors never carry the URL or the
  provider's own message, because the key travels in the query string
- feat: results are stored in artist_similar with a freshness ledger in
  provider_fetches: 7 days for an answer, 1 day for "unknown artist". Old data
  is served when the provider fails, and concurrent requests for the same
  artist share one call that does not depend on the first caller staying
- feat: artists named by a provider but without files become artists without
  tracks. GET /artists/{id} can show them (track_count 0), the lists of the
  library never include them, and a later scan adopts them by name. A provider
  id is never written onto an existing artist; names match only when at least
  one side has no id, and different ids always mean different artists
- feat: POST /providers/{name}/test (admin) checks a stored key with one cheap
  request, whether the provider is on or off, and reports ok, no_api_key,
  key_refused, rate_limited, unreachable or error
- test: the whole request is checked from the receiving end, with a user, a
  listen, a title and a library path in the database: only the artist and the
  key leave, with a User-Agent that names the project
- docs: README and OpenAPI updated; the contract test calls the new operations
2026-11-11 16:42:52 +01:00
tgiachi
9f4da2e77e feat(providers): provider settings in a table and a careful outbound HTTP client
- feat(storage): provider_settings table (provider, enabled, api_key, contact);
  migration 0007 for both drivers. A provider with no row, or turned off, is
  never contacted
- feat(providers): a service that validates and applies settings: a Last.fm key
  is 32 hexadecimal characters, enabling a provider that needs a key without
  one is refused, clearing the key turns the provider off, a contact cannot
  contain parentheses or control characters. Each provider states what it sends
  and under which terms, and what it never sends
- feat(api): admin-only GET /providers and PUT /providers/{name}. The API key is
  write-only: responses carry only has_api_key, errors never repeat it and the
  request log never contains it (tested)
- feat(providers/outbound): the only way Sonda talks to the outside: plain GET
  with an identifiable User-Agent, no cookies, no Referer, no Authorization, no
  redirects, bounded answers, a timeout per attempt, a minimum interval between
  requests, retries with exponential backoff that honour Retry-After without
  waiting past the caller's deadline or for an absurd time, and a circuit
  breaker that makes a failing provider fail fast
- feat(providers): UserAgent builds "Sonda/version ( contact )" as MusicBrainz
  asks, defaulting to the project address so no personal address is sent
- breaking(config): providers.lastfm and SONDA_LASTFM_API_KEY are gone; an old
  config, or the variable, fails with a hint instead of being ignored
- docs: OpenAPI operations and schemas, README section on the providers and on
  what leaves the server
- test: fake servers and a fake clock only; no network in the suite
2026-11-11 16:07:12 +01:00
tgiachi
e14c523828 feat(catalog): add the read API for artists, albums and tracks
- feat(api): GET /api/v1/artists, /albums and /tracks with q (case and accent
  insensitive substring), limit and offset (default 50, at most 200) and a
  total; filters library_id, artist_id and album_id; GET /artists/{id},
  /albums/{id} and /tracks/{id}; any signed-in user can read
- feat(catalog): only tracks with a file that is not flagged missing are
  visible, so everything shown can be played; a track with several files is
  listed once; album artists without credited tracks (compilations) are listed
- feat(storage): results are ordered the same way on SQLite and PostgreSQL
  (plain byte order), LIKE wildcards in the search text are escaped, and a
  page costs a fixed number of queries: related items and counts are loaded in
  batches
- feat(storage): tracks get a title_key computed in Go (migration 0006, with
  a startup refresh), like artists and albums
- test: ordering, paging, search with accents and wildcards, library filter,
  duplicate encodings, missing files, details, and a query-count guard on
  both databases
2026-11-11 13:43:38 +01:00
tgiachi
eb23ee426a fix(library): make the libraries upgrade and validation robust
- fix(migration): roots that differ only in case (/Music and /music are two
  directories on Linux) used to make migration 0005 fail on the unique name;
  the later ones now get a numeric suffix
- feat(library): at startup, log a warning for every library whose directory
  cannot be read or that overlaps another, because migrated rows skip
  validation and may still hold roots that were removed from the old config
- fix(library): creating or updating a library is now serialised, so two
  concurrent requests cannot both pass the overlap check
- fix(library): changing a library's directory or deleting it is refused with
  409 scan_running while a scan runs; the scan works from the list it started
  with and would write under a stale path
- fix(library): library names clash ignoring case and accents the same way on
  SQLite and PostgreSQL, checked in Go
- fix(library): changing only the path no longer revalidates an untouched
  name, which can be longer than the limit after a migration
- test: roots differing in case on both databases; concurrent creates with a
  slow store, verified to fail without the lock; the busy guard; Problems
2026-11-11 13:34:44 +01:00
tgiachi
46463846b3 feat(library): named media libraries managed through the API
- feat(storage): libraries table (name, path); audio_files points at a library
  by id instead of repeating the root path; migration 0005 for both drivers
  turns every existing root into a library and keeps file ids, tracks and the
  missing flag
- feat(library): Libraries service validates names and paths: absolute, an
  existing directory, never the filesystem root, never inside or around
  another library (checked on real paths, so symlinks cannot hide an overlap)
- feat(api): admin-only GET/POST /libraries, PATCH/DELETE /libraries/{id} and
  POST /libraries/{id}/scan; deleting a library drops its index entries and
  keeps tracks and listening history
- feat(library): the scanner and the scan manager work on libraries; scanning
  with no libraries answers 409 no_libraries
- feat(streaming): the library directory comes from the database instead of
  a configured list of roots
- breaking(config): library.roots is removed; an old config fails with a hint
  to use POST /api/v1/libraries
- test: CRUD and duplicates on both databases, path change keeps files,
  upgrade of a database written before libraries existed
2026-11-11 13:30:25 +01:00
tgiachi
dbe9338af3 feat(auth): add refresh tokens, admin role and user management
- feat(auth): short-lived access tokens plus rotating refresh tokens stored as
  SHA-256 hashes; replaying a spent token revokes every session of that user
- feat(auth): is_admin flag; the first registered user is the admin and the
  last admin can be neither demoted nor deleted
- feat(api): POST /auth/refresh, /auth/logout, /auth/password and GET /auth/me
- feat(api): admin-only GET/POST /users, PATCH /users/{id}, PUT
  /users/{id}/password and DELETE /users/{id}; the admin check reads the role
  from the database on every call
- feat(api): starting a library scan is now admin-only
- feat(storage): users and refresh token repositories for SQLite and
  PostgreSQL (migration 0004; the oldest existing account becomes admin)
- feat(config): auth.access_ttl_minutes and auth.refresh_ttl_hours replace
  auth.token_ttl_hours
- test: rotation, reuse detection, concurrent rotation with a single winner,
  admin rules, role changes taking effect immediately
2026-11-11 12:38:42 +01:00
tgiachi
d6e9e09ce7 fix(library): harden the scanner against special files, stat errors and lossy logging
- fix: skip FIFOs, sockets and devices named like audio files; opening a named
  pipe blocked a scan forever and left the manager stuck in "running"
- fix: a file that cannot be stat'ed for a transient reason is unknown, not
  gone, so it is never marked missing or used as a rename source
- fix: log the scan result from Manager.OnFinish instead of the lossy event
  hub, so the line can no longer be dropped
- fix(storage): match artists and albums on a Unicode-aware key computed in Go
  (migration 0003, both drivers), so SQLite and PostgreSQL agree
- test: FIFO, stat errors, OnFinish, Unicode artist and album matching
2026-11-11 12:28:44 +01:00
tgiachi
3a739dc3ae feat: add sondad server with config, storage, auth and listening API
- feat(config): TOML config with env overrides, strict parsing and validation
- feat(storage): SQLite (modernc) and PostgreSQL (pgx) backends through bun,
  embedded per-driver SQL migrations with a transactional runner
- feat(schema): catalog tables (artists, albums, tracks, audio_files with
  per-root paths), users and listening events with idempotency keys
- feat(auth): Argon2id password hashing, HS256 JWT tokens, first-user-only
  registration and login
- feat(api): gin REST API under /api/v1 (health, register, login, listens)
  with a consistent error envelope, body limit and request logging
- feat(sondad): zap logger, graceful shutdown, composition root in main
- chore: example config, .gitignore for secrets, DB files, IDE and agent files
2026-11-11 12:06:15 +01:00