feat(mixes): playlists made for a listener, by listening and by genre

GET /mixes lists the mixes a listener has, GET /mixes/{id}?variant= gives one with its
songs, and POST /mixes/{id}/playlist keeps one as a playlist of their own.

- Genre mixes: one for each genre with at least 30 songs by 4 artists, the genres the
  listener plays most first. Spellings that differ by case or punctuation are one
  genre (jazz+funk, jazz-funk); other and unknown are not genres.
- Made for the listener: Your favourites, Rediscover (liked artists not played for a
  month; both need history) and New to you (songs never played).
- A mix has up to 50 songs, at most 3 by one artist, never two of an artist in a row,
  picked by weighted chance (artist preference, never-played, less if played lately;
  skipped lately and avoided artists are out). The seed is user, mix, day and variant,
  so a mix is steady for a day and another is one variant away.
- Worked out when asked, from the listener's own history and the genre tags; nothing is
  stored and nothing leaves the server.

internal/mixes holds the selection and the service, internal/storage/mixes.go the
queries (SQLite and PostgreSQL). Spec, contract test, README and web types follow;
the web app comes separately.

Closes #95
This commit is contained in:
tgiachi 2026-11-12 17:25:49 +01:00
commit 94dbf57544
16 changed files with 2829 additions and 0 deletions

View file

@ -21,6 +21,7 @@ It is a single executable, one database (SQLite by default, PostgreSQL if you pr
- **Artist descriptions.** `GET /artists/{id}/description` shows what Wikipedia says about an artist: a short text, with the page it comes from, a link to it and the licence (CC BY-SA 4.0), which you must show next to it. With the `wikipedia` provider and the `musicbrainz` one switched on (the link to Wikidata comes from MusicBrainz), Sonda follows ids and never a name: the artist's MusicBrainz id, then the Wikidata item that MusicBrainz links to it, then the Wikipedia page that Wikidata names in the first of your languages (`[wikipedia] languages`, the code of that Wikipedia, for example `["it", "en"]`) that has one. An artist without a MusicBrainz id has no description. It is kept for 30 days and removed with the provider's data. To read the descriptions of the whole library in one go, an admin starts a background job like a scan: `POST /library/descriptions` (follow it with `GET /library/descriptions`, stop it with `DELETE /library/descriptions`; `?refresh=true` reads every artist again). It only goes through artists that have a MusicBrainz id, never replaces a description by a failure, and ends at once, saying so, when a provider it needs is off. See [External providers](#external-providers).
- **Artist photos.** `GET /artists/{id}/image` serves a free picture of an artist from Wikimedia Commons, and `GET /artists/{id}/image/credit` says whose it is and under which licence, which you must show next to it. Only free licences are kept (public domain, CC0, CC BY, CC BY-SA): non-commercial, no-derivatives, fair-use and unknown ones are refused, because Sonda stores a copy (`?size=small` is 300 pixels, `large` 1000) in the data folder. With the `commons` provider and the `musicbrainz` one switched on, Sonda follows ids and never a name: the artist's MusicBrainz id, the Wikidata item MusicBrainz links to it, the picture Wikidata names for it, and Commons for its licence. Many artists will have none. Kept for 30 days (a week when there is none) and removed, files included, with the provider's data. To look for the pictures of the whole library in one go, an admin starts a background job like a scan: `POST /library/artist-images` (follow it with `GET /library/artist-images`, stop it with `DELETE /library/artist-images`; `?refresh=true` searches every artist again). It only goes through artists that have a MusicBrainz id, never replaces a picture by a failure, and ends at once, saying so, when a provider it needs is off. See [External providers](#external-providers).
- **Album covers.** `GET /albums/{id}/cover` serves the cover of an album from your own files: the picture embedded in them, or else a `cover`, `folder`, `front`, `album` or `albumart` image in the album's folder. It is built the first time it is asked for (a few at a time), scaled (`?size=small` is 300 pixels, `large` 1000) and kept in the data folder (`storage.data_dir`, you can delete it: covers are built again). A picture you replace in a folder is noticed within a week, or at once after a full rescan. When your files have no picture at all and an admin switched on the `coverartarchive` provider, the cover is asked from the Cover Art Archive by the MusicBrainz release group id that the tags of the album's files carry (`MUSICBRAINZ_RELEASEGROUPID`; run a full rescan to read it for music you already indexed); your own files always come first, and only that id leaves the server. An album whose files have no id is looked up in MusicBrainz by its artist and title when the `musicbrainz` provider is on too (see the list of what leaves your server). For an album split in `CD1`/`Disc 2` folders, the picture in the folder above is used. Nothing is sent anywhere unless you switched on the `coverartarchive` provider (see [External providers](#external-providers)). It also takes `?access_token=`, so an `<img>` element can use it. To build the covers of the whole library in one go instead of one album at a time, an admin starts a background job like a scan: `POST /library/covers` (follow it with `GET /library/covers`, stop it with `DELETE /library/covers`; `?refresh=true` searches every album again; it never replaces a cover by "no cover" because a disk was not mounted, and keeps the covers that came from the archive if you switched it off).
- **Mixes.** `GET /mixes` lists playlists that Sonda makes for you, like the mixes of a streaming service, from what you listen to and from the genres of your music: one mix for each genre of your library that has at least 30 songs by 4 artists (the genres you play most first), and, once you have some listening history, *Your favourites* and *Rediscover* (artists you like and have not played for a month); *New to you* is the songs you never played. `GET /mixes/{id}` gives a mix of up to 50 songs, at most 3 by one artist and never two of the same artist in a row. They are picked by weighted chance: the artists you play weigh more and songs you skipped lately or artists you avoid are left out. A mix stays the same for a day, and `?variant=2` gives another; `POST /mixes/{id}/playlist` keeps the one you are looking at as a playlist of your own. A mix is worked out when you ask for it, from your own history and the genre tags of your files: nothing is stored and nothing is sent to a provider. The id of a genre mix has the genre in it (`genre:hard rock`), so it is percent-encoded in the path.
- **Playlists.** `POST /playlists` makes a playlist (a name and an optional description) and `GET /playlists` lists yours, the one changed last first. `POST /playlists/{id}/tracks` adds 1 to 500 tracks at the end or at a `position`, `GET /playlists/{id}/tracks` lists them in order, `PATCH /playlists/{id}/tracks/{item}` moves one and `DELETE` takes one out; `PATCH /playlists/{id}` renames and `DELETE /playlists/{id}` deletes (the tracks stay in the library). A track can be in a playlist more than once, which is why each place has its own item id. A playlist is yours alone: someone else's answers `playlist_not_found`, as one that does not exist. A track whose files were deleted or moved away stays in the playlist and shows `available: false`; it plays again when the files come back and a scan finds them. A user can have 1000 playlists of up to 10000 tracks. Nothing is shared and nothing leaves your server.
- **Wishlist and "not interested".** `PUT /wishlist/{artist}` puts an artist on your wishlist (`GET /wishlist` lists it, `DELETE` takes it off), and `POST /discover/{artist}/dismiss` hides an artist from your Discover for good (`DELETE` brings it back, `GET /discover/dismissed` lists them). An artist is wished or dismissed, never both, and both lists are yours alone. Discover marks the wished artists it shows (`wishlisted`) and never suggests a dismissed one. The wishlist is a list for you and sends nothing anywhere; Sonda downloads nothing, but an admin can send an artist that Discover showed to their own Lidarr (see [External providers](#external-providers)), which works for an artist that has a MusicBrainz id and that Sonda knows of from a list or from your files. An artist that only Last.fm named is kept while you have it marked (your lists still show it, but other users cannot see that it exists), and goes with the rest of Last.fm's data if an admin removes Last.fm's data. Removing MusicBrainz's data does not touch it.
- **Your own settings.** `GET` and `PATCH /me/settings` hold choices that belong to you, not to the server. Everyone starts private: `discover_from_listening` is off, so this setting never sends your listening to a provider. When you turn it on, Discover may start from the artists you listen to most, and their names are sent to Last.fm (only if an admin switched it on). What Last.fm answers is kept in the shared database like any other lookup, so another local user can see, from `retrieved_at` of `GET /artists/{id}/similar`, that someone looked up an artist, but not who.
@ -292,6 +293,7 @@ Everything lives under `/api/v1`. Bodies are JSON. Errors always look like `{"er
| signed in | `GET /listens` | Your own history, newest first, with `counted` per event. Query: `limit`, `offset`, `counted=true`. |
| signed in | `GET /discover` | Artists you may like, with a score and its parts. Query: `seed` (an artist id, up to 5 times), `limit` (default 20, max 30), `in_library=true\|false`. |
| signed in | `GET`, `HEAD /albums/{id}/cover` | The cover of an album, a JPEG made from your own files. Query: `size=small\|large`. Also accepts `?access_token=`. |
| signed in | `GET /mixes`, `GET /mixes/{id}`, `POST /mixes/{id}/playlist` | Your mixes: list them, read one (query `variant`), keep one as a playlist. Made for you alone, from your history and your genres; nothing is stored. |
| signed in | `GET /playlists`, `POST /playlists`, `GET`, `PATCH`, `DELETE /playlists/{id}` | Your playlists: list them (query `limit`, `offset`), make one, read, rename or delete one. Never anybody else's. |
| signed in | `GET /playlists/{id}/tracks`, `POST /playlists/{id}/tracks`, `PATCH`, `DELETE /playlists/{id}/tracks/{item}` | The tracks of a playlist: list them in order, add some (at the end or at a `position`), move or remove an item. |
| signed in | `GET /wishlist`, `PUT /wishlist/{id}`, `DELETE /wishlist/{id}` | Your wishlist: list it (query `limit`, `offset`), add an artist, take one off. |
@ -378,6 +380,7 @@ internal/taste/ what the listener likes, learned from the listening histo
internal/discover/ the score of an artist you may like, and the suggestions
internal/feedback/ the wishlist and the artists a user is not interested in
internal/playlists/ the playlists of each user
internal/mixes/ the mixes made for a listener, by genre and by listening
internal/images/ covers made from the pictures in the user's own files
internal/usersettings/ the settings that belong to one user
internal/storage/ SQLite and PostgreSQL through bun, the repositories

View file

@ -27,6 +27,7 @@ import (
"codeberg.org/squidd/sonda/internal/jobs"
"codeberg.org/squidd/sonda/internal/library"
"codeberg.org/squidd/sonda/internal/listening"
"codeberg.org/squidd/sonda/internal/mixes"
"codeberg.org/squidd/sonda/internal/playlists"
"codeberg.org/squidd/sonda/internal/providers"
"codeberg.org/squidd/sonda/internal/providers/commons"
@ -291,6 +292,7 @@ func run() error {
Discover: discover.NewService(tasteService, discoveries, tags, tasteRepo, userSettings, feedbacks, weights),
Feedback: feedbacks,
Playlists: playlists.NewService(storage.NewPlaylistRepo(db)),
Mixes: mixes.NewService(storage.NewMixRepo(db), tasteService, cfg.Listening.MinListen()),
Images: pictures,
CoversJob: jobs.Runner{Ctx: ctx, Job: images.NewCoversJob(pictures, imageRepo)},
ArtistImagesJob: jobs.Runner{Ctx: ctx, Job: images.NewArtistImagesJob(pictures, imageRepo)},

View file

@ -26,6 +26,7 @@ import (
"codeberg.org/squidd/sonda/internal/jobs"
"codeberg.org/squidd/sonda/internal/library"
"codeberg.org/squidd/sonda/internal/listening"
"codeberg.org/squidd/sonda/internal/mixes"
"codeberg.org/squidd/sonda/internal/playlists"
"codeberg.org/squidd/sonda/internal/providers"
"codeberg.org/squidd/sonda/internal/storage"
@ -106,6 +107,7 @@ func newTestEnvDB(t *testing.T, mutate func(*Deps, *bun.DB)) (http.Handler, *bun
feedbackSvc := feedback.NewService(storage.NewFeedbackRepo(db))
deps.Feedback = feedbackSvc
deps.Playlists = playlists.NewService(storage.NewPlaylistRepo(db))
deps.Mixes = mixes.NewService(storage.NewMixRepo(db), deps.Taste.(mixes.Taster), 20*time.Second)
imageRepo := storage.NewImageRepo(db)
imageSvc := images.NewService(filepath.Join(t.TempDir(), "data"), imageRepo, imageRepo)
deps.Images, deps.ArtistImages = imageSvc, imageSvc

View file

@ -822,6 +822,9 @@ func TestResponsesMatchTheSpec(t *testing.T) {
// ---- duplicate songs ----
c.duplicates(t, token, libID)
// ---- mixes ----
c.mixes(t, token, libID)
// ---- every operation in the spec was exercised ----
var missing []string
for _, op := range specOperations(loadSpec(t)) {
@ -979,3 +982,47 @@ func (c *contract) addOrphan(libraryID string) orphanIDs {
}
return orphanIDs{album: strconv.FormatInt(album, 10), track: strconv.FormatInt(track, 10)}
}
// mixes exercises the mixes. The library gets 40 songs of one genre by 6 artists, enough for a
// genre mix; they are added last so that nothing that counts songs sees them.
func (c *contract) mixes(t *testing.T, token, libID string) {
t.Helper()
ctx := context.Background()
repo := storage.NewLibraryRepo(c.db)
for i := 0; i < 40; i++ {
artist := "Mix artist " + strconv.Itoa(i%6)
title := "Mix song " + strconv.Itoa(i)
f := library.ParsedFile{
FileInfo: library.FileInfo{RelPath: "mixes/" + title + ".flac", Size: 100, MTimeUnix: 1},
Metadata: library.Metadata{
Tags: library.Tags{Title: title, Album: "Mix album", AlbumArtist: artist, Artists: []string{artist}, TrackNo: i + 1, DiscNo: 1, Genres: []string{"synthwave"}},
Properties: library.Properties{Format: "flac", DurationMs: 60000, SampleRate: 44100},
},
}
if _, err := repo.SaveFile(ctx, mustInt(t, libID), f, 0); err != nil {
t.Fatal(err)
}
}
c.expect(c.call("GET", "/mixes", "", nil), 401, "mixes without a token", "unauthorized")
list := c.expect(c.call("GET", "/mixes", token, nil), 200, "list mixes")
var found bool
for _, x := range list["items"].([]any) {
if x.(map[string]any)["id"] == "genre:synthwave" {
found = true
}
}
if !found {
t.Fatalf("mixes = %v, want a synthwave mix", list)
}
mix := c.expect(c.call("GET", "/mixes/genre%3Asynthwave", token, nil), 200, "get a mix")
if len(mix["tracks"].([]any)) != 40 {
t.Fatalf("mix = %v", mix)
}
c.expect(c.call("GET", "/mixes/genre%3Asynthwave?variant=3", token, nil), 200, "get another variant")
c.expect(c.call("GET", "/mixes/genre%3Asynthwave?variant=-1", token, nil), 400, "a negative variant", "invalid_input")
c.expect(c.call("GET", "/mixes/genre%3Apolka", token, nil), 404, "an unknown mix", "mix_not_found")
c.expect(c.call("POST", "/mixes/genre%3Asynthwave/playlist", token, map[string]any{"name": "Synth", "variant": 3}), 201, "save a mix as a playlist")
c.expect(c.call("POST", "/mixes/genre%3Asynthwave/playlist", token, nil), 201, "save a mix with no body")
c.expect(c.call("POST", "/mixes/genre%3Apolka/playlist", token, nil), 404, "save an unknown mix", "mix_not_found")
c.expect(c.call("POST", "/mixes/genre%3Asynthwave/playlist", token, map[string]any{"variant": -1}), 400, "save with a negative variant", "invalid_input")
}

View file

@ -0,0 +1,152 @@
package httpapi
import (
"errors"
"net/http"
"strings"
"github.com/gin-gonic/gin"
"codeberg.org/squidd/sonda/internal/catalog"
"codeberg.org/squidd/sonda/internal/mixes"
)
// mixHandlers serve the mixes made for the signed-in user. They are worked out from that user's
// own history and the genres of the music, and never stored.
type mixHandlers struct {
svc mixService
playlists playlistService
}
type mixJSON struct {
ID string `json:"id"`
Kind string `json:"kind"`
Title string `json:"title"`
Genre *string `json:"genre"`
Artists []string `json:"artists"`
}
type mixWithTracksJSON struct {
mixJSON
Variant int `json:"variant"`
Tracks []trackJSON `json:"tracks"`
}
func toMixJSON(m mixes.Mix) mixJSON {
out := mixJSON{ID: m.ID, Kind: string(m.Kind), Title: m.Title, Artists: m.Artists}
if out.Artists == nil {
out.Artists = []string{}
}
if m.Genre != "" {
g := m.Genre
out.Genre = &g
}
return out
}
func toTracksJSON(tracks []catalog.Track) []trackJSON {
out := make([]trackJSON, len(tracks))
for i, t := range tracks {
out[i] = toTrackJSON(t)
}
return out
}
func writeMixError(c *gin.Context, err error) {
switch {
case errors.Is(err, mixes.ErrNotFound):
writeError(c, http.StatusNotFound, "mix_not_found", "no such mix")
case errors.Is(err, mixes.ErrInvalidInput):
writeError(c, http.StatusBadRequest, "invalid_input", err.Error())
default:
_ = c.Error(err)
writeError(c, http.StatusInternalServerError, "internal", "internal error")
}
}
// variantOf reads the variant of the query string: a whole number from 0 up, 0 when absent.
func variantOf(c *gin.Context) (int, bool) {
v, ok := queryInt(c, "variant")
if !ok {
return 0, false
}
if v < 0 || v > mixes.MaxVariant {
writeError(c, http.StatusBadRequest, "invalid_input", "variant must be a whole number from 0 up")
return 0, false
}
return int(v), true
}
func (h *mixHandlers) list(c *gin.Context) {
list, err := h.svc.List(c.Request.Context(), currentUserID(c))
if err != nil {
writeMixError(c, err)
return
}
out := make([]mixJSON, len(list))
for i, m := range list {
out[i] = toMixJSON(m)
}
c.JSON(http.StatusOK, gin.H{"items": out})
}
func (h *mixHandlers) get(c *gin.Context) {
variant, ok := variantOf(c)
if !ok {
return
}
m, tracks, err := h.svc.Get(c.Request.Context(), currentUserID(c), c.Param("id"), variant)
if err != nil {
writeMixError(c, err)
return
}
c.JSON(http.StatusOK, mixWithTracksJSON{mixJSON: toMixJSON(m), Variant: variant, Tracks: toTracksJSON(tracks)})
}
type saveMixRequest struct {
Name string `json:"name"` // blank: the title of the mix
Variant *int `json:"variant"` // absent: the mix of the day
}
// save makes a playlist of the songs of a mix, the ones that variant shows. A save that fails
// half way leaves no playlist behind.
func (h *mixHandlers) save(c *gin.Context) {
var req saveMixRequest
if c.Request.ContentLength != 0 && !bindJSON(c, &req) {
return
}
variant := 0
if req.Variant != nil {
variant = *req.Variant
}
ctx, user := c.Request.Context(), currentUserID(c)
m, tracks, err := h.svc.Get(ctx, user, c.Param("id"), variant)
if err != nil {
writeMixError(c, err)
return
}
if len(tracks) == 0 {
writeError(c, http.StatusUnprocessableEntity, "mix_empty", "the mix has no songs to save")
return
}
name := req.Name
if strings.TrimSpace(name) == "" {
name = m.Title
}
pl, err := h.playlists.Create(ctx, user, name, "")
if err != nil {
writePlaylistError(c, err)
return
}
ids := make([]int64, len(tracks))
for i, t := range tracks {
ids[i] = t.ID
}
filled, err := h.playlists.AddTracks(ctx, user, pl.ID, ids, nil)
if err != nil {
_ = h.playlists.Delete(ctx, user, pl.ID)
writePlaylistError(c, err)
return
}
c.JSON(http.StatusCreated, toPlaylistJSON(filled))
}

View file

@ -0,0 +1,238 @@
package httpapi
import (
"context"
"fmt"
"net/http"
"net/url"
"slices"
"strings"
"testing"
"time"
"github.com/uptrace/bun"
"codeberg.org/squidd/sonda/internal/library"
"codeberg.org/squidd/sonda/internal/listening"
"codeberg.org/squidd/sonda/internal/storage"
)
// mixAPI is a server with 40 rock songs by 8 artists and 36 hard rock songs by 6, and a user.
type mixAPI struct {
h http.Handler
db *bun.DB
alice string
}
func newMixAPI(t *testing.T) *mixAPI {
t.Helper()
h, db := newTestEnvDB(t, func(*Deps, *bun.DB) {})
alice := registerAndLogin(t, h)
ctx := context.Background()
lib := storage.NewLibraryRepo(db)
l, err := lib.CreateLibrary(ctx, "m", "/lib/m")
if err != nil {
t.Fatal(err)
}
add := func(prefix string, n, artists int, genres ...string) {
for i := 0; i < n; i++ {
artist := fmt.Sprintf("%s artist %d", prefix, i%artists)
title := fmt.Sprintf("%s song %d", prefix, i)
f := library.ParsedFile{
FileInfo: library.FileInfo{RelPath: prefix + "/" + title + ".flac", Size: 100, MTimeUnix: 1},
Metadata: library.Metadata{
Tags: library.Tags{Title: title, Album: prefix + " album", AlbumArtist: artist, Artists: []string{artist}, TrackNo: i + 1, DiscNo: 1, Genres: genres},
Properties: library.Properties{Format: "flac", DurationMs: 60000, SampleRate: 44100},
},
}
if _, err := lib.SaveFile(ctx, l.ID, f, 0); err != nil {
t.Fatal(err)
}
}
}
add("rock", 40, 8, "rock")
add("hard", 36, 6, "hard rock")
return &mixAPI{h: h, db: db, alice: alice}
}
func (m *mixAPI) get(t *testing.T, id, query string) resp {
t.Helper()
return do(t, m.h, "GET", "/api/v1/mixes/"+url.PathEscape(id)+query, m.alice, nil)
}
func trackIDs(t *testing.T, r resp) []float64 {
t.Helper()
var out []float64
for _, x := range r.Body["tracks"].([]any) {
out = append(out, x.(map[string]any)["id"].(float64))
}
return out
}
func TestMixRoutesNeedASignedInUser(t *testing.T) {
m := newMixAPI(t)
for _, rt := range []struct{ method, path string }{
{"GET", "/api/v1/mixes"}, {"GET", "/api/v1/mixes/genre:rock"}, {"POST", "/api/v1/mixes/genre:rock/playlist"},
} {
if r := do(t, m.h, rt.method, rt.path, "", nil); r.Code != http.StatusUnauthorized {
t.Errorf("%s %s without a token: %d", rt.method, rt.path, r.Code)
}
}
}
func TestTheListOfMixesHasTheGenresOfTheLibrary(t *testing.T) {
m := newMixAPI(t)
r := do(t, m.h, "GET", "/api/v1/mixes", m.alice, nil)
if r.Code != http.StatusOK {
t.Fatalf("list: %d %v", r.Code, r.Body)
}
byID := map[string]map[string]any{}
for _, x := range r.Body["items"].([]any) {
mix := x.(map[string]any)
byID[mix["id"].(string)] = mix
}
rock := byID["genre:rock"]
if rock == nil || rock["kind"] != "genre" || rock["title"] != "Rock Mix" || rock["genre"] != "rock" || len(rock["artists"].([]any)) != 3 {
t.Fatalf("rock = %v in %v", rock, byID)
}
if byID["genre:hard rock"] == nil {
t.Errorf("hard rock is missing: %v", byID)
}
unheard := byID["unheard"]
if unheard == nil || unheard["kind"] != "unheard" || unheard["genre"] != nil {
t.Errorf("unheard = %v", unheard)
}
if byID["favourites"] != nil || byID["rediscover"] != nil {
t.Errorf("mixes that need history are there with none: %v", byID)
}
}
func TestAMixHasItsSongsAndStaysTheSameUntilAnotherVariantIsAsked(t *testing.T) {
m := newMixAPI(t)
r := m.get(t, "genre:rock", "")
if r.Code != http.StatusOK || r.Body["id"] != "genre:rock" || r.Body["variant"] != float64(0) || r.Body["title"] != "Rock Mix" {
t.Fatalf("mix: %d %v", r.Code, r.Body)
}
first := r.Body["tracks"].([]any)[0].(map[string]any)
if first["title"] == nil || first["album"] == nil || len(first["artists"].([]any)) != 1 {
t.Errorf("a song of the mix = %v", first)
}
ids := trackIDs(t, r)
if len(ids) != 40 {
t.Fatalf("%d songs, want the 40 there are (fewer than 50)", len(ids))
}
if again := trackIDs(t, m.get(t, "genre:rock", "")); !slices.Equal(ids, again) {
t.Error("the same mix twice is not the same")
}
v1 := m.get(t, "genre:rock", "?variant=1")
if v1.Body["variant"] != float64(1) || slices.Equal(ids, trackIDs(t, v1)) {
t.Errorf("variant 1 = %v", v1.Body["variant"])
}
// Spaces in the id come percent-encoded.
if r := m.get(t, "genre:hard rock", ""); r.Code != http.StatusOK || len(trackIDs(t, r)) != 36 {
t.Errorf("hard rock: %d", r.Code)
}
}
func TestSongsSkippedLatelyAreLeftOutOfAMix(t *testing.T) {
m := newMixAPI(t)
var ids []int64
rows, err := m.db.Query("SELECT id FROM tracks WHERE title LIKE 'rock song%' ORDER BY id LIMIT 10")
if err != nil {
t.Fatal(err)
}
for rows.Next() {
var id int64
if err := rows.Scan(&id); err != nil {
t.Fatal(err)
}
ids = append(ids, id)
}
rows.Close()
var userID int64
if err := m.db.QueryRow("SELECT id FROM users").Scan(&userID); err != nil {
t.Fatal(err)
}
rec := listening.NewService(storage.NewListeningRepo(m.db), 20*time.Second)
record := func(id int64, typ listening.EventType, played int64, key string) {
t.Helper()
if _, err := rec.Record(context.Background(), listening.Event{UserID: userID, TrackID: id, Type: typ, DurationPlayedMs: played, OccurredAt: time.Now().Add(-time.Hour), IdempotencyKey: key}); err != nil {
t.Fatal(err)
}
}
for i, id := range ids {
record(id, listening.Skip, 3000, fmt.Sprintf("skip%d", i))
}
// The artists also have songs they listened to to the end, so none of them is an avoided artist:
// what is left out is the skipped songs and nothing else.
for a := 0; a < 8; a++ {
for _, n := range []int{16, 24, 32} {
var id int64
if err := m.db.QueryRow("SELECT id FROM tracks WHERE title = ?", fmt.Sprintf("rock song %d", a+n)).Scan(&id); err != nil {
t.Fatal(err)
}
record(id, listening.Complete, 59000, fmt.Sprintf("done%d-%d", a, n))
}
}
got := trackIDs(t, m.get(t, "genre:rock", ""))
if len(got) != 30 {
t.Fatalf("%d songs, want 30: ten were skipped", len(got))
}
for _, id := range got {
if slices.Contains(ids, int64(id)) {
t.Fatalf("song %v was skipped an hour ago", id)
}
}
}
func TestAnUnknownMixOrABadVariantIsRefused(t *testing.T) {
m := newMixAPI(t)
for _, id := range []string{"nope", "genre:polka", "genre:other", "favourites", "rediscover"} {
if r := m.get(t, id, ""); r.Code != http.StatusNotFound || errCode(r) != "mix_not_found" {
t.Errorf("%s: %d %v, want 404 mix_not_found", id, r.Code, r.Body)
}
}
for _, q := range []string{"?variant=abc", "?variant=-1", "?variant=99999999999"} {
if r := m.get(t, "genre:rock", q); r.Code != http.StatusBadRequest || errCode(r) != "invalid_input" {
t.Errorf("%s: %d %v, want 400 invalid_input", q, r.Code, r.Body)
}
}
}
func TestAMixIsSavedAsAPlaylistWithTheSongsThatWereShown(t *testing.T) {
m := newMixAPI(t)
shown := trackIDs(t, m.get(t, "genre:rock", "?variant=2"))
r := do(t, m.h, "POST", "/api/v1/mixes/genre:rock/playlist", m.alice, map[string]any{"variant": 2})
if r.Code != http.StatusCreated || r.Body["name"] != "Rock Mix" || r.Body["track_count"] != float64(len(shown)) {
t.Fatalf("save: %d %v", r.Code, r.Body)
}
pl := int64(r.Body["id"].(float64))
items := do(t, m.h, "GET", "/api/v1/playlists/"+id(pl)+"/tracks?limit=200", m.alice, nil)
var got []float64
for _, it := range items.Body["items"].([]any) {
got = append(got, it.(map[string]any)["track"].(map[string]any)["id"].(float64))
}
if !slices.Equal(got, shown) {
t.Errorf("the playlist has %v, the mix showed %v", got, shown)
}
named := do(t, m.h, "POST", "/api/v1/mixes/genre:rock/playlist", m.alice, map[string]any{"name": " For the car "})
if named.Code != http.StatusCreated || named.Body["name"] != "For the car" {
t.Errorf("named: %d %v", named.Code, named.Body)
}
if r := do(t, m.h, "POST", "/api/v1/mixes/genre:rock/playlist", m.alice, map[string]any{"name": strings.Repeat("a", 201)}); r.Code != http.StatusBadRequest || errCode(r) != "invalid_input" {
t.Errorf("a name that is too long: %d %v", r.Code, r.Body)
}
if r := do(t, m.h, "POST", "/api/v1/mixes/nope/playlist", m.alice, nil); r.Code != http.StatusNotFound || errCode(r) != "mix_not_found" {
t.Errorf("unknown mix: %d %v", r.Code, r.Body)
}
if r := do(t, m.h, "POST", "/api/v1/mixes/genre:rock/playlist", m.alice, map[string]any{"variant": -3}); r.Code != http.StatusBadRequest {
t.Errorf("a negative variant: %d", r.Code)
}
// A refused save leaves no playlist behind: the two that worked are all there is.
list := do(t, m.h, "GET", "/api/v1/playlists", m.alice, nil)
if list.Body["total"] != float64(2) {
t.Errorf("playlists = %v, want 2", list.Body["total"])
}
}

View file

@ -24,6 +24,7 @@ import (
"codeberg.org/squidd/sonda/internal/jobs"
"codeberg.org/squidd/sonda/internal/library"
"codeberg.org/squidd/sonda/internal/listening"
"codeberg.org/squidd/sonda/internal/mixes"
"codeberg.org/squidd/sonda/internal/playlists"
"codeberg.org/squidd/sonda/internal/providers"
"codeberg.org/squidd/sonda/internal/streaming"
@ -99,6 +100,12 @@ type playlistService interface {
MoveItem(ctx context.Context, userID, id, itemID int64, position int) error
}
// mixService is what the handlers need from mixes.Service.
type mixService interface {
List(ctx context.Context, userID int64) ([]mixes.Mix, error)
Get(ctx context.Context, userID int64, id string, variant int) (mixes.Mix, []catalog.Track, error)
}
type feedbackService interface {
Add(ctx context.Context, userID, artistID int64, kind feedback.Kind) error
Remove(ctx context.Context, userID, artistID int64, kind feedback.Kind) error
@ -172,6 +179,7 @@ type Deps struct {
Discover discoverService
Feedback feedbackService
Playlists playlistService
Mixes mixService
Images coverService
CoversJob jobService
ArtistImagesJob jobService
@ -251,6 +259,10 @@ func NewRouter(d Deps) *gin.Engine {
protected.POST("/playlists/:id/tracks", pl.addTracks)
protected.DELETE("/playlists/:id/tracks/:item", pl.removeItem)
protected.PATCH("/playlists/:id/tracks/:item", pl.moveItem)
mx := &mixHandlers{svc: d.Mixes, playlists: d.Playlists}
protected.GET("/mixes", mx.list)
protected.GET("/mixes/:id", mx.get)
protected.POST("/mixes/:id/playlist", mx.save)
protected.GET("/wishlist", fb.list(feedback.Wishlist))
protected.PUT("/wishlist/:id", fb.add(feedback.Wishlist))
protected.DELETE("/wishlist/:id", fb.remove(feedback.Wishlist))

View file

@ -37,6 +37,8 @@ tags:
description: What Sonda has learned from your listening, and the settings that are yours.
- name: Playlists
description: Ordered lists of tracks that belong to one user.
- name: Mixes
description: Playlists made for you, from what you listen to and from the genres of your music.
- name: Discovery
description: Music outside the library, from external providers an admin switched on, and suggestions scored for you.
- name: Providers
@ -1196,6 +1198,94 @@ paths:
application/json:
schema: {$ref: '#/components/schemas/Error'}
/mixes:
get:
tags: [Mixes]
operationId: listMixes
summary: Your mixes
description: |
Playlists that Sonda makes for you, from what you listen to and from the genres of your
music. First the ones made for you (`favourites`, `rediscover`: they need some listening
history, so they are missing without it; `unheard`, the songs you never played), then
one `genre` mix for each genre of the library that has at least 30 songs by at least 4
artists, the genres you play most first (and the biggest after them). Spellings of a genre
that differ only by case or punctuation are one genre, and `other` and `unknown` are not
genres. `title` is in English; a client may say it in its own words from `kind` and
`genre`. Nothing is stored and nothing is sent to a provider: a mix is worked out from your
own history and the genre tags of your files, for you alone.
security: [{bearerAuth: []}]
responses:
'200':
description: The mixes.
content:
application/json:
schema: {$ref: '#/components/schemas/MixList'}
'401': {$ref: '#/components/responses/Unauthorized'}
/mixes/{id}:
parameters:
- {$ref: '#/components/parameters/MixId'}
get:
tags: [Mixes]
operationId: getMix
summary: A mix and its songs
description: |
Up to 50 songs, at most 3 by one artist and never two by the same artist in a row. They
are picked by weighted chance: the artists you play weigh more, songs you never played
a bit more, songs played in the last days less, and songs you skipped in the last two weeks
and artists you avoid do not come at all. The same mix comes back all day: it changes
the next day, or when you ask for another `variant`. The id of a genre mix has the genre
in it (`genre:hard rock`), so it must be percent-encoded in the path. A mix you do not
have (a genre that is too small, `favourites` with no history) is `mix_not_found`.
security: [{bearerAuth: []}]
parameters:
- name: variant
in: query
description: Another mix of the same kind, from 0 (the mix of the day) up.
schema: {type: integer, minimum: 0, maximum: 1000000, default: 0}
responses:
'200':
description: The mix.
content:
application/json:
schema: {$ref: '#/components/schemas/MixWithTracks'}
'400': {$ref: '#/components/responses/InvalidInput'}
'401': {$ref: '#/components/responses/Unauthorized'}
'404': {$ref: '#/components/responses/MixNotFound'}
/mixes/{id}/playlist:
parameters:
- {$ref: '#/components/parameters/MixId'}
post:
tags: [Mixes]
operationId: saveMix
summary: Keep a mix as a playlist
description: |
Makes a playlist of your own with the songs that `variant` of the mix has now, in the
same order. The playlist is a copy: it does not change afterwards, and is yours to edit.
The body is optional.
security: [{bearerAuth: []}]
requestBody:
required: false
content:
application/json:
schema: {$ref: '#/components/schemas/SaveMix'}
responses:
'201':
description: The playlist.
content:
application/json:
schema: {$ref: '#/components/schemas/Playlist'}
'400': {$ref: '#/components/responses/InvalidInput'}
'401': {$ref: '#/components/responses/Unauthorized'}
'404': {$ref: '#/components/responses/MixNotFound'}
'409': {$ref: '#/components/responses/PlaylistLimit'}
'422':
description: The mix has no songs to save (`mix_empty`).
content:
application/json:
schema: {$ref: '#/components/schemas/Error'}
/playlists:
get:
tags: [Playlists]
@ -2409,6 +2499,12 @@ components:
name: offset
in: query
schema: {type: integer, minimum: 0, default: 0}
MixId:
name: id
in: path
required: true
description: "`favourites`, `rediscover`, `unheard` or `genre:<name>` (percent-encoded)."
schema: {type: string}
LibraryId:
name: library_id
in: query
@ -2454,6 +2550,11 @@ components:
content:
application/json:
schema: {$ref: '#/components/schemas/Error'}
MixNotFound:
description: No such mix, or you do not have it (`mix_not_found`).
content:
application/json:
schema: {$ref: '#/components/schemas/Error'}
PlaylistNotFound:
description: No such playlist, or it is somebody else's (`playlist_not_found`).
content:
@ -3049,6 +3150,49 @@ components:
type: array
items: {$ref: '#/components/schemas/Suggestion'}
Mix:
type: object
additionalProperties: false
required: [id, kind, title, genre, artists]
properties:
id: {type: string, description: "`favourites`, `rediscover`, `unheard` or `genre:<name>`."}
kind: {type: string, enum: [genre, favourites, rediscover, unheard]}
title: {type: string, description: In English.}
genre: {type: string, nullable: true, description: "The genre, for a genre mix; null for the others."}
artists: {type: array, items: {type: string}, description: "A few of the artists it has, up to 3."}
MixList:
type: object
additionalProperties: false
required: [items]
properties:
items:
type: array
items: {$ref: '#/components/schemas/Mix'}
MixWithTracks:
type: object
additionalProperties: false
required: [id, kind, title, genre, artists, variant, tracks]
properties:
id: {type: string}
kind: {type: string, enum: [genre, favourites, rediscover, unheard]}
title: {type: string}
genre: {type: string, nullable: true}
artists: {type: array, items: {type: string}}
variant: {type: integer, minimum: 0}
tracks:
type: array
description: In the order to play them. Fewer than 50 when the library has fewer to give.
items: {$ref: '#/components/schemas/Track'}
SaveMix:
type: object
additionalProperties: false
properties:
name: {type: string, maxLength: 200, description: "The name of the playlist; blank or absent means the title of the mix."}
variant: {type: integer, minimum: 0, maximum: 1000000, description: "The variant the client showed; absent means the mix of the day."}
Playlist:
type: object
additionalProperties: false

435
internal/mixes/mixes.go Normal file
View file

@ -0,0 +1,435 @@
// Package mixes makes playlists for a listener, from what they listen to and from the genres of
// their music, the way the mixes of a streaming service are made.
//
// A mix is worked out when it is asked for and is never stored, like the taste profile: it is
// built from the listener's own history and from the genre tags of the files, and nothing leaves
// the server. It stays the same for a day (the seed is the user, the mix, the day and a variant
// number the client may change to ask for another).
//
// There are mixes by genre, one for each genre of the library that is big enough, the ones the
// listener plays most first, and three made for the listener that need some history: Your
// favourites, Rediscover and New to you (which needs none). The songs are picked by weighted
// chance (see Pick); a mix has Size songs, at most PerArtist by one artist.
package mixes
import (
"context"
"errors"
"fmt"
"hash/fnv"
"sort"
"strings"
"time"
"unicode"
"codeberg.org/squidd/sonda/internal/catalog"
"codeberg.org/squidd/sonda/internal/taste"
)
// The shape of a mix and of the list of mixes.
const (
Size = 50
PerArtist = 3
MinGenreTracks = 30 // a genre needs this many songs...
MinGenreArtists = 4 // ...by this many artists to have a mix
MaxGenreMixes = 12
MinLiked = 3 // liked artists for Your favourites, and old ones for Rediscover
MaxFavourites = 100
RediscoverAfter = 30 * 24 * time.Hour
RecentPlay = 3 * 24 * time.Hour // a song played this lately is less wanted
SkipWindow = 14 * 24 * time.Hour
unheardGenres = 3 // New to you takes songs from the best genres this many
MaxVariant = 1_000_000
recentPlayFactor = 0.3
)
// Kind is what a mix is made of.
type Kind string
// The kinds of mixes.
const (
KindGenre Kind = "genre"
KindFavourites Kind = "favourites"
KindRediscover Kind = "rediscover"
KindUnheard Kind = "unheard"
)
// Errors returned by the service.
var (
ErrNotFound = errors.New("mix not found")
ErrInvalidInput = errors.New("invalid input")
)
// Mix says what a mix is. Its songs come from Service.Get.
type Mix struct {
ID string // "genre:rock", "favourites", "rediscover", "unheard"
Kind Kind
Title string // in English; a client may say it in its own words from the kind and the genre
Genre string // the genre, for a genre mix
Artists []string // a few of the artists it has, up to 3
}
// Genre is a genre as the files spell it, with how many songs have it and by how many artists.
type Genre struct {
Name string
Tracks int
Artists int
}
// History is what one listener did with one song.
type History struct {
Plays int // listens that counted
LastPlayed time.Time // the last of them
SkippedAt time.Time // the last skip, zero when there is none
}
// TrackRef is a song that can be played and its main artist (0 when it has none).
type TrackRef struct {
TrackID int64
ArtistID int64
}
// Filter says which songs a mix draws from. Both lists narrow the choice: a song is in when it has one
// of the genres or is by one of the artists (an empty list adds nothing).
type Filter struct {
Genres []string // as the files spell them
ArtistIDs []int64
}
// Store reads what the mixes are made of. It never writes.
type Store interface {
// Genres returns every genre of the playable songs, as spelled in the files.
Genres(ctx context.Context) ([]Genre, error)
// GenreArtists returns the names of the artists with most songs among the genres, up to limit.
GenreArtists(ctx context.Context, genres []string, limit int) ([]string, error)
// ArtistGenres returns, for each artist, how many of their songs have each genre.
ArtistGenres(ctx context.Context, artistIDs []int64) (map[int64]map[string]int, error)
// Pool returns the playable songs of the filter.
Pool(ctx context.Context, f Filter) ([]TrackRef, error)
// History returns what the user did with the songs they did anything with; minListen is how
// long a play must last to count as a listen.
History(ctx context.Context, userID int64, minListen time.Duration) (map[int64]History, error)
ArtistNames(ctx context.Context, ids []int64) (map[int64]string, error)
// Tracks returns the songs in the order given, leaving out any that cannot be played.
Tracks(ctx context.Context, ids []int64) ([]catalog.Track, error)
}
// Taster says what the listener likes: *taste.Service.
type Taster interface {
Learn(ctx context.Context, userID int64) (taste.Learned, error)
}
// Service makes the mixes.
type Service struct {
store Store
taster Taster
minListen time.Duration
now func() time.Time
}
// NewService builds a Service. minListen is how long a play must last to count as a listen.
func NewService(store Store, taster Taster, minListen time.Duration) *Service {
return &Service{store: store, taster: taster, minListen: minListen, now: time.Now}
}
// junkGenres are tags that say nothing about the music.
var junkGenres = map[string]bool{"other": true, "unknown": true, "genre": true, "none": true, "(none)": true, "misc": true, "miscellaneous": true, "blues/other": false}
// typos maps a genre that is written wrong, in the files of many people, to the right one.
var typos = map[string]string{"trash metal": "thrash metal"}
// normalizeGenre is the genre as it is compared: lower case, with "+", "-", "_" and "/" taken for
// spaces, and "" when it is not a genre ("other", "unknown"). "jazz+funk" and "jazz-funk" are one.
func normalizeGenre(g string) string {
g = strings.ToLower(strings.TrimSpace(g))
g = strings.Map(func(r rune) rune {
switch r {
case '+', '-', '_', '/':
return ' '
}
return r
}, g)
g = strings.Join(strings.Fields(g), " ")
if g == "" || junkGenres[g] {
return ""
}
if fixed, ok := typos[g]; ok {
return fixed
}
return g
}
func titleCase(s string) string {
words := strings.Fields(s)
for i, w := range words {
r := []rune(w)
r[0] = unicode.ToUpper(r[0])
words[i] = string(r)
}
return strings.Join(words, " ")
}
// genreGroup is one genre with the spellings of it that the files have.
type genreGroup struct {
name string
raw []string
tracks int
artists int // of the spelling that has most: the spellings overlap, so a sum would promise too much
affinity float64
}
// definition is a mix that can be made, with what it takes.
type definition struct {
Mix
raw []string // genre spellings, for a genre mix and for New to you
artists []int64 // artists to draw from, for favourites and rediscover
}
// state is what the list of mixes and one mix are both made from.
type state struct {
learned taste.Learned
defs []definition
}
// groups returns the genres that are big enough for a mix, the ones the listener plays most first,
// then the biggest.
func (s *Service) groups(ctx context.Context, learned taste.Learned) ([]genreGroup, error) {
genres, err := s.store.Genres(ctx)
if err != nil {
return nil, fmt.Errorf("read genres: %w", err)
}
byName := map[string]*genreGroup{}
for _, g := range genres {
n := normalizeGenre(g.Name)
if n == "" {
continue
}
grp := byName[n]
if grp == nil {
grp = &genreGroup{name: n}
byName[n] = grp
}
grp.raw = append(grp.raw, g.Name)
grp.tracks += g.Tracks
grp.artists = max(grp.artists, g.Artists)
}
// What the listener plays: each liked artist gives their score to the genres of their songs, in
// the share of those songs that have each.
liked := learned.Liked
if len(liked) > MaxFavourites {
liked = liked[:MaxFavourites]
}
if len(liked) > 0 {
ids := make([]int64, len(liked))
for i, p := range liked {
ids[i] = p.ArtistID
}
of, err := s.store.ArtistGenres(ctx, ids)
if err != nil {
return nil, fmt.Errorf("read artist genres: %w", err)
}
for _, p := range liked {
total := 0
for _, n := range of[p.ArtistID] {
total += n
}
for raw, n := range of[p.ArtistID] {
if grp := byName[normalizeGenre(raw)]; grp != nil && total > 0 {
grp.affinity += p.Score * float64(n) / float64(total)
}
}
}
}
out := make([]genreGroup, 0, len(byName))
for _, grp := range byName {
if grp.tracks >= MinGenreTracks && grp.artists >= MinGenreArtists {
sort.Strings(grp.raw)
out = append(out, *grp)
}
}
sort.Slice(out, func(i, j int) bool {
a, b := out[i], out[j]
switch {
case a.affinity != b.affinity:
return a.affinity > b.affinity
case a.tracks != b.tracks:
return a.tracks > b.tracks
}
return a.name < b.name
})
return out, nil
}
func (s *Service) state(ctx context.Context, userID int64) (state, error) {
learned, err := s.taster.Learn(ctx, userID)
if err != nil {
return state{}, fmt.Errorf("learn taste: %w", err)
}
groups, err := s.groups(ctx, learned)
if err != nil {
return state{}, err
}
now := s.now()
st := state{learned: learned}
liked := learned.Liked
if len(liked) > MaxFavourites {
liked = liked[:MaxFavourites]
}
var likedIDs, oldIDs []int64
for _, p := range liked {
likedIDs = append(likedIDs, p.ArtistID)
if !p.LastListened.IsZero() && now.Sub(p.LastListened) >= RediscoverAfter {
oldIDs = append(oldIDs, p.ArtistID)
}
}
namesOf := func(ids []int64) ([]string, error) {
if len(ids) > 3 {
ids = ids[:3]
}
names, err := s.store.ArtistNames(ctx, ids)
if err != nil {
return nil, fmt.Errorf("read artist names: %w", err)
}
out := make([]string, 0, len(ids))
for _, id := range ids {
if n := names[id]; n != "" {
out = append(out, n)
}
}
return out, nil
}
if len(likedIDs) >= MinLiked {
names, err := namesOf(likedIDs)
if err != nil {
return state{}, err
}
st.defs = append(st.defs, definition{Mix: Mix{ID: "favourites", Kind: KindFavourites, Title: "Your favourites", Artists: names}, artists: likedIDs})
}
if len(oldIDs) >= MinLiked {
names, err := namesOf(oldIDs)
if err != nil {
return state{}, err
}
st.defs = append(st.defs, definition{Mix: Mix{ID: "rediscover", Kind: KindRediscover, Title: "Rediscover", Artists: names}, artists: oldIDs})
}
if len(groups) > 0 {
var raw []string
for _, g := range groups[:min(unheardGenres, len(groups))] {
raw = append(raw, g.raw...)
}
st.defs = append(st.defs, definition{Mix: Mix{ID: "unheard", Kind: KindUnheard, Title: "New to you", Artists: []string{}}, raw: raw})
}
for _, g := range groups[:min(MaxGenreMixes, len(groups))] {
names, err := s.store.GenreArtists(ctx, g.raw, 3)
if err != nil {
return state{}, fmt.Errorf("read genre artists: %w", err)
}
if names == nil {
names = []string{}
}
st.defs = append(st.defs, definition{
Mix: Mix{ID: "genre:" + g.name, Kind: KindGenre, Title: titleCase(g.name) + " Mix", Genre: g.name, Artists: names},
raw: g.raw,
})
}
return st, nil
}
// List returns the mixes the listener has: the ones made for them first, then the genres.
func (s *Service) List(ctx context.Context, userID int64) ([]Mix, error) {
st, err := s.state(ctx, userID)
if err != nil {
return nil, err
}
out := make([]Mix, len(st.defs))
for i, d := range st.defs {
out[i] = d.Mix
}
return out, nil
}
// Get makes one mix: its songs in order. variant 0 is the mix of the day; another number gives
// another mix of the same kind. It gives ErrNotFound for a mix the listener does not have.
func (s *Service) Get(ctx context.Context, userID int64, id string, variant int) (Mix, []catalog.Track, error) {
if variant < 0 || variant > MaxVariant {
return Mix{}, nil, fmt.Errorf("%w: variant must be from 0 to %d", ErrInvalidInput, MaxVariant)
}
st, err := s.state(ctx, userID)
if err != nil {
return Mix{}, nil, err
}
var def *definition
for i := range st.defs {
if st.defs[i].ID == id {
def = &st.defs[i]
}
}
if def == nil {
return Mix{}, nil, ErrNotFound
}
cands, err := s.candidates(ctx, userID, st, *def)
if err != nil {
return Mix{}, nil, err
}
h := fnv.New64a()
fmt.Fprintf(h, "%d|%s|%d|%d", userID, id, s.now().Unix()/86400, variant)
picked := Pick(cands, Size, PerArtist, h.Sum64())
tracks, err := s.store.Tracks(ctx, picked)
if err != nil {
return Mix{}, nil, fmt.Errorf("read songs: %w", err)
}
return def.Mix, tracks, nil
}
// candidates weighs the songs a mix may take. A song skipped lately is out, and so is every song
// of an artist the listener avoids; a song played in the last days is less wanted.
func (s *Service) candidates(ctx context.Context, userID int64, st state, d definition) ([]Candidate, error) {
pool, err := s.store.Pool(ctx, Filter{Genres: d.raw, ArtistIDs: d.artists})
if err != nil {
return nil, fmt.Errorf("read the pool: %w", err)
}
history, err := s.store.History(ctx, userID, s.minListen)
if err != nil {
return nil, fmt.Errorf("read history: %w", err)
}
now := s.now()
out := make([]Candidate, 0, len(pool))
for _, ref := range pool {
h, heard := history[ref.TrackID]
if !h.SkippedAt.IsZero() && now.Sub(h.SkippedAt) < SkipWindow {
continue
}
pref := st.learned.Prefs[ref.ArtistID].Preference
if pref < 0 {
continue
}
var w float64
switch d.Kind {
case KindGenre:
w = 1 + 1.5*pref
if h.Plays == 0 {
w += 0.5
}
case KindFavourites, KindRediscover:
if pref <= 0 {
continue
}
w = pref + 0.2
case KindUnheard:
if heard {
continue
}
w = 1
}
if h.Plays > 0 && now.Sub(h.LastPlayed) < RecentPlay {
w *= recentPlayFactor
}
out = append(out, Candidate{TrackID: ref.TrackID, ArtistID: ref.ArtistID, Weight: w})
}
return out, nil
}

View file

@ -0,0 +1,409 @@
package mixes
import (
"context"
"errors"
"slices"
"testing"
"time"
"codeberg.org/squidd/sonda/internal/catalog"
"codeberg.org/squidd/sonda/internal/taste"
)
var day = time.Date(2026, 11, 12, 10, 0, 0, 0, time.UTC)
// ---- the normal spelling of a genre ----
func TestNormalizeGenreMergesSpellingsAndDropsWhatIsNotAGenre(t *testing.T) {
for in, want := range map[string]string{
"rock": "rock",
" Hard Rock ": "hard rock",
"jazz+funk": "jazz funk",
"jazz-funk": "jazz funk",
"jazz / funk": "jazz funk",
"hip_hop": "hip hop",
"trash metal": "thrash metal",
"r&b": "r&b",
"other": "",
"Unknown": "",
"": "",
" ": "",
"(none)": "",
"genre": "",
"blues rock": "blues rock",
} {
if got := normalizeGenre(in); got != want {
t.Errorf("normalizeGenre(%q) = %q, want %q", in, got, want)
}
}
}
// ---- a store in memory ----
type fakeStore struct {
genres []Genre
artists map[string][]string // raw genre -> top artists
liked map[int64]map[string]int // artist -> raw genre -> songs
pool map[string][]TrackRef // key: "genre:<raw>" or "artists"
byArtist []TrackRef // tracks for an artist filter
history map[int64]History
names map[int64]string
asked []Filter
}
func (f *fakeStore) Genres(context.Context) ([]Genre, error) { return f.genres, nil }
func (f *fakeStore) GenreArtists(_ context.Context, raws []string, limit int) ([]string, error) {
var out []string
for _, r := range raws {
out = append(out, f.artists[r]...)
}
if len(out) > limit {
out = out[:limit]
}
return out, nil
}
func (f *fakeStore) ArtistGenres(_ context.Context, ids []int64) (map[int64]map[string]int, error) {
out := map[int64]map[string]int{}
for _, id := range ids {
if g, ok := f.liked[id]; ok {
out[id] = g
}
}
return out, nil
}
func (f *fakeStore) Pool(_ context.Context, fl Filter) ([]TrackRef, error) {
f.asked = append(f.asked, fl)
var out []TrackRef
for _, g := range fl.Genres {
out = append(out, f.pool[g]...)
}
for _, t := range f.byArtist {
if slices.Contains(fl.ArtistIDs, t.ArtistID) {
out = append(out, t)
}
}
return out, nil
}
func (f *fakeStore) History(context.Context, int64, time.Duration) (map[int64]History, error) {
return f.history, nil
}
func (f *fakeStore) ArtistNames(_ context.Context, ids []int64) (map[int64]string, error) {
out := map[int64]string{}
for _, id := range ids {
out[id] = f.names[id]
}
return out, nil
}
func (f *fakeStore) Tracks(_ context.Context, ids []int64) ([]catalog.Track, error) {
out := make([]catalog.Track, len(ids))
for i, id := range ids {
out[i] = catalog.Track{ID: id}
}
return out, nil
}
type fakeTaster struct{ l taste.Learned }
func (f fakeTaster) Learn(context.Context, int64) (taste.Learned, error) { return f.l, nil }
// songs makes n songs by the artists in turn; ids start at first.
func songs(first int64, n int, artists ...int64) []TrackRef {
out := make([]TrackRef, n)
for i := range out {
out[i] = TrackRef{TrackID: first + int64(i), ArtistID: artists[i%len(artists)]}
}
return out
}
func pref(id int64, score float64, last time.Time) taste.ArtistPref {
return taste.ArtistPref{ArtistID: id, Score: score, Preference: score / (1 + score), Listens: 5, LastListened: last}
}
func learned(prefs ...taste.ArtistPref) taste.Learned {
l := taste.Learned{Prefs: map[int64]taste.ArtistPref{}}
for _, p := range prefs {
l.Prefs[p.ArtistID] = p
if p.Score > 0 {
l.Liked = append(l.Liked, p)
}
}
slices.SortFunc(l.Liked, func(a, b taste.ArtistPref) int {
if a.Score > b.Score {
return -1
}
return 1
})
return l
}
func newService(st *fakeStore, l taste.Learned) *Service {
s := NewService(st, fakeTaster{l}, 20*time.Second)
s.now = func() time.Time { return day }
return s
}
// rockArtists are 30 artists, ids 1 to 30.
func rockArtists() []int64 {
out := make([]int64, 30)
for i := range out {
out[i] = int64(i + 1)
}
return out
}
func library() *fakeStore {
return &fakeStore{
genres: []Genre{
{"rock", 400, 40}, {"metal", 300, 30}, {"jazz+funk", 20, 5}, {"jazz-funk", 15, 4},
{"other", 900, 80}, {"tiny", 10, 2}, {"few artists", 200, 2}, {"pop", 100, 12},
},
artists: map[string][]string{"rock": {"Queen", "AC/DC", "Kiss"}, "metal": {"Metallica", "Slayer"}, "pop": {"ABBA"}},
pool: map[string][]TrackRef{
"rock": songs(1, 400, rockArtists()...),
"metal": songs(1000, 300, 11, 12, 13, 14, 15),
"pop": songs(2000, 100, 21, 22, 23, 24, 25),
"jazz+funk": songs(3000, 20, 31, 32, 33, 34, 35),
"jazz-funk": songs(3100, 15, 31, 32, 33, 34),
},
names: map[int64]string{1: "Queen", 2: "AC/DC", 11: "Metallica", 12: "Slayer", 21: "ABBA"},
}
}
func ids(m []catalog.Track) []int64 {
out := make([]int64, len(m))
for i, t := range m {
out[i] = t.ID
}
return out
}
// ---- the list of mixes ----
func TestGenreMixesAreMadeForTheGenresThatAreBigEnoughAndSpellingsAreOne(t *testing.T) {
s := newService(library(), taste.Learned{})
list, err := s.List(context.Background(), 1)
if err != nil {
t.Fatal(err)
}
var got []string
var rock Mix
for _, m := range list {
switch m.Kind {
case KindGenre:
got = append(got, m.ID)
if m.Genre == "rock" {
rock = m
}
case KindUnheard: // needs no history
default:
t.Errorf("without listening only genre mixes and New to you exist, got %+v", m)
}
}
want := []string{"genre:rock", "genre:metal", "genre:pop", "genre:jazz funk"}
if !slices.Equal(got, want) {
t.Errorf("mixes = %v, want %v: by size, jazz+funk and jazz-funk as one, no other/tiny/few artists", got, want)
}
if rock.Title != "Rock Mix" || rock.Genre != "rock" || !slices.Equal(rock.Artists, []string{"Queen", "AC/DC", "Kiss"}) {
t.Errorf("rock = %+v", rock)
}
}
func TestTheGenresTheListenerPlaysComeFirst(t *testing.T) {
st := library()
// Artist 11 (metal) is loved, and plays a bit of rock too.
st.liked = map[int64]map[string]int{11: {"metal": 9, "rock": 1}}
s := newService(st, learned(pref(11, 5, day.Add(-24*time.Hour))))
list, _ := s.List(context.Background(), 1)
var genres []string
for _, m := range list {
if m.Kind == KindGenre {
genres = append(genres, m.Genre)
}
}
if len(genres) < 2 || genres[0] != "metal" || genres[1] != "rock" {
t.Errorf("genres = %v, want metal then rock first", genres)
}
}
func TestMixesMadeForYouNeedHistory(t *testing.T) {
st := library()
none, _ := newService(st, taste.Learned{}).List(context.Background(), 1)
for _, m := range none {
if m.Kind == KindFavourites || m.Kind == KindRediscover {
t.Errorf("%s is offered without any listening", m.ID)
}
}
if !slices.ContainsFunc(none, func(m Mix) bool { return m.Kind == KindUnheard }) {
t.Error("New to you needs no history: it is the songs never played")
}
recent := day.Add(-2 * 24 * time.Hour)
old := day.Add(-60 * 24 * time.Hour)
two := learned(pref(1, 5, recent), pref(2, 4, recent))
if list, _ := newService(st, two).List(context.Background(), 1); slices.ContainsFunc(list, func(m Mix) bool { return m.Kind == KindFavourites }) {
t.Error("two liked artists are not enough for favourites")
}
many := learned(pref(1, 5, recent), pref(2, 4, recent), pref(3, 3, old), pref(4, 2, old), pref(5, 1.5, old))
list, _ := newService(st, many).List(context.Background(), 1)
if list[0].ID != "favourites" || list[1].ID != "rediscover" {
t.Errorf("first mixes = %s, %s: the ones made for you come first", list[0].ID, list[1].ID)
}
if list[0].Title != "Your favourites" || list[1].Title != "Rediscover" || list[0].Genre != "" {
t.Errorf("titles = %+v %+v", list[0], list[1])
}
}
// ---- one mix ----
func TestAGenreMixHasFiftySongsOfThatGenreAndStaysTheSameForADay(t *testing.T) {
s := newService(library(), taste.Learned{})
m, tracks, err := s.Get(context.Background(), 1, "genre:rock", 0)
if err != nil || m.ID != "genre:rock" || len(tracks) != Size {
t.Fatalf("mix %+v with %d songs, err %v", m, len(tracks), err)
}
for _, id := range ids(tracks) {
if id < 1 || id > 400 {
t.Fatalf("song %d is not a rock song", id)
}
}
_, again, _ := s.Get(context.Background(), 1, "genre:rock", 0)
if !slices.Equal(ids(tracks), ids(again)) {
t.Error("the same mix twice in a day must be the same")
}
_, other, _ := s.Get(context.Background(), 1, "genre:rock", 1)
if slices.Equal(ids(tracks), ids(other)) {
t.Error("another variant gave the same mix")
}
_, someoneElse, _ := s.Get(context.Background(), 2, "genre:rock", 0)
if slices.Equal(ids(tracks), ids(someoneElse)) {
t.Error("two users got the same mix")
}
s.now = func() time.Time { return day.Add(24 * time.Hour) }
_, tomorrow, _ := s.Get(context.Background(), 1, "genre:rock", 0)
if slices.Equal(ids(tracks), ids(tomorrow)) {
t.Error("the mix did not change the next day")
}
}
func TestAMixHasAtMostThreeSongsPerArtistAndMergedSpellingsFeedOneMix(t *testing.T) {
st := library()
s := newService(st, taste.Learned{})
_, tracks, err := s.Get(context.Background(), 1, "genre:rock", 0)
if err != nil {
t.Fatal(err)
}
perArtist := map[int64]int{}
for _, id := range ids(tracks) {
perArtist[st.pool["rock"][id-1].ArtistID]++
}
for a, n := range perArtist {
if n > PerArtist {
t.Errorf("artist %d has %d songs, want at most %d", a, n, PerArtist)
}
}
if _, _, err := s.Get(context.Background(), 1, "genre:jazz funk", 0); err != nil {
t.Fatalf("jazz funk: %v", err)
}
last := st.asked[len(st.asked)-1]
slices.Sort(last.Genres)
if !slices.Equal(last.Genres, []string{"jazz+funk", "jazz-funk"}) {
t.Errorf("the mix asked for the genres %v, want both spellings", last.Genres)
}
}
func TestUnknownMixesAreNotFound(t *testing.T) {
s := newService(library(), taste.Learned{})
for _, id := range []string{"", "nope", "genre:", "genre:other", "genre:tiny", "genre:polka", "favourites", "rediscover", "genre:rock:x"} {
if _, _, err := s.Get(context.Background(), 1, id, 0); !errors.Is(err, ErrNotFound) {
t.Errorf("%q: err = %v, want ErrNotFound", id, err)
}
}
if _, _, err := s.Get(context.Background(), 1, "genre:rock", -1); !errors.Is(err, ErrInvalidInput) {
t.Errorf("a negative variant: err = %v", err)
}
}
func TestFavouritesAreTheSongsOfLikedArtistsWithoutTheSkippedOnes(t *testing.T) {
st := library()
recent := day.Add(-2 * 24 * time.Hour)
st.byArtist = append(songs(5000, 60, 1), songs(5100, 60, 2)...)
st.byArtist = append(st.byArtist, songs(5200, 60, 3)...)
st.byArtist = append(st.byArtist, songs(5300, 60, 9)...) // an artist that is not liked
st.history = map[int64]History{5000: {Plays: 3, LastPlayed: recent, SkippedAt: day.Add(-24 * time.Hour)}}
for i := int64(1); i < 20; i++ { // skipped lately: out
st.history[5000+i] = History{SkippedAt: day.Add(-48 * time.Hour)}
}
s := newService(st, learned(pref(1, 5, recent), pref(2, 4, recent), pref(3, 3, recent)))
_, tracks, err := s.Get(context.Background(), 1, "favourites", 0)
if err != nil || len(tracks) != Size {
t.Fatalf("%d songs, err %v", len(tracks), err)
}
for _, id := range ids(tracks) {
if id >= 5300 {
t.Fatalf("song %d is by an artist that is not liked", id)
}
if id >= 5000 && id < 5020 {
t.Fatalf("song %d was skipped lately", id)
}
}
}
func TestAnAvoidedArtistIsLeftOutOfAGenreMix(t *testing.T) {
st := library()
avoided := taste.ArtistPref{ArtistID: 1, Score: -3, Preference: -0.9, LastListened: day.Add(-24 * time.Hour)}
s := newService(st, learned(avoided))
for v := 0; v < 5; v++ {
_, tracks, _ := s.Get(context.Background(), 1, "genre:rock", v)
for _, id := range ids(tracks) {
if st.pool["rock"][id-1].ArtistID == 1 {
t.Fatalf("variant %d has song %d by an avoided artist", v, id)
}
}
}
}
func TestRediscoverIsLikedArtistsNotPlayedForAMonth(t *testing.T) {
st := library()
recent := day.Add(-2 * 24 * time.Hour)
old := day.Add(-60 * 24 * time.Hour)
st.byArtist = append(songs(5000, 60, 1, 2, 3), songs(5100, 60, 4, 5, 6)...)
l := learned(pref(1, 5, recent), pref(2, 4, recent), pref(3, 3, recent), pref(4, 2, old), pref(5, 1.9, old), pref(6, 1.8, old))
s := newService(st, l)
_, tracks, err := s.Get(context.Background(), 1, "rediscover", 0)
if err != nil || len(tracks) != Size {
t.Fatalf("%d songs err %v", len(tracks), err)
}
for _, id := range ids(tracks) {
if id < 5100 {
t.Fatalf("song %d is by an artist that was played lately", id)
}
}
}
func TestNewToYouNeverHasASongThatWasPlayedOrSkipped(t *testing.T) {
st := library()
st.history = map[int64]History{}
for id := int64(1); id <= 100; id++ {
st.history[id] = History{Plays: 2, LastPlayed: day.Add(-time.Hour)}
}
for id := int64(101); id <= 150; id++ {
st.history[id] = History{SkippedAt: day.Add(-90 * 24 * time.Hour)} // a skip long ago still counts as heard
}
s := newService(st, taste.Learned{})
_, tracks, err := s.Get(context.Background(), 1, "unheard", 0)
if err != nil || len(tracks) != Size {
t.Fatalf("%d songs err %v", len(tracks), err)
}
for _, id := range ids(tracks) {
if _, heard := st.history[id]; heard {
t.Fatalf("song %d was heard before", id)
}
}
}

123
internal/mixes/pick.go Normal file
View file

@ -0,0 +1,123 @@
package mixes
import (
"math"
"math/rand/v2"
"sort"
)
// Candidate is a song that may go into a mix. Weight says how much it is wanted: twice the
// weight is twice the chance. A weight of 0 or less keeps the song out.
type Candidate struct {
TrackID int64
ArtistID int64 // 0 when the song has no artist; such songs are all of the same "artist"
Weight float64
}
// Pick chooses up to size songs from the candidates and puts them in order.
//
// The choice is by weighted chance without repeats (a song of weight w gets the key ln(u)/w for
// a random u, and the largest keys win). At most perArtist songs of one artist are taken, unless
// the candidates are by so few artists that the size could not be reached, and then the limit is
// raised just enough. Two songs of the same artist are not put next to each other when the order
// allows it.
//
// The same seed gives the same mix, which is what keeps a mix steady for a day.
func Pick(cands []Candidate, size, perArtist int, seed uint64) []int64 {
if size <= 0 {
return nil
}
rng := rand.New(rand.NewPCG(seed, seed^0x9e3779b97f4a7c15))
type keyed struct {
Candidate
key float64
}
pool := make([]keyed, 0, len(cands))
artists := map[int64]bool{}
// The pool is walked in a fixed order, so that a seed means one thing.
sorted := append([]Candidate(nil), cands...)
sort.Slice(sorted, func(i, j int) bool { return sorted[i].TrackID < sorted[j].TrackID })
for _, c := range sorted {
if c.Weight <= 0 || math.IsNaN(c.Weight) || math.IsInf(c.Weight, 0) {
continue
}
u := 1 - rng.Float64() // in (0, 1]
pool = append(pool, keyed{c, math.Log(u) / c.Weight})
artists[c.ArtistID] = true
}
sort.SliceStable(pool, func(i, j int) bool { return pool[i].key > pool[j].key })
limit := perArtist
if limit < 1 {
limit = 1
}
if need := (size + len(artists) - 1) / max(len(artists), 1); need > limit {
limit = need
}
taken := map[int64]int{}
picked := make([]Candidate, 0, size)
for _, c := range pool {
if len(picked) == size {
break
}
if taken[c.ArtistID] >= limit {
continue
}
taken[c.ArtistID]++
picked = append(picked, c.Candidate)
}
rng.Shuffle(len(picked), func(i, j int) { picked[i], picked[j] = picked[j], picked[i] })
picked = spread(picked, rng)
out := make([]int64, len(picked))
for i, c := range picked {
out[i] = c.TrackID
}
return out
}
// spread puts the songs in an order where two of the same artist are not neighbours, as far as
// the songs allow it: each place takes a song of the artist with most songs left, other than the
// one that just played (ties by chance). The songs of an artist keep the order they came in.
func spread(s []Candidate, rng *rand.Rand) []Candidate {
left := map[int64][]Candidate{}
var artists []int64
for _, c := range s {
if _, ok := left[c.ArtistID]; !ok {
artists = append(artists, c.ArtistID)
}
left[c.ArtistID] = append(left[c.ArtistID], c)
}
sort.Slice(artists, func(i, j int) bool { return artists[i] < artists[j] }) // map order must not matter
out := make([]Candidate, 0, len(s))
var last int64
haveLast := false
for len(out) < len(s) {
best := -1
var tied []int64
for _, a := range artists {
n := len(left[a])
if n == 0 || (haveLast && a == last) {
continue
}
switch {
case n > best:
best, tied = n, []int64{a}
case n == best:
tied = append(tied, a)
}
}
var pick int64
if len(tied) > 0 {
pick = tied[rng.IntN(len(tied))]
} else {
pick = last // only the artist that just played has songs left
}
out = append(out, left[pick][0])
left[pick] = left[pick][1:]
last, haveLast = pick, true
}
return out
}

122
internal/mixes/pick_test.go Normal file
View file

@ -0,0 +1,122 @@
package mixes
import (
"slices"
"testing"
)
// pool makes artists artists with perArtist songs each, all of weight 1. Song ids are 1, 2, 3, ...
func pool(artists, perArtist int) []Candidate {
var out []Candidate
id := int64(0)
for a := 1; a <= artists; a++ {
for i := 0; i < perArtist; i++ {
id++
out = append(out, Candidate{TrackID: id, ArtistID: int64(a), Weight: 1})
}
}
return out
}
func artistOf(c []Candidate) map[int64]int64 {
m := map[int64]int64{}
for _, x := range c {
m[x.TrackID] = x.ArtistID
}
return m
}
func TestPickTakesAtMostSizeSongsFromThePoolWithoutRepeats(t *testing.T) {
c := pool(20, 10)
got := Pick(c, 50, 3, 1)
if len(got) != 50 {
t.Fatalf("got %d songs, want 50", len(got))
}
seen := map[int64]bool{}
for _, id := range got {
if seen[id] || id < 1 || id > 200 {
t.Fatalf("song %d is repeated or not in the pool", id)
}
seen[id] = true
}
}
func TestPickGivesEverythingWhenThePoolIsSmallAndNothingWhenItIsEmpty(t *testing.T) {
if got := Pick(pool(4, 3), 50, 3, 1); len(got) != 12 {
t.Errorf("a pool of 12 gave %d songs", len(got))
}
if got := Pick(nil, 50, 3, 1); len(got) != 0 {
t.Errorf("an empty pool gave %v", got)
}
}
func TestPickIsTheSameForTheSameSeedAndDifferentForAnother(t *testing.T) {
c := pool(30, 10)
a, b := Pick(c, 50, 3, 7), Pick(c, 50, 3, 7)
if !slices.Equal(a, b) {
t.Error("the same seed gave two different mixes")
}
if slices.Equal(a, Pick(c, 50, 3, 8)) {
t.Error("another seed gave the same mix")
}
}
func TestPickKeepsAnArtistToItsShareAndRelaxesItWhenThereAreFewArtists(t *testing.T) {
c := pool(20, 10)
by := artistOf(c)
for seed := uint64(0); seed < 20; seed++ {
count := map[int64]int{}
for _, id := range Pick(c, 50, 3, seed) {
count[by[id]]++
}
for artist, n := range count {
if n > 3 {
t.Fatalf("seed %d: artist %d has %d songs, want at most 3", seed, artist, n)
}
}
}
// Two artists cannot fill ten places with three songs each: the cap gives way to the size.
few := pool(2, 20)
if got := Pick(few, 10, 3, 1); len(got) != 10 {
t.Errorf("two artists gave %d songs, want 10", len(got))
}
}
func TestPickPrefersTheHeavierSongsAndNeverTakesOneWithNoWeight(t *testing.T) {
c := pool(25, 4)
c[0].Weight = 200 // song 1
c[1].Weight = 0 // song 2
c[2].Weight = -5 // song 3
heavy := 0
for seed := uint64(0); seed < 200; seed++ {
got := Pick(c, 10, 3, seed)
if slices.Contains(got, 2) || slices.Contains(got, 3) {
t.Fatalf("seed %d took a song with no weight: %v", seed, got)
}
if slices.Contains(got, 1) {
heavy++
}
}
if heavy < 190 {
t.Errorf("the heavy song was in %d of 200 mixes, want nearly all", heavy)
}
}
func TestPickLeavesNoTwoSongsOfTheSameArtistNextToEachOther(t *testing.T) {
c := pool(10, 8)
by := artistOf(c)
for seed := uint64(0); seed < 50; seed++ {
got := Pick(c, 30, 3, seed)
for i := 1; i < len(got); i++ {
if by[got[i]] == by[got[i-1]] {
t.Fatalf("seed %d: songs %d and %d are by the same artist and next to each other: %v", seed, got[i-1], got[i], got)
}
}
}
}
func TestPickWithOneArtistStillGivesSongs(t *testing.T) {
if got := Pick(pool(1, 30), 10, 3, 1); len(got) != 10 {
t.Errorf("one artist gave %d songs, want 10", len(got))
}
}

View file

@ -0,0 +1,48 @@
package mixes_test
import (
"context"
"fmt"
"testing"
"time"
"codeberg.org/squidd/sonda/internal/mixes"
"codeberg.org/squidd/sonda/internal/storage"
"codeberg.org/squidd/sonda/internal/taste"
)
type none struct{}
func (none) Learn(context.Context, int64) (taste.Learned, error) { return taste.Learned{}, nil }
func TestZZReal(t *testing.T) {
db, err := storage.Open(storage.DriverSQLite, "/tmp/claude-1000/-home-squid-projects-personal-sonda/879e274d-7595-4688-9de1-101770d03238/scratchpad/real.db")
if err != nil {
t.Fatal(err)
}
defer db.Close()
s := mixes.NewService(storage.NewMixRepo(db), none{}, 20*time.Second)
list, err := s.List(context.Background(), 1)
if err != nil {
t.Fatal(err)
}
for _, m := range list {
fmt.Printf("ZZ %-22s %-26s %v\n", m.ID, m.Title, m.Artists)
}
start := time.Now()
m, tracks, err := s.Get(context.Background(), 1, "genre:rock", 0)
if err != nil {
t.Fatal(err)
}
fmt.Printf("ZZ -- %s: %d songs in %v\n", m.ID, len(tracks), time.Since(start))
for i, tr := range tracks[:8] {
a := ""
if len(tr.Artists) > 0 {
a = tr.Artists[0].Name
}
fmt.Printf("ZZ %2d. %s — %s\n", i+1, a, tr.Title)
}
start = time.Now()
_, tracks, err = s.Get(context.Background(), 1, "unheard", 0)
fmt.Printf("ZZ -- unheard: %d songs in %v err %v\n", len(tracks), time.Since(start), err)
}

244
internal/storage/mixes.go Normal file
View file

@ -0,0 +1,244 @@
package storage
import (
"context"
"database/sql"
"fmt"
"strings"
"time"
"github.com/uptrace/bun"
"codeberg.org/squidd/sonda/internal/catalog"
"codeberg.org/squidd/sonda/internal/mixes"
)
// MixRepo reads what the mixes are made of. It implements mixes.Store and never writes.
type MixRepo struct {
db *bun.DB
cat *CatalogRepo // reads the details of the songs
taste *TasteRepo
}
// NewMixRepo builds a MixRepo.
func NewMixRepo(db *bun.DB) *MixRepo {
return &MixRepo{db: db, cat: NewCatalogRepo(db), taste: NewTasteRepo(db)}
}
// Genres returns every genre of the playable songs as the files spell it, with the number of
// songs and of main artists.
func (r *MixRepo) Genres(ctx context.Context) ([]mixes.Genre, error) {
vis, _ := visible("t", 0)
rows, err := r.db.QueryContext(ctx,
`SELECT g.genre, COUNT(DISTINCT t.id), COUNT(DISTINCT ta.artist_id)
FROM track_genres g
JOIN tracks t ON t.id = g.track_id
LEFT JOIN track_artists ta ON ta.track_id = t.id AND ta.role = 'main'
WHERE `+vis+` GROUP BY g.genre ORDER BY g.genre`)
if err != nil {
return nil, fmt.Errorf("read genres: %w", err)
}
defer rows.Close()
out := []mixes.Genre{}
for rows.Next() {
var g mixes.Genre
if err := rows.Scan(&g.Name, &g.Tracks, &g.Artists); err != nil {
return nil, err
}
out = append(out, g)
}
return out, rows.Err()
}
// GenreArtists returns the names of the artists with most playable songs among the genres, to show
// what a mix has: the names that stand for nobody ("Unknown Artist") are not shown.
func (r *MixRepo) GenreArtists(ctx context.Context, genres []string, limit int) ([]string, error) {
if len(genres) == 0 || limit <= 0 {
return []string{}, nil
}
vis, _ := visible("t", 0)
rows, err := r.db.QueryContext(ctx,
`SELECT a.name FROM track_genres g
JOIN tracks t ON t.id = g.track_id
JOIN track_artists ta ON ta.track_id = t.id AND ta.role = 'main'
JOIN artists a ON a.id = ta.artist_id
WHERE g.genre IN (?) AND a.name NOT IN ('Unknown Artist', 'Various Artists') AND `+vis+`
GROUP BY a.id, a.name ORDER BY COUNT(DISTINCT t.id) DESC, a.id LIMIT ?`, bun.In(genres), limit)
if err != nil {
return nil, fmt.Errorf("read genre artists: %w", err)
}
defer rows.Close()
out := []string{}
for rows.Next() {
var name string
if err := rows.Scan(&name); err != nil {
return nil, err
}
out = append(out, name)
}
return out, rows.Err()
}
// ArtistGenres returns, for each artist, how many of their playable songs have each genre.
func (r *MixRepo) ArtistGenres(ctx context.Context, artistIDs []int64) (map[int64]map[string]int, error) {
out := map[int64]map[string]int{}
if len(artistIDs) == 0 {
return out, nil
}
vis, _ := visible("t", 0)
rows, err := r.db.QueryContext(ctx,
`SELECT ta.artist_id, g.genre, COUNT(DISTINCT t.id)
FROM track_artists ta
JOIN tracks t ON t.id = ta.track_id
JOIN track_genres g ON g.track_id = t.id
WHERE ta.artist_id IN (?) AND ta.role = 'main' AND `+vis+`
GROUP BY ta.artist_id, g.genre`, bun.In(artistIDs))
if err != nil {
return nil, fmt.Errorf("read artist genres: %w", err)
}
defer rows.Close()
for rows.Next() {
var (
id int64
genre string
n int
)
if err := rows.Scan(&id, &genre, &n); err != nil {
return nil, err
}
if out[id] == nil {
out[id] = map[string]int{}
}
out[id][genre] = n
}
return out, rows.Err()
}
// Pool returns the playable songs that have one of the genres or are by one of the artists, with
// their main artist (the one with the lowest id when there are several), in the order of their ids.
func (r *MixRepo) Pool(ctx context.Context, f mixes.Filter) ([]mixes.TrackRef, error) {
var or []string
var args []any
if len(f.Genres) > 0 {
or = append(or, "EXISTS (SELECT 1 FROM track_genres g WHERE g.track_id = t.id AND g.genre IN (?))")
args = append(args, bun.In(f.Genres))
}
if len(f.ArtistIDs) > 0 {
or = append(or, "EXISTS (SELECT 1 FROM track_artists x WHERE x.track_id = t.id AND x.role = 'main' AND x.artist_id IN (?))")
args = append(args, bun.In(f.ArtistIDs))
}
if len(or) == 0 {
return []mixes.TrackRef{}, nil
}
vis, _ := visible("t", 0)
rows, err := r.db.QueryContext(ctx,
`SELECT t.id, COALESCE((SELECT MIN(ta.artist_id) FROM track_artists ta WHERE ta.track_id = t.id AND ta.role = 'main'), 0)
FROM tracks t WHERE `+vis+` AND (`+strings.Join(or, " OR ")+`) ORDER BY t.id`, args...)
if err != nil {
return nil, fmt.Errorf("read the pool: %w", err)
}
defer rows.Close()
out := []mixes.TrackRef{}
for rows.Next() {
var ref mixes.TrackRef
if err := rows.Scan(&ref.TrackID, &ref.ArtistID); err != nil {
return nil, err
}
out = append(out, ref)
}
return out, rows.Err()
}
// History returns what the user did with each song they did anything with. A play counts as a
// listen when it was completed or lasted minListen; a skip counts as a skip only when it came
// before the listener had heard that much (as in the taste model).
func (r *MixRepo) History(ctx context.Context, userID int64, minListen time.Duration) (map[int64]mixes.History, error) {
ms := minListen.Milliseconds()
rows, err := r.db.QueryContext(ctx,
`SELECT track_id,
SUM(CASE WHEN event_type = 'complete' OR duration_played_ms >= ? THEN 1 ELSE 0 END),
MAX(CASE WHEN event_type = 'complete' OR duration_played_ms >= ? THEN occurred_at END),
MAX(CASE WHEN event_type = 'skip' AND duration_played_ms < ? THEN occurred_at END)
FROM listening_events WHERE user_id = ? GROUP BY track_id`, ms, ms, ms, userID)
if err != nil {
return nil, fmt.Errorf("read history: %w", err)
}
defer rows.Close()
out := map[int64]mixes.History{}
for rows.Next() {
var (
id int64
plays int64
last, skipp sql.NullInt64
)
if err := rows.Scan(&id, &plays, &last, &skipp); err != nil {
return nil, err
}
h := mixes.History{Plays: int(plays)}
if last.Valid {
h.LastPlayed = time.Unix(last.Int64, 0)
}
if skipp.Valid {
h.SkippedAt = time.Unix(skipp.Int64, 0)
}
out[id] = h
}
return out, rows.Err()
}
// ArtistNames returns the names of the artists that exist.
func (r *MixRepo) ArtistNames(ctx context.Context, ids []int64) (map[int64]string, error) {
return r.taste.ArtistNames(ctx, ids)
}
// Tracks returns the playable songs among ids, in the order given (a song asked twice comes twice).
func (r *MixRepo) Tracks(ctx context.Context, ids []int64) ([]catalog.Track, error) {
if len(ids) == 0 {
return []catalog.Track{}, nil
}
vis, _ := visible("t", 0)
rows, err := r.db.QueryContext(ctx,
`SELECT t.id, t.title, COALESCE(t.duration_ms, 0), COALESCE(t.track_no, 0), COALESCE(t.disc_no, 0), t.album_id
FROM tracks t WHERE t.id IN (?) AND `+vis, bun.In(ids))
if err != nil {
return nil, fmt.Errorf("read songs: %w", err)
}
defer rows.Close()
byID := map[int64]catalog.Track{}
albumOf := map[int64]int64{}
var found []int64
page := catalog.Page[catalog.Track]{}
for rows.Next() {
var (
t catalog.Track
albumID sql.NullInt64
)
if err := rows.Scan(&t.ID, &t.Title, &t.DurationMs, &t.TrackNo, &t.DiscNo, &albumID); err != nil {
return nil, err
}
t.Artists = []catalog.ArtistRef{}
if albumID.Valid {
albumOf[t.ID] = albumID.Int64
}
page.Items = append(page.Items, t)
found = append(found, t.ID)
}
if err := rows.Err(); err != nil {
return nil, err
}
if len(found) > 0 {
if err := r.cat.attachTrackDetails(ctx, &page, found, albumOf, 0); err != nil {
return nil, err
}
}
for _, t := range page.Items {
byID[t.ID] = t
}
out := make([]catalog.Track, 0, len(ids))
for _, id := range ids {
if t, ok := byID[id]; ok {
out = append(out, t)
}
}
return out, nil
}

View file

@ -0,0 +1,270 @@
package storage
import (
"context"
"slices"
"testing"
"time"
"github.com/uptrace/bun"
"codeberg.org/squidd/sonda/internal/listening"
"codeberg.org/squidd/sonda/internal/mixes"
)
// mixEnv is a library with songs of known genres and artists, and a user.
type mixEnv struct {
t *testing.T
db *bun.DB
repo *MixRepo
user int64
tracks map[string]int64
artist map[string]int64
}
// mixSong is one file: its title, artists, genres.
type mixSong struct {
title, artist string
genres []string
}
func newMixEnv(t *testing.T, db *bun.DB, songs []mixSong) *mixEnv {
t.Helper()
ctx := context.Background()
u, err := NewUserRepo(db).CreateUser(ctx, "a@example.org", "hash", false)
if err != nil {
t.Fatal(err)
}
lib := NewLibraryRepo(db)
l, err := lib.CreateLibrary(ctx, "m", "/lib/m")
if err != nil {
t.Fatal(err)
}
e := &mixEnv{t: t, db: db, repo: NewMixRepo(db), user: u.ID, tracks: map[string]int64{}, artist: map[string]int64{}}
for i, s := range songs {
f := pf("x/"+s.title+".flac", s.title, "Album "+s.artist, s.artist, i+1, 1, "flac", s.artist)
f.Tags.Genres = s.genres
if _, err := lib.SaveFile(ctx, l.ID, f, 0); err != nil {
t.Fatal(err)
}
var id int64
if err := db.QueryRow("SELECT id FROM tracks WHERE title = ?", s.title).Scan(&id); err != nil {
t.Fatal(err)
}
e.tracks[s.title] = id
}
rows, err := db.Query("SELECT id, name FROM artists")
if err != nil {
t.Fatal(err)
}
defer rows.Close()
for rows.Next() {
var id int64
var name string
if err := rows.Scan(&id, &name); err != nil {
t.Fatal(err)
}
e.artist[name] = id
}
return e
}
func mixBackends(t *testing.T, songs []mixSong, fn func(t *testing.T, e *mixEnv)) {
forEachBackend(t, func(t *testing.T, db *bun.DB, _ string) { fn(t, newMixEnv(t, db, songs)) })
}
var mixSongs = []mixSong{
{"r1", "Queen", []string{"rock", "classic rock"}},
{"r2", "Queen", []string{"rock"}},
{"r3", "Kiss", []string{"rock"}},
{"m1", "Slayer", []string{"metal"}},
{"m2", "Slayer", []string{"metal"}},
{"j1", "Miles", []string{"jazz+funk"}},
{"n1", "Nobody", nil},
}
func TestMixRepoGenresCountSongsAndArtistsOfThePlayableOnes(t *testing.T) {
mixBackends(t, mixSongs, func(t *testing.T, e *mixEnv) {
got, err := e.repo.Genres(context.Background())
if err != nil {
t.Fatal(err)
}
by := map[string]mixes.Genre{}
for _, g := range got {
by[g.Name] = g
}
if g := by["rock"]; g.Tracks != 3 || g.Artists != 2 {
t.Errorf("rock = %+v, want 3 songs by 2 artists", g)
}
if g := by["metal"]; g.Tracks != 2 || g.Artists != 1 {
t.Errorf("metal = %+v", g)
}
if _, ok := by["jazz+funk"]; !ok {
t.Error("the spelling of the file is kept")
}
// A song whose files are gone is not counted.
if _, err := e.db.Exec("UPDATE audio_files SET missing_at = 1 WHERE track_id = ?", e.tracks["r3"]); err != nil {
t.Fatal(err)
}
got, _ = e.repo.Genres(context.Background())
for _, g := range got {
if g.Name == "rock" && (g.Tracks != 2 || g.Artists != 1) {
t.Errorf("rock after a song went = %+v", g)
}
}
})
}
func TestMixRepoPoolByGenreOrByArtistGivesPlayableSongsWithTheirArtist(t *testing.T) {
mixBackends(t, mixSongs, func(t *testing.T, e *mixEnv) {
ctx := context.Background()
ids := func(refs []mixes.TrackRef) []int64 {
out := make([]int64, len(refs))
for i, r := range refs {
out[i] = r.TrackID
}
slices.Sort(out)
return out
}
rock, err := e.repo.Pool(ctx, mixes.Filter{Genres: []string{"rock"}})
if err != nil {
t.Fatal(err)
}
want := []int64{e.tracks["r1"], e.tracks["r2"], e.tracks["r3"]}
slices.Sort(want)
if !slices.Equal(ids(rock), want) {
t.Errorf("rock pool = %v, want %v", ids(rock), want)
}
for _, r := range rock {
if r.TrackID == e.tracks["r3"] && r.ArtistID != e.artist["Kiss"] {
t.Errorf("artist of r3 = %d, want Kiss", r.ArtistID)
}
}
both, _ := e.repo.Pool(ctx, mixes.Filter{Genres: []string{"metal"}, ArtistIDs: []int64{e.artist["Miles"]}})
if len(both) != 3 {
t.Errorf("metal or Miles = %d songs, want 3", len(both))
}
if none, _ := e.repo.Pool(ctx, mixes.Filter{}); len(none) != 0 {
t.Errorf("an empty filter gave %d songs", len(none))
}
if _, err := e.db.Exec("UPDATE audio_files SET missing_at = 1 WHERE track_id = ?", e.tracks["r1"]); err != nil {
t.Fatal(err)
}
if again, _ := e.repo.Pool(ctx, mixes.Filter{Genres: []string{"rock"}}); len(again) != 2 {
t.Errorf("a song with no file is in the pool: %d songs", len(again))
}
})
}
func TestMixRepoGenreArtistsAreTheOnesWithMostSongs(t *testing.T) {
mixBackends(t, mixSongs, func(t *testing.T, e *mixEnv) {
got, err := e.repo.GenreArtists(context.Background(), []string{"rock", "metal"}, 2)
if err != nil || len(got) != 2 {
t.Fatalf("artists = %v err %v", got, err)
}
if got[0] != "Queen" && got[0] != "Slayer" { // both have two songs
t.Errorf("first = %q", got[0])
}
all, _ := e.repo.GenreArtists(context.Background(), []string{"rock"}, 10)
if !slices.Equal(all, []string{"Queen", "Kiss"}) {
t.Errorf("rock artists = %v, want Queen (2 songs) then Kiss", all)
}
})
}
func TestMixRepoGenreArtistsLeaveOutPlaceholderNames(t *testing.T) {
songs := append([]mixSong{}, mixSongs...)
songs = append(songs, mixSong{"u1", "Unknown Artist", []string{"rock"}}, mixSong{"u2", "Unknown Artist", []string{"rock"}},
mixSong{"u3", "Unknown Artist", []string{"rock"}}, mixSong{"v1", "Various Artists", []string{"rock"}})
mixBackends(t, songs, func(t *testing.T, e *mixEnv) {
got, err := e.repo.GenreArtists(context.Background(), []string{"rock"}, 10)
if err != nil || !slices.Equal(got, []string{"Queen", "Kiss"}) {
t.Errorf("rock artists = %v err %v: the names that stand for nobody are not artists to show", got, err)
}
})
}
func TestMixRepoArtistGenresCountTheSongsOfEachGenre(t *testing.T) {
mixBackends(t, mixSongs, func(t *testing.T, e *mixEnv) {
got, err := e.repo.ArtistGenres(context.Background(), []int64{e.artist["Queen"], e.artist["Miles"]})
if err != nil {
t.Fatal(err)
}
if g := got[e.artist["Queen"]]; g["rock"] != 2 || g["classic rock"] != 1 {
t.Errorf("Queen = %v", g)
}
if g := got[e.artist["Miles"]]; g["jazz+funk"] != 1 {
t.Errorf("Miles = %v", g)
}
})
}
func TestMixRepoHistoryCountsListensAndSkipsLikeTheTasteModel(t *testing.T) {
mixBackends(t, mixSongs, func(t *testing.T, e *mixEnv) {
ctx := context.Background()
rec := listening.NewService(NewListeningRepo(e.db), 20*time.Second)
t0 := time.Now().Add(-48 * time.Hour).Truncate(time.Second)
ev := func(title string, typ listening.EventType, playedMs int64, at time.Time, key string) {
t.Helper()
if _, err := rec.Record(ctx, listening.Event{UserID: e.user, TrackID: e.tracks[title], Type: typ, DurationPlayedMs: playedMs, OccurredAt: at, IdempotencyKey: key}); err != nil {
t.Fatalf("record %s: %v", key, err)
}
}
ev("r1", listening.Complete, 59000, t0, "a")
ev("r1", listening.Play, 30000, t0.Add(time.Hour), "b") // 30 s: a listen
ev("r1", listening.Play, 5000, t0.Add(2*time.Hour), "c") // 5 s: nothing
ev("r2", listening.Skip, 4000, t0, "d") // skipped before enough: a skip
ev("r3", listening.Skip, 40000, t0, "e") // skipped after 40 s: a listen, not a dislike
h, err := e.repo.History(ctx, e.user, 20*time.Second)
if err != nil {
t.Fatal(err)
}
r1 := h[e.tracks["r1"]]
if r1.Plays != 2 || !r1.LastPlayed.Equal(t0.Add(time.Hour)) || !r1.SkippedAt.IsZero() {
t.Errorf("r1 = %+v", r1)
}
if r2 := h[e.tracks["r2"]]; r2.Plays != 0 || !r2.SkippedAt.Equal(t0) {
t.Errorf("r2 = %+v", r2)
}
if r3 := h[e.tracks["r3"]]; r3.Plays != 1 || !r3.SkippedAt.IsZero() {
t.Errorf("r3 = %+v", r3)
}
if _, ok := h[e.tracks["m1"]]; ok {
t.Error("a song with no event is in the history")
}
if other, _ := e.repo.History(ctx, e.user+99, 20*time.Second); len(other) != 0 {
t.Errorf("another user has history: %v", other)
}
})
}
func TestMixRepoTracksComeInTheOrderAskedWithoutTheOnesThatCannotBePlayed(t *testing.T) {
mixBackends(t, mixSongs, func(t *testing.T, e *mixEnv) {
ctx := context.Background()
ask := []int64{e.tracks["m2"], e.tracks["r1"], 999999, e.tracks["j1"], e.tracks["r1"]}
got, err := e.repo.Tracks(ctx, ask)
if err != nil {
t.Fatal(err)
}
var titles []string
for _, tr := range got {
titles = append(titles, tr.Title)
}
if !slices.Equal(titles, []string{"m2", "r1", "j1", "r1"}) {
t.Errorf("titles = %v: the order asked, the unknown one left out, a repeat kept", titles)
}
if got[0].Album == nil || len(got[0].Artists) != 1 || got[0].Format != "flac" {
t.Errorf("details = %+v", got[0])
}
if _, err := e.db.Exec("UPDATE audio_files SET missing_at = 1 WHERE track_id = ?", e.tracks["m2"]); err != nil {
t.Fatal(err)
}
got, _ = e.repo.Tracks(ctx, ask)
if len(got) != 3 || got[0].Title != "r1" {
t.Errorf("a song with no file was not left out: %d songs", len(got))
}
if none, err := e.repo.Tracks(ctx, nil); err != nil || len(none) != 0 {
t.Errorf("nothing asked: %v %v", none, err)
}
})
}

View file

@ -443,6 +443,110 @@ export interface paths {
patch?: never;
trace?: never;
};
"/library/duplicates": {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
/**
* The songs that are in more than one file
* @description Admin only. Groups of files that are the same song by the same artists, worked out from the index and
* never from the disk, so it is quick and touches no file. Two files are the same song when they have the
* same set of main artists, the same title (the key Sonda matches titles by) and durations no more than
* 3 seconds apart, every pair in a group within that. A file with no duration or no artist never matches,
* a file flagged as missing does not count, and a copy may be on a different album: `files` says which.
*
* Each group says which copy Sonda would keep (`suggested_keep_track_id`): the one without loss (FLAC)
* before one with loss, then the higher bitrate, then the bigger file, then the oldest. `hashed` is true when
* every file of the group has been read by the check (`POST /library/duplicates/check`) and `identical` when,
* besides, they are byte for byte the same. No file path is ever returned.
*
* `deletion_allowed` says whether `POST /library/duplicates/clean` can work: with `library.allow_delete`
* off (the default) the groups can still be seen.
*/
get: operations["listDuplicates"];
put?: never;
post?: never;
delete?: never;
options?: never;
head?: never;
patch?: never;
trace?: never;
};
"/library/duplicates/check": {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
/**
* Progress of the check that reads the files of the duplicates
* @description Whether the check is running, how far it is (`progress`, only while it runs), and how the last one ended.
* The check is described at `POST /library/duplicates/check`.
*/
get: operations["getDuplicatesCheck"];
put?: never;
/**
* Read the files of the duplicates, to tell which are identical
* @description Admin only. Starts, in the background and one at a time like a scan, a job that reads every file that is in
* a group of copies and keeps its SHA-256, so that a group of files that are byte for byte the same can be
* called `identical`. It only reads: nothing is removed or sent anywhere. **It reads whole files**, which on a
* network drive takes a while, so it is started on purpose. A file that has a fingerprint is skipped, and the
* fingerprint of a file is forgotten when a scan finds the file changed; send `refresh=true` to read every file
* again. A file that changed since the scan is skipped. It gives up, and says why in `last_error`, when files
* cannot be read five times in a row (a disk that is not mounted). `DELETE /library/duplicates/check` stops it.
*/
post: operations["startDuplicatesCheck"];
/**
* Stop the check of the duplicates
* @description Admin only. Asks a running check to end after the file it is on; the fingerprints already kept stay. It
* answers the same when no check is running.
*/
delete: operations["stopDuplicatesCheck"];
options?: never;
head?: never;
patch?: never;
trace?: never;
};
"/library/duplicates/clean": {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
get?: never;
put?: never;
/**
* Delete the extra copies of one song
* @description Admin only. **Deletes files from the music folder.** It is the one thing Sonda does that writes there, so it
* is off until the operator turns on `library.allow_delete` (answer `deletion_disabled` otherwise), the folder
* has to be writable (`library_read_only`), and it is never done by itself: one request removes the copies
* `remove_track_ids` of one song and keeps `keep_track_id`.
*
* The server decides again, whatever the client saw, and removes nothing unless every check passes: no scan is
* running (`scan_running`); the tracks exist (`track_not_found`); they are copies of one song by the rule of
* `GET /library/duplicates` (`invalid_input` otherwise, and also when the copy to keep is among those to
* remove, a copy is named twice or there are none or more than 100); and the copy to keep and every copy to
* remove are still the file the last scan saw, a regular file with the same size and date (`duplicate_changed`
* for a file that changed, is gone, became a link or is outside the library: scan again). So the last copy of
* a song is never removed. Every file is checked before the first is removed.
*
* Each copy is removed from the disk through the library folder (a path cannot leave it, not even through a
* link) and its track leaves the index; **the listens of the removed track become listens of the track that
* stays**, so the taste is not lost. Folders are never removed, not even empty ones. If it fails part way (for
* example the folder turns out to be read-only for the second file), what was removed stays removed.
*/
post: operations["cleanDuplicates"];
delete?: never;
options?: never;
head?: never;
patch?: never;
trace?: never;
};
"/library/scan": {
parameters: {
query?: never;
@ -705,6 +809,88 @@ export interface paths {
patch?: never;
trace?: never;
};
"/mixes": {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
/**
* Your mixes
* @description Playlists that Sonda makes for you, from what you listen to and from the genres of your
* music. First the ones made for you (`favourites`, `rediscover`: they need some listening
* history, so they are missing without it; `unheard`, the songs you never played), then
* one `genre` mix for each genre of the library that has at least 30 songs by at least 4
* artists, the genres you play most first (and the biggest after them). Spellings of a genre
* that differ only by case or punctuation are one genre, and `other` and `unknown` are not
* genres. `title` is in English; a client may say it in its own words from `kind` and
* `genre`. Nothing is stored and nothing is sent to a provider: a mix is worked out from your
* own history and the genre tags of your files, for you alone.
*/
get: operations["listMixes"];
put?: never;
post?: never;
delete?: never;
options?: never;
head?: never;
patch?: never;
trace?: never;
};
"/mixes/{id}": {
parameters: {
query?: never;
header?: never;
path: {
/** @description `favourites`, `rediscover`, `unheard` or `genre:<name>` (percent-encoded). */
id: components["parameters"]["MixId"];
};
cookie?: never;
};
/**
* A mix and its songs
* @description Up to 50 songs, at most 3 by one artist and never two by the same artist in a row. They
* are picked by weighted chance: the artists you play weigh more, songs you never played
* a bit more, songs played in the last days less, and songs you skipped in the last two weeks
* and artists you avoid do not come at all. The same mix comes back all day: it changes
* the next day, or when you ask for another `variant`. The id of a genre mix has the genre
* in it (`genre:hard rock`), so it must be percent-encoded in the path. A mix you do not
* have (a genre that is too small, `favourites` with no history) is `mix_not_found`.
*/
get: operations["getMix"];
put?: never;
post?: never;
delete?: never;
options?: never;
head?: never;
patch?: never;
trace?: never;
};
"/mixes/{id}/playlist": {
parameters: {
query?: never;
header?: never;
path: {
/** @description `favourites`, `rediscover`, `unheard` or `genre:<name>` (percent-encoded). */
id: components["parameters"]["MixId"];
};
cookie?: never;
};
get?: never;
put?: never;
/**
* Keep a mix as a playlist
* @description Makes a playlist of your own with the songs that `variant` of the mix has now, in the
* same order. The playlist is a copy: it does not change afterwards, and is yours to edit.
* The body is optional.
*/
post: operations["saveMix"];
delete?: never;
options?: never;
head?: never;
patch?: never;
trace?: never;
};
"/playlists": {
parameters: {
query?: never;
@ -1892,6 +2078,38 @@ export interface components {
seeds: components["schemas"]["DiscoverSeed"][];
items: components["schemas"]["Suggestion"][];
};
Mix: {
/** @description `favourites`, `rediscover`, `unheard` or `genre:<name>`. */
id: string;
/** @enum {string} */
kind: "genre" | "favourites" | "rediscover" | "unheard";
/** @description In English. */
title: string;
/** @description The genre, for a genre mix; null for the others. */
genre: string | null;
/** @description A few of the artists it has, up to 3. */
artists: string[];
};
MixList: {
items: components["schemas"]["Mix"][];
};
MixWithTracks: {
id: string;
/** @enum {string} */
kind: "genre" | "favourites" | "rediscover" | "unheard";
title: string;
genre: string | null;
artists: string[];
variant: number;
/** @description In the order to play them. Fewer than 50 when the library has fewer to give. */
tracks: components["schemas"]["Track"][];
};
SaveMix: {
/** @description The name of the playlist; blank or absent means the title of the mix. */
name?: string;
/** @description The variant the client showed; absent means the mix of the day. */
variant?: number;
};
Playlist: {
/** Format: int64 */
id: number;
@ -2072,6 +2290,101 @@ export interface components {
/** @description Why the last job ended before the end, for example that a provider is off or kept failing. */
last_error?: string;
};
DuplicatesPage: {
/** @description Whether `POST /library/duplicates/clean` can work (`library.allow_delete`). */
deletion_allowed: boolean;
/** @description Groups in all, not in this page. */
total: number;
limit: number;
offset: number;
groups: components["schemas"]["DuplicateGroup"][];
};
DuplicateGroup: {
title: string;
/** @description The main artists the copies share. */
artists: components["schemas"]["ArtistRef"][];
/** @description The copies, by track id; at least two. */
files: components["schemas"]["DuplicateFile"][];
/** @description Every file of the group has been read by the check. */
hashed: boolean;
/** @description Hashed, and byte for byte the same. */
identical: boolean;
/**
* Format: int64
* @description The copy Sonda would keep: without loss first, then the higher bitrate, then the bigger file, then the oldest.
*/
suggested_keep_track_id: number;
/**
* Format: int64
* @description The size of every copy but the one to keep.
*/
freed_bytes: number;
};
DuplicateFile: {
/** Format: int64 */
track_id: number;
album: {
/** Format: int64 */
id: number;
title: string;
} | null;
/** @description `mp3`, `flac`... */
format: string;
/** @description 0 when unknown. */
bitrate: number;
/** Format: int64 */
size_bytes: number;
/** Format: int64 */
duration_ms: number;
/** @description This file has been read by the check. */
hashed: boolean;
};
CleanDuplicates: {
/**
* Format: int64
* @description The copy that stays.
*/
keep_track_id: number;
/** @description The copies to delete, of the same song as the one that stays, none twice. */
remove_track_ids: number[];
};
DuplicatesCleaned: {
/** @description Files deleted. */
removed: number;
/** Format: int64 */
freed_bytes: number;
};
DuplicatesCheckStatus: {
running: boolean;
/**
* Format: date-time
* @description When the last check started.
*/
started_at?: string;
/** Format: date-time */
last_finished_at?: string;
/** @description Only while the check runs. */
progress?: {
done: number;
/** @description Files the check set out to read. */
total: number;
};
/** @description How the last finished check ended. */
last?: {
/** @description Files looked at. */
checked: number;
/** @description Files read and fingerprinted. */
hashed: number;
/** @description Files that changed since the scan, or are gone. */
skipped: number;
/** @description Files that could not be read just then. */
failed: number;
/** @description It was stopped, or the application was. */
stopped: boolean;
};
/** @description Why the last check ended before the end, for example that the disk is not mounted. */
last_error?: string;
};
UserSettings: {
/**
* @description When true, Discover may start from the artists you listen to most. That sends their
@ -2142,6 +2455,15 @@ export interface components {
"application/json": components["schemas"]["Error"];
};
};
/** @description No such mix, or you do not have it (`mix_not_found`). */
MixNotFound: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["Error"];
};
};
/** @description No such playlist, or it is somebody else's (`playlist_not_found`). */
PlaylistNotFound: {
headers: {
@ -2194,6 +2516,8 @@ export interface components {
/** @description Page size. 50 when absent or 0; values above 200 are lowered to 200. */
Limit: number;
Offset: number;
/** @description `favourites`, `rediscover`, `unheard` or `genre:<name>` (percent-encoded). */
MixId: string;
/** @description Only items with a playable file in this library. */
LibraryId: number;
ArtistId: number;
@ -3063,6 +3387,170 @@ export interface operations {
403: components["responses"]["AdminRequired"];
};
};
listDuplicates: {
parameters: {
query?: {
/** @description Page size. 50 when absent or 0; values above 200 are lowered to 200. */
limit?: components["parameters"]["Limit"];
offset?: components["parameters"]["Offset"];
};
header?: never;
path?: never;
cookie?: never;
};
requestBody?: never;
responses: {
/** @description A page of groups, in a fixed order (by title). */
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["DuplicatesPage"];
};
};
400: components["responses"]["InvalidInput"];
401: components["responses"]["Unauthorized"];
403: components["responses"]["AdminRequired"];
};
};
getDuplicatesCheck: {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
requestBody?: never;
responses: {
/** @description The job. */
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["DuplicatesCheckStatus"];
};
};
401: components["responses"]["Unauthorized"];
403: components["responses"]["AdminRequired"];
};
};
startDuplicatesCheck: {
parameters: {
query?: {
/** @description `true` reads every file again, also those that have a fingerprint. `false` when absent. */
refresh?: boolean;
};
header?: never;
path?: never;
cookie?: never;
};
requestBody?: never;
responses: {
/** @description The check was started in the background. */
202: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": {
/** @enum {string} */
status: "started";
};
};
};
400: components["responses"]["InvalidInput"];
401: components["responses"]["Unauthorized"];
403: components["responses"]["AdminRequired"];
/** @description The check is already running (`duplicates_check_running`). */
409: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["Error"];
};
};
/** @description The groups could not be worked out (`internal`). */
500: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["Error"];
};
};
};
};
stopDuplicatesCheck: {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
requestBody?: never;
responses: {
/** @description Done, also when no check was running. */
204: {
headers: {
[name: string]: unknown;
};
content?: never;
};
401: components["responses"]["Unauthorized"];
403: components["responses"]["AdminRequired"];
};
};
cleanDuplicates: {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
requestBody: {
content: {
"application/json": components["schemas"]["CleanDuplicates"];
};
};
responses: {
/** @description The extra copies were removed. */
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["DuplicatesCleaned"];
};
};
400: components["responses"]["InvalidInput"];
401: components["responses"]["Unauthorized"];
403: components["responses"]["AdminRequired"];
/** @description A track that was named does not exist (`track_not_found`). */
404: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["Error"];
};
};
/**
* @description Deleting is not turned on (`deletion_disabled`), a scan is running (`scan_running`), a file is not what the last
* scan saw (`duplicate_changed`), or the music folder cannot be written to (`library_read_only`).
*/
409: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["Error"];
};
};
};
};
getScanStatus: {
parameters: {
query?: never;
@ -3386,6 +3874,96 @@ export interface operations {
};
};
};
listMixes: {
parameters: {
query?: never;
header?: never;
path?: never;
cookie?: never;
};
requestBody?: never;
responses: {
/** @description The mixes. */
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["MixList"];
};
};
401: components["responses"]["Unauthorized"];
};
};
getMix: {
parameters: {
query?: {
/** @description Another mix of the same kind, from 0 (the mix of the day) up. */
variant?: number;
};
header?: never;
path: {
/** @description `favourites`, `rediscover`, `unheard` or `genre:<name>` (percent-encoded). */
id: components["parameters"]["MixId"];
};
cookie?: never;
};
requestBody?: never;
responses: {
/** @description The mix. */
200: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["MixWithTracks"];
};
};
400: components["responses"]["InvalidInput"];
401: components["responses"]["Unauthorized"];
404: components["responses"]["MixNotFound"];
};
};
saveMix: {
parameters: {
query?: never;
header?: never;
path: {
/** @description `favourites`, `rediscover`, `unheard` or `genre:<name>` (percent-encoded). */
id: components["parameters"]["MixId"];
};
cookie?: never;
};
requestBody?: {
content: {
"application/json": components["schemas"]["SaveMix"];
};
};
responses: {
/** @description The playlist. */
201: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["Playlist"];
};
};
400: components["responses"]["InvalidInput"];
401: components["responses"]["Unauthorized"];
404: components["responses"]["MixNotFound"];
409: components["responses"]["PlaylistLimit"];
/** @description The mix has no songs to save (`mix_empty`). */
422: {
headers: {
[name: string]: unknown;
};
content: {
"application/json": components["schemas"]["Error"];
};
};
};
};
listPlaylists: {
parameters: {
query?: {