diff --git a/README.md b/README.md index d6f068e..3da66ee 100644 --- a/README.md +++ b/README.md @@ -28,6 +28,7 @@ It is a single executable, one database (SQLite by default, PostgreSQL if you pr - **Wishlist and "not interested".** `PUT /wishlist/{artist}` puts an artist on your wishlist (`GET /wishlist` lists it, `DELETE` takes it off), and `POST /discover/{artist}/dismiss` hides an artist from your Discover for good (`DELETE` brings it back, `GET /discover/dismissed` lists them). An artist is wished or dismissed, never both, and both lists are yours alone. Discover marks the wished artists it shows (`wishlisted`) and never suggests a dismissed one. The wishlist is a list for you and sends nothing anywhere; Sonda downloads nothing, but an admin can send an artist that Discover showed to their own Lidarr (see [External providers](#external-providers)), which works for an artist that has a MusicBrainz id and that Sonda knows of from a list or from your files. An artist that only Last.fm named is kept while you have it marked (your lists still show it, but other users cannot see that it exists), and goes with the rest of Last.fm's data if an admin removes Last.fm's data. Removing MusicBrainz's data does not touch it. - **Your own settings.** `GET` and `PATCH /me/settings` hold choices that belong to you, not to the server. Everyone starts private: `discover_from_listening` is off, so this setting never sends your listening to a provider. When you turn it on, Discover may start from the artists you listen to most, and their names are sent to Last.fm (only if an admin switched it on). What Last.fm answers is kept in the shared database like any other lookup, so another local user can see, from `retrieved_at` of `GET /artists/{id}/similar`, that someone looked up an artist, but not who. - **Importing from Spotify.** `POST /me/import/spotify` brings your listening history and your playlists from a Spotify data export into your own account. Ask Spotify for your data (Account, Privacy, "Extended streaming history" and "Account data") and send what it gives, as a multipart form with a `files` field for each file: the zip itself, or the JSON files in it (`Streaming_History_Audio_*.json`, the older `StreamingHistory_music_*.json` and `Playlist*.json`); a request may be up to 512 MB, and the rest of the export (the account, the searches, the library) is never opened. It all happens on your server and nothing is sent anywhere; the files are read from the request and not kept, and of each play only the song, the artist, the album, the time and how it ended are read, never the IP addresses, the country or the platform in the file. Of a play of a song you do not have that counts as listened, the name of the artist and the time are kept (and nothing else), so that Sonda can learn which artists you play together. A play is matched to a song of your library by artist and title (case, accents and punctuation do not matter, a `Remastered` note on either side does not, any credited artist will do, and a play of a compilation is matched by its album when Spotify says `Various Artists`). A match is recorded as a listen at the time of the play: `complete` when it ran to its end (or nine tenths were played), `skip` when Spotify says it was skipped or it lasted less than a listen, `play` otherwise; nothing is posted to Mastodon. Importing the same files again records nothing twice, and the same play in the extended and in the older history is one. A playlist becomes a playlist of yours with the songs that were found, in order; one whose name you already have is left alone, and one with none of its songs in your library is not made. The report counts what was added, what was already there and what is not in your library (and lists those artists, most played first), so you can see what is missing. One import of yours runs at a time. The report is kept: `GET /me/import/spotify` lists your past imports, newest first, with the numbers of each (never the files). The web app has an Import page for the form and for those numbers. +- **Suggestions from your own listening.** `GET /me/suggestions` learns which artists you play together: your history is split into sittings (plays less than 30 minutes apart) and two artists that turn up in the same sitting often are linked (a cosine over the sittings of each). It is a count over your own data, built on your server in a moment and again when plays are added, with no provider and nothing sent anywhere; it needs `discover_from_listening` in `PATCH /me/settings`. The model reads your listening events and the plays of artists you have no music of that an import kept (the artist and the time only). `library` are artists you have that go with what you play and that you did not play for two months; `missing` are artists you played at least three times and have no music of, the ones you play most that also go with what you play now first. Each says which of your artists it goes with. `POST /me/suggestions/wishlist` puts an artist you played on your wishlist by name (it is made without music and without a MusicBrainz id: Sonda never looks one up by name, so it cannot be sent to Lidarr from here). The web app has a "From your listening" tab in Discover. - **Sharing on Mastodon.** `PUT /me/mastodon` connects your own Mastodon account (the address of your instance and an access token that can only write posts); `GET` shows which account is connected and `DELETE` forgets it and the token. Nothing is connected by default. Once it is, each track of this server that counts as a listen is posted to your account as an unlisted status with the title, artists, album and `#NowPlaying`, and no link: at most one post a minute, never music of a friend. The token stays on the server and is never sent back. If the instance refuses the token on a post (it was revoked or expired), `GET /me/mastodon` says `problem: token_refused`, Settings shows it, and nothing is posted until you connect again with a new token. The address of the instance must be a domain name on https, and the server never connects to a private, loopback or link-local address for it, never follows a redirect and ignores proxy settings. - **Tags.** `GET /artists/{id}/tags` shows what Last.fm and MusicBrainz say kind of music an artist plays, each with its own weights, and the genres of the artist's own files, read offline. See [External providers](#external-providers). - **Similar artists.** With Last.fm switched on, `GET /artists/{id}/similar` lists artists similar to an artist, and says which of them you already have. See [External providers](#external-providers). Without it, Sonda makes no outbound network requests at all. diff --git a/cmd/sondad/main.go b/cmd/sondad/main.go index 597c2b1..66afb6f 100644 --- a/cmd/sondad/main.go +++ b/cmd/sondad/main.go @@ -15,6 +15,7 @@ import ( "go.uber.org/zap" "codeberg.org/squidd/sonda/internal/about" + "codeberg.org/squidd/sonda/internal/affinity" httpapi "codeberg.org/squidd/sonda/internal/api/http" "codeberg.org/squidd/sonda/internal/auth" "codeberg.org/squidd/sonda/internal/catalog" @@ -273,6 +274,7 @@ func run() error { tasteService := taste.NewService(tasteRepo, taste.Options{MinListen: cfg.Listening.MinListen(), HalfLife: cfg.Taste.HalfLife()}) userSettings := usersettings.NewService(storage.NewUserSettingsRepo(db)) feedbacks := feedback.NewService(storage.NewFeedbackRepo(db)) + affinityRepo := storage.NewAffinityRepo(db) weights := discover.Weights{ Affinity: cfg.Scoring.WeightAffinity, Genre: cfg.Scoring.WeightGenre, Preference: cfg.Scoring.WeightPreference, Discovery: cfg.Scoring.WeightDiscovery, @@ -312,6 +314,7 @@ func run() error { Discover: discover.NewService(tasteService, discoveries, tags, tasteRepo, userSettings, feedbacks, weights), Feedback: feedbacks, Playlists: playlistSvc, + Suggestions: affinity.NewService(affinityRepo, affinityRepo, feedbacks, affinityRepo, userSettings, cfg.Listening.MinListen()), SpotifyImport: spotifyimport.NewService(storage.NewCatalogRepo(db), storage.NewListeningRepo(db), playlistSvc, storage.NewSpotifyImportRepo(db), storage.NewOutsidePlayRepo(db), storage.SearchKey, cfg.Listening.MinListen()), Mixes: mixes.NewService(storage.NewMixRepo(db), tasteService, cfg.Listening.MinListen()), Smart: smart.NewService(storage.NewSmartRepo(db), tasteService, cfg.Listening.MinListen()), diff --git a/internal/affinity/model.go b/internal/affinity/model.go new file mode 100644 index 0000000..82ecea4 --- /dev/null +++ b/internal/affinity/model.go @@ -0,0 +1,154 @@ +// Package affinity learns which artists a person plays together, from their own listening: the plays are split into +// sittings, and two artists that often turn up in the same sitting are linked. It is a count over the person's own +// data, with no provider and no training service; it can be built again in a moment. +package affinity + +import ( + "math" + "sort" + "time" +) + +// Options are the knobs of the model. +type Options struct { + // Gap is the longest pause inside one sitting: a play that follows the one before by more than this starts another. + Gap time.Duration + // MaxArtists is how many different artists a sitting may have to teach anything: a sitting of a whole library on + // shuffle says nothing about what goes together. + MaxArtists int + // MinTogether is in how many sittings two artists must meet to be linked: once is chance. + MinTogether int + // SeedHalfLife is how long it takes a play to weigh half as much when the artists are ranked by how much they + // are played. It is long, because this is the whole history and not only the last months. + SeedHalfLife time.Duration +} + +// DefaultOptions are what Sonda uses. +func DefaultOptions() Options { + return Options{Gap: 30 * time.Minute, MaxArtists: 60, MinTogether: 2, SeedHalfLife: 2 * 365 * 24 * time.Hour} +} + +// Play is one play that counts as listened: the artist (as a key that is the same for the same artist wherever the +// play comes from) and when it ended. +type Play struct { + Key string + Name string + At time.Time +} + +// Artist is what the model knows of one artist. +type Artist struct { + Key string + Name string + Plays int + Sessions int // the sittings it was in that were used for the links + Last time.Time // its last play + Weight float64 // its plays, each worth 1 and halving every SeedHalfLife +} + +// Link says that two artists are played together. A is before B. Score is from 0 to 1: the sittings they share over +// the square root of the product of the sittings of each (a cosine), so a pair that is always together scores 1 +// and a pair of which one is everywhere scores low. +type Link struct { + A, B string + Score float64 +} + +// Model is what was learned. +type Model struct { + Artists []Artist // by key + Links []Link // by A, then B + Plays int // the plays read + Sessions int // the sittings they made +} + +type pair struct{ a, b string } + +// Build learns from the plays. now is the moment the age of a play is counted to. +func Build(plays []Play, now time.Time, o Options) Model { + sorted := make([]Play, 0, len(plays)) + for _, p := range plays { + if p.Key != "" { + sorted = append(sorted, p) + } + } + sort.SliceStable(sorted, func(i, j int) bool { return sorted[i].At.Before(sorted[j].At) }) + + artists := map[string]*Artist{} + use := func(p Play) *Artist { + a := artists[p.Key] + if a == nil { + a = &Artist{Key: p.Key, Name: p.Name} + artists[p.Key] = a + } + return a + } + sittings := map[string]int{} + together := map[pair]int{} + m := Model{Plays: len(sorted)} + + var current map[string]bool + closeSitting := func() { + if len(current) == 0 { + return + } + m.Sessions++ + if len(current) > o.MaxArtists { + return + } + keys := make([]string, 0, len(current)) + for k := range current { + keys = append(keys, k) + sittings[k]++ + } + sort.Strings(keys) + for i := range keys { + for j := i + 1; j < len(keys); j++ { + together[pair{keys[i], keys[j]}]++ + } + } + } + var previous time.Time + for i, p := range sorted { + if i == 0 || p.At.Sub(previous) > o.Gap { + closeSitting() + current = map[string]bool{} + } + previous = p.At + current[p.Key] = true + + a := use(p) + a.Plays++ + if p.At.After(a.Last) { + a.Last = p.At + } + age := now.Sub(p.At) + if age < 0 { + age = 0 // a play dated after now weighs as one that just happened + } + a.Weight += math.Exp2(-float64(age) / float64(o.SeedHalfLife)) + } + closeSitting() + + for k, n := range sittings { + artists[k].Sessions = n + } + for pr, n := range together { + if n < o.MinTogether { + continue + } + m.Links = append(m.Links, Link{A: pr.a, B: pr.b, Score: float64(n) / math.Sqrt(float64(sittings[pr.a])*float64(sittings[pr.b]))}) + } + sort.Slice(m.Links, func(i, j int) bool { + if m.Links[i].A != m.Links[j].A { + return m.Links[i].A < m.Links[j].A + } + return m.Links[i].B < m.Links[j].B + }) + m.Artists = make([]Artist, 0, len(artists)) + for _, a := range artists { + m.Artists = append(m.Artists, *a) + } + sort.Slice(m.Artists, func(i, j int) bool { return m.Artists[i].Key < m.Artists[j].Key }) + return m +} diff --git a/internal/affinity/model_test.go b/internal/affinity/model_test.go new file mode 100644 index 0000000..275b0e8 --- /dev/null +++ b/internal/affinity/model_test.go @@ -0,0 +1,151 @@ +package affinity + +import ( + "math" + "testing" + "time" +) + +var t0 = time.Date(2023, 3, 1, 9, 0, 0, 0, time.UTC) + +// at builds a play of an artist, minutes after t0. +func at(artist string, minutes int) Play { + return Play{Key: artist, Name: artist, At: t0.Add(time.Duration(minutes) * time.Minute)} +} + +// sittings are plays of the given artists in order, one minute apart, starting a day after the day before. +func sitting(day int, artists ...string) []Play { + var out []Play + for i, a := range artists { + out = append(out, at(a, day*24*60+i)) + } + return out +} + +func link(m Model, a, b string) (float64, bool) { + for _, l := range m.Links { + if l.A == a && l.B == b { + return l.Score, true + } + } + return 0, false +} + +func TestArtistsPlayedInTheSameSittingAreLinked(t *testing.T) { + var plays []Play + for day := 0; day < 3; day++ { + plays = append(plays, sitting(day, "blur", "oasis")...) + } + m := Build(plays, t0.AddDate(0, 0, 10), DefaultOptions()) + score, ok := link(m, "blur", "oasis") + if !ok || math.Abs(score-1) > 1e-9 { + t.Fatalf("link = %v, %v, want 1 (always together)", score, ok) + } + if len(m.Links) != 1 || m.Sessions != 3 || m.Plays != 6 { + t.Errorf("model = %d links, %d sittings, %d plays, want 1, 3, 6", len(m.Links), m.Sessions, m.Plays) + } +} + +func TestTheLinkIsTheCosineOfTheSittingsOfTheTwo(t *testing.T) { + var plays []Play + plays = append(plays, sitting(0, "a", "b")...) + plays = append(plays, sitting(1, "a", "b")...) + plays = append(plays, sitting(2, "a")...) + plays = append(plays, sitting(3, "a")...) + plays = append(plays, sitting(4, "a", "c")...) // a is in five sittings, b in two, together twice + m := Build(plays, t0.AddDate(0, 0, 10), DefaultOptions()) + want := 2 / math.Sqrt(5*2) + if score, ok := link(m, "a", "b"); !ok || math.Abs(score-want) > 1e-9 { + t.Errorf("a-b = %v, %v, want %v", score, ok, want) + } + if _, ok := link(m, "a", "c"); ok { + t.Error("a and c were together once: not enough to be linked") + } +} + +func TestAPauseLongerThanTheGapStartsAnotherSitting(t *testing.T) { + o := DefaultOptions() + gap := int(o.Gap / time.Minute) + together := []Play{at("a", 0), at("b", gap)} // exactly the gap: still one sitting + apart := []Play{at("a", 1000), at("b", 1000+gap+1)} // one minute more: two + m := Build(append(together, apart...), t0.AddDate(0, 0, 10), Options{Gap: o.Gap, MaxArtists: o.MaxArtists, MinTogether: 1, SeedHalfLife: o.SeedHalfLife}) + if m.Sessions != 3 { + t.Fatalf("%d sittings, want 3", m.Sessions) + } + if score, ok := link(m, "a", "b"); !ok || math.Abs(score-1/math.Sqrt(2*2)) > 1e-9 { + t.Errorf("a-b = %v, %v, want together in one of the two sittings each is in", score, ok) + } +} + +func TestAnArtistRepeatedInASittingCountsOnce(t *testing.T) { + var plays []Play + for day := 0; day < 2; day++ { + plays = append(plays, sitting(day, "a", "a", "a", "b")...) + } + m := Build(plays, t0.AddDate(0, 0, 10), DefaultOptions()) + if score, ok := link(m, "a", "b"); !ok || math.Abs(score-1) > 1e-9 { + t.Errorf("a-b = %v, %v, want 1", score, ok) + } + for _, a := range m.Artists { + if a.Key == "a" && (a.Plays != 6 || a.Sessions != 2) { + t.Errorf("a = %+v, want 6 plays in 2 sittings", a) + } + } +} + +func TestAMarathonOfManyArtistsTeachesNothing(t *testing.T) { + o := DefaultOptions() + o.MaxArtists = 3 + var names []string + for _, n := range []string{"a", "b", "c", "d"} { + names = append(names, n) + } + var plays []Play + for day := 0; day < 3; day++ { + plays = append(plays, sitting(day, names...)...) + } + m := Build(plays, t0.AddDate(0, 0, 10), o) + if len(m.Links) != 0 { + t.Errorf("%d links from sittings of more artists than the limit, want none", len(m.Links)) + } + if m.Plays != 12 { + t.Errorf("%d plays, want all 12 still counted", m.Plays) + } +} + +func TestRecentPlaysWeighMoreAndTheFutureIsNow(t *testing.T) { + now := t0.AddDate(3, 0, 0) + o := DefaultOptions() + plays := []Play{ + {Key: "old", Name: "Old", At: now.AddDate(0, 0, -int(o.SeedHalfLife/(24*time.Hour)))}, // one half-life ago + {Key: "new", Name: "New", At: now}, + {Key: "future", Name: "Future", At: now.Add(48 * time.Hour)}, + } + m := Build(plays, now, o) + w := map[string]float64{} + for _, a := range m.Artists { + w[a.Key] = a.Weight + } + if math.Abs(w["old"]-0.5) > 0.01 || math.Abs(w["new"]-1) > 1e-9 || math.Abs(w["future"]-1) > 1e-9 { + t.Errorf("weights = %v, want old 0.5, new 1, future 1", w) + } +} + +func TestAnArtistKnowsItsNameItsLastPlayAndItsOrderIsStable(t *testing.T) { + plays := []Play{{Key: "b", Name: "Björk", At: t0}, {Key: "a", Name: "Air", At: t0.Add(time.Hour)}, {Key: "b", Name: "BJORK", At: t0.Add(48 * time.Hour)}, {Key: "", Name: "nobody", At: t0}} + m := Build(plays, t0.AddDate(0, 0, 10), DefaultOptions()) + if len(m.Artists) != 2 || m.Artists[0].Key != "a" || m.Artists[1].Key != "b" { + t.Fatalf("artists = %+v, want a then b, and none without a key", m.Artists) + } + b := m.Artists[1] + if b.Name != "Björk" || b.Plays != 2 || !b.Last.Equal(t0.Add(48*time.Hour)) { + t.Errorf("b = %+v, want the first name seen, 2 plays and the last time", b) + } +} + +func TestNoPlaysMakeAnEmptyModel(t *testing.T) { + m := Build(nil, t0, DefaultOptions()) + if len(m.Artists) != 0 || len(m.Links) != 0 || m.Plays != 0 || m.Sessions != 0 { + t.Errorf("model = %+v, want empty", m) + } +} diff --git a/internal/affinity/service.go b/internal/affinity/service.go new file mode 100644 index 0000000..7e0c1f1 --- /dev/null +++ b/internal/affinity/service.go @@ -0,0 +1,414 @@ +package affinity + +import ( + "context" + "errors" + "fmt" + "math" + "sort" + "strings" + "sync" + "time" + + "golang.org/x/sync/singleflight" + + "codeberg.org/squidd/sonda/internal/feedback" + "codeberg.org/squidd/sonda/internal/usersettings" +) + +// Limits and rules. +const ( + DefaultLimit = 20 + MaxLimit = 50 + + // seedCount is how many of the most played artists the suggestions start from. + seedCount = 40 + // poolCount is how many of the most played artists are considered at all, besides the ones linked to a seed. + poolCount = 300 + // recentDays: an artist of the library played in this many days is not "forgotten", so it is not suggested. + recentDays = 60 + // minPlays is how many times an artist that is not in the library must have been played to be worth getting. + minPlays = 3 + + // staleAfter and bigChange say when the model is built again: when plays were added and the last build is older + // than staleAfter, or at once when as many as bigChange were added (an import). + staleAfter = 10 * time.Minute + bigChange = 100 +) + +// Errors of the service. +var ( + // ErrNotAllowed means the listener did not allow Discover to start from their listening. + ErrNotAllowed = errors.New("allow Discover to start from your listening to get suggestions") + // ErrInvalidLimit means a negative limit. + ErrInvalidLimit = errors.New("the limit cannot be negative") + // ErrUnknownArtist means the name is not an artist the listener has played. + ErrUnknownArtist = errors.New("that is not an artist you have played") +) + +// Built says when and from what the model was built. +type Built struct { + BuiltAt time.Time + Plays int + Sessions int + Artists int + Links int +} + +// LibraryArtist is an artist that exists on this server. +type LibraryArtist struct { + ID int64 + Name string + Playable bool // there is music of it to play +} + +// Store reads the plays of a person and keeps what was learned from them. +type Store interface { + Plays(ctx context.Context, userID int64, minPlayed time.Duration) ([]Play, error) + PlayCount(ctx context.Context, userID int64, minPlayed time.Duration) (int, error) + Save(ctx context.Context, userID int64, at time.Time, m Model) error + Build(ctx context.Context, userID int64) (Built, bool, error) + Seeds(ctx context.Context, userID int64, limit int) ([]Artist, error) + Artists(ctx context.Context, userID int64, keys []string) (map[string]Artist, error) + Neighbours(ctx context.Context, userID int64, keys []string) ([]Link, error) +} + +// Library says which of the artists exist on this server, by search key. +type Library interface { + LibraryArtists(ctx context.Context, keys []string) (map[string]LibraryArtist, error) +} + +// Marks reads what the listener decided about artists. +type Marks interface { + Marks(ctx context.Context, userID int64) (map[int64]feedback.Kind, error) +} + +// Wishes puts an artist on the wishlist by name and gives its id. +type Wishes interface { + Wishlist(ctx context.Context, userID int64, name string, at time.Time) (int64, error) +} + +// Settings reads the listener's own settings. +type Settings interface { + Get(ctx context.Context, userID int64) (usersettings.Settings, error) +} + +// Reason is an artist the listener plays, because of which another is suggested. +type Reason struct { + Name string + Score float64 +} + +// Suggestion is an artist put forward. +type Suggestion struct { + key string + Name string + ArtistID int64 // 0 when no artist of this name exists on this server yet + Score float64 + Plays int + LastPlayed time.Time + Wishlisted bool + Because []Reason +} + +// Suggestions is the answer. +type Suggestions struct { + // Library are artists the listener has, linked to what they play now, not played for a while. + Library []Suggestion + // Missing are artists the listener plays that they have no music of, the ones they play most and that go with + // what they play now first. + Missing []Suggestion + Built Built +} + +// Service learns from the listening of a person and makes suggestions. +type Service struct { + store Store + library Library + marks Marks + wishes Wishes + settings Settings + minListen time.Duration + opts Options + + now func() time.Time + flight singleflight.Group + mu sync.Mutex +} + +// NewService builds a Service. minListen is how long a play must last to count as a listen. +func NewService(store Store, lib Library, marks Marks, wishes Wishes, settings Settings, minListen time.Duration) *Service { + return &Service{store: store, library: lib, marks: marks, wishes: wishes, settings: settings, minListen: minListen, opts: DefaultOptions(), now: time.Now} +} + +// Rebuild learns again from every play of the person, and keeps it. +func (s *Service) Rebuild(ctx context.Context, userID int64) (Built, error) { + v, err, _ := s.flight.Do(fmt.Sprint(userID), func() (any, error) { return s.rebuild(ctx, userID) }) + if err != nil { + return Built{}, err + } + return v.(Built), nil +} + +func (s *Service) rebuild(ctx context.Context, userID int64) (Built, error) { + plays, err := s.store.Plays(ctx, userID, s.minListen) + if err != nil { + return Built{}, err + } + at := s.now() + m := Build(plays, at, s.opts) + if err := s.store.Save(ctx, userID, at, m); err != nil { + return Built{}, err + } + return Built{BuiltAt: at, Plays: m.Plays, Sessions: m.Sessions, Artists: len(m.Artists), Links: len(m.Links)}, nil +} + +// ensure builds the model when there is none or when plays were added since. +func (s *Service) ensure(ctx context.Context, userID int64) (Built, error) { + have, ok, err := s.store.Build(ctx, userID) + if err != nil { + return Built{}, err + } + if !ok { + return s.Rebuild(ctx, userID) + } + n, err := s.store.PlayCount(ctx, userID, s.minListen) + if err != nil { + return Built{}, err + } + changed := n - have.Plays + if changed < 0 { + changed = -changed + } + if changed >= bigChange || (changed > 0 && s.now().Sub(have.BuiltAt) > staleAfter) { + return s.Rebuild(ctx, userID) + } + return have, nil +} + +// Suggest makes the suggestions for a person. It needs them to have allowed Discover to start from their listening. +func (s *Service) Suggest(ctx context.Context, userID int64, limit int) (Suggestions, error) { + switch { + case limit < 0: + return Suggestions{}, ErrInvalidLimit + case limit == 0: + limit = DefaultLimit + case limit > MaxLimit: + limit = MaxLimit + } + set, err := s.settings.Get(ctx, userID) + if err != nil { + return Suggestions{}, err + } + if !set.DiscoverFromListening { + return Suggestions{}, ErrNotAllowed + } + build, err := s.ensure(ctx, userID) + if err != nil { + return Suggestions{}, err + } + out := Suggestions{Library: []Suggestion{}, Missing: []Suggestion{}, Built: build} + + pool, err := s.store.Seeds(ctx, userID, poolCount) + if err != nil { + return Suggestions{}, err + } + if len(pool) == 0 { + return out, nil + } + seeds := pool[:min(seedCount, len(pool))] + weight := map[string]float64{} + keys := make([]string, 0, len(seeds)) + for _, a := range seeds { + weight[a.Key] = math.Log1p(a.Weight) + keys = append(keys, a.Key) + } + links, err := s.store.Neighbours(ctx, userID, keys) + if err != nil { + return Suggestions{}, err + } + + type reason struct { + key string + score float64 + } + affinity := map[string]float64{} + because := map[string][]reason{} + link := func(seed, other string, score float64) { + w, ok := weight[seed] + if !ok || seed == other { + return + } + affinity[other] += w * score + because[other] = append(because[other], reason{seed, w * score}) + } + for _, l := range links { + link(l.A, l.B, l.Score) + link(l.B, l.A, l.Score) + } + + stats := map[string]Artist{} + for _, a := range pool { + stats[a.Key] = a + } + var unknown []string + for k := range affinity { + if _, ok := stats[k]; !ok { + unknown = append(unknown, k) + } + } + extra, err := s.store.Artists(ctx, userID, unknown) + if err != nil { + return Suggestions{}, err + } + for k, a := range extra { + stats[k] = a + } + + all := make([]string, 0, len(stats)) + for k := range stats { + all = append(all, k) + } + sort.Strings(all) + inLibrary, err := s.library.LibraryArtists(ctx, all) + if err != nil { + return Suggestions{}, err + } + marks, err := s.marks.Marks(ctx, userID) + if err != nil { + return Suggestions{}, err + } + + names := func(key string) string { + if a, ok := stats[key]; ok { + return a.Name + } + return key + } + make1 := func(key string) Suggestion { + a := stats[key] + sg := Suggestion{key: key, Name: a.Name, Plays: a.Plays, LastPlayed: a.Last} + if la, ok := inLibrary[key]; ok { + sg.ArtistID = la.ID + sg.Name = la.Name + sg.Wishlisted = marks[la.ID] == feedback.Wishlist + } + rs := because[key] + sort.Slice(rs, func(i, j int) bool { + if rs[i].score != rs[j].score { + return rs[i].score > rs[j].score + } + return rs[i].key < rs[j].key + }) + for _, r := range rs[:min(3, len(rs))] { + sg.Because = append(sg.Because, Reason{Name: names(r.key), Score: r.score}) + } + return sg + } + dismissed := func(key string) bool { + la, ok := inLibrary[key] + return ok && marks[la.ID] == feedback.Dismissed + } + + // The artists of the library that go with what is played now and have not been played for a while. + recent := s.now().AddDate(0, 0, -recentDays) + var lib []Suggestion + maxAff := 0.0 + for _, key := range all { + la, ok := inLibrary[key] + if !ok || !la.Playable || affinity[key] <= 0 || dismissed(key) || stats[key].Last.After(recent) { + continue + } + maxAff = math.Max(maxAff, affinity[key]) + sg := make1(key) + sg.Score = affinity[key] + lib = append(lib, sg) + } + for i := range lib { + lib[i].Score /= maxAff + } + out.Library = top(lib, limit) + + // The artists that are played and are not in the library: how much they are played and how well they go with + // what is played now, half and half. + var away []Suggestion + maxOwn, maxAway := 0.0, 0.0 + for _, key := range all { + if la, ok := inLibrary[key]; ok && la.Playable { + continue + } + if stats[key].Plays < minPlays || dismissed(key) { + continue + } + maxOwn = math.Max(maxOwn, math.Log1p(stats[key].Weight)) + maxAway = math.Max(maxAway, affinity[key]) + away = append(away, make1(key)) + } + for i := range away { + key := away[i].key + own := math.Log1p(stats[key].Weight) / maxOwn + with := 0.0 + if maxAway > 0 { + with = affinity[key] / maxAway + } + away[i].Score = 0.5*own + 0.5*with + } + out.Missing = top(away, limit) + return out, nil +} + +// top keeps the best of the suggestions, the highest score first, then the most played, then by name. +func top(in []Suggestion, limit int) []Suggestion { + sort.SliceStable(in, func(i, j int) bool { + a, b := in[i], in[j] + switch { + case a.Score != b.Score: + return a.Score > b.Score + case a.Plays != b.Plays: + return a.Plays > b.Plays + default: + return a.Name < b.Name + } + }) + if len(in) > limit { + in = in[:limit] + } + if in == nil { + in = []Suggestion{} + } + return in +} + +// AddToWishlist puts an artist the listener has played on their wishlist, by name, and gives its id. +func (s *Service) AddToWishlist(ctx context.Context, userID int64, name string) (int64, error) { + name = strings.TrimSpace(name) + if name == "" { + return 0, ErrUnknownArtist + } + set, err := s.settings.Get(ctx, userID) + if err != nil { + return 0, err + } + if !set.DiscoverFromListening { + return 0, ErrNotAllowed + } + if _, err := s.ensure(ctx, userID); err != nil { + return 0, err + } + found := false + seeds, err := s.store.Seeds(ctx, userID, math.MaxInt32) + if err != nil { + return 0, err + } + for _, a := range seeds { + if a.Name == name { + found = true + name = a.Name + break + } + } + if !found { + return 0, ErrUnknownArtist + } + return s.wishes.Wishlist(ctx, userID, name, s.now()) +} diff --git a/internal/affinity/service_test.go b/internal/affinity/service_test.go new file mode 100644 index 0000000..f8f24d6 --- /dev/null +++ b/internal/affinity/service_test.go @@ -0,0 +1,302 @@ +package affinity + +import ( + "context" + "errors" + "sort" + "testing" + "time" + + "codeberg.org/squidd/sonda/internal/feedback" + "codeberg.org/squidd/sonda/internal/usersettings" +) + +var now = time.Date(2026, 11, 15, 12, 0, 0, 0, time.UTC) + +// memStore keeps the model in memory and counts how often it is built. +type memStore struct { + plays []Play + model Model + built Built + has bool + reads int +} + +func (m *memStore) Plays(context.Context, int64, time.Duration) ([]Play, error) { + m.reads++ + return m.plays, nil +} +func (m *memStore) PlayCount(context.Context, int64, time.Duration) (int, error) { + return len(m.plays), nil +} +func (m *memStore) Save(_ context.Context, _ int64, at time.Time, model Model) error { + m.model, m.has = model, true + m.built = Built{BuiltAt: at, Plays: model.Plays, Sessions: model.Sessions, Artists: len(model.Artists), Links: len(model.Links)} + return nil +} +func (m *memStore) Build(context.Context, int64) (Built, bool, error) { return m.built, m.has, nil } +func (m *memStore) Seeds(_ context.Context, _ int64, limit int) ([]Artist, error) { + out := append([]Artist(nil), m.model.Artists...) + sort.Slice(out, func(i, j int) bool { + if out[i].Weight != out[j].Weight { + return out[i].Weight > out[j].Weight + } + return out[i].Key < out[j].Key + }) + return out[:min(limit, len(out))], nil +} +func (m *memStore) Artists(_ context.Context, _ int64, keys []string) (map[string]Artist, error) { + out := map[string]Artist{} + for _, a := range m.model.Artists { + for _, k := range keys { + if a.Key == k { + out[k] = a + } + } + } + return out, nil +} +func (m *memStore) Neighbours(_ context.Context, _ int64, keys []string) ([]Link, error) { + var out []Link + for _, l := range m.model.Links { + for _, k := range keys { + if l.A == k || l.B == k { + out = append(out, l) + break + } + } + } + return out, nil +} + +type fakeLibrary map[string]LibraryArtist + +func (f fakeLibrary) LibraryArtists(_ context.Context, keys []string) (map[string]LibraryArtist, error) { + out := map[string]LibraryArtist{} + for _, k := range keys { + if a, ok := f[k]; ok { + out[k] = a + } + } + return out, nil +} + +type fakeMarks map[int64]feedback.Kind + +func (f fakeMarks) Marks(context.Context, int64) (map[int64]feedback.Kind, error) { return f, nil } + +type fakeWishes struct { + names []string + id int64 +} + +func (f *fakeWishes) Wishlist(_ context.Context, _ int64, name string, _ time.Time) (int64, error) { + f.names = append(f.names, name) + return f.id, nil +} + +type fakeSettings struct{ on bool } + +func (f fakeSettings) Get(context.Context, int64) (usersettings.Settings, error) { + return usersettings.Settings{DiscoverFromListening: f.on}, nil +} + +// ago is a moment this many days before now, and the sitting of artists played one after the other from there. +func ago(days int, artists ...string) []Play { + var out []Play + for i, a := range artists { + out = append(out, Play{Key: a, Name: a, At: now.AddDate(0, 0, -days).Add(time.Duration(i) * time.Minute)}) + } + return out +} + +type rig struct { + svc *Service + store *memStore + wishes *fakeWishes + marks fakeMarks +} + +// newRig plays out this history: three old sittings of radiohead, pixies and canali; two recent ones of radiohead and +// placebo; one play of an artist met once. pixies and placebo are in the library, canali and oneoff are not. +func newRig(on bool) rig { + var plays []Play + for _, d := range []int{200, 199, 198} { + plays = append(plays, ago(d, "radiohead", "pixies", "canali")...) + } + for _, d := range []int{10, 9} { + plays = append(plays, ago(d, "radiohead", "placebo")...) + } + plays = append(plays, ago(150, "oneoff")...) + store := &memStore{plays: plays} + lib := fakeLibrary{ + "radiohead": {ID: 1, Name: "Radiohead", Playable: true}, + "pixies": {ID: 2, Name: "Pixies", Playable: true}, + "placebo": {ID: 3, Name: "Placebo", Playable: true}, + } + wishes := &fakeWishes{id: 77} + marks := fakeMarks{} + svc := NewService(store, lib, marks, wishes, fakeSettings{on}, 20*time.Second) + svc.now = func() time.Time { return now } + return rig{svc, store, wishes, marks} +} + +func names(in []Suggestion) []string { + out := []string{} + for _, s := range in { + out = append(out, s.Name) + } + return out +} + +func equal(a, b []string) bool { + if len(a) != len(b) { + return false + } + for i := range a { + if a[i] != b[i] { + return false + } + } + return true +} + +func TestSuggestionsNeedTheListenerToAllowListening(t *testing.T) { + r := newRig(false) + if _, err := r.svc.Suggest(context.Background(), 1, 0); !errors.Is(err, ErrNotAllowed) { + t.Fatalf("err = %v, want ErrNotAllowed", err) + } + if r.store.reads != 0 { + t.Error("a refused request read the plays") + } + if _, err := r.svc.AddToWishlist(context.Background(), 1, "canali"); !errors.Is(err, ErrNotAllowed) { + t.Errorf("wishlist err = %v, want ErrNotAllowed", err) + } +} + +func TestTheLibraryListHasTheArtistsThatGoWithWhatIsPlayedAndWereNotPlayedLately(t *testing.T) { + r := newRig(true) + got, err := r.svc.Suggest(context.Background(), 1, 0) + if err != nil { + t.Fatal(err) + } + // placebo goes with radiohead too, but it was played ten days ago; radiohead itself was too. + if !equal(names(got.Library), []string{"Pixies"}) { + t.Fatalf("library = %v, want only Pixies", names(got.Library)) + } + p := got.Library[0] + if p.ArtistID != 2 || p.Score != 1 || p.Plays != 3 || !p.LastPlayed.Equal(now.AddDate(0, 0, -198).Add(time.Minute)) { + t.Errorf("Pixies = %+v", p) + } + if len(p.Because) == 0 || p.Because[0].Name == "Pixies" { + t.Errorf("because = %+v, want the artists it goes with", p.Because) + } +} + +func TestTheMissingListHasTheArtistsPlayedThatAreNotInTheLibrary(t *testing.T) { + r := newRig(true) + got, err := r.svc.Suggest(context.Background(), 1, 0) + if err != nil { + t.Fatal(err) + } + // oneoff was played once: not worth getting. Radiohead, Pixies and Placebo are in the library. + if !equal(names(got.Missing), []string{"canali"}) { + t.Fatalf("missing = %v, want only canali", names(got.Missing)) + } + c := got.Missing[0] + if c.ArtistID != 0 || c.Wishlisted || c.Plays != 3 || c.Score <= 0 || c.Score > 1 { + t.Errorf("canali = %+v", c) + } +} + +func TestAMissingArtistThatIsOnTheWishlistSaysSoAndOneNotWantedIsLeftOut(t *testing.T) { + r := newRig(true) + lib := fakeLibrary{ + "radiohead": {ID: 1, Name: "Radiohead", Playable: true}, + "pixies": {ID: 2, Name: "Pixies", Playable: true}, + "canali": {ID: 9, Name: "Giorgio Canali", Playable: false}, // a row without music + } + r.svc.library = lib + r.marks[9] = feedback.Wishlist + got, _ := r.svc.Suggest(context.Background(), 1, 0) + if len(got.Missing) != 1 || got.Missing[0].ArtistID != 9 || !got.Missing[0].Wishlisted || got.Missing[0].Name != "Giorgio Canali" { + t.Fatalf("missing = %+v, want the row 9 on the wishlist", got.Missing) + } + r.marks[9] = feedback.Dismissed + r.marks[2] = feedback.Dismissed + got, _ = r.svc.Suggest(context.Background(), 1, 0) + if len(got.Missing) != 0 || len(got.Library) != 0 { + t.Errorf("after dismissing: library %v, missing %v, want nothing", names(got.Library), names(got.Missing)) + } +} + +func TestTheModelIsBuiltOnceAndAgainWhenPlaysAreAdded(t *testing.T) { + r := newRig(true) + ctx := context.Background() + for i := 0; i < 3; i++ { + if _, err := r.svc.Suggest(ctx, 1, 0); err != nil { + t.Fatal(err) + } + } + if r.store.reads != 1 { + t.Fatalf("plays read %d times for three requests, want 1", r.store.reads) + } + + // A few plays, soon after: the model is left alone. + r.store.plays = append(r.store.plays, ago(0, "radiohead")...) + r.svc.now = func() time.Time { return now.Add(time.Minute) } + _, _ = r.svc.Suggest(ctx, 1, 0) + if r.store.reads != 1 { + t.Errorf("read %d times after one new play a minute later, want 1", r.store.reads) + } + // The same, but the model is old now. + r.svc.now = func() time.Time { return now.Add(time.Hour) } + _, _ = r.svc.Suggest(ctx, 1, 0) + if r.store.reads != 2 { + t.Errorf("read %d times after an hour, want 2", r.store.reads) + } + // An import adds many at once: at once. + for i := 0; i < bigChange; i++ { + r.store.plays = append(r.store.plays, ago(300+i, "radiohead", "pixies")...) + } + _, _ = r.svc.Suggest(ctx, 1, 0) + if r.store.reads != 3 { + t.Errorf("read %d times after a big import, want 3", r.store.reads) + } +} + +func TestTheLimitIsAppliedAndBadOnesRefused(t *testing.T) { + r := newRig(true) + if _, err := r.svc.Suggest(context.Background(), 1, -1); !errors.Is(err, ErrInvalidLimit) { + t.Errorf("err = %v, want ErrInvalidLimit", err) + } + got, err := r.svc.Suggest(context.Background(), 1, 1) + if err != nil || len(got.Library) > 1 || len(got.Missing) > 1 { + t.Errorf("limit 1 = %v, %v", got, err) + } +} + +func TestNoPlaysGiveEmptyListsAndNotNull(t *testing.T) { + r := newRig(true) + r.store.plays = nil + got, err := r.svc.Suggest(context.Background(), 1, 0) + if err != nil || got.Library == nil || got.Missing == nil || len(got.Library)+len(got.Missing) != 0 { + t.Errorf("got %+v, %v, want two empty lists", got, err) + } +} + +func TestOnlyAnArtistThatWasPlayedGoesOnTheWishlist(t *testing.T) { + r := newRig(true) + ctx := context.Background() + // The model is built by the wish itself when no request for suggestions did it. + if _, err := r.svc.AddToWishlist(ctx, 1, "nobody I played"); !errors.Is(err, ErrUnknownArtist) { + t.Errorf("err = %v, want ErrUnknownArtist", err) + } + if _, err := r.svc.AddToWishlist(ctx, 1, " "); !errors.Is(err, ErrUnknownArtist) { + t.Errorf("a blank name: %v, want ErrUnknownArtist", err) + } + id, err := r.svc.AddToWishlist(ctx, 1, "canali") + if err != nil || id != 77 || len(r.wishes.names) != 1 || r.wishes.names[0] != "canali" { + t.Errorf("AddToWishlist = %d, %v, names %v", id, err, r.wishes.names) + } +} diff --git a/internal/api/http/api_test.go b/internal/api/http/api_test.go index 8477219..02b76a0 100644 --- a/internal/api/http/api_test.go +++ b/internal/api/http/api_test.go @@ -16,6 +16,7 @@ import ( "go.uber.org/zap" "codeberg.org/squidd/sonda/internal/about" + "codeberg.org/squidd/sonda/internal/affinity" "codeberg.org/squidd/sonda/internal/auth" "codeberg.org/squidd/sonda/internal/catalog" "codeberg.org/squidd/sonda/internal/discover" @@ -149,6 +150,10 @@ func newTestEnvDB(t *testing.T, mutate func(*Deps, *bun.DB)) (http.Handler, *bun } } // Built from the final services, so a test that swaps one gets a Discover that uses it. + if deps.Suggestions == nil { + repo := storage.NewAffinityRepo(db) + deps.Suggestions = affinity.NewService(repo, repo, feedbackSvc, repo, usersettings.NewService(storage.NewUserSettingsRepo(db)), 20*time.Second) + } if deps.Discover == nil { deps.Discover = discover.NewService(deps.Taste.(discover.Taster), deps.Discovery.(discover.Similar), discovery.NewTagService(storage.NewDiscoveryRepo(db), map[string]discovery.TagSource{}, prov, zap.NewNop()), diff --git a/internal/api/http/contract_test.go b/internal/api/http/contract_test.go index 1b464e7..bc0122c 100644 --- a/internal/api/http/contract_test.go +++ b/internal/api/http/contract_test.go @@ -881,6 +881,21 @@ func TestResponsesMatchTheSpec(t *testing.T) { c.expect(c.upload("/me/import/spotify", token, export[:len(export)-20]), 400, "a file cut short", "unreadable_file") c.expect(c.upload("/me/import/spotify", ""), 401, "an import without a token", "unauthorized") + // Suggestions from the listening: refused until it is allowed, then empty for a history with nothing to learn from. + c.expect(c.call("PATCH", "/me/settings", token, map[string]any{"discover_from_listening": false}), 200, "forbid Discover from the listening") + c.expect(c.call("GET", "/me/suggestions", token, nil), 400, "suggestions before they are allowed", "listening_not_allowed") + c.expect(c.call("POST", "/me/suggestions/wishlist", token, map[string]any{"name": "Nobody"}), 400, "a wish before it is allowed", "listening_not_allowed") + c.expect(c.call("PATCH", "/me/settings", token, map[string]any{"discover_from_listening": true}), 200, "allow Discover from the listening") + suggested := c.expect(c.call("GET", "/me/suggestions?limit=5", token, nil), 200, "suggestions") + if lib, _ := suggested["library"].([]any); lib == nil { + c.t.Errorf("suggestions = %v, want empty lists and not null", suggested) + } + c.expect(c.call("GET", "/me/suggestions?limit=-1", token, nil), 400, "a negative limit", "invalid_input") + c.expect(c.call("GET", "/me/suggestions", "", nil), 401, "suggestions without a token", "unauthorized") + c.expect(c.call("POST", "/me/suggestions/wishlist", token, map[string]any{"name": "Nobody I played"}), 404, "an artist not played", "artist_not_played") + c.expect(c.call("POST", "/me/suggestions/wishlist", "", map[string]any{"name": "x"}), 401, "a wish without a token", "unauthorized") + c.expect(c.call("PATCH", "/me/settings", token, map[string]any{"discover_from_listening": false}), 200, "forbid it again") + // ---- duplicate songs ---- c.duplicates(t, token, libID) diff --git a/internal/api/http/handlers_suggestions.go b/internal/api/http/handlers_suggestions.go new file mode 100644 index 0000000..5307d71 --- /dev/null +++ b/internal/api/http/handlers_suggestions.go @@ -0,0 +1,120 @@ +package httpapi + +import ( + "context" + "errors" + "net/http" + "strings" + "time" + + "github.com/gin-gonic/gin" + + "codeberg.org/squidd/sonda/internal/affinity" + "codeberg.org/squidd/sonda/internal/feedback" +) + +// listeningSuggestionService is what the handlers need from affinity.Service. +type listeningSuggestionService interface { + Suggest(ctx context.Context, userID int64, limit int) (affinity.Suggestions, error) + AddToWishlist(ctx context.Context, userID int64, name string) (int64, error) +} + +type listeningSuggestionHandlers struct{ svc listeningSuggestionService } + +type ownedReasonJSON struct { + Name string `json:"name"` + Score float64 `json:"score"` +} + +type listeningSuggestionJSON struct { + Name string `json:"name"` + ArtistID *int64 `json:"artist_id"` + Score float64 `json:"score"` + Plays int `json:"plays"` + LastPlayedAt time.Time `json:"last_played_at"` + Wishlisted bool `json:"wishlisted"` + Because []ownedReasonJSON `json:"because"` +} + +type listeningSuggestionsJSON struct { + Library []listeningSuggestionJSON `json:"library"` + Missing []listeningSuggestionJSON `json:"missing"` + Built struct { + At time.Time `json:"at"` + Plays int `json:"plays"` + Sessions int `json:"sessions"` + Artists int `json:"artists"` + Links int `json:"links"` + } `json:"built"` +} + +func toListeningSuggestions(in []affinity.Suggestion) []listeningSuggestionJSON { + out := make([]listeningSuggestionJSON, 0, len(in)) + for _, s := range in { + j := listeningSuggestionJSON{Name: s.Name, Score: s.Score, Plays: s.Plays, LastPlayedAt: s.LastPlayed.UTC(), Wishlisted: s.Wishlisted, Because: make([]ownedReasonJSON, 0, len(s.Because))} + if s.ArtistID != 0 { + id := s.ArtistID + j.ArtistID = &id + } + for _, r := range s.Because { + j.Because = append(j.Because, ownedReasonJSON{Name: r.Name, Score: r.Score}) + } + out = append(out, j) + } + return out +} + +// list suggests artists from how the signed-in person listens. +func (h *listeningSuggestionHandlers) list(c *gin.Context) { + limit, ok := queryInt(c, "limit") + if !ok { + return + } + got, err := h.svc.Suggest(c.Request.Context(), currentUserID(c), clampInt(limit)) + switch { + case errors.Is(err, affinity.ErrNotAllowed): + writeError(c, http.StatusBadRequest, "listening_not_allowed", err.Error()) + return + case errors.Is(err, affinity.ErrInvalidLimit): + writeError(c, http.StatusBadRequest, "invalid_input", err.Error()) + return + case err != nil: + _ = c.Error(err) + writeError(c, http.StatusInternalServerError, "internal", "internal error") + return + } + out := listeningSuggestionsJSON{Library: toListeningSuggestions(got.Library), Missing: toListeningSuggestions(got.Missing)} + out.Built.At, out.Built.Plays, out.Built.Sessions = got.Built.BuiltAt.UTC(), got.Built.Plays, got.Built.Sessions + out.Built.Artists, out.Built.Links = got.Built.Artists, got.Built.Links + c.JSON(http.StatusOK, out) +} + +type wishRequest struct { + Name string `json:"name"` +} + +// wish puts an artist the person has played on their wishlist, by name. +func (h *listeningSuggestionHandlers) wish(c *gin.Context) { + var req wishRequest + if !bindJSON(c, &req) { + return + } + if strings.TrimSpace(req.Name) == "" { + writeError(c, http.StatusBadRequest, "invalid_input", "name is required") + return + } + id, err := h.svc.AddToWishlist(c.Request.Context(), currentUserID(c), req.Name) + switch { + case err == nil: + c.JSON(http.StatusOK, gin.H{"artist_id": id}) + case errors.Is(err, affinity.ErrNotAllowed): + writeError(c, http.StatusBadRequest, "listening_not_allowed", err.Error()) + case errors.Is(err, affinity.ErrUnknownArtist): + writeError(c, http.StatusNotFound, "artist_not_played", err.Error()) + case errors.Is(err, feedback.ErrUserNotFound): + writeError(c, http.StatusUnauthorized, "unauthorized", "sign in again") + default: + _ = c.Error(err) + writeError(c, http.StatusInternalServerError, "internal", "internal error") + } +} diff --git a/internal/api/http/router.go b/internal/api/http/router.go index e460e14..b48d7cc 100644 --- a/internal/api/http/router.go +++ b/internal/api/http/router.go @@ -110,6 +110,9 @@ type spotifyImportService interface { History(ctx context.Context, userID int64, limit, offset int) (spotifyimport.RunPage, error) } +// listeningSuggestionService is what the handlers need from affinity.Service. +// (declared in handlers_suggestions.go) + // mixService is what the handlers need from mixes.Service. type mixService interface { List(ctx context.Context, userID int64) ([]mixes.Mix, error) @@ -227,6 +230,7 @@ type Deps struct { Feedback feedbackService Playlists playlistService SpotifyImport spotifyImportService + Suggestions listeningSuggestionService Mixes mixService Smart smartService Naming namingService @@ -304,6 +308,9 @@ func NewRouter(d Deps) *gin.Engine { protected.POST("/listens", l.record) protected.GET("/listens", l.list) + sg := &listeningSuggestionHandlers{svc: d.Suggestions} + protected.GET("/me/suggestions", sg.list) + protected.POST("/me/suggestions/wishlist", sg.wish) ts := &tasteHandlers{svc: d.Taste} protected.GET("/me/taste", ts.taste) dc := &discoverHandlers{svc: d.Discover} diff --git a/internal/api/http/suggestions_test.go b/internal/api/http/suggestions_test.go new file mode 100644 index 0000000..c503cc6 --- /dev/null +++ b/internal/api/http/suggestions_test.go @@ -0,0 +1,174 @@ +package httpapi + +import ( + "fmt" + "net/http" + "testing" + "time" + + "github.com/uptrace/bun" + + "codeberg.org/squidd/sonda/internal/storage" +) + +const suggestionsPath = "/api/v1/me/suggestions" + +// listeningHistory plays this out for the only user: three old sittings of Radiohead, Pixies and (outside the library) +// Giorgio Canali, and two recent ones of Radiohead and Placebo. Pixies and Placebo have a song in the library. +func listeningHistory(t *testing.T, db *bun.DB) { + t.Helper() + seedSong(t, db, "a.flac", "Airbag", "OK Computer", "Radiohead") + seedSong(t, db, "b.flac", "Debaser", "Doolittle", "Pixies") + seedSong(t, db, "c.flac", "Nancy Boy", "Placebo", "Placebo") + var user int64 + if err := db.QueryRow("SELECT id FROM users LIMIT 1").Scan(&user); err != nil { + t.Fatal(err) + } + track := func(title string) int64 { + var id int64 + if err := db.QueryRow("SELECT id FROM tracks WHERE title = ?", title).Scan(&id); err != nil { + t.Fatal(err) + } + return id + } + n := 0 + event := func(title string, at time.Time) { + n++ + if _, err := db.Exec("INSERT INTO listening_events (user_id, track_id, event_type, occurred_at, duration_played_ms, idempotency_key) VALUES (?, ?, 'complete', ?, 200000, ?)", + user, track(title), at.Unix(), fmt.Sprintf("k%d", n)); err != nil { + t.Fatal(err) + } + } + away := func(artist string, at time.Time) { + n++ + if _, err := db.Exec("INSERT INTO outside_plays (user_id, idempotency_key, artist, artist_key, played_at) VALUES (?, ?, ?, ?, ?)", + user, fmt.Sprintf("o%d", n), artist, storage.SearchKey(artist), at.Unix()); err != nil { + t.Fatal(err) + } + } + now := time.Now() + for _, days := range []int{200, 199, 198} { + at := now.AddDate(0, 0, -days) + event("Airbag", at) + event("Debaser", at.Add(time.Minute)) + away("Giorgio Canali", at.Add(2*time.Minute)) + } + for _, days := range []int{10, 9} { + at := now.AddDate(0, 0, -days) + event("Airbag", at) + event("Nancy Boy", at.Add(time.Minute)) + } +} + +func allowListening(t *testing.T, h http.Handler, tok string) { + t.Helper() + if r := do(t, h, "PATCH", "/api/v1/me/settings", tok, map[string]any{"discover_from_listening": true}); r.Code != http.StatusOK { + t.Fatalf("settings: %d %v", r.Code, r.Body) + } +} + +func artistNames(body map[string]any, list string) []string { + out := []string{} + items, _ := body[list].([]any) + for _, it := range items { + out = append(out, it.(map[string]any)["name"].(string)) + } + return out +} + +func TestSuggestionsComeFromHowThePersonListensAndNeedTheirPermission(t *testing.T) { + h, db := newTestEnv(t, func(*Deps) {}) + tok := registerAndLogin(t, h) + listeningHistory(t, db) + + if r := do(t, h, "GET", suggestionsPath, tok, nil); r.Code != http.StatusBadRequest || errCode(r) != "listening_not_allowed" { + t.Fatalf("without permission: %d %v, want 400 listening_not_allowed", r.Code, r.Body) + } + allowListening(t, h, tok) + + r := do(t, h, "GET", suggestionsPath, tok, nil) + if r.Code != http.StatusOK { + t.Fatalf("suggestions: %d %v", r.Code, r.Body) + } + sameKeys(t, "suggestions", r.Body, "library", "missing", "built") + if got := artistNames(r.Body, "library"); len(got) != 1 || got[0] != "Pixies" { + t.Errorf("library = %v, want Pixies (Placebo and Radiohead were played lately)", got) + } + if got := artistNames(r.Body, "missing"); len(got) != 1 || got[0] != "Giorgio Canali" { + t.Errorf("missing = %v, want Giorgio Canali", got) + } + pixies := r.Body["library"].([]any)[0].(map[string]any) + sameKeys(t, "suggestion", pixies, "name", "artist_id", "score", "plays", "last_played_at", "wishlisted", "because") + if pixies["artist_id"] == nil || pixies["score"] != 1.0 || pixies["plays"] != 3.0 || pixies["wishlisted"] != false { + t.Errorf("Pixies = %v", pixies) + } + canali := r.Body["missing"].([]any)[0].(map[string]any) + if canali["artist_id"] != nil || canali["plays"] != 3.0 { + t.Errorf("Canali = %v, want no artist yet and 3 plays", canali) + } + built, _ := r.Body["built"].(map[string]any) + sameKeys(t, "built", built, "at", "plays", "sessions", "artists", "links") + if built["sessions"] != 5.0 || built["artists"] != 4.0 { + t.Errorf("built = %v, want 5 sittings and 4 artists", built) + } + + if r := do(t, h, "GET", suggestionsPath+"?limit=-1", tok, nil); r.Code != http.StatusBadRequest || errCode(r) != "invalid_input" { + t.Errorf("a negative limit: %d %v", r.Code, r.Body) + } + if r := do(t, h, "GET", suggestionsPath, "", nil); r.Code != http.StatusUnauthorized { + t.Errorf("without a token: %d, want 401", r.Code) + } +} + +func TestAMissingArtistGoesOnTheWishlistByName(t *testing.T) { + h, db := newTestEnv(t, func(*Deps) {}) + tok := registerAndLogin(t, h) + listeningHistory(t, db) + allowListening(t, h, tok) + wish := suggestionsPath + "/wishlist" + + if r := do(t, h, "POST", wish, tok, map[string]any{"name": "Nobody I played"}); r.Code != http.StatusNotFound || errCode(r) != "artist_not_played" { + t.Errorf("an artist not played: %d %v, want 404 artist_not_played", r.Code, r.Body) + } + if r := do(t, h, "POST", wish, tok, map[string]any{"name": "Giorgio Canali"}); r.Code != http.StatusOK || r.Body["artist_id"] == nil { + t.Fatalf("wish: %d %v", r.Code, r.Body) + } + if r := do(t, h, "POST", wish, tok, map[string]any{"name": "Giorgio Canali"}); r.Code != http.StatusOK { + t.Errorf("wishing twice: %d %v, want 200", r.Code, r.Body) + } + + list := do(t, h, "GET", "/api/v1/wishlist", tok, nil) + items, _ := list.Body["items"].([]any) + if len(items) != 1 || items[0].(map[string]any)["artist"].(map[string]any)["name"] != "Giorgio Canali" { + t.Errorf("wishlist = %v, want the artist", list.Body) + } + if got := do(t, h, "GET", suggestionsPath, tok, nil); !got.Body["missing"].([]any)[0].(map[string]any)["wishlisted"].(bool) { + t.Errorf("the suggestion does not say it is on the wishlist: %v", got.Body["missing"]) + } + var named int + if err := db.QueryRow("SELECT COUNT(*) FROM artists WHERE name = 'Giorgio Canali' AND mbid IS NULL AND discovered = ?", true).Scan(&named); err != nil || named != 1 { + t.Errorf("%d artists made for it (%v), want one without a MusicBrainz id", named, err) + } + + if r := do(t, h, "POST", wish, tok, map[string]any{"nope": 1}); r.Code != http.StatusBadRequest { + t.Errorf("a body that is not valid: %d, want 400", r.Code) + } + if r := do(t, h, "POST", wish, "", map[string]any{"name": "x"}); r.Code != http.StatusUnauthorized { + t.Errorf("without a token: %d, want 401", r.Code) + } +} + +func TestAnotherPersonSeesNothingOfMyListening(t *testing.T) { + h, db := newTestEnv(t, func(*Deps) {}) + tok := registerAndLogin(t, h) + listeningHistory(t, db) + _, other := addUser(t, h, tok, "other", false) + allowListening(t, h, other) + r := do(t, h, "GET", suggestionsPath, other, nil) + if r.Code != http.StatusOK || len(artistNames(r.Body, "library")) != 0 || len(artistNames(r.Body, "missing")) != 0 { + t.Errorf("another person: %d %v, want empty lists", r.Code, r.Body) + } + if r := do(t, h, "POST", suggestionsPath+"/wishlist", other, map[string]any{"name": "Giorgio Canali"}); r.Code != http.StatusNotFound { + t.Errorf("wishing an artist only someone else played: %d, want 404", r.Code) + } +} diff --git a/internal/api/openapi/openapi.yaml b/internal/api/openapi/openapi.yaml index 9b66347..47a1ead 100644 --- a/internal/api/openapi/openapi.yaml +++ b/internal/api/openapi/openapi.yaml @@ -2053,6 +2053,86 @@ paths: '400': {$ref: '#/components/responses/InvalidInput'} '401': {$ref: '#/components/responses/Unauthorized'} + /me/suggestions: + get: + tags: [Taste] + operationId: getMySuggestions + summary: Artists suggested by how you listen + description: | + Suggestions learned from your own listening, with no provider and nothing sent anywhere. Sonda splits your + history into sittings (plays less than 30 minutes apart) and links the artists that are often played in the + same sitting; the model is built from your listening events and from the plays of artists you have no music + of that an import kept (see `POST /me/import/spotify`), and is built again when plays are added. It needs + `discover_from_listening` in `PATCH /me/settings` (`listening_not_allowed` without it). + + `library` are artists you have, linked to what you play most, that you did not play in the last 60 days, + best first. `missing` are artists you have played at least three times and have no music of, the ones you play + most that also go with what you play now first. Each says which of the artists you play it goes with + (`because`) and has a `score` from 0 to 1 within its list. `artist_id` is null for an artist that does not + exist on this server yet. Artists you dismissed are left out. + security: [{bearerAuth: []}] + parameters: + - name: limit + in: query + description: How many artists in each list. 20 when absent or 0; values above 50 are lowered to 50. + schema: {type: integer, minimum: 0, default: 20} + responses: + '200': + description: The suggestions. Both lists are empty until you have listened to something. + content: + application/json: + schema: {$ref: '#/components/schemas/ListeningSuggestions'} + '400': + description: A negative limit (`invalid_input`), or Discover is not allowed to start from your listening (`listening_not_allowed`). + content: + application/json: + schema: {$ref: '#/components/schemas/Error'} + '401': {$ref: '#/components/responses/Unauthorized'} + + /me/suggestions/wishlist: + post: + tags: [Taste] + operationId: wishArtistFromSuggestions + summary: Put a suggested artist you do not have on your wishlist + description: | + Puts an artist you have played on your wishlist by its name, for the artists of `missing` that have no + `artist_id`: the artist is made on this server without any music, like the ones a provider names, and has + no MusicBrainz id. The name must be one the model knows from your listening (`artist_not_played`). Doing it + twice keeps the first date. It needs `discover_from_listening` like the suggestions. + security: [{bearerAuth: []}] + requestBody: + required: true + content: + application/json: + schema: + type: object + additionalProperties: false + required: [name] + properties: + name: {type: string} + responses: + '200': + description: The artist is on your wishlist. + content: + application/json: + schema: + type: object + additionalProperties: false + required: [artist_id] + properties: + artist_id: {type: integer} + '400': + description: The body is not valid (`invalid_input`), or Discover is not allowed to start from your listening (`listening_not_allowed`). + content: + application/json: + schema: {$ref: '#/components/schemas/Error'} + '401': {$ref: '#/components/responses/Unauthorized'} + '404': + description: You have not played an artist with this name (`artist_not_played`). + content: + application/json: + schema: {$ref: '#/components/schemas/Error'} + /me/settings: get: tags: [Taste] @@ -4866,6 +4946,51 @@ components: duration_played_ms: {type: integer, format: int64} counted: {type: boolean, description: Whether the event counts as a listen under the current rule.} + ListeningSuggestion: + type: object + additionalProperties: false + required: [name, artist_id, score, plays, last_played_at, wishlisted, because] + properties: + name: {type: string} + artist_id: {type: integer, nullable: true, description: 'The artist on this server, null when there is none yet.'} + score: {type: number, description: 'From 0 to 1 within its list.'} + plays: {type: integer, description: 'How many times you played it.'} + last_played_at: {type: string, format: date-time} + wishlisted: {type: boolean} + because: + type: array + description: The artists you play that it goes with, the strongest first, at most three. + items: + type: object + additionalProperties: false + required: [name, score] + properties: + name: {type: string} + score: {type: number} + + ListeningSuggestions: + type: object + additionalProperties: false + required: [library, missing, built] + properties: + library: + type: array + items: {$ref: '#/components/schemas/ListeningSuggestion'} + missing: + type: array + items: {$ref: '#/components/schemas/ListeningSuggestion'} + built: + type: object + additionalProperties: false + required: [at, plays, sessions, artists, links] + description: When the model was built and from how much. + properties: + at: {type: string, format: date-time} + plays: {type: integer} + sessions: {type: integer} + artists: {type: integer} + links: {type: integer} + Taste: type: object additionalProperties: false diff --git a/internal/storage/affinity.go b/internal/storage/affinity.go new file mode 100644 index 0000000..50bffe0 --- /dev/null +++ b/internal/storage/affinity.go @@ -0,0 +1,315 @@ +package storage + +import ( + "context" + "database/sql" + "fmt" + "strings" + "time" + + "github.com/uptrace/bun" + + "codeberg.org/squidd/sonda/internal/affinity" + "codeberg.org/squidd/sonda/internal/feedback" +) + +// AffinityRepo reads the plays of a person and keeps what was learned from them. It implements affinity.Store, +// affinity.Library and affinity.Wishes. +type AffinityRepo struct{ db *bun.DB } + +// NewAffinityRepo builds an AffinityRepo. +func NewAffinityRepo(db *bun.DB) *AffinityRepo { return &AffinityRepo{db: db} } + +// counted is the condition "this listening event counts as a listen" (see POST /listens), with its one argument. +const counted = "(e.event_type = 'complete' OR e.duration_played_ms >= ?)" + +// Plays gives every play of a person that counts as listened, once for each artist credited on it: the listening +// events of the library and the plays kept from imports, leaving out the kept ones that have a listening event now. +func (r *AffinityRepo) Plays(ctx context.Context, userID int64, minPlayed time.Duration) ([]affinity.Play, error) { + var plays []affinity.Play + rows, err := r.db.QueryContext(ctx, + `SELECT a.name, e.occurred_at FROM listening_events e + JOIN track_artists ta ON ta.track_id = e.track_id JOIN artists a ON a.id = ta.artist_id + WHERE e.user_id = ? AND `+counted, userID, minPlayed.Milliseconds()) + if err != nil { + return nil, fmt.Errorf("read listens: %w", err) + } + defer rows.Close() + for rows.Next() { + var ( + name string + at int64 + ) + if err := rows.Scan(&name, &at); err != nil { + return nil, err + } + plays = append(plays, affinity.Play{Key: searchKey(name), Name: name, At: time.Unix(at, 0)}) + } + if err := rows.Err(); err != nil { + return nil, err + } + + away, err := r.db.QueryContext(ctx, + `SELECT o.artist, o.artist_key, o.played_at FROM outside_plays o WHERE o.user_id = ? + AND NOT EXISTS (SELECT 1 FROM listening_events e WHERE e.user_id = o.user_id AND e.idempotency_key = o.idempotency_key)`, userID) + if err != nil { + return nil, fmt.Errorf("read outside plays: %w", err) + } + defer away.Close() + for away.Next() { + var ( + name, key string + at int64 + ) + if err := away.Scan(&name, &key, &at); err != nil { + return nil, err + } + plays = append(plays, affinity.Play{Key: key, Name: name, At: time.Unix(at, 0)}) + } + return plays, away.Err() +} + +// PlayCount says how many plays Plays would give, without reading them. +func (r *AffinityRepo) PlayCount(ctx context.Context, userID int64, minPlayed time.Duration) (int, error) { + var events, away int + if err := r.db.QueryRowContext(ctx, + `SELECT COUNT(*) FROM listening_events e JOIN track_artists ta ON ta.track_id = e.track_id + WHERE e.user_id = ? AND `+counted, userID, minPlayed.Milliseconds()).Scan(&events); err != nil { + return 0, fmt.Errorf("count listens: %w", err) + } + if err := r.db.QueryRowContext(ctx, + `SELECT COUNT(*) FROM outside_plays o WHERE o.user_id = ? + AND NOT EXISTS (SELECT 1 FROM listening_events e WHERE e.user_id = o.user_id AND e.idempotency_key = o.idempotency_key)`, userID).Scan(&away); err != nil { + return 0, fmt.Errorf("count outside plays: %w", err) + } + return events + away, nil +} + +// affinityChunk is how many rows one statement inserts: at most seven columns each. +const affinityChunk = 500 + +type affinityArtistRow struct { + bun.BaseModel `bun:"table:affinity_artists"` + + UserID int64 `bun:"user_id,pk"` + ArtistKey string `bun:"artist_key,pk"` + Name string `bun:"name,notnull"` + Plays int `bun:"plays,notnull"` + Sessions int `bun:"sessions,notnull"` + LastPlayedAt int64 `bun:"last_played_at,notnull"` + Weight float64 `bun:"weight,notnull"` +} + +type affinityLinkRow struct { + bun.BaseModel `bun:"table:affinity_links"` + + UserID int64 `bun:"user_id,pk"` + AKey string `bun:"a_key,pk"` + BKey string `bun:"b_key,pk"` + Score float64 `bun:"score,notnull"` +} + +type affinityBuildRow struct { + bun.BaseModel `bun:"table:affinity_builds"` + + UserID int64 `bun:"user_id,pk"` + BuiltAt int64 `bun:"built_at,notnull"` + Plays int `bun:"plays,notnull"` + Sessions int `bun:"sessions,notnull"` + Artists int `bun:"artists,notnull"` + Links int `bun:"links,notnull"` +} + +// Save replaces what was learned about a person with a new model, in one transaction. The plays kept from imports +// that have a listening event now are removed: the event says it better. +func (r *AffinityRepo) Save(ctx context.Context, userID int64, at time.Time, m affinity.Model) error { + return r.db.RunInTx(ctx, nil, func(ctx context.Context, tx bun.Tx) error { + for _, table := range []string{"affinity_links", "affinity_artists", "affinity_builds"} { + if _, err := tx.ExecContext(ctx, "DELETE FROM "+table+" WHERE user_id = ?", userID); err != nil { + return fmt.Errorf("clear %s: %w", table, err) + } + } + for start := 0; start < len(m.Artists); start += affinityChunk { + end := min(start+affinityChunk, len(m.Artists)) + rows := make([]affinityArtistRow, 0, end-start) + for _, a := range m.Artists[start:end] { + rows = append(rows, affinityArtistRow{UserID: userID, ArtistKey: a.Key, Name: a.Name, Plays: a.Plays, Sessions: a.Sessions, LastPlayedAt: a.Last.Unix(), Weight: a.Weight}) + } + if _, err := tx.NewInsert().Model(&rows).Exec(ctx); err != nil { + return fmt.Errorf("insert affinity artists: %w", err) + } + } + for start := 0; start < len(m.Links); start += affinityChunk { + end := min(start+affinityChunk, len(m.Links)) + rows := make([]affinityLinkRow, 0, end-start) + for _, l := range m.Links[start:end] { + rows = append(rows, affinityLinkRow{UserID: userID, AKey: l.A, BKey: l.B, Score: l.Score}) + } + if _, err := tx.NewInsert().Model(&rows).Exec(ctx); err != nil { + return fmt.Errorf("insert affinity links: %w", err) + } + } + build := &affinityBuildRow{UserID: userID, BuiltAt: at.Unix(), Plays: m.Plays, Sessions: m.Sessions, Artists: len(m.Artists), Links: len(m.Links)} + if _, err := tx.NewInsert().Model(build).Exec(ctx); err != nil { + return fmt.Errorf("insert affinity build: %w", err) + } + if _, err := tx.ExecContext(ctx, + `DELETE FROM outside_plays WHERE user_id = ? AND idempotency_key IN + (SELECT idempotency_key FROM listening_events WHERE user_id = ?)`, userID, userID); err != nil { + return fmt.Errorf("drop outside plays that are listens now: %w", err) + } + return nil + }) +} + +// Build says when the model of a person was built and from how much; ok is false when it never was. +func (r *AffinityRepo) Build(ctx context.Context, userID int64) (b affinity.Built, ok bool, err error) { + var row affinityBuildRow + err = r.db.NewSelect().Model(&row).Where("user_id = ?", userID).Scan(ctx) + if err == sql.ErrNoRows { + return b, false, nil + } + if err != nil { + return b, false, fmt.Errorf("read affinity build: %w", err) + } + return affinity.Built{BuiltAt: time.Unix(row.BuiltAt, 0), Plays: row.Plays, Sessions: row.Sessions, Artists: row.Artists, Links: row.Links}, true, nil +} + +// Seeds gives the artists a person plays the most (by weight), best first. +func (r *AffinityRepo) Seeds(ctx context.Context, userID int64, limit int) ([]affinity.Artist, error) { + var rows []affinityArtistRow + if err := r.db.NewSelect().Model(&rows).Where("user_id = ?", userID).OrderExpr("weight DESC, artist_key").Limit(int64(limit)).Scan(ctx); err != nil { + return nil, fmt.Errorf("read affinity seeds: %w", err) + } + return artistsOf(rows), nil +} + +// Artists gives what is known of the artists with these keys. +func (r *AffinityRepo) Artists(ctx context.Context, userID int64, keys []string) (map[string]affinity.Artist, error) { + out := map[string]affinity.Artist{} + for start := 0; start < len(keys); start += affinityChunk { + end := min(start+affinityChunk, len(keys)) + var rows []affinityArtistRow + if err := r.db.NewSelect().Model(&rows).Where("user_id = ?", userID).Where("artist_key IN (?)", bun.In(keys[start:end])).Scan(ctx); err != nil { + return nil, fmt.Errorf("read affinity artists: %w", err) + } + for _, a := range artistsOf(rows) { + out[a.Key] = a + } + } + return out, nil +} + +func artistsOf(rows []affinityArtistRow) []affinity.Artist { + out := make([]affinity.Artist, 0, len(rows)) + for _, row := range rows { + out = append(out, affinity.Artist{Key: row.ArtistKey, Name: row.Name, Plays: row.Plays, Sessions: row.Sessions, Last: time.Unix(row.LastPlayedAt, 0), Weight: row.Weight}) + } + return out +} + +// Neighbours gives every link that has one of these artists at one end. +func (r *AffinityRepo) Neighbours(ctx context.Context, userID int64, keys []string) ([]affinity.Link, error) { + if len(keys) == 0 { + return nil, nil + } + var rows []affinityLinkRow + if err := r.db.NewSelect().Model(&rows).Where("user_id = ?", userID). + WhereGroup(" AND ", func(q *bun.SelectQuery) *bun.SelectQuery { + return q.Where("a_key IN (?)", bun.In(keys)).WhereOr("b_key IN (?)", bun.In(keys)) + }).Scan(ctx); err != nil { + return nil, fmt.Errorf("read affinity links: %w", err) + } + out := make([]affinity.Link, 0, len(rows)) + for _, row := range rows { + out = append(out, affinity.Link{A: row.AKey, B: row.BKey, Score: row.Score}) + } + return out, nil +} + +// LibraryArtists says, for these search keys, which artists exist: the id, and whether there is music to play. When +// several artists share a key the one with music comes first. +func (r *AffinityRepo) LibraryArtists(ctx context.Context, keys []string) (map[string]affinity.LibraryArtist, error) { + out := map[string]affinity.LibraryArtist{} + for start := 0; start < len(keys); start += affinityChunk { + end := min(start+affinityChunk, len(keys)) + rows, err := r.db.QueryContext(ctx, + "SELECT a.id, a.name, a.search_key, "+playableArtist("a.id")+" FROM artists a WHERE a.search_key IN (?) ORDER BY a.id", + bun.In(keys[start:end])) + if err != nil { + return nil, fmt.Errorf("find artists: %w", err) + } + for rows.Next() { + var la affinity.LibraryArtist + var key string + if err := rows.Scan(&la.ID, &la.Name, &key, &la.Playable); err != nil { + rows.Close() + return nil, err + } + if have, ok := out[key]; !ok || (la.Playable && !have.Playable) { + out[key] = la + } + } + if err := rows.Err(); err != nil { + rows.Close() + return nil, err + } + rows.Close() + } + return out, nil +} + +// Wishlist puts an artist on the wishlist of a person by name: the artist that already has this name (the one with +// music first), or a new one that exists only because of this, like the ones a provider names. It gives the id. +func (r *AffinityRepo) Wishlist(ctx context.Context, userID int64, name string, at time.Time) (int64, error) { + var id int64 + err := r.db.RunInTx(ctx, nil, func(ctx context.Context, tx bun.Tx) error { + if err := lockArtistWrites(ctx, tx); err != nil { + return err + } + var one int + if err := tx.QueryRowContext(ctx, "SELECT 1 FROM users WHERE id = ?", userID).Scan(&one); err != nil { + return mapNoRows(err, feedback.ErrUserNotFound, "check user") + } + key := searchKey(name) + rows, err := tx.QueryContext(ctx, "SELECT a.id, "+playableArtist("a.id")+" FROM artists a WHERE a.search_key = ? ORDER BY a.id", key) + if err != nil { + return fmt.Errorf("find artist: %w", err) + } + var playable bool + for rows.Next() { + var cand int64 + var p bool + if err := rows.Scan(&cand, &p); err != nil { + rows.Close() + return err + } + if id == 0 || (p && !playable) { + id, playable = cand, p + } + } + if err := rows.Err(); err != nil { + rows.Close() + return err + } + rows.Close() + if id == 0 { + if id, err = insertDiscovered(ctx, tx, strings.TrimSpace(name), ""); err != nil { + return err + } + } + if _, err := tx.ExecContext(ctx, + `INSERT INTO artist_feedback (user_id, artist_id, kind, created_at) VALUES (?, ?, ?, ?) + ON CONFLICT (user_id, artist_id) DO UPDATE SET + created_at = CASE WHEN artist_feedback.kind = excluded.kind THEN artist_feedback.created_at ELSE excluded.created_at END, + kind = excluded.kind`, + userID, id, string(feedback.Wishlist), at.Unix()); err != nil { + return fmt.Errorf("write feedback: %w", err) + } + return nil + }) + if err != nil { + return 0, err + } + return id, nil +} diff --git a/internal/storage/affinity_test.go b/internal/storage/affinity_test.go new file mode 100644 index 0000000..0ec2421 --- /dev/null +++ b/internal/storage/affinity_test.go @@ -0,0 +1,210 @@ +package storage + +import ( + "context" + "testing" + "time" + + "github.com/uptrace/bun" + + "codeberg.org/squidd/sonda/internal/affinity" + "codeberg.org/squidd/sonda/internal/library" + "codeberg.org/squidd/sonda/internal/listening" + "codeberg.org/squidd/sonda/internal/spotifyimport" +) + +// affinitySong puts a song with one artist in the library and gives the id of its track. +func affinitySong(t *testing.T, db *bun.DB, lib int64, rel, title, artist string) int64 { + t.Helper() + f := library.ParsedFile{ + FileInfo: library.FileInfo{RelPath: rel, Size: 100, MTimeUnix: 1}, + Metadata: library.Metadata{ + Tags: library.Tags{Title: title, Album: "Album of " + artist, AlbumArtist: artist, Artists: []string{artist}}, + Properties: library.Properties{Format: "flac", DurationMs: 200_000, SampleRate: 44100, BitRate: 900}, + }, + } + if _, err := NewLibraryRepo(db).SaveFile(context.Background(), lib, f, 0); err != nil { + t.Fatal(err) + } + var id int64 + if err := db.QueryRow("SELECT id FROM tracks WHERE title = ?", title).Scan(&id); err != nil { + t.Fatal(err) + } + return id +} + +func TestTheAffinityRepoReadsThePlaysThatCountAndKeepsTheModel(t *testing.T) { + forEachBackend(t, func(t *testing.T, db *bun.DB, _ string) { + ctx := context.Background() + u, err := NewUserRepo(db).CreateUser(ctx, "alice", "h", false) + if err != nil { + t.Fatal(err) + } + other, err := NewUserRepo(db).CreateUser(ctx, "bob", "h", false) + if err != nil { + t.Fatal(err) + } + lib, err := NewLibraryRepo(db).CreateLibrary(ctx, "music", "/music") + if err != nil { + t.Fatal(err) + } + pixies := affinitySong(t, db, lib.ID, "a.flac", "Debaser", "Pixies") + blur := affinitySong(t, db, lib.ID, "b.flac", "Song 2", "Blur") + minListen := 20 * time.Second + + listens := NewListeningRepo(db) + ev := func(user, track int64, typ listening.EventType, ms int64, at int64, key string) listening.Event { + return listening.Event{UserID: user, TrackID: track, Type: typ, DurationPlayedMs: ms, OccurredAt: time.Unix(at, 0), IdempotencyKey: key} + } + for _, e := range []listening.Event{ + ev(u.ID, pixies, listening.Complete, 1000, 1000, "p1"), // counts: complete + ev(u.ID, pixies, listening.Play, 30_000, 2000, "p2"), // counts: long enough + ev(u.ID, pixies, listening.Play, 5_000, 3000, "p3"), // too short + ev(u.ID, blur, listening.Skip, 1_000, 4000, "p4"), // a skip + ev(other.ID, blur, listening.Complete, 1000, 5000, "p5"), + } { + if _, err := listens.Record(ctx, e); err != nil { + t.Fatal(err) + } + } + outside := NewOutsidePlayRepo(db) + if _, err := outside.RecordOutside(ctx, u.ID, []spotifyimport.OutsidePlay{ + {Key: "o1", Artist: "Giorgio Canali", PlayedAt: time.Unix(6000, 0)}, + {Key: "p1", Artist: "Pixies", PlayedAt: time.Unix(1000, 0)}, // the same play as the event p1 + }); err != nil { + t.Fatal(err) + } + + repo := NewAffinityRepo(db) + plays, err := repo.Plays(ctx, u.ID, minListen) + if err != nil { + t.Fatal(err) + } + got := map[string]int{} + for _, p := range plays { + got[p.Name]++ + if p.Key != SearchKey(p.Name) { + t.Errorf("key %q of %q, want its search key", p.Key, p.Name) + } + } + if len(plays) != 3 || got["Pixies"] != 2 || got["Giorgio Canali"] != 1 { + t.Errorf("plays = %v (%d), want Pixies twice and Canali once: the short one, the skip, the other person's and the duplicate are out", got, len(plays)) + } + if n, err := repo.PlayCount(ctx, u.ID, minListen); err != nil || n != len(plays) { + t.Errorf("PlayCount = %d, %v, want %d", n, err, len(plays)) + } + + if _, ok, err := repo.Build(ctx, u.ID); err != nil || ok { + t.Errorf("Build before any = %v, %v, want none", ok, err) + } + at := time.Unix(9000, 0) + model := affinity.Model{ + Plays: 3, Sessions: 2, + Artists: []affinity.Artist{ + {Key: "pixies", Name: "Pixies", Plays: 2, Sessions: 2, Last: time.Unix(2000, 0), Weight: 1.5}, + {Key: "blur", Name: "Blur", Plays: 1, Sessions: 1, Last: time.Unix(1000, 0), Weight: 0.5}, + {Key: "giorgio canali", Name: "Giorgio Canali", Plays: 1, Sessions: 1, Last: time.Unix(6000, 0), Weight: 3}, + }, + Links: []affinity.Link{{A: "blur", B: "pixies", Score: 0.5}, {A: "giorgio canali", B: "pixies", Score: 0.25}}, + } + if err := repo.Save(ctx, u.ID, at, model); err != nil { + t.Fatal(err) + } + b, ok, err := repo.Build(ctx, u.ID) + if err != nil || !ok || b.Plays != 3 || b.Sessions != 2 || b.Artists != 3 || b.Links != 2 || b.BuiltAt.Unix() != 9000 { + t.Errorf("Build = %+v, %v, %v", b, ok, err) + } + seeds, err := repo.Seeds(ctx, u.ID, 2) + if err != nil || len(seeds) != 2 || seeds[0].Key != "giorgio canali" || seeds[1].Key != "pixies" || seeds[1].Last.Unix() != 2000 { + t.Errorf("Seeds = %+v, %v, want the two heaviest, heaviest first", seeds, err) + } + arts, err := repo.Artists(ctx, u.ID, []string{"blur", "nobody"}) + if err != nil || len(arts) != 1 || arts["blur"].Name != "Blur" { + t.Errorf("Artists = %+v, %v", arts, err) + } + links, err := repo.Neighbours(ctx, u.ID, []string{"blur"}) + if err != nil || len(links) != 1 || links[0].A != "blur" || links[0].B != "pixies" || links[0].Score != 0.5 { + t.Errorf("Neighbours(blur) = %+v, %v", links, err) + } + if links, _ := repo.Neighbours(ctx, u.ID, []string{"pixies"}); len(links) != 2 { + t.Errorf("Neighbours(pixies) = %+v, want both links, from either end", links) + } + if links, _ := repo.Neighbours(ctx, other.ID, []string{"pixies"}); len(links) != 0 { + t.Errorf("another person's links = %+v, want none", links) + } + if links, err := repo.Neighbours(ctx, u.ID, nil); err != nil || links != nil { + t.Errorf("Neighbours of nobody = %v, %v", links, err) + } + + // Building again replaces it, and drops the kept play that has an event now. + if err := repo.Save(ctx, u.ID, at.Add(time.Hour), affinity.Model{Plays: 1}); err != nil { + t.Fatal(err) + } + if b, _, _ := repo.Build(ctx, u.ID); b.Artists != 0 || b.Links != 0 || b.Plays != 1 { + t.Errorf("Build after a smaller model = %+v", b) + } + if seeds, _ := repo.Seeds(ctx, u.ID, 10); len(seeds) != 0 { + t.Errorf("Seeds = %+v, want none after it was replaced", seeds) + } + if got := count(t, db, "SELECT COUNT(*) FROM outside_plays WHERE user_id = ?", u.ID); got != 1 { + t.Errorf("%d kept plays left, want 1 (the one that is a listening event was dropped)", got) + } + }) +} + +func TestLibraryArtistsSayWhichHaveMusicAndWishlistFindsOrMakesThem(t *testing.T) { + forEachBackend(t, func(t *testing.T, db *bun.DB, _ string) { + ctx := context.Background() + u, err := NewUserRepo(db).CreateUser(ctx, "alice", "h", false) + if err != nil { + t.Fatal(err) + } + lib, err := NewLibraryRepo(db).CreateLibrary(ctx, "music", "/music") + if err != nil { + t.Fatal(err) + } + affinitySong(t, db, lib.ID, "a.flac", "Debaser", "Pixies") + if _, err := db.ExecContext(ctx, "INSERT INTO artists (name, name_key, search_key, discovered) VALUES ('Björk', 'björk', ?, ?)", SearchKey("Björk"), true); err != nil { + t.Fatal(err) + } + repo := NewAffinityRepo(db) + + got, err := repo.LibraryArtists(ctx, []string{SearchKey("Pixies"), SearchKey("Björk"), "nobody"}) + if err != nil { + t.Fatal(err) + } + if len(got) != 2 || !got[SearchKey("Pixies")].Playable || got[SearchKey("Pixies")].Name != "Pixies" || got[SearchKey("Björk")].Playable || got[SearchKey("Björk")].ID == 0 { + t.Errorf("LibraryArtists = %+v, want Pixies with music, Björk without, nobody not there", got) + } + + // The artist that has the name is the one on the wishlist (also written without the accent) and none is made. + id, err := repo.Wishlist(ctx, u.ID, "Bjork", time.Unix(100, 0)) + if err != nil || id != got[SearchKey("Björk")].ID { + t.Errorf("Wishlist(Bjork) = %d, %v, want the artist that exists (%d)", id, err, got[SearchKey("Björk")].ID) + } + // A new name makes an artist that nothing names but the mark, with no MusicBrainz id. + id2, err := repo.Wishlist(ctx, u.ID, " Giorgio Canali ", time.Unix(200, 0)) + if err != nil || id2 == 0 { + t.Fatalf("Wishlist(new) = %d, %v", id2, err) + } + var name string + var mbidNull, discovered bool + if err := db.QueryRow("SELECT name, mbid IS NULL, discovered FROM artists WHERE id = ?", id2).Scan(&name, &mbidNull, &discovered); err != nil || name != "Giorgio Canali" || !mbidNull || !discovered { + t.Errorf("made artist = %q, no mbid %v, discovered %v, %v", name, mbidNull, discovered, err) + } + again, err := repo.Wishlist(ctx, u.ID, "Giorgio Canali", time.Unix(300, 0)) + if err != nil || again != id2 { + t.Errorf("wishing again = %d, %v, want the same artist %d", again, err, id2) + } + var at int64 + if err := db.QueryRow("SELECT created_at FROM artist_feedback WHERE user_id = ? AND artist_id = ?", u.ID, id2).Scan(&at); err != nil || at != 200 { + t.Errorf("mark date = %d, %v, want the first date 200", at, err) + } + if got := count(t, db, "SELECT COUNT(*) FROM artists WHERE search_key = ?", SearchKey("Giorgio Canali")); got != 1 { + t.Errorf("%d artists named Canali, want 1", got) + } + if _, err := repo.Wishlist(ctx, 999999, "Whoever", time.Unix(1, 0)); err == nil { + t.Error("a wish of a person that does not exist was accepted") + } + }) +} diff --git a/internal/storage/storage_test.go b/internal/storage/storage_test.go index 7ecc04b..7f828bb 100644 --- a/internal/storage/storage_test.go +++ b/internal/storage/storage_test.go @@ -156,8 +156,8 @@ func TestMigrateCreatesSchemaAndIsIdempotent(t *testing.T) { if err := db.QueryRow("SELECT COUNT(*) FROM schema_migrations").Scan(&applied); err != nil { t.Fatal(err) } - if applied != 31 { - t.Errorf("applied migrations = %d, want 31", applied) + if applied != 32 { + t.Errorf("applied migrations = %d, want 32", applied) } }) } diff --git a/migrations/postgres/0032_affinity.sql b/migrations/postgres/0032_affinity.sql new file mode 100644 index 0000000..72ca711 --- /dev/null +++ b/migrations/postgres/0032_affinity.sql @@ -0,0 +1,33 @@ +-- PostgreSQL flavour of sqlite/0032_affinity.sql. + +CREATE TABLE affinity_builds ( + user_id BIGINT PRIMARY KEY REFERENCES users(id) ON DELETE CASCADE, + built_at BIGINT NOT NULL, + plays INTEGER NOT NULL, + sessions INTEGER NOT NULL, + artists INTEGER NOT NULL, + links INTEGER NOT NULL +); + +CREATE TABLE affinity_artists ( + user_id BIGINT NOT NULL REFERENCES users(id) ON DELETE CASCADE, + artist_key TEXT NOT NULL, + name TEXT NOT NULL, + plays INTEGER NOT NULL, + sessions INTEGER NOT NULL, + last_played_at BIGINT NOT NULL, + weight DOUBLE PRECISION NOT NULL, + PRIMARY KEY (user_id, artist_key) +); + +CREATE INDEX idx_affinity_artists_weight ON affinity_artists(user_id, weight DESC); + +CREATE TABLE affinity_links ( + user_id BIGINT NOT NULL REFERENCES users(id) ON DELETE CASCADE, + a_key TEXT NOT NULL, + b_key TEXT NOT NULL, + score DOUBLE PRECISION NOT NULL, + PRIMARY KEY (user_id, a_key, b_key) +); + +CREATE INDEX idx_affinity_links_b ON affinity_links(user_id, b_key); diff --git a/migrations/sqlite/0032_affinity.sql b/migrations/sqlite/0032_affinity.sql new file mode 100644 index 0000000..4987bca --- /dev/null +++ b/migrations/sqlite/0032_affinity.sql @@ -0,0 +1,35 @@ +-- What Sonda learned about which artists a person plays together (see internal/affinity), built again from the +-- listening events and the plays of artists outside the library. It is only a cache of that: it can be dropped and +-- built again. Artists are known by their search key, so one that is not in the library has a place too. + +CREATE TABLE affinity_builds ( + user_id INTEGER PRIMARY KEY REFERENCES users(id) ON DELETE CASCADE, + built_at INTEGER NOT NULL, -- unix seconds + plays INTEGER NOT NULL, -- the plays it was built from + sessions INTEGER NOT NULL, + artists INTEGER NOT NULL, + links INTEGER NOT NULL +); + +CREATE TABLE affinity_artists ( + user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE, + artist_key TEXT NOT NULL, + name TEXT NOT NULL, + plays INTEGER NOT NULL, + sessions INTEGER NOT NULL, + last_played_at INTEGER NOT NULL, -- unix seconds + weight REAL NOT NULL, + PRIMARY KEY (user_id, artist_key) +); + +CREATE INDEX idx_affinity_artists_weight ON affinity_artists(user_id, weight DESC); + +CREATE TABLE affinity_links ( + user_id INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE, + a_key TEXT NOT NULL, -- a_key < b_key + b_key TEXT NOT NULL, + score REAL NOT NULL, + PRIMARY KEY (user_id, a_key, b_key) +); + +CREATE INDEX idx_affinity_links_b ON affinity_links(user_id, b_key); diff --git a/web/src/api/listeningSuggestions.ts b/web/src/api/listeningSuggestions.ts new file mode 100644 index 0000000..3b743f0 --- /dev/null +++ b/web/src/api/listeningSuggestions.ts @@ -0,0 +1,38 @@ +import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query' +import { api } from '@/api/client' +import type { components } from '@/api/schema' + +export type ListeningSuggestions = components['schemas']['ListeningSuggestions'] +export type ListeningSuggestion = components['schemas']['ListeningSuggestion'] + +const KEY = ['me', 'suggestions'] + +/** + * The artists suggested by how you listen: the ones you have and have not played lately, and the ones you play and + * have no music of. Everything is worked out on the server from your own listening. It is asked for only when + * `enabled` (when Discover may start from your listening), and the server builds the model on the first call. + */ +export function useListeningSuggestions({ enabled }: { enabled: boolean }) { + return useQuery({ + queryKey: KEY, + queryFn: () => api.get('/me/suggestions', { limit: 30 }), + enabled, + retry: false, + }) +} + +/** Puts an artist you have played on your wishlist by its name, and reads the lists again. */ +export function useWishPlayedArtist() { + const client = useQueryClient() + return useMutation({ + mutationFn: (name: string) => api.post<{ artist_id: number }>('/me/suggestions/wishlist', { name }), + onSuccess: () => + Promise.all([client.invalidateQueries({ queryKey: KEY }), client.invalidateQueries({ queryKey: ['/wishlist'] }), client.invalidateQueries({ queryKey: ['/discover/dismissed'] })]), + }) +} + +/** Reads the suggestions again, after the artist was wished or dismissed in another way. */ +export function useRefreshListeningSuggestions() { + const client = useQueryClient() + return () => client.invalidateQueries({ queryKey: KEY }) +} diff --git a/web/src/api/schema.d.ts b/web/src/api/schema.d.ts index c528c33..3bb87b3 100644 --- a/web/src/api/schema.d.ts +++ b/web/src/api/schema.d.ts @@ -1480,6 +1480,59 @@ export interface paths { patch?: never; trace?: never; }; + "/me/suggestions": { + parameters: { + query?: never; + header?: never; + path?: never; + cookie?: never; + }; + /** + * Artists suggested by how you listen + * @description Suggestions learned from your own listening, with no provider and nothing sent anywhere. Sonda splits your + * history into sittings (plays less than 30 minutes apart) and links the artists that are often played in the + * same sitting; the model is built from your listening events and from the plays of artists you have no music + * of that an import kept (see `POST /me/import/spotify`), and is built again when plays are added. It needs + * `discover_from_listening` in `PATCH /me/settings` (`listening_not_allowed` without it). + * + * `library` are artists you have, linked to what you play most, that you did not play in the last 60 days, + * best first. `missing` are artists you have played at least three times and have no music of, the ones you play + * most that also go with what you play now first. Each says which of the artists you play it goes with + * (`because`) and has a `score` from 0 to 1 within its list. `artist_id` is null for an artist that does not + * exist on this server yet. Artists you dismissed are left out. + */ + get: operations["getMySuggestions"]; + put?: never; + post?: never; + delete?: never; + options?: never; + head?: never; + patch?: never; + trace?: never; + }; + "/me/suggestions/wishlist": { + parameters: { + query?: never; + header?: never; + path?: never; + cookie?: never; + }; + get?: never; + put?: never; + /** + * Put a suggested artist you do not have on your wishlist + * @description Puts an artist you have played on your wishlist by its name, for the artists of `missing` that have no + * `artist_id`: the artist is made on this server without any music, like the ones a provider names, and has + * no MusicBrainz id. The name must be one the model knows from your listening (`artist_not_played`). Doing it + * twice keeps the first date. It needs `discover_from_listening` like the suggestions. + */ + post: operations["wishArtistFromSuggestions"]; + delete?: never; + options?: never; + head?: never; + patch?: never; + trace?: never; + }; "/me/settings": { parameters: { query?: never; @@ -3257,6 +3310,36 @@ export interface components { /** @description Whether the event counts as a listen under the current rule. */ counted: boolean; }; + ListeningSuggestion: { + name: string; + /** @description The artist on this server, null when there is none yet. */ + artist_id: number | null; + /** @description From 0 to 1 within its list. */ + score: number; + /** @description How many times you played it. */ + plays: number; + /** Format: date-time */ + last_played_at: string; + wishlisted: boolean; + /** @description The artists you play that it goes with, the strongest first, at most three. */ + because: { + name: string; + score: number; + }[]; + }; + ListeningSuggestions: { + library: components["schemas"]["ListeningSuggestion"][]; + missing: components["schemas"]["ListeningSuggestion"][]; + /** @description When the model was built and from how much. */ + built: { + /** Format: date-time */ + at: string; + plays: number; + sessions: number; + artists: number; + links: number; + }; + }; Taste: { artists: components["schemas"]["TasteArtist"][]; tags: components["schemas"]["TasteTag"][]; @@ -6532,6 +6615,86 @@ export interface operations { 401: components["responses"]["Unauthorized"]; }; }; + getMySuggestions: { + parameters: { + query?: { + /** @description How many artists in each list. 20 when absent or 0; values above 50 are lowered to 50. */ + limit?: number; + }; + header?: never; + path?: never; + cookie?: never; + }; + requestBody?: never; + responses: { + /** @description The suggestions. Both lists are empty until you have listened to something. */ + 200: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["ListeningSuggestions"]; + }; + }; + /** @description A negative limit (`invalid_input`), or Discover is not allowed to start from your listening (`listening_not_allowed`). */ + 400: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["Error"]; + }; + }; + 401: components["responses"]["Unauthorized"]; + }; + }; + wishArtistFromSuggestions: { + parameters: { + query?: never; + header?: never; + path?: never; + cookie?: never; + }; + requestBody: { + content: { + "application/json": { + name: string; + }; + }; + }; + responses: { + /** @description The artist is on your wishlist. */ + 200: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": { + artist_id: number; + }; + }; + }; + /** @description The body is not valid (`invalid_input`), or Discover is not allowed to start from your listening (`listening_not_allowed`). */ + 400: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["Error"]; + }; + }; + 401: components["responses"]["Unauthorized"]; + /** @description You have not played an artist with this name (`artist_not_played`). */ + 404: { + headers: { + [name: string]: unknown; + }; + content: { + "application/json": components["schemas"]["Error"]; + }; + }; + }; + }; getMySettings: { parameters: { query?: never; diff --git a/web/src/i18n/locales/en.json b/web/src/i18n/locales/en.json index 14610ce..ff5144f 100644 --- a/web/src/i18n/locales/en.json +++ b/web/src/i18n/locales/en.json @@ -585,6 +585,7 @@ "tabs": { "label": "Discover sections", "suggestions": "Suggestions", + "mine": "From your listening", "wishlist": "Wishlist", "dismissed": "Not interested" }, @@ -609,6 +610,22 @@ "partial": "Last.fm could not be reached for some artists, so this list may be shorter.", "busy": "Discover is busy with another request. Try again in a moment.", "retryIn": "Try again in {{seconds}} s", + "mine": { + "needs": "These suggestions are learned from what you play. Turn on starting Discover from your listening to see them.", + "intro": "Sonda looks at which artists you play in the same sitting, and suggests more of what goes together. It is worked out on your server from your own listening, and nothing is sent anywhere.", + "built": "Learned from {{plays}} plays in {{sessions}} sittings, on {{when}}.", + "nothing": "You have not played anything yet. Play some music, or import your history from Spotify, and the suggestions appear here.", + "libraryTitle": "In your library, not played lately", + "libraryHint": "Artists you have that go with what you play, and that you have not played for two months.", + "libraryNone": "Nothing to bring back right now.", + "missingTitle": "You play them, but you do not have them", + "missingHint": "Artists you have played at least three times and have no music of here, the ones you play most that also go with what you play now first.", + "missingNone": "No artist you play is missing from your library.", + "lastPlayed": "Last played {{when}}", + "plays_one": "{{count}} play", + "plays_other": "{{count}} plays", + "because": "Goes with {{names}}" + }, "marked": { "hideOwned": "Hide the ones I already have", "remove": "Remove", diff --git a/web/src/i18n/locales/it.json b/web/src/i18n/locales/it.json index 606852b..6c7f830 100644 --- a/web/src/i18n/locales/it.json +++ b/web/src/i18n/locales/it.json @@ -585,6 +585,7 @@ "tabs": { "label": "Sezioni di Discover", "suggestions": "Suggerimenti", + "mine": "Dai tuoi ascolti", "wishlist": "Wishlist", "dismissed": "Non mi interessa" }, @@ -609,6 +610,22 @@ "partial": "Last.fm non era raggiungibile per alcuni artisti, quindi la lista può essere più corta.", "busy": "Discover è occupato con un'altra richiesta. Riprova tra poco.", "retryIn": "Riprova tra {{seconds}} s", + "mine": { + "needs": "Questi suggerimenti nascono da quello che ascolti. Attiva l'avvio di Discover dai tuoi ascolti per vederli.", + "intro": "Sonda guarda quali artisti ascolti nella stessa sessione e propone altro di ciò che sta bene insieme. Si calcola sul tuo server dai tuoi ascolti e non viene inviato niente a nessuno.", + "built": "Imparato da {{plays}} ascolti in {{sessions}} sessioni, il {{when}}.", + "nothing": "Non hai ancora ascoltato niente. Ascolta un po' di musica, o importa la tua cronologia da Spotify, e qui compaiono i suggerimenti.", + "libraryTitle": "Nella tua libreria, non ascoltati di recente", + "libraryHint": "Artisti che hai e che stanno bene con quello che ascolti, ma che non ascolti da due mesi.", + "libraryNone": "Per ora non c'è niente da riportare a galla.", + "missingTitle": "Li ascolti, ma non li hai", + "missingHint": "Artisti che hai ascoltato almeno tre volte e di cui qui non hai musica: prima quelli che ascolti di più e che stanno bene con quello che ascolti ora.", + "missingNone": "Nessun artista che ascolti manca dalla tua libreria.", + "lastPlayed": "Ultimo ascolto {{when}}", + "plays_one": "{{count}} ascolto", + "plays_other": "{{count}} ascolti", + "because": "Sta bene con {{names}}" + }, "marked": { "hideOwned": "Nascondi quelli che ho già", "remove": "Togli", diff --git a/web/src/pages/DiscoverPage.test.tsx b/web/src/pages/DiscoverPage.test.tsx index f9e3555..c3c2345 100644 --- a/web/src/pages/DiscoverPage.test.tsx +++ b/web/src/pages/DiscoverPage.test.tsx @@ -84,8 +84,10 @@ test('the arrow keys move between the tabs', async () => { renderPage() tab('Suggestions').focus() await userEvent.keyboard('{ArrowRight}') + expect(tab('From your listening')).toHaveAttribute('aria-selected', 'true') + expect(tab('From your listening')).toHaveFocus() + await userEvent.keyboard('{ArrowRight}') expect(tab('Wishlist')).toHaveAttribute('aria-selected', 'true') - expect(tab('Wishlist')).toHaveFocus() - await userEvent.keyboard('{ArrowLeft}{ArrowLeft}') + await userEvent.keyboard('{ArrowLeft}{ArrowLeft}{ArrowLeft}') expect(tab('Not interested')).toHaveAttribute('aria-selected', 'true') }) diff --git a/web/src/pages/DiscoverPage.tsx b/web/src/pages/DiscoverPage.tsx index 6dcb20a..e65bd63 100644 --- a/web/src/pages/DiscoverPage.tsx +++ b/web/src/pages/DiscoverPage.tsx @@ -3,11 +3,13 @@ import { useTranslation } from 'react-i18next' import { useSearchParams } from 'react-router-dom' import { cn } from '@/lib/utils' import MarkedList from './discover/MarkedList' +import ListeningTab from './discover/ListeningTab' import SuggestionsTab from './discover/SuggestionsTab' import PageTitle from '@/components/PageTitle' const TABS = [ { id: 'suggestions', key: 'discover.tabs.suggestions' }, + { id: 'mine', key: 'discover.tabs.mine' }, { id: 'wishlist', key: 'discover.tabs.wishlist' }, { id: 'dismissed', key: 'discover.tabs.dismissed' }, ] as const @@ -67,7 +69,7 @@ export default function DiscoverPage() {
- {current === 'suggestions' ? : } + {current === 'suggestions' ? : current === 'mine' ? : }
) diff --git a/web/src/pages/discover/ListeningTab.test.tsx b/web/src/pages/discover/ListeningTab.test.tsx new file mode 100644 index 0000000..ba8ad36 --- /dev/null +++ b/web/src/pages/discover/ListeningTab.test.tsx @@ -0,0 +1,142 @@ +import { QueryClient, QueryClientProvider } from '@tanstack/react-query' +import { render, screen, within } from '@testing-library/react' +import userEvent from '@testing-library/user-event' +import { MemoryRouter } from 'react-router-dom' +import { beforeEach, expect, test, vi } from 'vitest' +import { sessionStore } from '@/auth/session' +import { initI18n } from '@/i18n' +import { apiError, mockFetch, type MockRoutes } from '@/test/mockFetch' +import ListeningTab from './ListeningTab' + +const none = () => new Response(null, { status: 204 }) + +const item = (name: string, id: number | null, over: object = {}) => ({ + name, + artist_id: id, + score: 0.9, + plays: 3, + last_played_at: '2025-01-10T10:00:00Z', + wishlisted: false, + because: [{ name: 'Radiohead', score: 0.7 }, { name: 'Blur', score: 0.2 }], + ...over, +}) + +const built = { at: '2026-11-15T10:00:00Z', plays: 1200, sessions: 340, artists: 80, links: 500 } + +const suggestions = (over: object = {}) => ({ library: [item('Pixies', 2)], missing: [item('Giorgio Canali', null, { plays: 40, score: 0.6 })], built, ...over }) + +function setup(routes: MockRoutes, allowed = true) { + const mocked = mockFetch({ + 'GET /api/v1/me/settings': () => ({ discover_from_listening: allowed }), + 'GET /api/v1/me/suggestions': () => suggestions(), + ...routes, + }) + const client = new QueryClient({ defaultOptions: { queries: { retry: false }, mutations: { retry: false } } }) + render( + + + + + , + ) + return mocked +} + +beforeEach(async () => { + vi.spyOn(navigator, 'language', 'get').mockReturnValue('en-US') + await initI18n() + sessionStore.setState({ status: 'authenticated', user: { id: 1, email: 'tom@example.org', is_admin: false } }) +}) + +test('without permission it explains and offers the switch, and asks the server for nothing', async () => { + const { calls } = setup({}, false) + expect(await screen.findByText(/Turn on starting Discover from your listening/)).toBeInTheDocument() + expect(screen.getByRole('switch')).toBeInTheDocument() + expect(calls.some((c) => c.key === 'GET /api/v1/me/suggestions')).toBe(false) +}) + +test('the two lists say why each artist is there and when it was last played', async () => { + setup({}) + const library = await screen.findByRole('region', { name: 'In your library, not played lately' }) + expect(within(library).getByRole('link', { name: 'Pixies' })).toHaveAttribute('href', '/artists/2') + expect(within(library).getByText('90% match')).toBeInTheDocument() + expect(within(library).getByText('Last played Jan 10, 2025')).toBeInTheDocument() + expect(within(library).getByText('3 plays')).toBeInTheDocument() + expect(within(library).getByText('Goes with Radiohead, Blur')).toBeInTheDocument() + expect(within(library).queryByRole('button', { name: 'Wishlist' })).not.toBeInTheDocument() + + const missing = screen.getByRole('region', { name: 'You play them, but you do not have them' }) + expect(within(missing).getByText('Giorgio Canali')).toBeInTheDocument() + expect(within(missing).queryByRole('link', { name: 'Giorgio Canali' })).not.toBeInTheDocument() + expect(within(missing).getByText('40 plays')).toBeInTheDocument() + expect(within(missing).queryByRole('button', { name: 'Not interested' })).not.toBeInTheDocument() // no artist to dismiss yet + expect(screen.getByText(/Learned from 1200 plays in 340 sittings/)).toBeInTheDocument() +}) + +test('an artist that is missing goes on the wishlist by its name, and the list shows it', async () => { + let wished = false + const { calls } = setup({ + 'GET /api/v1/me/suggestions': () => suggestions({ missing: [item('Giorgio Canali', wished ? 9 : null, { wishlisted: wished })] }), + 'POST /api/v1/me/suggestions/wishlist': () => { + wished = true + return { artist_id: 9 } + }, + }) + const button = await screen.findByRole('button', { name: 'Wishlist' }) + expect(button).toHaveAttribute('aria-pressed', 'false') + await userEvent.click(button) + const sent = calls.filter((c) => c.key === 'POST /api/v1/me/suggestions/wishlist') + expect(sent).toHaveLength(1) + expect(JSON.parse(sent[0].init.body as string)).toEqual({ name: 'Giorgio Canali' }) + expect(await screen.findByRole('button', { name: 'Wishlist', pressed: true })).toBeInTheDocument() +}) + +test('an artist already on the wishlist can be taken off', async () => { + let wished = true + const { calls } = setup({ + 'GET /api/v1/me/suggestions': () => suggestions({ missing: [item('Giorgio Canali', 9, { wishlisted: wished })] }), + 'DELETE /api/v1/wishlist/9': () => { + wished = false + return none() + }, + 'GET /api/v1/wishlist': () => ({ items: [], total: 0, limit: 50, offset: 0 }), + }) + await userEvent.click(await screen.findByRole('button', { name: 'Wishlist', pressed: true })) + expect(await screen.findByRole('button', { name: 'Wishlist', pressed: false })).toBeInTheDocument() + expect(calls.some((c) => c.key === 'DELETE /api/v1/wishlist/9')).toBe(true) +}) + +test('"Not interested" dismisses the artist and the list is read again without it', async () => { + let gone = false + const { calls } = setup({ + 'GET /api/v1/me/suggestions': () => suggestions({ library: gone ? [] : [item('Pixies', 2)] }), + 'POST /api/v1/discover/2/dismiss': () => { + gone = true + return none() + }, + }) + const library = await screen.findByRole('region', { name: 'In your library, not played lately' }) + await userEvent.click(within(library).getByRole('button', { name: 'Not interested' })) + expect(await screen.findByText('Nothing to bring back right now.')).toBeInTheDocument() + expect(calls.some((c) => c.key === 'POST /api/v1/discover/2/dismiss')).toBe(true) +}) + +test('with no plays yet it says what to do', async () => { + setup({ 'GET /api/v1/me/suggestions': () => suggestions({ library: [], missing: [], built: { ...built, plays: 0, sessions: 0, artists: 0, links: 0 } }) }) + expect(await screen.findByText(/You have not played anything yet/)).toBeInTheDocument() +}) + +test('a list with nothing in it says so', async () => { + setup({ 'GET /api/v1/me/suggestions': () => suggestions({ library: [], missing: [] }) }) + expect(await screen.findByText('Nothing to bring back right now.')).toBeInTheDocument() + expect(screen.getByText('No artist you play is missing from your library.')).toBeInTheDocument() +}) + +test('a server error has a way to try again', async () => { + let fail = true + setup({ 'GET /api/v1/me/suggestions': () => (fail ? apiError(500, 'internal', 'x') : suggestions()) }) + const retry = await screen.findByRole('button', { name: 'Try again' }) + fail = false + await userEvent.click(retry) + expect(await screen.findByRole('link', { name: 'Pixies' })).toBeInTheDocument() +}) diff --git a/web/src/pages/discover/ListeningTab.tsx b/web/src/pages/discover/ListeningTab.tsx new file mode 100644 index 0000000..9b367b9 --- /dev/null +++ b/web/src/pages/discover/ListeningTab.tsx @@ -0,0 +1,138 @@ +import { useTranslation } from 'react-i18next' +import { Link } from 'react-router-dom' +import { ApiError } from '@/api/client' +import { useArtistMarks, useMySettings } from '@/api/discover' +import { + useListeningSuggestions, + useRefreshListeningSuggestions, + useWishPlayedArtist, + type ListeningSuggestion, +} from '@/api/listeningSuggestions' +import ListeningSwitch from '@/components/ListeningSwitch' +import { EmptyMessage, ErrorMessage, Loading } from '@/components/Status' +import { Button } from '@/components/ui/button' +import { errorMessageKey } from '@/i18n' + +/** One artist: how well it goes with what you play, why, when you last played it, and what you can do about it. */ +function Row({ item, owned }: { item: ListeningSuggestion; owned: boolean }) { + const { t, i18n } = useTranslation() + const marks = useArtistMarks() + const wish = useWishPlayedArtist() + const refresh = useRefreshListeningSuggestions() + const id = item.artist_id + const when = new Intl.DateTimeFormat(i18n.language, { dateStyle: 'medium' }).format(new Date(item.last_played_at)) + const failure = wish.error ?? marks.addToWishlist.error ?? marks.removeFromWishlist.error ?? marks.dismiss.error + + function toggleWish() { + if (item.wishlisted && id !== null) marks.removeFromWishlist.mutate(id, { onSuccess: refresh }) + else wish.mutate(item.name) + } + + return ( +
  • +
    +
    + {owned && id !== null ? ( + + {item.name} + + ) : ( + {item.name} + )} +
    + {t('discover.score', { percent: Math.round(item.score * 100) })} + {t('discover.mine.lastPlayed', { when })} + {t('discover.mine.plays', { count: item.plays })} +
    + {item.because.length > 0 && ( +

    + {t('discover.mine.because', { names: item.because.map((b) => b.name).join(', ') })} +

    + )} +
    +
    + {!owned && ( + + )} + {id !== null && ( + + )} +
    +
    + {failure ? ( +

    + {t(errorMessageKey(failure instanceof ApiError ? failure.code : 'unknown'))} +

    + ) : null} +
  • + ) +} + +function Group({ title, hint, empty, items, owned }: { title: string; hint: string; empty: string; items: ListeningSuggestion[]; owned: boolean }) { + return ( +
    +

    {title}

    +

    {hint}

    + {items.length === 0 ? ( + {empty} + ) : ( + + )} +
    + ) +} + +/** + * Suggestions learned from your own listening: Sonda looks at which artists you play in the same sitting, and + * suggests the ones you have and have not played lately, and the ones you play and have no music of. It needs the + * switch that lets Discover start from your listening; without it, this is the place to turn it on. + */ +export default function ListeningTab() { + const { t, i18n } = useTranslation() + const settings = useMySettings() + const allowed = settings.data?.discover_from_listening === true + const suggestions = useListeningSuggestions({ enabled: allowed }) + + if (settings.isPending) return + if (!allowed) { + return ( +
    +

    {t('discover.mine.needs')}

    + +
    + ) + } + if (suggestions.isError) return void suggestions.refetch()} /> + if (suggestions.isPending) return + + const data = suggestions.data + const built = new Intl.DateTimeFormat(i18n.language, { dateStyle: 'medium', timeStyle: 'short' }).format(new Date(data.built.at)) + return ( +
    +
    +

    {t('discover.mine.intro')}

    + {data.built.plays > 0 && ( +

    + {t('discover.mine.built', { plays: data.built.plays, sessions: data.built.sessions, when: built })} +

    + )} +
    + {data.built.plays === 0 ? ( + {t('discover.mine.nothing')} + ) : ( + <> + + + + )} +
    + ) +}