feat: favourites teach Sonda my taste
A heart adds to what the listening says of an artist: a favourite artist counts as much as three complete listens, a favourite album 1.5 for its album artist and a favourite song 1 for each of its main artists, at most 6 for one artist, without fading with time. An artist hearted but never played is liked, with no listens. That feeds Discover, the mixes and the smart playlists that use the taste, and GET /me/taste. In the mixes of what the listener likes (genre mixes, Your favourites, Rediscover) a hearted song is wanted 1.5 times as much and a song on a hearted album 1.25 times. New to you is not changed.
This commit is contained in:
parent
84b61ce6aa
commit
ca5cbbf088
12 changed files with 467 additions and 17 deletions
|
|
@ -21,7 +21,7 @@ It is a single executable, one database (SQLite by default, PostgreSQL if you pr
|
|||
- **Artist descriptions.** `GET /artists/{id}/description` shows what Wikipedia says about an artist: a short text, with the page it comes from, a link to it and the licence (CC BY-SA 4.0), which you must show next to it. With the `wikipedia` provider and the `musicbrainz` one turned on (the link to Wikidata comes from MusicBrainz), Sonda follows ids and never a name: the artist's MusicBrainz id, then the Wikidata item that MusicBrainz links to it, then the Wikipedia page that Wikidata names in the first of your languages (`[wikipedia] languages`, the code of that Wikipedia, for example `["it", "en"]`) that has one. An artist without a MusicBrainz id has no description. It is kept for 30 days and removed with the provider's data. To read the descriptions of the whole library in one go, an admin starts a background job like a scan: `POST /library/descriptions` (follow it with `GET /library/descriptions`, stop it with `DELETE /library/descriptions`; `?refresh=true` reads every artist again). It only goes through artists that have a MusicBrainz id, never replaces a description by a failure, and ends at once, saying so, when a provider it needs is off. See [External providers](#external-providers).
|
||||
- **Artist photos.** `GET /artists/{id}/image` serves a free picture of an artist from Wikimedia Commons, and `GET /artists/{id}/image/credit` says whose it is and under which licence, which you must show next to it. Only free licences are kept (public domain, CC0, CC BY, CC BY-SA): non-commercial, no-derivatives, fair-use and unknown ones are refused, because Sonda stores a copy (`?size=small` is 300 pixels, `large` 1000) in the data folder. With the `commons` provider and the `musicbrainz` one turned on, Sonda follows ids and never a name: the artist's MusicBrainz id, the Wikidata item MusicBrainz links to it, the picture Wikidata names for it, and Commons for its licence. Many artists will have none. Kept for 30 days (a week when there is none) and removed, files included, with the provider's data. To look for the pictures of the whole library in one go, an admin starts a background job like a scan: `POST /library/artist-images` (follow it with `GET /library/artist-images`, stop it with `DELETE /library/artist-images`; `?refresh=true` searches every artist again). It only goes through artists that have a MusicBrainz id, never replaces a picture by a failure, and ends at once, saying so, when a provider it needs is off. See [External providers](#external-providers).
|
||||
- **Album covers.** `GET /albums/{id}/cover` serves the cover of an album from your own files: the picture embedded in them, or else a `cover`, `folder`, `front`, `album` or `albumart` image in the album's folder. It is built the first time it is asked for (a few at a time), scaled (`?size=small` is 300 pixels, `large` 1000) and kept in the data folder (`storage.data_dir`, you can delete it: covers are built again). A picture you replace in a folder is noticed within a week, or at once after a full rescan. When your files have no picture at all and an admin turned on the `coverartarchive` provider, the cover is asked from the Cover Art Archive by the MusicBrainz release group id that the tags of the album's files carry (`MUSICBRAINZ_RELEASEGROUPID`; run a full rescan to read it for music you already indexed); your own files always come first, and only that id leaves the server. An album whose files have no id is looked up in MusicBrainz by its artist and title when the `musicbrainz` provider is on too (see the list of what leaves your server). For an album split in `CD1`/`Disc 2` folders, the picture in the folder above is used. Nothing is sent anywhere unless you turned on the `coverartarchive` provider (see [External providers](#external-providers)). It also takes `?access_token=`, so an `<img>` element can use it. To build the covers of the whole library in one go instead of one album at a time, an admin starts a background job like a scan: `POST /library/covers` (follow it with `GET /library/covers`, stop it with `DELETE /library/covers`; `?refresh=true` searches every album again; it never replaces a cover by "no cover" because a disk was not mounted, and keeps the covers that came from the archive if you switched it off).
|
||||
- **Mixes.** `GET /mixes` lists playlists that Sonda makes for you, like the mixes of a streaming service, from what you listen to and from the genres of your music: one mix for each genre of your library that has at least 30 songs by 4 artists (the genres you play most first), and, once you have some listening history, *Your favourites* and *Rediscover* (artists you like and have not played for a month); *New to you* is the songs you never played. `GET /mixes/{id}` gives a mix of up to 50 songs, at most 3 by one artist and never two of the same artist in a row. They are picked by weighted chance: the artists you play weigh more and songs you skipped lately or artists you avoid are left out. A mix stays the same for a day, and `?variant=2` gives another; `POST /mixes/{id}/playlist` keeps the one you are looking at as a playlist of your own. A mix is worked out when you ask for it, from your own history and the genre tags of your files: nothing is stored and nothing is sent to a provider. The id of a genre mix has the genre in it (`genre:hard rock`), so it is percent-encoded in the path. Each mix says which albums stand for it (`album_ids`: up to 4 albums that have a cover, by different artists, with most of the songs the mix draws from), for a picture made of their covers; the covers are the ones the library already has, so a mix has a picture once the covers of its albums were looked up (browsing the albums, or the covers job), and none is fetched for it.
|
||||
- **Mixes.** `GET /mixes` lists playlists that Sonda makes for you, like the mixes of a streaming service, from what you listen to and from the genres of your music: one mix for each genre of your library that has at least 30 songs by 4 artists (the genres you play most first), and, once you have some listening history, *Your favourites* (the artists you like: the ones you play, and the ones you marked with a heart) and *Rediscover* (artists you like and have not played for a month); *New to you* is the songs you never played. `GET /mixes/{id}` gives a mix of up to 50 songs, at most 3 by one artist and never two of the same artist in a row. They are picked by weighted chance: the artists you play weigh more and songs you skipped lately or artists you avoid are left out. A mix stays the same for a day, and `?variant=2` gives another; `POST /mixes/{id}/playlist` keeps the one you are looking at as a playlist of your own. A mix is worked out when you ask for it, from your own history and the genre tags of your files: nothing is stored and nothing is sent to a provider. The id of a genre mix has the genre in it (`genre:hard rock`), so it is percent-encoded in the path. Each mix says which albums stand for it (`album_ids`: up to 4 albums that have a cover, by different artists, with most of the songs the mix draws from), for a picture made of their covers; the covers are the ones the library already has, so a mix has a picture once the covers of its albums were looked up (browsing the albums, or the covers job), and none is fetched for it.
|
||||
- **ReplayGain.** The scan reads the ReplayGain tags of each file (`REPLAYGAIN_TRACK_GAIN`, `REPLAYGAIN_ALBUM_GAIN` and the two peaks), offline, and keeps them on the file; a tag that makes no sense (not a number, more than 60 dB) is left out. A song in the API has them as `replay_gain` (gains in decibels, peaks as a fraction of full scale), or `null` when its first playable file says nothing; the files indexed before this existed have none until a full rescan (`POST /library/scan?full=true`) reads their tags again. The server never changes a file or the sound: a client uses the numbers to play every song at a similar volume, as the web app does (see [Web app](#web-app)).
|
||||
- **Smart playlists.** A smart playlist is a set of rules instead of a list of songs, worked out again every time you open it, so new songs and your listening change it by themselves. `POST /smart-playlists` keeps a name, the rules, an order (`random`, `most_played`, `least_played`, `recently_added` or `title`) and a limit (50 by default, 500 at most); `GET /smart-playlists/{id}/tracks` gives the songs now. Every rule that is set must hold: genres, artists and albums (is, or is not), file formats and a minimum bitrate (about the same file), added in the last N days (by the date of the file), never played, played or not played in the last N days, played at least N times (optionally counting only the plays of the last N days, which is what makes the songs in rotation lately and the top of the year), skipped or not skipped lately, only artists you like or without the ones you avoid, the length of the song, and the year it came out (`min_year` and `max_year`, both included: 1980 to 1989 is the eighties). The year comes from the tags of the file (the original date first, then the date); a song with no year never matches a year rule, and the files indexed before this existed have none until a full rescan (`POST /library/scan?full=true`) reads their tags again. `POST /smart-playlists/preview` tries rules without keeping them, and `POST /smart-playlists/{id}/playlist` keeps what you see as a normal playlist (a copy). Only the rules are stored; the songs come from the files and from your own history, nothing is sent to a provider, and a smart playlist is yours alone, and `GET /genres` lists the genres to choose from. A random order stays the same for a day, and `?variant=2` gives another. `GET /smart-playlists/{id}/covers` and `GET /smart-playlists/suggested/{key}/covers` say which albums stand for one (up to 4 that have a cover, in the order of the songs it gives now), for its picture; they are apart from the songs so that the list of smart playlists shows at once and each card fills in its cover when it arrives. Some are there without anybody making them: `GET /smart-playlists/suggested` lists a fixed set (`recently-added`, `on-rotation`, `top-year`, `most-played`, `never-played`, `forgotten`, `high-quality`, `long-tracks`), the same rules for everyone and worked out on your own library and history, nothing stored; `GET /smart-playlists/suggested/{key}/tracks` gives their songs and `POST /smart-playlists/suggested/{key}/playlist` keeps them as a playlist, and `on-rotation` is the songs you played most in the last 30 days and `top-year` the same over a year, so they change as you listen; a copy of their rules becomes a smart playlist of your own with `POST /smart-playlists`. A user can have 200.
|
||||
- **Playlists.** `POST /playlists` makes a playlist (a name and an optional description) and `GET /playlists` lists yours, the one changed last first. `POST /playlists/{id}/tracks` adds 1 to 500 songs at the end or at a `position`, `GET /playlists/{id}/tracks` lists them in order, `PATCH /playlists/{id}/tracks/{item}` moves one and `DELETE` takes one out; `PATCH /playlists/{id}` renames and `DELETE /playlists/{id}` deletes (the songs stay in the library). A song can be in a playlist more than once, which is why each place has its own item id. A playlist is yours alone: someone else's answers `playlist_not_found`, as one that does not exist. A song whose files were deleted or moved away stays in the playlist and shows `available: false`; it plays again when the files come back and a scan finds them. Each playlist says which albums stand for it (`album_ids`: up to 4 albums that have a cover, in the order they first appear in it, each once), for a picture made of their covers; nothing is fetched for it. A user can have 1000 playlists of up to 10000 songs. Nothing is shared and nothing leaves your server.
|
||||
|
|
@ -37,7 +37,7 @@ It is a single executable, one database (SQLite by default, PostgreSQL if you pr
|
|||
- **Similar artists.** With Last.fm turned on, `GET /artists/{id}/similar` lists artists similar to an artist, and says which of them you already have. See [External providers](#external-providers). Without it, Sonda makes no outbound network requests at all.
|
||||
- **Friends (federation).** Two Sonda servers can become friends, and the listeners of one can read what the other chose to share. `PUT /federation` (admin) turns federation on, which is off by default, and gives this server a name and a public address (`https`; plain `http` only on a network of your own); the server then makes its own Ed25519 key, kept in the database and never returned. `POST /federation/invites` makes an invite that works once and expires in 24 hours (only a hash of its code is kept). The other admin uses it with `POST /federation/friends`: that server is asked to accept this one, the key it answers with must be the one named in the invite, and the friend waits as `outgoing` until the first admin approves it with `POST /federation/friends/{id}/approve` (`POST /federation/friends/{id}/check` lets the waiting side find out). Either side can remove a friend with `DELETE /federation/friends/{id}`. Nothing is shared until an admin turns on `shared` for a library (`PATCH /libraries/{id}`). A friend's server then reads, with requests signed by its key (the method, the path and query, a date, a nonce and the hash of the body), the routes under `/peer`: the artists, albums and songs that have a file in a shared library, their covers once they are built, and the audio itself with `Range`. A friend never sees a path, a library id, a listen, a taste or a user, never makes this server build a cover or ask a provider, and has a few streams at a time and a number of KiB a second to share (`max_streams`, `max_kib_per_sec`); the admin sees the last requests it made (`GET /federation/requests`, the route and the size, never what was searched for). Sharing a library lets friends play it and, in practice, copy its files. A listener browses the music of a friend with `GET /friends` (the names of the friends whose friendship both admins agreed to, never an address or a key) and `GET /friends/{friend}/artists`, `albums` and `tracks` with `q`, `limit` and `offset`, and one item at a time: this server asks the friend with a signed request, checks the shape of the answer and passes it on, and nothing is kept; the ids are the friend's own. A friend that cannot be reached or no longer takes this server for a friend is told in words. The web app has a Friends section for it, shown only when there is a friend to browse. A listener plays it with `GET /friends/{friend}/tracks/{id}/stream` and sees its covers with `GET /friends/{friend}/albums/{id}/cover`, which accept the access token in the query like the local ones: this server asks the friend with a signed request, forwards `Range` and the conditional headers, and passes on only audio (or a JPEG for a cover) with `Content-Type`, `Content-Length`, `Content-Range`, `Accept-Ranges`, `Last-Modified` and `ETag`, so the friend sees this server and never the listener, and a friend cannot make this server serve anything else. A friend's song plays in the same player as the others but is never reported as a listen and does not feed taste. The admin area has a Federation section for all of it.
|
||||
|
||||
- **Favourites.** `PUT /me/favorites/{kind}/{id}` marks a song, an album or an artist (`kind` is `tracks`, `albums` or `artists`) as a favourite and `DELETE` takes it back; `GET /me/favorites/{kind}` lists them, the newest first, and `GET /me/favorites/ids` gives every id for the hearts of a list. They are yours alone: never shown to a friend and never sent anywhere. The web app has a heart on every song, album and artist (and in the player) and a Favourites page. Apps that speak OpenSubsonic see them as starred things.
|
||||
- **Favourites.** `PUT /me/favorites/{kind}/{id}` marks a song, an album or an artist (`kind` is `tracks`, `albums` or `artists`) as a favourite and `DELETE` takes it back; `GET /me/favorites/{kind}` lists them, the newest first, and `GET /me/favorites/ids` gives every id for the hearts of a list. They are yours alone: never shown to a friend and never sent anywhere. Favourites also teach Sonda your taste: a favourite artist counts as much as three complete listens, a favourite album for its artist 1.5, a favourite song 1 (at most 6 for one artist), and the mixes of what you like give a favourite song more chance (and a song of a favourite album a little more). That feeds Discover, the mixes and the smart playlists that use your taste, and `GET /me/taste`. The web app has a heart on every song, album and artist (and in the player) and a Favourites page. Apps that speak OpenSubsonic see them as starred things.
|
||||
- **Other apps.** Apps that speak OpenSubsonic (Symfonium, Feishin, Amperfy, Supersonic and others) can browse, search, play, show covers, use your playlists and report what you listened to, with a password of their own for each app (see [Other apps](#other-apps-opensubsonic)).
|
||||
|
||||
## What is not there yet
|
||||
|
|
|
|||
|
|
@ -2060,6 +2060,13 @@ paths:
|
|||
one skip does not sink an artist and listening makes one loved at most. A track with
|
||||
several artists credits each of them.
|
||||
|
||||
**Favourites** count too (see `/me/favorites`), added to the sum of the artist they point at, without fading
|
||||
with time: a favourite artist +3 (as much as three complete listens), a favourite album +1.5 for its album
|
||||
artist, and a favourite song +1 for each of its main artists, at most +6 for one artist. An artist you
|
||||
marked but never played is liked, with `listens` 0 and `last_listened_at` at the zero date
|
||||
(`0001-01-01T00:00:00Z`). A heart is stronger than one skip. Discover, the mixes and the smart playlists that
|
||||
use your taste follow.
|
||||
|
||||
`artists` are the ones you like (a `preference` above 0), the most played first. They are
|
||||
ordered by the sum itself, so artists whose `preference` rounds to 1 keep their order;
|
||||
ties go to the lower id. `tags` are the kinds of music you like, at most 50: the tags of
|
||||
|
|
@ -6258,7 +6265,7 @@ components:
|
|||
name: {type: string}
|
||||
preference: {type: number, minimum: 0, exclusiveMinimum: true, maximum: 1, description: How much you like the artist. Only artists above 0 are listed.}
|
||||
listens: {type: integer, description: "Events that count as a listen, with no decay, within the period that is read (ten half-lives)."}
|
||||
last_listened_at: {type: string, format: date-time, description: The most recent listen that counted.}
|
||||
last_listened_at: {type: string, format: date-time, description: "The most recent listen that counted; the zero date (`0001-01-01T00:00:00Z`) for an artist that is liked only because of a favourite."}
|
||||
|
||||
TasteTag:
|
||||
type: object
|
||||
|
|
|
|||
|
|
@ -87,6 +87,31 @@ type History struct {
|
|||
SkippedAt time.Time // the last skip, zero when there is none
|
||||
}
|
||||
|
||||
// Heart says why a song is a favourite of the listener: they hearted the song, or the album it is on.
|
||||
type Heart struct {
|
||||
Track bool
|
||||
Album bool
|
||||
}
|
||||
|
||||
// The extra chance a heart gives a song in the mixes of what the listener likes: a hearted song 1.5 times, a song of a
|
||||
// hearted album 1.25 times, and both 1.75 times.
|
||||
const (
|
||||
heartTrackBonus = 0.5
|
||||
heartAlbumBonus = 0.25
|
||||
)
|
||||
|
||||
// heartFactor is how many times more a song is wanted for its hearts.
|
||||
func heartFactor(h Heart) float64 {
|
||||
f := 1.0
|
||||
if h.Track {
|
||||
f += heartTrackBonus
|
||||
}
|
||||
if h.Album {
|
||||
f += heartAlbumBonus
|
||||
}
|
||||
return f
|
||||
}
|
||||
|
||||
// TrackRef is a song that can be played and its main artist (0 when it has none).
|
||||
type TrackRef struct {
|
||||
TrackID int64
|
||||
|
|
@ -116,6 +141,8 @@ type Store interface {
|
|||
// History returns what the user did with the songs they did anything with; minListen is how
|
||||
// long a play must last to count as a listen.
|
||||
History(ctx context.Context, userID int64, minListen time.Duration) (map[int64]History, error)
|
||||
// Hearts returns, for the songs that are favourites of the user or are on a favourite album, which of the two.
|
||||
Hearts(ctx context.Context, userID int64) (map[int64]Heart, error)
|
||||
ArtistNames(ctx context.Context, ids []int64) (map[int64]string, error)
|
||||
// Tracks returns the songs in the order given, leaving out any that cannot be played.
|
||||
Tracks(ctx context.Context, ids []int64) ([]catalog.Track, error)
|
||||
|
|
@ -414,7 +441,8 @@ func (s *Service) Get(ctx context.Context, userID int64, id string, variant int)
|
|||
}
|
||||
|
||||
// candidates weighs the songs a mix may take. A song skipped lately is out, and so is every song
|
||||
// of an artist the listener avoids; a song played in the last days is less wanted.
|
||||
// of an artist the listener avoids; a song played in the last days is less wanted; a song the listener hearted, or one
|
||||
// on an album they hearted, is wanted more (not in New to you, which is about what they have not met).
|
||||
func (s *Service) candidates(ctx context.Context, userID int64, st state, d definition) ([]Candidate, error) {
|
||||
pool, err := s.store.Pool(ctx, Filter{Genres: d.raw, ArtistIDs: d.artists})
|
||||
if err != nil {
|
||||
|
|
@ -424,6 +452,10 @@ func (s *Service) candidates(ctx context.Context, userID int64, st state, d defi
|
|||
if err != nil {
|
||||
return nil, fmt.Errorf("read history: %w", err)
|
||||
}
|
||||
hearts, err := s.store.Hearts(ctx, userID)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read favourites: %w", err)
|
||||
}
|
||||
now := s.now()
|
||||
out := make([]Candidate, 0, len(pool))
|
||||
for _, ref := range pool {
|
||||
|
|
@ -442,11 +474,12 @@ func (s *Service) candidates(ctx context.Context, userID int64, st state, d defi
|
|||
if h.Plays == 0 {
|
||||
w += 0.5
|
||||
}
|
||||
w *= heartFactor(hearts[ref.TrackID])
|
||||
case KindFavourites, KindRediscover:
|
||||
if pref <= 0 {
|
||||
continue
|
||||
}
|
||||
w = pref + 0.2
|
||||
w = (pref + 0.2) * heartFactor(hearts[ref.TrackID])
|
||||
case KindUnheard:
|
||||
if heard {
|
||||
continue
|
||||
|
|
|
|||
|
|
@ -3,6 +3,7 @@ package mixes
|
|||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"math"
|
||||
"slices"
|
||||
"testing"
|
||||
"time"
|
||||
|
|
@ -42,16 +43,22 @@ func TestNormalizeGenreMergesSpellingsAndDropsWhatIsNotAGenre(t *testing.T) {
|
|||
// ---- a store in memory ----
|
||||
|
||||
type fakeStore struct {
|
||||
genres []Genre
|
||||
artists map[string][]string // raw genre -> top artists
|
||||
liked map[int64]map[string]int // artist -> raw genre -> songs
|
||||
pool map[string][]TrackRef // key: "genre:<raw>" or "artists"
|
||||
byArtist []TrackRef // tracks for an artist filter
|
||||
history map[int64]History
|
||||
names map[int64]string
|
||||
asked []Filter
|
||||
covers map[string][]int64 // key: the first genre spelling or "artists"; the albums to give
|
||||
coverAsk []Filter
|
||||
genres []Genre
|
||||
artists map[string][]string // raw genre -> top artists
|
||||
liked map[int64]map[string]int // artist -> raw genre -> songs
|
||||
pool map[string][]TrackRef // key: "genre:<raw>" or "artists"
|
||||
byArtist []TrackRef // tracks for an artist filter
|
||||
history map[int64]History
|
||||
names map[int64]string
|
||||
asked []Filter
|
||||
covers map[string][]int64 // key: the first genre spelling or "artists"; the albums to give
|
||||
coverAsk []Filter
|
||||
hearts map[int64]Heart
|
||||
heartsErr error
|
||||
}
|
||||
|
||||
func (f *fakeStore) Hearts(context.Context, int64) (map[int64]Heart, error) {
|
||||
return f.hearts, f.heartsErr
|
||||
}
|
||||
|
||||
func (f *fakeStore) Covers(_ context.Context, fl Filter, limit int) ([]int64, error) {
|
||||
|
|
@ -479,3 +486,98 @@ func TestOneMixAsksForTheAlbumsOfItself(t *testing.T) {
|
|||
t.Errorf("%d questions about covers for one mix: %+v", len(st.coverAsk), st.coverAsk)
|
||||
}
|
||||
}
|
||||
|
||||
// ---- hearts ----
|
||||
|
||||
// weightOf is the weight a mix gives to a song, with everything else equal.
|
||||
func weightOf(t *testing.T, s *Service, mix string, track int64) float64 {
|
||||
t.Helper()
|
||||
ctx := context.Background()
|
||||
st, err := s.state(ctx, 1)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, d := range st.defs {
|
||||
if d.ID != mix {
|
||||
continue
|
||||
}
|
||||
cands, err := s.candidates(ctx, 1, st, d)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, c := range cands {
|
||||
if c.TrackID == track {
|
||||
return c.Weight
|
||||
}
|
||||
}
|
||||
t.Fatalf("song %d is not a candidate of %s", track, mix)
|
||||
}
|
||||
t.Fatalf("no mix %s", mix)
|
||||
return 0
|
||||
}
|
||||
|
||||
func TestAHeartedSongIsWantedMoreInTheMixesOfWhatIsLiked(t *testing.T) {
|
||||
st := library()
|
||||
recent := day.Add(-2 * 24 * time.Hour)
|
||||
st.byArtist = append(songs(5000, 10, 1), songs(5100, 10, 2)...)
|
||||
st.byArtist = append(st.byArtist, songs(5200, 10, 3)...)
|
||||
// 5000 is a favourite song, 5001 is on a favourite album, 5002 is both, 5003 is neither.
|
||||
st.hearts = map[int64]Heart{5000: {Track: true}, 5001: {Album: true}, 5002: {Track: true, Album: true}}
|
||||
s := newService(st, learned(pref(1, 5, recent), pref(2, 4, recent), pref(3, 3, recent)))
|
||||
|
||||
plain := weightOf(t, s, "favourites", 5003)
|
||||
for track, factor := range map[int64]float64{5000: 1.5, 5001: 1.25, 5002: 1.75} {
|
||||
if got := weightOf(t, s, "favourites", track); math.Abs(got-plain*factor) > 1e-9 {
|
||||
t.Errorf("song %d weighs %v, want %v (x%v of %v)", track, got, plain*factor, factor, plain)
|
||||
}
|
||||
}
|
||||
// A genre mix does the same for the songs of its genre.
|
||||
genrePlain := weightOf(t, s, "genre:rock", 20)
|
||||
st.hearts = map[int64]Heart{20: {Track: true}}
|
||||
if got := weightOf(t, s, "genre:rock", 20); math.Abs(got-genrePlain*1.5) > 1e-9 {
|
||||
t.Errorf("a hearted song of a genre weighs %v, want %v", got, genrePlain*1.5)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAHeartDoesNotMakeAHeardSongNewToYou(t *testing.T) {
|
||||
st := library()
|
||||
st.history = map[int64]History{3: {Plays: 2, LastPlayed: day.Add(-90 * 24 * time.Hour)}}
|
||||
st.hearts = map[int64]Heart{3: {Track: true}, 4: {Track: true}}
|
||||
s := newService(st, learned())
|
||||
ctx := context.Background()
|
||||
state, err := s.state(ctx, 1)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var def definition
|
||||
for _, d := range state.defs {
|
||||
if d.ID == "unheard" {
|
||||
def = d
|
||||
}
|
||||
}
|
||||
cands, err := s.candidates(ctx, 1, state, def)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var w4 float64
|
||||
for _, c := range cands {
|
||||
if c.TrackID == 3 {
|
||||
t.Error("a song that was played is New to you")
|
||||
}
|
||||
if c.TrackID == 4 {
|
||||
w4 = c.Weight
|
||||
}
|
||||
}
|
||||
if w4 != 1 {
|
||||
t.Errorf("a hearted song New to you weighs %v: hearts do not change that mix", w4)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAMixThatCannotReadTheHeartsFails(t *testing.T) {
|
||||
st := library()
|
||||
st.heartsErr = errors.New("boom")
|
||||
s := newService(st, learned())
|
||||
if _, _, err := s.Get(context.Background(), 1, "genre:rock", 0); err == nil {
|
||||
t.Fatal("no error")
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -294,3 +294,31 @@ func (r *MixRepo) Tracks(ctx context.Context, ids []int64) ([]catalog.Track, err
|
|||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// Hearts returns, for the songs that are favourites of the user or are on a favourite album, which of the two. It lists
|
||||
// songs that can no longer be played too: they are never offered, so it does no harm.
|
||||
func (r *MixRepo) Hearts(ctx context.Context, userID int64) (map[int64]mixes.Heart, error) {
|
||||
out := map[int64]mixes.Heart{}
|
||||
for _, q := range []struct {
|
||||
sql string
|
||||
mark func(*mixes.Heart)
|
||||
}{
|
||||
{`SELECT f.item_id FROM favorites f WHERE f.user_id = ? AND f.kind = 'track'`, func(h *mixes.Heart) { h.Track = true }},
|
||||
{`SELECT t.id FROM favorites f JOIN tracks t ON t.album_id = f.item_id WHERE f.user_id = ? AND f.kind = 'album'`, func(h *mixes.Heart) { h.Album = true }},
|
||||
} {
|
||||
rows, err := r.db.QueryContext(ctx, q.sql, userID)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read favourites: %w", err)
|
||||
}
|
||||
ids, err := scanIDs(rows)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, id := range ids {
|
||||
h := out[id]
|
||||
q.mark(&h)
|
||||
out[id] = h
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
|
|
|||
|
|
@ -8,6 +8,7 @@ import (
|
|||
|
||||
"github.com/uptrace/bun"
|
||||
|
||||
"git.kmalloc.org/squid/sonda/internal/favorites"
|
||||
"git.kmalloc.org/squid/sonda/internal/listening"
|
||||
"git.kmalloc.org/squid/sonda/internal/mixes"
|
||||
)
|
||||
|
|
@ -347,3 +348,38 @@ func TestMixRepoCoversAreAlbumsWithACoverByDifferentArtistsWithMostSongsFirst(t
|
|||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestHeartsAreTheSongsHeartedAndTheSongsOfHeartedAlbums(t *testing.T) {
|
||||
forEachBackend(t, func(t *testing.T, db *bun.DB, _ string) {
|
||||
ctx := context.Background()
|
||||
s := seedCatalog(t, db)
|
||||
fav := NewFavoritesRepo(db)
|
||||
uid, other := seedUser(t, db, "a@example.org"), seedUser(t, db, "b@example.org")
|
||||
var ok int64
|
||||
if err := db.QueryRowContext(ctx, "SELECT id FROM albums WHERE title = 'OK Computer'").Scan(&ok); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
at := time.Unix(1_700_000_000, 0)
|
||||
_ = fav.Add(ctx, uid, favorites.Track, s.track["Airbag"], at)
|
||||
_ = fav.Add(ctx, uid, favorites.Track, s.track["Army of Me"], at)
|
||||
_ = fav.Add(ctx, uid, favorites.Album, ok, at)
|
||||
_ = fav.Add(ctx, other, favorites.Track, s.track["So What"], at)
|
||||
|
||||
got, err := NewMixRepo(db).Hearts(ctx, uid)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if h := got[s.track["Airbag"]]; !h.Track || !h.Album { // hearted, and on a hearted album
|
||||
t.Errorf("Airbag = %+v", h)
|
||||
}
|
||||
if h := got[s.track["Karma Police"]]; h.Track || !h.Album {
|
||||
t.Errorf("Karma Police = %+v", h)
|
||||
}
|
||||
if h := got[s.track["Army of Me"]]; !h.Track || h.Album {
|
||||
t.Errorf("Army of Me = %+v", h)
|
||||
}
|
||||
if _, ok := got[s.track["So What"]]; ok {
|
||||
t.Error("another person's heart counted")
|
||||
}
|
||||
})
|
||||
}
|
||||
|
|
|
|||
|
|
@ -193,3 +193,44 @@ func scaleToBest(rows interface {
|
|||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// ArtistFavorites returns, per artist, the hearts of the user that point at it: the artist itself, the albums it is the
|
||||
// album artist of, and the songs it is a main artist of. A heart counts whether or not the music can be played now.
|
||||
func (r *TasteRepo) ArtistFavorites(ctx context.Context, userID int64) (map[int64]taste.Favorites, error) {
|
||||
rows, err := r.db.QueryContext(ctx,
|
||||
`SELECT artist_id, kind, COUNT(*) FROM (
|
||||
SELECT a.id AS artist_id, 'artist' AS kind FROM favorites f JOIN artists a ON a.id = f.item_id
|
||||
WHERE f.user_id = ? AND f.kind = 'artist'
|
||||
UNION ALL
|
||||
SELECT al.artist_id, 'album' FROM favorites f JOIN albums al ON al.id = f.item_id
|
||||
WHERE f.user_id = ? AND f.kind = 'album' AND al.artist_id IS NOT NULL
|
||||
UNION ALL
|
||||
SELECT ta.artist_id, 'track' FROM favorites f JOIN track_artists ta ON ta.track_id = f.item_id AND ta.role = 'main'
|
||||
WHERE f.user_id = ? AND f.kind = 'track'
|
||||
) x GROUP BY artist_id, kind`, userID, userID, userID)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read favourites: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
out := map[int64]taste.Favorites{}
|
||||
for rows.Next() {
|
||||
var (
|
||||
id, n int64
|
||||
kind string
|
||||
)
|
||||
if err := rows.Scan(&id, &kind, &n); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
f := out[id]
|
||||
switch kind {
|
||||
case "artist":
|
||||
f.Artist = true
|
||||
case "album":
|
||||
f.Albums = int(n)
|
||||
default:
|
||||
f.Tracks = int(n)
|
||||
}
|
||||
out[id] = f
|
||||
}
|
||||
return out, rows.Err()
|
||||
}
|
||||
|
|
|
|||
|
|
@ -8,6 +8,7 @@ import (
|
|||
|
||||
"github.com/uptrace/bun"
|
||||
|
||||
"git.kmalloc.org/squid/sonda/internal/favorites"
|
||||
"git.kmalloc.org/squid/sonda/internal/listening"
|
||||
"git.kmalloc.org/squid/sonda/internal/taste"
|
||||
)
|
||||
|
|
@ -160,3 +161,56 @@ func TestTasteArtistTags(t *testing.T) {
|
|||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestTheHeartsOfAListenerAreCountedPerArtist(t *testing.T) {
|
||||
forEachBackend(t, func(t *testing.T, db *bun.DB, _ string) {
|
||||
ctx := context.Background()
|
||||
s := seedCatalog(t, db)
|
||||
repo := NewTasteRepo(db)
|
||||
fav := NewFavoritesRepo(db)
|
||||
uid, other := seedUser(t, db, "a@example.org"), seedUser(t, db, "b@example.org")
|
||||
id := func(table, col, v string) int64 {
|
||||
var n int64
|
||||
if err := db.QueryRowContext(ctx, "SELECT id FROM "+table+" WHERE "+col+" = ?", v).Scan(&n); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return n
|
||||
}
|
||||
radiohead, bjork, miles := id("artists", "name", "Radiohead"), id("artists", "name", "Björk"), id("artists", "name", "Miles Davis")
|
||||
at := time.Unix(1_700_000_000, 0)
|
||||
add := func(user int64, kind favorites.Kind, item int64) {
|
||||
t.Helper()
|
||||
if err := fav.Add(ctx, user, kind, item, at); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
add(uid, favorites.Artist, radiohead)
|
||||
add(uid, favorites.Album, id("albums", "title", "OK Computer")) // credited to Radiohead
|
||||
add(uid, favorites.Track, s.track["Airbag"]) // Radiohead
|
||||
add(uid, favorites.Track, s.track["Karma Police"]) // Radiohead
|
||||
add(uid, favorites.Track, s.track["Army of Me"]) // Björk
|
||||
add(uid, favorites.Album, id("albums", "title", "Various Hits"))
|
||||
add(other, favorites.Artist, miles) // somebody else's
|
||||
|
||||
got, err := repo.ArtistFavorites(ctx, uid)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if r := got[radiohead]; !r.Artist || r.Albums != 1 || r.Tracks != 2 {
|
||||
t.Errorf("Radiohead = %+v", r)
|
||||
}
|
||||
if b := got[bjork]; b.Artist || b.Albums != 0 || b.Tracks != 1 {
|
||||
t.Errorf("Björk = %+v", b)
|
||||
}
|
||||
if _, ok := got[miles]; ok {
|
||||
t.Error("another person's heart counted")
|
||||
}
|
||||
// The album of two other artists is credited to its album artist, Various Artists.
|
||||
if va := got[id("artists", "name", "Various Artists")]; va.Albums != 1 {
|
||||
t.Errorf("Various Artists = %+v", va)
|
||||
}
|
||||
if none, err := repo.ArtistFavorites(ctx, other+100); err != nil || len(none) != 0 {
|
||||
t.Errorf("a person with no hearts: %v %v", none, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
|
|
|||
|
|
@ -39,6 +39,8 @@ type Store interface {
|
|||
// from 0 to 1 (genres of the files and tags that providers gave). It reads
|
||||
// the database only.
|
||||
ArtistTags(ctx context.Context, artistIDs []int64) (map[int64]map[string]float64, error)
|
||||
// ArtistFavorites returns, per artist, the hearts of the listener that point at it.
|
||||
ArtistFavorites(ctx context.Context, userID int64) (map[int64]Favorites, error)
|
||||
// ArtistNames returns the names of the given artists.
|
||||
ArtistNames(ctx context.Context, artistIDs []int64) (map[int64]string, error)
|
||||
}
|
||||
|
|
@ -81,7 +83,8 @@ type Learned struct {
|
|||
Tags []TagWeight // the tag profile of the liked artists, best first, not cut
|
||||
}
|
||||
|
||||
// Learn works out what the listener's history says. Artists are ordered by how much
|
||||
// Learn works out what the listener's history says, and what their favourites say: each heart adds to the artist it
|
||||
// points at (see FavoriteValue). Artists are ordered by how much
|
||||
// the listener played them, recent plays counting more; ties go to the lower id. The
|
||||
// tag profile is built from the MaxProfileArtists most played liked artists.
|
||||
func (s *Service) Learn(ctx context.Context, userID int64) (Learned, error) {
|
||||
|
|
@ -90,7 +93,11 @@ func (s *Service) Learn(ctx context.Context, userID int64) (Learned, error) {
|
|||
if err != nil {
|
||||
return Learned{}, fmt.Errorf("read events: %w", err)
|
||||
}
|
||||
out := Learned{Prefs: Preferences(events, now, s.opts), Liked: []ArtistPref{}, Tags: []TagWeight{}}
|
||||
favs, err := s.store.ArtistFavorites(ctx, userID)
|
||||
if err != nil {
|
||||
return Learned{}, fmt.Errorf("read favourites: %w", err)
|
||||
}
|
||||
out := Learned{Prefs: WithFavorites(Preferences(events, now, s.opts), favs), Liked: []ArtistPref{}, Tags: []TagWeight{}}
|
||||
for _, p := range out.Prefs {
|
||||
if p.Score > 0 {
|
||||
out.Liked = append(out.Liked, p)
|
||||
|
|
|
|||
|
|
@ -19,6 +19,14 @@ type fakeStore struct {
|
|||
tagCalls [][]int64
|
||||
nameCalls [][]int64
|
||||
err error
|
||||
favs map[int64]Favorites
|
||||
favUser int64
|
||||
favErr error
|
||||
}
|
||||
|
||||
func (f *fakeStore) ArtistFavorites(_ context.Context, userID int64) (map[int64]Favorites, error) {
|
||||
f.favUser = userID
|
||||
return f.favs, f.favErr
|
||||
}
|
||||
|
||||
func (f *fakeStore) ArtistEvents(_ context.Context, userID int64, since time.Time) ([]Event, error) {
|
||||
|
|
@ -235,3 +243,37 @@ func TestLearnOfSomeoneWhoHasListenedToNothing(t *testing.T) {
|
|||
t.Errorf("learned = %+v err=%v, want empty and not nil", l, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLearnCountsTheFavouritesOfTheListener(t *testing.T) {
|
||||
f := &fakeStore{
|
||||
events: complete(1, 1),
|
||||
favs: map[int64]Favorites{2: {Artist: true}, 1: {Tracks: 1}},
|
||||
tags: map[int64]map[string]float64{1: {"rock": 1}, 2: {"jazz": 1}},
|
||||
names: map[int64]string{1: "One", 2: "Two"},
|
||||
}
|
||||
l, err := newService(f).Learn(context.Background(), 7)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if f.favUser != 7 {
|
||||
t.Errorf("favourites of user %d", f.favUser)
|
||||
}
|
||||
// Two (a heart: 3) before One (a listen and a heart: 2).
|
||||
if len(l.Liked) != 2 || l.Liked[0].ArtistID != 2 || l.Liked[1].ArtistID != 1 {
|
||||
t.Fatalf("liked = %+v", l.Liked)
|
||||
}
|
||||
if len(l.Tags) != 2 || l.Tags[0].Tag != "jazz" {
|
||||
t.Errorf("tags = %+v", l.Tags)
|
||||
}
|
||||
p, err := newService(f).Taste(context.Background(), 7, 0)
|
||||
if err != nil || len(p.Artists) != 2 || p.Artists[0].ID != 2 || p.Artists[0].Listens != 0 {
|
||||
t.Fatalf("taste = %+v, %v", p, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLearnFailsWhenTheFavouritesCannotBeRead(t *testing.T) {
|
||||
f := &fakeStore{events: complete(1, 1), favErr: errors.New("boom")}
|
||||
if _, err := newService(f).Learn(context.Background(), 7); err == nil {
|
||||
t.Fatal("no error")
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -25,6 +25,51 @@ import (
|
|||
// full recent listens.
|
||||
const sumScale = 3.0
|
||||
|
||||
// What a heart is worth, in the same units as a complete listen (1). A heart is a statement the listener made on
|
||||
// purpose and it does not fade with time, so these are plain sums, not weighed by age.
|
||||
const (
|
||||
FavoriteArtistValue = 3.0 // a favourite artist: three complete listens
|
||||
FavoriteAlbumValue = 1.5 // a favourite album, for its album artist
|
||||
FavoriteTrackValue = 1.0 // a favourite song, for each of its main artists
|
||||
// MaxFavoriteValue is the most the hearts of one artist can add, so that a thousand hearted songs do not put one
|
||||
// artist above everything else.
|
||||
MaxFavoriteValue = 6.0
|
||||
)
|
||||
|
||||
// Favorites are the hearts that point at one artist: the artist itself, the albums it made and the songs it is a main
|
||||
// artist of.
|
||||
type Favorites struct {
|
||||
Artist bool
|
||||
Albums int
|
||||
Tracks int
|
||||
}
|
||||
|
||||
// FavoriteValue is what the hearts of one artist add to what the listening says of it.
|
||||
func FavoriteValue(f Favorites) float64 {
|
||||
v := float64(f.Albums)*FavoriteAlbumValue + float64(f.Tracks)*FavoriteTrackValue
|
||||
if f.Artist {
|
||||
v += FavoriteArtistValue
|
||||
}
|
||||
return math.Min(v, MaxFavoriteValue)
|
||||
}
|
||||
|
||||
// WithFavorites adds what the hearts say to the preferences, in place, and returns them. An artist the listener
|
||||
// never played but hearted becomes liked; its listens stay as they are, because a heart is not a listen.
|
||||
func WithFavorites(prefs map[int64]ArtistPref, favs map[int64]Favorites) map[int64]ArtistPref {
|
||||
for id, f := range favs {
|
||||
v := FavoriteValue(f)
|
||||
if v == 0 {
|
||||
continue
|
||||
}
|
||||
p := prefs[id]
|
||||
p.ArtistID = id
|
||||
p.Score += v
|
||||
p.Preference = math.Tanh(p.Score / sumScale)
|
||||
prefs[id] = p
|
||||
}
|
||||
return prefs
|
||||
}
|
||||
|
||||
// Options are the knobs of the model.
|
||||
type Options struct {
|
||||
MinListen time.Duration // how long a play must last to count as a listen
|
||||
|
|
|
|||
|
|
@ -170,3 +170,58 @@ func TestTagProfileIsEmptyWithoutLikedArtists(t *testing.T) {
|
|||
t.Errorf("got %v", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAHeartCountsLikeListening(t *testing.T) {
|
||||
for name, c := range map[string]struct {
|
||||
f Favorites
|
||||
want float64
|
||||
}{
|
||||
"nothing": {Favorites{}, 0},
|
||||
"the artist": {Favorites{Artist: true}, 3},
|
||||
"two albums": {Favorites{Albums: 2}, 3},
|
||||
"three songs": {Favorites{Tracks: 3}, 3},
|
||||
"all three": {Favorites{Artist: true, Albums: 1, Tracks: 1}, 5.5},
|
||||
"a great many songs": {Favorites{Artist: true, Albums: 40, Tracks: 4000}, MaxFavoriteValue},
|
||||
} {
|
||||
if got := FavoriteValue(c.f); math.Abs(got-c.want) > 1e-9 {
|
||||
t.Errorf("%s: %v, want %v", name, got, c.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestFavouritesMakeAnArtistLikedWithoutAnyListen(t *testing.T) {
|
||||
prefs := Preferences(complete(1, 1), now, options)
|
||||
prefs = WithFavorites(prefs, map[int64]Favorites{
|
||||
1: {Tracks: 1}, // one listen and one song hearted
|
||||
2: {Artist: true}, // never listened to
|
||||
3: {Albums: 1},
|
||||
})
|
||||
if p := prefs[2]; p.Listens != 0 || !p.LastListened.IsZero() || p.Score != 3 || math.Abs(p.Preference-math.Tanh(1)) > 1e-9 {
|
||||
t.Errorf("a favourite artist with no listens: %+v", p)
|
||||
}
|
||||
if p := prefs[1]; p.Listens != 1 || p.Score != 2 { // 1 listen + 1 heart
|
||||
t.Errorf("an artist with both: %+v", p)
|
||||
}
|
||||
if p := prefs[3]; p.Score != 1.5 || p.ArtistID != 3 {
|
||||
t.Errorf("an artist with a favourite album: %+v", p)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAHeartOutweighsASkip(t *testing.T) {
|
||||
prefs := Preferences([]Event{{ArtistID: 1, Type: listening.Skip, PlayedMs: 1000, At: now}}, now, options)
|
||||
if prefs[1].Preference >= 0 {
|
||||
t.Fatalf("one skip: %+v", prefs[1])
|
||||
}
|
||||
prefs = WithFavorites(prefs, map[int64]Favorites{1: {Artist: true}})
|
||||
if prefs[1].Preference <= 0 {
|
||||
t.Errorf("a skip and a heart: %+v", prefs[1])
|
||||
}
|
||||
}
|
||||
|
||||
func TestNoFavouritesChangeNothing(t *testing.T) {
|
||||
before := Preferences(complete(1, 2), now, options)
|
||||
after := WithFavorites(Preferences(complete(1, 2), now, options), nil)
|
||||
if len(before) != len(after) || before[1] != after[1] {
|
||||
t.Fatalf("%+v vs %+v", before, after)
|
||||
}
|
||||
}
|
||||
|
|
|
|||
Loading…
Reference in a new issue