fix(taste): order by the score, clamp future events, keep tags short

- Order the liked artists by the weighted sum, not by tanh, which rounds to
  exactly 1 after about 58 listens and made heavy artists tie.
- An event dated after now counts as happening now, also for the last listen.
- Genres of files that are gone no longer shape the tag profile, as in
  /artists/{id}/tags.
- The tag list is capped at 50.
- A zero half-life falls back to 90 days instead of producing NaN.
- Document the 10 half-lives window and the new rules in the spec and README.
- Tests: heavy artists keep their order, tracks with two artists credit both,
  genres of a missing file are left out, the tag cap, a zero half-life.
This commit is contained in:
tgiachi 2026-11-11 18:52:08 +01:00
commit d251cc4f39
8 changed files with 135 additions and 40 deletions

View file

@ -15,7 +15,7 @@ It is a single executable, one database (SQLite by default, PostgreSQL if you pr
- **Streaming.** `GET` and `HEAD` with `Range` support, so players can seek. Opening or prefetching a stream does not record a listen.
- **API documentation.** The full API is described in OpenAPI and browsable with Swagger UI, served by Sonda itself. The UI is embedded in the binary, so opening it contacts no other server.
- **Listening history.** Clients report explicit `play`, `complete` and `skip` events. An event counts as a listen when it is `complete`, or when it was played for at least 20 seconds (`listening.min_listen_seconds`); the rule is applied when the history is read, so changing it re-reads the past. You can read your own history, with a `counted` flag on every event. It stays in your database and is never sent anywhere.
- **Your taste.** `GET /me/taste` shows what Sonda has learned from your own listening: the artists you like and the kinds of music (tags) they have in common. A complete listen counts for more than a short one, a skip counts against the artist, and older listens weigh less (they halve every 90 days, `taste.half_life_days`). It is worked out on every request from data already in your database; nothing is sent to any provider.
- **Your taste.** `GET /me/taste` shows what Sonda has learned from your own listening: the artists you like and the kinds of music (tags) they have in common. A complete listen counts for more than a short one, a skip counts against the artist, and older listens weigh less (they halve every 90 days, `taste.half_life_days`; only the last ten half-lives are read). It is worked out on every request from data already in your database; nothing is sent to any provider.
- **Tags.** `GET /artists/{id}/tags` shows what Last.fm and MusicBrainz say kind of music an artist plays, each with its own weights, and the genres of the artist's own files, read offline. See [External providers](#external-providers).
- **Similar artists.** With Last.fm switched on, `GET /artists/{id}/similar` lists artists similar to an artist, and says which of them you already have. See [External providers](#external-providers). Without it, Sonda makes no outbound network requests at all.

View file

@ -661,22 +661,26 @@ paths:
Each event has a value: a `complete` listen +1, any other listen that counts +0.5 (see
`POST /listens`), a `skip` before that -1, and a play too short to mean anything 0. An
event weighs half as much for every 90 days (`taste.half_life_days`) that has passed. For
each artist the values add up, and `preference` is `tanh(sum / 3)`: from -1 to 1, never
reaching either, so one skip does not sink an artist and no amount of listening makes
one perfect. A track with several artists credits each of them.
event weighs half as much for every 90 days (`taste.half_life_days`) that has passed, and
one dated after the moment of the request weighs as if it happened then. Only the last
ten half-lives are read (900 days by default); older events weigh less than a thousandth
and are left out, also from `listens`. For each artist the values add up, and `preference`
is `tanh(sum / 3)`: from -1 to 1, approaching both (a large sum rounds to exactly 1), so
one skip does not sink an artist and listening makes one loved at most. A track with
several artists credits each of them.
`artists` are the ones you like (a `preference` above 0), the most liked first, ties going
to the most recent listen. `tags` are the kinds of music you like: the tags of those
artists (the genres of your files, and what Last.fm and MusicBrainz said earlier, if
they were ever asked) added up, each artist weighing as much as its `preference`, and
scaled so the best tag is 1. Tags are read from what is already stored; this request never
asks a provider. The tag profile is built from your 200 most liked artists.
`artists` are the ones you like (a `preference` above 0), the most played first. They are
ordered by the sum itself, so artists whose `preference` rounds to 1 keep their order;
ties go to the lower id. `tags` are the kinds of music you like, at most 50: the tags of
those artists (the genres of your files that still exist, and what Last.fm and MusicBrainz
said earlier, if they were ever asked) added up, each artist weighing as much as its
`preference`, and scaled so the best tag is 1, best first. Tags are read from what is already stored; this
request never asks a provider. The tag profile is built from your 200 most liked artists.
security: [{bearerAuth: []}]
parameters:
- name: limit
in: query
description: How many artists to return. 20 when absent or 0; values above 50 are lowered to 50. Tags are not limited by it.
description: How many artists to return. 20 when absent or 0; values above 50 are lowered to 50. It does not limit `tags`.
schema: {type: integer, minimum: 0, default: 20}
responses:
'200':
@ -1619,7 +1623,7 @@ components:
id: {type: integer, format: int64}
name: {type: string}
preference: {type: number, minimum: 0, exclusiveMinimum: true, maximum: 1, description: How much you like the artist. Only artists above 0 are listed.}
listens: {type: integer, description: "Events that count as a listen, with no decay."}
listens: {type: integer, description: "Events that count as a listen, with no decay, within the period that is read (ten half-lives)."}
last_listened_at: {type: string, format: date-time, description: The most recent listen that counted.}
TasteTag:

View file

@ -95,12 +95,15 @@ func (r *TasteRepo) ArtistTags(ctx context.Context, ids []int64) (map[int64]map[
}
}
// Files: track counts, scaled to the artist's most common genre.
// Files: track counts, scaled to the artist's most common genre. Only tracks
// that still have a file: the genre of a file that is gone says nothing about
// the music that is there (the same rule as the genres of /artists/{id}/tags).
vis, visArgs := visible("t", 0)
files, err := r.db.QueryContext(ctx,
`SELECT ta.artist_id, g.genre, COUNT(DISTINCT ta.track_id)
FROM track_artists ta JOIN track_genres g ON g.track_id = ta.track_id
WHERE ta.artist_id IN (?) AND ta.role = 'main' GROUP BY ta.artist_id, g.genre`,
bun.In(ids))
FROM track_artists ta JOIN track_genres g ON g.track_id = ta.track_id JOIN tracks t ON t.id = ta.track_id
WHERE ta.artist_id IN (?) AND ta.role = 'main' AND `+vis+` GROUP BY ta.artist_id, g.genre`,
append([]any{bun.In(ids)}, visArgs...)...)
if err != nil {
return nil, fmt.Errorf("read genres: %w", err)
}

View file

@ -2,6 +2,7 @@ package storage
import (
"context"
"math"
"testing"
"time"
@ -41,6 +42,10 @@ func TestTasteArtistEvents(t *testing.T) {
record(alice.ID, "Army of Me", "a3", listening.Play, 30_000, recent)
record(bob.ID, "Airbag", "b1", listening.Complete, 1, recent) // somebody else's
// Two artists credited on one track: both are credited with the event.
if _, err := db.Exec("INSERT INTO track_artists (track_id, artist_id, role) VALUES (?, ?, 'main')",
s.track["Song A"], artistID(t, db, "Artist B")); err != nil {
t.Fatal(err)
}
record(alice.ID, "Song A", "a4", listening.Complete, 100_000, recent)
got, err := NewTasteRepo(db).ArtistEvents(ctx, alice.ID, time.Unix(1_600_000_000, 0))
@ -62,15 +67,18 @@ func TestTasteArtistEvents(t *testing.T) {
if len(by[artistID(t, db, "Artist A")]) != 1 {
t.Errorf("the artist credited on Song A has %d events, want 1", len(by[artistID(t, db, "Artist A")]))
}
if len(got) != 3 {
t.Errorf("got %d events, want 3: %+v", len(got), got)
if len(by[artistID(t, db, "Artist B")]) != 1 || by[artistID(t, db, "Artist B")][0].PlayedMs != 100_000 {
t.Errorf("the second artist credited on Song A has %+v, want the same event", by[artistID(t, db, "Artist B")])
}
if len(got) != 4 {
t.Errorf("got %d events, want 4: %+v", len(got), got)
}
// A track whose files are gone still counts: the history is the listener's own.
record(alice.ID, "Ghost", "a5", listening.Complete, 100_000, recent)
got, _ = NewTasteRepo(db).ArtistEvents(ctx, alice.ID, time.Unix(1_600_000_000, 0))
if len(got) != 4 {
t.Errorf("with a track that has no file: %d events, want 4", len(got))
if len(got) != 5 {
t.Errorf("with a track that has no file: %d events, want 5", len(got))
}
})
}
@ -96,7 +104,8 @@ func TestTasteArtistTags(t *testing.T) {
ctx := context.Background()
radiohead, bjork := artistID(t, db, "Radiohead"), artistID(t, db, "Björk")
// Genres of the files: Radiohead has 4 tracks, 4 rock and 2 art rock.
// Genres of the files: Radiohead has 4 tracks. Ghost's file is gone, so its
// genre says nothing about the music that is there: 3 rock and 2 art rock.
for _, g := range []struct{ track, genre string }{
{"Airbag", "rock"}, {"Paranoid Android", "rock"}, {"Karma Police", "rock"}, {"Ghost", "rock"},
{"Airbag", "art rock"}, {"Karma Police", "art rock"},
@ -105,6 +114,9 @@ func TestTasteArtistTags(t *testing.T) {
t.Fatal(err)
}
}
if _, err := db.Exec("INSERT INTO track_genres (track_id, genre) VALUES (?, 'trip hop')", s.track["Ghost"]); err != nil {
t.Fatal(err)
}
// Provider tags: Last.fm is 0-100, MusicBrainz is votes.
for _, row := range []struct {
artist int64
@ -128,12 +140,12 @@ func TestTasteArtistTags(t *testing.T) {
if err != nil {
t.Fatal(err)
}
want := map[string]float64{"rock": 1, "art rock": 0.5, "alternative": 1, "electronic": 0.5}
want := map[string]float64{"rock": 1, "art rock": 2.0 / 3, "alternative": 1, "electronic": 0.5}
if len(got[radiohead]) != len(want) {
t.Errorf("Radiohead tags = %v, want %v", got[radiohead], want)
}
for tag, w := range want {
if got[radiohead][tag] != w {
if math.Abs(got[radiohead][tag]-w) > 1e-9 {
t.Errorf("Radiohead %q = %v, want %v", tag, got[radiohead][tag], w)
}
}

View file

@ -16,6 +16,9 @@ const (
DefaultLimit = 20
MaxLimit = 50
// MaxTags is how many tags the profile keeps, the best first.
MaxTags = 50
// MaxProfileArtists is how many of the liked artists, the most liked first,
// shape the tag profile. The rest weigh too little to change it and cost a read each.
MaxProfileArtists = 200
@ -23,6 +26,8 @@ const (
// lookbackHalfLives is how far back the history is read: an event ten
// half-lives old weighs less than a thousandth of a new one.
lookbackHalfLives = 10
defaultHalfLife = 90 * 24 * time.Hour
)
// Store reads what the model needs. It never writes.
@ -62,11 +67,16 @@ type Service struct {
// NewService builds a Service.
func NewService(store Store, o Options) *Service {
if o.HalfLife <= 0 {
o.HalfLife = defaultHalfLife // a zero would make every weight NaN
}
return &Service{store: store, opts: o, now: time.Now}
}
// Taste returns the listener's profile: up to limit liked artists (zero means
// DefaultLimit) and the tag profile built from the liked artists.
// DefaultLimit) and the tag profile built from the liked artists (the best
// MaxTags). Artists are ordered by how much the listener played them, recent
// plays counting more; ties go to the lower id.
func (s *Service) Taste(ctx context.Context, userID int64, limit int) (Profile, error) {
if limit < 0 {
return Profile{}, fmt.Errorf("%w: limit cannot be negative", ErrInvalidQuery)
@ -85,17 +95,15 @@ func (s *Service) Taste(ctx context.Context, userID int64, limit int) (Profile,
}
var liked []ArtistPref
for _, p := range Preferences(events, now, s.opts) {
if p.Preference > 0 {
if p.Score > 0 {
liked = append(liked, p)
}
}
sort.Slice(liked, func(i, j int) bool {
a, b := liked[i], liked[j]
switch {
case a.Preference != b.Preference:
return a.Preference > b.Preference
case !a.LastListened.Equal(b.LastListened):
return a.LastListened.After(b.LastListened)
case a.Score != b.Score:
return a.Score > b.Score
default:
return a.ArtistID < b.ArtistID
}
@ -120,6 +128,9 @@ func (s *Service) Taste(ctx context.Context, userID int64, limit int) (Profile,
return Profile{}, fmt.Errorf("read tags: %w", err)
}
profile.Tags = TagProfile(prefs, tags)
if len(profile.Tags) > MaxTags {
profile.Tags = profile.Tags[:MaxTags]
}
shown := liked
if len(shown) > limit {

View file

@ -3,6 +3,7 @@ package taste
import (
"context"
"errors"
"math"
"testing"
"time"
@ -129,8 +130,8 @@ func TestTasteLimit(t *testing.T) {
}
func TestTasteTiesAreStable(t *testing.T) {
// Same preference, so the one heard more recently goes first, then the lower id.
f := &fakeStore{
// Same listens at the same time: the lower id goes first.
same := &fakeStore{
events: []Event{
{ArtistID: 5, Type: listening.Complete, At: now.Add(-time.Hour)},
{ArtistID: 4, Type: listening.Complete, At: now.Add(-time.Hour)},
@ -138,9 +139,44 @@ func TestTasteTiesAreStable(t *testing.T) {
},
names: map[int64]string{},
}
p, _ := newService(f).Taste(context.Background(), 1, 0)
p, _ := newService(same).Taste(context.Background(), 1, 0)
if len(p.Artists) != 3 || p.Artists[0].ID != 4 || p.Artists[1].ID != 5 || p.Artists[2].ID != 6 {
t.Errorf("order = %+v", p.Artists)
t.Errorf("order by id = %+v", p.Artists)
}
}
func TestTasteOrdersByTheScoreNotByTheRoundedPreference(t *testing.T) {
var events []Event
for i := 0; i < 500; i++ {
events = append(events, Event{ArtistID: 1, Type: listening.Complete, At: now.Add(-time.Hour)})
}
for i := 0; i < 60; i++ {
events = append(events, Event{ArtistID: 2, Type: listening.Complete, At: now})
}
p, _ := newService(&fakeStore{events: events, names: map[int64]string{}}).Taste(context.Background(), 1, 0)
if len(p.Artists) != 2 || p.Artists[0].ID != 1 {
t.Errorf("order = %+v, want the artist heard 500 times first", p.Artists)
}
}
func TestTasteKeepsTheTagListShort(t *testing.T) {
tags := map[string]float64{}
for i := 0; i < MaxTags+30; i++ {
tags[string(rune('a'+i%26))+string(rune('a'+i/26))] = 1 - float64(i)/1000
}
f := &fakeStore{events: complete(1, 1), tags: map[int64]map[string]float64{1: tags}, names: map[int64]string{1: "One"}}
p, _ := newService(f).Taste(context.Background(), 1, 0)
if len(p.Tags) != MaxTags || p.Tags[0].Weight != 1 {
t.Errorf("%d tags, first %+v; want %d, best first", len(p.Tags), p.Tags[0], MaxTags)
}
}
func TestAZeroHalfLifeDoesNotBreakTheModel(t *testing.T) {
s := NewService(&fakeStore{events: complete(1, 3), names: map[int64]string{1: "One"}}, Options{MinListen: 20 * time.Second})
s.now = func() time.Time { return now }
p, err := s.Taste(context.Background(), 1, 0)
if err != nil || len(p.Artists) != 1 || math.IsNaN(p.Artists[0].Preference) || p.Artists[0].Preference <= 0 {
t.Errorf("profile = %+v err=%v, want a normal preference", p, err)
}
}
@ -154,7 +190,7 @@ func TestTasteBoundsTheArtistsBehindTheTagProfile(t *testing.T) {
t.Fatal(err)
}
if len(f.tagCalls) != 1 || len(f.tagCalls[0]) != MaxProfileArtists {
t.Errorf("tags were read for %d artists in %d calls, want %d in one", len(f.tagCalls[len(f.tagCalls)-1]), len(f.tagCalls), MaxProfileArtists)
t.Errorf("tags were read in %d calls (%v), want one call for %d artists", len(f.tagCalls), f.tagCalls, MaxProfileArtists)
}
}

View file

@ -8,7 +8,9 @@
// Each listening event has a value (see EventValue). Older events weigh less
// (the weight halves every Options.HalfLife). The values of one artist add up
// and are squeezed into (-1, 1) by tanh, so a few listens already say something
// but no amount of them reaches 1, and one skip does not sink an artist.
// but they only approach 1, and one skip does not sink an artist. tanh is for
// showing: it rounds to exactly 1 for a very large sum, so artists are ordered by
// the sum itself (Score).
package taste
import (
@ -41,6 +43,7 @@ type Event struct {
type ArtistPref struct {
ArtistID int64
Preference float64 // from -1 (avoided) to 1 (loved); 0 is neutral
Score float64 // the weighted sum behind Preference; it keeps order where tanh rounds to 1
Listens int // events that count as a listen, with no decay
LastListened time.Time // the latest counted event; zero when there is none
}
@ -76,10 +79,10 @@ func Preferences(events []Event, now time.Time, o Options) map[int64]ArtistPref
sums := map[int64]float64{}
out := map[int64]ArtistPref{}
for _, e := range events {
age := now.Sub(e.At)
if age < 0 {
age = 0 // a fast clock does not make an event more than new
if e.At.After(now) {
e.At = now // a wrong clock does not make an event newer than now
}
age := now.Sub(e.At)
weight := math.Exp2(-float64(age) / float64(o.HalfLife))
sums[e.ArtistID] += weight * EventValue(e.Type, e.PlayedMs, o.MinListen)
@ -94,6 +97,7 @@ func Preferences(events []Event, now time.Time, o Options) map[int64]ArtistPref
out[e.ArtistID] = p
}
for id, p := range out {
p.Score = sums[id]
p.Preference = math.Tanh(sums[id] / sumScale)
out[id] = p
}

View file

@ -58,6 +58,31 @@ func TestPreferenceGrowsWithListensAndStaysBelowOne(t *testing.T) {
}
}
func TestHeavilyPlayedArtistsKeepTheirOrder(t *testing.T) {
var events []Event
for i := 0; i < 500; i++ {
events = append(events, Event{ArtistID: 1, Type: listening.Complete, At: now.Add(-time.Hour)})
}
for i := 0; i < 60; i++ {
events = append(events, Event{ArtistID: 2, Type: listening.Complete, At: now})
}
got := Preferences(events, now, options)
if got[1].Score <= got[2].Score {
t.Errorf("500 listens score %v, 60 listens score %v: the one heard more must score more", got[1].Score, got[2].Score)
}
if got[1].Preference > 1 || got[1].Preference < 0.999 {
t.Errorf("preference = %v, want at most 1 and close to it", got[1].Preference)
}
}
func TestAnEventFromTheFutureIsNewAndNothingMore(t *testing.T) {
future := now.Add(365 * 24 * time.Hour)
got := Preferences([]Event{{ArtistID: 1, Type: listening.Complete, At: future}}, now, options)[1]
if !got.LastListened.Equal(now) {
t.Errorf("last listened = %v, want now (%v): a clock that is wrong cannot make a listen later than now", got.LastListened, now)
}
}
func TestOneSkipDoesNotSinkAnArtist(t *testing.T) {
got := Preferences([]Event{{ArtistID: 1, Type: listening.Skip, PlayedMs: 2_000, At: now}}, now, options)[1]
if want := math.Tanh(-1.0 / 3); !near(got.Preference, want) {