fix(images): limit parallel builds, no stale saves, safer files

- At most two covers are built at the same time: a page of albums asked for at once
  could decode dozens of large pictures together.
- A change to an album's files marks its record stale and moves a generation instead
  of deleting the row. A build that started before the change is not saved over it
  and starts again, so a "no cover" found too early cannot win for a day.
- A cover that was found is checked again after a week, so a picture replaced in a
  folder is noticed.
- For CD1/Disc 2 folders the picture in the folder above is used. Folder pictures
  are read one at a time, and the pixel count is computed in 64 bits.
- Stored files are synced before they get their name, an empty one counts as
  missing, and old temporary files from a crash are removed.
- Files without track or disc numbers sort the same on both databases.
- storage.data_dir defaults next to the SQLite database, so a service started from
  another folder does not need a writable working directory.
- Tests: a listener who leaves, a change during a build, the build limit, a named
  pipe, an embedded picture of a file outside the library that exists, builds that
  really overlap.
This commit is contained in:
tgiachi 2026-11-11 21:03:26 +01:00
commit 7bdbc9e160
14 changed files with 546 additions and 90 deletions

View file

@ -17,7 +17,7 @@ It is a single executable, one database (SQLite by default, PostgreSQL if you pr
- **Listening history.** Clients report explicit `play`, `complete` and `skip` events. An event counts as a listen when it is `complete`, or when it was played for at least 20 seconds (`listening.min_listen_seconds`); the rule is applied when the history is read, so changing it re-reads the past. You can read your own history, with a `counted` flag on every event. It stays in your database and is never sent anywhere.
- **Your taste.** `GET /me/taste` shows what Sonda has learned from your own listening: the artists you like and the kinds of music (tags) they have in common. A complete listen counts for more than a short one, a skip counts against the artist, and older listens weigh less (they halve every 90 days, `taste.half_life_days`; only the last ten half-lives are read). It is worked out on every request from data already in your database; nothing is sent to any provider.
- **Discover.** `GET /discover` suggests artists you may like, in your library or not, each with a score from 0 to 1 and the parts it is made of: how close it is to the artists you like (affinity), how close its tags are to yours (genre), what your listening says about it (preference) and a bonus for what you have not played (discovery). The weights are in `[scoring]` in the config. A part with no data is left out and the rest rescaled, and every answer says which seeds led to each suggestion. It starts from the artists you name with `?seed=`, or, only if you turned on `discover_from_listening`, from the artists you play most. It needs Last.fm to be switched on to ask for anything new: when Last.fm is off and a seed's similar artists are not stored yet, the answer says `disabled` (what was stored earlier is still used). See [External providers](#external-providers).
- **Album covers.** `GET /albums/{id}/cover` serves the cover of an album from your own files: the picture embedded in them, or else a `cover`, `folder`, `front`, `album` or `albumart` image in the album's folder. It is built the first time it is asked for, scaled (`?size=small` is 300 pixels, `large` 1000) and kept in the data folder (`storage.data_dir`, you can delete it: covers are built again). Nothing is sent anywhere. It also takes `?access_token=`, so an `<img>` element can use it.
- **Album covers.** `GET /albums/{id}/cover` serves the cover of an album from your own files: the picture embedded in them, or else a `cover`, `folder`, `front`, `album` or `albumart` image in the album's folder. It is built the first time it is asked for (a few at a time), scaled (`?size=small` is 300 pixels, `large` 1000) and kept in the data folder (`storage.data_dir`, you can delete it: covers are built again). A picture you replace in a folder is noticed within a week, or at once after a full rescan. For an album split in `CD1`/`Disc 2` folders, the picture in the folder above is used. Nothing is sent anywhere. It also takes `?access_token=`, so an `<img>` element can use it.
- **Wishlist and "not interested".** `PUT /wishlist/{artist}` puts an artist on your wishlist (`GET /wishlist` lists it, `DELETE` takes it off), and `POST /discover/{artist}/dismiss` hides an artist from your Discover for good (`DELETE` brings it back, `GET /discover/dismissed` lists them). An artist is wished or dismissed, never both, and both lists are yours alone. Discover marks the wished artists it shows (`wishlisted`) and never suggests a dismissed one. Nothing is downloaded or sent anywhere: the wishlist is a list for you, and for a download client you may set up later. An artist that only Last.fm named is kept while you have it marked (your lists still show it, but other users cannot see that it exists), and goes with the rest of Last.fm's data if an admin removes Last.fm's data. Removing MusicBrainz's data does not touch it.
- **Your own settings.** `GET` and `PATCH /me/settings` hold choices that belong to you, not to the server. Everyone starts private: `discover_from_listening` is off, so this setting never sends your listening to a provider. When you turn it on, Discover may start from the artists you listen to most, and their names are sent to Last.fm (only if an admin switched it on). What Last.fm answers is kept in the shared database like any other lookup, so another local user can see, from `retrieved_at` of `GET /artists/{id}/similar`, that someone looked up an artist, but not who.
- **Tags.** `GET /artists/{id}/tags` shows what Last.fm and MusicBrainz say kind of music an artist plays, each with its own weights, and the genres of the artist's own files, read offline. See [External providers](#external-providers).
@ -66,7 +66,7 @@ Precedence is: built-in defaults, then the file, then environment variables. An
| `auth.refresh_ttl_hours` | | `720` | Lifetime of a refresh token (30 days). |
| `listening.min_listen_seconds` | | `20` | A play counts as a listen after this long (1 to 3600), or when it is `complete`. |
| `taste.half_life_days` | | `90` | After this many days a listen weighs half as much when learning your taste (1 to 3650). |
| `storage.data_dir` | `SONDA_DATA_DIR` | `data` | Where Sonda keeps the files it makes (covers). They are built from your music, so they need no backup. |
| `storage.data_dir` | `SONDA_DATA_DIR` | `data` next to the SQLite database (`data` in the working directory with PostgreSQL) | Where Sonda keeps the files it makes (covers). They are built from your music, so they need no backup. |
| `scoring.weight_affinity`, `weight_genre`, `weight_preference`, `weight_discovery` | | `0.40`, `0.25`, `0.25`, `0.10` | The weights of the parts of a Discover score (0 to 10 each, not all zero). Only their proportions matter. |
Libraries and the external providers are not in the config: you set them up through the API (see the demo and [External providers](#external-providers)).

View file

@ -554,6 +554,9 @@ func TestResponsesMatchTheSpec(t *testing.T) {
if rec := c.call("GET", cover, token, nil, "If-None-Match", big.Header().Get("ETag")); rec.Code != 304 {
t.Fatalf("cover If-None-Match: %d", rec.Code)
}
if rec := c.call("GET", cover, token, nil, "Range", "bytes=0-9"); rec.Code != 206 || rec.Body.Len() != 10 {
t.Fatalf("cover Range: %d, %d bytes", rec.Code, rec.Body.Len())
}
if rec := c.call("HEAD", cover, token, nil); rec.Code != 200 {
t.Fatalf("cover HEAD: %d", rec.Code)
}

View file

@ -990,8 +990,14 @@ paths:
contacts anybody. Pictures that are too big (over 20 MB or 16 million pixels) or are not
pictures are ignored, and only files inside the library are read.
For an album split in folders named like `CD1` or `Disc 2`, the picture in the folder
above them is used too. At most a couple of covers are built at the same time, so a
client that asks for many at once may wait.
An album with no picture answers `cover_not_found`, and the server does not look again
for a day; a new or changed file of the album, or a full rescan, makes it look again.
A cover that was found is checked again after a week, so a picture replaced in the
folder is noticed.
The picture is `private` for an hour in caches and has an `ETag`: send it back as
`If-None-Match`.

View file

@ -9,6 +9,7 @@ import (
"fmt"
"math"
"os"
"path/filepath"
"time"
"github.com/pelletier/go-toml/v2"
@ -104,10 +105,21 @@ type Scoring struct {
WeightDiscovery float64 `toml:"weight_discovery"` // bonus for what I have not played
}
// defaultDataDir puts the data folder next to a SQLite database, so that a service
// started from another working directory finds the same place; with PostgreSQL it is
// "data" in the working directory.
func defaultDataDir(db Database) string {
if db.Driver == "sqlite" && db.Path != "" {
return filepath.Join(filepath.Dir(db.Path), "data")
}
return "data"
}
// Storage says where Sonda keeps the files it makes (covers and, later, other
// pictures). They can be built again from the music, so this folder needs no backup,
// but losing it makes every cover be built again.
type Storage struct {
// DataDir is the folder; empty means "data" next to the SQLite database.
DataDir string `toml:"data_dir"`
}
@ -121,7 +133,6 @@ func Default() Config {
Auth: Auth{AccessTTLMinutes: 15, RefreshTTLHours: 720}, // 15 minutes, 30 days
Listening: Listening{MinListenSeconds: 20},
Taste: Taste{HalfLifeDays: 90},
Storage: Storage{DataDir: "data"},
Scoring: Scoring{WeightAffinity: 0.40, WeightGenre: 0.25, WeightPreference: 0.25, WeightDiscovery: 0.10},
}
}
@ -150,6 +161,9 @@ func Load(path string) (Config, error) {
}
applyEnv(&cfg)
if cfg.Storage.DataDir == "" {
cfg.Storage.DataDir = defaultDataDir(cfg.Database)
}
return cfg, nil
}

View file

@ -221,10 +221,22 @@ func TestScoringDefaultsAndFile(t *testing.T) {
}
}
func TestStorageDataDir(t *testing.T) {
if got := Default().Storage.DataDir; got != "data" {
t.Errorf("default data dir = %q, want data", got)
func TestStorageDataDirDefaultsNextToTheDatabase(t *testing.T) {
for name, tt := range map[string]struct{ toml, want string }{
"default database": {"", "data"},
"database elsewhere": {"[database]\npath = \"/var/lib/sonda/sonda.db\"\n", "/var/lib/sonda/data"},
"relative database": {"[database]\npath = \"state/sonda.db\"\n", "state/data"},
"set explicitly": {"[database]\npath = \"/var/lib/sonda/sonda.db\"\n[storage]\ndata_dir = \"/srv/x\"\n", "/srv/x"},
"postgres": {"[database]\ndriver = \"postgres\"\ndsn = \"postgres://u:p@h/d\"\npath = \"\"\n", "data"},
} {
cfg, err := Load(writeFile(t, t.TempDir(), tt.toml))
if err != nil || cfg.Storage.DataDir != tt.want {
t.Errorf("%s: data dir = %q err=%v, want %q", name, cfg.Storage.DataDir, err, tt.want)
}
}
}
func TestStorageDataDir(t *testing.T) {
cfg, err := Load(writeFile(t, t.TempDir(), "[storage]\ndata_dir = \"/srv/sonda\"\n"))
if err != nil || cfg.Storage.DataDir != "/srv/sonda" {
t.Errorf("data dir = %q err=%v", cfg.Storage.DataDir, err)

View file

@ -0,0 +1,33 @@
//go:build unix
package images
import (
"context"
"errors"
"path/filepath"
"syscall"
"testing"
"time"
)
// Opening a named pipe waits for a writer: a pipe called cover.jpg must not hang the build.
func TestAPipeCalledCoverDoesNotHangTheBuild(t *testing.T) {
r := newRig(t)
if err := syscall.Mkfifo(filepath.Join(r.root, "Album One", "cover.jpg"), 0o644); err != nil {
t.Skip("cannot create a FIFO:", err)
}
done := make(chan error, 1)
go func() {
_, err := r.svc.AlbumCover(context.Background(), 1, Large)
done <- err
}()
select {
case err := <-done:
if !errors.Is(err, ErrNoCover) {
t.Errorf("err = %v, want no cover", err)
}
case <-time.After(5 * time.Second):
t.Fatal("the build hung on a named pipe")
}
}

View file

@ -53,7 +53,7 @@ func Process(data []byte) (Processed, error) {
if err != nil {
return Processed{}, fmt.Errorf("%w: %v", ErrUnusable, err)
}
if cfg.Width <= 0 || cfg.Height <= 0 || cfg.Width*cfg.Height > MaxPixels {
if cfg.Width <= 0 || cfg.Height <= 0 || int64(cfg.Width)*int64(cfg.Height) > MaxPixels { // int64: a 32-bit int would overflow
return Processed{}, fmt.Errorf("%w: %dx%d pixels", ErrUnusable, cfg.Width, cfg.Height)
}
src, _, err := image.Decode(bytes.NewReader(data))

View file

@ -11,9 +11,11 @@ import (
"os"
"path"
"path/filepath"
"regexp"
"sort"
"strconv"
"strings"
"sync"
"time"
"go.senan.xyz/taglib"
@ -25,8 +27,9 @@ const kindAlbum = "album"
// What is stored about a picture, and where it came from.
const (
StatusOK = "ok" // a cover was found and is stored
StatusNone = "none" // the files were searched and have none
StatusOK = "ok" // a cover was found and is stored
StatusNone = "none" // the files were searched and have none
StatusStale = "stale" // a file of the album changed: look again
SourceEmbedded = "embedded" // the picture inside an audio file
SourceFolder = "folder" // a cover.jpg, folder.png ... next to the audio
@ -37,6 +40,15 @@ const (
// MaxFilesTried is how many files of an album are asked for an embedded picture:
// they all carry the same one, and reading a file costs.
MaxFilesTried = 3
// MaxParallelBuilds is how many covers are built at the same time. Building one
// decodes a picture that can take 200 MB, and a client showing a page of albums
// asks for dozens at once.
MaxParallelBuilds = 2
// CoverTTL is how long a cover that was found is believed before the files are
// looked at again: a picture replaced in a folder changes no audio file.
CoverTTL = 7 * 24 * time.Hour
// tempMaxAge is how old a leftover temporary file must be to be removed.
tempMaxAge = time.Hour
// NoCoverTTL is how long "no cover" is believed. Adding a picture to a folder
// changes no audio file, so the answer has to be checked again now and then.
NoCoverTTL = 24 * time.Hour
@ -75,10 +87,14 @@ type Source interface {
// Record is what is known about the picture of one owner.
type Record struct {
Status string // StatusOK or StatusNone
Source string // SourceEmbedded or SourceFolder; empty for StatusNone
ETag string // identifies the stored picture
Width int // of the original
Status string // StatusOK or StatusNone
Source string // SourceEmbedded or SourceFolder; empty for StatusNone
ETag string // identifies the stored picture
// Gen counts the changes of the album's files. A build saves its result only if
// the count is still the one it started from, so that it cannot bring back an
// answer that a change made out of date.
Gen int64
Width int // of the original
Height int
CheckedAt time.Time
}
@ -86,7 +102,9 @@ type Record struct {
// Store keeps the records. The files themselves are in the data folder.
type Store interface {
Get(ctx context.Context, kind string, ownerID int64) (Record, bool, error)
Save(ctx context.Context, kind string, ownerID int64, r Record) error
// Save keeps the record unless the owner's Gen is no longer r.Gen, and says
// whether it was kept.
Save(ctx context.Context, kind string, ownerID int64, r Record) (bool, error)
}
// Cover is a stored picture ready to be served. The caller must close File.
@ -105,11 +123,14 @@ type Service struct {
embedded func(path string) ([]byte, error)
now func() time.Time
flight singleflight.Group
slots chan struct{} // one per build allowed at the same time
tidy sync.Once
}
// NewService builds a Service that keeps its files under dir.
func NewService(dir string, store Store, src Source) *Service {
return &Service{dir: dir, store: store, src: src, embedded: taglib.ReadImage, now: time.Now}
return &Service{dir: dir, store: store, src: src, embedded: taglib.ReadImage, now: time.Now,
slots: make(chan struct{}, MaxParallelBuilds)}
}
// AlbumCover returns the cover of an album in the given size, building it first when
@ -132,7 +153,7 @@ func (s *Service) AlbumCover(ctx context.Context, albumID int64, size Size) (*Co
if ok && rec.Status == StatusNone && s.now().Sub(rec.CheckedAt) < NoCoverTTL {
return nil, ErrNoCover
}
if ok && rec.Status == StatusOK {
if ok && rec.Status == StatusOK && s.now().Sub(rec.CheckedAt) < CoverTTL {
if c, err := s.open(albumID, size, rec); err == nil {
return c, nil
} // the stored file is gone: build it again
@ -174,9 +195,9 @@ func (s *Service) open(albumID int64, size Size, rec Record) (*Cover, error) {
return nil, err
}
info, err := f.Stat()
if err != nil {
if err != nil || info.Size() == 0 { // empty: a write that did not finish before a crash
_ = f.Close()
return nil, err
return nil, errors.Join(err, os.ErrNotExist)
}
etag := rec.ETag
if size == Small {
@ -185,13 +206,44 @@ func (s *Service) open(albumID int64, size Size, rec Record) (*Cover, error) {
return &Cover{File: f, ModTime: info.ModTime(), ETag: etag}, nil
}
// build looks for a picture, stores it and records the outcome.
// build looks for a picture, stores it and records the outcome. If a file of the
// album changed while it was working, what it found may be out of date: it starts again.
func (s *Service) build(albumID int64) (Record, error) {
ctx, cancel := context.WithTimeout(context.Background(), buildTimeout)
defer cancel()
files, err := s.src.AlbumFiles(ctx, albumID)
if err != nil {
return Record{}, err
var rec Record
for attempt := 0; attempt < 3; attempt++ {
cur, _, err := s.store.Get(ctx, kindAlbum, albumID)
if err != nil {
return Record{}, fmt.Errorf("read picture record: %w", err)
}
files, err := s.src.AlbumFiles(ctx, albumID)
if err != nil {
return Record{}, err
}
rec, err = s.search(ctx, albumID, files)
if err != nil {
return Record{}, err
}
rec.Gen = cur.Gen
saved, err := s.store.Save(ctx, kindAlbum, albumID, rec)
if err != nil {
return Record{}, fmt.Errorf("save picture record: %w", err)
}
if saved {
return rec, nil
}
}
return rec, nil // it kept changing; this one is served and the record stays stale
}
// search finds a picture and stores it, at most MaxParallelBuilds at a time.
func (s *Service) search(ctx context.Context, albumID int64, files []File) (Record, error) {
select {
case s.slots <- struct{}{}:
defer func() { <-s.slots }()
case <-ctx.Done():
return Record{}, ctx.Err()
}
rec := Record{Status: StatusNone, CheckedAt: s.now()}
if p, source, ok := s.find(files); ok {
@ -201,9 +253,6 @@ func (s *Service) build(albumID int64) (Record, error) {
sum := sha256.Sum256(p.Large)
rec = Record{Status: StatusOK, Source: source, ETag: hex.EncodeToString(sum[:8]), Width: p.Width, Height: p.Height, CheckedAt: rec.CheckedAt}
}
if err := s.store.Save(ctx, kindAlbum, albumID, rec); err != nil {
return Record{}, fmt.Errorf("save picture record: %w", err)
}
return rec, nil
}
@ -221,13 +270,26 @@ func (s *Service) find(files []File) (Processed, string, bool) {
}
}
seen := map[string]bool{}
var dirs []File // each folder once, then the folders above disc folders
for _, f := range files {
dir := path.Dir(f.RelPath)
if seen[f.Root+"\x00"+dir] {
continue
if !seen[f.Root+"\x00"+dir] {
seen[f.Root+"\x00"+dir] = true
dirs = append(dirs, File{Root: f.Root, RelPath: dir})
}
seen[f.Root+"\x00"+dir] = true
for _, data := range folderPictures(f.Root, dir) {
}
for _, d := range append([]File(nil), dirs...) {
if parent := path.Dir(d.RelPath); discFolder.MatchString(path.Base(d.RelPath)) && parent != "." && !seen[d.Root+"\x00"+parent] {
seen[d.Root+"\x00"+parent] = true
dirs = append(dirs, File{Root: d.Root, RelPath: parent})
}
}
for _, d := range dirs {
for _, name := range folderPictureNames(d.Root, d.RelPath) {
data, ok := readFolderPicture(d.Root, d.RelPath, name)
if !ok {
continue
}
if p, err := Process(data); err == nil {
return p, SourceFolder, true
}
@ -236,6 +298,10 @@ func (s *Service) find(files []File) (Processed, string, bool) {
return Processed{}, "", false
}
// discFolder matches the name of a folder that holds one disc of an album: "CD1",
// "Disc 2", "disk 03". The picture of such an album is usually in the folder above.
var discFolder = regexp.MustCompile(`(?i)^(cd|dis[ck])[ _.-]*\d+$`)
// embeddedPicture reads the picture inside an audio file, after checking through
// os.Root that the file is a regular file inside its library.
func (s *Service) embeddedPicture(f File) ([]byte, bool) {
@ -258,10 +324,10 @@ func (s *Service) embeddedPicture(f File) ([]byte, bool) {
// folderNames are the names of picture files that mean "the cover", best first.
var folderNames = []string{"cover", "folder", "front", "album", "albumart"}
// folderPictures returns the contents of the cover-like picture files of a folder of
// the library, best name first. Only regular files inside the library are read (a link
// is not followed), and none above MaxSourceBytes.
func folderPictures(rootDir, dir string) [][]byte {
// folderPictureNames returns the names of the cover-like picture files of a folder of
// the library, best name first. Only regular files inside the library are listed (a
// link is not followed), and none above MaxSourceBytes.
func folderPictureNames(rootDir, dir string) []string {
root, err := os.OpenRoot(filepath.Clean(rootDir))
if err != nil {
return nil
@ -300,21 +366,36 @@ func folderPictures(rootDir, dir string) [][]byte {
}
return picks[i].name < picks[j].name
})
var out [][]byte
for _, p := range picks {
f, err := root.OpenFile(path.Join(dir, p.name), os.O_RDONLY|nonBlock, 0)
if err != nil {
continue
}
data, err := io.ReadAll(io.LimitReader(f, MaxSourceBytes+1))
_ = f.Close()
if err == nil && len(data) <= MaxSourceBytes {
out = append(out, data)
}
out := make([]string, len(picks))
for i, p := range picks {
out[i] = p.name
}
return out
}
// readFolderPicture reads one of the files folderPictureNames returned, through
// os.Root, without waiting on a pipe, and not more than MaxSourceBytes.
func readFolderPicture(rootDir, dir, name string) ([]byte, bool) {
root, err := os.OpenRoot(filepath.Clean(rootDir))
if err != nil {
return nil, false
}
defer root.Close()
f, err := root.OpenFile(path.Join(dir, name), os.O_RDONLY|nonBlock, 0)
if err != nil {
return nil, false
}
defer f.Close()
if info, err := f.Stat(); err != nil || !info.Mode().IsRegular() {
return nil, false // what was opened is not what the listing showed
}
data, err := io.ReadAll(io.LimitReader(f, MaxSourceBytes+1))
if err != nil || len(data) > MaxSourceBytes {
return nil, false
}
return data, true
}
// write stores both sizes. Each goes to a temporary file first and is renamed, so a
// reader never sees half a picture.
func (s *Service) write(albumID int64, p Processed) error {
@ -322,16 +403,18 @@ func (s *Service) write(albumID int64, p Processed) error {
if err := os.MkdirAll(dir, 0o700); err != nil {
return fmt.Errorf("create the data folder: %w", err)
}
s.tidy.Do(func() { removeOldTemps(dir, time.Now()) }) // file times are the disk's, not the clock of the service
for size, data := range map[Size][]byte{Large: p.Large, Small: p.Small} {
tmp, err := os.CreateTemp(dir, "tmp-*")
if err != nil {
return fmt.Errorf("create a picture file: %w", err)
}
_, werr := tmp.Write(data)
serr := tmp.Sync() // on disk before it gets its name: a crash must not leave a short file under it
cerr := tmp.Close()
if werr != nil || cerr != nil {
if werr != nil || serr != nil || cerr != nil {
_ = os.Remove(tmp.Name())
return fmt.Errorf("write a picture file: %w", errors.Join(werr, cerr))
return fmt.Errorf("write a picture file: %w", errors.Join(werr, serr, cerr))
}
if err := os.Rename(tmp.Name(), s.filePath(albumID, size)); err != nil {
_ = os.Remove(tmp.Name())
@ -340,3 +423,20 @@ func (s *Service) write(albumID int64, p Processed) error {
}
return nil
}
// removeOldTemps removes the temporary files that a crash left behind. A recent one may
// belong to a build that is running, so it stays.
func removeOldTemps(dir string, now time.Time) {
entries, err := os.ReadDir(dir)
if err != nil {
return
}
for _, e := range entries {
if !strings.HasPrefix(e.Name(), "tmp-") {
continue
}
if info, err := e.Info(); err == nil && now.Sub(info.ModTime()) > tempMaxAge {
_ = os.Remove(filepath.Join(dir, e.Name()))
}
}
}

View file

@ -10,6 +10,7 @@ import (
"io"
"os"
"path/filepath"
"strconv"
"sync"
"sync/atomic"
"testing"
@ -23,7 +24,7 @@ type fakeStore struct {
err error
}
func key(kind string, id int64) string { return kind + "/" + string(rune('0'+id)) }
func key(kind string, id int64) string { return kind + "/" + strconv.FormatInt(id, 10) }
func (f *fakeStore) Get(_ context.Context, kind string, id int64) (Record, bool, error) {
f.mu.Lock()
@ -33,17 +34,27 @@ func (f *fakeStore) Get(_ context.Context, kind string, id int64) (Record, bool,
return r, ok, f.err
}
func (f *fakeStore) Save(_ context.Context, kind string, id int64, r Record) error {
// Save keeps the record only when nothing was cleared since the Gen it carries was
// read, like the real store.
func (f *fakeStore) Save(_ context.Context, kind string, id int64, r Record) (bool, error) {
f.mu.Lock()
defer f.mu.Unlock()
if f.err != nil {
return false, f.err
}
if cur, ok := f.recs[key(kind, id)]; ok && cur.Gen != r.Gen {
return false, nil
}
f.recs[key(kind, id)] = r
return f.err
return true, nil
}
func (f *fakeStore) drop(kind string, id int64) {
// clear is what a changed file does: the record is marked stale and its generation moves.
func (f *fakeStore) clear(kind string, id int64) {
f.mu.Lock()
defer f.mu.Unlock()
delete(f.recs, key(kind, id))
cur := f.recs[key(kind, id)]
f.recs[key(kind, id)] = Record{Status: StatusStale, Gen: cur.Gen + 1}
}
type fakeSource struct {
@ -61,6 +72,7 @@ func (f *fakeSource) AlbumFiles(_ context.Context, id int64) ([]File, error) {
}
type rig struct {
gate chan struct{} // when set, the embedded stub waits for it to be closed
svc *Service
store *fakeStore
src *fakeSource
@ -103,6 +115,9 @@ func newRig(t *testing.T) *rig {
r.svc = NewService(r.data, r.store, r.src)
r.svc.embedded = func(path string) ([]byte, error) {
atomic.AddInt32(&r.reads, 1)
if r.gate != nil {
<-r.gate
}
if b, ok := r.embedded[path]; ok {
return b, nil
}
@ -172,7 +187,7 @@ func TestWithoutAnEmbeddedPictureTheFolderImageIsUsedInPriorityOrder(t *testing.
t.Errorf("source = %q", got)
}
// "cover" outranks "folder" and "front".
r.store.drop("album", 1)
r.store.clear("album", 1)
r.write(t, "Album One/cover.jpeg", jpegBytes(t, 300, 300, color.RGBA{0, 200, 0, 255}))
img, _ = jpeg.Decode(bytes.NewReader(r.read(t, 1, Large)))
_, green, _, _ := img.At(100, 100).RGBA()
@ -244,7 +259,7 @@ func TestAnAlbumWhoseRecordWasClearedIsBuiltAgain(t *testing.T) {
if second := r.read(t, 1, Large); !bytes.Equal(first, second) {
t.Error("a cover that is stored was built again for nothing")
}
r.store.drop("album", 1) // what a changed file does (see the storage)
r.store.clear("album", 1) // what a changed file does (see the storage)
if third := r.read(t, 1, Large); bytes.Equal(first, third) {
t.Error("after the record was cleared the old picture was served")
}
@ -317,9 +332,13 @@ func TestPicturesOutsideTheLibraryAreNeverRead(t *testing.T) {
t.Errorf("album %d: err = %v, want no cover: the picture is outside the library", id, err)
}
}
// An embedded picture is not read from a file that leaves the library either.
// An embedded picture is not read from a file that leaves the library either: the
// file exists (outside), so only the check can be what refuses it.
if err := os.WriteFile(filepath.Join(outside, "x.mp3"), []byte("audio"), 0o644); err != nil {
t.Fatal(err)
}
r.embedded[filepath.Join(r.root, "linked/x.mp3")] = jpegBytes(t, 300, 300, color.White)
r.store.drop("album", 2)
r.store.clear("album", 2)
if _, err := r.svc.AlbumCover(context.Background(), 2, Large); !errors.Is(err, ErrNoCover) {
t.Errorf("embedded picture of a file outside the library: err = %v", err)
}
@ -341,6 +360,7 @@ func TestOnlyRegularPictureFilesAreRead(t *testing.T) {
func TestManyAsksAtOnceBuildItOnce(t *testing.T) {
r := newRig(t)
r.embedded[filepath.Join(r.root, "Album One/01.mp3")] = jpegBytes(t, 1200, 1200, color.White)
r.gate = make(chan struct{}) // the build waits here, so that every ask arrives while it runs
var wg sync.WaitGroup
errs := make(chan error, 20)
for i := 0; i < 20; i++ {
@ -354,6 +374,8 @@ func TestManyAsksAtOnceBuildItOnce(t *testing.T) {
errs <- err
}()
}
time.Sleep(100 * time.Millisecond)
close(r.gate)
wg.Wait()
close(errs)
for err := range errs {
@ -368,11 +390,179 @@ func TestManyAsksAtOnceBuildItOnce(t *testing.T) {
func TestAListenerWhoLeavesDoesNotStopTheBuild(t *testing.T) {
r := newRig(t)
r.write(t, "Album One/cover.jpg", jpegBytes(t, 300, 300, color.White))
r.embedded[filepath.Join(r.root, "Album One/01.mp3")] = jpegBytes(t, 600, 600, color.White)
r.gate = make(chan struct{})
ctx, cancel := context.WithCancel(context.Background())
cancel()
if _, err := r.svc.AlbumCover(ctx, 1, Large); !errors.Is(err, context.Canceled) {
t.Errorf("err = %v", err)
errc := make(chan error, 1)
go func() {
_, err := r.svc.AlbumCover(ctx, 1, Large)
errc <- err
}()
time.Sleep(50 * time.Millisecond)
cancel() // the first listener gives up while the build is still reading
if err := <-errc; !errors.Is(err, context.Canceled) {
t.Fatalf("err = %v, want the listener to be told it left", err)
}
close(r.gate)
// The build went on: the next one finds the cover without reading the file again.
deadline := time.Now().Add(5 * time.Second)
for {
if rec, ok, _ := r.store.Get(context.Background(), "album", 1); ok && rec.Status == StatusOK {
break
}
if time.Now().After(deadline) {
t.Fatal("the build was stopped when the listener left")
}
time.Sleep(10 * time.Millisecond)
}
reads := atomic.LoadInt32(&r.reads)
if got := r.read(t, 1, Large); len(got) == 0 || atomic.LoadInt32(&r.reads) != reads {
t.Error("the second listener did not get the cover that was built")
}
}
func TestAChangeDuringABuildIsNotUndoneByIt(t *testing.T) {
r := newRig(t)
r.gate = make(chan struct{})
done := make(chan error, 1)
go func() {
_, err := r.svc.AlbumCover(context.Background(), 1, Large)
done <- err
}()
time.Sleep(50 * time.Millisecond) // the build looked at the files and found no picture
// A new file arrives in the album, and with it a picture.
r.write(t, "Album One/cover.jpg", jpegBytes(t, 400, 400, color.RGBA{0, 200, 0, 255}))
r.store.clear("album", 1)
close(r.gate)
// The build that started before the change must not leave its "no cover" behind.
got, err := func() ([]byte, error) {
<-done
c, err := r.svc.AlbumCover(context.Background(), 1, Large)
if err != nil {
return nil, err
}
defer c.File.Close()
return io.ReadAll(c.File)
}()
if err != nil || len(got) == 0 {
t.Fatalf("after the change: %v, want the new cover", err)
}
if rec := r.store.recs["album/1"]; rec.Status != StatusOK {
t.Errorf("record = %+v, want the cover that was added", rec)
}
}
func TestAtMostAFewCoversAreBuiltAtOnce(t *testing.T) {
r := newRig(t)
var inFlight, maxSeen int32
r.gate = nil
r.svc.embedded = func(string) ([]byte, error) {
n := atomic.AddInt32(&inFlight, 1)
for {
m := atomic.LoadInt32(&maxSeen)
if n <= m || atomic.CompareAndSwapInt32(&maxSeen, m, n) {
break
}
}
time.Sleep(30 * time.Millisecond)
atomic.AddInt32(&inFlight, -1)
return nil, errors.New("no picture")
}
for id := int64(10); id < 30; id++ {
r.src.files[id] = []File{{Root: r.root, RelPath: "Album One/01.mp3"}}
}
var wg sync.WaitGroup
for id := int64(10); id < 30; id++ {
wg.Add(1)
go func() {
defer wg.Done()
_, _ = r.svc.AlbumCover(context.Background(), id, Large)
}()
}
wg.Wait()
if m := atomic.LoadInt32(&maxSeen); m < 1 || m > MaxParallelBuilds {
t.Errorf("%d builds at once, want at most %d", m, MaxParallelBuilds)
}
}
func TestACoverThatWasFoundIsCheckedAgainAfterAWhile(t *testing.T) {
r := newRig(t)
r.write(t, "Album One/cover.jpg", jpegBytes(t, 300, 300, color.RGBA{200, 0, 0, 255}))
first := r.read(t, 1, Large)
r.write(t, "Album One/cover.jpg", jpegBytes(t, 300, 300, color.RGBA{0, 0, 200, 255}))
if again := r.read(t, 1, Large); !bytes.Equal(first, again) {
t.Error("rebuilt for nothing")
}
r.now = r.now.Add(CoverTTL + time.Minute)
if later := r.read(t, 1, Large); bytes.Equal(first, later) {
t.Error("a picture replaced in the folder was never noticed")
}
}
func TestACDFolderUsesThePictureOfItsParent(t *testing.T) {
r := newRig(t)
for _, d := range []string{"Double/CD1", "Double/Disc 2"} {
if err := os.MkdirAll(filepath.Join(r.root, filepath.FromSlash(d)), 0o755); err != nil {
t.Fatal(err)
}
r.write(t, d+"/01.mp3", []byte("audio"))
}
r.write(t, "Double/cover.jpg", jpegBytes(t, 300, 300, color.White))
r.src.files[5] = []File{{Root: r.root, RelPath: "Double/CD1/01.mp3"}, {Root: r.root, RelPath: "Double/Disc 2/01.mp3"}}
if got := r.read(t, 5, Large); len(got) == 0 {
t.Fatal("no cover from the folder above the disc folders")
}
// A folder that is not a disc folder does not borrow the picture of its parent.
if err := os.MkdirAll(filepath.Join(r.root, "Various", "Album A"), 0o755); err != nil {
t.Fatal(err)
}
r.write(t, "Various/Album A/01.mp3", []byte("audio"))
r.write(t, "Various/cover.jpg", jpegBytes(t, 300, 300, color.White))
r.src.files[6] = []File{{Root: r.root, RelPath: "Various/Album A/01.mp3"}}
if _, err := r.svc.AlbumCover(context.Background(), 6, Large); !errors.Is(err, ErrNoCover) {
t.Errorf("err = %v, want no cover: that picture belongs to the folder, not to this album", err)
}
}
func TestAStoredFileThatIsEmptyIsBuiltAgain(t *testing.T) {
r := newRig(t)
r.write(t, "Album One/cover.jpg", jpegBytes(t, 300, 300, color.White))
r.read(t, 1, Large)
if err := os.WriteFile(filepath.Join(r.data, "images", "album", "1-large.jpg"), nil, 0o600); err != nil {
t.Fatal(err)
}
if got := r.read(t, 1, Large); len(got) == 0 {
t.Error("an empty file was served")
}
}
func TestOldLeftoverTemporaryFilesAreRemoved(t *testing.T) {
r := newRig(t)
dir := filepath.Join(r.data, "images", "album")
if err := os.MkdirAll(dir, 0o700); err != nil {
t.Fatal(err)
}
old, fresh := filepath.Join(dir, "tmp-old"), filepath.Join(dir, "tmp-fresh")
for _, p := range []string{old, fresh} {
if err := os.WriteFile(p, []byte("x"), 0o600); err != nil {
t.Fatal(err)
}
}
past := time.Now().Add(-2 * time.Hour)
if err := os.Chtimes(old, past, past); err != nil {
t.Fatal(err)
}
r.write(t, "Album One/cover.jpg", jpegBytes(t, 300, 300, color.White))
r.read(t, 1, Large)
if _, err := os.Stat(old); !os.IsNotExist(err) {
t.Error("a temporary file from a crash was kept")
}
if _, err := os.Stat(fresh); err != nil {
t.Error("a recent temporary file (maybe of a build in progress) was removed")
}
info, err := os.Stat(filepath.Join(r.data, "images"))
if err != nil || info.Mode().Perm()&0o077 != 0 {
t.Errorf("the data folder is %v, want no access for others", info.Mode())
}
}
@ -398,7 +588,7 @@ func TestTheETagChangesWithThePictureNotTheSize(t *testing.T) {
if c1.ETag == "" || c1.ETag == c2.ETag {
t.Errorf("etags: large %q, small %q: they must be set and differ by size", c1.ETag, c2.ETag)
}
r.store.drop("album", 1)
r.store.clear("album", 1)
r.write(t, "Album One/cover.jpg", jpegBytes(t, 600, 600, color.RGBA{0, 0, 200, 255}))
c3, _ := r.svc.AlbumCover(context.Background(), 1, Large)
c3.File.Close()

View file

@ -29,8 +29,8 @@ func (r *ImageRepo) Get(ctx context.Context, kind string, ownerID int64) (images
at int64
)
err := r.db.QueryRowContext(ctx,
"SELECT status, source, etag, width, height, checked_at FROM images WHERE kind = ? AND owner_id = ?",
kind, ownerID).Scan(&rec.Status, &rec.Source, &rec.ETag, &rec.Width, &rec.Height, &at)
"SELECT status, source, etag, width, height, gen, checked_at FROM images WHERE kind = ? AND owner_id = ?",
kind, ownerID).Scan(&rec.Status, &rec.Source, &rec.ETag, &rec.Width, &rec.Height, &rec.Gen, &at)
if errors.Is(err, sql.ErrNoRows) {
return images.Record{}, false, nil
}
@ -41,17 +41,23 @@ func (r *ImageRepo) Get(ctx context.Context, kind string, ownerID int64) (images
return rec, true, nil
}
// Save stores the record, replacing the one there was.
func (r *ImageRepo) Save(ctx context.Context, kind string, ownerID int64, rec images.Record) error {
_, err := r.db.ExecContext(ctx,
`INSERT INTO images (kind, owner_id, status, source, etag, width, height, checked_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?)
// Save stores the record unless the owner's files changed since the generation the
// record carries (see clearAlbumCovers), and says whether it did.
func (r *ImageRepo) Save(ctx context.Context, kind string, ownerID int64, rec images.Record) (bool, error) {
res, err := r.db.ExecContext(ctx,
`INSERT INTO images (kind, owner_id, status, source, etag, width, height, gen, checked_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (kind, owner_id) DO UPDATE SET status = excluded.status, source = excluded.source, etag = excluded.etag,
width = excluded.width, height = excluded.height, checked_at = excluded.checked_at`,
kind, ownerID, rec.Status, rec.Source, rec.ETag, rec.Width, rec.Height, rec.CheckedAt.Unix())
width = excluded.width, height = excluded.height, checked_at = excluded.checked_at
WHERE images.gen = excluded.gen`,
kind, ownerID, rec.Status, rec.Source, rec.ETag, rec.Width, rec.Height, rec.Gen, rec.CheckedAt.Unix())
if err != nil {
return fmt.Errorf("save picture record: %w", err)
return false, fmt.Errorf("save picture record: %w", err)
}
return nil
n, err := res.RowsAffected()
if err != nil {
return false, fmt.Errorf("save picture record: %w", err)
}
return n > 0, nil
}
// AlbumFiles returns the files of the album that can be played, in play order, as
@ -62,7 +68,7 @@ func (r *ImageRepo) AlbumFiles(ctx context.Context, albumID int64) ([]images.Fil
`SELECT l.path, f.rel_path
FROM tracks t JOIN audio_files f ON f.track_id = t.id JOIN libraries l ON l.id = f.library_id
WHERE t.album_id = ? AND f.missing_at IS NULL
ORDER BY t.disc_no, t.track_no, t.id, f.id LIMIT ?`, albumID, maxAlbumFiles)
ORDER BY COALESCE(t.disc_no, 0), COALESCE(t.track_no, 0), t.id, f.id LIMIT ?`, albumID, maxAlbumFiles)
if err != nil {
return nil, fmt.Errorf("list album files: %w", err)
}
@ -84,14 +90,19 @@ func (r *ImageRepo) AlbumFiles(ctx context.Context, albumID int64) ([]images.Fil
return out, nil
}
// clearAlbumCovers forgets the pictures of the albums, so that they are looked for
// again: a file of the album was added or changed. Albums with id 0 are ignored.
// clearAlbumCovers marks the pictures of the albums out of date, so that they are
// looked for again: a file of the album was added or changed. The row is kept, with its
// generation moved, instead of deleted: a build that is running right now started from
// the old generation, and its result must not be saved over the change. Albums with id
// 0 are ignored.
func clearAlbumCovers(ctx context.Context, tx bun.Tx, albumIDs ...int64) error {
for _, id := range albumIDs {
if id == 0 {
continue
}
if _, err := tx.ExecContext(ctx, "DELETE FROM images WHERE kind = 'album' AND owner_id = ?", id); err != nil {
if _, err := tx.ExecContext(ctx,
`INSERT INTO images (kind, owner_id, status, gen, checked_at) VALUES ('album', ?, 'stale', 1, 0)
ON CONFLICT (kind, owner_id) DO UPDATE SET status = 'stale', source = '', etag = '', gen = images.gen + 1`, id); err != nil {
return fmt.Errorf("forget the cover: %w", err)
}
}

View file

@ -20,8 +20,8 @@ func TestImageRepoSaveAndGet(t *testing.T) {
}
at := time.Unix(1_700_000_000, 0)
rec := images.Record{Status: images.StatusOK, Source: images.SourceFolder, ETag: "abc", Width: 800, Height: 600, CheckedAt: at}
if err := repo.Save(ctx, "album", 5, rec); err != nil {
t.Fatal(err)
if saved, err := repo.Save(ctx, "album", 5, rec); err != nil || !saved {
t.Fatalf("Save: saved=%v err=%v", saved, err)
}
got, ok, err := repo.Get(ctx, "album", 5)
if err != nil || !ok || got.Status != rec.Status || got.Source != rec.Source || got.ETag != "abc" || got.Width != 800 || got.Height != 600 || !got.CheckedAt.Equal(at) {
@ -29,8 +29,8 @@ func TestImageRepoSaveAndGet(t *testing.T) {
}
// Saved again, it replaces.
none := images.Record{Status: images.StatusNone, CheckedAt: at.Add(time.Hour)}
if err := repo.Save(ctx, "album", 5, none); err != nil {
t.Fatal(err)
if saved, err := repo.Save(ctx, "album", 5, none); err != nil || !saved {
t.Fatalf("Save again: saved=%v err=%v", saved, err)
}
if got, _, _ = repo.Get(ctx, "album", 5); got.Status != images.StatusNone || got.Source != "" || got.ETag != "" {
t.Errorf("after replacing: %+v", got)
@ -45,6 +45,82 @@ func TestImageRepoSaveAndGet(t *testing.T) {
})
}
func TestAChangeIsNotUndoneByABuildThatStartedBeforeIt(t *testing.T) {
forEachBackend(t, func(t *testing.T, db *bun.DB, _ string) {
ctx := context.Background()
repo := NewImageRepo(db)
at := time.Unix(1_700_000_000, 0)
clear := func(id int64) {
t.Helper()
err := db.RunInTx(ctx, nil, func(ctx context.Context, tx bun.Tx) error { return clearAlbumCovers(ctx, tx, id) })
if err != nil {
t.Fatal(err)
}
}
// A change with no record yet leaves a stale one, with a generation the build never saw.
clear(7)
got, ok, _ := repo.Get(ctx, "album", 7)
if !ok || got.Status != images.StatusStale || got.Gen != 1 {
t.Fatalf("after a change: %+v ok=%v, want stale at generation 1", got, ok)
}
// A build that read generation 0 before the change is not saved.
if saved, err := repo.Save(ctx, "album", 7, images.Record{Status: images.StatusNone, Gen: 0, CheckedAt: at}); err != nil || saved {
t.Errorf("an out of date build: saved=%v err=%v, want it dropped", saved, err)
}
if got, _, _ = repo.Get(ctx, "album", 7); got.Status != images.StatusStale {
t.Errorf("record = %+v, want it still stale", got)
}
// One that read the current generation is, and the generation stays.
if saved, err := repo.Save(ctx, "album", 7, images.Record{Status: images.StatusOK, Source: images.SourceFolder, ETag: "e", Gen: 1, CheckedAt: at}); err != nil || !saved {
t.Errorf("a current build: saved=%v err=%v", saved, err)
}
if got, _, _ = repo.Get(ctx, "album", 7); got.Status != images.StatusOK || got.Gen != 1 || got.ETag != "e" {
t.Errorf("record = %+v", got)
}
// Another change moves it again and forgets the old picture's details.
clear(7)
if got, _, _ = repo.Get(ctx, "album", 7); got.Status != images.StatusStale || got.Gen != 2 || got.ETag != "" || got.Source != "" {
t.Errorf("record = %+v, want stale at generation 2 with no picture details", got)
}
// A build that finds nothing with no record at all is saved (generation 0).
if saved, err := repo.Save(ctx, "album", 8, images.Record{Status: images.StatusNone, CheckedAt: at}); err != nil || !saved {
t.Errorf("first save: saved=%v err=%v", saved, err)
}
})
}
func TestImageRepoAlbumFilesPutFilesWithoutNumbersFirstOnBothDatabases(t *testing.T) {
forEachBackend(t, func(t *testing.T, db *bun.DB, _ string) {
ctx := context.Background()
lib := NewLibraryRepo(db)
l, _ := lib.CreateLibrary(ctx, "m", "/lib/m")
for _, f := range []struct {
rel string
track int
}{{"b.flac", 2}, {"x.flac", 0}, {"a.flac", 1}, {"y.flac", 0}} {
if _, err := lib.SaveFile(ctx, l.ID, pf(f.rel, f.rel, "Mixed", "Band", f.track, 0, "flac", "Band"), 0); err != nil {
t.Fatal(err)
}
}
var album int64
if err := db.QueryRow("SELECT id FROM albums WHERE title = 'Mixed'").Scan(&album); err != nil {
t.Fatal(err)
}
files, err := NewImageRepo(db).AlbumFiles(ctx, album)
if err != nil {
t.Fatal(err)
}
var got []string
for _, f := range files {
got = append(got, f.RelPath)
}
want := []string{"x.flac", "y.flac", "a.flac", "b.flac"} // no number counts as 0, then by id
if len(got) != 4 || got[0] != want[0] || got[1] != want[1] || got[2] != want[2] || got[3] != want[3] {
t.Errorf("order = %v, want %v", got, want)
}
})
}
func TestImageRepoAlbumFiles(t *testing.T) {
forEachBackend(t, func(t *testing.T, db *bun.DB, _ string) {
seedCatalog(t, db)
@ -121,19 +197,25 @@ func TestSaveFileClearsTheCoversThatMayHaveChanged(t *testing.T) {
}
return id
}
mark := func(ids ...int64) {
mark := func(ids ...int64) { // a built cover at the current generation
for _, id := range ids {
if err := images.Save(ctx, "album", id, imagesRec()); err != nil {
cur, _, err := images.Get(ctx, "album", id)
if err != nil {
t.Fatal(err)
}
rec := imagesRec()
rec.Gen = cur.Gen
if _, err := images.Save(ctx, "album", id, rec); err != nil {
t.Fatal(err)
}
}
}
has := func(id int64) bool {
_, ok, err := images.Get(ctx, "album", id)
has := func(id int64) bool { // a cover that is still believed
rec, ok, err := images.Get(ctx, "album", id)
if err != nil {
t.Fatal(err)
}
return ok
return ok && rec.Status != "stale"
}
save("a/1.flac", "First", 0)

View file

@ -3,17 +3,19 @@
-- came from, and when the files were last searched, so a search that found nothing
-- is not repeated at every request.
--
-- Nothing points at the albums table on purpose: the row of a changed album is
-- removed by the code that saves the file.
-- Nothing points at the albums table on purpose. When a file of an album changes, the
-- code that saves it marks the row 'stale' and bumps gen; a build that started before
-- the change is not saved over it.
CREATE TABLE images (
kind TEXT NOT NULL, -- 'album'
owner_id BIGINT NOT NULL,
status TEXT NOT NULL CHECK (status IN ('ok', 'none')),
status TEXT NOT NULL CHECK (status IN ('ok', 'none', 'stale')),
source TEXT NOT NULL DEFAULT '', -- 'embedded', 'folder'
etag TEXT NOT NULL DEFAULT '',
width INTEGER NOT NULL DEFAULT 0, -- of the original
height INTEGER NOT NULL DEFAULT 0,
checked_at BIGINT NOT NULL, -- unix seconds
gen BIGINT NOT NULL DEFAULT 0, -- counts the changes of the owner's files since the record was made
PRIMARY KEY (kind, owner_id)
);

View file

@ -3,17 +3,19 @@
-- came from, and when the files were last searched, so a search that found nothing
-- is not repeated at every request.
--
-- Nothing points at the albums table on purpose: the row of a changed album is
-- removed by the code that saves the file.
-- Nothing points at the albums table on purpose. When a file of an album changes, the
-- code that saves it marks the row 'stale' and bumps gen; a build that started before
-- the change is not saved over it.
CREATE TABLE images (
kind TEXT NOT NULL, -- 'album'
owner_id INTEGER NOT NULL,
status TEXT NOT NULL CHECK (status IN ('ok', 'none')),
status TEXT NOT NULL CHECK (status IN ('ok', 'none', 'stale')),
source TEXT NOT NULL DEFAULT '', -- 'embedded', 'folder'
etag TEXT NOT NULL DEFAULT '',
width INTEGER NOT NULL DEFAULT 0, -- of the original
height INTEGER NOT NULL DEFAULT 0,
checked_at INTEGER NOT NULL, -- unix seconds
gen INTEGER NOT NULL DEFAULT 0, -- counts the changes of the owner's files since the record was made
PRIMARY KEY (kind, owner_id)
);

View file

@ -30,7 +30,8 @@ min_listen_seconds = 20 # a play counts as a listen after this long, or when
half_life_days = 90 # an old listen weighs half as much after this many days
[storage]
data_dir = "data" # files Sonda makes (covers); built from your music, no backup needed
# data_dir = "data" # files Sonda makes (covers); built from your music, no backup needed.
# Default: "data" next to the SQLite database.
[scoring] # weights of the parts of a Discover score (only their proportions matter)
weight_affinity = 0.40 # similar to the artists I like