feat(images): album covers from my own files

- GET and HEAD /albums/{id}/cover serve a JPEG made from the picture embedded in
  the album's files, or else a cover, folder, front, album or albumart image
  (jpg, jpeg, png) in the album's folder. ?size=small (300 px) or large (1000 px).
  The token can be in the query, so an <img> element can use it.
- Built the first time it is asked for, scaled, and kept in storage.data_dir
  (SONDA_DATA_DIR, default "data"); an album with no cover is remembered for a day.
  A new or changed file of the album, or a full rescan, clears the record.
- Nothing is sent anywhere. Pictures are untrusted: files are opened inside the
  library only (links are not followed), over 20 MB or 16 million pixels are
  ignored, and what is served is always a JPEG made here. Stored files are 0600.
- images table (migration 0013, SQLite and PostgreSQL).
- Spec, contract test, README and example config updated.

Closes #37
This commit is contained in:
tgiachi 2026-11-11 20:52:36 +01:00
commit 0e121ead8e
24 changed files with 1847 additions and 4 deletions

1
.gitignore vendored
View file

@ -5,6 +5,7 @@ sonda.toml
*.db
*.db-wal
*.db-shm
/data/
# AI agent instruction files: never published (Codeberg EULA)
agent.md

View file

@ -17,6 +17,7 @@ It is a single executable, one database (SQLite by default, PostgreSQL if you pr
- **Listening history.** Clients report explicit `play`, `complete` and `skip` events. An event counts as a listen when it is `complete`, or when it was played for at least 20 seconds (`listening.min_listen_seconds`); the rule is applied when the history is read, so changing it re-reads the past. You can read your own history, with a `counted` flag on every event. It stays in your database and is never sent anywhere.
- **Your taste.** `GET /me/taste` shows what Sonda has learned from your own listening: the artists you like and the kinds of music (tags) they have in common. A complete listen counts for more than a short one, a skip counts against the artist, and older listens weigh less (they halve every 90 days, `taste.half_life_days`; only the last ten half-lives are read). It is worked out on every request from data already in your database; nothing is sent to any provider.
- **Discover.** `GET /discover` suggests artists you may like, in your library or not, each with a score from 0 to 1 and the parts it is made of: how close it is to the artists you like (affinity), how close its tags are to yours (genre), what your listening says about it (preference) and a bonus for what you have not played (discovery). The weights are in `[scoring]` in the config. A part with no data is left out and the rest rescaled, and every answer says which seeds led to each suggestion. It starts from the artists you name with `?seed=`, or, only if you turned on `discover_from_listening`, from the artists you play most. It needs Last.fm to be switched on to ask for anything new: when Last.fm is off and a seed's similar artists are not stored yet, the answer says `disabled` (what was stored earlier is still used). See [External providers](#external-providers).
- **Album covers.** `GET /albums/{id}/cover` serves the cover of an album from your own files: the picture embedded in them, or else a `cover`, `folder`, `front`, `album` or `albumart` image in the album's folder. It is built the first time it is asked for, scaled (`?size=small` is 300 pixels, `large` 1000) and kept in the data folder (`storage.data_dir`, you can delete it: covers are built again). Nothing is sent anywhere. It also takes `?access_token=`, so an `<img>` element can use it.
- **Wishlist and "not interested".** `PUT /wishlist/{artist}` puts an artist on your wishlist (`GET /wishlist` lists it, `DELETE` takes it off), and `POST /discover/{artist}/dismiss` hides an artist from your Discover for good (`DELETE` brings it back, `GET /discover/dismissed` lists them). An artist is wished or dismissed, never both, and both lists are yours alone. Discover marks the wished artists it shows (`wishlisted`) and never suggests a dismissed one. Nothing is downloaded or sent anywhere: the wishlist is a list for you, and for a download client you may set up later. An artist that only Last.fm named is kept while you have it marked (your lists still show it, but other users cannot see that it exists), and goes with the rest of Last.fm's data if an admin removes Last.fm's data. Removing MusicBrainz's data does not touch it.
- **Your own settings.** `GET` and `PATCH /me/settings` hold choices that belong to you, not to the server. Everyone starts private: `discover_from_listening` is off, so this setting never sends your listening to a provider. When you turn it on, Discover may start from the artists you listen to most, and their names are sent to Last.fm (only if an admin switched it on). What Last.fm answers is kept in the shared database like any other lookup, so another local user can see, from `retrieved_at` of `GET /artists/{id}/similar`, that someone looked up an artist, but not who.
- **Tags.** `GET /artists/{id}/tags` shows what Last.fm and MusicBrainz say kind of music an artist plays, each with its own weights, and the genres of the artist's own files, read offline. See [External providers](#external-providers).
@ -65,6 +66,7 @@ Precedence is: built-in defaults, then the file, then environment variables. An
| `auth.refresh_ttl_hours` | | `720` | Lifetime of a refresh token (30 days). |
| `listening.min_listen_seconds` | | `20` | A play counts as a listen after this long (1 to 3600), or when it is `complete`. |
| `taste.half_life_days` | | `90` | After this many days a listen weighs half as much when learning your taste (1 to 3650). |
| `storage.data_dir` | `SONDA_DATA_DIR` | `data` | Where Sonda keeps the files it makes (covers). They are built from your music, so they need no backup. |
| `scoring.weight_affinity`, `weight_genre`, `weight_preference`, `weight_discovery` | | `0.40`, `0.25`, `0.25`, `0.10` | The weights of the parts of a Discover score (0 to 10 each, not all zero). Only their proportions matter. |
Libraries and the external providers are not in the config: you set them up through the API (see the demo and [External providers](#external-providers)).
@ -233,6 +235,7 @@ Everything lives under `/api/v1`. Bodies are JSON. Errors always look like `{"er
| signed in | `POST /listens` | Record a `play`, `complete` or `skip`; the answer says whether it counts as a listen. |
| signed in | `GET /listens` | Your own history, newest first, with `counted` per event. Query: `limit`, `offset`, `counted=true`. |
| signed in | `GET /discover` | Artists you may like, with a score and its parts. Query: `seed` (an artist id, up to 5 times), `limit` (default 20, max 30), `in_library=true\|false`. |
| signed in | `GET`, `HEAD /albums/{id}/cover` | The cover of an album, a JPEG made from your own files. Query: `size=small\|large`. Also accepts `?access_token=`. |
| signed in | `GET /wishlist`, `PUT /wishlist/{id}`, `DELETE /wishlist/{id}` | Your wishlist: list it (query `limit`, `offset`), add an artist, take one off. |
| signed in | `GET /discover/dismissed`, `POST /discover/{id}/dismiss`, `DELETE /discover/{id}/dismiss` | The artists you are not interested in: list them, dismiss one, bring it back. |
| signed in | `GET /me/taste` | The artists and tags Sonda learned you like, from your own listening. Query: `limit` (default 20, max 50). |
@ -293,6 +296,7 @@ internal/listening/ listen events
internal/taste/ what the listener likes, learned from the listening history
internal/discover/ the score of an artist you may like, and the suggestions
internal/feedback/ the wishlist and the artists a user is not interested in
internal/images/ covers made from the pictures in the user's own files
internal/usersettings/ the settings that belong to one user
internal/storage/ SQLite and PostgreSQL through bun, the repositories
internal/streaming/ opening a track's file safely

View file

@ -21,6 +21,7 @@ import (
"codeberg.org/squidd/sonda/internal/discover"
"codeberg.org/squidd/sonda/internal/discovery"
"codeberg.org/squidd/sonda/internal/feedback"
"codeberg.org/squidd/sonda/internal/images"
"codeberg.org/squidd/sonda/internal/library"
"codeberg.org/squidd/sonda/internal/listening"
"codeberg.org/squidd/sonda/internal/providers"
@ -145,6 +146,10 @@ func run() error {
tasteService := taste.NewService(tasteRepo, taste.Options{MinListen: cfg.Listening.MinListen(), HalfLife: cfg.Taste.HalfLife()})
userSettings := usersettings.NewService(storage.NewUserSettingsRepo(db))
feedbacks := feedback.NewService(storage.NewFeedbackRepo(db))
if err := os.MkdirAll(cfg.Storage.DataDir, 0o700); err != nil {
return fmt.Errorf("create the data folder: %w", err)
}
imageRepo := storage.NewImageRepo(db)
weights := discover.Weights{
Affinity: cfg.Scoring.WeightAffinity, Genre: cfg.Scoring.WeightGenre,
Preference: cfg.Scoring.WeightPreference, Discovery: cfg.Scoring.WeightDiscovery,
@ -164,6 +169,7 @@ func run() error {
Taste: tasteService,
Discover: discover.NewService(tasteService, discoveries, tags, tasteRepo, userSettings, feedbacks, weights),
Feedback: feedbacks,
Images: images.NewService(cfg.Storage.DataDir, imageRepo, imageRepo),
Settings: userSettings,
Streams: streaming.NewService(libraryRepo),
})

View file

@ -19,6 +19,7 @@ import (
"codeberg.org/squidd/sonda/internal/discover"
"codeberg.org/squidd/sonda/internal/discovery"
"codeberg.org/squidd/sonda/internal/feedback"
"codeberg.org/squidd/sonda/internal/images"
"codeberg.org/squidd/sonda/internal/library"
"codeberg.org/squidd/sonda/internal/listening"
"codeberg.org/squidd/sonda/internal/providers"
@ -96,6 +97,8 @@ func newTestEnvDB(t *testing.T, mutate func(*Deps, *bun.DB)) (http.Handler, *bun
}
feedbackSvc := feedback.NewService(storage.NewFeedbackRepo(db))
deps.Feedback = feedbackSvc
imageRepo := storage.NewImageRepo(db)
deps.Images = images.NewService(filepath.Join(t.TempDir(), "data"), imageRepo, imageRepo)
mutate(&deps, db)
// Built from the final services, so a test that swaps one gets a Discover that uses it.
if deps.Discover == nil {

View file

@ -4,6 +4,8 @@ import (
"bytes"
"context"
"encoding/json"
"image"
"image/png"
"io"
"mime"
"net/http"
@ -143,6 +145,14 @@ func newContract(t *testing.T) *contract {
t.Fatal(err)
}
}
// A picture in the folder of the music, for the cover of its album.
var pic bytes.Buffer
if err := png.Encode(&pic, image.NewRGBA(image.Rect(0, 0, 40, 30))); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(music, "cover.png"), pic.Bytes(), 0o644); err != nil {
t.Fatal(err)
}
return &contract{lastfm: lf, t: t, h: h, db: db, reader: reader, router: router, seen: map[string]bool{}, music: music}
}
@ -526,6 +536,28 @@ func TestResponsesMatchTheSpec(t *testing.T) {
c.expect(c.call("GET", "/me/taste?limit=-1", token, nil), 400, "taste with a bad limit", "invalid_input")
c.expect(c.call("GET", "/me/taste?limit=abc", token, nil), 400, "taste with a limit that is not a number", "invalid_input")
// ---- covers ----
cover := "/albums/" + albumID + "/cover"
big := c.call("GET", cover, token, nil)
if big.Code != 200 || big.Header().Get("Content-Type") != "image/jpeg" {
t.Fatalf("cover: %d %s", big.Code, big.Header().Get("Content-Type"))
}
if sm := c.call("GET", cover+"?size=small", token, nil); sm.Code != 200 {
t.Fatalf("small cover: %d", sm.Code)
}
c.expect(c.call("GET", cover+"?size=huge", token, nil), 400, "a cover of a size that does not exist", "invalid_input")
c.expect(c.call("GET", "/albums/999/cover", token, nil), 404, "the cover of an unknown album", "album_not_found")
c.expect(c.call("GET", cover, "", nil), 401, "a cover without a token", "unauthorized")
if rec := c.call("GET", cover+"?access_token="+token, "", nil); rec.Code != 200 {
t.Fatalf("cover with the token in the query: %d", rec.Code)
}
if rec := c.call("GET", cover, token, nil, "If-None-Match", big.Header().Get("ETag")); rec.Code != 304 {
t.Fatalf("cover If-None-Match: %d", rec.Code)
}
if rec := c.call("HEAD", cover, token, nil); rec.Code != 200 {
t.Fatalf("cover HEAD: %d", rec.Code)
}
// ---- discover ----
c.expect(c.call("GET", "/discover", token, nil), 400, "discover with no seed and no permission", "seed_required")
found := c.expect(c.call("GET", "/discover?seed="+artistID+"&limit=5", token, nil), 200, "discover from a named seed")

View file

@ -0,0 +1,229 @@
package httpapi
import (
"bytes"
"encoding/json"
"image"
"image/color"
"image/jpeg"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"testing"
"github.com/uptrace/bun"
"codeberg.org/squidd/sonda/internal/images"
"codeberg.org/squidd/sonda/internal/library"
"codeberg.org/squidd/sonda/internal/storage"
)
type coversEnv struct {
h http.Handler
db *bun.DB
token string
root string
data string
withArt int64 // an album with a cover.jpg in its folder
without int64 // an album with no picture at all
}
func testJPEG(t *testing.T, w, h int, c color.Color) []byte {
t.Helper()
img := image.NewRGBA(image.Rect(0, 0, w, h))
for y := 0; y < h; y++ {
for x := 0; x < w; x++ {
img.Set(x, y, c)
}
}
var b bytes.Buffer
if err := jpeg.Encode(&b, img, nil); err != nil {
t.Fatal(err)
}
return b.Bytes()
}
func newCoversEnv(t *testing.T) *coversEnv {
t.Helper()
e := &coversEnv{root: t.TempDir(), data: filepath.Join(t.TempDir(), "data")}
for _, dir := range []string{"With Art", "Without"} {
if err := os.MkdirAll(filepath.Join(e.root, dir), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(e.root, dir, "01.mp3"), []byte("not audio"), 0o644); err != nil {
t.Fatal(err)
}
}
if err := os.WriteFile(filepath.Join(e.root, "With Art", "cover.jpg"), testJPEG(t, 1500, 1200, color.RGBA{200, 30, 30, 255}), 0o644); err != nil {
t.Fatal(err)
}
e.h, e.db = newTestEnvDB(t, func(d *Deps, db *bun.DB) {
repo := storage.NewImageRepo(db)
d.Images = images.NewService(e.data, repo, repo)
})
e.token = registerAndLogin(t, e.h)
lib := storage.NewLibraryRepo(e.db)
l, err := lib.CreateLibrary(t.Context(), "music", e.root)
if err != nil {
t.Fatal(err)
}
for _, name := range []string{"With Art", "Without"} {
f := library.ParsedFile{
FileInfo: library.FileInfo{RelPath: name + "/01.mp3", Size: 9, MTimeUnix: 1},
Metadata: library.Metadata{
Tags: library.Tags{Title: name + " song", Album: name, AlbumArtist: "Band", Artists: []string{"Band"}},
Properties: library.Properties{Format: "mpeg", DurationMs: 1000},
},
}
if _, err := lib.SaveFile(t.Context(), l.ID, f, 0); err != nil {
t.Fatal(err)
}
}
e.withArt, e.without = e.album(t, "With Art"), e.album(t, "Without")
return e
}
func (e *coversEnv) album(t *testing.T, title string) int64 {
t.Helper()
var id int64
if err := e.db.QueryRow("SELECT id FROM albums WHERE title = ?", title).Scan(&id); err != nil {
t.Fatal(err)
}
return id
}
func (e *coversEnv) get(method, path, token string, header ...string) *httptest.ResponseRecorder {
req := httptest.NewRequest(method, "/api/v1"+path, nil)
if token != "" {
req.Header.Set("Authorization", "Bearer "+token)
}
for i := 0; i+1 < len(header); i += 2 {
req.Header.Set(header[i], header[i+1])
}
rec := httptest.NewRecorder()
e.h.ServeHTTP(rec, req)
return rec
}
func TestCoverNeedsASignedInUserAndAcceptsTheTokenInTheQuery(t *testing.T) {
e := newCoversEnv(t)
path := "/albums/" + id(e.withArt) + "/cover"
if rec := e.get("GET", path, ""); rec.Code != http.StatusUnauthorized {
t.Errorf("no token: %d", rec.Code)
}
// An <img> element cannot send a header.
if rec := e.get("GET", path+"?access_token="+e.token, ""); rec.Code != http.StatusOK {
t.Errorf("token in the query: %d %s", rec.Code, rec.Body.String())
}
if rec := e.get("GET", path+"?access_token=wrong", ""); rec.Code != http.StatusUnauthorized {
t.Errorf("a bad token in the query: %d", rec.Code)
}
}
func TestCoverIsAJPEGInTheSizeAsked(t *testing.T) {
e := newCoversEnv(t)
path := "/albums/" + id(e.withArt) + "/cover"
for query, width := range map[string]int{"": 1000, "?size=large": 1000, "?size=small": 300} {
rec := e.get("GET", path+query, e.token)
if rec.Code != http.StatusOK {
t.Fatalf("%q: %d %s", query, rec.Code, rec.Body.String())
}
if ct := rec.Header().Get("Content-Type"); ct != "image/jpeg" {
t.Errorf("%q: content type %q", query, ct)
}
cfg, format, err := image.DecodeConfig(bytes.NewReader(rec.Body.Bytes()))
if err != nil || format != "jpeg" || cfg.Width != width || cfg.Height != width*1200/1500 {
t.Errorf("%q: %v %s %v, want a jpeg %d wide, same shape as the original", query, cfg, format, err, width)
}
if rec.Header().Get("X-Content-Type-Options") != "nosniff" {
t.Errorf("%q: no nosniff", query)
}
if cc := rec.Header().Get("Cache-Control"); cc != "private, max-age=3600" {
t.Errorf("%q: cache-control %q, want it private", query, cc)
}
}
}
func TestCoverIsCachedWithAnETag(t *testing.T) {
e := newCoversEnv(t)
path := "/albums/" + id(e.withArt) + "/cover"
first := e.get("GET", path, e.token)
etag := first.Header().Get("ETag")
if etag == "" || etag[0] != '"' || first.Header().Get("Last-Modified") == "" {
t.Fatalf("etag %q, last-modified %q", etag, first.Header().Get("Last-Modified"))
}
if rec := e.get("GET", path, e.token, "If-None-Match", etag); rec.Code != http.StatusNotModified || rec.Body.Len() != 0 {
t.Errorf("If-None-Match: %d with %d bytes", rec.Code, rec.Body.Len())
}
if small := e.get("GET", path+"?size=small", e.token); small.Header().Get("ETag") == etag {
t.Error("the small and the large picture have the same etag")
}
if rec := e.get("HEAD", path, e.token); rec.Code != http.StatusOK || rec.Body.Len() != 0 || rec.Header().Get("Content-Length") == "" {
t.Errorf("HEAD: %d, %d bytes, length %q", rec.Code, rec.Body.Len(), rec.Header().Get("Content-Length"))
}
if rec := e.get("GET", path, e.token, "Range", "bytes=0-9"); rec.Code != http.StatusPartialContent || rec.Body.Len() != 10 {
t.Errorf("Range: %d, %d bytes", rec.Code, rec.Body.Len())
}
}
func TestCoverErrors(t *testing.T) {
e := newCoversEnv(t)
for name, tt := range map[string]struct {
path string
code int
err string
}{
"no picture": {"/albums/" + id(e.without) + "/cover", 404, "cover_not_found"},
"unknown album": {"/albums/99999/cover", 404, "album_not_found"},
"bad id": {"/albums/abc/cover", 400, "invalid_input"},
"zero id": {"/albums/0/cover", 400, "invalid_input"},
"bad size": {"/albums/" + id(e.withArt) + "/cover?size=huge", 400, "invalid_input"},
"size twice": {"/albums/" + id(e.withArt) + "/cover?size=small&size=large", 400, "invalid_input"},
"size in capital": {"/albums/" + id(e.withArt) + "/cover?size=Small", 400, "invalid_input"},
} {
rec := e.get("GET", tt.path, e.token)
var body struct {
Error struct{ Code string } `json:"error"`
}
_ = json.Unmarshal(rec.Body.Bytes(), &body)
if rec.Code != tt.code || body.Error.Code != tt.err {
t.Errorf("%s: %d %s, want %d %s", name, rec.Code, rec.Body.String(), tt.code, tt.err)
}
}
}
// A picture is looked for again when a file of the album changes.
func TestCoverIsBuiltAgainAfterTheFilesChange(t *testing.T) {
e := newCoversEnv(t)
path := "/albums/" + id(e.without) + "/cover"
if rec := e.get("GET", path, e.token); rec.Code != http.StatusNotFound {
t.Fatalf("before: %d", rec.Code)
}
if err := os.WriteFile(filepath.Join(e.root, "Without", "folder.jpg"), testJPEG(t, 500, 500, color.White), 0o644); err != nil {
t.Fatal(err)
}
if rec := e.get("GET", path, e.token); rec.Code != http.StatusNotFound {
t.Fatalf("the absence is remembered for a day: %d", rec.Code)
}
// What the scanner does when a file of the album changes.
lib := storage.NewLibraryRepo(e.db)
var libID, fileID int64
if err := e.db.QueryRow("SELECT library_id, id FROM audio_files WHERE rel_path = 'Without/01.mp3'").Scan(&libID, &fileID); err != nil {
t.Fatal(err)
}
f := library.ParsedFile{
FileInfo: library.FileInfo{RelPath: "Without/01.mp3", Size: 9, MTimeUnix: 2},
Metadata: library.Metadata{
Tags: library.Tags{Title: "Without song", Album: "Without", AlbumArtist: "Band", Artists: []string{"Band"}},
Properties: library.Properties{Format: "mpeg", DurationMs: 1000},
},
}
if _, err := lib.SaveFile(t.Context(), libID, f, fileID); err != nil {
t.Fatal(err)
}
if rec := e.get("GET", path, e.token); rec.Code != http.StatusOK {
t.Errorf("after the file changed: %d %s", rec.Code, rec.Body.String())
}
}

View file

@ -0,0 +1,65 @@
package httpapi
import (
"context"
"errors"
"net/http"
"github.com/gin-gonic/gin"
"codeberg.org/squidd/sonda/internal/images"
)
type coverHandlers struct{ svc coverService }
// sizeParam reads ?size=: absent means large, otherwise exactly "small" or "large" once.
func sizeParam(c *gin.Context) (images.Size, bool) {
values := c.QueryArray("size")
switch {
case len(values) == 0:
return images.Large, true
case len(values) == 1 && values[0] == string(images.Small):
return images.Small, true
case len(values) == 1 && values[0] == string(images.Large):
return images.Large, true
}
writeError(c, http.StatusBadRequest, "invalid_input", "size must be small or large")
return "", false
}
// album serves the cover of an album: a JPEG made from the picture in the album's own
// files, built on the first request. It never contacts anybody.
func (h *coverHandlers) album(c *gin.Context) {
id, ok := pathID(c)
if !ok {
return
}
size, ok := sizeParam(c)
if !ok {
return
}
cover, err := h.svc.AlbumCover(c.Request.Context(), id, size)
switch {
case errors.Is(err, images.ErrAlbumNotFound):
writeError(c, http.StatusNotFound, "album_not_found", "no such album")
return
case errors.Is(err, images.ErrNoCover):
writeError(c, http.StatusNotFound, "cover_not_found", "this album has no cover")
return
case errors.Is(err, context.Canceled):
c.AbortWithStatus(statusClientClosed) // the client left; nobody is waiting
return
case err != nil:
_ = c.Error(err)
writeError(c, http.StatusInternalServerError, "internal", "internal error")
return
}
defer cover.File.Close()
hdr := c.Writer.Header()
hdr.Set("Content-Type", "image/jpeg")
hdr.Set("ETag", `"`+cover.ETag+`"`)
hdr.Set("Cache-Control", "private, max-age=3600") // it is the listener's own library
hdr.Set("X-Content-Type-Options", "nosniff")
http.ServeContent(c.Writer, c.Request, "cover.jpg", cover.ModTime, cover.File)
}

View file

@ -15,6 +15,7 @@ import (
"codeberg.org/squidd/sonda/internal/discover"
"codeberg.org/squidd/sonda/internal/discovery"
"codeberg.org/squidd/sonda/internal/feedback"
"codeberg.org/squidd/sonda/internal/images"
"codeberg.org/squidd/sonda/internal/library"
"codeberg.org/squidd/sonda/internal/listening"
"codeberg.org/squidd/sonda/internal/providers"
@ -84,6 +85,10 @@ type feedbackService interface {
List(ctx context.Context, userID int64, kind feedback.Kind, p feedback.Page) (feedback.Listing, error)
}
type coverService interface {
AlbumCover(ctx context.Context, albumID int64, size images.Size) (*images.Cover, error)
}
type tasteService interface {
Taste(ctx context.Context, userID int64, limit int) (taste.Profile, error)
}
@ -124,6 +129,7 @@ type Deps struct {
Taste tasteService
Discover discoverService
Feedback feedbackService
Images coverService
Settings userSettingsService
Streams streamService
}
@ -189,6 +195,10 @@ func NewRouter(d Deps) *gin.Engine {
audio := v1.Group("", requireAuth(d.Tokens, d.Auth, true))
audio.GET("/tracks/:id/stream", st.stream)
audio.HEAD("/tracks/:id/stream", st.stream)
// An <img> element cannot send headers either, so covers take the token the same way.
cv := &coverHandlers{svc: d.Images}
audio.GET("/albums/:id/cover", cv.album)
audio.HEAD("/albums/:id/cover", cv.album)
// Admins only.
admin := protected.Group("", requireAdmin(d.Auth))

View file

@ -974,6 +974,101 @@ paths:
application/json:
schema: {$ref: '#/components/schemas/Error'}
/albums/{id}/cover:
parameters:
- {$ref: '#/components/parameters/Id'}
get:
tags: [Catalog]
operationId: getAlbumCover
summary: The cover of an album
description: |
A JPEG made from the picture in the album's own files: the picture embedded in one of
its first files, or else a `cover`, `folder`, `front`, `album` or `albumart` image
(`.jpg`, `.jpeg` or `.png`, any case) in the album's folder, in that order of names. It
is built the first time it is asked for (that request can take a moment), scaled to fit
and saved in the server's data folder, and served from there afterwards. It never
contacts anybody. Pictures that are too big (over 20 MB or 16 million pixels) or are not
pictures are ignored, and only files inside the library are read.
An album with no picture answers `cover_not_found`, and the server does not look again
for a day; a new or changed file of the album, or a full rescan, makes it look again.
The picture is `private` for an hour in caches and has an `ETag`: send it back as
`If-None-Match`.
An `<img>` element cannot send headers, so, like the stream, this also accepts the access
token as the `access_token` query parameter. A header, when present, always wins.
security:
- bearerAuth: []
- streamQueryToken: []
parameters:
- name: size
in: query
description: "`small` (300 pixels at most on the longest side) or `large` (1000). Large when absent. Never enlarged: a small picture stays as it is."
schema: {type: string, enum: [small, large], default: large}
- name: If-None-Match
in: header
required: false
schema: {type: string}
- name: Range
in: header
required: false
schema: {type: string}
responses:
'200':
description: The cover.
headers:
ETag: {$ref: '#/components/headers/ETag'}
Cache-Control: {$ref: '#/components/headers/CacheControl'}
Content-Length: {$ref: '#/components/headers/ContentLength'}
Last-Modified: {$ref: '#/components/headers/LastModified'}
content:
image/jpeg:
schema: {type: string, format: binary}
'206':
description: The requested byte range of the cover.
headers:
Content-Range:
schema: {type: string}
content:
image/jpeg:
schema: {type: string, format: binary}
'304': {description: Not modified since the `ETag` sent.}
'400': {$ref: '#/components/responses/InvalidInput'}
'401': {$ref: '#/components/responses/Unauthorized'}
'404':
description: No such album (`album_not_found`), or it has no cover (`cover_not_found`).
content:
application/json:
schema: {$ref: '#/components/schemas/Error'}
head:
tags: [Catalog]
operationId: headAlbumCover
summary: Headers of a cover
description: Same as `GET`, without the body.
security:
- bearerAuth: []
- streamQueryToken: []
parameters:
- name: size
in: query
schema: {type: string, enum: [small, large], default: large}
- name: If-None-Match
in: header
required: false
schema: {type: string}
responses:
'200':
description: Headers of the cover.
headers:
ETag: {$ref: '#/components/headers/ETag'}
Cache-Control: {$ref: '#/components/headers/CacheControl'}
Content-Length: {$ref: '#/components/headers/ContentLength'}
Last-Modified: {$ref: '#/components/headers/LastModified'}
'304': {description: Not modified since the `ETag` sent.}
'400': {$ref: '#/components/responses/InvalidInput'}
'401': {$ref: '#/components/responses/Unauthorized'}
'404': {description: "No such album, or it has no cover."}
/tracks:
get:
tags: [Catalog]
@ -1348,7 +1443,7 @@ components:
type: apiKey
in: query
name: access_token
description: The same access token, for stream URLs used by an `<audio>` element. Accepted only on `/tracks/{id}/stream`.
description: The same access token, for stream URLs used by an `<audio>` element. Accepted only on `/tracks/{id}/stream` and `/albums/{id}/cover`.
parameters:
Id:
@ -1392,6 +1487,12 @@ components:
schema: {type: integer, format: int64, minimum: 1}
headers:
ETag:
description: Identifies the stored picture, in quotes.
schema: {type: string}
CacheControl:
description: "`private, max-age=3600`."
schema: {type: string}
AcceptRanges:
description: Always `bytes`.
schema: {type: string}

View file

@ -26,6 +26,7 @@ type Config struct {
Listening Listening `toml:"listening"`
Taste Taste `toml:"taste"`
Scoring Scoring `toml:"scoring"`
Storage Storage `toml:"storage"`
}
// Server holds HTTP listener settings.
@ -103,6 +104,13 @@ type Scoring struct {
WeightDiscovery float64 `toml:"weight_discovery"` // bonus for what I have not played
}
// Storage says where Sonda keeps the files it makes (covers and, later, other
// pictures). They can be built again from the music, so this folder needs no backup,
// but losing it makes every cover be built again.
type Storage struct {
DataDir string `toml:"data_dir"`
}
// Default returns the configuration used when nothing else is set.
func Default() Config {
return Config{
@ -113,6 +121,7 @@ func Default() Config {
Auth: Auth{AccessTTLMinutes: 15, RefreshTTLHours: 720}, // 15 minutes, 30 days
Listening: Listening{MinListenSeconds: 20},
Taste: Taste{HalfLifeDays: 90},
Storage: Storage{DataDir: "data"},
Scoring: Scoring{WeightAffinity: 0.40, WeightGenre: 0.25, WeightPreference: 0.25, WeightDiscovery: 0.10},
}
}
@ -145,6 +154,9 @@ func Load(path string) (Config, error) {
}
func applyEnv(cfg *Config) {
if v := os.Getenv("SONDA_DATA_DIR"); v != "" {
cfg.Storage.DataDir = v
}
if v := os.Getenv("SONDA_ENV"); v != "" {
cfg.Log.Env = v
}
@ -220,6 +232,9 @@ func (c Config) Validate() error {
if sum == 0 {
errs = append(errs, errors.New("scoring.weight_*: at least one weight must be above 0"))
}
if c.Storage.DataDir == "" {
errs = append(errs, errors.New("storage.data_dir must not be empty"))
}
if c.Library.ScanWorkers < 1 {
errs = append(errs, errors.New("library.scan_workers must be at least 1"))
}

View file

@ -138,6 +138,7 @@ func TestValidate(t *testing.T) {
{"huge weight", func(c *Config) { c.Scoring.WeightAffinity = 11 }, "scoring.weight_affinity"},
{"all weights zero", func(c *Config) { c.Scoring = Scoring{} }, "scoring.weight"},
{"one weight is enough", func(c *Config) { c.Scoring = Scoring{WeightPreference: 1} }, ""},
{"empty data dir", func(c *Config) { c.Storage.DataDir = "" }, "storage.data_dir"},
{"bad workers", func(c *Config) { c.Library.ScanWorkers = 0 }, "library.scan_workers"},
{"bad access ttl", func(c *Config) { c.Auth.AccessTTLMinutes = 0 }, "auth.access_ttl_minutes"},
{"bad refresh ttl", func(c *Config) { c.Auth.RefreshTTLHours = 0 }, "auth.refresh_ttl_hours"},
@ -220,6 +221,20 @@ func TestScoringDefaultsAndFile(t *testing.T) {
}
}
func TestStorageDataDir(t *testing.T) {
if got := Default().Storage.DataDir; got != "data" {
t.Errorf("default data dir = %q, want data", got)
}
cfg, err := Load(writeFile(t, t.TempDir(), "[storage]\ndata_dir = \"/srv/sonda\"\n"))
if err != nil || cfg.Storage.DataDir != "/srv/sonda" {
t.Errorf("data dir = %q err=%v", cfg.Storage.DataDir, err)
}
t.Setenv("SONDA_DATA_DIR", "/var/lib/sonda")
if cfg, err = Load(""); err != nil || cfg.Storage.DataDir != "/var/lib/sonda" {
t.Errorf("from the environment: %q err=%v", cfg.Storage.DataDir, err)
}
}
// Provider settings moved into the database. An old config must fail with a
// hint, never be ignored in silence.
func TestLoadExplainsRemovedProviderSettings(t *testing.T) {

View file

@ -0,0 +1,5 @@
//go:build !unix
package images
const nonBlock = 0

View file

@ -0,0 +1,8 @@
//go:build unix
package images
import "syscall"
// nonBlock makes open(2) on a FIFO return at once instead of waiting for a writer.
const nonBlock = syscall.O_NONBLOCK

120
internal/images/process.go Normal file
View file

@ -0,0 +1,120 @@
// Package images turns pictures found on the user's own disk into the covers the
// API serves, and keeps them in a data folder.
//
// Pictures are untrusted input: a file called cover.jpg can be anything. They are
// size-checked before they are decoded, decoded with the standard library only, and
// stored re-encoded, so what is served is always a plain JPEG made here.
package images
import (
"bytes"
"errors"
"fmt"
"image"
"image/color"
"image/draw"
_ "image/gif" // registered for image.Decode
"image/jpeg"
_ "image/png" // registered for image.Decode
)
// Limits and sizes.
const (
// MaxSourceBytes is the largest picture file that is read at all.
MaxSourceBytes = 20 << 20
// MaxPixels is the largest picture that is decoded: a 4000 by 4000 cover. A file
// of a few kilobytes can claim billions of pixels and fill the memory.
MaxPixels = 16_000_000
// LargeEdge and SmallEdge are the longest side of the two stored sizes.
LargeEdge = 1000
SmallEdge = 300
jpegQuality = 85
)
// ErrUnusable is returned for a file that is not a picture Sonda can use.
var ErrUnusable = errors.New("not a usable picture")
// Processed is a picture in the two stored sizes, as JPEG.
type Processed struct {
Large, Small []byte
Width, Height int // of the original
}
// Process decodes a GIF, JPEG or PNG and returns it scaled down to fit LargeEdge and
// SmallEdge (never enlarged), on a white background where it was transparent. The
// same input always gives the same bytes.
func Process(data []byte) (Processed, error) {
if len(data) == 0 || len(data) > MaxSourceBytes {
return Processed{}, fmt.Errorf("%w: %d bytes", ErrUnusable, len(data))
}
cfg, _, err := image.DecodeConfig(bytes.NewReader(data))
if err != nil {
return Processed{}, fmt.Errorf("%w: %v", ErrUnusable, err)
}
if cfg.Width <= 0 || cfg.Height <= 0 || cfg.Width*cfg.Height > MaxPixels {
return Processed{}, fmt.Errorf("%w: %dx%d pixels", ErrUnusable, cfg.Width, cfg.Height)
}
src, _, err := image.Decode(bytes.NewReader(data))
if err != nil {
return Processed{}, fmt.Errorf("%w: %v", ErrUnusable, err)
}
flat := image.NewRGBA(src.Bounds())
draw.Draw(flat, flat.Bounds(), image.NewUniform(color.White), image.Point{}, draw.Src)
draw.Draw(flat, flat.Bounds(), src, src.Bounds().Min, draw.Over)
large := fit(flat, LargeEdge)
small := fit(large, SmallEdge)
out := Processed{Width: cfg.Width, Height: cfg.Height}
if out.Large, err = encode(large); err != nil {
return Processed{}, err
}
if out.Small, err = encode(small); err != nil {
return Processed{}, err
}
return out, nil
}
func encode(img *image.RGBA) ([]byte, error) {
var b bytes.Buffer
if err := jpeg.Encode(&b, img, &jpeg.Options{Quality: jpegQuality}); err != nil {
return nil, fmt.Errorf("encode picture: %w", err)
}
return b.Bytes(), nil
}
// fit returns src scaled down so that its longest side is at most edge, averaging
// the source pixels under each new one. A picture that is small enough is returned as it is.
func fit(src *image.RGBA, edge int) *image.RGBA {
b := src.Bounds()
sw, sh := b.Dx(), b.Dy()
if sw <= edge && sh <= edge {
return src
}
dw, dh := edge, edge
if sw >= sh {
dh = max(1, sh*edge/sw)
} else {
dw = max(1, sw*edge/sh)
}
dst := image.NewRGBA(image.Rect(0, 0, dw, dh))
for y := 0; y < dh; y++ {
y0, y1 := y*sh/dh, max(y*sh/dh+1, (y+1)*sh/dh)
for x := 0; x < dw; x++ {
x0, x1 := x*sw/dw, max(x*sw/dw+1, (x+1)*sw/dw)
var r, g, bl, n uint32
for sy := y0; sy < y1; sy++ {
row := src.Pix[(sy)*src.Stride:]
for sx := x0; sx < x1; sx++ {
p := row[sx*4 : sx*4+4]
r, g, bl, n = r+uint32(p[0]), g+uint32(p[1]), bl+uint32(p[2]), n+1
}
}
o := dst.PixOffset(x, y)
dst.Pix[o], dst.Pix[o+1], dst.Pix[o+2], dst.Pix[o+3] = uint8(r/n), uint8(g/n), uint8(bl/n), 255
}
}
return dst
}

View file

@ -0,0 +1,162 @@
package images
import (
"bytes"
"encoding/binary"
"hash/crc32"
"image"
"image/color"
"image/gif"
"image/jpeg"
"image/png"
"strings"
"testing"
)
func solid(w, h int, c color.Color) image.Image {
img := image.NewRGBA(image.Rect(0, 0, w, h))
for y := 0; y < h; y++ {
for x := 0; x < w; x++ {
img.Set(x, y, c)
}
}
return img
}
func encodePNG(t *testing.T, img image.Image) []byte {
t.Helper()
var b bytes.Buffer
if err := png.Encode(&b, img); err != nil {
t.Fatal(err)
}
return b.Bytes()
}
func encodeJPEG(t *testing.T, img image.Image) []byte {
t.Helper()
var b bytes.Buffer
if err := jpeg.Encode(&b, img, nil); err != nil {
t.Fatal(err)
}
return b.Bytes()
}
// pngHeader is the start of a valid PNG: signature and IHDR, claiming a w by h picture.
func pngHeader(w, h uint32) []byte {
ihdr := make([]byte, 0, 17)
ihdr = append(ihdr, "IHDR"...)
ihdr = binary.BigEndian.AppendUint32(ihdr, w)
ihdr = binary.BigEndian.AppendUint32(ihdr, h)
ihdr = append(ihdr, 8, 6, 0, 0, 0) // 8 bits, RGBA, no interlace
out := []byte("\x89PNG\r\n\x1a\n")
out = binary.BigEndian.AppendUint32(out, 13)
out = append(out, ihdr...)
return binary.BigEndian.AppendUint32(out, crc32.ChecksumIEEE(ihdr))
}
func dims(t *testing.T, data []byte) (int, int, string) {
t.Helper()
cfg, format, err := image.DecodeConfig(bytes.NewReader(data))
if err != nil {
t.Fatalf("the result is not an image: %v", err)
}
return cfg.Width, cfg.Height, format
}
func TestProcessScalesToFitAndKeepsTheShape(t *testing.T) {
out, err := Process(encodePNG(t, solid(2000, 1000, color.RGBA{200, 30, 30, 255})))
if err != nil {
t.Fatal(err)
}
if w, h, f := dims(t, out.Large); w != 1000 || h != 500 || f != "jpeg" {
t.Errorf("large = %dx%d %s, want 1000x500 jpeg", w, h, f)
}
if w, h, f := dims(t, out.Small); w != 300 || h != 150 || f != "jpeg" {
t.Errorf("small = %dx%d %s, want 300x150 jpeg", w, h, f)
}
if out.Width != 2000 || out.Height != 1000 {
t.Errorf("original size = %dx%d, want 2000x1000", out.Width, out.Height)
}
}
func TestProcessNeverEnlarges(t *testing.T) {
out, err := Process(encodeJPEG(t, solid(200, 200, color.RGBA{10, 10, 200, 255})))
if err != nil {
t.Fatal(err)
}
for name, data := range map[string][]byte{"large": out.Large, "small": out.Small} {
if w, h, _ := dims(t, data); w != 200 || h != 200 {
t.Errorf("%s = %dx%d, want it left at 200x200", name, w, h)
}
}
}
func TestProcessKeepsTheColours(t *testing.T) {
out, err := Process(encodePNG(t, solid(1500, 1500, color.RGBA{220, 20, 20, 255})))
if err != nil {
t.Fatal(err)
}
img, err := jpeg.Decode(bytes.NewReader(out.Large))
if err != nil {
t.Fatal(err)
}
r, g, b, _ := img.At(500, 500).RGBA()
if r>>8 < 190 || g>>8 > 60 || b>>8 > 60 {
t.Errorf("the middle is (%d,%d,%d), want about (220,20,20)", r>>8, g>>8, b>>8)
}
}
func TestProcessPutsTransparencyOnWhite(t *testing.T) {
out, err := Process(encodePNG(t, solid(400, 400, color.NRGBA{0, 0, 0, 0})))
if err != nil {
t.Fatal(err)
}
img, _ := jpeg.Decode(bytes.NewReader(out.Large))
r, g, b, _ := img.At(10, 10).RGBA()
if r>>8 < 240 || g>>8 < 240 || b>>8 < 240 {
t.Errorf("a transparent picture became (%d,%d,%d), want white", r>>8, g>>8, b>>8)
}
}
func TestProcessReadsGIFAndJPEGAndPNG(t *testing.T) {
var g bytes.Buffer
if err := gif.Encode(&g, solid(50, 50, color.RGBA{1, 2, 3, 255}), nil); err != nil {
t.Fatal(err)
}
for name, data := range map[string][]byte{"gif": g.Bytes(), "jpeg": encodeJPEG(t, solid(50, 50, color.White)), "png": encodePNG(t, solid(50, 50, color.White))} {
if _, err := Process(data); err != nil {
t.Errorf("%s: %v", name, err)
}
}
}
func TestProcessRefusesWhatIsNotAUsablePicture(t *testing.T) {
bomb := pngHeader(60000, 60000) // a valid header that claims 3.6 billion pixels
for name, data := range map[string][]byte{
"empty": nil,
"text": []byte(strings.Repeat("not an image ", 100)),
"truncated": encodeJPEG(t, solid(300, 300, color.White))[:80],
"claims huge": bomb,
"zero width": pngHeader(0, 5),
} {
if _, err := Process(data); err == nil {
t.Errorf("%s: accepted", name)
}
}
}
func TestProcessRefusesAFileThatIsTooBig(t *testing.T) {
big := append(encodeJPEG(t, solid(10, 10, color.White)), make([]byte, MaxSourceBytes)...)
if _, err := Process(big); err == nil {
t.Error("a file over the size limit was accepted")
}
}
func TestProcessIsDeterministicSoTheEtagIsStable(t *testing.T) {
src := encodePNG(t, solid(1200, 900, color.RGBA{5, 100, 200, 255}))
a, _ := Process(src)
b, _ := Process(src)
if !bytes.Equal(a.Large, b.Large) || !bytes.Equal(a.Small, b.Small) {
t.Error("the same picture gave different bytes")
}
}

342
internal/images/service.go Normal file
View file

@ -0,0 +1,342 @@
package images
import (
"context"
"crypto/sha256"
"encoding/hex"
"errors"
"fmt"
"io"
"io/fs"
"os"
"path"
"path/filepath"
"sort"
"strconv"
"strings"
"time"
"go.senan.xyz/taglib"
"golang.org/x/sync/singleflight"
)
// Kinds of owner a picture belongs to.
const kindAlbum = "album"
// What is stored about a picture, and where it came from.
const (
StatusOK = "ok" // a cover was found and is stored
StatusNone = "none" // the files were searched and have none
SourceEmbedded = "embedded" // the picture inside an audio file
SourceFolder = "folder" // a cover.jpg, folder.png ... next to the audio
)
// Tuning.
const (
// MaxFilesTried is how many files of an album are asked for an embedded picture:
// they all carry the same one, and reading a file costs.
MaxFilesTried = 3
// NoCoverTTL is how long "no cover" is believed. Adding a picture to a folder
// changes no audio file, so the answer has to be checked again now and then.
NoCoverTTL = 24 * time.Hour
buildTimeout = 2 * time.Minute
)
// Errors returned by the service.
var (
ErrAlbumNotFound = errors.New("album not found")
ErrNoCover = errors.New("this album has no cover")
ErrInvalidSize = errors.New("invalid size")
)
// Size is one of the two stored sizes.
type Size string
// The sizes.
const (
Small Size = "small" // 300 pixels at most
Large Size = "large" // 1000 pixels at most
)
// File is where an audio file lives: the directory of its library and the path
// inside it, as the scanner stored them.
type File struct {
Root string
RelPath string // slash-separated
}
// Source finds the playable files of an album, in play order. It returns
// ErrAlbumNotFound for an album that does not exist or has nothing to play.
type Source interface {
AlbumFiles(ctx context.Context, albumID int64) ([]File, error)
}
// Record is what is known about the picture of one owner.
type Record struct {
Status string // StatusOK or StatusNone
Source string // SourceEmbedded or SourceFolder; empty for StatusNone
ETag string // identifies the stored picture
Width int // of the original
Height int
CheckedAt time.Time
}
// Store keeps the records. The files themselves are in the data folder.
type Store interface {
Get(ctx context.Context, kind string, ownerID int64) (Record, bool, error)
Save(ctx context.Context, kind string, ownerID int64, r Record) error
}
// Cover is a stored picture ready to be served. The caller must close File.
type Cover struct {
File *os.File
ModTime time.Time
ETag string
}
// Service builds covers from the user's own files, on the first request, and keeps
// them in the data folder.
type Service struct {
dir string
store Store
src Source
embedded func(path string) ([]byte, error)
now func() time.Time
flight singleflight.Group
}
// NewService builds a Service that keeps its files under dir.
func NewService(dir string, store Store, src Source) *Service {
return &Service{dir: dir, store: store, src: src, embedded: taglib.ReadImage, now: time.Now}
}
// AlbumCover returns the cover of an album in the given size, building it first when
// it was never built or its record was cleared (a changed file does that), and
// searching again for an album that had none after NoCoverTTL.
func (s *Service) AlbumCover(ctx context.Context, albumID int64, size Size) (*Cover, error) {
if size != Small && size != Large {
return nil, fmt.Errorf("%w: %q", ErrInvalidSize, string(size))
}
if albumID <= 0 {
return nil, ErrAlbumNotFound
}
if err := ctx.Err(); err != nil {
return nil, err
}
rec, ok, err := s.store.Get(ctx, kindAlbum, albumID)
if err != nil {
return nil, fmt.Errorf("read picture record: %w", err)
}
if ok && rec.Status == StatusNone && s.now().Sub(rec.CheckedAt) < NoCoverTTL {
return nil, ErrNoCover
}
if ok && rec.Status == StatusOK {
if c, err := s.open(albumID, size, rec); err == nil {
return c, nil
} // the stored file is gone: build it again
}
// One build per album, however many ask at once. It runs on its own context, so
// the first one to leave does not fail the others.
done := s.flight.DoChan(strconv.FormatInt(albumID, 10), func() (v any, err error) {
defer func() {
if r := recover(); r != nil {
err = fmt.Errorf("building a cover panicked: %v", r)
}
}()
return s.build(albumID)
})
var built Record
select {
case <-ctx.Done():
return nil, ctx.Err()
case r := <-done:
if r.Err != nil {
return nil, r.Err
}
built = r.Val.(Record)
}
if built.Status == StatusNone {
return nil, ErrNoCover
}
return s.open(albumID, size, built)
}
func (s *Service) filePath(albumID int64, size Size) string {
return filepath.Join(s.dir, "images", kindAlbum, strconv.FormatInt(albumID, 10)+"-"+string(size)+".jpg")
}
func (s *Service) open(albumID int64, size Size, rec Record) (*Cover, error) {
f, err := os.Open(s.filePath(albumID, size))
if err != nil {
return nil, err
}
info, err := f.Stat()
if err != nil {
_ = f.Close()
return nil, err
}
etag := rec.ETag
if size == Small {
etag += "s"
}
return &Cover{File: f, ModTime: info.ModTime(), ETag: etag}, nil
}
// build looks for a picture, stores it and records the outcome.
func (s *Service) build(albumID int64) (Record, error) {
ctx, cancel := context.WithTimeout(context.Background(), buildTimeout)
defer cancel()
files, err := s.src.AlbumFiles(ctx, albumID)
if err != nil {
return Record{}, err
}
rec := Record{Status: StatusNone, CheckedAt: s.now()}
if p, source, ok := s.find(files); ok {
if err := s.write(albumID, p); err != nil {
return Record{}, err
}
sum := sha256.Sum256(p.Large)
rec = Record{Status: StatusOK, Source: source, ETag: hex.EncodeToString(sum[:8]), Width: p.Width, Height: p.Height, CheckedAt: rec.CheckedAt}
}
if err := s.store.Save(ctx, kindAlbum, albumID, rec); err != nil {
return Record{}, fmt.Errorf("save picture record: %w", err)
}
return rec, nil
}
// find returns the first usable picture: embedded in one of the first files, then an
// image in the folder of one of them.
func (s *Service) find(files []File) (Processed, string, bool) {
if len(files) > MaxFilesTried {
files = files[:MaxFilesTried]
}
for _, f := range files {
if data, ok := s.embeddedPicture(f); ok {
if p, err := Process(data); err == nil {
return p, SourceEmbedded, true
}
}
}
seen := map[string]bool{}
for _, f := range files {
dir := path.Dir(f.RelPath)
if seen[f.Root+"\x00"+dir] {
continue
}
seen[f.Root+"\x00"+dir] = true
for _, data := range folderPictures(f.Root, dir) {
if p, err := Process(data); err == nil {
return p, SourceFolder, true
}
}
}
return Processed{}, "", false
}
// embeddedPicture reads the picture inside an audio file, after checking through
// os.Root that the file is a regular file inside its library.
func (s *Service) embeddedPicture(f File) ([]byte, bool) {
root, err := os.OpenRoot(filepath.Clean(f.Root))
if err != nil {
return nil, false
}
defer root.Close()
info, err := root.Stat(f.RelPath)
if err != nil || !info.Mode().IsRegular() {
return nil, false
}
data, err := s.embedded(filepath.Join(filepath.Clean(f.Root), filepath.FromSlash(f.RelPath)))
if err != nil || len(data) == 0 {
return nil, false
}
return data, true
}
// folderNames are the names of picture files that mean "the cover", best first.
var folderNames = []string{"cover", "folder", "front", "album", "albumart"}
// folderPictures returns the contents of the cover-like picture files of a folder of
// the library, best name first. Only regular files inside the library are read (a link
// is not followed), and none above MaxSourceBytes.
func folderPictures(rootDir, dir string) [][]byte {
root, err := os.OpenRoot(filepath.Clean(rootDir))
if err != nil {
return nil
}
defer root.Close()
entries, err := fs.ReadDir(root.FS(), dir)
if err != nil {
return nil
}
type pick struct {
rank int
name string
}
var picks []pick
for _, e := range entries {
if !e.Type().IsRegular() {
continue
}
base := strings.ToLower(e.Name())
ext := path.Ext(base)
if ext != ".jpg" && ext != ".jpeg" && ext != ".png" {
continue
}
for rank, n := range folderNames {
if strings.TrimSuffix(base, ext) == n {
if info, err := e.Info(); err == nil && info.Size() <= MaxSourceBytes {
picks = append(picks, pick{rank, e.Name()})
}
break
}
}
}
sort.Slice(picks, func(i, j int) bool {
if picks[i].rank != picks[j].rank {
return picks[i].rank < picks[j].rank
}
return picks[i].name < picks[j].name
})
var out [][]byte
for _, p := range picks {
f, err := root.OpenFile(path.Join(dir, p.name), os.O_RDONLY|nonBlock, 0)
if err != nil {
continue
}
data, err := io.ReadAll(io.LimitReader(f, MaxSourceBytes+1))
_ = f.Close()
if err == nil && len(data) <= MaxSourceBytes {
out = append(out, data)
}
}
return out
}
// write stores both sizes. Each goes to a temporary file first and is renamed, so a
// reader never sees half a picture.
func (s *Service) write(albumID int64, p Processed) error {
dir := filepath.Dir(s.filePath(albumID, Large))
if err := os.MkdirAll(dir, 0o700); err != nil {
return fmt.Errorf("create the data folder: %w", err)
}
for size, data := range map[Size][]byte{Large: p.Large, Small: p.Small} {
tmp, err := os.CreateTemp(dir, "tmp-*")
if err != nil {
return fmt.Errorf("create a picture file: %w", err)
}
_, werr := tmp.Write(data)
cerr := tmp.Close()
if werr != nil || cerr != nil {
_ = os.Remove(tmp.Name())
return fmt.Errorf("write a picture file: %w", errors.Join(werr, cerr))
}
if err := os.Rename(tmp.Name(), s.filePath(albumID, size)); err != nil {
_ = os.Remove(tmp.Name())
return fmt.Errorf("store a picture file: %w", err)
}
}
return nil
}

View file

@ -0,0 +1,408 @@
package images
import (
"bytes"
"context"
"errors"
"image"
"image/color"
"image/jpeg"
"io"
"os"
"path/filepath"
"sync"
"sync/atomic"
"testing"
"time"
)
type fakeStore struct {
mu sync.Mutex
recs map[string]Record
gets int
err error
}
func key(kind string, id int64) string { return kind + "/" + string(rune('0'+id)) }
func (f *fakeStore) Get(_ context.Context, kind string, id int64) (Record, bool, error) {
f.mu.Lock()
defer f.mu.Unlock()
f.gets++
r, ok := f.recs[key(kind, id)]
return r, ok, f.err
}
func (f *fakeStore) Save(_ context.Context, kind string, id int64, r Record) error {
f.mu.Lock()
defer f.mu.Unlock()
f.recs[key(kind, id)] = r
return f.err
}
func (f *fakeStore) drop(kind string, id int64) {
f.mu.Lock()
defer f.mu.Unlock()
delete(f.recs, key(kind, id))
}
type fakeSource struct {
files map[int64][]File
calls int32
}
func (f *fakeSource) AlbumFiles(_ context.Context, id int64) ([]File, error) {
atomic.AddInt32(&f.calls, 1)
files, ok := f.files[id]
if !ok {
return nil, ErrAlbumNotFound
}
return files, nil
}
type rig struct {
svc *Service
store *fakeStore
src *fakeSource
root string
data string
embedded map[string][]byte // by full path
reads int32 // embedded reads
now time.Time
}
func jpegBytes(t *testing.T, w, h int, c color.Color) []byte {
t.Helper()
img := image.NewRGBA(image.Rect(0, 0, w, h))
for y := 0; y < h; y++ {
for x := 0; x < w; x++ {
img.Set(x, y, c)
}
}
var b bytes.Buffer
if err := jpeg.Encode(&b, img, nil); err != nil {
t.Fatal(err)
}
return b.Bytes()
}
func newRig(t *testing.T) *rig {
t.Helper()
r := &rig{
store: &fakeStore{recs: map[string]Record{}}, root: t.TempDir(), data: filepath.Join(t.TempDir(), "data"),
embedded: map[string][]byte{}, now: time.Unix(1_700_000_000, 0),
}
if err := os.MkdirAll(filepath.Join(r.root, "Album One"), 0o755); err != nil {
t.Fatal(err)
}
r.write(t, "Album One/01.mp3", []byte("audio"))
r.write(t, "Album One/02.mp3", []byte("audio"))
r.src = &fakeSource{files: map[int64][]File{
1: {{Root: r.root, RelPath: "Album One/01.mp3"}, {Root: r.root, RelPath: "Album One/02.mp3"}},
}}
r.svc = NewService(r.data, r.store, r.src)
r.svc.embedded = func(path string) ([]byte, error) {
atomic.AddInt32(&r.reads, 1)
if b, ok := r.embedded[path]; ok {
return b, nil
}
return nil, errors.New("no picture")
}
r.svc.now = func() time.Time { return r.now }
return r
}
func (r *rig) write(t *testing.T, rel string, data []byte) {
t.Helper()
if err := os.WriteFile(filepath.Join(r.root, filepath.FromSlash(rel)), data, 0o644); err != nil {
t.Fatal(err)
}
}
func (r *rig) read(t *testing.T, id int64, size Size) []byte {
t.Helper()
c, err := r.svc.AlbumCover(context.Background(), id, size)
if err != nil {
t.Fatalf("AlbumCover: %v", err)
}
defer c.File.Close()
b, _ := io.ReadAll(c.File)
return b
}
func TestTheEmbeddedPictureComesFirst(t *testing.T) {
r := newRig(t)
r.embedded[filepath.Join(r.root, "Album One/01.mp3")] = jpegBytes(t, 1500, 1500, color.RGBA{200, 0, 0, 255})
r.write(t, "Album One/cover.jpg", jpegBytes(t, 400, 400, color.RGBA{0, 0, 200, 255}))
img, _ := jpeg.Decode(bytes.NewReader(r.read(t, 1, Large)))
if w := img.Bounds().Dx(); w != 1000 {
t.Errorf("large is %d wide, want 1000", w)
}
red, _, blue, _ := img.At(500, 500).RGBA()
if red>>8 < 150 || blue>>8 > 80 {
t.Errorf("the cover is not the embedded one: (%d, %d)", red>>8, blue>>8)
}
rec := r.store.recs["album/1"]
if rec.Status != StatusOK || rec.Source != SourceEmbedded || rec.Width != 1500 || rec.ETag == "" {
t.Errorf("record = %+v", rec)
}
}
func TestTheSmallSizeIsSmall(t *testing.T) {
r := newRig(t)
r.embedded[filepath.Join(r.root, "Album One/01.mp3")] = jpegBytes(t, 1500, 1500, color.White)
cfg, _, err := image.DecodeConfig(bytes.NewReader(r.read(t, 1, Small)))
if err != nil || cfg.Width != 300 {
t.Errorf("small = %v err %v, want 300 wide", cfg, err)
}
}
func TestWithoutAnEmbeddedPictureTheFolderImageIsUsedInPriorityOrder(t *testing.T) {
r := newRig(t)
r.write(t, "Album One/scan-back.jpg", jpegBytes(t, 300, 300, color.RGBA{0, 200, 0, 255}))
r.write(t, "Album One/Folder.PNG", jpegBytes(t, 300, 300, color.RGBA{0, 0, 200, 255})) // wrong extension for the content, still a picture
r.write(t, "Album One/FRONT.jpg", jpegBytes(t, 300, 300, color.RGBA{200, 0, 0, 255}))
// "folder" outranks "front"; "scan-back" is not a cover name at all.
img, _ := jpeg.Decode(bytes.NewReader(r.read(t, 1, Large)))
_, _, blue, _ := img.At(100, 100).RGBA()
if blue>>8 < 150 {
t.Errorf("not the Folder.PNG: blue = %d", blue>>8)
}
if got := r.store.recs["album/1"].Source; got != SourceFolder {
t.Errorf("source = %q", got)
}
// "cover" outranks "folder" and "front".
r.store.drop("album", 1)
r.write(t, "Album One/cover.jpeg", jpegBytes(t, 300, 300, color.RGBA{0, 200, 0, 255}))
img, _ = jpeg.Decode(bytes.NewReader(r.read(t, 1, Large)))
_, green, _, _ := img.At(100, 100).RGBA()
if green>>8 < 150 {
t.Errorf("not the cover.jpeg: green = %d", green>>8)
}
}
func TestAPictureThatCannotBeUsedFallsThroughToTheNextOne(t *testing.T) {
r := newRig(t)
r.embedded[filepath.Join(r.root, "Album One/01.mp3")] = []byte("this is not a picture")
r.write(t, "Album One/cover.jpg", []byte("neither is this"))
r.write(t, "Album One/folder.jpg", jpegBytes(t, 200, 200, color.White))
if got := r.read(t, 1, Large); len(got) == 0 {
t.Fatal("nothing served")
}
if got := r.store.recs["album/1"].Source; got != SourceFolder {
t.Errorf("source = %q, want the folder.jpg that works", got)
}
}
func TestOnlyTheFirstFewFilesAreTriedForAnEmbeddedPicture(t *testing.T) {
r := newRig(t)
var files []File
for i := 0; i < 10; i++ {
files = append(files, File{Root: r.root, RelPath: "Album One/01.mp3"})
}
r.src.files[1] = files
_, err := r.svc.AlbumCover(context.Background(), 1, Large)
if !errors.Is(err, ErrNoCover) {
t.Fatalf("err = %v", err)
}
if n := atomic.LoadInt32(&r.reads); n > MaxFilesTried {
t.Errorf("%d files read, want at most %d", n, MaxFilesTried)
}
}
func TestAnAlbumWithoutACoverIsRememberedForADayOnly(t *testing.T) {
r := newRig(t)
if _, err := r.svc.AlbumCover(context.Background(), 1, Large); !errors.Is(err, ErrNoCover) {
t.Fatalf("err = %v", err)
}
if rec := r.store.recs["album/1"]; rec.Status != StatusNone {
t.Fatalf("record = %+v", rec)
}
calls := atomic.LoadInt32(&r.src.calls)
reads := atomic.LoadInt32(&r.reads)
for i := 0; i < 5; i++ { // the disk is not searched again
if _, err := r.svc.AlbumCover(context.Background(), 1, Large); !errors.Is(err, ErrNoCover) {
t.Fatalf("err = %v", err)
}
}
if atomic.LoadInt32(&r.src.calls) != calls || atomic.LoadInt32(&r.reads) != reads {
t.Error("the files were looked at again within the day")
}
// A cover.jpg was added meanwhile; a day later it is found.
r.write(t, "Album One/cover.jpg", jpegBytes(t, 300, 300, color.White))
r.now = r.now.Add(NoCoverTTL + time.Minute)
if got := r.read(t, 1, Large); len(got) == 0 {
t.Error("the new cover was not found after the day")
}
}
func TestAnAlbumWhoseRecordWasClearedIsBuiltAgain(t *testing.T) {
r := newRig(t)
r.write(t, "Album One/cover.jpg", jpegBytes(t, 300, 300, color.RGBA{200, 0, 0, 255}))
first := r.read(t, 1, Large)
r.write(t, "Album One/cover.jpg", jpegBytes(t, 300, 300, color.RGBA{0, 0, 200, 255}))
if second := r.read(t, 1, Large); !bytes.Equal(first, second) {
t.Error("a cover that is stored was built again for nothing")
}
r.store.drop("album", 1) // what a changed file does (see the storage)
if third := r.read(t, 1, Large); bytes.Equal(first, third) {
t.Error("after the record was cleared the old picture was served")
}
}
func TestAStoredFileThatWentMissingIsBuiltAgain(t *testing.T) {
r := newRig(t)
r.write(t, "Album One/cover.jpg", jpegBytes(t, 300, 300, color.White))
r.read(t, 1, Large)
if err := os.Remove(filepath.Join(r.data, "images", "album", "1-large.jpg")); err != nil {
t.Fatal(err)
}
if got := r.read(t, 1, Large); len(got) == 0 {
t.Error("nothing served after the stored file was removed")
}
}
func TestStoredFilesArePrivate(t *testing.T) {
r := newRig(t)
r.write(t, "Album One/cover.jpg", jpegBytes(t, 300, 300, color.White))
r.read(t, 1, Large)
for _, p := range []string{filepath.Join(r.data, "images", "album", "1-large.jpg"), filepath.Join(r.data, "images", "album", "1-small.jpg")} {
info, err := os.Stat(p)
if err != nil || info.Mode().Perm()&0o077 != 0 {
t.Errorf("%s: %v %v, want no access for group or others", p, info, err)
}
}
// And no half-written files are left behind.
entries, _ := os.ReadDir(filepath.Join(r.data, "images", "album"))
for _, e := range entries {
if filepath.Ext(e.Name()) != ".jpg" {
t.Errorf("stray file %s", e.Name())
}
}
}
func TestAnUnknownAlbumIsNotFoundAndLeavesNoRecord(t *testing.T) {
r := newRig(t)
if _, err := r.svc.AlbumCover(context.Background(), 9, Large); !errors.Is(err, ErrAlbumNotFound) {
t.Errorf("err = %v", err)
}
if len(r.store.recs) != 0 {
t.Errorf("records = %v", r.store.recs)
}
if _, err := r.svc.AlbumCover(context.Background(), 1, Size("huge")); !errors.Is(err, ErrInvalidSize) {
t.Errorf("a bad size: err = %v", err)
}
if _, err := r.svc.AlbumCover(context.Background(), 0, Large); !errors.Is(err, ErrAlbumNotFound) {
t.Errorf("album 0: err = %v", err)
}
}
func TestPicturesOutsideTheLibraryAreNeverRead(t *testing.T) {
r := newRig(t)
outside := t.TempDir()
if err := os.WriteFile(filepath.Join(outside, "secret.jpg"), jpegBytes(t, 300, 300, color.RGBA{255, 0, 0, 255}), 0o644); err != nil {
t.Fatal(err)
}
// A cover.jpg that is a link to a picture outside the library...
if err := os.Symlink(filepath.Join(outside, "secret.jpg"), filepath.Join(r.root, "Album One", "cover.jpg")); err != nil {
t.Skip("symlinks not supported:", err)
}
// ...and a folder that is a link to a directory outside it.
if err := os.Symlink(outside, filepath.Join(r.root, "linked")); err != nil {
t.Fatal(err)
}
r.src.files[2] = []File{{Root: r.root, RelPath: "linked/x.mp3"}}
for _, id := range []int64{1, 2} {
if _, err := r.svc.AlbumCover(context.Background(), id, Large); !errors.Is(err, ErrNoCover) {
t.Errorf("album %d: err = %v, want no cover: the picture is outside the library", id, err)
}
}
// An embedded picture is not read from a file that leaves the library either.
r.embedded[filepath.Join(r.root, "linked/x.mp3")] = jpegBytes(t, 300, 300, color.White)
r.store.drop("album", 2)
if _, err := r.svc.AlbumCover(context.Background(), 2, Large); !errors.Is(err, ErrNoCover) {
t.Errorf("embedded picture of a file outside the library: err = %v", err)
}
}
func TestOnlyRegularPictureFilesAreRead(t *testing.T) {
r := newRig(t)
if err := os.Mkdir(filepath.Join(r.root, "Album One", "cover.jpg"), 0o755); err != nil { // a folder called cover.jpg
t.Fatal(err)
}
big := make([]byte, MaxSourceBytes+10)
copy(big, jpegBytes(t, 300, 300, color.White))
r.write(t, "Album One/folder.jpg", big) // too big
if _, err := r.svc.AlbumCover(context.Background(), 1, Large); !errors.Is(err, ErrNoCover) {
t.Errorf("err = %v, want no cover", err)
}
}
func TestManyAsksAtOnceBuildItOnce(t *testing.T) {
r := newRig(t)
r.embedded[filepath.Join(r.root, "Album One/01.mp3")] = jpegBytes(t, 1200, 1200, color.White)
var wg sync.WaitGroup
errs := make(chan error, 20)
for i := 0; i < 20; i++ {
wg.Add(1)
go func() {
defer wg.Done()
c, err := r.svc.AlbumCover(context.Background(), 1, Large)
if err == nil {
c.File.Close()
}
errs <- err
}()
}
wg.Wait()
close(errs)
for err := range errs {
if err != nil {
t.Errorf("err = %v", err)
}
}
if n := atomic.LoadInt32(&r.reads); n != 1 {
t.Errorf("the embedded picture was read %d times, want 1", n)
}
}
func TestAListenerWhoLeavesDoesNotStopTheBuild(t *testing.T) {
r := newRig(t)
r.write(t, "Album One/cover.jpg", jpegBytes(t, 300, 300, color.White))
ctx, cancel := context.WithCancel(context.Background())
cancel()
if _, err := r.svc.AlbumCover(ctx, 1, Large); !errors.Is(err, context.Canceled) {
t.Errorf("err = %v", err)
}
}
func TestStoreErrorsPass(t *testing.T) {
r := newRig(t)
boom := errors.New("boom")
r.store.err = boom
if _, err := r.svc.AlbumCover(context.Background(), 1, Large); !errors.Is(err, boom) {
t.Errorf("err = %v", err)
}
}
func TestTheETagChangesWithThePictureNotTheSize(t *testing.T) {
r := newRig(t)
r.write(t, "Album One/cover.jpg", jpegBytes(t, 600, 600, color.RGBA{200, 0, 0, 255}))
c1, err := r.svc.AlbumCover(context.Background(), 1, Large)
if err != nil {
t.Fatal(err)
}
c1.File.Close()
c2, _ := r.svc.AlbumCover(context.Background(), 1, Small)
c2.File.Close()
if c1.ETag == "" || c1.ETag == c2.ETag {
t.Errorf("etags: large %q, small %q: they must be set and differ by size", c1.ETag, c2.ETag)
}
r.store.drop("album", 1)
r.write(t, "Album One/cover.jpg", jpegBytes(t, 600, 600, color.RGBA{0, 0, 200, 255}))
c3, _ := r.svc.AlbumCover(context.Background(), 1, Large)
c3.File.Close()
if c3.ETag == c1.ETag {
t.Error("the picture changed but the etag did not")
}
}

View file

@ -0,0 +1,99 @@
package storage
import (
"context"
"database/sql"
"errors"
"fmt"
"time"
"github.com/uptrace/bun"
"codeberg.org/squidd/sonda/internal/images"
)
// maxAlbumFiles bounds how many files of an album are offered for a cover search.
const maxAlbumFiles = 20
// ImageRepo stores what is known about pictures. It implements images.Store and
// images.Source.
type ImageRepo struct{ db *bun.DB }
// NewImageRepo builds an ImageRepo.
func NewImageRepo(db *bun.DB) *ImageRepo { return &ImageRepo{db: db} }
// Get returns the record of one owner's picture, with ok=false when there is none.
func (r *ImageRepo) Get(ctx context.Context, kind string, ownerID int64) (images.Record, bool, error) {
var (
rec images.Record
at int64
)
err := r.db.QueryRowContext(ctx,
"SELECT status, source, etag, width, height, checked_at FROM images WHERE kind = ? AND owner_id = ?",
kind, ownerID).Scan(&rec.Status, &rec.Source, &rec.ETag, &rec.Width, &rec.Height, &at)
if errors.Is(err, sql.ErrNoRows) {
return images.Record{}, false, nil
}
if err != nil {
return images.Record{}, false, fmt.Errorf("read picture record: %w", err)
}
rec.CheckedAt = time.Unix(at, 0)
return rec, true, nil
}
// Save stores the record, replacing the one there was.
func (r *ImageRepo) Save(ctx context.Context, kind string, ownerID int64, rec images.Record) error {
_, err := r.db.ExecContext(ctx,
`INSERT INTO images (kind, owner_id, status, source, etag, width, height, checked_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT (kind, owner_id) DO UPDATE SET status = excluded.status, source = excluded.source, etag = excluded.etag,
width = excluded.width, height = excluded.height, checked_at = excluded.checked_at`,
kind, ownerID, rec.Status, rec.Source, rec.ETag, rec.Width, rec.Height, rec.CheckedAt.Unix())
if err != nil {
return fmt.Errorf("save picture record: %w", err)
}
return nil
}
// AlbumFiles returns the files of the album that can be played, in play order, as
// the scanner stored them. An album that does not exist, or has nothing to play, is
// images.ErrAlbumNotFound.
func (r *ImageRepo) AlbumFiles(ctx context.Context, albumID int64) ([]images.File, error) {
rows, err := r.db.QueryContext(ctx,
`SELECT l.path, f.rel_path
FROM tracks t JOIN audio_files f ON f.track_id = t.id JOIN libraries l ON l.id = f.library_id
WHERE t.album_id = ? AND f.missing_at IS NULL
ORDER BY t.disc_no, t.track_no, t.id, f.id LIMIT ?`, albumID, maxAlbumFiles)
if err != nil {
return nil, fmt.Errorf("list album files: %w", err)
}
defer rows.Close()
var out []images.File
for rows.Next() {
var f images.File
if err := rows.Scan(&f.Root, &f.RelPath); err != nil {
return nil, err
}
out = append(out, f)
}
if err := rows.Err(); err != nil {
return nil, err
}
if len(out) == 0 {
return nil, images.ErrAlbumNotFound
}
return out, nil
}
// clearAlbumCovers forgets the pictures of the albums, so that they are looked for
// again: a file of the album was added or changed. Albums with id 0 are ignored.
func clearAlbumCovers(ctx context.Context, tx bun.Tx, albumIDs ...int64) error {
for _, id := range albumIDs {
if id == 0 {
continue
}
if _, err := tx.ExecContext(ctx, "DELETE FROM images WHERE kind = 'album' AND owner_id = ?", id); err != nil {
return fmt.Errorf("forget the cover: %w", err)
}
}
return nil
}

View file

@ -0,0 +1,166 @@
package storage
import (
"context"
"errors"
"testing"
"time"
"github.com/uptrace/bun"
"codeberg.org/squidd/sonda/internal/images"
)
func TestImageRepoSaveAndGet(t *testing.T) {
forEachBackend(t, func(t *testing.T, db *bun.DB, _ string) {
ctx := context.Background()
repo := NewImageRepo(db)
if _, ok, err := repo.Get(ctx, "album", 5); ok || err != nil {
t.Fatalf("empty: ok=%v err=%v", ok, err)
}
at := time.Unix(1_700_000_000, 0)
rec := images.Record{Status: images.StatusOK, Source: images.SourceFolder, ETag: "abc", Width: 800, Height: 600, CheckedAt: at}
if err := repo.Save(ctx, "album", 5, rec); err != nil {
t.Fatal(err)
}
got, ok, err := repo.Get(ctx, "album", 5)
if err != nil || !ok || got.Status != rec.Status || got.Source != rec.Source || got.ETag != "abc" || got.Width != 800 || got.Height != 600 || !got.CheckedAt.Equal(at) {
t.Errorf("got %+v ok=%v err=%v, want %+v", got, ok, err, rec)
}
// Saved again, it replaces.
none := images.Record{Status: images.StatusNone, CheckedAt: at.Add(time.Hour)}
if err := repo.Save(ctx, "album", 5, none); err != nil {
t.Fatal(err)
}
if got, _, _ = repo.Get(ctx, "album", 5); got.Status != images.StatusNone || got.Source != "" || got.ETag != "" {
t.Errorf("after replacing: %+v", got)
}
// Another owner and another kind are separate.
if _, ok, _ := repo.Get(ctx, "album", 6); ok {
t.Error("another album has a record")
}
if _, ok, _ := repo.Get(ctx, "artist", 5); ok {
t.Error("an artist has the record of an album")
}
})
}
func TestImageRepoAlbumFiles(t *testing.T) {
forEachBackend(t, func(t *testing.T, db *bun.DB, _ string) {
seedCatalog(t, db)
ctx := context.Background()
repo := NewImageRepo(db)
var kind, ghost, hidden int64
if err := db.QueryRow("SELECT id FROM albums WHERE title = 'Kind of Blue'").Scan(&kind); err != nil {
t.Fatal(err)
}
if err := db.QueryRow("SELECT id FROM albums WHERE title = 'OK Computer'").Scan(&ghost); err != nil {
t.Fatal(err)
}
if err := db.QueryRow("SELECT id FROM albums WHERE title = 'Hidden Album'").Scan(&hidden); err != nil {
t.Fatal(err)
}
files, err := repo.AlbumFiles(ctx, kind)
if err != nil || len(files) != 3 {
t.Fatalf("files = %+v err=%v", files, err)
}
// In play order: track 1, 2, 3 (they were saved as 3, 1, 2).
want := []string{"m/1.mp3", "m/2.mp3", "m/3.mp3"}
for i, f := range files {
if f.RelPath != want[i] || f.Root != "/lib/jazz" {
t.Errorf("file %d = %+v, want %s in /lib/jazz", i, f, want[i])
}
}
// The file that is gone (Ghost) is not offered.
files, err = repo.AlbumFiles(ctx, ghost)
if err != nil {
t.Fatal(err)
}
for _, f := range files {
if f.RelPath == "r/4.flac" {
t.Error("a missing file was offered")
}
}
// An album whose files are all gone, and one that does not exist.
if _, err := repo.AlbumFiles(ctx, hidden); !errors.Is(err, images.ErrAlbumNotFound) {
t.Errorf("album with nothing to play: err = %v", err)
}
if _, err := repo.AlbumFiles(ctx, 99999); !errors.Is(err, images.ErrAlbumNotFound) {
t.Errorf("unknown album: err = %v", err)
}
})
}
func TestSaveFileClearsTheCoversThatMayHaveChanged(t *testing.T) {
forEachBackend(t, func(t *testing.T, db *bun.DB, _ string) {
ctx := context.Background()
lib := NewLibraryRepo(db)
l, err := lib.CreateLibrary(ctx, "m", "/lib/m")
if err != nil {
t.Fatal(err)
}
images := NewImageRepo(db)
save := func(rel, album string, existing int64) {
t.Helper()
if _, err := lib.SaveFile(ctx, l.ID, pf(rel, rel, album, "Band", 1, 1, "flac", "Band"), existing); err != nil {
t.Fatal(err)
}
}
albumID := func(title string) int64 {
var id int64
if err := db.QueryRow("SELECT id FROM albums WHERE title = ?", title).Scan(&id); err != nil {
t.Fatal(err)
}
return id
}
fileID := func(rel string) int64 {
var id int64
if err := db.QueryRow("SELECT id FROM audio_files WHERE rel_path = ?", rel).Scan(&id); err != nil {
t.Fatal(err)
}
return id
}
mark := func(ids ...int64) {
for _, id := range ids {
if err := images.Save(ctx, "album", id, imagesRec()); err != nil {
t.Fatal(err)
}
}
}
has := func(id int64) bool {
_, ok, err := images.Get(ctx, "album", id)
if err != nil {
t.Fatal(err)
}
return ok
}
save("a/1.flac", "First", 0)
save("b/1.flac", "Second", 0)
first, second := albumID("First"), albumID("Second")
// A new file in the first album clears only that album.
mark(first, second)
save("a/2.flac", "First", 0)
if has(first) || !has(second) {
t.Errorf("after a new file: first=%v second=%v, want the first cleared only", has(first), has(second))
}
// A changed file clears its album.
mark(first)
save("a/1.flac", "First", fileID("a/1.flac"))
if has(first) || !has(second) {
t.Errorf("after a changed file: first=%v second=%v", has(first), has(second))
}
// A file retagged into another album clears both: the old one lost a track.
mark(first, second)
save("a/1.flac", "Second", fileID("a/1.flac"))
if has(first) || has(second) {
t.Errorf("after a move: first=%v second=%v, want both cleared", has(first), has(second))
}
})
}
func imagesRec() images.Record {
return images.Record{Status: images.StatusOK, Source: images.SourceFolder, ETag: "x", CheckedAt: time.Unix(1_700_000_000, 0)}
}

View file

@ -156,6 +156,9 @@ func (r *LibraryRepo) SaveFile(ctx context.Context, libraryID int64, f library.P
return fmt.Errorf("insert audio file: %w", err)
}
added = true
if err := clearAlbumCovers(ctx, tx, albumID); err != nil {
return err
}
if err := replaceGenres(ctx, tx, track.ID, f.Genres); err != nil {
return err
}
@ -167,6 +170,14 @@ func (r *LibraryRepo) SaveFile(ctx context.Context, libraryID int64, f library.P
return fmt.Errorf("load audio file %d: %w", existingID, err)
}
track.ID = file.TrackID
// The album it leaves, if it moves, and the one it is in now: their covers may change.
var oldAlbum sql.NullInt64
if err := tx.QueryRowContext(ctx, "SELECT album_id FROM tracks WHERE id = ?", track.ID).Scan(&oldAlbum); err != nil {
return fmt.Errorf("read the track's album: %w", err)
}
if err := clearAlbumCovers(ctx, tx, albumID, oldAlbum.Int64); err != nil {
return err
}
if _, err := tx.NewUpdate().Model(track).
Column("title", "title_key", "duration_ms", "track_no", "disc_no", "album_id").
WherePK().Exec(ctx); err != nil {

Some files were not shown because too many files have changed in this diff Show more