From 89826fcd070c3ccbbfa8edcc91a96737bb2e67f0 Mon Sep 17 00:00:00 2001 From: tgiachi Date: Fri, 13 Nov 2026 15:04:07 +0100 Subject: [PATCH] feat(federation): shared libraries and the signed read API for friends A friend's server can now read what an admin chose to share. - A library has a shared flag, off by default (migration 0024), set with PATCH /libraries/{id}. Changing it never waits for a scan. - Requests between friends are signed with each server's Ed25519 key: the method, the path and query from /api/v1/peer on, a date, a nonce and the hash of the empty body. A date more than two minutes off, a nonce used before, an unknown or removed key and a bad signature all get the same 401. A friend still waiting for approval gets 403, and while federation is off the routes are missing. - Routes under /api/v1/peer: info, artists, albums, tracks, covers and streaming with Range. The catalogue, the counts and the format come from the shared libraries only, and the file served is the one in a shared library. No path, library id, listen, taste, user or provider data is part of an answer. - A friend never makes this server build a cover, read an audio file or ask a provider: only covers already built from the audio files are served. - Per-friend limits, set by the admin: parallel streams (default 3) and KiB a second (default 4096, 0 for none), shared by all the streams of a friend. - A log of the last requests friends made (route, status, bytes; never the query), kept for 7 days, in GET /federation/requests. - POST /federation/friends/{id}/check lets the side that pasted an invite find out that the other admin approved; a signed call from a friend this server is waiting for makes it active too. - Admin: a Shared with friends switch on each library, which asks first, and limits, recent requests and a Check button in the Federation section, in English and Italian. - OpenAPI paths and schemas, contract test, two real servers checked end to end with tagged audio files. --- README.md | 4 +- cmd/sondad/main.go | 3 + internal/api/http/api_test.go | 4 + internal/api/http/contract_test.go | 90 ++- internal/api/http/federation_test.go | 2 +- internal/api/http/handlers_federation.go | 74 +- internal/api/http/handlers_libraries.go | 18 +- internal/api/http/handlers_peer.go | 194 +++++ internal/api/http/libraries_test.go | 3 + internal/api/http/peer_test.go | 526 +++++++++++++ internal/api/http/router.go | 9 + internal/api/openapi/openapi.yaml | 398 +++++++++- internal/federation/client.go | 50 ++ internal/federation/client_test.go | 80 ++ internal/federation/federation.go | 39 +- internal/federation/limits.go | 123 +++ internal/federation/peer_test.go | 400 ++++++++++ internal/federation/service.go | 197 ++++- internal/federation/service_test.go | 53 ++ internal/federation/sign.go | 50 ++ internal/images/service.go | 33 + internal/images/service_test.go | 49 ++ internal/library/libraries.go | 8 + internal/library/libraries_test.go | 32 + internal/storage/catalog.go | 40 +- internal/storage/federation.go | 87 ++- internal/storage/federation_test.go | 71 ++ internal/storage/library.go | 38 +- internal/storage/sharing_test.go | 160 ++++ internal/storage/storage_test.go | 6 +- .../postgres/0024_federation_sharing.sql | 17 + migrations/sqlite/0024_federation_sharing.sql | 18 + web/src/api/adminFederation.ts | 23 + web/src/api/libraries.ts | 4 +- web/src/api/schema.d.ts | 734 +++++++++++++++++- web/src/i18n/locales/en.json | 30 +- web/src/i18n/locales/it.json | 30 +- web/src/pages/admin/FederationFriends.tsx | 10 +- web/src/pages/admin/FederationLimits.tsx | 60 ++ web/src/pages/admin/FederationPage.test.tsx | 106 ++- web/src/pages/admin/FederationPage.tsx | 4 + web/src/pages/admin/FederationRequests.tsx | 61 ++ web/src/pages/admin/LibraryList.test.tsx | 65 +- web/src/pages/admin/LibraryList.tsx | 27 +- 44 files changed, 3970 insertions(+), 60 deletions(-) create mode 100644 internal/api/http/handlers_peer.go create mode 100644 internal/api/http/peer_test.go create mode 100644 internal/federation/limits.go create mode 100644 internal/federation/peer_test.go create mode 100644 internal/federation/sign.go create mode 100644 internal/storage/sharing_test.go create mode 100644 migrations/postgres/0024_federation_sharing.sql create mode 100644 migrations/sqlite/0024_federation_sharing.sql create mode 100644 web/src/pages/admin/FederationLimits.tsx create mode 100644 web/src/pages/admin/FederationRequests.tsx diff --git a/README.md b/README.md index 845483a..24ccf50 100644 --- a/README.md +++ b/README.md @@ -28,11 +28,11 @@ It is a single executable, one database (SQLite by default, PostgreSQL if you pr - **Your own settings.** `GET` and `PATCH /me/settings` hold choices that belong to you, not to the server. Everyone starts private: `discover_from_listening` is off, so this setting never sends your listening to a provider. When you turn it on, Discover may start from the artists you listen to most, and their names are sent to Last.fm (only if an admin switched it on). What Last.fm answers is kept in the shared database like any other lookup, so another local user can see, from `retrieved_at` of `GET /artists/{id}/similar`, that someone looked up an artist, but not who. - **Tags.** `GET /artists/{id}/tags` shows what Last.fm and MusicBrainz say kind of music an artist plays, each with its own weights, and the genres of the artist's own files, read offline. See [External providers](#external-providers). - **Similar artists.** With Last.fm switched on, `GET /artists/{id}/similar` lists artists similar to an artist, and says which of them you already have. See [External providers](#external-providers). Without it, Sonda makes no outbound network requests at all. -- **Friends (federation, first part).** Two Sonda servers can become friends. `PUT /federation` (admin) turns federation on, which is off by default, and gives this server a name and a public address (`https`; plain `http` only on a network of your own); the server then makes its own Ed25519 key, kept in the database and never returned. `POST /federation/invites` makes an invite that works once and expires in 24 hours (only a hash of its code is kept). The other admin uses it with `POST /federation/friends`: that server is asked to accept this one, the key it answers with must be the one named in the invite, and the friend waits as `outgoing` until the first admin approves it with `POST /federation/friends/{id}/approve`. Either side can remove a friend with `DELETE /federation/friends/{id}`. Nothing is sent until an admin makes or uses an invite, and then only this server's name, address and public key. Sharing libraries and playing a friend's music are not there yet: for now a friend is only recorded. The admin area has a Federation section for all of it. +- **Friends (federation).** Two Sonda servers can become friends, and the listeners of one can read what the other chose to share. `PUT /federation` (admin) turns federation on, which is off by default, and gives this server a name and a public address (`https`; plain `http` only on a network of your own); the server then makes its own Ed25519 key, kept in the database and never returned. `POST /federation/invites` makes an invite that works once and expires in 24 hours (only a hash of its code is kept). The other admin uses it with `POST /federation/friends`: that server is asked to accept this one, the key it answers with must be the one named in the invite, and the friend waits as `outgoing` until the first admin approves it with `POST /federation/friends/{id}/approve` (`POST /federation/friends/{id}/check` lets the waiting side find out). Either side can remove a friend with `DELETE /federation/friends/{id}`. Nothing is shared until an admin turns on `shared` for a library (`PATCH /libraries/{id}`). A friend's server then reads, with requests signed by its key (the method, the path and query, a date, a nonce and the hash of the body), the routes under `/peer`: the artists, albums and tracks that have a file in a shared library, their covers once they are built, and the audio itself with `Range`. A friend never sees a path, a library id, a listen, a taste or a user, never makes this server build a cover or ask a provider, and has a few streams at a time and a number of KiB a second to share (`max_streams`, `max_kib_per_sec`); the admin sees the last requests it made (`GET /federation/requests`, the route and the size, never what was searched for). Sharing a library lets friends play it and, in practice, copy its files. Browsing a friend's music from the web app is not there yet. The admin area has a Federation section for all of it. ## What is not there yet -Playing a friend's libraries (federation: shared libraries, the read API and streaming), radio queues, OpenSubsonic compatibility, transcoding, and the ListenBrainz integration. The web client is built into the binary (see [Web app](#web-app)), and its admin area covers everything the API can do for an admin: libraries and scanning, users, the external providers and the bulk jobs. +Browsing and playing a friend's libraries from the web app (federation), radio queues, OpenSubsonic compatibility, transcoding, and the ListenBrainz integration. The web client is built into the binary (see [Web app](#web-app)), and its admin area covers everything the API can do for an admin: libraries and scanning, users, the external providers and the bulk jobs. ## Requirements diff --git a/cmd/sondad/main.go b/cmd/sondad/main.go index ea61dbf..f4199ea 100644 --- a/cmd/sondad/main.go +++ b/cmd/sondad/main.go @@ -317,6 +317,9 @@ func run() error { Settings: userSettings, Streams: streaming.NewService(libraryRepo), Federation: federation.NewService(storage.NewFederationRepo(db), federation.NewClient()), + PeerCatalog: catalog.NewService(storage.NewSharedCatalogRepo(db)), + PeerStreams: streaming.NewService(libraryRepo.Shared()), + PeerCovers: images.StoredOnly{Service: pictures}, Web: webui.FS(), }) diff --git a/internal/api/http/api_test.go b/internal/api/http/api_test.go index 4527ca6..8f459af 100644 --- a/internal/api/http/api_test.go +++ b/internal/api/http/api_test.go @@ -109,6 +109,9 @@ func newTestEnvDB(t *testing.T, mutate func(*Deps, *bun.DB)) (http.Handler, *bun Streams: streaming.NewService(noFiles{}), Federation: federation.NewService(storage.NewFederationRepo(db), federation.NewClient()), } + libRepo := storage.NewLibraryRepo(db) + deps.PeerCatalog = catalog.NewService(storage.NewSharedCatalogRepo(db)) + deps.PeerStreams = streaming.NewService(libRepo.Shared()) feedbackSvc := feedback.NewService(storage.NewFeedbackRepo(db)) deps.Feedback = feedbackSvc deps.Playlists = playlists.NewService(storage.NewPlaylistRepo(db)) @@ -120,6 +123,7 @@ func newTestEnvDB(t *testing.T, mutate func(*Deps, *bun.DB)) (http.Handler, *bun imageRepo := storage.NewImageRepo(db) imageSvc := images.NewService(filepath.Join(t.TempDir(), "data"), imageRepo, imageRepo) deps.Images, deps.ArtistImages = imageSvc, imageSvc + deps.PeerCovers = images.StoredOnly{Service: imageSvc} deps.CoversJob = jobs.Runner{Ctx: context.Background(), Job: images.NewCoversJob(imageSvc, imageRepo)} deps.ArtistImagesJob = jobs.Runner{Ctx: context.Background(), Job: images.NewArtistImagesJob(imageSvc, imageRepo)} mutate(&deps, db) diff --git a/internal/api/http/contract_test.go b/internal/api/http/contract_test.go index 6d4981b..80f7a5f 100644 --- a/internal/api/http/contract_test.go +++ b/internal/api/http/contract_test.go @@ -3,6 +3,8 @@ package httpapi import ( "bytes" "context" + "crypto/ed25519" + "crypto/rand" "encoding/base64" "encoding/json" "image" @@ -29,6 +31,7 @@ import ( "codeberg.org/squidd/sonda/internal/catalog" "codeberg.org/squidd/sonda/internal/discovery" "codeberg.org/squidd/sonda/internal/duplicates" + "codeberg.org/squidd/sonda/internal/federation" "codeberg.org/squidd/sonda/internal/images" "codeberg.org/squidd/sonda/internal/jobs" "codeberg.org/squidd/sonda/internal/library" @@ -157,6 +160,7 @@ func newContract(t *testing.T) *contract { pictures.SetArtistSource(&fakeArtistPictures{found: images.Found{Data: testJPEG(t, 800, 600, color.White), Credit: images.Credit{Author: "Someone", License: "CC BY 4.0", LicenseURL: "https://creativecommons.org/licenses/by/4.0", PageURL: "https://commons.wikimedia.org/wiki/File:X.jpg"}}}) d.Images, d.ArtistImages = pictures, pictures + d.PeerCovers = images.StoredOnly{Service: pictures} d.LidarrArtists = lidarr.NewService(storage.NewDiscoveryRepo(db), prov, fakeLidarrRemote{}) }) @@ -241,8 +245,8 @@ func (c *contract) validate(req *http.Request, payload []byte, rec *httptest.Res // The library skips HEAD and redirects, and cannot read audio or YAML. // Those are checked by hand: the status above, and the content type here. skipBody := req.Method == http.MethodHead || route.Path == "/openapi.yaml" || - (rec.Code >= 300 && rec.Code < 400) || (route.Path == "/tracks/{id}/stream" && rec.Code/100 == 2) - if skipBody && documented != nil && documented.Value != nil && rec.Code/100 == 2 && req.Method != http.MethodHead && route.Path == "/tracks/{id}/stream" { + (rec.Code >= 300 && rec.Code < 400) || ((route.Path == "/tracks/{id}/stream" || route.Path == "/peer/tracks/{id}/stream") && rec.Code/100 == 2) + if skipBody && documented != nil && documented.Value != nil && rec.Code/100 == 2 && req.Method != http.MethodHead && (route.Path == "/tracks/{id}/stream" || route.Path == "/peer/tracks/{id}/stream") { mediaType, _, err := mime.ParseMediaType(rec.Header().Get("Content-Type")) if err != nil || documented.Value.Content.Get(mediaType) == nil { c.t.Errorf("%s -> %d: Content-Type %q is not one the spec lists", where, rec.Code, rec.Header().Get("Content-Type")) @@ -828,7 +832,7 @@ func TestResponsesMatchTheSpec(t *testing.T) { c.mixes(t, token, libID) c.smartPlaylists(t, token) c.names(t, token) - c.federation(t, token) + c.federation(t, token, libID, albumID) // ---- every operation in the spec was exercised ---- var missing []string @@ -1126,7 +1130,7 @@ func (c *contract) names(t *testing.T, token string) { c.expect(c.call("POST", "/smart-playlists/999999/suggest-name", token, nil), 404, "a name for an unknown smart playlist", "smart_playlist_not_found") } -func (c *contract) federation(t *testing.T, token string) { +func (c *contract) federation(t *testing.T, token, libID, albumID string) { t.Helper() for _, r := range []struct{ method, path string }{ {"GET", "/federation"}, {"PUT", "/federation"}, {"POST", "/federation/invites"}, {"POST", "/federation/friends"}, @@ -1170,6 +1174,8 @@ func (c *contract) federation(t *testing.T, token string) { c.expect(c.call("DELETE", "/federation/friends/"+id, token, nil), 204, "end the friendship") c.expect(c.call("DELETE", "/federation/friends/"+id, token, nil), 404, "end it twice", "friend_not_found") + c.peer(t, token, libID, albumID) + // This server uses an invite of another one, which is a real server of its own. other := newFriendServer(t, true) other.setUp(t, "Anna", other.ts.URL) @@ -1180,6 +1186,8 @@ func (c *contract) federation(t *testing.T, token string) { t.Fatalf("friend = %v", friend) } c.expect(c.call("POST", "/federation/friends", token, map[string]any{"invite": theirs.Body["invite"]}), 409, "use the invite of a friend", "already_friends") + c.expect(c.call("POST", "/federation/friends/"+idOf(friend)+"/check", token, nil), 409, "ask a friend that has not approved", "not_approved") + c.expect(c.call("POST", "/federation/friends/999999/check", token, nil), 404, "check a missing friend", "friend_not_found") c.expect(c.call("DELETE", "/federation/friends/"+idOf(friend), token, nil), 204, "withdraw the request") again := other.call(t, "POST", "/federation/invites", nil) other.ts.Close() @@ -1201,3 +1209,77 @@ func inviteCode(t *testing.T, text string) string { } return p.C } + +// peer plays the part of a friend: it holds a key of its own, is paired and approved, and reads what is shared. +func (c *contract) peer(t *testing.T, token, libID, albumID string) { + t.Helper() + c.expect(c.call("PATCH", "/libraries/"+libID, token, map[string]any{"shared": true}), 200, "share the library") + + inv := c.expect(c.call("POST", "/federation/invites", token, nil), 201, "an invite for the friend") + pub, priv, _ := ed25519.GenerateKey(rand.Reader) + c.expect(c.call("POST", "/peer/pair", "", map[string]any{ + "code": inviteCode(t, inv["invite"].(string)), "name": "Reader", "address": "https://reader.example.org", + "public_key": base64.RawURLEncoding.EncodeToString(pub), + }), 200, "the friend pairs") + var friendID string + for _, f := range c.expect(c.call("GET", "/federation", token, nil), 200, "federation")["friends"].([]any) { + if f.(map[string]any)["name"] == "Reader" { + friendID = idOf(f.(map[string]any)) + } + } + c.expect(c.call("POST", "/federation/friends/"+friendID+"/approve", token, nil), 200, "approve the friend") + + var n int + signed := func(path string, extra ...string) *httptest.ResponseRecorder { + n++ + h := federation.Sign(priv, "GET", "/api/v1"+path, time.Now(), "contract-nonce-"+strconv.Itoa(n)+"-0123456789") + headers := append([]string{ + federation.HeaderKeyID, federation.KeyID(pub), federation.HeaderDate, h.Date, + federation.HeaderNonce, h.Nonce, federation.HeaderSignature, h.Signature, + }, extra...) + return c.call("GET", path, "", nil, headers...) + } + + c.expect(c.call("GET", "/peer/info", "", nil), 401, "a friend's route without a signature", "unauthorized") + info := c.expect(signed("/peer/info"), 200, "who this server is") + if info["shared_libraries"] != float64(1) { + t.Fatalf("info = %v", info) + } + artists := c.expect(signed("/peer/artists?limit=5"), 200, "the artists of the shared libraries") + if artists["total"].(float64) == 0 { + t.Fatalf("artists = %v", artists) + } + c.expect(signed("/peer/artists/"+idOf(artists["items"].([]any)[0].(map[string]any))), 200, "one artist") + c.expect(signed("/peer/artists/999999"), 404, "an artist that is not there", "artist_not_found") + c.expect(signed("/peer/albums?q=a&limit=5"), 200, "the albums") + c.expect(signed("/peer/albums/"+albumID), 200, "one album") + c.expect(signed("/peer/albums/999999"), 404, "an album that is not there", "album_not_found") + tracks := c.expect(signed("/peer/tracks?limit=5"), 200, "the tracks") + trackID := idOf(tracks["items"].([]any)[0].(map[string]any)) + c.expect(signed("/peer/tracks/"+trackID), 200, "one track") + c.expect(signed("/peer/tracks/999999"), 404, "a track that is not there", "track_not_found") + + if rec := signed("/peer/tracks/" + trackID + "/stream"); rec.Code != 200 { + t.Fatalf("stream: %d %s", rec.Code, rec.Body.String()) + } + if rec := signed("/peer/tracks/"+trackID+"/stream", "Range", "bytes=0-9"); rec.Code != 206 { + t.Fatalf("stream range: %d", rec.Code) + } + c.expect(signed("/peer/tracks/999999/stream"), 404, "the stream of a track that is not there", "track_not_found") + // A friend gets a cover once a listener's request has built it, never before. + c.expect(signed("/peer/albums/"+albumID+"/cover?size=small"), 404, "a cover that was not asked for yet", "cover_not_found") + if rec := c.call("GET", "/albums/"+albumID+"/cover", token, nil); rec.Code != 200 { + t.Fatalf("the listener's cover: %d %s", rec.Code, rec.Body.String()) + } + if rec := signed("/peer/albums/" + albumID + "/cover"); rec.Code != 200 || rec.Header().Get("Content-Type") != "image/jpeg" { + t.Fatalf("cover: %d %q", rec.Code, rec.Header().Get("Content-Type")) + } + c.expect(signed("/peer/albums/999999/cover"), 404, "the cover of an album that is not there", "album_not_found") + + reqs := c.expect(c.call("GET", "/federation/requests", token, nil), 200, "what friends asked")["requests"].([]any) + if len(reqs) == 0 { + t.Fatal("no request was logged") + } + c.expect(c.call("DELETE", "/federation/friends/"+friendID, token, nil), 204, "remove the reader") + c.expect(c.call("PATCH", "/libraries/"+libID, token, map[string]any{"shared": false}), 200, "stop sharing the library") +} diff --git a/internal/api/http/federation_test.go b/internal/api/http/federation_test.go index da64aeb..3238e26 100644 --- a/internal/api/http/federation_test.go +++ b/internal/api/http/federation_test.go @@ -83,7 +83,7 @@ func TestFederationStartsOffAndSaysWhatItSends(t *testing.T) { if r.Code != http.StatusOK { t.Fatalf("get: %d %v", r.Code, r.Body) } - sameKeys(t, "federation", r.Body, "enabled", "name", "address", "key_id", "sends", "friends") + sameKeys(t, "federation", r.Body, "enabled", "name", "address", "key_id", "sends", "friends", "max_streams", "max_kib_per_sec") if r.Body["enabled"] != false || r.Body["key_id"] != "" || r.Body["address"] != "" { t.Errorf("federation must start off with no key and no address: %v", r.Body) } diff --git a/internal/api/http/handlers_federation.go b/internal/api/http/handlers_federation.go index a500dec..66a5e6d 100644 --- a/internal/api/http/handlers_federation.go +++ b/internal/api/http/handlers_federation.go @@ -20,6 +20,13 @@ type federationService interface { Pair(ctx context.Context, req federation.PairRequest) (federation.PairReply, error) Approve(ctx context.Context, id int64) (federation.Friend, error) Remove(ctx context.Context, id int64) error + Check(ctx context.Context, id int64) (federation.Friend, error) + Requests(ctx context.Context) ([]federation.FriendRequest, error) + // What the routes friends use need. + Authenticate(ctx context.Context, in federation.Signed) (federation.Friend, error) + StartStream(ctx context.Context, friendID int64) (*federation.Stream, error) + Pace(ctx context.Context, friendID int64) (*federation.Stream, error) + Record(ctx context.Context, r federation.FriendRequest) } // federationHandlers are the admin routes that set up federation and its friends, and the one public route @@ -48,18 +55,20 @@ func toFriendJSON(f federation.Friend) friendJSON { } type federationJSON struct { - Enabled bool `json:"enabled"` - Name string `json:"name"` - Address string `json:"address"` - KeyID string `json:"key_id"` - Sends string `json:"sends"` - Friends []friendJSON `json:"friends"` + Enabled bool `json:"enabled"` + Name string `json:"name"` + Address string `json:"address"` + KeyID string `json:"key_id"` + Sends string `json:"sends"` + MaxStreams int `json:"max_streams"` + MaxKiBPerSec int `json:"max_kib_per_sec"` + Friends []friendJSON `json:"friends"` } func toFederationJSON(st federation.Status) federationJSON { out := federationJSON{ Enabled: st.Enabled, Name: st.Name, Address: st.Address, KeyID: st.KeyID, Sends: st.Sends, - Friends: make([]friendJSON, len(st.Friends)), + MaxStreams: st.MaxStreams, MaxKiBPerSec: st.MaxKiBPerSec, Friends: make([]friendJSON, len(st.Friends)), } for i, f := range st.Friends { out.Friends[i] = toFriendJSON(f) @@ -92,6 +101,10 @@ func writeFederationError(c *gin.Context, err error) { writeError(c, http.StatusNotFound, "friend_not_found", "no such friend") case errors.Is(err, federation.ErrWrongState): writeError(c, http.StatusConflict, "not_waiting", err.Error()) + case errors.Is(err, federation.ErrNotApproved): + writeError(c, http.StatusConflict, "not_approved", "the other admin has not approved yet") + case errors.Is(err, federation.ErrNotFriend): + writeError(c, http.StatusConflict, "friend_gone", err.Error()) default: _ = c.Error(err) writeError(c, http.StatusInternalServerError, "internal", "internal error") @@ -108,9 +121,11 @@ func (h *federationHandlers) get(c *gin.Context) { } type updateFederationRequest struct { - Enabled *bool `json:"enabled"` // pointers: an absent field is left alone - Name *string `json:"name"` - Address *string `json:"address"` + Enabled *bool `json:"enabled"` // pointers: an absent field is left alone + Name *string `json:"name"` + Address *string `json:"address"` + MaxStreams *int `json:"max_streams"` + MaxKiBPerSec *int `json:"max_kib_per_sec"` } func (h *federationHandlers) update(c *gin.Context) { @@ -118,7 +133,9 @@ func (h *federationHandlers) update(c *gin.Context) { if !bindJSON(c, &req) { return } - st, err := h.svc.Update(c.Request.Context(), federation.Patch{Enabled: req.Enabled, Name: req.Name, Address: req.Address}) + st, err := h.svc.Update(c.Request.Context(), federation.Patch{ + Enabled: req.Enabled, Name: req.Name, Address: req.Address, MaxStreams: req.MaxStreams, MaxKiBPerSec: req.MaxKiBPerSec, + }) if err != nil { writeFederationError(c, err) return @@ -219,3 +236,38 @@ func (h *federationHandlers) pair(c *gin.Context) { writeError(c, http.StatusInternalServerError, "internal", "internal error") } } + +func (h *federationHandlers) check(c *gin.Context) { + id, ok := pathID(c) + if !ok { + return + } + f, err := h.svc.Check(c.Request.Context(), id) + if err != nil { + writeFederationError(c, err) + return + } + c.JSON(http.StatusOK, toFriendJSON(f)) +} + +type requestJSON struct { + At *string `json:"at"` + FriendID int64 `json:"friend_id"` + Friend string `json:"friend"` + Route string `json:"route"` + Status int `json:"status"` + Bytes int64 `json:"bytes"` +} + +func (h *federationHandlers) requests(c *gin.Context) { + list, err := h.svc.Requests(c.Request.Context()) + if err != nil { + writeFederationError(c, err) + return + } + out := make([]requestJSON, len(list)) + for i, r := range list { + out[i] = requestJSON{At: rfc3339(r.At), FriendID: r.FriendID, Friend: r.FriendName, Route: r.Route, Status: r.Status, Bytes: r.Bytes} + } + c.JSON(http.StatusOK, gin.H{"requests": out}) +} diff --git a/internal/api/http/handlers_libraries.go b/internal/api/http/handlers_libraries.go index 46f65e9..10ce05f 100644 --- a/internal/api/http/handlers_libraries.go +++ b/internal/api/http/handlers_libraries.go @@ -17,11 +17,12 @@ type libraryJSON struct { ID int64 `json:"id"` Name string `json:"name"` Path string `json:"path"` + Shared bool `json:"shared"` CreatedAt string `json:"created_at"` } func toLibraryJSON(l library.Library) libraryJSON { - return libraryJSON{ID: l.ID, Name: l.Name, Path: l.Path, CreatedAt: l.CreatedAt.UTC().Format(time.RFC3339)} + return libraryJSON{ID: l.ID, Name: l.Name, Path: l.Path, Shared: l.Shared, CreatedAt: l.CreatedAt.UTC().Format(time.RFC3339)} } func writeLibraryError(c *gin.Context, err error) { @@ -74,6 +75,8 @@ func (h *librariesHandlers) create(c *gin.Context) { type updateLibraryRequest struct { Name *string `json:"name"` // pointers: an absent field keeps the current value Path *string `json:"path"` + // Shared says whether friends (federation) may read the library. + Shared *bool `json:"shared"` } func (h *librariesHandlers) update(c *gin.Context) { @@ -85,11 +88,18 @@ func (h *librariesHandlers) update(c *gin.Context) { if !bindJSON(c, &req) { return } - if req.Name == nil && req.Path == nil { - writeError(c, http.StatusBadRequest, "invalid_input", "send name, path or both") + if req.Name == nil && req.Path == nil && req.Shared == nil { + writeError(c, http.StatusBadRequest, "invalid_input", "send name, path, shared or any of them") return } - lib, err := h.svc.Update(c.Request.Context(), id, req.Name, req.Path) + var lib library.Library + var err error + if req.Name != nil || req.Path != nil { + lib, err = h.svc.Update(c.Request.Context(), id, req.Name, req.Path) + } + if err == nil && req.Shared != nil { + lib, err = h.svc.SetShared(c.Request.Context(), id, *req.Shared) + } if err != nil { writeLibraryError(c, err) return diff --git a/internal/api/http/handlers_peer.go b/internal/api/http/handlers_peer.go new file mode 100644 index 0000000..efd18a0 --- /dev/null +++ b/internal/api/http/handlers_peer.go @@ -0,0 +1,194 @@ +package httpapi + +import ( + "context" + "errors" + "net/http" + "strings" + + "github.com/gin-gonic/gin" + + "codeberg.org/squidd/sonda/internal/catalog" + "codeberg.org/squidd/sonda/internal/federation" +) + +const ( + peerPrefix = "/api/v1/peer/" + friendKey = "friend" + peerChunkSize = 32 << 10 +) + +// registerPeer adds the routes another Sonda server reads, to v1. A friend signs every request (see +// federation.Sign); what it can read is the catalogue and the files of the libraries that are shared, and nothing +// else: the services it is given are the ones that look only at those. +func registerPeer(v1 *gin.RouterGroup, d Deps) { + peer := v1.Group("/peer", requireFriend(d.Federation)) + peer.GET("/info", (&peerInfoHandlers{fed: d.Federation, libs: d.Libraries}).info) + + cat := &catalogHandlers{svc: d.PeerCatalog} + peer.GET("/artists", cat.listArtists) + peer.GET("/artists/:id", cat.getArtist) + peer.GET("/albums", cat.listAlbums) + peer.GET("/albums/:id", cat.getAlbum) + peer.GET("/tracks", cat.listTracks) + peer.GET("/tracks/:id", cat.getTrack) + + peer.GET("/albums/:id/cover", (&peerCoverHandlers{fed: d.Federation, catalog: d.PeerCatalog, covers: &coverHandlers{svc: d.PeerCovers}}).album) + peer.GET("/tracks/:id/stream", (&peerStreamHandlers{fed: d.Federation, inner: &streamHandlers{svc: d.PeerStreams}}).stream) +} + +// requireFriend lets a request through only when a friend signed it. Everything that is not properly signed is +// the same 401, and while federation is off the routes are missing. Each request of a friend is logged afterwards: +// the route (never the query), the status and the bytes sent. +func requireFriend(fed federationService) gin.HandlerFunc { + return func(c *gin.Context) { + uri := c.Request.URL.RequestURI() + // A proxy may serve Sonda under a sub-path: what is signed starts at /api/v1/peer. + if i := strings.Index(uri, peerPrefix); i > 0 { + uri = uri[i:] + } + friend, err := fed.Authenticate(c.Request.Context(), federation.Signed{ + KeyID: c.GetHeader(federation.HeaderKeyID), + Date: c.GetHeader(federation.HeaderDate), + Nonce: c.GetHeader(federation.HeaderNonce), + Signature: c.GetHeader(federation.HeaderSignature), + Method: c.Request.Method, + URI: uri, + }) + switch { + case err == nil: + case errors.Is(err, federation.ErrOff): + writeError(c, http.StatusNotFound, "not_found", "no such route") + return + case errors.Is(err, federation.ErrUnauthorized): + writeError(c, http.StatusUnauthorized, "unauthorized", "the request is not signed by a friend") + return + case errors.Is(err, federation.ErrNotApproved): + writeError(c, http.StatusForbidden, "not_approved", "the friendship is not approved yet") + return + default: + _ = c.Error(err) + writeError(c, http.StatusInternalServerError, "internal", "internal error") + return + } + c.Set(friendKey, friend) + c.Next() + sent := int64(c.Writer.Size()) + if sent < 0 { + sent = 0 + } + fed.Record(c.Request.Context(), federation.FriendRequest{FriendID: friend.ID, Route: c.FullPath(), Status: c.Writer.Status(), Bytes: sent}) + } +} + +func friendOf(c *gin.Context) federation.Friend { + f, _ := c.Get(friendKey) + friend, _ := f.(federation.Friend) + return friend +} + +type peerInfoHandlers struct { + fed federationService + libs libraryService +} + +// info says who this server is and how many libraries are shared; no more. +func (h *peerInfoHandlers) info(c *gin.Context) { + st, err := h.fed.Status(c.Request.Context()) + if err != nil { + writeFederationError(c, err) + return + } + libs, err := h.libs.List(c.Request.Context()) + if err != nil { + writeLibraryError(c, err) + return + } + shared := 0 + for _, l := range libs { + if l.Shared { + shared++ + } + } + c.JSON(http.StatusOK, gin.H{"name": st.Name, "shared_libraries": shared}) +} + +// pacedWriter slows the body of an answer to what the friend's allowance says. +type pacedWriter struct { + gin.ResponseWriter + st *federation.Stream + ctx context.Context +} + +func (w *pacedWriter) Write(p []byte) (int, error) { + written := 0 + for len(p) > 0 { + n := min(len(p), peerChunkSize) + if err := w.st.Wait(w.ctx, n); err != nil { + return written, err + } + m, err := w.ResponseWriter.Write(p[:n]) + written += m + if err != nil { + return written, err + } + p = p[n:] + } + return written, nil +} + +type peerStreamHandlers struct { + fed federationService + inner *streamHandlers +} + +// stream serves a track of a shared library to a friend, as a listener would get it, within the friend's +// allowance: a few streams at a time, and a number of bytes a second. +func (h *peerStreamHandlers) stream(c *gin.Context) { + st, err := h.fed.StartStream(c.Request.Context(), friendOf(c).ID) + if errors.Is(err, federation.ErrTooManyStreams) { + c.Header("Retry-After", "5") + writeError(c, http.StatusTooManyRequests, "too_many_streams", "this server already streams as much to you as it allows") + return + } + if err != nil { + _ = c.Error(err) + writeError(c, http.StatusInternalServerError, "internal", "internal error") + return + } + defer st.Close() + c.Writer = &pacedWriter{ResponseWriter: c.Writer, st: st, ctx: c.Request.Context()} + h.inner.stream(c) +} + +type peerCoverHandlers struct { + fed federationService + catalog catalogService + covers *coverHandlers +} + +// album serves a cover that was already built, for an album that has a track in a shared library. +func (h *peerCoverHandlers) album(c *gin.Context) { + id, ok := pathID(c) + if !ok { + return + } + if _, err := h.catalog.Album(c.Request.Context(), id); err != nil { + if errors.Is(err, catalog.ErrNotFound) { + writeError(c, http.StatusNotFound, "album_not_found", "no such album") + return + } + _ = c.Error(err) + writeError(c, http.StatusInternalServerError, "internal", "internal error") + return + } + st, err := h.fed.Pace(c.Request.Context(), friendOf(c).ID) + if err != nil { + _ = c.Error(err) + writeError(c, http.StatusInternalServerError, "internal", "internal error") + return + } + defer st.Close() + c.Writer = &pacedWriter{ResponseWriter: c.Writer, st: st, ctx: c.Request.Context()} + h.covers.album(c) +} diff --git a/internal/api/http/libraries_test.go b/internal/api/http/libraries_test.go index ceee8ff..1fe317e 100644 --- a/internal/api/http/libraries_test.go +++ b/internal/api/http/libraries_test.go @@ -190,6 +190,9 @@ func (busyLibraries) Create(context.Context, string, string) (library.Library, e func (busyLibraries) Update(context.Context, int64, *string, *string) (library.Library, error) { return library.Library{}, library.ErrScanRunning } +func (busyLibraries) SetShared(context.Context, int64, bool) (library.Library, error) { + return library.Library{}, nil +} func (busyLibraries) Delete(context.Context, int64) error { return library.ErrScanRunning } func TestLibraryChangesDuringAScanAreConflicts(t *testing.T) { diff --git a/internal/api/http/peer_test.go b/internal/api/http/peer_test.go new file mode 100644 index 0000000..8c00838 --- /dev/null +++ b/internal/api/http/peer_test.go @@ -0,0 +1,526 @@ +package httpapi + +import ( + "bytes" + "crypto/ed25519" + "crypto/rand" + "encoding/base64" + "encoding/json" + "net/http" + "net/http/httptest" + "os" + "path/filepath" + "strconv" + "strings" + "sync/atomic" + "testing" + "time" + + "github.com/uptrace/bun" + + "codeberg.org/squidd/sonda/internal/federation" + "codeberg.org/squidd/sonda/internal/library" + "codeberg.org/squidd/sonda/internal/storage" +) + +// peerFixture is a server with two libraries, one that can be shared and one that cannot, and a friend that holds +// a key of its own. +type peerFixture struct { + h http.Handler + db *bun.DB + admin string + fed *federation.Service + pub ed25519.PublicKey + priv ed25519.PrivateKey + shared int64 // library ids + private int64 + track map[string]int64 + friendID string + bytes map[string][]byte // what each file holds + nonce atomic.Int64 +} + +func peerParsed(rel, title, album, artist string) library.ParsedFile { + return library.ParsedFile{ + FileInfo: library.FileInfo{RelPath: rel, Size: 400, MTimeUnix: 1_700_000_000}, + Metadata: library.Metadata{ + Tags: library.Tags{Title: title, Album: album, AlbumArtist: artist, Artists: []string{artist}, TrackNo: 1, DiscNo: 1}, + Properties: library.Properties{Format: "mpeg", DurationMs: 60000, SampleRate: 44100}, + }, + } +} + +func newPeerFixture(t *testing.T) *peerFixture { + t.Helper() + f := &peerFixture{track: map[string]int64{}, bytes: map[string][]byte{}} + h, db := newTestEnvDB(t, func(d *Deps, db *bun.DB) { + f.fed = federation.NewService(storage.NewFederationRepo(db), federation.NewClient()) + d.Federation = f.fed + }) + f.h, f.db = h, db + f.admin = registerAndLogin(t, h) + + root := t.TempDir() + repo := storage.NewLibraryRepo(db) + for _, l := range []struct { + name, title, album, artist string + fill byte + id *int64 + }{ + {"shared", "Shared Song", "Shared Album", "Shared Band", 0xA1, &f.shared}, + {"private", "Private Song", "Private Album", "Private Band", 0xB2, &f.private}, + } { + dir := filepath.Join(root, l.name) + if err := os.MkdirAll(dir, 0o755); err != nil { + t.Fatal(err) + } + data := bytes.Repeat([]byte{l.fill}, 400) + for i := range data { + data[i] ^= byte(i) + } + f.bytes[l.title] = data + if err := os.WriteFile(filepath.Join(dir, "song.mp3"), data, 0o644); err != nil { + t.Fatal(err) + } + lib, err := repo.CreateLibrary(t.Context(), l.name, dir) + if err != nil { + t.Fatal(err) + } + *l.id = lib.ID + if _, err := repo.SaveFile(t.Context(), lib.ID, peerParsed("song.mp3", l.title, l.album, l.artist), 0); err != nil { + t.Fatal(err) + } + } + rows, err := db.Query("SELECT id, title FROM tracks WHERE title <> 'song'") + if err != nil { + t.Fatal(err) + } + for rows.Next() { + var id int64 + var title string + _ = rows.Scan(&id, &title) + f.track[title] = id + } + _ = rows.Close() + return f +} + +func (f *peerFixture) admin1(t *testing.T, method, path string, body any) resp { + t.Helper() + return do(t, f.h, method, "/api/v1"+path, f.admin, body) +} + +// befriend turns federation on and makes the friend (its key is in f.priv) an active friend, the way a real one +// would: with an invite, and then the admin's approval. +func (f *peerFixture) befriend(t *testing.T) { + t.Helper() + if r := f.admin1(t, "PUT", "/federation", map[string]any{"enabled": true, "name": "Anna", "address": "https://anna.example.org"}); r.Code != 200 { + t.Fatalf("turn on: %d %v", r.Code, r.Body) + } + inv := f.admin1(t, "POST", "/federation/invites", nil) + f.pub, f.priv, _ = ed25519.GenerateKey(rand.Reader) + body := map[string]any{ + "code": inviteCode(t, str(inv.Body, "invite")), "name": "Bruno", "address": "https://bruno.example.org", + "public_key": base64.RawURLEncoding.EncodeToString(f.pub), + } + if r := do(t, f.h, "POST", "/api/v1/peer/pair", "", body); r.Code != 200 { + t.Fatalf("pair: %d %v", r.Code, r.Body) + } + list := f.admin1(t, "GET", "/federation", nil).Body["friends"].([]any) + f.friendID = strconv.Itoa(int(list[0].(map[string]any)["id"].(float64))) + if r := f.admin1(t, "POST", "/federation/friends/"+f.friendID+"/approve", nil); r.Code != 200 { + t.Fatalf("approve: %d %v", r.Code, r.Body) + } +} + +func (f *peerFixture) share(t *testing.T, id int64, on bool) { + t.Helper() + r := f.admin1(t, "PATCH", "/libraries/"+strconv.FormatInt(id, 10), map[string]any{"shared": on}) + if r.Code != 200 || r.Body["shared"] != on { + t.Fatalf("share %d: %d %v", id, r.Code, r.Body) + } +} + +// signed makes a request a friend would make, signed with its key. +func (f *peerFixture) signed(method, path string, mutate ...func(*http.Request)) *http.Request { + uri := "/api/v1/peer" + path + n := f.nonce.Add(1) + h := federation.Sign(f.priv, method, uri, time.Now(), "nonce-"+strconv.FormatInt(n, 10)+"-0123456789abcdef") + req := httptest.NewRequest(method, uri, nil) + req.Header.Set(federation.HeaderKeyID, federation.KeyID(f.pub)) + req.Header.Set(federation.HeaderDate, h.Date) + req.Header.Set(federation.HeaderNonce, h.Nonce) + req.Header.Set(federation.HeaderSignature, h.Signature) + for _, m := range mutate { + m(req) + } + return req +} + +func (f *peerFixture) serve(req *http.Request) *httptest.ResponseRecorder { + rec := httptest.NewRecorder() + f.h.ServeHTTP(rec, req) + return rec +} + +func (f *peerFixture) get(t *testing.T, path string) resp { + t.Helper() + rec := f.serve(f.signed("GET", path)) + out := resp{Code: rec.Code, Body: map[string]any{}} + _ = json.Unmarshal(rec.Body.Bytes(), &out.Body) + return out +} + +func titlesIn(r resp) []string { + var out []string + for _, it := range r.Body["items"].([]any) { + m := it.(map[string]any) + if v, ok := m["title"].(string); ok { + out = append(out, v) + } else { + out = append(out, m["name"].(string)) + } + } + return out +} + +func TestAFriendMustSignItsRequests(t *testing.T) { + f := newPeerFixture(t) + f.share(t, f.shared, true) + + // Federation is off: the routes are as good as missing, whoever asks. + if rec := f.serve(httptest.NewRequest("GET", "/api/v1/peer/info", nil)); rec.Code != 404 { + t.Errorf("while off: %d", rec.Code) + } + f.befriend(t) + + for name, req := range map[string]*http.Request{ + "no signature": httptest.NewRequest("GET", "/api/v1/peer/info", nil), + "another path": f.signed("GET", "/info", func(r *http.Request) { r.URL.Path = "/api/v1/peer/artists" }), + "a bad signature": f.signed("GET", "/info", func(r *http.Request) { r.Header.Set(federation.HeaderSignature, "AAAA") }), + "an old date": f.signed("GET", "/info", func(r *http.Request) { + r.Header.Set(federation.HeaderDate, strconv.FormatInt(time.Now().Add(-time.Hour).Unix(), 10)) + }), + "another key": f.signed("GET", "/info", func(r *http.Request) { r.Header.Set(federation.HeaderKeyID, "0123456789abcdef") }), + } { + rec := f.serve(req) + var body struct{ Error struct{ Code string } } + _ = json.Unmarshal(rec.Body.Bytes(), &body) + if rec.Code != 401 || body.Error.Code != "unauthorized" { + t.Errorf("%s: %d %s", name, rec.Code, rec.Body.String()) + } + } + + // A request is good once. + req := f.signed("GET", "/info") + if rec := f.serve(req); rec.Code != 200 { + t.Fatalf("the good request: %d %s", rec.Code, rec.Body.String()) + } + if rec := f.serve(req.Clone(req.Context())); rec.Code != 401 { + t.Errorf("the same request again: %d, want 401", rec.Code) + } + + // Removing the friend ends it at once. + f.admin1(t, "DELETE", "/federation/friends/"+f.friendID, nil) + if rec := f.serve(f.signed("GET", "/info")); rec.Code != 401 { + t.Errorf("after removal: %d, want 401", rec.Code) + } +} + +func TestAFriendWaitingForApprovalIsToldSo(t *testing.T) { + f := newPeerFixture(t) + f.admin1(t, "PUT", "/federation", map[string]any{"enabled": true, "address": "https://anna.example.org"}) + inv := f.admin1(t, "POST", "/federation/invites", nil) + f.pub, f.priv, _ = ed25519.GenerateKey(rand.Reader) + do(t, f.h, "POST", "/api/v1/peer/pair", "", map[string]any{ + "code": inviteCode(t, str(inv.Body, "invite")), "name": "Bruno", "address": "https://bruno.example.org", + "public_key": base64.RawURLEncoding.EncodeToString(f.pub), + }) + rec := f.serve(f.signed("GET", "/info")) + if rec.Code != 403 || !strings.Contains(rec.Body.String(), `"not_approved"`) { + t.Errorf("unapproved friend: %d %s", rec.Code, rec.Body.String()) + } +} + +func TestAFriendSeesWhoWeAreAndHowManyLibrariesAreShared(t *testing.T) { + f := newPeerFixture(t) + f.befriend(t) + r := f.get(t, "/info") + sameKeys(t, "info", r.Body, "name", "shared_libraries") + if r.Code != 200 || r.Body["name"] != "Anna" || r.Body["shared_libraries"] != float64(0) { + t.Errorf("info = %d %v", r.Code, r.Body) + } + f.share(t, f.shared, true) + if r := f.get(t, "/info"); r.Body["shared_libraries"] != float64(1) { + t.Errorf("info = %v", r.Body) + } +} + +func TestAFriendReadsOnlyTheSharedLibraries(t *testing.T) { + f := newPeerFixture(t) + f.befriend(t) + + // Nothing is shared: nothing is there, by list, by id or by search. + for _, p := range []string{"/artists", "/albums", "/tracks"} { + if r := f.get(t, p); r.Code != 200 || r.Body["total"] != float64(0) { + t.Errorf("%s with nothing shared = %d %v", p, r.Code, r.Body) + } + } + for _, p := range []string{"/tracks/" + strconv.FormatInt(f.track["Shared Song"], 10), "/tracks/" + strconv.FormatInt(f.track["Private Song"], 10)} { + if r := f.get(t, p); r.Code != 404 { + t.Errorf("%s with nothing shared = %d", p, r.Code) + } + } + + f.share(t, f.shared, true) + if got := titlesIn(f.get(t, "/tracks")); len(got) != 1 || got[0] != "Shared Song" { + t.Errorf("tracks = %v", got) + } + if got := titlesIn(f.get(t, "/albums")); len(got) != 1 || got[0] != "Shared Album" { + t.Errorf("albums = %v", got) + } + if got := titlesIn(f.get(t, "/artists")); len(got) != 1 || got[0] != "Shared Band" { + t.Errorf("artists = %v", got) + } + if got := titlesIn(f.get(t, "/tracks?q=private")); len(got) != 0 { + t.Errorf("a search found %v in the library that is not shared", got) + } + if got := titlesIn(f.get(t, "/tracks?library_id="+strconv.FormatInt(f.private, 10))); len(got) != 0 { + t.Errorf("asking for the other library found %v", got) + } + one := f.get(t, "/tracks/"+strconv.FormatInt(f.track["Shared Song"], 10)) + sameKeys(t, "track", one.Body, "id", "title", "duration_ms", "track_no", "disc_no", "album", "artists", "format") + if one.Code != 200 || one.Body["title"] != "Shared Song" { + t.Errorf("track = %d %v", one.Code, one.Body) + } + if r := f.get(t, "/tracks/"+strconv.FormatInt(f.track["Private Song"], 10)); r.Code != 404 || errCode(r) != "track_not_found" { + t.Errorf("the track of the other library: %d %v", r.Code, r.Body) + } + for _, p := range []string{"/artists/", "/albums/"} { + if r := f.get(t, p+"999999"); r.Code != 404 { + t.Errorf("%s999999 = %d", p, r.Code) + } + } + // Whatever the answer says, it is never a path. + rec := f.serve(f.signed("GET", "/tracks")) + if strings.Contains(rec.Body.String(), "song.mp3") || strings.Contains(rec.Body.String(), "/shared") { + t.Errorf("the answer names a path: %s", rec.Body.String()) + } + + // Unsharing takes it away at once. + f.share(t, f.shared, false) + if r := f.get(t, "/tracks"); r.Body["total"] != float64(0) { + t.Errorf("after unsharing: %v", r.Body) + } +} + +func TestAFriendStreamsSharedMusicWithRanges(t *testing.T) { + f := newPeerFixture(t) + f.befriend(t) + f.share(t, f.shared, true) + want := f.bytes["Shared Song"] + id := strconv.FormatInt(f.track["Shared Song"], 10) + + rec := f.serve(f.signed("GET", "/tracks/"+id+"/stream")) + if rec.Code != 200 || !bytes.Equal(rec.Body.Bytes(), want) { + t.Fatalf("stream: %d, %d bytes", rec.Code, rec.Body.Len()) + } + if ct := rec.Header().Get("Content-Type"); !strings.HasPrefix(ct, "audio/") { + t.Errorf("Content-Type = %q", ct) + } + rec = f.serve(f.signed("GET", "/tracks/"+id+"/stream", func(r *http.Request) { r.Header.Set("Range", "bytes=100-199") })) + if rec.Code != 206 || !bytes.Equal(rec.Body.Bytes(), want[100:200]) || rec.Header().Get("Content-Range") != "bytes 100-199/400" { + t.Errorf("range: %d %q %d bytes", rec.Code, rec.Header().Get("Content-Range"), rec.Body.Len()) + } + + // The other library is out of reach, by its id too. + other := strconv.FormatInt(f.track["Private Song"], 10) + if rec := f.serve(f.signed("GET", "/tracks/"+other+"/stream")); rec.Code != 404 { + t.Errorf("stream of a track that is not shared: %d", rec.Code) + } + if rec := f.serve(f.signed("GET", "/tracks/999999/stream")); rec.Code != 404 { + t.Errorf("stream of no track: %d", rec.Code) + } +} + +func TestAFriendCannotHaveMoreStreamsThanAllowed(t *testing.T) { + f := newPeerFixture(t) + f.befriend(t) + f.share(t, f.shared, true) + one := 1 + if _, err := f.fed.Update(t.Context(), federation.Patch{MaxStreams: &one}); err != nil { + t.Fatal(err) + } + fid, _ := strconv.ParseInt(f.friendID, 10, 64) + open, err := f.fed.StartStream(t.Context(), fid) + if err != nil { + t.Fatal(err) + } + id := strconv.FormatInt(f.track["Shared Song"], 10) + rec := f.serve(f.signed("GET", "/tracks/"+id+"/stream")) + if rec.Code != 429 || !strings.Contains(rec.Body.String(), `"too_many_streams"`) || rec.Header().Get("Retry-After") == "" { + t.Errorf("one stream too many: %d %s (Retry-After %q)", rec.Code, rec.Body.String(), rec.Header().Get("Retry-After")) + } + open.Close() + if rec := f.serve(f.signed("GET", "/tracks/"+id+"/stream")); rec.Code != 200 { + t.Errorf("after the first ended: %d", rec.Code) + } +} + +func TestACoverIsOnlyServedOnceBuiltAndOnlyForSharedAlbums(t *testing.T) { + f := newPeerFixture(t) + f.befriend(t) + f.share(t, f.shared, true) + var shared, private int64 + _ = f.db.QueryRow("SELECT id FROM albums WHERE title = 'Shared Album'").Scan(&shared) + _ = f.db.QueryRow("SELECT id FROM albums WHERE title = 'Private Album'").Scan(&private) + sid, pid := strconv.FormatInt(shared, 10), strconv.FormatInt(private, 10) + + r := f.get(t, "/albums/"+sid+"/cover") + if r.Code != 404 || errCode(r) != "cover_not_found" { + t.Errorf("a cover that was not built: %d %v: a friend must not make this server build one", r.Code, r.Body) + } + if r := f.get(t, "/albums/"+pid+"/cover"); r.Code != 404 || errCode(r) != "album_not_found" { + t.Errorf("the cover of an album that is not shared: %d %v", r.Code, r.Body) + } + if r := f.get(t, "/albums/"+sid+"/cover?size=huge"); r.Code != 400 { + t.Errorf("a bad size: %d", r.Code) + } +} + +func TestSharingALibraryIsAnAdminsChoice(t *testing.T) { + f := newPeerFixture(t) + id := strconv.FormatInt(f.shared, 10) + list := f.admin1(t, "GET", "/libraries", nil).Body["libraries"].([]any) + for _, l := range list { + if l.(map[string]any)["shared"] != false { + t.Errorf("library %v starts shared", l) + } + } + _, bob := addUser(t, f.h, f.admin, "bob", false) + if r := do(t, f.h, "PATCH", "/api/v1/libraries/"+id, bob, map[string]any{"shared": true}); r.Code != 403 { + t.Errorf("a user who is not an admin shares a library: %d", r.Code) + } + r := f.admin1(t, "PATCH", "/libraries/"+id, map[string]any{"shared": true}) + if r.Code != 200 || r.Body["shared"] != true { + t.Fatalf("share: %d %v", r.Code, r.Body) + } + // Name and sharing can be changed together or apart. + r = f.admin1(t, "PATCH", "/libraries/"+id, map[string]any{"name": "renamed"}) + if r.Code != 200 || r.Body["shared"] != true || r.Body["name"] != "renamed" { + t.Errorf("rename keeps sharing: %d %v", r.Code, r.Body) + } + r = f.admin1(t, "PATCH", "/libraries/"+id, map[string]any{"name": "again", "shared": false}) + if r.Code != 200 || r.Body["shared"] != false || r.Body["name"] != "again" { + t.Errorf("both at once: %d %v", r.Code, r.Body) + } + if r := f.admin1(t, "PATCH", "/libraries/"+id, map[string]any{}); r.Code != 400 { + t.Errorf("nothing to change: %d", r.Code) + } + if r := f.admin1(t, "PATCH", "/libraries/99999", map[string]any{"shared": true}); r.Code != 404 { + t.Errorf("unknown library: %d", r.Code) + } +} + +func TestTheAdminSeesWhatFriendsAsked(t *testing.T) { + f := newPeerFixture(t) + f.befriend(t) + f.share(t, f.shared, true) + f.get(t, "/info") + f.get(t, "/tracks?q=secret+words&limit=5") + f.serve(f.signed("GET", "/tracks/"+strconv.FormatInt(f.track["Shared Song"], 10)+"/stream")) + + r := f.admin1(t, "GET", "/federation/requests", nil) + if r.Code != 200 { + t.Fatalf("requests: %d %v", r.Code, r.Body) + } + list := r.Body["requests"].([]any) + if len(list) != 3 { + t.Fatalf("requests = %v", list) + } + newest := list[0].(map[string]any) + sameKeys(t, "request", newest, "at", "friend_id", "friend", "route", "status", "bytes") + if newest["route"] != "/api/v1/peer/tracks/:id/stream" || newest["status"] != float64(200) || newest["bytes"] != float64(400) || newest["friend"] != "Bruno" { + t.Errorf("newest = %v", newest) + } + if body := strings.ToLower(jsonOf(t, r.Body)); strings.Contains(body, "secret") { + t.Errorf("the log keeps what was searched for: %s", body) + } + _, bob := addUser(t, f.h, f.admin, "bob", false) + if r := do(t, f.h, "GET", "/api/v1/federation/requests", bob, nil); r.Code != 403 { + t.Errorf("a user who is not an admin reads the log: %d", r.Code) + } +} + +func jsonOf(t *testing.T, v any) string { + t.Helper() + b, err := json.Marshal(v) + if err != nil { + t.Fatal(err) + } + return string(b) +} + +func TestTheLimitsAreSetWithTheOtherSettings(t *testing.T) { + f := newPeerFixture(t) + r := f.admin1(t, "GET", "/federation", nil) + if r.Body["max_streams"] != float64(3) || r.Body["max_kib_per_sec"] != float64(4096) { + t.Errorf("defaults = %v %v", r.Body["max_streams"], r.Body["max_kib_per_sec"]) + } + r = f.admin1(t, "PUT", "/federation", map[string]any{"max_streams": 2, "max_kib_per_sec": 0}) + if r.Code != 200 || r.Body["max_streams"] != float64(2) || r.Body["max_kib_per_sec"] != float64(0) { + t.Errorf("update = %d %v", r.Code, r.Body) + } + for _, body := range []map[string]any{{"max_streams": 0}, {"max_streams": 99}, {"max_kib_per_sec": 5}} { + if r := f.admin1(t, "PUT", "/federation", body); r.Code != 400 || errCode(r) != "invalid_input" { + t.Errorf("%v: %d %v", body, r.Code, r.Body) + } + } +} + +func TestAServerChecksWhetherTheOtherApproved(t *testing.T) { + a := newFriendServer(t, true) + b := newFriendServer(t, false) + a.setUp(t, "Anna", a.ts.URL) + b.setUp(t, "Bruno", "https://bruno.example.org") + inv := a.call(t, "POST", "/federation/invites", nil) + mine := b.call(t, "POST", "/federation/friends", map[string]any{"invite": inv.Body["invite"]}) + bID := strconv.Itoa(int(mine.Body["id"].(float64))) + + // Anna has not approved yet. + r := b.call(t, "POST", "/federation/friends/"+bID+"/check", nil) + if r.Code != http.StatusConflict || errCode(r) != "not_approved" { + t.Fatalf("before approval: %d %v", r.Code, r.Body) + } + if got := b.call(t, "GET", "/federation", nil).Body["friends"].([]any)[0].(map[string]any)["state"]; got != "outgoing" { + t.Errorf("state = %v", got) + } + + list := a.call(t, "GET", "/federation", nil).Body["friends"].([]any) + aID := strconv.Itoa(int(list[0].(map[string]any)["id"].(float64))) + a.call(t, "POST", "/federation/friends/"+aID+"/approve", nil) + + r = b.call(t, "POST", "/federation/friends/"+bID+"/check", nil) + if r.Code != 200 || r.Body["state"] != "active" || r.Body["approved_at"] == nil { + t.Fatalf("after approval: %d %v", r.Code, r.Body) + } + // Anna logged that call. + if reqs := a.call(t, "GET", "/federation/requests", nil).Body["requests"].([]any); len(reqs) != 1 || reqs[0].(map[string]any)["route"] != "/api/v1/peer/info" { + t.Errorf("Anna's log = %v", reqs) + } + + // Anna removes Bruno: Bruno is told it is no friend any more, and stays as it was. + a.call(t, "DELETE", "/federation/friends/"+aID, nil) + r = b.call(t, "POST", "/federation/friends/"+bID+"/check", nil) + if r.Code != http.StatusConflict || errCode(r) != "friend_gone" { + t.Errorf("after removal: %d %v", r.Code, r.Body) + } + // And a server that is gone is "unreachable", not a verdict. + a.ts.Close() + if r := b.call(t, "POST", "/federation/friends/"+bID+"/check", nil); r.Code != http.StatusBadGateway || errCode(r) != "friend_unreachable" { + t.Errorf("server gone: %d %v", r.Code, r.Body) + } + if r := b.call(t, "POST", "/federation/friends/99999/check", nil); r.Code != 404 { + t.Errorf("unknown friend: %d", r.Code) + } +} diff --git a/internal/api/http/router.go b/internal/api/http/router.go index 163ac6f..a5064c2 100644 --- a/internal/api/http/router.go +++ b/internal/api/http/router.go @@ -183,6 +183,7 @@ type libraryService interface { List(ctx context.Context) ([]library.Library, error) Create(ctx context.Context, name, path string) (library.Library, error) Update(ctx context.Context, id int64, name, path *string) (library.Library, error) + SetShared(ctx context.Context, id int64, shared bool) (library.Library, error) Delete(ctx context.Context, id int64) error } @@ -218,6 +219,11 @@ type Deps struct { Settings userSettingsService Streams streamService Federation federationService + // What a friend's server reads: the catalogue and the files of the libraries that are shared, and only covers + // that were already built. + PeerCatalog catalogService + PeerStreams streamService + PeerCovers coverService // Web is the built web app, served on / and for its own links. It is optional: with none (a build // without the app) / shows a page that says how to build it, and the API works the same. Web fs.FS @@ -262,6 +268,7 @@ func NewRouter(d Deps) *gin.Engine { v1.POST("/auth/logout", a.logout) // Another Sonda server pairs here, with the code of an invite instead of a login. v1.POST("/peer/pair", (&federationHandlers{svc: d.Federation}).pair) + registerPeer(v1, d) // Any signed-in user. protected := v1.Group("", requireAuth(d.Tokens, d.Auth, false)) @@ -397,6 +404,8 @@ func NewRouter(d Deps) *gin.Engine { admin.POST("/federation/invites", fed.invite) admin.POST("/federation/friends", fed.redeem) admin.POST("/federation/friends/:id/approve", fed.approve) + admin.POST("/federation/friends/:id/check", fed.check) + admin.GET("/federation/requests", fed.requests) admin.DELETE("/federation/friends/:id", fed.remove) u := &usersHandlers{svc: d.Auth} admin.GET("/users", u.list) diff --git a/internal/api/openapi/openapi.yaml b/internal/api/openapi/openapi.yaml index 2bd6c5c..68bdde4 100644 --- a/internal/api/openapi/openapi.yaml +++ b/internal/api/openapi/openapi.yaml @@ -3162,6 +3162,346 @@ paths: application/json: schema: {$ref: '#/components/schemas/Error'} + /federation/friends/{id}/check: + parameters: + - {name: id, in: path, required: true, schema: {type: integer}} + post: + tags: [Federation] + operationId: checkFederationFriend + summary: Ask a friend whether it approved + description: | + Admin only. Sends the friend one signed request (`GET /peer/info`). An answer means it approved, and a friend that + was `outgoing` becomes `active`. The friend is returned. A friend in state `incoming` is a `409` (`not_waiting`): + it waits for this server's admin, not the other way round. + security: [{bearerAuth: []}] + responses: + '200': + description: The friend, active when it answered. + content: + application/json: + schema: {$ref: '#/components/schemas/FederationFriend'} + '400': {$ref: '#/components/responses/InvalidInput'} + '401': {$ref: '#/components/responses/Unauthorized'} + '403': {$ref: '#/components/responses/AdminRequired'} + '404': + description: No such friend (`friend_not_found`). + content: + application/json: + schema: {$ref: '#/components/schemas/Error'} + '409': + description: | + Federation is off (`federation_off`), the friend is not waiting for the other admin (`not_waiting`), the other + admin has not approved yet (`not_approved`), or the other server does not take this one for a friend any more + (`friend_gone`). + content: + application/json: + schema: {$ref: '#/components/schemas/Error'} + '502': + description: The other server could not be reached or did not answer as a Sonda server (`friend_unreachable`). + content: + application/json: + schema: {$ref: '#/components/schemas/Error'} + + /federation/requests: + get: + tags: [Federation] + operationId: listFederationRequests + summary: The last requests friends made + description: | + Admin only. The last 50 requests that friends made to this server, newest first, kept for 7 days: when, which + friend, which route (the pattern, never the query, so what was searched for is not kept), the status and how many + bytes were sent. + security: [{bearerAuth: []}] + responses: + '200': + description: The requests. + content: + application/json: + schema: + type: object + additionalProperties: false + required: [requests] + properties: + requests: + type: array + items: {$ref: '#/components/schemas/FederationRequest'} + '401': {$ref: '#/components/responses/Unauthorized'} + '403': {$ref: '#/components/responses/AdminRequired'} + + /peer/info: + get: + tags: [Federation] + operationId: peerInfo + summary: Who this server is (called by a friend) + description: | + Not for people: a friend's server calls this, signed (see the `friendSignature` scheme), and also to find out whether + this server's admin approved it. Says the name of this server and how many libraries are shared. + security: [{friendSignature: []}] + responses: + '200': + description: The name and the number of shared libraries. + content: + application/json: + schema: {$ref: '#/components/schemas/PeerInfo'} + '401': {$ref: '#/components/responses/FriendUnauthorized'} + '403': {$ref: '#/components/responses/FriendNotApproved'} + '404': {$ref: '#/components/responses/FederationOff'} + + /peer/artists: + get: + tags: [Federation] + operationId: peerListArtists + summary: The artists of the shared libraries (called by a friend) + description: | + Like `GET /artists`, over the libraries that are shared and nothing else: an artist is listed only when it has a + playable track in one of them, and the counts are of those tracks. Paths, library ids and everything that providers + said are never part of it. Signed like every `/peer` route. + security: [{friendSignature: []}] + parameters: + - {$ref: '#/components/parameters/Q'} + - {$ref: '#/components/parameters/Limit'} + - {$ref: '#/components/parameters/Offset'} + responses: + '200': + description: A page of artists. + content: + application/json: + schema: {$ref: '#/components/schemas/ArtistPage'} + '400': {$ref: '#/components/responses/InvalidInput'} + '401': {$ref: '#/components/responses/FriendUnauthorized'} + '403': {$ref: '#/components/responses/FriendNotApproved'} + '404': {$ref: '#/components/responses/FederationOff'} + + /peer/artists/{id}: + parameters: + - {$ref: '#/components/parameters/Id'} + get: + tags: [Federation] + operationId: peerGetArtist + summary: One artist of the shared libraries (called by a friend) + security: [{friendSignature: []}] + responses: + '200': + description: The artist. + content: + application/json: + schema: {$ref: '#/components/schemas/Artist'} + '400': {$ref: '#/components/responses/InvalidInput'} + '401': {$ref: '#/components/responses/FriendUnauthorized'} + '403': {$ref: '#/components/responses/FriendNotApproved'} + '404': + description: No such artist among the shared libraries (`artist_not_found`), or federation is off (`not_found`). + content: + application/json: + schema: {$ref: '#/components/schemas/Error'} + + /peer/albums: + get: + tags: [Federation] + operationId: peerListAlbums + summary: The albums of the shared libraries (called by a friend) + description: Like `GET /albums`, over the libraries that are shared. Signed like every `/peer` route. + security: [{friendSignature: []}] + parameters: + - {$ref: '#/components/parameters/Q'} + - {$ref: '#/components/parameters/Limit'} + - {$ref: '#/components/parameters/Offset'} + - {$ref: '#/components/parameters/ArtistId'} + responses: + '200': + description: A page of albums. + content: + application/json: + schema: {$ref: '#/components/schemas/AlbumPage'} + '400': {$ref: '#/components/responses/InvalidInput'} + '401': {$ref: '#/components/responses/FriendUnauthorized'} + '403': {$ref: '#/components/responses/FriendNotApproved'} + '404': {$ref: '#/components/responses/FederationOff'} + + /peer/albums/{id}: + parameters: + - {$ref: '#/components/parameters/Id'} + get: + tags: [Federation] + operationId: peerGetAlbum + summary: One album of the shared libraries (called by a friend) + security: [{friendSignature: []}] + responses: + '200': + description: The album. + content: + application/json: + schema: {$ref: '#/components/schemas/Album'} + '400': {$ref: '#/components/responses/InvalidInput'} + '401': {$ref: '#/components/responses/FriendUnauthorized'} + '403': {$ref: '#/components/responses/FriendNotApproved'} + '404': + description: No such album among the shared libraries (`album_not_found`), or federation is off (`not_found`). + content: + application/json: + schema: {$ref: '#/components/schemas/Error'} + + /peer/albums/{id}/cover: + parameters: + - {$ref: '#/components/parameters/Id'} + get: + tags: [Federation] + operationId: peerGetAlbumCover + summary: The cover of an album of the shared libraries (called by a friend) + description: | + Only a cover that this server already built from the audio files of the album: a friend never makes this server + build one, read an audio file or ask a provider. A cover that came from the Cover Art Archive is not given. Any other + album answers `cover_not_found`. The bytes go out within the friend's allowance. + security: [{friendSignature: []}] + parameters: + - name: size + in: query + description: "`small` or `large`. Large when absent." + schema: {type: string, enum: [small, large], default: large} + - name: If-None-Match + in: header + required: false + schema: {type: string} + responses: + '200': + description: The cover. + headers: + ETag: {$ref: '#/components/headers/ETag'} + Cache-Control: {$ref: '#/components/headers/CacheControl'} + Content-Length: {$ref: '#/components/headers/ContentLength'} + content: + image/jpeg: + schema: {type: string, format: binary} + '304': {description: Not modified since `If-None-Match`.} + '400': {$ref: '#/components/responses/InvalidInput'} + '401': {$ref: '#/components/responses/FriendUnauthorized'} + '403': {$ref: '#/components/responses/FriendNotApproved'} + '404': + description: No such album among the shared libraries (`album_not_found`), no cover built (`cover_not_found`), or federation is off (`not_found`). + content: + application/json: + schema: {$ref: '#/components/schemas/Error'} + + /peer/tracks: + get: + tags: [Federation] + operationId: peerListTracks + summary: The tracks of the shared libraries (called by a friend) + description: Like `GET /tracks`, over the libraries that are shared. Signed like every `/peer` route. + security: [{friendSignature: []}] + parameters: + - {$ref: '#/components/parameters/Q'} + - {$ref: '#/components/parameters/Limit'} + - {$ref: '#/components/parameters/Offset'} + - {$ref: '#/components/parameters/ArtistId'} + - {$ref: '#/components/parameters/AlbumId'} + responses: + '200': + description: A page of tracks. + content: + application/json: + schema: {$ref: '#/components/schemas/TrackPage'} + '400': {$ref: '#/components/responses/InvalidInput'} + '401': {$ref: '#/components/responses/FriendUnauthorized'} + '403': {$ref: '#/components/responses/FriendNotApproved'} + '404': {$ref: '#/components/responses/FederationOff'} + + /peer/tracks/{id}: + parameters: + - {$ref: '#/components/parameters/Id'} + get: + tags: [Federation] + operationId: peerGetTrack + summary: One track of the shared libraries (called by a friend) + security: [{friendSignature: []}] + responses: + '200': + description: The track. + content: + application/json: + schema: {$ref: '#/components/schemas/Track'} + '400': {$ref: '#/components/responses/InvalidInput'} + '401': {$ref: '#/components/responses/FriendUnauthorized'} + '403': {$ref: '#/components/responses/FriendNotApproved'} + '404': + description: No such track among the shared libraries (`track_not_found`), or federation is off (`not_found`). + content: + application/json: + schema: {$ref: '#/components/schemas/Error'} + + /peer/tracks/{id}/stream: + parameters: + - {$ref: '#/components/parameters/Id'} + get: + tags: [Federation] + operationId: peerStreamTrack + summary: Stream a track of the shared libraries (called by a friend) + description: | + The file of the track that lives in a shared library, with `Range` support, as a listener would get it. A track that is + only in libraries that are not shared is `track_not_found`. A friend has a few streams open at a time and a number + of bytes a second to share among them, both set by the admin (`max_streams`, `max_kib_per_sec`): one stream too many + is a `429` (`too_many_streams`) with `Retry-After`. + security: [{friendSignature: []}] + parameters: + - name: Range + in: header + required: false + description: A byte range, for example `bytes=0-99`. + schema: {type: string} + - name: If-Modified-Since + in: header + required: false + schema: {type: string} + responses: + '200': + description: The whole file. + headers: + Accept-Ranges: {$ref: '#/components/headers/AcceptRanges'} + Content-Length: {$ref: '#/components/headers/ContentLength'} + Last-Modified: {$ref: '#/components/headers/LastModified'} + content: + audio/mpeg: + schema: {type: string, format: binary} + audio/flac: + schema: {type: string, format: binary} + '206': + description: The requested byte range. + headers: + Content-Range: + description: The range sent and the size of the file, for example `bytes 0-99/40635`. + schema: {type: string} + content: + audio/mpeg: + schema: {type: string, format: binary} + audio/flac: + schema: {type: string, format: binary} + multipart/byteranges: + schema: {type: string, format: binary} + '304': {description: Not modified since `If-Modified-Since`.} + '400': {$ref: '#/components/responses/InvalidInput'} + '401': {$ref: '#/components/responses/FriendUnauthorized'} + '403': {$ref: '#/components/responses/FriendNotApproved'} + '404': + description: No such track among the shared libraries (`track_not_found`), its file is not available (`file_missing`), or federation is off (`not_found`). + content: + application/json: + schema: {$ref: '#/components/schemas/Error'} + '416': + description: The range cannot be satisfied. + headers: + Content-Range: + description: '`bytes */`.' + schema: {type: string} + '429': + description: The friend already has as many streams open as allowed (`too_many_streams`). + headers: + Retry-After: + description: Seconds to wait. + schema: {type: integer} + content: + application/json: + schema: {$ref: '#/components/schemas/Error'} + components: securitySchemes: bearerAuth: @@ -3175,6 +3515,18 @@ components: name: access_token description: The same access token, for stream URLs used by an `