sonda/internal/storage/search_key.go
tgiachi c08a234159 feat(import): bring my Spotify history and playlists in
I want my listening history and my playlists from Spotify in Sonda, so that Mixes, Discover and the smart playlists know what I listened to before.

- POST /me/import/spotify takes the zip of a Spotify data export, or the JSON files in it (extended streaming history, the older StreamingHistory files, the playlists), as a multipart form. A request may be up to 512 MB; the zip is limited in how much it may inflate and how many files it may hold, and only the history and playlist files are opened.
- Everything is read on the server and nothing is sent anywhere. The files are not kept, and of each play only the song, artist, album, time and how it ended are read: never the IP addresses, the country or the platform.
- A play is matched to a song of the library by artist and title (no case, accents or punctuation; a "Remastered" note on either side; any credited artist; a compilation by its album). A match is recorded as a listen at the time of the play: complete, skip or play. Nothing is posted to Mastodon.
- Importing the same files again records nothing twice, and the same play in the two shapes of history file is counted once. One import per person at a time.
- A playlist becomes a playlist of mine with the songs that were found; one with a name I already have is left alone, and one with none of its songs found is not made.
- The report counts what was added, what was already there and what is not in the library, and lists the missing artists, most played first.
- Settings gets an Import from Spotify section; the API client can send files.
- English and Italian.
2026-11-14 16:25:58 +01:00

40 lines
1.5 KiB
Go
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

package storage
import (
"strings"
"unicode"
"golang.org/x/text/unicode/norm"
)
// letters that have no accent to take off but are written with the plain letters in a search:
// the key of "Straße" is "strasse" and the one of "Ørsted" is "orsted".
var searchLetters = strings.NewReplacer(
"ς", "σ", "ß", "ss", "æ", "ae", "œ", "oe", "ø", "o", "đ", "d", "ð", "d", "ł", "l", "ı", "i", "ħ", "h",
)
// searchKey is the key a search looks in: matchKey with the accents taken off, so "bjork" finds
// "Björk". It is not the key that identifies artists and albums, which keeps accents: otherwise
// "Beyoncé" and "Beyonce" would be one artist.
//
// Compatibility forms are folded too (the ligature "fi" is "fi", a full-width "A" is "a").
//
// Only the combining marks of the Latin, Greek and Cyrillic range (U+0300 to U+036F) go, so the
// marks that are part of a letter in other scripts (the voicing mark of a kana, a Hangul syllable)
// stay. It is computed here, not in SQL, so SQLite and PostgreSQL agree.
func searchKey(s string) string {
d := norm.NFKD.String(matchKey(s))
var b strings.Builder
b.Grow(len(d))
for _, r := range d {
if r >= 0x0300 && r <= 0x036F && unicode.Is(unicode.Mn, r) {
continue
}
b.WriteRune(r)
}
return searchLetters.Replace(norm.NFC.String(b.String()))
}
// SearchKey is searchKey for the packages that compare names with the ones in the library (the import of a Spotify
// export): the same key, so that "Björk" and "Bjork" are one name there too.
func SearchKey(s string) string { return searchKey(s) }