A scan writes no rows for the names it records #57

Closed
opened 2026-08-03 23:37:11 +02:00 by Claude · 1 comment
Collaborator

A scan writes no rows for the names it records.

Upstream's scan creates a BaseItems row per genre, studio and artist it sees —
GenresValidator, MusicGenresValidator, StudiosValidator — and the by-name
listings are queries over those rows. This server's scan writes none, except the
MusicArtist rows an artist folder produces.

The listings still answer, because they fall back to deriving names from
ItemValues for anything no row stands for. On a library upstream scanned that
fallback contributes nothing (measured: zero orphan values across all five
listings), so a database written by upstream reads identically either way.

On a library this server scanned, the difference is everything the listings
report about a name: a row carries a sort name, an id that survives a rescan,
images, and a date. Derived names have none of that, which is why an id from
/Genres/{name} still disagrees with upstream's.

MILESTONES.md states that a by-name item is never a row here. That is the rule
this issue proposes changing, so the sentence wants revisiting with it.

A scan writes no rows for the names it records. Upstream's scan creates a `BaseItems` row per genre, studio and artist it sees — `GenresValidator`, `MusicGenresValidator`, `StudiosValidator` — and the by-name listings are queries over those rows. This server's scan writes none, except the `MusicArtist` rows an artist folder produces. The listings still answer, because they fall back to deriving names from `ItemValues` for anything no row stands for. On a library upstream scanned that fallback contributes nothing (measured: zero orphan values across all five listings), so a database written by upstream reads identically either way. On a library **this** server scanned, the difference is everything the listings report about a name: a row carries a sort name, an id that survives a rescan, images, and a date. Derived names have none of that, which is why an id from `/Genres/{name}` still disagrees with upstream's. `MILESTONES.md` states that a by-name item is never a row here. That is the rule this issue proposes changing, so the sentence wants revisiting with it.
Author
Collaborator

A scan now writes the rows, as upstream's validators do.

Upstream runs GenresValidator, MusicGenresValidator, StudiosValidator and
ArtistsValidator after a scan. Each takes its names from GetItemValueNames
(ItemValuesMap ⋈ ItemValues, grouped on CleanValue, taking MIN(Value)) and
calls CreateItemByName per name. The genre split is by owner type —
GetGenreNames excludes ItemTypeLookup.MusicGenreTypes, GetMusicGenreNames
requires them.

The id is the part that had to be exact: GetItemByNameId<T>(T.GetPath(name))
makes the path relative to the program data directory, swaps separators,
lowercases it, prefixes the .NET type name, and takes MD5 over UTF-16LE.
GetValidFilename maps " < > | : * ? \ / and the C0 range to spaces, and a
person is filed under the first letter-or-digit of that.

Measured over 4889 tracks:

before after upstream
by-name rows written 0 338 —
MusicGenre rows 0 220 224
MusicArtist rows 67 (folders only) 185 186

All 220 music genre rows are identical to upstream's on id, name, sort name,
presentation key, path and folder flag. 174 of 185 artist rows are identical on
every column; the other 11 differ only on SortName, which is a pre-existing
folding gap — upstream transliterates CJK to pinyin and maps Ø→O, ®→r where
NFD does not — and is present in the before database too. Upstream's four extra
genres come from artist metadata refresh, which is a separate ingest gap.

The id derivation matches even where it degenerates: an artist named ?????
has an empty valid filename, and both servers hash metadata\artists.

A rescan writes nothing — named=0, and DateLastSaved on all 405 rows is
byte-identical to the first run.

What the listings report changed even where their membership did not: every
music genre now carries a real DateCreated where it had the lowest date, and
an Etag where it had none.

The one claim the work could not check itself was IsFolder, being the only
existing read-path output it changed. Confirmed against a running upstream
afterwards: a genre, a music genre, a studio and a by-name artist all omit the
field, while an artist that is a real folder on disk reports true. Both hold
here now.

Left undone and worth their own issues: PeopleValidator and years are still
derived rather than written, this port records no Studios values for the
studio validator to act on, and the sort_name/fold_diacritics gaps above are
now visible in the rows as well as in the items.

A scan now writes the rows, as upstream's validators do. Upstream runs `GenresValidator`, `MusicGenresValidator`, `StudiosValidator` and `ArtistsValidator` after a scan. Each takes its names from `GetItemValueNames` (`ItemValuesMap ⋈ ItemValues`, grouped on `CleanValue`, taking `MIN(Value)`) and calls `CreateItemByName` per name. The genre split is by owner type — `GetGenreNames` excludes `ItemTypeLookup.MusicGenreTypes`, `GetMusicGenreNames` requires them. The id is the part that had to be exact: `GetItemByNameId<T>(T.GetPath(name))` makes the path relative to the program data directory, swaps separators, lowercases it, prefixes the .NET type name, and takes MD5 over UTF-16LE. `GetValidFilename` maps `" < > | : * ? \ /` and the C0 range to spaces, and a person is filed under the first letter-or-digit of that. Measured over 4889 tracks: | | before | after | upstream | |---|---|---|---| | by-name rows written | 0 | **338** | — | | `MusicGenre` rows | 0 | 220 | 224 | | `MusicArtist` rows | 67 (folders only) | 185 | 186 | All 220 music genre rows are identical to upstream's on id, name, sort name, presentation key, path and folder flag. 174 of 185 artist rows are identical on every column; the other 11 differ only on `SortName`, which is a pre-existing folding gap — upstream transliterates CJK to pinyin and maps `Ø→O`, `®→r` where NFD does not — and is present in the before database too. Upstream's four extra genres come from artist metadata refresh, which is a separate ingest gap. The id derivation matches even where it degenerates: an artist named `?????` has an empty valid filename, and both servers hash `metadata\artists`. A rescan writes nothing — `named=0`, and `DateLastSaved` on all 405 rows is byte-identical to the first run. What the listings report changed even where their membership did not: every music genre now carries a real `DateCreated` where it had the lowest date, and an `Etag` where it had none. The one claim the work could not check itself was `IsFolder`, being the only existing read-path output it changed. Confirmed against a running upstream afterwards: a genre, a music genre, a studio and a by-name artist all omit the field, while an artist that is a real folder on disk reports `true`. Both hold here now. Left undone and worth their own issues: `PeopleValidator` and years are still derived rather than written, this port records no `Studios` values for the studio validator to act on, and the `sort_name`/`fold_diacritics` gaps above are now visible in the rows as well as in the items.
Sign in to join this conversation.
No description provided.