A database scanned by an older build keeps what that build did not write #28

Closed
opened 2026-08-02 18:22:10 +02:00 by Claude · 1 comment
Collaborator

Nothing backfills a table a previous scan left empty.

pass() skips an item whose file has not moved (crates/jellyfin-app/src/scan/run.rs:123),
comparing the size and mtime the previous scan itself recorded. So on a server that
already scanned its library, every item is unchanged, filing.items is empty, and
whatever a newer build learned to write is never written for it.

Concretely: a database scanned before #9 has every BaseItems row and no ItemValues.
Restart on a build that writes them and /Genres, /Artists and /MusicGenres keep
answering an empty list until each individual file's size or mtime changes.

The same holds for every table this series adds — #10 Chapters, #11
BaseItemProviders, #12 Peoples, #13 AttachmentStreamInfos — and for the corrected
values in #3, #5, #7 and #8, which only reach a row that gets rewritten.

Upstream's answer is a migration: Jellyfin.Server/Migrations/Routines/ carries one per
change of this kind, and 20260610120000_RefreshCleanNamesAndValues is exactly this
case. This server applies nothing on top of the baseline schema, so there is nowhere
for such a step to live yet.

Found while reviewing #9. Deciding what a migration is here — a version stamp, a
forced rewrite, or a rescan that ignores the unchanged check once — is the work.

Nothing backfills a table a previous scan left empty. `pass()` skips an item whose file has not moved (`crates/jellyfin-app/src/scan/run.rs:123`), comparing the size and mtime the previous scan itself recorded. So on a server that already scanned its library, every item is unchanged, `filing.items` is empty, and whatever a newer build learned to write is never written for it. Concretely: a database scanned before #9 has every `BaseItems` row and no `ItemValues`. Restart on a build that writes them and `/Genres`, `/Artists` and `/MusicGenres` keep answering an empty list until each individual file's size or mtime changes. The same holds for every table this series adds — #10 `Chapters`, #11 `BaseItemProviders`, #12 `Peoples`, #13 `AttachmentStreamInfos` — and for the corrected values in #3, #5, #7 and #8, which only reach a row that gets rewritten. Upstream's answer is a migration: `Jellyfin.Server/Migrations/Routines/` carries one per change of this kind, and `20260610120000_RefreshCleanNamesAndValues` is exactly this case. This server applies nothing on top of the baseline schema, so there is nowhere for such a step to live yet. Found while reviewing #9. Deciding what a migration is here — a version stamp, a forced rewrite, or a rescan that ignores the unchanged check once — is the work.
Author
Collaborator

Closing without the fix, and recording why.

Not a stamp of our own

The obvious answer — a number naming what this build's scan knows how to write,
kept beside the database, a mismatch forcing one full re-read — was built and
measured (5387 rows re-read on a stale stamp, nothing on a current one) and
then thrown away. It is a second source of truth that upstream's own migrations
would never touch, and it drifts the moment the pinned commit moves.

The direction is the opposite: pull upstream more often and inherit its
migrations, so that a database this server wrote stays one upstream can reason
about.

What is already true

Upstream records every migration in __EFMigrationsHistory — the code routines
in Jellyfin.Server/Migrations/Routines/ as much as the schema ones, keyed by
the timestamped id. A database this server creates carries all 93 of them, the
same ids in the same order as the reference instance's own, because the
baseline schema is created that way (D21).

So at the pinned commit there is nothing to apply. When the pin moves, the new
ids are ones the database does not carry, and those are what should run —
including the refresh routines that are exactly this issue's shape
(20260610120000_RefreshCleanNamesAndValues,
20260722120000_RefreshForcedSortNames,
20260723120000_RecomputeSeriesPresentationKey).

The work, when it is wanted

  • xtask gen-migrations writing the pinned upstream's migration ids, so a pull
    surfaces new ones as a reviewable diff rather than as a silent gap.
  • A runner reading __EFMigrationsHistory, applying this server's routine for
    each id it does not carry, and recording the row.
  • What to do about an id with no routine here. Refusing to start names the gap
    and makes a pull a deliberate step; recording it anyway keeps the server up
    and accepts a database upstream would have migrated.

What stays broken meanwhile

A database written by an older build of this server keeps whatever that build
left empty, until an upstream migration happens to refresh those rows or the
files themselves move. That is the gap this issue opened for, and it is real —
it is just not one a private version stamp should paper over.

Closing without the fix, and recording why. ## Not a stamp of our own The obvious answer — a number naming what this build's scan knows how to write, kept beside the database, a mismatch forcing one full re-read — was built and measured (5387 rows re-read on a stale stamp, nothing on a current one) and then thrown away. It is a second source of truth that upstream's own migrations would never touch, and it drifts the moment the pinned commit moves. The direction is the opposite: pull upstream more often and inherit its migrations, so that a database this server wrote stays one upstream can reason about. ## What is already true Upstream records every migration in `__EFMigrationsHistory` — the code routines in `Jellyfin.Server/Migrations/Routines/` as much as the schema ones, keyed by the timestamped id. A database this server creates carries all 93 of them, the same ids in the same order as the reference instance's own, because the baseline schema is created that way (D21). So at the pinned commit there is nothing to apply. When the pin moves, the new ids are ones the database does not carry, and those are what should run — including the refresh routines that are exactly this issue's shape (`20260610120000_RefreshCleanNamesAndValues`, `20260722120000_RefreshForcedSortNames`, `20260723120000_RecomputeSeriesPresentationKey`). ## The work, when it is wanted - `xtask gen-migrations` writing the pinned upstream's migration ids, so a pull surfaces new ones as a reviewable diff rather than as a silent gap. - A runner reading `__EFMigrationsHistory`, applying this server's routine for each id it does not carry, and recording the row. - What to do about an id with no routine here. Refusing to start names the gap and makes a pull a deliberate step; recording it anyway keeps the server up and accepts a database upstream would have migrated. ## What stays broken meanwhile A database written by an older build of *this* server keeps whatever that build left empty, until an upstream migration happens to refresh those rows or the files themselves move. That is the gap this issue opened for, and it is real — it is just not one a private version stamp should paper over.
Sign in to join this conversation.
No description provided.