A library that is not movies, tvshows or music is configured, listed and never scanned #194

Closed
opened 2026-09-12 08:50:57 +02:00 by Claude · 3 comments
Collaborator

Every milestone is closed and the scanner still reads three kinds of library.
scan::Collection is the whole of what it knows:

// crates/ephyra-app/src/scan/mod.rs:30
pub enum Collection {
    Movies,
    TvShows,
    Music,
}

A library of any other kind is configured, listed, kept — and never scanned.
Two call sites decide it:

  • scan::configured (crates/ephyra-app/src/scan/run/mod.rs:661) pushes the
    kind onto refused where Collection::named answers None, so the library
    contributes no Library to the pass.
  • libraries::reconcile (crates/ephyra-server/src/libraries.rs:291) passes
    Collection::EVERY as the set of kinds a sweep may forget rows for, which is
    what keeps an unscanned library's rows rather than deleting them.

The browse side is already ahead of the scanner and says so:
item::kind::browsed_as (crates/ephyra-core/src/item/kind.rs:126) maps
books, photos, homevideos, musicvideos, boxsets and folders to the
types each lists, under a doc comment reading the kind on a row may be one
this server never scanned
. A database handed those rows renders them; a
directory of the files produces nothing.

What upstream resolves that we do not

One resolver each, under upstream/Emby.Server.Implementations/Library/Resolvers/:

Library kind Upstream
books Books/BookResolver.cs
photos, homevideos PhotoResolver.cs, PhotoAlbumResolver.cs, and upstream/Emby.Photos/PhotoProvider.cs (175 lines) for what a photo's metadata is
musicvideos GenericVideoResolver.cs over MusicVideo
folders, mixed content GenericFolderResolver.cs, and Movies/MovieResolver.cs's mixed-content branch

Emby.Photos/PhotoProvider.cs is the one that is not just a resolver: a photo's
width, height, orientation and EXIF date come off the file, which is work
ephyra-images may already be close to.

Where to start

crates/ephyra-app/src/scan/ is the scanner: mod.rs holds Collection,
tree/ resolves a directory into Nodes, ingest.rs writes the rows, and
run/ drives a pass. Adding a kind means a Collection variant, the naming
rules in ephyra-naming for it, a resolver in tree/, and the row shape in
ingest.rs.

Open, and worth settling before anything is written

  • Whether this is one piece of work or four. Books and photos are each a
    file extension and a naming rule rather than a hierarchy, and are the cheap
    pair; mixed content is the expensive one because it changes how a library's
    root is walked rather than what a leaf resolves to.
  • Which fixture it is diffed against. The reference library is nineteen
    short films and has no book, photo or music video in it, and
    crates/ephyra-harness/README.md is where a diff gets its meaning. A
    scanner for a kind no fixture holds cannot be compared to upstream at all,
    and a test that cannot fail on the fixture it runs against is worse than
    none. So this needs either the reference library to grow a directory per kind
    (which redraws the snapshot — see MILESTONES.md and the rebuild note) or a
    fixture of its own.
Every milestone is closed and the scanner still reads three kinds of library. `scan::Collection` is the whole of what it knows: ```rust // crates/ephyra-app/src/scan/mod.rs:30 pub enum Collection { Movies, TvShows, Music, } ``` A library of any other kind is configured, listed, kept — and never scanned. Two call sites decide it: - `scan::configured` (`crates/ephyra-app/src/scan/run/mod.rs:661`) pushes the kind onto `refused` where `Collection::named` answers `None`, so the library contributes no `Library` to the pass. - `libraries::reconcile` (`crates/ephyra-server/src/libraries.rs:291`) passes `Collection::EVERY` as the set of kinds a sweep may forget rows for, which is what keeps an unscanned library's rows rather than deleting them. The browse side is already ahead of the scanner and says so: `item::kind::browsed_as` (`crates/ephyra-core/src/item/kind.rs:126`) maps `books`, `photos`, `homevideos`, `musicvideos`, `boxsets` and `folders` to the types each lists, under a doc comment reading *the kind on a row may be one this server never scanned*. A database handed those rows renders them; a directory of the files produces nothing. ## What upstream resolves that we do not One resolver each, under `upstream/Emby.Server.Implementations/Library/Resolvers/`: | Library kind | Upstream | |---|---| | `books` | `Books/BookResolver.cs` | | `photos`, `homevideos` | `PhotoResolver.cs`, `PhotoAlbumResolver.cs`, and `upstream/Emby.Photos/PhotoProvider.cs` (175 lines) for what a photo's metadata is | | `musicvideos` | `GenericVideoResolver.cs` over `MusicVideo` | | `folders`, mixed content | `GenericFolderResolver.cs`, and `Movies/MovieResolver.cs`'s mixed-content branch | `Emby.Photos/PhotoProvider.cs` is the one that is not just a resolver: a photo's width, height, orientation and EXIF date come off the file, which is work `ephyra-images` may already be close to. ## Where to start `crates/ephyra-app/src/scan/` is the scanner: `mod.rs` holds `Collection`, `tree/` resolves a directory into `Node`s, `ingest.rs` writes the rows, and `run/` drives a pass. Adding a kind means a `Collection` variant, the naming rules in `ephyra-naming` for it, a resolver in `tree/`, and the row shape in `ingest.rs`. ## Open, and worth settling before anything is written - **Whether this is one piece of work or four.** Books and photos are each a file extension and a naming rule rather than a hierarchy, and are the cheap pair; mixed content is the expensive one because it changes how a library's root is walked rather than what a leaf resolves to. - **Which fixture it is diffed against.** The reference library is nineteen short films and has no book, photo or music video in it, and `crates/ephyra-harness/README.md` is where a diff gets its meaning. A scanner for a kind no fixture holds cannot be compared to upstream at all, and a test that cannot fail on the fixture it runs against is worse than none. So this needs either the reference library to grow a directory per kind (which redraws the snapshot — see `MILESTONES.md` and the rebuild note) or a fixture of its own.
Author
Collaborator

Books and home videos are done in d818759c. Music videos and mixed content
are not, so this stays open.

The fixture question the issue flagged

It has an answer that does not touch the reference. Registering a library is
a write
, and the rule for a write is already written down — its own container
from the pinned image over a cp -a copy of a snapshot, on its own ports. So
cargo xtask reference gen-library-kinds lays a deterministic tree under
reference/var/library-kinds, stands a second container up over a copy of
snapshots/default, registers a books and a homevideos library with every
fetcher off, scans, and writes upstream's rows and the listing they were
resolved from
into crates/ephyra-app/corpus/library-kinds.json.

crates/ephyra-app/tests/library_kinds.rs replays library_tree over that
listing and compares. So it is a real differential against upstream that needs
neither docker nor a disk to run, and the shared reference, its ids and its
snapshots never move.

One finding that changed the scope

photos is not a registerable collection type. CollectionTypeOptions
offers homevideos — "Home Videos (and Photos)" — and nothing else; a photos
library is a legacy stored value. So the home-video library is the photo
library. Both are resolved here, and they part only over a disc rip, which
homevideos resolves and photos does not.

Left, and named rather than hidden

  • Music videos — GenericVideoResolver over a MusicVideo type this
    server does not model at all; only the type name exists.
  • Mixed content and folders — changes how a library's root is walked
    rather than what a leaf resolves to.
  • EXIF (aperture, orientation, camera, date taken) from upstream's photo
    provider: rows carry width and height and no EXIF.
  • The PhotoAlbum folder image upstream copies from the first picture inside.
  • RunTimeTicks on an .epub, which upstream fills with exactly one second.

One test that could not fail

The sixth mutation — removing the rule that a picture is its own Primary —
passed. The fixture could not tell that code from the generic artwork
rules. Going back to the container established what upstream actually does:
17x16.jpg and 17x16.png in one folder each get themselves, a .dng gets
itself despite not being an artwork extension, and a poster.jpg beside them
illustrates the album. The fixture was widened to those cases and the mutation
then failed. Six of the seven failed first time; this is the one that did not.

For the sweep

forget_libraries gains two tests: a books library the configuration still
names keeps its folder and its rows, and one nothing names goes with everything
under it while the film library survives. scanned_kind — the guard on
POST /Library/VirtualFolders — is derived from Collection::EVERY now rather
than a literal, so a new kind follows automatically.

Books and home videos are done in `d818759c`. Music videos and mixed content are not, so this stays open. ### The fixture question the issue flagged It has an answer that does not touch the reference. *Registering a library is a write*, and the rule for a write is already written down — its own container from the pinned image over a `cp -a` copy of a snapshot, on its own ports. So `cargo xtask reference gen-library-kinds` lays a deterministic tree under `reference/var/library-kinds`, stands a second container up over a copy of `snapshots/default`, registers a `books` and a `homevideos` library with every fetcher off, scans, and writes upstream's rows **and the listing they were resolved from** into `crates/ephyra-app/corpus/library-kinds.json`. `crates/ephyra-app/tests/library_kinds.rs` replays `library_tree` over that listing and compares. So it is a real differential against upstream that needs neither docker nor a disk to run, and the shared reference, its ids and its snapshots never move. ### One finding that changed the scope **`photos` is not a registerable collection type.** `CollectionTypeOptions` offers `homevideos` — "Home Videos (and Photos)" — and nothing else; a `photos` library is a legacy stored value. So the home-video library *is* the photo library. Both are resolved here, and they part only over a disc rip, which `homevideos` resolves and `photos` does not. ### Left, and named rather than hidden - **Music videos** — `GenericVideoResolver` over a `MusicVideo` type this server does not model at all; only the type name exists. - **Mixed content and `folders`** — changes how a library's root is walked rather than what a leaf resolves to. - **EXIF** (aperture, orientation, camera, date taken) from upstream's photo provider: rows carry width and height and no EXIF. - The `PhotoAlbum` folder image upstream copies from the first picture inside. - `RunTimeTicks` on an `.epub`, which upstream fills with exactly one second. ### One test that could not fail The sixth mutation — removing the rule that a picture is its own `Primary` — **passed**. The fixture could not tell that code from the generic artwork rules. Going back to the container established what upstream actually does: `17x16.jpg` and `17x16.png` in one folder each get themselves, a `.dng` gets itself despite not being an artwork extension, and a `poster.jpg` beside them illustrates the album. The fixture was widened to those cases and the mutation then failed. Six of the seven failed first time; this is the one that did not. ### For the sweep `forget_libraries` gains two tests: a books library the configuration still names keeps its folder and its rows, and one nothing names goes with everything under it while the film library survives. `scanned_kind` — the guard on `POST /Library/VirtualFolders` — is derived from `Collection::EVERY` now rather than a literal, so a new kind follows automatically.
Author
Collaborator

Music videos are done, on main as 89dcdca3 and a441bc3d. Mixed content and folders are not, for reasons that turned out to change what is left on this issue — see the bottom.

Music videos

Kind::MusicVideo end to end: type name, from_type_name, not a folder, media type Video, unrated type Music (upstream/MediaBrowser.Controller/Entities/MusicVideo.cs:25, GetBlockUnratedType() => UnratedItem.Music), held_as_video; Collection::MusicVideos in EVERY and REGISTERABLE.

Upstream resolves one with the same walk as a film — MovieResolver.cs:200 ResolveVideos<MusicVideo>(parent, files, true, ct, false) against :227 ResolveVideos<Movie>(parent, files, true, ct, true), differing only in parseName — so scan/tree/movies.rs now takes a Leaves { kind, parse_name, named_by_its_folder } and serves both, one function where upstream has one class instantiated twice. (The hint in this issue's description pointed at GenericVideoResolver, which is wrong: that class is only ever instantiated by ExtraResolver.cs:35, for trailers.)

Everything that follows from a new kind went with it: providers::from_path, the artwork name lists, a music genre's collage, the library-options dialog row, and the sweep's scanned-kinds list.

Three film bugs the new fixture found

The generated answer key (cargo xtask reference gen-library-kinds) now registers four libraries — books, homevideos, musicvideos and movies. The film library is there because three changes touched the film path and nothing else could check them. It immediately disagreed with upstream three times:

  1. A folder several films share kept no row. The rule was "a folder with no film in it"; upstream's is "a folder exactly one film collapsed into keeps no row, every other folder does" (FindMovie returns null when result.Items.Count != 1). Films in a folder that keeps a row now hang from it.
  2. An extras folder counted as a folder a film did not have to itself, so Extras Beside (2011)/Feature.mp4 was not named after its folder. MovieResolver.cs:427 steps over extras first.
  3. No year was read off a name or the folder above it (Movie.BeforeMetadataRefresh / MusicVideo.BeforeMetadataRefresh): upstream files Yearless Folder (2014)/Untitled.mp4 with ProductionYear 2014 and we filed none.

A fourth came out of the review: a multi-disc rip read no year at all, because Video.ContainingFolderPath for a stacked video is the release folder, not the disc (Video.cs:201-218). Upstream files Release (1999)/Feature - cd 1|2/VIDEO_TS/ as Feature, year 1999; the answer key now says so.

What the review caught

Three of its findings were rules with no test — the shape that cost this issue a round already:

  • Collection::MusicVideos in REGISTERABLE was unasserted. Deleting it left all 3241 tests green, and it is the only thing POST /Library/VirtualFolders?collectionType=musicvideos consults — the switch that turns the whole feature on from the dashboard. Now pinned by a test that posts all six registerable kinds and reads the configuration back.
  • reads_a_year(Kind::Movie) was unasserted, because the generated tree's film half contained no disc rip at all. It does now.
  • disc_names omitted BoxSet where LocalImageProvider.cs:221 has item is Video || item is BoxSet — a wrong row left in a table transcribed from upstream.

Each was mutated and watched to fail. 3260 tests, clippy silent, fmt clean. The regenerated corpus is insertions only: 807 added lines across both commits, nothing moved or removed, so every row the port and upstream already agreed on still agrees.

Filed rather than buried

  • #205 — a stack's later parts do not inherit the owner's mixed-folder flag (Video.cs:691)
  • #206 — a film alone in a disc folder takes the wrong folder's name, and a test asserts our answer as though it were upstream's
  • #207 — a scan files one row where upstream files a row per alternate version; VideoInfo::alternate_versions is read nowhere

Also worth knowing: from_type_name now answering for MusicVideo changes two call sites for rows seeded from a real Jellyfin database — ephyra-db/src/items/writes.rs:37 (the sort-name rule) and routes/library/remote_images.rs:225 (which providers a music-video row is offered). Both look benign; neither has a test.

What is left on this issue, and a correction to its scope

folders is not a library kind and never was. CollectionTypeOptions — what LibraryStructureController.cs:87 binds — has no folders member, so no .collection file can name one. CollectionType.folders exists only for GetNamedView("folders"), the Folders user view. There is nothing to scan, and our read-side browsed_as("folders") is already right. This part of the issue can be struck.

Mixed content is a resolver chain, not a resolver. A mixed library writes mixed.collection, Enum.TryParse<CollectionType> fails on it, and CollectionType is null — at which point every resolver's collectionType is null branch comes into play: SeriesResolver promotes a directory holding tvshow.nfo or a season folder, MovieResolver.ResolveMultiple aborts when it sees tvshow.nfo/season.nfo beside the files, and ResolvePaths tries multi-item resolvers first and falls back per file, ordered by ResolverPriority. Our tree/ picks one function per collection kind up front, so this is a different shape for the module rather than another arm — it deserves its own issue rather than a fourth round here.

So: with folders struck and mixed content split out, music videos and box sets were the remainder. Box sets are still open.

Music videos are done, on `main` as `89dcdca3` and `a441bc3d`. Mixed content and `folders` are not, for reasons that turned out to change what is left on this issue — see the bottom. ## Music videos `Kind::MusicVideo` end to end: type name, `from_type_name`, not a folder, media type Video, unrated type Music (`upstream/MediaBrowser.Controller/Entities/MusicVideo.cs:25`, `GetBlockUnratedType() => UnratedItem.Music`), `held_as_video`; `Collection::MusicVideos` in `EVERY` and `REGISTERABLE`. Upstream resolves one with the *same* walk as a film — `MovieResolver.cs:200` `ResolveVideos<MusicVideo>(parent, files, true, ct, false)` against `:227` `ResolveVideos<Movie>(parent, files, true, ct, true)`, differing only in `parseName` — so `scan/tree/movies.rs` now takes a `Leaves { kind, parse_name, named_by_its_folder }` and serves both, one function where upstream has one class instantiated twice. (The hint in this issue's description pointed at `GenericVideoResolver`, which is wrong: that class is only ever instantiated by `ExtraResolver.cs:35`, for trailers.) Everything that follows from a new kind went with it: `providers::from_path`, the artwork name lists, a music genre's collage, the library-options dialog row, and the sweep's scanned-kinds list. ## Three film bugs the new fixture found The generated answer key (`cargo xtask reference gen-library-kinds`) now registers four libraries — books, homevideos, musicvideos and **movies**. The film library is there because three changes touched the film path and nothing else could check them. It immediately disagreed with upstream three times: 1. **A folder several films share kept no row.** The rule was "a folder with no film in it"; upstream's is "a folder exactly one film collapsed into keeps no row, every other folder does" (`FindMovie` returns null when `result.Items.Count != 1`). Films in a folder that keeps a row now hang from it. 2. **An extras folder counted as a folder a film did not have to itself**, so `Extras Beside (2011)/Feature.mp4` was not named after its folder. `MovieResolver.cs:427` steps over extras first. 3. **No year was read off a name or the folder above it** (`Movie.BeforeMetadataRefresh` / `MusicVideo.BeforeMetadataRefresh`): upstream files `Yearless Folder (2014)/Untitled.mp4` with `ProductionYear` 2014 and we filed none. A fourth came out of the review: a **multi-disc rip** read no year at all, because `Video.ContainingFolderPath` for a stacked video is the *release* folder, not the disc (`Video.cs:201-218`). Upstream files `Release (1999)/Feature - cd 1|2/VIDEO_TS/` as `Feature`, year 1999; the answer key now says so. ## What the review caught Three of its findings were rules with no test — the shape that cost this issue a round already: - **`Collection::MusicVideos` in `REGISTERABLE` was unasserted.** Deleting it left all 3241 tests green, and it is the only thing `POST /Library/VirtualFolders?collectionType=musicvideos` consults — *the switch that turns the whole feature on from the dashboard*. Now pinned by a test that posts all six registerable kinds and reads the configuration back. - **`reads_a_year(Kind::Movie)` was unasserted**, because the generated tree's film half contained no disc rip at all. It does now. - **`disc_names` omitted `BoxSet`** where `LocalImageProvider.cs:221` has `item is Video || item is BoxSet` — a wrong row left in a table transcribed from upstream. Each was mutated and watched to fail. `3260` tests, clippy silent, fmt clean. The regenerated corpus is insertions only: 807 added lines across both commits, nothing moved or removed, so every row the port and upstream already agreed on still agrees. ## Filed rather than buried - #205 — a stack's later parts do not inherit the owner's mixed-folder flag (`Video.cs:691`) - #206 — a film alone in a disc folder takes the wrong folder's name, **and a test asserts our answer as though it were upstream's** - #207 — a scan files one row where upstream files a row per alternate version; `VideoInfo::alternate_versions` is read nowhere Also worth knowing: `from_type_name` now answering for `MusicVideo` changes two call sites for rows seeded from a real Jellyfin database — `ephyra-db/src/items/writes.rs:37` (the sort-name rule) and `routes/library/remote_images.rs:225` (which providers a music-video row is offered). Both look benign; neither has a test. ## What is left on this issue, and a correction to its scope **`folders` is not a library kind and never was.** `CollectionTypeOptions` — what `LibraryStructureController.cs:87` binds — has no `folders` member, so no `.collection` file can name one. `CollectionType.folders` exists only for `GetNamedView("folders")`, the *Folders* user view. There is nothing to scan, and our read-side `browsed_as("folders")` is already right. This part of the issue can be struck. **Mixed content is a resolver chain, not a resolver.** A `mixed` library writes `mixed.collection`, `Enum.TryParse<CollectionType>` fails on it, and `CollectionType` is null — at which point every resolver's `collectionType is null` branch comes into play: `SeriesResolver` promotes a directory holding `tvshow.nfo` or a season folder, `MovieResolver.ResolveMultiple` aborts when it sees `tvshow.nfo`/`season.nfo` beside the files, and `ResolvePaths` tries multi-item resolvers first and falls back per file, ordered by `ResolverPriority`. Our `tree/` picks one function per collection kind up front, so this is a different shape for the module rather than another arm — it deserves its own issue rather than a fourth round here. So: with `folders` struck and mixed content split out, music videos and box sets were the remainder. Box sets are still open.
Author
Collaborator

Closing this. Every kind in its table is scanned, with one struck and the rest split out:

  • books — done, and what a book's file still does not say is #210
  • photos, homevideos — done; the EXIF columns Emby.Photos/PhotoProvider.cs fills, the PhotoAlbum folder image and LibraryOptions.EnablePhotos are #209
  • musicvideos — done (89dcdca3, a441bc3d)
  • folders — struck. It is not a library kind: CollectionTypeOptions, which LibraryStructureController.cs:87 binds, has no such member, so no .collection file can name one. CollectionType.folders exists only for GetNamedView("folders"), the Folders user view. Our read-side browsed_as("folders") is already right and there is nothing to scan.
  • mixed content — #208. Not another resolver: a mixed library leaves CollectionType null, and null is the case every resolver has its own branch for, tried in ResolverPriority order. That is a change to how a root is walked, not another arm on a match.

Also filed from the two review rounds on this issue: #205, #206, #207.

Closing this. Every kind in its table is scanned, with one struck and the rest split out: - **books** — done, and what a book's file still does not say is #210 - **photos**, **homevideos** — done; the EXIF columns `Emby.Photos/PhotoProvider.cs` fills, the `PhotoAlbum` folder image and `LibraryOptions.EnablePhotos` are #209 - **musicvideos** — done (`89dcdca3`, `a441bc3d`) - **`folders`** — **struck**. It is not a library kind: `CollectionTypeOptions`, which `LibraryStructureController.cs:87` binds, has no such member, so no `.collection` file can name one. `CollectionType.folders` exists only for `GetNamedView("folders")`, the *Folders* user view. Our read-side `browsed_as("folders")` is already right and there is nothing to scan. - **mixed content** — #208. Not another resolver: a `mixed` library leaves `CollectionType` null, and null is the case every resolver has its own branch for, tried in `ResolverPriority` order. That is a change to how a root is walked, not another arm on a match. Also filed from the two review rounds on this issue: #205, #206, #207.
Sign in to join this conversation.
No description provided.