alex/ephyra
Watch
1
0
Fork
You've already forked ephyra
 0
Code Issues 2 Pull requests Projects Releases Packages 1 Wiki Activity Actions

The generated images upstream draws for a folder with no picture of its own #110

New issue
Closed
opened 2026-08-13 22:59:42 +02:00 by Claude · 2 comments
Claude commented 2026-08-13 22:59:42 +02:00
Collaborator
Copy link

We draw two of upstream's seven dynamic image providers — the genre collage and the music genre one. The other five leave a folder with no picture where upstream shows a mosaic of what is inside it:

  • ArtistImageProvider — an artist with no photograph
  • CollectionFolderImageProvider — a library's own tile on the home screen
  • DynamicImageProvider (user view) — the same for a view
  • PlaylistImageProvider — a playlist's tile
  • CollectionImageProvider — a box set's tile

All five subclass BaseDynamicImageProvider (Emby.Server.Implementations/Images/), which is the code the collage already ports — so this is mostly a matter of which items each one draws from and how many, plus registering them where the collage is registered (crates/jellyfin-server/src/main.rs:734).

Separate and more interesting: the three that read the media itself rather than composing other pictures, in MediaBrowser.Providers/MediaInfo/.

  • AudioImageProvider — the cover art embedded in a track, which is how most music libraries actually carry artwork
  • EmbeddedImageProvider — an attached picture in a video container (mkv cover art)
  • VideoImageProvider — a frame from the video as a primary image where nothing else supplied one

scan/images.rs says it in its own module doc: local only — an image already on disk. The prober is explicitly told not to read cover art (crates/jellyfin-ffmpeg/src/prober.rs:288), which is the right call for a probe and means the extraction needs its own pass.

Where to start

Two halves, and the second is the interesting one.

The composed five. ArtistImageProvider, CollectionFolderImageProvider,
DynamicImageProvider (user view), PlaylistImageProvider and
CollectionImageProvider all subclass BaseDynamicImageProvider
(Emby.Server.Implementations/Images/) — which is the code the genre collage
already ports. So this is mostly which items each one draws from, and how
many, plus registering them where the collage is registered
(crates/jellyfin-server/src/main.rs:734).

The three that read the media. MediaBrowser.Providers/MediaInfo/:
AudioImageProvider (cover art embedded in a track — how most music libraries
actually carry artwork), EmbeddedImageProvider (an attached picture in a video
container), VideoImageProvider (a frame, where nothing else supplied one).
crates/jellyfin-app/src/scan/images.rs says in its own module doc that it is
local-only, and the prober is deliberately told not to read cover art
(crates/jellyfin-ffmpeg/src/prober.rs:288) — correctly, since a cover is
megabytes and a probe is meant to be quick. So the extraction needs a pass of
its own rather than a flag on the probe.

How to know it is done

diff-db on BaseItemImageInfos over a library both servers scanned, and the
files compared on bytes. The blurhash comes with them — it is a property of
Skia's resampler, and M10's port is the thing that makes a generated picture's
hash agree at all.

House rules

CLAUDE.md is the short version and these are the parts that bite: tests run
under cargo nextest; upstream's source is the checkout in upstream/ at the
pin (v12.0-rc5, 6d501ba4) and not the clone beside the repository;
generated files are regenerated rather than edited; a comment says what the code
does, not how it got there; and Dn references stay out of code and comments.
ARCHITECTURE.md decides which crate a thing goes in and which dependency edges
are illegal, DECISIONS.md says why a rule exists.

The reference pair is cargo xtask reference up (and reference provision,
which is idempotent); a run is
cargo run -p jellyfin-harness -- replay --upstream URL --ours URL --scenario FILE --out DIR,
with diff-db for the two databases. crates/jellyfin-harness/README.md says
what a diff means and what an ignore entry has to carry — every entry needs a
reason and an expiry milestone, and an adds-only history is the smell the risk
table names.

We draw two of upstream's seven dynamic image providers — the genre collage and the music genre one. The other five leave a folder with no picture where upstream shows a mosaic of what is inside it: - `ArtistImageProvider` — an artist with no photograph - `CollectionFolderImageProvider` — a library's own tile on the home screen - `DynamicImageProvider` (user view) — the same for a view - `PlaylistImageProvider` — a playlist's tile - `CollectionImageProvider` — a box set's tile All five subclass `BaseDynamicImageProvider` (`Emby.Server.Implementations/Images/`), which is the code the collage already ports — so this is mostly a matter of which items each one draws from and how many, plus registering them where the collage is registered (`crates/jellyfin-server/src/main.rs:734`). Separate and more interesting: the three that read the media itself rather than composing other pictures, in `MediaBrowser.Providers/MediaInfo/`. - `AudioImageProvider` — the cover art embedded in a track, which is how most music libraries actually carry artwork - `EmbeddedImageProvider` — an attached picture in a video container (mkv cover art) - `VideoImageProvider` — a frame from the video as a primary image where nothing else supplied one `scan/images.rs` says it in its own module doc: *local only — an image already on disk*. The prober is explicitly told not to read cover art (`crates/jellyfin-ffmpeg/src/prober.rs:288`), which is the right call for a probe and means the extraction needs its own pass. --- ## Where to start Two halves, and the second is the interesting one. **The composed five.** `ArtistImageProvider`, `CollectionFolderImageProvider`, `DynamicImageProvider` (user view), `PlaylistImageProvider` and `CollectionImageProvider` all subclass `BaseDynamicImageProvider` (`Emby.Server.Implementations/Images/`) — which is the code the genre collage already ports. So this is mostly *which items each one draws from, and how many*, plus registering them where the collage is registered (`crates/jellyfin-server/src/main.rs:734`). **The three that read the media.** `MediaBrowser.Providers/MediaInfo/`: `AudioImageProvider` (cover art embedded in a track — how most music libraries actually carry artwork), `EmbeddedImageProvider` (an attached picture in a video container), `VideoImageProvider` (a frame, where nothing else supplied one). `crates/jellyfin-app/src/scan/images.rs` says in its own module doc that it is local-only, and the prober is deliberately told not to read cover art (`crates/jellyfin-ffmpeg/src/prober.rs:288`) — correctly, since a cover is megabytes and a probe is meant to be quick. So the extraction needs a pass of its own rather than a flag on the probe. ## How to know it is done `diff-db` on `BaseItemImageInfos` over a library both servers scanned, and the files compared on bytes. The blurhash comes with them — it is a property of Skia's resampler, and M10's port is the thing that makes a generated picture's hash agree at all. ### House rules `CLAUDE.md` is the short version and these are the parts that bite: tests run under `cargo nextest`; upstream's source is the checkout in `upstream/` **at the pin** (`v12.0-rc5`, `6d501ba4`) and not the clone beside the repository; generated files are regenerated rather than edited; a comment says what the code does, not how it got there; and `Dn` references stay out of code and comments. `ARCHITECTURE.md` decides which crate a thing goes in and which dependency edges are illegal, `DECISIONS.md` says why a rule exists. The reference pair is `cargo xtask reference up` (and `reference provision`, which is idempotent); a run is `cargo run -p jellyfin-harness -- replay --upstream URL --ours URL --scenario FILE --out DIR`, with `diff-db` for the two databases. `crates/jellyfin-harness/README.md` says what a diff means and what an ignore entry has to carry — every entry needs a reason and an expiry milestone, and an adds-only history is the smell the risk table names.
Claude commented 2026-08-16 15:15:37 +02:00
Author
Collaborator
Copy link

One of the eight landed in batch D, on main. Staying open for the other seven,
with what was measured about each below so the next pass does not re-derive it.

Landed: CollectionFolderImageProvider

The tile a library shows on the home screen. jellyfin-images grew a second
drawing shape (Drawn::Dimmed beside Drawn::Quarters), jellyfin-db grew
libraries_to_illustrate, and libraries.rs draws it where the genre collage is
drawn. Measured on a copy of the enriched snapshot with upstream's two library
image rows deleted and our scan re-run:

Path Type Width Height
upstream enriched …/library/e5/e563…/poster.png Primary 960 540
ours same path Primary 960 540
upstream enrichedshows …/library/4e/4e4b…/poster.png Primary 960 540
ours same path Primary 960 540

Same backdrop, same scaling, same 0x78 black wash.

The blur hash cannot agree, and this is the finding. BuildThumbCollage is
not a collage: it takes the first backdrop that decodes, scales it, washes the
canvas with half-opaque black, and then draws the library's name across the
middle at 112 points in SkiaEncoder.DefaultTypeFace — _typefaces.Last(),
whatever the host's fontconfig answers with. Upstream's own enriched tile is
dominated by the word "enriched" in white, and two upstream installs on different
base images do not draw the same tile either. Matching it means committing a font
and a rasteriser. What this server draws is the picture underneath, and
collage.rs and libraries.rs say so in their doc comments.

Also measured, and both change what the remaining work is:

  • CollectionFolderImageProvider and DynamicImageProvider pick their sources
    with OrderBy: Random / Shuffle(). Even without the text, which backdrop
    upstream used is not reproducible. Ours takes a stable order and says so.
  • ArtistImageProvider draws nothing at all. GetItemsWithImages returns
    Array.Empty<BaseItem>(), so an artist with no photograph gets none from
    upstream either. There is no work here, only a fact — that is seven remaining,
    not eight.
  • An empty IncludeItemTypes upstream is the absence of a type filter, not a
    filter that admits nothing     (BaseItemRepository.cs:185, and line 218 the
    same way). So CollectionType.folders returning [] from
    GetBaseItemKindsForCollectionType means a folders library's tile is drawn
    from everything inside it, not that upstream draws none.

The review round is what made the query upstream's: it now narrows by
GetBaseItemKindsForCollectionType with the music → MusicArtist override at
the call site where upstream applies it, requires a candidate to carry a
Primary, and does Backdrop→Primary→Thumb as a per-item CASE rather than a
global ORDER BY. Without the type narrowing a tvshows library drew an episode
still where upstream draws the series poster. crates/jellyfin-db/tests/library_tile.rs
is seven cases, one per rule.

A note for whoever writes the next one: the thumbnail branch of
GetStripCollageImagePaths is unreachable for a library tile — a candidate
has to carry a Primary, so there is always a poster to fall back to before the
thumbnail is reached.

The seven, and what each needs

Provider What it is What stands in the way
DynamicImageProvider (user view) The same dimmed-plus-text tile, for a UserView This server files no UserView rows — the views it serves are computed. Nothing to hang it on, and the only such row in the fixture is Live TV, which upstream's Supports refuses anyway
PlaylistImageProvider 600×600 square from the playlist's members, in order Deterministic and small, reusing the existing square. No playlist in either fixture carries an image, so it would land with a unit test and nothing that can fail
CollectionImageProvider A file copy of the first member's Primary — byte-exact by construction Gated on item.IsLocked, which this server never sets. It would be dead code
AudioImageProvider Cover art embedded in a track — how most music libraries carry artwork No fixture coverage at all: the generated music files carry no embedded picture
EmbeddedImageProvider An attached picture in a video container Neither .mkv carries an attached_pic
VideoImageProvider A frame at 10% as a poster where nothing else supplied one reference/README.md records that provision deliberately turns the screen grabber off server-wide. Implementing it creates rows the reference, as configured, does not have. Cheap now: it is Stills::cut at one offset, and #99's intervals.rs proved the plumbing
Drawing the library name 112pt, white, centred after the wash, with right-to-left detection for Arabic, Hebrew, Syriac and Thaana A committed font and a rasteriser. Everything else about the tile already agrees

The precondition for the three media readers is a fixture that carries embedded
artwork — growing make-sample-media.sh to attach a cover to the mp3, the flac
and one .mkv. Without it none of them is measurable.

The blur hash and the ignore list

No entry was opened for it, and the reason is in
crates/jellyfin-harness/README.md instead: Exemption cannot name rows, only
columns, so an entry would hide Blurhash for the ~120 fetched pictures whose
hashes do agree in order to cover two CollectionFolder rows — and no run has
produced the difference, because both snapshots already carry upstream's tiles
and either server leaves a row that has a picture alone. That is filed as #151.

One of the eight landed in batch D, on `main`. Staying open for the other seven, with what was measured about each below so the next pass does not re-derive it. ## Landed: `CollectionFolderImageProvider` The tile a library shows on the home screen. `jellyfin-images` grew a second drawing shape (`Drawn::Dimmed` beside `Drawn::Quarters`), `jellyfin-db` grew `libraries_to_illustrate`, and `libraries.rs` draws it where the genre collage is drawn. Measured on a copy of the `enriched` snapshot with upstream's two library image rows deleted and our scan re-run: | | Path | Type | Width | Height | |---|---|---|---|---| | upstream `enriched` | `…/library/e5/e563…/poster.png` | Primary | 960 | 540 | | ours | same path | Primary | 960 | 540 | | upstream `enrichedshows` | `…/library/4e/4e4b…/poster.png` | Primary | 960 | 540 | | ours | same path | Primary | 960 | 540 | Same backdrop, same scaling, same `0x78` black wash. **The blur hash cannot agree, and this is the finding.** `BuildThumbCollage` is not a collage: it takes the first backdrop that decodes, scales it, washes the canvas with half-opaque black, and then **draws the library's name across the middle at 112 points** in `SkiaEncoder.DefaultTypeFace` — `_typefaces.Last()`, whatever the host's fontconfig answers with. Upstream's own `enriched` tile is dominated by the word "enriched" in white, and two upstream installs on different base images do not draw the same tile either. Matching it means committing a font and a rasteriser. What this server draws is the picture underneath, and `collage.rs` and `libraries.rs` say so in their doc comments. Also measured, and both change what the remaining work is: - `CollectionFolderImageProvider` and `DynamicImageProvider` pick their sources with `OrderBy: Random` / `Shuffle()`. Even without the text, *which* backdrop upstream used is not reproducible. Ours takes a stable order and says so. - **`ArtistImageProvider` draws nothing at all.** `GetItemsWithImages` returns `Array.Empty<BaseItem>()`, so an artist with no photograph gets none from upstream either. There is no work here, only a fact — that is seven remaining, not eight. - **An empty `IncludeItemTypes` upstream is the absence of a type filter, not a filter that admits nothing** (`BaseItemRepository.cs:185`, and line 218 the same way). So `CollectionType.folders` returning `[]` from `GetBaseItemKindsForCollectionType` means a `folders` library's tile is drawn from everything inside it, not that upstream draws none. The review round is what made the query upstream's: it now narrows by `GetBaseItemKindsForCollectionType` with the `music → MusicArtist` override at the call site where upstream applies it, requires a candidate to carry a `Primary`, and does Backdrop→Primary→Thumb as a **per-item** `CASE` rather than a global `ORDER BY`. Without the type narrowing a tvshows library drew an episode still where upstream draws the series poster. `crates/jellyfin-db/tests/library_tile.rs` is seven cases, one per rule. A note for whoever writes the next one: the thumbnail branch of `GetStripCollageImagePaths` is **unreachable for a library tile** — a candidate has to carry a `Primary`, so there is always a poster to fall back to before the thumbnail is reached. ## The seven, and what each needs | Provider | What it is | What stands in the way | |---|---|---| | `DynamicImageProvider` (user view) | The same dimmed-plus-text tile, for a `UserView` | This server files no `UserView` rows — the views it serves are computed. Nothing to hang it on, and the only such row in the fixture is Live TV, which upstream's `Supports` refuses anyway | | `PlaylistImageProvider` | 600×600 square from the playlist's members, in order | Deterministic and small, reusing the existing square. No playlist in either fixture carries an image, so it would land with a unit test and nothing that can fail | | `CollectionImageProvider` | **A file copy** of the first member's `Primary` — byte-exact by construction | Gated on `item.IsLocked`, which this server never sets. It would be dead code | | `AudioImageProvider` | Cover art embedded in a track — how most music libraries carry artwork | **No fixture coverage at all**: the generated music files carry no embedded picture | | `EmbeddedImageProvider` | An attached picture in a video container | Neither `.mkv` carries an `attached_pic` | | `VideoImageProvider` | A frame at 10% as a poster where nothing else supplied one | `reference/README.md` records that `provision` deliberately turns the screen grabber **off** server-wide. Implementing it creates rows the reference, as configured, does not have. Cheap now: it is `Stills::cut` at one offset, and #99's `intervals.rs` proved the plumbing | | Drawing the library name | 112pt, white, centred after the wash, with right-to-left detection for Arabic, Hebrew, Syriac and Thaana | A committed font and a rasteriser. Everything else about the tile already agrees | The precondition for the three media readers is a fixture that carries embedded artwork — growing `make-sample-media.sh` to attach a cover to the mp3, the flac and one `.mkv`. Without it none of them is measurable. ## The blur hash and the ignore list No entry was opened for it, and the reason is in `crates/jellyfin-harness/README.md` instead: `Exemption` cannot name rows, only columns, so an entry would hide `Blurhash` for the ~120 fetched pictures whose hashes do agree in order to cover two `CollectionFolder` rows — and **no run has produced the difference**, because both snapshots already carry upstream's tiles and either server leaves a row that has a picture alone. That is filed as #151.
Claude commented 2026-08-16 17:03:24 +02:00
Author
Collaborator
Copy link

Landed in 915c37f (the composed half) and 4ccdb3d (the three that read the
media).

The composed five

Two of the five draw nothing here, and both because upstream draws nothing
either:

  • ArtistImageProvider asks for no items at all. GetItemsWithImages
    returns Array.Empty<BaseItem>() at the pin — the query is commented out
    above a TODO — so CreateImage is handed an empty collection and returns
    null. An artist with no photograph keeps none upstream.
  • DynamicImageProvider draws for a UserView, and it only supports the
    three collection-strip types. This server synthesizes its views from the
    library folders (jellyfin_app::library::views) and files no UserView row,
    so there is nothing for it to run against.

Both are recorded in the note beside the runner rather than left as a gap.

The other two are ported:

  • PlaylistImageProvider — the same square of quarters a genre gets, drawn
    from the first four items the playlist names.
  • CollectionImageProvider — CreateSingleImage, so the first picture is
    copied whole and keeps its own extension, and only while IsLocked, which
    is upstream's way of keeping a drawn tile from landing ahead of a fetched one.

Each named item offers one picture in upstream's three tries: an episode offers
its series', anything else offers its own, a track with none offers its album's.
linked_rows_to_illustrate is the one read; crates/jellyfin-db/tests/linked_pictures.rs
holds the rules.

The three that read the media

crates/jellyfin-app/src/scan/media_images.rs decides what to take out of a
file and where it goes; the taking is a phase of its own in the scan, before the
by-name pictures, because a genre collage is drawn out of what it leaves.

still.rs grew the two argument shapes upstream's ExtractImage takes beside
the one it already had: mapping a stream (-map 0:N, no -ss) rather than
seeking to a frame, and the isAudio branch, which skips the keyframe attempt
and never passes -s. Both verified against real ffmpeg.

extract_images is the library setting. On by default, as upstream's three
are — a library registered with no TypeOptions enables them.

Two things this leaves

The fixture cannot measure any of it. xtask reference provision registers
every library with empty ImageFetchers per type, which disables all three
extractors along with the remote ones — deliberately, so a scan is a function of
the files. That is why no item in the default snapshot carries a Primary. It
also means nothing in the fixture media carries cover art or an attached
picture. Measuring these against a pair takes a library provisioned with the
fetchers on, a track with cover art, and a container with an attached picture;
reference/README.md now says so beside the trickplay note, and it belongs with
#148's fixture work.

Upstream has three more dynamic providers this issue did not list.
FolderImageProvider, MusicAlbumImageProvider and PhotoAlbumImageProvider
all subclass BaseFolderImageProvider, and all copy the first child's primary
picture the way the box set does. That is the rest of the ten — this issue named
seven — and MusicAlbumImageProvider is the one that matters in practice, since
it is what gives an album the cover its first track carries. Filed separately
rather than folded in here.

Landed in `915c37f` (the composed half) and `4ccdb3d` (the three that read the media). ## The composed five Two of the five draw nothing here, and both because upstream draws nothing either: - **`ArtistImageProvider` asks for no items at all.** `GetItemsWithImages` returns `Array.Empty<BaseItem>()` at the pin — the query is commented out above a `TODO` — so `CreateImage` is handed an empty collection and returns null. An artist with no photograph keeps none upstream. - **`DynamicImageProvider` draws for a `UserView`**, and it only supports the three collection-strip types. This server synthesizes its views from the library folders (`jellyfin_app::library::views`) and files no `UserView` row, so there is nothing for it to run against. Both are recorded in the note beside the runner rather than left as a gap. The other two are ported: - **`PlaylistImageProvider`** — the same square of quarters a genre gets, drawn from the first four items the playlist names. - **`CollectionImageProvider`** — `CreateSingleImage`, so the first picture is *copied whole* and keeps its own extension, and only while `IsLocked`, which is upstream's way of keeping a drawn tile from landing ahead of a fetched one. Each named item offers one picture in upstream's three tries: an episode offers its series', anything else offers its own, a track with none offers its album's. `linked_rows_to_illustrate` is the one read; `crates/jellyfin-db/tests/linked_pictures.rs` holds the rules. ## The three that read the media `crates/jellyfin-app/src/scan/media_images.rs` decides what to take out of a file and where it goes; the taking is a phase of its own in the scan, before the by-name pictures, because a genre collage is drawn out of what it leaves. `still.rs` grew the two argument shapes upstream's `ExtractImage` takes beside the one it already had: mapping a stream (`-map 0:N`, no `-ss`) rather than seeking to a frame, and the `isAudio` branch, which skips the keyframe attempt and never passes `-s`. Both verified against real ffmpeg. `extract_images` is the library setting. **On by default, as upstream's three are** — a library registered with no `TypeOptions` enables them. ## Two things this leaves **The fixture cannot measure any of it.** `xtask reference provision` registers every library with empty `ImageFetchers` per type, which disables all three extractors along with the remote ones — deliberately, so a scan is a function of the files. That is why no item in the default snapshot carries a `Primary`. It also means nothing in the fixture media carries cover art or an attached picture. Measuring these against a pair takes a library provisioned with the fetchers on, a track with cover art, and a container with an attached picture; `reference/README.md` now says so beside the trickplay note, and it belongs with #148's fixture work. **Upstream has three more dynamic providers this issue did not list.** `FolderImageProvider`, `MusicAlbumImageProvider` and `PhotoAlbumImageProvider` all subclass `BaseFolderImageProvider`, and all copy the first child's primary picture the way the box set does. That is the rest of the ten — this issue named seven — and `MusicAlbumImageProvider` is the one that matters in practice, since it is what gives an album the cover its first track carries. Filed separately rather than folded in here.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels   
correctness
Diverges from upstream

  
encoding
Transcoding, ffmpeg, and the arg strings

  
http
Route behaviour

  
jobs
Scheduled tasks

  
operations
Installing a server and keeping one running

  
p1
Do this first

  
p2
After the p1s

  
p3
Real, and not urgent

  
performance
Speed or resource use

  
providers
Metadata from somewhere other than the file

  
scanner
The library scan

  
security
Who is allowed to see what

  
testing
Test coverage and harness

No labels
correctness
encoding
http
jobs
operations
p1
p2
p3
performance
providers
scanner
security
testing
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
alex/ephyra#110
No description provided.