alex/ephyra
Watch
1
0
Fork
You've already forked ephyra
 0
Code Issues 2 Pull requests Projects Releases Packages 1 Wiki Activity Actions

Trickplay sheets are never generated, so scrubbing a video shows nothing #99

New issue
Closed
opened 2026-08-13 22:59:39 +02:00 by Claude · 2 comments
Claude commented 2026-08-13 22:59:39 +02:00
Collaborator
Copy link

/Videos/{id}/Trickplay/{width}/tiles.m3u8 and the .jpg route beside it are answered and serve whatever TrickplayInfos holds (crates/jellyfin-db/src/items/children.rs and crates/jellyfin-db/src/items/repository.rs) — and nothing in the workspace ever inserts a row. Both routes therefore serve a database somebody else filled, which is what M7 said and is not an install.

Upstream has three pieces: TrickplayProvider (a custom metadata provider that fires on refresh), TrickplayImagesTask (key RefreshTrickplayImages, daily), and TrickplayMoveImagesTask (key MoveTrickplayImages), all in MediaBrowser.Providers/Trickplay/. It decodes at an interval, tiles the frames into sheets, and records width, height, tile counts and interval per resolution in TrickplayInfos.

This is the most visible missing artefact for a viewer: hovering the scrub bar in jellyfin-web shows a preview against upstream and nothing here. The generation is ffmpeg work the encoding crate is already built to drive, and the row shape is fixed by the schema.

Note the interaction with #92 and with the scheduler: this is expensive per file and has to be resumable, which is why upstream makes it a task rather than part of a scan.

Where to start

The read side is done and empty: crates/jellyfin-http/src/routes/streaming/trickplay.rs
serves the playlist and the sheets, and crates/jellyfin-db/src/items/children.rs and crates/jellyfin-db/src/items/repository.rs
reads TrickplayInfos. Nothing inserts.

Upstream is three files in MediaBrowser.Providers/Trickplay/: TrickplayProvider
(fires on refresh), TrickplayImagesTask (key RefreshTrickplayImages, daily)
and TrickplayMoveImagesTask (key MoveTrickplayImages). Take the interval,
the widths, the tile grid and the "where do the sheets live" rules from there —
each of them is a number a client reads back out of TrickplayInfos, so
inventing one produces a sheet no player can index into.

The decode-and-tile work belongs with the image code (crates/jellyfin-images,
which already carries the resampler ported for the blurhash) driven by
crates/jellyfin-ffmpeg.

How to know it is done

  • Turn the library option on in the reference, let it generate, scan the same
    library here, and diff-db TrickplayInfos: same widths, heights, tile counts
    and interval per resolution.
  • The sheets themselves compared on bytes against upstream's, the way M10
    compares the pictures a provider won.
  • In a browser: hovering the scrub bar on the fixture's long file shows a
    preview. That is the thing a viewer notices and no diff will tell you.

House rules

CLAUDE.md is the short version and these are the parts that bite: tests run
under cargo nextest; upstream's source is the checkout in upstream/ at the
pin (v12.0-rc5, 6d501ba4) and not the clone beside the repository;
generated files are regenerated rather than edited; a comment says what the code
does, not how it got there; and Dn references stay out of code and comments.
ARCHITECTURE.md decides which crate a thing goes in and which dependency edges
are illegal, DECISIONS.md says why a rule exists.

The reference pair is cargo xtask reference up (and reference provision,
which is idempotent); a run is
cargo run -p jellyfin-harness -- replay --upstream URL --ours URL --scenario FILE --out DIR,
with diff-db for the two databases. crates/jellyfin-harness/README.md says
what a diff means and what an ignore entry has to carry — every entry needs a
reason and an expiry milestone, and an adds-only history is the smell the risk
table names.

`/Videos/{id}/Trickplay/{width}/tiles.m3u8` and the `.jpg` route beside it are answered and serve whatever `TrickplayInfos` holds (`crates/jellyfin-db/src/items/children.rs` and `crates/jellyfin-db/src/items/repository.rs`) — and nothing in the workspace ever inserts a row. Both routes therefore serve a database somebody else filled, which is what M7 said and is not an install. Upstream has three pieces: `TrickplayProvider` (a custom metadata provider that fires on refresh), `TrickplayImagesTask` (key `RefreshTrickplayImages`, daily), and `TrickplayMoveImagesTask` (key `MoveTrickplayImages`), all in `MediaBrowser.Providers/Trickplay/`. It decodes at an interval, tiles the frames into sheets, and records width, height, tile counts and interval per resolution in `TrickplayInfos`. This is the most visible missing artefact for a viewer: hovering the scrub bar in jellyfin-web shows a preview against upstream and nothing here. The generation is ffmpeg work the encoding crate is already built to drive, and the row shape is fixed by the schema. Note the interaction with #92 and with the scheduler: this is expensive per file and has to be resumable, which is why upstream makes it a task rather than part of a scan. --- ## Where to start The read side is done and empty: `crates/jellyfin-http/src/routes/streaming/trickplay.rs` serves the playlist and the sheets, and `crates/jellyfin-db/src/items/children.rs` and `crates/jellyfin-db/src/items/repository.rs` reads `TrickplayInfos`. Nothing inserts. Upstream is three files in `MediaBrowser.Providers/Trickplay/`: `TrickplayProvider` (fires on refresh), `TrickplayImagesTask` (key `RefreshTrickplayImages`, daily) and `TrickplayMoveImagesTask` (key `MoveTrickplayImages`). Take the interval, the widths, the tile grid and the "where do the sheets live" rules from there — each of them is a number a client reads back out of `TrickplayInfos`, so inventing one produces a sheet no player can index into. The decode-and-tile work belongs with the image code (`crates/jellyfin-images`, which already carries the resampler ported for the blurhash) driven by `crates/jellyfin-ffmpeg`. ## How to know it is done - Turn the library option on in the reference, let it generate, scan the same library here, and `diff-db` `TrickplayInfos`: same widths, heights, tile counts and interval per resolution. - The sheets themselves compared on bytes against upstream's, the way M10 compares the pictures a provider won. - In a browser: hovering the scrub bar on the fixture's long file shows a preview. That is the thing a viewer notices and no diff will tell you. ### House rules `CLAUDE.md` is the short version and these are the parts that bite: tests run under `cargo nextest`; upstream's source is the checkout in `upstream/` **at the pin** (`v12.0-rc5`, `6d501ba4`) and not the clone beside the repository; generated files are regenerated rather than edited; a comment says what the code does, not how it got there; and `Dn` references stay out of code and comments. `ARCHITECTURE.md` decides which crate a thing goes in and which dependency edges are illegal, `DECISIONS.md` says why a rule exists. The reference pair is `cargo xtask reference up` (and `reference provision`, which is idempotent); a run is `cargo run -p jellyfin-harness -- replay --upstream URL --ours URL --scenario FILE --out DIR`, with `diff-db` for the two databases. `crates/jellyfin-harness/README.md` says what a diff means and what an ignore entry has to carry — every entry needs a reason and an expiry milestone, and an adds-only history is the smell the risk table names.
Claude commented 2026-08-14 15:44:45 +02:00
Author
Collaborator
Copy link

Not implemented, and here is the measured reason plus a split. Everything below was measured by running upstream's RefreshTrickplayImages on the fixture.

Night Signal -> 320x180, 10x10, 9 thumbnails, 10000 ms, Bandwidth 888
one tile: 110967 bytes
ceil(110967 * 8 / 10 / 10 / 10) = 888

Bandwidth is a pure function of the encoded tile's byte length. A one-byte difference in the JPEG changes the row. Upstream encodes the tile through SkiaSharp at quality 90, so matching that row means a libjpeg-turbo-faithful JPEG encoder — crates/jellyfin-images/src/jpeg.rs is the comparable decoder and runs to 1,804 lines, and collage.rs already records this project's position that a drawn picture is not upstream's bytes.

Something never measured before was confirmed on the way: our existing trickplay read path serves upstream's real sheets correctly — same playlist with the api key masked, and a byte-identical tile, on the <data>/data/trickplay/<xx>/<hyphenated guid>/<width> - <tw>x<th>/<n>.jpg convention.

The split

  1. Generation and the row — ffmpeg interval extraction (GetInputArgument and GetVideoProcessingFilterParam, both already ported), tiling, the TrickplayInfos write, RefreshTrickplayImages, and ExtractTrickplayImagesDuringLibraryScan. Everything matches except Bandwidth, which needs an ignore entry expiring on (2). This issue.
  2. A Skia-compatible JPEG encoder in jellyfin-images — what makes a tile byte-identical and retires that entry. Filed separately; it is the large one.
  3. MoveTrickplayImages and DiscoverExistingTrickplayAsync — moving sheets between the media and data directories, and cataloguing user-placed ones with upstream's interval-snapping arithmetic. Independent of both.

Doing (1) with an argued ignore entry is the honest path: a client gets working scrub previews, and the one row that cannot match says why and when it will.

Not implemented, and here is the measured reason plus a split. Everything below was measured by running upstream's `RefreshTrickplayImages` on the fixture. ``` Night Signal -> 320x180, 10x10, 9 thumbnails, 10000 ms, Bandwidth 888 one tile: 110967 bytes ceil(110967 * 8 / 10 / 10 / 10) = 888 ``` **`Bandwidth` is a pure function of the encoded tile's byte length.** A one-byte difference in the JPEG changes the row. Upstream encodes the tile through SkiaSharp at quality 90, so matching that row means a libjpeg-turbo-faithful JPEG *encoder* — `crates/jellyfin-images/src/jpeg.rs` is the comparable decoder and runs to 1,804 lines, and `collage.rs` already records this project's position that a drawn picture is not upstream's bytes. Something never measured before was confirmed on the way: **our existing trickplay read path serves upstream's real sheets correctly** — same playlist with the api key masked, and a byte-identical tile, on the `<data>/data/trickplay/<xx>/<hyphenated guid>/<width> - <tw>x<th>/<n>.jpg` convention. ## The split 1. **Generation and the row** — ffmpeg interval extraction (`GetInputArgument` and `GetVideoProcessingFilterParam`, both already ported), tiling, the `TrickplayInfos` write, `RefreshTrickplayImages`, and `ExtractTrickplayImagesDuringLibraryScan`. Everything matches except `Bandwidth`, which needs an ignore entry expiring on (2). **This issue.** 2. **A Skia-compatible JPEG encoder** in `jellyfin-images` — what makes a tile byte-identical and retires that entry. Filed separately; it is the large one. 3. **`MoveTrickplayImages` and `DiscoverExistingTrickplayAsync`** — moving sheets between the media and data directories, and cataloguing user-placed ones with upstream's interval-snapping arithmetic. Independent of both. Doing (1) with an argued ignore entry is the honest path: a client gets working scrub previews, and the one row that cannot match says why and when it will.
Claude commented 2026-08-16 15:15:37 +02:00
Author
Collaborator
Copy link

Done, on main. Split (1) of the comment above landed; split (2) landed too and
is #140, so there is no Bandwidth ignore entry — it was opened in 7f31074
and deleted in 3cd112d, and db-ignore-list.json is byte-for-byte unchanged
across the batch.

What landed

  • crates/jellyfin-ffmpeg/src/intervals.rs — ExtractVideoImagesOnIntervalAccelerated
    on its software path, through the already-ported input_argument and
    video_processing_filter_param, with the setpts in front of the rate filter.
  • crates/jellyfin-images/src/trickplay.rs — SkiaEncoder.CreateTrickplayTile:
    the grid, the black padding of a short sheet, and the refusal when a frame is
    not the width asked for.
  • crates/jellyfin-server/src/artefacts.rs — the task body: page the videos of
    every library that asks, the CanGenerateTrickplay guards, cut, tile, file,
    prune the widths nothing is configured at, and forget the sheets of a library
    that stopped asking.
  • A Trickplay config section on upstream's defaults, LibraryConfig::extract_trickplay_images,
    ItemWriter::save_trickplay / forget_trickplay, and Kind::TrickplayImages.

The numbers

The command line builds character for character against what upstream logged:

-loglevel error -threads 1 -i file:"…/Night Signal (2022).mkv" -an -sn
-vf "setpts=N/24.000/TB,fps=0.10000000149011612,setparams=…,scale=trunc(min(max(iw\,ih*(a*sar))\,320)/2)*2:trunc(ow/(a*sar)/2)*2,format=yuv420p"
-threads 1 -c:v mjpeg -qscale:v 4 -fps_mode passthrough -f image2 "…/%08d.jpg"

The row, from a real run of each server over a cp -a copy of the default
snapshot — upstream in its own container from the pinned image on :8099, ours
on :8098:

Width Height TileWidth TileHeight ThumbnailCount Interval Bandwidth
upstream 320 180 10 10 9 10000 888
ours 320 180 10 10 9 10000 888

The sheet is cmp-identical at 110,967 bytes, at
<data>/data/trickplay/6a/6af10317-…/320 - 10x10/0.jpg on both. Over HTTP the
playlist matches with the key masked and /Videos/{id}/Trickplay/320/0.jpg
returns the same bytes from each. The task id
64f5f44cd30dc273cb9890205473bbcc falls out of
Guid::hashed("MediaBrowser.Providers.Trickplay.TrickplayImagesTask") unchanged
and is asserted in the route test.

What the review round changed

  • One bad video no longer fails the whole sweep — it catches per video, the way
    the sibling chapter-image sweep already did and the way upstream does twice.
  • A stream whose frame rate is unknown is refused rather than written a frame
    at a time. Upstream throws on the same condition; without it a remux or an
    MPEG-TS capture yields ~130k JPEGs for a 90-minute film and a row claiming
    ThumbnailCount=130000.
  • The keep-set is built from the rows, not the configuration
    (TrickplayManager.cs:360-366). Measured: generate at [320], move the
    configuration to [480], re-run — the database holds 320|180|9|888 and
    480|270|9|1554, both folders survive, and the 320 tile still answers 200
    where it used to 404.
  • IsShortcut and IsPlaceHolder guards added, and the runtime guard moved onto
    the raw interval where upstream has it, before the clamp to 1000.
  • Intervals::cut takes a Cancellation and races it against the deadline, so
    a cancelled task no longer waits up to an hour on one file; stderr is kept
    and logged on a non-zero exit.

Two guards were deliberately left out, both because they cannot fire here:
SourceTypes = [Library] is dead upstream at this pin — eight callers set
InternalItemsQuery.SourceTypes and nothing reads it; there is no reference in
BaseItemRepository.cs or BaseItemRepository.TranslateQuery.cs and no
SourceType column on BaseItemEntity. IsCompleteMedia needs a live-TV
recordings manager this server does not have, and its channel branch needs
channel items nothing files. If live TV lands, this is one of the places that
grows a guard.

Filed forward

  • #147 — MoveTrickplayImages and DiscoverExistingTrickplayAsync, split (3)
    above, untouched.
  • #148 — the fixture carries no sheets, so none of the columns above is diffed
    by any harness run and two refusal scenarios refuse for a reason they do not
    name.
Done, on `main`. Split (1) of the comment above landed; split (2) landed too and is #140, so **there is no `Bandwidth` ignore entry** — it was opened in `7f31074` and deleted in `3cd112d`, and `db-ignore-list.json` is byte-for-byte unchanged across the batch. ## What landed - `crates/jellyfin-ffmpeg/src/intervals.rs` — `ExtractVideoImagesOnIntervalAccelerated` on its software path, through the already-ported `input_argument` and `video_processing_filter_param`, with the `setpts` in front of the rate filter. - `crates/jellyfin-images/src/trickplay.rs` — `SkiaEncoder.CreateTrickplayTile`: the grid, the black padding of a short sheet, and the refusal when a frame is not the width asked for. - `crates/jellyfin-server/src/artefacts.rs` — the task body: page the videos of every library that asks, the `CanGenerateTrickplay` guards, cut, tile, file, prune the widths nothing is configured at, and forget the sheets of a library that stopped asking. - A `Trickplay` config section on upstream's defaults, `LibraryConfig::extract_trickplay_images`, `ItemWriter::save_trickplay` / `forget_trickplay`, and `Kind::TrickplayImages`. ## The numbers The command line builds character for character against what upstream logged: ``` -loglevel error -threads 1 -i file:"…/Night Signal (2022).mkv" -an -sn -vf "setpts=N/24.000/TB,fps=0.10000000149011612,setparams=…,scale=trunc(min(max(iw\,ih*(a*sar))\,320)/2)*2:trunc(ow/(a*sar)/2)*2,format=yuv420p" -threads 1 -c:v mjpeg -qscale:v 4 -fps_mode passthrough -f image2 "…/%08d.jpg" ``` The row, from a real run of each server over a `cp -a` copy of the `default` snapshot — upstream in its own container from the pinned image on `:8099`, ours on `:8098`: | | Width | Height | TileWidth | TileHeight | ThumbnailCount | Interval | Bandwidth | |---|---|---|---|---|---|---|---| | upstream | 320 | 180 | 10 | 10 | 9 | 10000 | 888 | | ours | 320 | 180 | 10 | 10 | 9 | 10000 | **888** | The sheet is `cmp`-identical at 110,967 bytes, at `<data>/data/trickplay/6a/6af10317-…/320 - 10x10/0.jpg` on both. Over HTTP the playlist matches with the key masked and `/Videos/{id}/Trickplay/320/0.jpg` returns the same bytes from each. The task id `64f5f44cd30dc273cb9890205473bbcc` falls out of `Guid::hashed("MediaBrowser.Providers.Trickplay.TrickplayImagesTask")` unchanged and is asserted in the route test. ## What the review round changed - One bad video no longer fails the whole sweep — it catches per video, the way the sibling chapter-image sweep already did and the way upstream does twice. - A stream whose frame rate is unknown is refused rather than written a frame at a time. Upstream throws on the same condition; without it a remux or an MPEG-TS capture yields ~130k JPEGs for a 90-minute film and a row claiming `ThumbnailCount=130000`. - The keep-set is built from the **rows**, not the configuration (`TrickplayManager.cs:360-366`). Measured: generate at `[320]`, move the configuration to `[480]`, re-run — the database holds `320|180|9|888` *and* `480|270|9|1554`, both folders survive, and the 320 tile still answers `200` where it used to 404. - `IsShortcut` and `IsPlaceHolder` guards added, and the runtime guard moved onto the raw interval where upstream has it, before the clamp to 1000. - `Intervals::cut` takes a `Cancellation` and races it against the deadline, so a cancelled task no longer waits up to an hour on one file; `stderr` is kept and logged on a non-zero exit. Two guards were deliberately left out, both because they cannot fire here: `SourceTypes = [Library]` is **dead upstream at this pin** — eight callers set `InternalItemsQuery.SourceTypes` and nothing reads it; there is no reference in `BaseItemRepository.cs` or `BaseItemRepository.TranslateQuery.cs` and no `SourceType` column on `BaseItemEntity`. `IsCompleteMedia` needs a live-TV recordings manager this server does not have, and its channel branch needs channel items nothing files. If live TV lands, this is one of the places that grows a guard. ## Filed forward - #147 — `MoveTrickplayImages` and `DiscoverExistingTrickplayAsync`, split (3) above, untouched. - #148 — the fixture carries no sheets, so none of the columns above is diffed by any harness run and two refusal scenarios refuse for a reason they do not name.
alex closed this issue 2026-08-16 15:20:04 +02:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
No results found.
Labels
Clear labels   
correctness
Diverges from upstream

  
encoding
Transcoding, ffmpeg, and the arg strings

  
http
Route behaviour

  
jobs
Scheduled tasks

  
operations
Installing a server and keeping one running

  
p1
Do this first

  
p2
After the p1s

  
p3
Real, and not urgent

  
performance
Speed or resource use

  
providers
Metadata from somewhere other than the file

  
scanner
The library scan

  
security
Who is allowed to see what

  
testing
Test coverage and harness

No labels
correctness
encoding
http
jobs
operations
p1
p2
p3
performance
providers
scanner
security
testing
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
alex/ephyra#99
No description provided.