KeyframeData is never written, so every HLS playlist re-reads the file #100
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
M5 named this and left it: upstream caches an extraction in the
KeyframeDatatable and we read the container's index each time a playlist is asked for — three seeks into the mkv, per request, forever.Two consequences, one of them already predicted in
MILESTONES.md:Upstream's task is
src/Jellyfin.MediaEncoding.Hls/ScheduledTasks/KeyframeExtractionScheduledTask.cs, keyKeyframeExtraction. It has two extractors — the cheap one that reads container metadata and the ffprobe one that decodes — and only the cheap one runs inline; the expensive one is exactly why the task exists.crates/jellyfin-matroskaalready does the cheap half for Matroska, so what is missing is the table and the mp4 side.The playlist we build from the Cues element agrees with upstream's to the millisecond on the fixture's HEVC mkv, so this is a caching and coverage issue rather than a correctness one.
Where to start
M5 named this and left it, and the note there is the specification: upstream
caches the extraction in
KeyframeData, ours reads the container index on everyplaylist request — three seeks per request, forever.
crates/jellyfin-matroskareads the Cues element, and the playlist isbuilt in
crates/jellyfin-app/src/hls/.src/Jellyfin.MediaEncoding.Hls/ScheduledTasks/KeyframeExtractionScheduledTask.cs(key
KeyframeExtraction), and the two extractors behind it. Only the cheapone — the one that reads container metadata — runs inline; the ffmpeg one
decodes the file and is exactly why the task exists. Keep that split.
The mp4 side is not written here at all; Matroska is.
How to know it is done
upstream's — same nine segments to the millisecond on the fixture's HEVC mkv —
still holds, and now costs one read per file rather than one per request.
diff-dbreports no difference onKeyframeDataafter a run where upstreamremuxed an mkv over HLS. That run does not exist yet; making it exist is part
of this.
reason.
House rules
CLAUDE.mdis the short version and these are the parts that bite: tests rununder
cargo nextest; upstream's source is the checkout inupstream/at thepin (
v12.0-rc5,6d501ba4) and not the clone beside the repository;generated files are regenerated rather than edited; a comment says what the code
does, not how it got there; and
Dnreferences stay out of code and comments.ARCHITECTURE.mddecides which crate a thing goes in and which dependency edgesare illegal,
DECISIONS.mdsays why a rule exists.The reference pair is
cargo xtask reference up(andreference provision,which is idempotent); a run is
cargo run -p jellyfin-harness -- replay --upstream URL --ours URL --scenario FILE --out DIR,with
diff-dbfor the two databases.crates/jellyfin-harness/README.mdsayswhat a diff means and what an ignore entry has to carry — every entry needs a
reason and an expiry milestone, and an adds-only history is the smell the risk
table names.
Done, on
main—434843f.KeyframeDatais read and written. A playlist request reads the row and reaches the container only on a miss, which is upstream'sCacheDecorator; the ffprobe reader that walks every packet is reachable only from the task, because upstream's playlist generator keeps just theIsMetadataBasedextractors. TwoCachedvalues over one set of rows.The mp4 index reader this issue expected does not exist upstream either. At the pin there are exactly two extractors — Matroska and ffprobe — so this was much smaller than it was billed as.
Measured against a pair built for it: nine rows each,
diff-dbclean on the table, ids andKeyframeTicksidentical, and the variant playlist for the HEVC mkv byte-identical to upstream's before and after. Cost: 20 requests at 237 ms with the row against ~409 ms reaching the container each time.