A scan's rows differ from upstream's though the counts nearly agree #75
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
The rows a scan writes differ from upstream's, though the counts nearly agree.
First
diff-dbbetween upstream's own scan of/mnt/media(
var/big-upstream) and this port's scan of the same three directories.The shape agrees. Item counts by type are identical — 7905 Episode, 4889
Audio, 708 Season, 329 MusicAlbum, 289 Series, 284 Movie — and three tables
match row for row:
The content does not. 47,632 rows differ on at least one column, dominated
by
MediaStreamInfosat 16,772 — roughly one row in seven of that table. Thatis the largest single thing left, and it is invisible to the response harness,
which compares what a route serves rather than what a scan wrote.
Not defects, and to exclude before counting:
Users,Permissions,Preferences,Devicesare empty here only becausethe scan config provisioned no account.
ActivityLogsand the display-preference tables are already exempt withrecorded reasons.
Seen in the
BaseItemssample and worth pulling apart first: upstream stores acopyright string in
Studios(2026 BBC (programme data only)) where this portstores
'', and carries aPremiereDatethis port leaves null. Both come fromthe probe rather than from a provider, so both should be reachable.
Rerun with:
Scoped. The stream table is not 16,772 rows of trouble — it is two columns.
All 116,917 rows this port wrote match upstream's on every column but three:
RefFrames1Hdr10PlusPresentFlag1BitRateEverything else — codec, profile, dimensions, channel layout, language, the
dynamic-range columns, the disposition flags — agrees row for row. The harness
counts a row as differing when any column does, which is what made two missing
fields look like one row in seven.
RefFramesis the reference-frame count ffprobe reports asrefs, and thisport never records it.
Hdr10PlusPresentFlagis already written here but isreached only where the binary takes
-only_first_vframe; the 248 are rowsupstream measured and this port did not.
The remaining 70 rows are streams this port did not write at all, on 6 items it
also did not write — worth pulling apart separately from the columns.
So the order of work is:
RefFrames, then the 70 absent rows, then the twobitrates. The other tables' differences are still unexamined.
The
BaseItemshalf, scoped the same way.Comparing the 15,139 items both scans wrote (ignoring
Dataand thesave/refresh timestamps):
ProductionLocations''Tags''PremiereDate2018-12-19 00:01:06ProductionYearDateLastMediaAddedDateModifiedRunTimeTicksGenresName/SortNameOverviewThe first two are one bug and the cheapest win here: upstream's columns are
non-nullable, so nothing reads as
''where this port writes NULL. That is thesame shape as the
DateCreatedfix in #51, and it accounts for 30,278 of thediffering values on its own.
ProductionYearis the one to look at hardest — those are different values,not a missing one (2018 against 2001), so something is being read from a
different place rather than not read at all.
NormalizationGain(83) is already fixed by #72; this database was scannedbefore that merged. Check the others for the same before assuming they are open.
Eight items upstream has and this port does not, and most are known scope
rather than defects: the
MØLartist (#71's unresolvable name), threeMusicGenrerows from upstream's artist metadata refresh, the playlists folder,and the Live TV view.
The stream half is not a defect. It was my measurement.
The scan those numbers came from ran with
ffprobe = "ffprobe"— the distrobuild.
-only_first_vframeis a jellyfin-ffmpeg patch, and a distro binaryrefuses it, so the probe drops
-show_frames -only_first_vframe. That isupstream's own rule (
MediaEncoder.cs:517, gated by the capability check at:238) and this port implements it identically. Without-show_frames,ffprobe never opens a decoder, and all three columns follow from that one fact
— measured against the pinned image's own binary on the same file:
refs-show_streams -show_format+ -show_frames -only_first_vframe1The two
BitRaterows follow arithmetically: video is estimated as containerminus audio, so the audio's 4,069 moves to video with the opposite sign. Both
were on one file and cancelled.
Rescanning the same subsets with the right prober closes every column:
Neither side was broken:
refsis parsed, mapped, written, read back andserved, and the read path was proven live against the upstream-scanned database
— both servers return byte-identical
RefFrames,Hdr10PlusPresentFlagandBitRatefor the same item.No source change was made, and that is deliberate. Rather than manufacture
one to satisfy a test requirement, the existing behaviour is now pinned: a
stream round trip over every probe-filled field (that file claimed to cover them
and covered only the dynamic-range columns, so
RefFrameshad no write-sideassertion at all), and three unit tests for the HDR10+ rule that decides all 248
rows and had none. Both shown load-bearing by reverting the source.
The 70 absent rows are also not this. My pairing of them with the 6 missing
items was wrong: all 70 are Lyric streams on 70 audio items this port did
write. Upstream extracts a
lyrics-engtag to an.lrcfile under its metadatadirectory and then resolves that file back as a stream. No
.lrcexists under/mnt/media; it is a metadata-writing feature this port does not have.README now records that a scan being diffed wants the prober upstream ships, and
that the
first_video_frameline at startup says which one you got.Closing this on what it settled, with the remainder split out as #76.
The stream columns were not a defect. The scan behind the original numbers
ran with the distro ffprobe, which refuses
-only_first_vframe— ajellyfin-ffmpeg patch — so the probe correctly drops
-show_frames, and withouta decoder
refs, the HDR10+ side data and the aac bitrate all disappeartogether. Rescanning the same subsets with the prober upstream ships closes all
three to zero. No source change was warranted; instead the existing behaviour is
now pinned, and that turned up real coverage gaps —
crates/jellyfin-db/tests/ streams.rsclaimed to cover every probe-filled field and covered only thedynamic-range columns, so
RefFrameshad no write-side assertion at all, andthe HDR10+ rule deciding 248 rows had none either.
The item columns were, and are fixed. Upstream reads an audio date from its
tag library rather than from the prober, and the two disagree: ffmpeg merges
ID3v2.3'sTYERandTDATinto one date where ATL does not. Every ID3v2block in the library is v2.3, which is why 1,437 mp3s and 61 flacs carried a
full date where upstream has a year. Measured against upstream's own scan of the
same files:
ProductionLocations,Tags,PremiereDateProductionYear,AlbumDateLastMediaAddedProductionLocationsandTagswere the cheapest of it: upstream's columns arenon-nullable so every row holds
'', and this port wrote NULL — 30,278 valuesfrom one omission.
A rescan of an unchanged library still writes nothing, verified on both real
libraries with every column byte-identical across the second pass.
Also corrected here: the 70 absent stream rows are not paired with the 6 missing
items. They are Lyric streams on 70 items this port did write, extracted by
upstream from a
lyrics-engtag into an.lrcfile it then resolves back. No.lrcexists under the measured library.What is left is in #76 — it is a different kind of work from this, not more of
the same.