Two listings put a different row first than upstream, where a large library has ties #78
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Two listings put a different row first than upstream does, on a library large
enough to have ties.
Measured against a database copied from a running Jellyfin — 55,523 items,
43,466
UserDatarows, 14 users — with both servers given their own copy.Filters=IsPlayed&SortBy=DatePlayed&SortOrder=Descendingdisagrees on thefirst row outright:
Everything downstream of that —
CommunityRating, the blur hashes, the wholepage — differs because it is a different item, not because any field is mapped
wrong.
/Shows/NextUpdisagrees at index 4, in all three of its forms(
/Shows/NextUp,?SeriesId=…,?EnableResumable=false):DatePlayedacross 43k rows of real play state has ties that a fixture with noplay state cannot produce, and
NextUporders by a date too. D15 settles theties upstream's
ORDER BYleaves open with the id; whether that is what thesetwo paths do, and whether upstream's tiebreak for these particular sorts is the
same one, is what wants checking.
Neither is visible against the committed snapshot: with 25 items and no play
state there is nothing to tie.
To reproduce. Point both servers at their own copy of a library with real
play state across several users, then:
A third listing may have the same cause and is not confirmed:
/Items?Recursive=true&IncludeItemTypes=Audio&SortBy=AlbumArtist,SortNameshowsItems[0]differing onAlbumArtists,ArtistItemsandAlbumId, which iseither a mapping difference or a different row in first place. It needs the ids
compared before it is called either.
The third listing is confirmed: it is a different row in first place, not a mapping difference.
From bench run
20260805T083531Z, same anchor.IdandNamediffer on all fifty rows, which settles it — nothing is mapped wrong, the page is simply a different page:1,200 differences over the page, and
AlbumArtists/ArtistItems/AlbumId— the fields this issue was unsure about — are just the consequence.It is not the same tiebreak problem as the other two, though. Upstream's leading rows carry no album artist at all:
Upstream leads an ascending
AlbumArtistsort with the tracks that have none, then orders those among themselves bySortName— "3rd Prototype…", "Absolve_313775811…" is alphabetical. We put a named artist first, so we are either sorting empty album artists last or excluding them from the ordering key. That is a null-ordering difference rather than an unstable tiebreak, and it will not be fixed by whatever settlesDatePlayedandNextUp. Worth splitting out if the other two turn out to share a cause.It gates a benchmark request.
audio-by-album-artistis one of the eight the bench gate drops, so its cost is currently unmeasured — see #83.Two of the three were one bug, and it was not a tiebreak. The third does not reproduce.
Fixed on
mainin44c07ad,08cf310andba9156d.The cause
dateplayedandalbumartistwere both missing from the sort-name table incrates/jellyfin-http/src/routes/items.rs. A name that table does not know is dropped, so the query fell back toSortName ASC, Id ASCand answered a different page. Nothing was tied and nothing was ordered wrong — the ordering asked for was never applied.played-descendingcame back in exactly reversed order, on rows four months apart:bd17017e— 2025-10-31T20:17a9972238— 2025-07-28T19:062a83e1e5— 2025-07-28T21:102a83e1e5a9972238— 2025-07-28T19:06bd17017eaudio-by-album-artistshared 0 of 50 ids with upstream's page. Upstream orders by theCleanValueof the first credited album artist (OrderMapper.MapOrderByField), which is NULL for an uncredited track and therefore leads ascending — the null-ordering difference this issue's later comment described, and it falls out of applying the sort at all.Two related things came with it:
DatePlayednow takes the newest of an item's own progress and that of its alternate versions, as upstream'sConcat(...).Max()does; andSortOrderis paired positionally withSortBy, soSortBy=AlbumArtist,SortName&SortOrder=Descending,Ascendingno longer silently sorts ascending./Shows/NextUpdoes not reproduceAgainst a correctly arranged pair the two sides agree id for id, in all three forms — plain,
?EnableResumable=false, and?SeriesId=for each of the four series. Upstream returns four rows (TotalRecordCount: 4), so the reported disagreement "at index 4" cannot exist here. That measurement came from the fixture layout described in the comment above.After
played-descendingid order equalaudio-by-album-artistid order equalplayed-descendingproduces no difference at all now.audio-by-album-artisthas three left, all on one row (Items[25].ParentBackdropItemId,ParentBackdropImageTags, and the blurhash keyed by it) — an inherited parent backdrop, unrelated to ordering and not tracked anywhere yet.What the fix had to survive
The obvious way to write the alternate-version leg is quadratic. Without
sqlite_stat1SQLite drives it fromUserDatarather than from the partial index onPrimaryVersionId, scanning the user's wholeUserDatatable once per sorted row: 4.46 s against 0.005 s at 5,000 items, and past 120 s at 50,000. It is planned with aCROSS JOINnamingBaseItemsfirst instead, which does not depend on statistics — 0.038 s at 50,000 rows with none.Worth recording for future measurements: the anchor carries
sqlite_stat1with 80 rows, because it came off a server that had runANALYZE. It could never have shown this. A synthetic fixture built from the committed baseline schema was the only thing that could.Albumis now ordered by as well, since the canonical music sort a client sends isAlbumArtist,Album,SortNameand leaving the middle key out hides the divergence.ArtistandStudiouse the sameItemValuesmechanism and are still unmapped — no request exercises them.