A movie's playback key changes with the fields the request asked for #84

Closed
opened 2026-08-05 11:04:24 +02:00 by Claude · 1 comment
Collaborator

A movie's playback key changes with the fields the request asked for, and only the richer form disagrees with upstream.

Measured against a database copied from a running Jellyfin — 55,523 items, 43,466 UserData rows — with both servers given their own copy, in bench run 20260805T083531Z.

Two requests over the same fifty movies in the same order. One asks for no fields and agrees with upstream completely. The other adds Fields=Overview,People,MediaStreams,ProviderIds and disagrees on UserData.Key for 33 of the 50:

GET /Items?Recursive=true&IncludeItemTypes=Movie&SortBy=SortName&SortOrder=Ascending&Limit=50
  agrees — this request is in the timed set

GET /Items?Recursive=true&IncludeItemTypes=Movie&SortBy=SortName&SortOrder=Ascending&Limit=50
    &Fields=Overview,People,MediaStreams,ProviderIds
  Items[0].UserData.Key: upstream "f88d8636-81e8-ca38-8e58-359ee0656ce6", ours "1316092"
  Items[1].UserData.Key: upstream "edc55773-7a7c-4c18-7e8a-3d5def226ccc", ours "9805"
  Items[4].UserData.Key: upstream "a5cb611a-a8ac-c050-2301-a2a4167133cc", ours "696806"
  Items[5].UserData.Key: upstream "b7c88710-9b0e-4966-0f91-eea94ca8c4e7", ours "1062215"
  Items[6].UserData.Key: upstream "f9b894d8-4f01-b2ac-cc43-307a08bbf574", ours "348"

The key is a property of the item and of the user's row, not of the field list, so one of the two answers is wrong on our side whatever upstream does — a client that pages a listing and then re-reads it with fields gets two different keys for one film.

Ours is a provider id and upstream's is a guid. The 17 that agree are presumably the films with no provider id to derive from, which is what makes this look like the derivation firing only when ProviderIds was loaded: asking for the field loads the rows, the loaded rows change the key, and the request the client did not decorate keeps the guid.

This is the inverse of #66 and may be a regression from its fix. That issue was an episode whose key fell back to the guid because the series' provider rows were not read on routes that did not ask for them, and it recorded that a movie's key agreed with upstream exactly. It no longer does, in the direction that fix pushed. Worth checking 3729318 against a movie before assuming this is independent.

Which of the two is right is the open question. Upstream sends a guid for these films on this route, so if upstream is the target then the derivation should not run for a movie here at all — but #66's evidence says a movie's key did agree when it was derived, on /Items/Latest. The two need reconciling against GetUserDataKeys() rather than against each other.

It gates a benchmark request. movies-page-1-fields is one of the eight requests the bench gate drops, so its cost is currently unmeasured — see #83. People and MediaStreams[].Score on the same request are #81; this is the third difference on it and is not tracked anywhere.

To reproduce. Any library with provider ids on its films and play state against them.

for port in 8096 8097; do
  for f in "" "&Fields=ProviderIds"; do
    curl -s -H "Authorization: MediaBrowser Token=\"$TOKEN\"" \
      "http://127.0.0.1:$port/Items?Recursive=true&IncludeItemTypes=Movie&SortBy=SortName&Limit=5$f" \
      | jq -r '[.Items[].UserData.Key] | @csv'
  done
done

Not visible against the committed snapshot: it has no provider ids to derive a key from, so both forms answer the guid.

A movie's playback key changes with the fields the request asked for, and only the richer form disagrees with upstream. Measured against a database copied from a running Jellyfin — 55,523 items, 43,466 `UserData` rows — with both servers given their own copy, in bench run `20260805T083531Z`. Two requests over the **same fifty movies in the same order**. One asks for no fields and agrees with upstream completely. The other adds `Fields=Overview,People,MediaStreams,ProviderIds` and disagrees on `UserData.Key` for 33 of the 50: GET /Items?Recursive=true&IncludeItemTypes=Movie&SortBy=SortName&SortOrder=Ascending&Limit=50 agrees — this request is in the timed set GET /Items?Recursive=true&IncludeItemTypes=Movie&SortBy=SortName&SortOrder=Ascending&Limit=50 &Fields=Overview,People,MediaStreams,ProviderIds Items[0].UserData.Key: upstream "f88d8636-81e8-ca38-8e58-359ee0656ce6", ours "1316092" Items[1].UserData.Key: upstream "edc55773-7a7c-4c18-7e8a-3d5def226ccc", ours "9805" Items[4].UserData.Key: upstream "a5cb611a-a8ac-c050-2301-a2a4167133cc", ours "696806" Items[5].UserData.Key: upstream "b7c88710-9b0e-4966-0f91-eea94ca8c4e7", ours "1062215" Items[6].UserData.Key: upstream "f9b894d8-4f01-b2ac-cc43-307a08bbf574", ours "348" The key is a property of the item and of the user's row, not of the field list, so **one of the two answers is wrong on our side whatever upstream does** — a client that pages a listing and then re-reads it with fields gets two different keys for one film. Ours is a provider id and upstream's is a guid. The 17 that agree are presumably the films with no provider id to derive from, which is what makes this look like the derivation firing only when `ProviderIds` was loaded: asking for the field loads the rows, the loaded rows change the key, and the request the client did not decorate keeps the guid. **This is the inverse of #66 and may be a regression from its fix.** That issue was an *episode* whose key fell back to the guid because the series' provider rows were not read on routes that did not ask for them, and it recorded that *a movie's key agreed with upstream exactly*. It no longer does, in the direction that fix pushed. Worth checking `3729318` against a movie before assuming this is independent. Which of the two is right is the open question. Upstream sends a guid for these films on this route, so if upstream is the target then the derivation should not run for a movie here at all — but #66's evidence says a movie's key did agree when it was derived, on `/Items/Latest`. The two need reconciling against `GetUserDataKeys()` rather than against each other. **It gates a benchmark request.** `movies-page-1-fields` is one of the eight requests the bench gate drops, so its cost is currently unmeasured — see #83. `People` and `MediaStreams[].Score` on the same request are #81; this is the third difference on it and is not tracked anywhere. **To reproduce.** Any library with provider ids on its films and play state against them. for port in 8096 8097; do for f in "" "&Fields=ProviderIds"; do curl -s -H "Authorization: MediaBrowser Token=\"$TOKEN\"" \ "http://127.0.0.1:$port/Items?Recursive=true&IncludeItemTypes=Movie&SortBy=SortName&Limit=5$f" \ | jq -r '[.Items[].UserData.Key] | @csv' done done Not visible against the committed snapshot: it has no provider ids to derive a key from, so both forms answer the guid.
Author
Collaborator

Upstream has no rule here — it has a cache. The rule it does have is about the route, not the request.

Fixed on main in 294f19f and 7be4ecb.

What upstream actually does

Measured on genuinely fresh pairs — one request per start, nothing before it, because anything else measures the cache:

the first request after a restart upstream's UserData.Key
a plain movie listing f88d8636-81e8-ca38-8e58-359ee0656ce6
the same, plus Fields=…,ProviderIds 1316092
GET /Items/{id} 1316092

So the field list does move upstream's key on a cold server, and this issue's premise — that upstream sends the guid with Fields — is not upstream's rule but upstream's cache. Three mechanisms, each confirmed in the pinned source:

  • ApplyNavigations joins BaseItemProviders only when the request asked for ProviderIds, and GetUserDataKeys() reads whatever was joined.
  • UserDataManager keeps one UserItemData per user and item. Only a query fills that cache and reads it, so the first query to touch an item after a restart freezes the key every later query reports. In the scenario, the plain listing runs first and freezes the guid.
  • GET /Items/{id} goes through GetUserData, which never consults that cache, over an item RetrieveItem fetched with .Include(e => e.Provider) hard-coded. It derives the key every time — the one deterministic answer, and it leaves what the queries report untouched.

That last point also corrects crates/jellyfin-harness/README.md, which claimed a by-id read poisons what later listings report. It does not: it never touches the cache.

The rule ours now answers

One key per item per user, independent of the field list: an item's own external ids key its playback state where the route reached the item by id; a query keys on the item's id.

That is upstream everywhere upstream is deterministic. Where it is not — a listing whose first-ever request after a restart asks for ProviderIds — no rule can match both that and the plain listing, and the plain listing is what every client sends first. Ours matches from the second request onward, forever.

"Reached by id" is a property of how the item in hand was fetched, not only of the route's subject. /Items/{itemId}/Ancestors is the case that makes the difference: upstream describes each parent with the singular GetBaseItemDto over an item GetItemById returned, so each ancestor keys on its own external ids, cold or warm. Walking that chain as a query — which the first version of this fix did — moved the bug rather than fixing it, reporting a series under its guid where upstream says 368611. The chain is walked by id now, and the test for it fails against the query form and passes against the by-id form.

After

Anchor replay, main's binary against this one, fresh pair each: 741 → 692 differences. Diffing the two reports request by request: exactly one changed — movies-page-1-fields, 49 UserData.Key differences fixed, 0 new, every other request byte-identical in its difference set.

/Items/{season}/Ancestors verified live against a deliberately warm upstream: Moon Knight 368611, Breaking Bad 81189, both sides identical.

Left standing

/Items/Suggestions is described through the cached batch and always joins the ids, so upstream's answer there is cold-versus-warm dependent in the same way. It stays in the acknowledged-unmatchable class; the guid is what a browsed instance reports.

**Upstream has no rule here — it has a cache. The rule it does have is about the route, not the request.** Fixed on `main` in `294f19f` and `7be4ecb`. ## What upstream actually does Measured on genuinely fresh pairs — one request per start, nothing before it, because anything else measures the cache: | the first request after a restart | upstream's `UserData.Key` | |---|---| | a plain movie listing | `f88d8636-81e8-ca38-8e58-359ee0656ce6` | | the same, plus `Fields=…,ProviderIds` | `1316092` | | `GET /Items/{id}` | `1316092` | So the field list *does* move upstream's key on a cold server, and this issue's premise — that upstream sends the guid with `Fields` — is not upstream's rule but upstream's cache. Three mechanisms, each confirmed in the pinned source: - `ApplyNavigations` joins `BaseItemProviders` **only when the request asked for `ProviderIds`**, and `GetUserDataKeys()` reads whatever was joined. - `UserDataManager` keeps one `UserItemData` per user and item. **Only a query fills that cache and reads it**, so the first query to touch an item after a restart freezes the key every later query reports. In the scenario, the plain listing runs first and freezes the guid. - `GET /Items/{id}` goes through `GetUserData`, which never consults that cache, over an item `RetrieveItem` fetched with `.Include(e => e.Provider)` hard-coded. It derives the key every time — the one deterministic answer, and it leaves what the queries report untouched. That last point also corrects `crates/jellyfin-harness/README.md`, which claimed a by-id read poisons what later listings report. It does not: it never touches the cache. ## The rule ours now answers One key per item per user, independent of the field list: **an item's own external ids key its playback state where the route reached the item by id; a query keys on the item's id.** That is upstream everywhere upstream is deterministic. Where it is not — a listing whose first-ever request after a restart asks for `ProviderIds` — no rule can match both that and the plain listing, and the plain listing is what every client sends first. Ours matches from the second request onward, forever. "Reached by id" is a property of **how the item in hand was fetched**, not only of the route's subject. `/Items/{itemId}/Ancestors` is the case that makes the difference: upstream describes each parent with the *singular* `GetBaseItemDto` over an item `GetItemById` returned, so each ancestor keys on its own external ids, cold or warm. Walking that chain as a query — which the first version of this fix did — moved the bug rather than fixing it, reporting a series under its guid where upstream says `368611`. The chain is walked by id now, and the test for it fails against the query form and passes against the by-id form. ## After Anchor replay, main's binary against this one, fresh pair each: **741 → 692 differences**. Diffing the two reports request by request: exactly one changed — `movies-page-1-fields`, 49 `UserData.Key` differences fixed, **0 new**, every other request byte-identical in its difference set. `/Items/{season}/Ancestors` verified live against a deliberately warm upstream: Moon Knight `368611`, Breaking Bad `81189`, both sides identical. ## Left standing `/Items/Suggestions` is described through the cached batch and always joins the ids, so upstream's answer there is cold-versus-warm dependent in the same way. It stays in the acknowledged-unmatchable class; the guid is what a browsed instance reports.
Sign in to join this conversation.
No description provided.