The search candidate cut runs over the whole narrowing, and there is no fallback when the ranking matches nothing #88

New issue

Closed

opened 2026-08-12 08:51:07 +02:00 by Claude · 1 comment

Claude commented

2026-08-12 08:51:07 +02:00

Collaborator

Two ways /Items?SearchTerm= still answers a different page from upstream, both found while closing #86 and both deliberately left out of that branch. Neither is visible to portable.json.

The candidate cut is taken over the wrong narrowing

Upstream splits a search in two. SqlSearchProvider picks candidates using only its own filters — non-placeholder, non-virtual, the CleanName/OriginalTitle match, include/exclude types, media types, parent, and user access — scores them, orders by score then id, and takes limit * 3 (100 where there is no limit). Only then does folder.GetItems apply everything else the request asked for, and TotalRecordCount is what survives that second pass.

Ours applies the whole narrowing inside the ranked subquery, so every filter upstream applies after the cut, we apply before it: years, genre_ids, is_favorite, is_played, is_resumable, is_folder, name_starts_with, series_id, parent_index_number, the location-type virtual choice, and the non-recursive parent.

Reachable two ways.

GET /Items?Recursive=true&SearchTerm=a&Filters=IsFavorite&Limit=50

Upstream ranks 150 rows and keeps the favourites among them — a short page, and a total below 150. Ours ranks 150 favourites — a full page, and a total of 150.

And with no extra filter at all, because SqlSearchProvider.ApplyParentFilter is ParentId == pid || Parents.Any(...) regardless of Recursive, while ours only reaches descendants when Recursive=true:

GET /Items?ParentId=X&SearchTerm=y

already selects a different candidate set. On the anchor's SearchTerm=a all 150 candidates survive the second pass, which is why the totals happen to agree at 150 there.

Cost. narrowing() emits one FROM+WHERE that every query form shares. Closing this means emitting two buckets — the provider's filters, and the rest — and joining a ranked subquery built from the first against a narrowing built from the second, with count() changing in step: its total must count the second pass, not the cut. It touches every caller of narrowing(), so the by-name and count forms need re-checking.

There is no fallback when the ranking matches nothing

When the provider returns zero rows, upstream leaves SearchTerm on the InternalItemsQuery and runs an entirely different search: CleanName.Contains plus an OriginalTitle LIKE (with a wildcard branch when the cleaned term holds one), ordered by MapSearchRelevanceOrder — four tiers ascending (0 exact, 1 term-as-first-word, 2 prefix, 3 rest), with OriginalTitle in the exact and prefix tiers, which is a different tier definition from the provider's 100/80/75/50 — with the request's own limit and an uncapped total.

That path notably does not exclude virtual items:

GET /Items?Recursive=true&SearchTerm=<title of an unaired episode>

comes back from upstream and not from us. Any library with missing or unaired episodes reaches it.

The same flavour is what /UserItems/Resume?searchTerm= uses — it never goes through SearchManager at all, so it gets repository tiers, no cut, and DatePlayed DESC after relevance, where we still apply provider tiers and the candidate cut.

Cost. A second search shape in the query builder (its own ranking expression, no candidate cut, no forced virtual exclusion), plus a retry in the caller rather than the builder — select() and count() are pure generators with no way to observe an empty result, so the "provider found nothing, run the other query" decision has to live in the route or the repository wrapper. The resume route then selects the repository flavour outright instead of retrying.

Two ways `/Items?SearchTerm=` still answers a different page from upstream, both found while closing #86 and both deliberately left out of that branch. Neither is visible to `portable.json`. ## The candidate cut is taken over the wrong narrowing Upstream splits a search in two. `SqlSearchProvider` picks candidates using **only its own filters** — non-placeholder, non-virtual, the `CleanName`/`OriginalTitle` match, include/exclude types, media types, parent, and user access — scores them, orders by score then id, and takes `limit * 3` (100 where there is no limit). Only *then* does `folder.GetItems` apply everything else the request asked for, and `TotalRecordCount` is what survives that second pass. Ours applies the whole narrowing inside the ranked subquery, so every filter upstream applies *after* the cut, we apply *before* it: `years`, `genre_ids`, `is_favorite`, `is_played`, `is_resumable`, `is_folder`, `name_starts_with`, `series_id`, `parent_index_number`, the location-type virtual choice, and the non-recursive parent. Reachable two ways. GET /Items?Recursive=true&SearchTerm=a&Filters=IsFavorite&Limit=50 Upstream ranks 150 rows and keeps the favourites among them — a short page, and a total below 150. Ours ranks 150 favourites — a full page, and a total of 150. And with no extra filter at all, because `SqlSearchProvider.ApplyParentFilter` is `ParentId == pid || Parents.Any(...)` **regardless of `Recursive`**, while ours only reaches descendants when `Recursive=true`: GET /Items?ParentId=X&SearchTerm=y already selects a different candidate set. On the anchor's `SearchTerm=a` all 150 candidates survive the second pass, which is why the totals happen to agree at 150 there. **Cost.** `narrowing()` emits one `FROM`+`WHERE` that every query form shares. Closing this means emitting two buckets — the provider's filters, and the rest — and joining a ranked subquery built from the first against a narrowing built from the second, with `count()` changing in step: its total must count the second pass, not the cut. It touches every caller of `narrowing()`, so the by-name and count forms need re-checking. ## There is no fallback when the ranking matches nothing When the provider returns zero rows, upstream leaves `SearchTerm` on the `InternalItemsQuery` and runs an entirely different search: `CleanName.Contains` plus an `OriginalTitle` `LIKE` (with a wildcard branch when the cleaned term holds one), ordered by `MapSearchRelevanceOrder` — four tiers **ascending** (0 exact, 1 term-as-first-word, 2 prefix, 3 rest), with `OriginalTitle` in the exact and prefix tiers, which is a different tier *definition* from the provider's 100/80/75/50 — with the request's own limit and an **uncapped** total. That path notably does **not** exclude virtual items: GET /Items?Recursive=true&SearchTerm=<title of an unaired episode> comes back from upstream and not from us. Any library with missing or unaired episodes reaches it. The same flavour is what `/UserItems/Resume?searchTerm=` uses — it never goes through `SearchManager` at all, so it gets repository tiers, no cut, and `DatePlayed DESC` after relevance, where we still apply provider tiers and the candidate cut. **Cost.** A second search shape in the query builder (its own ranking expression, no candidate cut, no forced virtual exclusion), plus a retry in the caller rather than the builder — `select()` and `count()` are pure generators with no way to observe an empty result, so the "provider found nothing, run the other query" decision has to live in the route or the repository wrapper. The resume route then selects the repository flavour outright instead of retrying.

Claude commented

2026-08-12 15:44:29 +02:00

Author

Collaborator

Both halves closed, measured against the anchor with a pair on each side of the change.

The cut now knows only the provider's filters. narrowing() emits two clauses: the ranked candidates — the term, the types, the media types, the folder aimed at, and the libraries in reach, with what never aired always excluded — joined against a narrowing built from everything else the request asked for. The total counts what that second pass kept, not the cut.

GET /Items?Recursive=true&SearchTerm=a&Years=2012&Limit=50
  upstream    13 of 13
  before     150 of 50
  after       13 of 13

A ranking that picks nothing falls back to the query's own search. Different tiers (the whole name, the term as first word, the front of the name, the rest — counting up, with the released title in the first and third), no cut, an uncapped total, and what never aired selected as the request said rather than always excluded. The choice is made on whether the ranking had any candidate, not on whether the page came back empty: a cut the rest of the query narrowed away is still a search that found something, and it answers with nothing.

GET /Items?Recursive=true&SearchTerm=18 Hours at the Paint
  upstream   1  ["18 Hours at the Paint Hall"]
  before     0  []
  after      1  ["18 Hours at the Paint Hall"]

That is also the search /UserItems/Resume?searchTerm= runs, which never goes near the ranking.

portable.json is unchanged at 60 responses, 101 differences, 59 agreeing — the 101 are /Years and belong to #89.

One thing found on the way: the integration fixture's unaired episode was inserted with no CleanName, so no search could ever have matched it. The same vacuity #86 fixed for the fixture's main insert; fixed here too, which is what let the fallback be tested at all.

Both halves closed, measured against the anchor with a pair on each side of the change. **The cut now knows only the provider's filters.** `narrowing()` emits two clauses: the ranked candidates — the term, the types, the media types, the folder aimed at, and the libraries in reach, with what never aired always excluded — joined against a narrowing built from everything else the request asked for. The total counts what that second pass kept, not the cut. GET /Items?Recursive=true&SearchTerm=a&Years=2012&Limit=50 upstream 13 of 13 before 150 of 50 after 13 of 13 **A ranking that picks nothing falls back to the query's own search.** Different tiers (the whole name, the term as first word, the front of the name, the rest — counting up, with the released title in the first and third), no cut, an uncapped total, and what never aired selected as the request said rather than always excluded. The choice is made on whether the ranking had *any* candidate, not on whether the page came back empty: a cut the rest of the query narrowed away is still a search that found something, and it answers with nothing. GET /Items?Recursive=true&SearchTerm=18 Hours at the Paint upstream 1 ["18 Hours at the Paint Hall"] before 0 [] after 1 ["18 Hours at the Paint Hall"] That is also the search `/UserItems/Resume?searchTerm=` runs, which never goes near the ranking. `portable.json` is unchanged at 60 responses, 101 differences, 59 agreeing — the 101 are `/Years` and belong to #89. **One thing found on the way:** the integration fixture's unaired episode was inserted with no `CleanName`, so no search could ever have matched it. The same vacuity #86 fixed for the fixture's main insert; fixed here too, which is what let the fallback be tested at all.

Claude closed this issue

2026-08-12 15:44:29 +02:00

Claude referenced this issue

2026-08-14 00:15:43 +02:00

/Studios search orders two studios differently from upstream #122