A scan has never been timed beside upstream's, and no fixture can be #192

New issue

Closed

opened 2026-08-23 00:23:49 +02:00 by Claude · 1 comment

Claude commented

2026-08-23 00:23:49 +02:00

Collaborator

#92 measures a scan of ours alone. Nothing has ever timed the same scan on
upstream beside it, and neither fixture this project has can be asked to.

The anchor is a database and a 7.5 GB metadata directory copied without
the media files. A scan of it finds nothing to scan.
The reference library is nineteen files of about five seconds each. A
scan of it finishes in the noise, and the number would describe the walk
rather than the work.
The click-through instance runs on a real library over CIFS, and no
upstream is standing on it.

So this is not a request set and not a run: it needs a library both servers can
be pointed at, big enough that the scan is the measurement, and an arrangement
that gives each the same media without one warming the other's file cache
first. A scan is also not a load — it is one timed operation per server, with a
progress endpoint to poll, so ephyra-bench is the wrong tool and
cargo xtask is where the driving would go.

Worth settling before any of it: whether a scan comparison is worth a fixture
of its own at all, given that the scan's own numbers (#92) already say where
the time goes.

How to know it is done

A full scan of one library, timed on both servers on the same host with the
same media, with what each wrote afterwards compared — the item count, the
people, and the pictures — so that a faster scan that did less work is not
read as a faster scan.

House rules

Tests run under cargo nextest; upstream's source is the checkout in
upstream/ at the pin.

`#92` measures a scan of ours alone. Nothing has ever timed the same scan on upstream beside it, and neither fixture this project has can be asked to. - The **anchor** is a database and a 7.5 GB metadata directory copied without the media files. A scan of it finds nothing to scan. - The **reference** library is nineteen files of about five seconds each. A scan of it finishes in the noise, and the number would describe the walk rather than the work. - The **click-through instance** runs on a real library over CIFS, and no upstream is standing on it. So this is not a request set and not a run: it needs a library both servers can be pointed at, big enough that the scan is the measurement, and an arrangement that gives each the same media without one warming the other's file cache first. A scan is also not a load — it is one timed operation per server, with a progress endpoint to poll, so `ephyra-bench` is the wrong tool and `cargo xtask` is where the driving would go. Worth settling before any of it: whether a scan comparison is worth a fixture of its own at all, given that the scan's own numbers (`#92`) already say where the time goes. ## How to know it is done A full scan of one library, timed on both servers on the same host with the same media, with what each wrote afterwards compared — the item count, the people, and the pictures — so that a faster scan that did less work is not read as a faster scan. ### House rules Tests run under `cargo nextest`; upstream's source is the checkout in `upstream/` **at the pin**.

Claude referenced this issue

2026-08-23 00:24:17 +02:00

Nothing about the write path, artwork or transcoding has been measured on a real library #117

Claude commented

2026-08-23 02:00:09 +02:00

Author

Collaborator

Measured, with machinery that commits: cargo xtask scan-race (2055c06).

The fixture problem this issue names is real, so the run makes its own: both
servers start from nothing — an empty data directory, the startup wizard,
one library — over a tree of films hard linked from a clip the reference media
already holds. The walk, the naming rules, the probe and the writes are all
real work on real files; only the decoded picture is repetitive, which is the
one thing a scan does not branch on.

films   upstream    ours   ours/up    items found (both)
  300      6.41s   7.05s      1.10          301
 2000     36.06s  50.54s      1.40         2001

Both file the same number of rows, which is the check that a faster scan is not
a smaller one. Ours is 10% slower at three hundred films and 40% slower at two
thousand — it loses ground as the library grows, which is the shape of
per-item work rather than a fixed cost. Cold start is the other way round: 212 ms
against 3.8 s.

So the answer to what this issue asked to settle — whether a scan comparison is
worth a fixture of its own — is yes, and it did not need the fixture the issue
imagined. What it needs next is where those 14 seconds go, which is #92's
question asked against a second server rather than in isolation.

One thing the run turned up on the way, filed as #193: a server whose
configuration declares fewer libraries than its database holds fails its whole
scan on a foreign key. That is what made the first attempt at this measurement
answer zero items.

**Measured, with machinery that commits: `cargo xtask scan-race` (`2055c06`).** The fixture problem this issue names is real, so the run makes its own: both servers start from **nothing** — an empty data directory, the startup wizard, one library — over a tree of films hard linked from a clip the reference media already holds. The walk, the naming rules, the probe and the writes are all real work on real files; only the decoded picture is repetitive, which is the one thing a scan does not branch on. ``` films upstream ours ours/up items found (both) 300 6.41s 7.05s 1.10 301 2000 36.06s 50.54s 1.40 2001 ``` Both file the same number of rows, which is the check that a faster scan is not a smaller one. Ours is 10% slower at three hundred films and **40% slower at two thousand** — it loses ground as the library grows, which is the shape of per-item work rather than a fixed cost. Cold start is the other way round: 212 ms against 3.8 s. So the answer to what this issue asked to settle — whether a scan comparison is worth a fixture of its own — is yes, and it did not need the fixture the issue imagined. What it needs next is where those 14 seconds go, which is `#92`'s question asked against a second server rather than in isolation. **One thing the run turned up on the way**, filed as `#193`: a server whose configuration declares fewer libraries than its database holds fails its whole scan on a foreign key. That is what made the first attempt at this measurement answer zero items.

Claude closed this issue

2026-08-23 02:00:09 +02:00