The byte-exact JPEG encoder is only used for trickplay #149

New issue

Closed

opened 2026-08-16 15:13:38 +02:00 by Claude · 1 comment

Claude commented

2026-08-16 15:13:38 +02:00

Collaborator

#140 landed crates/jellyfin-images/src/jpeg_encode.rs, whose output is
byte-identical to libjpeg-turbo's — verified at 793/793 over 61 images × 13
qualities against cjpeg -baseline -optimize -sample 2x2 -dct int, including
1×1, odd width and odd height, grayscale content, both sides of the
5000/q ↔ 200-2q branch at quality 50 and 51, and quality 100.

Only trickplay.rs calls it. The served-image path still encodes through
image::codecs::jpeg::JpegEncoder (crates/jellyfin-images/src/lib.rs:306), and
collages are PNG.

#140's own aside was that a byte-exact encoder would close the position
collage.rs records — that a picture this server draws is not upstream's
bytes. It did not: a scaled JPEG this server serves is still not upstream's
bytes, and it now could be, for the cost of routing that path through the
encoder that already exists.

What has to be settled first

The encoder is exact given identical pixels, so the scaling step is where the
two would still differ, and that is the whole of the work:

What quality and subsampling upstream's served-image path actually asks Skia
for per request. It is not necessarily 90 — DEFAULT_QUALITY is what a
request that names none gets, and a request may name one.
Whether the resampler agrees. The blurhash port is the precedent and the
warning: matching Skia there needed the scale factor applied twice, nearest
sampling with a float reciprocal nudged one unit in the last place, and fancy
chroma upsampling off below a two-pixel block.

If the resampler does not agree, this closes as "the encoder is exact and the
scaler is not, and here is the measurement" — which is a better recorded
position than the one collage.rs carries today.

House rules

Tests run under cargo nextest; upstream's source is the checkout in
upstream/ at the pin. Generated artifacts are regenerated, never edited.
The answer keys under crates/jellyfin-images/tests/edges/ are generated with
-baseline: cjpeg defaults to force_baseline=FALSE and writes 16-bit
quantisation tables at low qualities, which is not what Skia asks for.

#140 landed `crates/jellyfin-images/src/jpeg_encode.rs`, whose output is byte-identical to libjpeg-turbo's — verified at 793/793 over 61 images × 13 qualities against `cjpeg -baseline -optimize -sample 2x2 -dct int`, including 1×1, odd width *and* odd height, grayscale content, both sides of the `5000/q` ↔ `200-2q` branch at quality 50 and 51, and quality 100. **Only `trickplay.rs` calls it.** The served-image path still encodes through `image::codecs::jpeg::JpegEncoder` (`crates/jellyfin-images/src/lib.rs:306`), and collages are PNG. #140's own aside was that a byte-exact encoder would close the position `collage.rs` records — that a picture this server *draws* is not upstream's bytes. It did not: a scaled JPEG this server serves is still not upstream's bytes, and it now could be, for the cost of routing that path through the encoder that already exists. ## What has to be settled first The encoder is exact **given identical pixels**, so the scaling step is where the two would still differ, and that is the whole of the work: - What quality and subsampling upstream's served-image path actually asks Skia for per request. It is not necessarily 90 — `DEFAULT_QUALITY` is what a request that names none gets, and a request may name one. - Whether the resampler agrees. The blurhash port is the precedent and the warning: matching Skia there needed the scale factor applied twice, nearest sampling with a float reciprocal nudged one unit in the last place, and fancy chroma upsampling off below a two-pixel block. If the resampler does not agree, this closes as "the encoder is exact and the scaler is not, and here is the measurement" — which is a better recorded position than the one `collage.rs` carries today. ### House rules Tests run under `cargo nextest`; upstream's source is the checkout in `upstream/` **at the pin**. Generated artifacts are regenerated, never edited. The answer keys under `crates/jellyfin-images/tests/edges/` are generated with `-baseline`: `cjpeg` defaults to `force_baseline=FALSE` and writes 16-bit quantisation tables at low qualities, which is not what Skia asks for.

Claude added the

encoding

labels

2026-08-16 15:13:38 +02:00

Claude referenced this issue

2026-08-16 15:15:37 +02:00

A JPEG encoder that agrees with Skia, byte for byte #140

Claude commented

2026-08-16 17:03:00 +02:00

Author

Collaborator

Landed in 9139e6c. Closes as the issue anticipated: the encoder is exact and
the scaler is not, and here is the measurement.

The encoder half

scale() now encodes through encode_jpeg rather than image::codecs::jpeg,
so the served path and the trickplay sheets go through the same byte-exact
encoder. Collages stay PNG.

What has to be settled first — settled

Upstream's served-image path asks for quality 100, not 90. The
documentation on GetImageInternal says "Defaults to 90", and it is wrong for
this route: ImageController.cs:1910 is Quality = quality ?? 100. Measured
against a pinned server rather than read off the source, on one poster at
maxWidth=200&format=jpg:

quality	bytes
(none)	22386
100	22386
90	10571
50	4957

Ninety is what the account picture (ImageController.cs:1505) and the
splashscreen (:1696) pass, and this server already passed ninety for both.
The item route now passes a hundred; it was serving half-size pictures.

The scaler half — it does not agree

Skia downscales through a mipmap chain sampled trilinearly
(SKSamplingOptions(SKFilterMode.Linear, SKMipmapMode.Linear)) and upscales
with Mitchell cubic (SkiaEncoder.cs:542-566). Nothing the image crate offers
reproduces it. Measured against what a pinned server served for
tests/pictures/band-500x334.jpg, mean absolute difference per sample:

width	round trip alone	box	triangle	catmullrom	gaussian	lanczos3
50	5.31	2.60	3.56	2.95	4.36	3.02
100	3.27	1.59	1.28	1.07	1.81	1.24
137	2.58	1.19	1.21	0.87	1.55	1.00
250	1.60	0.76	1.67	1.20	1.86	1.12
400	0.98	1.29	0.85	0.54	1.05	0.55

The same measured on three film posters (1000x1500 to 2000x3000, not
committable), where the round trip is a cleaner floor:

	round trip alone	lanczos3
mean, across sizes	0.18 – 1.43	0.55 – 5.51
max sample	11 – 30	37 – 105

Lanczos3 — what this server already used — is the closest of what is on offer,
tied with CatmullRom. No size agrees byte for byte at any filter. So the
resampler stays as it was, and the position is now a check rather than a
comment: crates/jellyfin-images/src/scaled.rs holds five committed answer keys
and asserts both bounds, and it fails the day a resampler makes one agree.

The keys are of a picture already in tests/pictures/, uploaded to a container
of its own over a copy of the enriched snapshot and read back — so the source is
synthetic and the answer is regenerable.

Landed in `9139e6c`. Closes as the issue anticipated: **the encoder is exact and the scaler is not, and here is the measurement.** ## The encoder half `scale()` now encodes through `encode_jpeg` rather than `image::codecs::jpeg`, so the served path and the trickplay sheets go through the same byte-exact encoder. Collages stay PNG. ## What has to be settled first — settled **Upstream's served-image path asks for quality 100, not 90.** The documentation on `GetImageInternal` says "Defaults to 90", and it is wrong for this route: `ImageController.cs:1910` is `Quality = quality ?? 100`. Measured against a pinned server rather than read off the source, on one poster at `maxWidth=200&format=jpg`: | quality | bytes | |---|---| | (none) | 22386 | | 100 | 22386 | | 90 | 10571 | | 50 | 4957 | Ninety is what the account picture (`ImageController.cs:1505`) and the splashscreen (`:1696`) pass, and this server already passed ninety for both. The item route now passes a hundred; it was serving half-size pictures. ## The scaler half — it does not agree Skia downscales through a mipmap chain sampled trilinearly (`SKSamplingOptions(SKFilterMode.Linear, SKMipmapMode.Linear)`) and upscales with Mitchell cubic (`SkiaEncoder.cs:542-566`). Nothing the `image` crate offers reproduces it. Measured against what a pinned server served for `tests/pictures/band-500x334.jpg`, mean absolute difference per sample: | width | round trip alone | box | triangle | catmullrom | gaussian | lanczos3 | |---|---|---|---|---|---|---| | 50 | 5.31 | 2.60 | 3.56 | 2.95 | 4.36 | 3.02 | | 100 | 3.27 | 1.59 | 1.28 | 1.07 | 1.81 | 1.24 | | 137 | 2.58 | 1.19 | 1.21 | 0.87 | 1.55 | 1.00 | | 250 | 1.60 | 0.76 | 1.67 | 1.20 | 1.86 | 1.12 | | 400 | 0.98 | 1.29 | 0.85 | 0.54 | 1.05 | 0.55 | The same measured on three film posters (1000x1500 to 2000x3000, not committable), where the round trip is a cleaner floor: | | round trip alone | lanczos3 | |---|---|---| | mean, across sizes | 0.18 – 1.43 | 0.55 – 5.51 | | max sample | 11 – 30 | 37 – 105 | Lanczos3 — what this server already used — is the closest of what is on offer, tied with CatmullRom. **No size agrees byte for byte at any filter.** So the resampler stays as it was, and the position is now a check rather than a comment: `crates/jellyfin-images/src/scaled.rs` holds five committed answer keys and asserts both bounds, and it fails the day a resampler makes one agree. The keys are of a picture already in `tests/pictures/`, uploaded to a container of its own over a copy of the enriched snapshot and read back — so the source is synthetic and the answer is regenerable.

Claude closed this issue

2026-08-16 17:04:01 +02:00