A transcode writes a third more bytes than upstream's for the same request #190

New issue

Closed

opened 2026-08-22 23:55:26 +02:00 by Claude · 1 comment

Claude commented

2026-08-22 23:55:26 +02:00

Collaborator

Four video shapes were gated and measured against the reference instance, and
ours wrote a larger file every time — the transcoding scenario's whole request
set, taken from var/bench-tc/*/manifest.json:

h264 -> smaller box    upstream 61866    ours 82599    +33%
hevc -> another codec  upstream 1249636  ours 1481301  +19%
h264 -> .ts            upstream 106596   ours 127840   +20%
copy + mp3             upstream 76594    ours 116927   +53%

Both servers were given the same request and the same file, so the difference
is in what each asked ffmpeg for. The encoding corpus covers none of this
regime — it checks the argument list a request builds, and these four requests
build lists both servers accept, so a size difference of a third is invisible
to it.

The last row is the interesting one: a stream copy with an mp3 audio track
should be nearly the same bytes on both sides, and is half again as large
here.

Nothing about this is diagnosed. The numbers above are the whole evidence
base, and the first thing to do is put the two argument lists side by side for
one of the four.

How to know it is done

The four requests produce files within a percent of upstream's, or each
difference that remains is explained by an argument the port deliberately
spells differently — recorded here, and covered by a corpus case so that it
cannot drift back.

House rules

Tests run under cargo nextest; upstream's source is the checkout in
upstream/ at the pin.

Four video shapes were gated and measured against the reference instance, and ours wrote a larger file every time — the transcoding scenario's whole request set, taken from `var/bench-tc/*/manifest.json`: ``` h264 -> smaller box upstream 61866 ours 82599 +33% hevc -> another codec upstream 1249636 ours 1481301 +19% h264 -> .ts upstream 106596 ours 127840 +20% copy + mp3 upstream 76594 ours 116927 +53% ``` Both servers were given the same request and the same file, so the difference is in what each asked ffmpeg for. The encoding corpus covers none of this regime — it checks the argument list a request builds, and these four requests build lists both servers accept, so a size difference of a third is invisible to it. The last row is the interesting one: a stream copy with an mp3 audio track should be nearly the same bytes on both sides, and is half again as large here. Nothing about this is diagnosed. The numbers above are the whole evidence base, and the first thing to do is put the two argument lists side by side for one of the four. ## How to know it is done The four requests produce files within a percent of upstream's, or each difference that remains is explained by an argument the port deliberately spells differently — recorded here, and covered by a corpus case so that it cannot drift back. ### House rules Tests run under `cargo nextest`; upstream's source is the checkout in `upstream/` **at the pin**.

Claude referenced this issue

2026-08-23 00:24:17 +02:00

Nothing about the write path, artwork or transcoding has been measured on a real library #117

Claude commented

2026-08-23 01:07:08 +02:00

Author

Collaborator

Diagnosed and fixed in a764076. It was not the arg builders — it was the
query binder.

Four parameters of the streaming routes were spelled in a way the binder could
never reach. Keys are respelled to snake_case before serde binds them, so
videoBitRate arrives as video_bit_rate and the field was called
video_bitrate: no match, no value, silently.

videoBitRate          -> video_bit_rate            field was video_bitrate
audioBitRate          -> audio_bit_rate            field was audio_bitrate
deInterlace           -> de_interlace              field was deinterlace
enableMpegtsM2TsMode  -> enable_mpegts_m2_ts_mode  field was enable_mpegts_m2ts_mode

So a request asking for 200 kbps of video and 64 kbps of audio was transcoded
with no rate cap at all, and its audio was stream-copied because nothing
told the copy check a lower bitrate had been asked for. Both make a bigger
file, which is exactly what the byte counts said.

The two command lines for the first of the four, side by side:

upstream  -crf 23 -maxrate 200000 -bufsize 400000 … -codec:a:0 libfdk_aac -ac 1 -ab 64000
ours      -crf 23 -maxrate 0      -bufsize 0      … -codec:a:0 copy

Measured

cargo xtask bench --fixture default --workload transcoding, the whole
committed request set, before → after:

before   2 of 6 requests agreed — the four video shapes differed by 19–53%
after    6 of 6 requests agreed

Bytes per response now run 1.00–1.09 of upstream's at low concurrency and
1.00 from c=32 up, which is a response cut off at a different point rather than
a different file.

What it uncovered. With the gate passing, the workload is timed for the
first time, and the number is bad: ours sustains ~2.9 transcode requests a
second against upstream's ~240, with a p99 approaching a minute. That is #191's
question, answered — recorded there.

**Diagnosed and fixed in `a764076`. It was not the arg builders — it was the query binder.** Four parameters of the streaming routes were spelled in a way the binder could never reach. Keys are respelled to `snake_case` before serde binds them, so `videoBitRate` arrives as `video_bit_rate` and the field was called `video_bitrate`: no match, no value, silently. ``` videoBitRate -> video_bit_rate field was video_bitrate audioBitRate -> audio_bit_rate field was audio_bitrate deInterlace -> de_interlace field was deinterlace enableMpegtsM2TsMode -> enable_mpegts_m2_ts_mode field was enable_mpegts_m2ts_mode ``` So a request asking for 200 kbps of video and 64 kbps of audio was transcoded with **no rate cap at all**, and its audio was stream-copied because nothing told the copy check a lower bitrate had been asked for. Both make a bigger file, which is exactly what the byte counts said. The two command lines for the first of the four, side by side: ``` upstream -crf 23 -maxrate 200000 -bufsize 400000 … -codec:a:0 libfdk_aac -ac 1 -ab 64000 ours -crf 23 -maxrate 0 -bufsize 0 … -codec:a:0 copy ``` ## Measured `cargo xtask bench --fixture default --workload transcoding`, the whole committed request set, before → after: ``` before 2 of 6 requests agreed — the four video shapes differed by 19–53% after 6 of 6 requests agreed ``` Bytes per response now run 1.00–1.09 of upstream's at low concurrency and 1.00 from c=32 up, which is a response cut off at a different point rather than a different file. **What it uncovered.** With the gate passing, the workload is timed for the first time, and the number is bad: ours sustains ~2.9 transcode requests a second against upstream's ~240, with a p99 approaching a minute. That is #191's question, answered — recorded there.

Claude closed this issue

2026-08-23 01:07:08 +02:00