The published ffmpeg cannot run on the host, so no benchmark can start #173

New issue

Closed

opened 2026-08-21 12:28:46 +02:00 by Claude · 1 comment

Claude commented

2026-08-21 12:28:46 +02:00

Collaborator

The published upstream cannot start on the host, so cargo xtask bench cannot
run at all — which is what #116 and #117 are waiting on.

What happens

reference/publish copies ffmpeg/ffmpeg and ffmpeg/lib out of the image.
The binary's RPATH is the container's absolute /usr/lib/jellyfin-ffmpeg/lib,
so on the host it resolves nothing:

$ reference/published/ffmpeg/ffmpeg -version
error while loading shared libraries: libavdevice.so.62: cannot open shared object file

LD_LIBRARY_PATH pointing at the copied lib gets one step further and then
stops: libbluray.so.2 is a system library of the image and was never copied.

Both servers refuse to start without an ffmpeg they consider valid, so a run
dies before it measures anything:

[FTL] Main: Error while starting server
MediaBrowser.Common.FfmpegException: Failed to find valid ffmpeg

ERROR reference/published/ffmpeg/ffmpeg: names no version it can be held to

xtask bench --ffmpeg defaults to exactly this binary, and docs/benchmarks.md
says both servers are pointed at it on purpose — a stock ffmpeg has neither
tonemapx nor alphasrc and earns different arguments. So pointing the run at
/usr/bin/ffmpeg is a workaround for a search or listing workload and wrong for
anything that transcodes.

Where to start

Either the publish stage carries what the binary loads — the rest of the image's
shared libraries beside the ones already copied, with the RPATH made relative
($ORIGIN/../lib) — or the publish writes a launcher beside it that names the
directory. The first is what makes --ffmpeg mean the pinned encoder; the
second only makes it start.

Whichever it is, publish should refuse rather than write an artifact whose
ffmpeg cannot run: it already checks that the stage produced the two files, and
a -version that fails is the same class of failure one step later.

House rules

Tests run under cargo nextest; upstream's source is the checkout in upstream/
at the pin. The anchor library and the scripts that stand a pair up on it are
outside the repository; README.md says where.

The published upstream cannot start on the host, so `cargo xtask bench` cannot run at all — which is what #116 and #117 are waiting on. ## What happens `reference/publish` copies `ffmpeg/ffmpeg` and `ffmpeg/lib` out of the image. The binary's `RPATH` is the container's absolute `/usr/lib/jellyfin-ffmpeg/lib`, so on the host it resolves nothing: ``` $ reference/published/ffmpeg/ffmpeg -version error while loading shared libraries: libavdevice.so.62: cannot open shared object file ``` `LD_LIBRARY_PATH` pointing at the copied `lib` gets one step further and then stops: `libbluray.so.2` is a system library of the image and was never copied. Both servers refuse to start without an ffmpeg they consider valid, so a run dies before it measures anything: ``` [FTL] Main: Error while starting server MediaBrowser.Common.FfmpegException: Failed to find valid ffmpeg ``` ``` ERROR reference/published/ffmpeg/ffmpeg: names no version it can be held to ``` `xtask bench --ffmpeg` defaults to exactly this binary, and `docs/benchmarks.md` says both servers are pointed at it on purpose — a stock ffmpeg has neither `tonemapx` nor `alphasrc` and earns different arguments. So pointing the run at `/usr/bin/ffmpeg` is a workaround for a search or listing workload and wrong for anything that transcodes. ## Where to start Either the publish stage carries what the binary loads — the rest of the image's shared libraries beside the ones already copied, with the `RPATH` made relative (`$ORIGIN/../lib`) — or the publish writes a launcher beside it that names the directory. The first is what makes `--ffmpeg` mean the pinned encoder; the second only makes it start. Whichever it is, `publish` should refuse rather than write an artifact whose ffmpeg cannot run: it already checks that the stage produced the two files, and a `-version` that fails is the same class of failure one step later. ### House rules Tests run under `cargo nextest`; upstream's source is the checkout in `upstream/` **at the pin**. The anchor library and the scripts that stand a pair up on it are outside the repository; `README.md` says where.

Claude referenced this issue

2026-08-21 12:41:42 +02:00

The by-name second pass is unmeasured: ten materialisations per restricted search #145

Claude closed this issue

2026-08-21 14:18:11 +02:00

Claude commented

2026-08-21 14:18:21 +02:00

Author

Collaborator

Landed in 0a52954.

reference/published/ffmpeg/ffmpeg runs on the host now. An ffmpeg-portable
stage between ffmpeg and published gathers the closure the binaries load —
the four libraries the image installs rather than the package ships, plus what
those name in turn — beside the ones the deb carries, and repoints every binary
and every shipped library at $ORIGIN. publish runs the result before
recording it, and reuses an existing artifact only when its encoder still
starts.

Verified: ffmpeg -version and ffprobe -version answer, ldd resolves
everything, tonemapx and alphasrc are both there, a real encode completes,
and cargo xtask bench runs end to end with upstream starting against it.

Worth knowing for anyone touching relocate-ffmpeg.sh: ldconfig -p | awk '... exit' leaves ldconfig writing into a closed pipe, and under pipefail
the SIGPIPE kills the script with status 141 before it can name the library it
was looking for. The cache is read once into a variable and searched from there.

Landed in 0a52954. `reference/published/ffmpeg/ffmpeg` runs on the host now. An `ffmpeg-portable` stage between `ffmpeg` and `published` gathers the closure the binaries load — the four libraries the image installs rather than the package ships, plus what those name in turn — beside the ones the deb carries, and repoints every binary and every shipped library at `$ORIGIN`. `publish` runs the result before recording it, and reuses an existing artifact only when its encoder still starts. Verified: `ffmpeg -version` and `ffprobe -version` answer, `ldd` resolves everything, `tonemapx` and `alphasrc` are both there, a real encode completes, and `cargo xtask bench` runs end to end with upstream starting against it. Worth knowing for anyone touching `relocate-ffmpeg.sh`: `ldconfig -p | awk '... exit'` leaves ldconfig writing into a closed pipe, and under `pipefail` the SIGPIPE kills the script with status 141 before it can name the library it was looking for. The cache is read once into a variable and searched from there.