ephyra-plugin-asr

Generated subtitles for the videos in an ephyra library that have no subtitle track of any kind.

A scheduled pass walks the library, and for every film or episode carrying no subtitle stream it extracts one audio track, transcribes it, cuts the result into cues, and files an SRT beside the media.

One WebAssembly component and nothing else. The only thing outside the server is the transcriber.

Installing

Add the repository once, under Dashboard → Plugins → Repositories:

https://git.shiverpeak.xyz/alex/ephyra-plugin-asr/raw/branch/main/manifest.json

Every release is offered there from then on, under Catalogue → Subtitles.

Or the same three files by hand, in a folder named by the manifest's id:

rustup target add wasm32-wasip2
cargo build --release --target wasm32-wasip2

<program data>/plugins/asr/
    plugin.toml
    plugin.wasm
    settings.html

Then set the transcriber's address on the plugin's own page, or in the server's configuration:

[plugins.asr]
enabled = true
endpoint = "http://127.0.0.1:8080/v1/audio/transcriptions"
api_key = ""
model = "Systran/faster-whisper-large-v3"
language = "en"            # what the audio is sent as; empty lets it detect
prompt = ""                # vocabulary the model is primed with
subtitle_language = "eng"  # only for a track that names no language
include = ""               # or movies / episodes
title_filter = ""

Everything but enabled is the plugin's own and reaches it as JSON. Saving the page takes effect on the next pass; only enabled and grants want a restart.

When it runs

Nightly at 03:00, out of the box. The manifest declares that as the job's default, so an install that is left alone still transcribes what arrives. A pass over a library is hours of the transcriber's time and there is nothing to gain from looking oftener than new media turns up.

Change it under Dashboard → Scheduled Tasks → Generated Subtitles, or run one by hand from the same page. A trigger set there is stored and replaces the default entirely.

There is no queue to keep: each pass walks the library and does whatever has no subtitle track, so anything added since the last pass is picked up on the next one. A pass asked for while one is already running is refused rather than stacked.

What a pass does

1. Walks every film and episode, a page at a time.
2. Skips the ones that already have a subtitle stream, and the ones the filter leaves out. Both are decided off the page the walk already carries, so neither costs a call.
3. For each of the rest: picks an audio track, has the host extract it as 16 kHz mono PCM, sends it to the transcriber, cuts the answer into cues, and has the host file the SRT beside the media.

The audio track is not guessed at. Tracks in the configured language are preferred; among whichever set that leaves, the file's own default wins, then a sole track. Anything still ambiguous is an error for that item rather than a guess — filing a commentary track's transcript leaves the item looking done. The fallback to all tracks happens only when none matches the language, never when several do.

One item failing does not stop the pass. A library will hold a file the encoder cannot open. It is logged, counted, and the walk carries on.

A pass runs for as long as it takes. A job is the one plugin capability nothing is waiting on, and the host holds it to an hour rather than the thirty seconds every other call gets.

Stopping it is the task's own Stop button. The component checks between items, and the host checks while the encoder runs — but a transcription already in flight is not interrupted, so Stop takes effect within one item rather than at once.

Resuming

One file per item in the plugin's own directory, named for the language that was actually spoken, which is not always the configured one:

<item id>.<language>.json      the transcriber's answer, verbatim

A second pass reuses it where it finds one, so an item whose transcription succeeded but whose placement did not is refiled without asking the transcriber again. The extracted audio is deleted as soon as the transcriber has answered, and whether or not it answered usefully — a library's worth of PCM would fill the disk.

The JSON is kept because it cannot be recovered from the SRT: it carries word timestamps, so the cue rules can be changed and re-run without asking the transcriber a second time.

What says an item is done is the walk. Placing a file beside the media makes the item stale, so the subtitle becomes a stream and has-subtitles turns true — an item this pass finished is one the next pass never sees. The server's own answer is the whole record; this plugin keeps no account of what it has placed.

That answer is not immediate: the read again is a pass over the library, held until the placing stops. A second pass started before it lands walks the same items again — and files nothing, because a placement that finds its own name answers where the file already is rather than putting a numbered copy beside it. What it costs is a wasted transcription, not a duplicate.

Delete the .srt beside the media and the item comes back into the walk on the next scan, which is the only thing that ought to bring it back.

What crosses the boundary

Nothing that would let this reach the library on its own. The component names an item id, a stream index, and files in its own directory; the host resolves the media, runs the encoder, sends the file and places the result.

task-host.extract-audio    one track, as PCM, into the plugin's directory
task-host.place-subtitle   bytes, beside the media, as an upload lands
task-host.post-file        that file, to the transcriber

post-file names the file rather than handing it over, so a hundred megabytes of audio never crosses a sixty-four megabyte instance. It is also the one outbound call no recording can answer — a cassette keys on a URL and this carries a body — which is why the transcriber's address is the one thing here that is genuinely off the server.

The net grant is not declared and is not needed.

The interface pin

wit/deps/ephyra-plugin/plugin.wit is the ephyra:plugin package at the version wit_version names in plugin.toml — 0.2.0, which is the first with extract-audio, place-subtitle and post-file. It is copied from the server and never edited; a server refuses a component built against another minor.

wit/world.wit beside it is this repository's own, and declares the one thing the pinned package does not: a world exporting task and http-handler together. The host binds one world per capability, and both are satisfied by a component exporting the two interfaces.

Tests

cargo test --lib

Covers what can be wrong without anything crossing the boundary: the audio-track choice, the language a subtitle is filed under, and the cue rules. What they do not cover is the crossing itself — that is an install.

Releasing

git tag -a v0.2.0 -m "what changed"
git push origin v0.2.0

.forgejo/workflows/release.yml does the rest: it builds the component, attaches asr-0.2.0.zip to the release, and rewrites manifest.json from every release the forge holds. The tag must name the version plugin.toml declares, and the tag's own message is the changelog the catalogue shows.

The index is derived, not accumulated. It is rebuilt whole from the releases each run — so a wrong entry is repaired by running the workflow again rather than by editing the file, and a change to catalogue.toml (the card: owner, category, the two blurbs, and the lowest server version a release is offered to) reaches every published version at the next run.

Licence

GPL-3.0-only, as the server is.