Three subtitle inputs still read differently from upstream: short legacy-encoded files, WEBVTT FILE with numbered cues, non-ASCII digits #252

New issue

Open

opened 2026-10-11 13:27:29 +02:00 by Claude · 0 comments

Claude commented

2026-10-11 13:27:29 +02:00

Collaborator

What is left of #251, each established against a private upstream v12.2.

Short cp1252 sidecars. UtfUnknown keeps probers above 0.20 confidence and takes the highest; Latin1Prober is capped at 0.5 and loses to the windows-1250 model on two- and three-line samples (upstream serves Seńor), and a single line gets no result at all, so upstream serves the raw bytes. The port's detector decodes these as cp1252. Longer French, German and Spanish samples agree. Reproducing it needs the single-byte language models and the multibyte frequency tables — the whole detector.
WEBVTT FILE with more than three numbered cues is handed to a second SubtitleEdit reader (WebVTTFileWithLineNumber); the port has only the one.
Non-ASCII digits where upstream's regexes use .NET \d/\s: an rgb()/rgba() colour in a STYLE block, a tag class name such as <c.١٢٣>, and a {T line in a .sub.
An ASS header carrying a .NET format item with a format spec after : is accepted and ignored; not captured.

What is left of #251, each established against a private upstream v12.2. - **Short cp1252 sidecars.** UtfUnknown keeps probers above 0.20 confidence and takes the highest; Latin1Prober is capped at 0.5 and loses to the windows-1250 model on two- and three-line samples (upstream serves `Seńor`), and a single line gets no result at all, so upstream serves the raw bytes. The port's detector decodes these as cp1252. Longer French, German and Spanish samples agree. Reproducing it needs the single-byte language models and the multibyte frequency tables — the whole detector. - **`WEBVTT FILE` with more than three numbered cues** is handed to a second SubtitleEdit reader (`WebVTTFileWithLineNumber`); the port has only the one. - **Non-ASCII digits** where upstream's regexes use .NET `\d`/`\s`: an `rgb()`/`rgba()` colour in a `STYLE` block, a tag class name such as `<c.١٢٣>`, and a `{T` line in a `.sub`. - An ASS header carrying a .NET format item with a format spec after `:` is accepted and ignored; not captured.