The ASS Subtitle Format: Everything That Can Go Wrong Will

The ASS (Advanced SubStation Alpha) format was designed for anime fansubs in the early 2000s. It is now the backbone of modern subtitle rendering — ffmpeg's subtitles filter uses libass, YouTube accepts ASS-derived timing, and every video editor that claims "professional captioning" is probably writing ASS under the hood.

It is also a format where everything can go wrong in ways that are invisible until you watch the video.

I hit four distinct classes of ASS bugs while building the YouTube Shorts caption pipeline. None of them are documented in one place. This is that place.

The Setup

The pipeline takes audiobook audio, runs it through WhisperX for word-level timestamps, then generates portrait-mode videos with burned-in captions. The caption text layout uses a three-zone system (top, middle, bottom of the screen), and the timing comes directly from WhisperX's word alignment data.

The output format is ASS — because ffmpeg's subtitles filter renders it natively, and because ASS supports per-word karaoke timing, positioned text, and styled regions. Everything you need for captions that look intentional rather than auto-generated.

Then the bugs started.

Bug 1: The splitn Off-by-One

The first step in generating ASS from word-level timing is grouping words into caption lines. Each line should display for a readable duration — roughly 1.5 to 3 seconds — and contain a natural phrase.

The grouping function used a sliding window over the word list, splitting at phrase boundaries. The split logic used splitn to divide the ASS dialogue line's text field at the boundary between two caption groups.

The problem: splitn(n, &str) in Rust splits a string into at most n parts. But the ASS dialogue line format separates fields with commas, and the text field — the last field — can contain commas. When the splitter ran on a line like:

Dialogue: 0,0:00:01.00,0:00:04.50,Default,,0,0,0,,Hello, world. This is a test.

splitn(10, ',') would split on the comma in "Hello, world." — putting "world." into what the code thought was the text field, and shifting everything after it into a non-existent eleventh field.

ASS has no escaping for commas in the text field. The format spec says the text field is "everything after the tenth comma," but splitn does not know that. It splits on every comma.

The fix was to split exactly 10 times and treat the remainder as the text field — not to split on all commas and reassemble. In Rust terms:

let parts: Vec<&str> = line.splitn(10, ',').collect();
let text = parts[9]; // everything after the 10th comma, unsplit

This is a one-character fix in practice (splitn(10, ...) instead of split(',')), but it took three hours to diagnose because the bug only manifested when caption text contained a comma. Which, in English prose, is quite often.

Bug 2: Karaoke Dedup Destroying Valid Events

ASS supports karaoke timing with \k tags — each tag specifies the duration a word should be highlighted. The pipeline generated these tags from WhisperX's word-level timestamps.

The ASS writer had a "dedup" pass that removed consecutive dialogue events with identical start and end times. This was supposed to clean up a different bug where WhisperX occasionally produced duplicate word entries.

But karaoke-style captions don't use separate dialogue events per word. They use one dialogue event with multiple \k tags. The dedup pass was comparing the start time of the dialogue event — which was the start time of the first word in the line — across all lines. Two consecutive caption lines that started at the same timestamp (possible if WhisperX produced overlapping segments) were being collapsed into one, losing the second line entirely.

The dedup logic was correct for event-per-word subtitle formats. It was wrong for line-level grouping with karaoke timing.

The fix was to dedup on the full content hash of the dialogue event, not just the start time. Two events with the same start time but different text are not duplicates — they are different caption lines that happen to begin simultaneously (which is itself a bug worth investigating, but not one the dedup pass should silently swallow).

Bug 3: Group Bleed

The three-zone layout system assigns each caption to a screen region: top, middle, or bottom. This is done via ASS styles — Style: Top, Style: Middle, Style: Bottom — each with different margins and alignment.

The problem appeared when a caption assigned to the "Top" zone was rendering in the "Middle" zone. Not occasionally. Every time a specific word appeared.

The cause: the style assignment was based on the caption group's index in the output vector. But the output vector was being modified in place during generation — groups were being merged when they were too short, split when they were too long. The index no longer corresponded to the zone assignment.

Zone assignment was coupled to position in a mutable data structure. When the structure changed, the assignment drifted.

The fix was to assign zones as an explicit field on each caption group object, set at creation time and never recomputed:

struct CaptionGroup {
    text: String,
    start: Time,
    end: Time,
    zone: Zone, // Top, Middle, Bottom — set once, never derived
}

This is a classic separation-of-concerns issue, but it manifested as a visual bug that looked like a rendering problem rather than a data problem. I spent an hour investigating libass margin calculations before realizing the wrong style name was being written to the ASS file.

Bug 4: Caption Flicker From 1-Frame Gaps

The final video had captions that flickered — briefly disappearing for one frame between consecutive lines. It was barely visible at 30fps but obvious at 60fps, and it looked unprofessional.

The cause: consecutive caption lines had a 1-frame gap between the end of one line and the start of the next. WhisperX's word timestamps are accurate to roughly 10ms, and when the pipeline rounded to ASS's centisecond timing format, rounding errors created gaps.

ASS timing is in centiseconds (1/100 second). A word ending at 1.045 seconds becomes 0:00:01.04 in ASS. The next word starting at 1.050 seconds becomes 0:00:01.05. That is a 1-centisecond gap — which at 30fps is almost exactly one frame.

One frame of empty screen between captions is visible as flicker. The fix is not to eliminate the gap — it is to overlap the captions by at least one frame so libass never has a frame with no active dialogue event.

The fix was to extend each caption's end time by 2 centiseconds (0.02 seconds). This creates a slight overlap between consecutive lines, but since they are in different screen zones, the overlap is invisible — the outgoing caption is already fading while the incoming caption is fading in. No visible artifact, no flicker.

// Extend end time by 2cs to prevent 1-frame flicker gaps
let adjusted_end = original_end + Duration::from_millis(20);

The Pattern

Every one of these bugs shares a shape: the ASS format looks simple, so the code that generates it does not defend against its edge cases.

ASS is a text format. You can write it by hand. You can generate it with println!. There is no schema validator, no type system, no compiler that catches a malformed dialogue line. The format spec is a wiki page from 2004, and the reference implementation is a C library that silently renders whatever you give it.

When the output looks wrong, you have to work backward from a visual artifact to a data generation bug — through a rendering engine that makes no complaints.

The practical lesson: when generating ASS files, validate the output structurally. Check that every dialogue line has exactly 10 comma-separated fields. Check that no two events have identical start times unless they are in different styles. Check for sub-frame gaps between consecutive events in the same zone. These checks take five minutes to write and save hours of visual debugging.

The ASS format is not hard. It is just unforgiving in ways that only become visible at 30 frames per second.