Your video thumbnail sprite sheet is fine. The VTT cues drift.

You hover the scrub bar and the preview shows a frame from thirty seconds ago. The sprite sheet looks fine when you open it, the WebVTT parses without errors, and every cue still points at the wrong tile. That drift isn't a player bug, and it isn't your grid math.

Quick answer: A thumbnail sprite sheet video preview is two artifacts built from one sampling interval: a tiled JPEG made with ffmpeg -i in.mp4 -vf "fps=1/10,scale=160:-2,tile=10x10" -qscale:v 4 sprite_%03d.jpg, and a WebVTT file whose cues point at sprite_001.jpg#xywh=0,0,160,90 rectangles inside it. Generate the WebVTT from the frame count and tile size FFmpeg actually produced instead of writing cues by hand, and the drift disappears. FFmpeg Micro runs the extraction and tiling pass as one API job, so you can ship preview strips without hosting an encoder.

The hover preview is two files that have to agree

A scrub-bar preview strip is not a player feature you switch on. It's a static image plus a text file, and every serious player reads the same pair: one sprite holding every sampled frame in a grid, and a WebVTT track whose cues carry a Media Fragments URI. The #xywh= syntax comes from the W3C Media Fragments URI 1.0 spec, and it tells the player which rectangle of the sprite to crop for a given moment.

Four cues for a 10-second interval look like this:

WEBVTT

00:00:00.000 --> 00:00:10.000
sprite_001.jpg#xywh=0,0,160,90

00:00:10.000 --> 00:00:20.000
sprite_001.jpg#xywh=160,0,160,90

00:00:20.000 --> 00:00:30.000
sprite_001.jpg#xywh=320,0,160,90

00:00:30.000 --> 00:00:40.000
sprite_001.jpg#xywh=480,0,160,90

The tiling exists for one reason: a 60-minute video sampled every 10 seconds is 360 thumbnails. As individual files that's 360 HTTP requests fired off during a single scrub. As four sprites of 100 tiles each, it's four. The browser decodes each sheet once and crops from memory.

Build the sprite sheet with FFmpeg

The whole extraction is one filter chain. Sample, scale, tile, write:

ffmpeg -i input.mp4 \
  -vf "fps=1/10,scale=160:-2,tile=10x10" \
  -qscale:v 4 \
  sprite_%03d.jpg

fps=1/10 means one frame every 10 seconds, not 10 frames per second. That inversion trips people up constantly. scale=160:-2 sizes each tile to 160 pixels wide and picks a height divisible by 2, which matters more than it looks: -1 will happily hand you an odd height on some sources and break downstream encoders, the same class of problem covered in FFmpeg's "height not divisible by 2" error. tile=10x10 packs 100 frames into each output image, and %03d makes FFmpeg roll over to a new file every time the grid fills.

Filter order matters for speed. Put fps before scale and FFmpeg scales 360 frames. Reverse them and it scales every single frame in the source, which on a 60-minute 1080p file is roughly 90,000 wasted scale operations.

Get the numbers FFmpeg actually produced

The one-liner is fine for a demo and dangerous in a pipeline, because you never learn how many tiles came out or what size they ended up. Split it into two passes and both facts become observable:

mkdir -p frames
ffmpeg -i input.mp4 -vf "fps=1/10,scale=160:-2" -qscale:v 4 frames/t_%05d.jpg
ffmpeg -framerate 1 -i frames/t_%05d.jpg -vf "tile=10x10" -qscale:v 4 sprite_%03d.jpg
ffprobe -v error -select_streams v:0 -show_entries stream=width,height \
  -of csv=p=0 frames/t_00001.jpg

That last command prints something like 160,90. Now you have the real tile dimensions and a real frame count from ls frames | wc -l, instead of two numbers you assumed. The temp directory costs you a few megabytes and buys exact agreement between the sprite and the VTT.

Generate the WebVTT from the same interval

Every drift bug I've seen comes from the same place: the VTT was written by a different process than the sprite, using a duration read off the container instead of the frames on disk. The fix is to make one script own both numbers. This one reads the frame list and the probed tile size, then emits cues that cannot disagree with the image:

#!/usr/bin/env python3
import glob, subprocess

INTERVAL = 10          # seconds between thumbnails
COLS, ROWS = 10, 10    # must match the tile= filter exactly

frames = sorted(glob.glob("frames/t_*.jpg"))
probe = subprocess.run(
    ["ffprobe", "-v", "error", "-select_streams", "v:0",
     "-show_entries", "stream=width,height", "-of", "csv=p=0", frames[0]],
    capture_output=True, text=True).stdout.strip()
TW, TH = (int(v) for v in probe.split(","))

def ts(seconds):
    h, rem = divmod(seconds, 3600)
    m, s = divmod(rem, 60)
    return f"{h:02d}:{m:02d}:{s:06.3f}"

per_sheet = COLS * ROWS
lines = ["WEBVTT", ""]
for i in range(len(frames)):
    cell = i % per_sheet
    x, y = (cell % COLS) * TW, (cell // COLS) * TH
    lines += [
        f"{ts(i * INTERVAL)} --> {ts((i + 1) * INTERVAL)}",
        f"sprite_{i // per_sheet + 1:03d}.jpg#xywh={x},{y},{TW},{TH}",
        "",
    ]

open("thumbnails.vtt", "w").write("\n".join(lines))

Three constants drive everything: the interval, the column count, and the row count. Change the tile grid and you change one line in both the FFmpeg command and this script. If you're already running FFmpeg from Python, the ffmpeg in Python patterns fold this straight into the same job.

One detail worth clamping: the final cue's end time runs past the actual video duration by up to one interval. Most players don't care because they select cues by hover position, but if yours renders a cue list, clamp the last end time to the probed duration.

Split long videos across several sprites

A single sprite holding every thumbnail from a three-hour video is a decode bomb on mobile. Mobile Safari has a long-standing ceiling around 16.7 million pixels per decoded image, roughly 4096x4096, and older Android GPUs cap textures at 4096 pixels per side. Past that the image gets downsampled or silently refused, and your preview strip goes blank on exactly the devices you can't debug.

Memory is the sharper constraint. A decoded image costs width x height x 4 bytes of RGBA. A 27x27 grid of 160x90 tiles is 4320x2430, which is about 42 MB resident per sheet. A 10x10 grid at 1600x900 is 5.8 MB. Keep sheets in that range and %03d handles the rest.

FFmpeg flushes the final partial sheet at end of file and fills the unused cells with the tile filter's color value, black by default. So your last sprite is always full-size and usually partly empty. This is precisely why cue count has to come from the frame count and not from sheets multiplied by grid size.

Generating those 360 frames means decoding the entire source stream. On a modern CPU, 1080p H.264 decodes at roughly 10 to 20 times realtime, so a 60-minute file takes three to six minutes of wall clock per video. That's an awkward number: too slow for a request handler, too spiky to justify a always-on worker. Sending the extraction and tiling pass to FFmpeg Micro as one job keeps the exact same filter chain and moves the six minutes off your box, with the frame count coming back in the job output so your VTT generator still reads real numbers. The docs have the request shape.

Hand the storyboard to the player

Most players want the VTT URL and nothing else. Plyr takes it as a config option:

const player = new Plyr('#player', {
  previewThumbnails: { enabled: true, src: '/thumbs/thumbnails.vtt' }
});

JW Player takes a track with kind: 'thumbnails'. Video.js needs the videojs-vtt-thumbnails plugin pointed at the same file. Shaka Player accepts a thumbnail track directly, and for DASH packaging the thumbnail tiles are described with the DASH-IF thumbnail_tile essential property on a separate AdaptationSet. If you're already packaging with an HLS ladder built in FFmpeg, the sprite job runs alongside it against the same mezzanine file.

Pitfalls that produce drift

Five sprite-and-VTT pitfalls cost real debugging hours, and none of them throw an error.

When a sprite sheet is the wrong call

Sprite sheets are the right tool for on-demand files with a fixed duration. For anything under about 30 seconds, skip the machinery: three or four individual stills cost less to build and less to serve. For live streams, sprites need continuous regeneration and a rolling VTT, which is a different problem with a different shape, and a managed live packaging service usually earns its money there. And if your platform already emits storyboards as part of its packaging step, generating a second set by hand just gives you two sources of truth to keep in sync.

FAQ

How many thumbnails should a sprite sheet have?

Aim for 100 to 200 tiles per sheet at 160 pixels wide, which keeps each sprite under 2000x1200 and well inside every mobile decode limit. Pick the interval from duration: 5 seconds for clips under 10 minutes, 10 seconds up to an hour, 20 seconds beyond that.

Why do my WebVTT thumbnail cues show the wrong frame?

Wrong-frame cues almost always mean the VTT was generated from assumed numbers rather than measured ones. The two usual causes are a tile height that came out different from what you hardcoded, because scale=160:-1 derived it from the source aspect ratio, and a cue count computed as duration / interval when FFmpeg emitted one more or one fewer frame.

Can I use WebP or PNG instead of JPEG for the sprite?

WebP works in every current browser and typically saves 25 to 30 percent over JPEG at matching quality, so it's a reasonable swap for the sprite. PNG is a bad fit because video frames are photographic and lossless compression makes the sheet several times larger for no visible gain.

Do I need a separate sprite sheet for mobile?

One sheet at 160 pixels wide covers phones and desktops for most players, since preview popups render around 120 to 200 pixels wide anyway. Build a second 320-pixel set only if your player scales previews up on wide screens and the upscaling looks soft.

How long does generating a sprite sheet take?

Generating a sprite sheet requires a full decode of the source, so budget three to six minutes for a 60-minute 1080p H.264 file on a typical CPU core. Shorter clips scale down roughly linearly, and the tiling pass itself is negligible next to the decode.

If you'd rather not keep a machine warm for a job that runs three minutes and then sits idle, run the extraction and tiling pass through the API and keep the twenty lines of VTT generation in your own code, where you can see the numbers. Sign up free and the same filter chain in this post works as one job on the free tier.