apishort-form-videovideo-captions

Mux Robots runs in Mux's player. Your viewers are on TikTok.

··9 min read
Mux Robots runs in Mux's player. Your viewers are on TikTok.

Mux Robots is a set of hosted AI workflows that generate chapters, summaries, key moments, caption tracks and thumbnail picks for a video that already exists as a Mux asset, and the output is metadata attached to that asset. Four things follow from that if you publish short-form:

  • A Robots job needs a ready Mux asset with a playback ID.
  • Chapters, summaries and key moments render inside a Mux player.
  • TikTok, Reels and Shorts give you no player to render them in.
  • What survives an upload is whatever got baked into the pixels.

What Robots makes, and where that output lives

The Robots workflows cover video understanding: Mux's own workflow docs on GitHub list generate-premium-captions, edit-captions, translate-audio, find-best-thumbnails and generate-engagement-insights, along with detection of subtitles already burned into frames (muxinc/ai WORKFLOWS.md). Each one returns structured data about your video, or writes a track back onto the asset.

In November 2026 Mux went a step further and put that output in its React Native player behind a robots prop, with chapter data written back to the asset so opening a panel doesn't fire a new generation. Timeline markers for chapters, highlighted ranges for key moments, a summary panel. That's good engineering, and it's the right move for anyone shipping a video app of their own.

It also tells you exactly where the value sits. The player's source options take playbackId as a required string, read off an asset's Playback and Thumbnails tab (Mux React Native Player). Hosting isn't a side detail in that sentence. Hosting is the thing that makes any of it work.

Why none of it reaches a viewer on TikTok

A repurposing pipeline ends in someone else's app, and that app renders its own UI around your file. There is no robots prop in the TikTok feed, no chapter rail in Reels, no summary panel in Shorts. The only channel you control is the frames and the audio inside the file you upload.

Captions are the clearest case. TikTok generates its own captions from speech recognition after upload and documents no way to attach a sidecar SRT or VTT at upload time (TikTok accessibility help). So a premium caption track sitting on a hosted asset is invisible the moment the MP4 leaves. If you want your wording, your font, your line breaks and your placement, they have to be pixels.

Chapters have exactly one off-platform home, and it's YouTube's description field: the first timestamp must be 00:00, you need at least three in ascending order, and each chapter has to run 10 seconds or longer (YouTube Help). That's a text field you can fill from any source, including your own notes. It does not need a hosted asset at all.

Captions earn their place on the frame for a plain reason. In a 2019 Verizon Media and Publicis Media survey of 5,616 US adults reported by Forbes, 69% said they watch video with the sound off in public and 80% said they're more likely to finish a video when captions are available; the same study put the lift in ad view time from subtitles at roughly 12% (Streaming Media).

The four jobs that survive an upload

For a faceless channel or a repurposing queue, the whole AI feature set collapses into four file-level jobs: burned captions, a hook on frame one, a 9:16 reframe, and a still for the cover. Here they are as FFmpeg, then as one API call.

Burn an SRT into the frames, keeping the audio untouched:

ffmpeg -i clip.mp4 \
  -vf "subtitles=clip.srt:force_style='FontName=Inter,FontSize=18,Outline=2'" \
  -c:a copy out.mp4

Put the hook on screen for the first two and a half seconds:

ffmpeg -i clip.mp4 -vf "drawtext=fontfile=/fonts/Inter-Bold.ttf:\
text='I tested five video APIs':fontsize=64:fontcolor=white:\
box=1:boxcolor=black@0.6:boxborderw=20:\
x=(w-text_w)/2:y=h*0.12:enable='lt(t,2.5)'" -c:a copy out.mp4

Reframe a 1920x1080 source to a center-cropped 1080x1920, which clears TikTok's recommended 9:16 and its 720x1280 minimum (TikTok video ad specs, which also caps uploads at 500 MB):

ffmpeg -i clip.mp4 -vf "scale=-2:1920,crop=1080:1920" -c:a copy out.mp4

Pull a cover frame at three seconds:

ffmpeg -ss 3 -i clip.mp4 -frames:v 1 -q:v 2 cover.jpg

Each of those is one POST to FFmpeg Micro with no FFmpeg on your machine, and the job hands back a file you download:

curl -X POST https://api.ffmpeg-micro.com/v1/transcodes \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": [{"url": "https://storage.example.com/input.mp4"}],
    "outputFormat": "mp4"
  }'

Submit, poll for status, download. The text overlay runs as an @text-overlay option on the same endpoint, which is how the bulk shorts script fires 100 of these in parallel from a CSV. If you'd rather not write the call, Auto Captions and Resize for Format are the one-click versions of two of these four jobs.

Worth knowing before you plan the caption step: you can't extract a caption track from an MP4 that never had one, so the real job is transcription first, burn second. That split is covered in the real job is transcription, and the fuller list of what a file will and won't give up is in what you can extract from a video file.

What 80 shorts a month costs on each kind of meter

The two billing models answer different questions, so compare them on the same workload: 20 vertical shorts a week, 30 seconds each, three processing passes per short (reframe plus burned captions, hook text, cover frame).

MeterWhat it charges forThis workload
Per input secondSeconds of video you submitted90 input seconds per short, 9 tokens, 720 tokens a month
Per job plus per minuteEach AI workflow run, by durationPriced per workflow, separate from the encode
Storage per assetEvery asset you keep, monthlyA second copy of a file TikTok already stores
Delivery per minuteMinutes streamed to viewersZero, because TikTok's CDN serves the views

On FFmpeg Micro's meter, one token buys 10 seconds of input video and it's metered per second with no rounding up to the minute. Those 720 tokens sit inside the 2,500 that the $19.99 Starter plan grants each month. The free tier runs 200 tokens, roughly 33 minutes of input, which is about 22 finished shorts at three passes each, and it needs no card. Checking status, listing jobs and downloading outputs cost nothing, and a failed job is never billed.

The structural difference matters more than any single rate. Billing is on the duration of the video you processed, not on how long our servers took and not on how many people watched. Trim a 30 second clip out of a one hour recording and the bill is 30 seconds. Storage and delivery meters, by contrast, keep running after the work is finished, which is the point of which meters keep billing after the job ends.

What an API with no player genuinely doesn't do

FFmpeg Micro has no player, no hosting and no editor UI, and those absences are real limits rather than modesty. There's no playback ID, no signed URL, no analytics on watch time, no adaptive stream served to your users unless you ask for HLS output and host it yourself. There's no timeline you drag clips around on.

If your product is a video app, those are the exact things you need, and a hosting-plus-player platform with AI workflows bolted onto the asset is the correct purchase. Robots inside a Mux player is a genuinely better experience than anything a processing API can hand you, because the information stays live and queryable next to the video.

The split is simple. When the viewer is inside your player, buy the player. When the viewer is inside TikTok, you're buying hosting and delivery for a copy nobody will ever watch, which is the same trap described in most alternatives are hosts, not processors.

FAQ about Mux Robots and short-form pipelines

Can I run Mux Robots on a video that isn't hosted at Mux?

A Robots job runs against a ready Mux asset in the same environment as your token, with a public or signed playback ID, and you pass the asset ID. A file sitting in S3, Google Drive or an n8n workflow has to be ingested as a Mux asset first, which means you're on the hosting meter before the AI job starts.

Does TikTok accept an SRT file when I upload a video?

TikTok documents no way to attach a sidecar SRT or VTT at upload; it runs its own speech recognition after the fact. Captions you control have to be rendered into the frames before the file leaves your pipeline.

Where do generated chapters show up if I post to YouTube instead?

YouTube reads chapters from timestamps in the video description, so generated chapters are useful there as text you paste, with the first at 00:00, at least three of them, and a 10 second minimum per chapter. YouTube does not read a WebVTT chapters track attached to an asset hosted somewhere else.

How do I burn captions into pixels instead of attaching a track?

Burning captions means re-encoding the video with the subtitles filter, which draws each cue onto the frames and makes them part of the picture. As an API job, you send the video and the caption file in one request and download an MP4 with the captions already in it.

Can I run these four jobs from n8n or Make?

All four run as ordinary HTTP jobs, so n8n, Make and Zapier drive them with the official node, the Make app, or a plain HTTP request. The pattern is the same in every tool: upload, submit, poll, download, then hand the file to your publishing step.

Burned captions, a hook on frame one, a 9:16 crop and a cover frame are the whole AI feature set once the viewer is inside someone else's app, and the pipelines that run them on a schedule are collected on the content repurposing page. Sign up free and you get a key with a free tier attached, nothing to install and no encoder to keep alive, callable from your own code, from n8n, Make and Zapier, or from an AI agent.

About Javid Jamae

Founder & CEO at FFmpeg Micro

Javid is a software engineer, author, and entrepreneur with over 25 years of professional software development experience across enterprise, startup, and consulting environments. He founded FFmpeg Micro to make video processing accessible to developers through a simple, automation-first REST API.

Software EngineeringVideo ProcessingFFmpegCloud ArchitectureAPI DesignAutomation

Skip the command line

The Auto Captions blueprint transcribes your video and burns the captions in. You just review the transcript.

Run it (free)