Batch transcode a folder of videos with one workflow

You have 340 MP4s in a folder and a deadline. The shell loop you wrote works fine on the first twelve and then your laptop fan sounds like a leaf blower, one file with a broken moov atom kills the script at #47, and you have no idea which ones finished.

Quick answer: To batch transcode a folder of videos with one workflow, don't loop FFmpeg sequentially on one machine. List the folder, submit one API job per file to a cloud FFmpeg endpoint, and let the queue run them in parallel while a webhook records each completion. With FFmpeg Micro that's one POST per file with no servers to run and no FFmpeg to install, and a failed file retries on its own instead of stopping the batch.

Conventional wisdom: batch video processing is a throughput problem, so you need a bigger box or more cores. Most setups behave exactly that way. But the thing that actually breaks a batch of 340 files isn't encode speed, it's state. A for loop has no memory. It doesn't know what finished, it can't resume, and one malformed input takes the whole run with it.

Once you treat each file as an independent job with an ID and a status, batch transcoding stops being a hardware question.

The shell loop, and exactly where it falls over

The classic version:

for f in ./input/*.mp4; do
  ffmpeg -i "$f" -c:v libx264 -crf 23 -preset medium \
    -c:a aac -b:a 128k "./output/$(basename "$f")"
done

This is correct code. It's also serial: 340 files at roughly 40 seconds of encode each is about 3.8 hours of wall clock, during which your machine is unusable and the terminal must stay open. Add -preset slow for better compression and you're past 6 hours.

Three failure modes show up in real folders, not synthetic ones:

GNU parallel fixes the first problem only: ls ./input/*.mp4 | parallel -j 4 ffmpeg -i {} ... gets you 4x on a 8-core laptop, then thermal throttling eats the gains. Still no state, still no resume.

Fan out one job per file instead

The mechanism shift: instead of one long-running process that iterates, you make N short-lived HTTP calls and let a managed queue do the concurrency. Each file gets a job ID. The job ID is your state.

for f in ./input/*.mp4; do
  curl -s -X POST https://api.ffmpeg-micro.com/jobs \
    -H "Authorization: Bearer $FFMPEG_MICRO_API_KEY" \
    -H "Content-Type: application/json" \
    -d "{\"input_url\":\"https://cdn.example.com/in/$(basename "$f")\",
         \"operation\":\"transcode\",
         \"webhook_url\":\"https://hooks.example.com/video-done\"}" \
    | tee -a job-ids.jsonl
done

Submitting 340 jobs takes about as long as 340 HTTP round trips. The encoding happens after, in parallel, on infrastructure that isn't your laptop. Your loop's only job is submission, so a bad file returns a 4xx for that one file and the loop keeps going.

job-ids.jsonl is the part people skip and then regret. Write every response to a file before you do anything else. That's your resume point, your audit log, and your answer to "did clip 212 ever finish."

Check the exact request and response shapes in the FFmpeg Micro docs before you script against them, and try one file in the playground first so you know the output looks right at scale.

Comparison: four ways to run a batch

The Kubernetes row is where most teams quietly lose a week. Getting FFmpeg into a container is the easy half; we wrote about the hard half in FFmpeg in Docker: the Dockerfile isn't the hard part. If you're weighing the self-hosted math seriously, FFmpeg in the cloud vs running FFmpeg yourself walks the total cost.

Batch video processing in n8n, Make, or Zapier

You don't need code for this. The pattern is identical in all three: a list step, a loop step, an HTTP step, and a wait-for-webhook step.

In n8n:

Make and Zapier map the same way: an Iterator or a Looping step in front of an HTTP module. The one thing to get right is not making n8n wait synchronously for each encode. A 6-minute encode inside a workflow execution will hit the default execution timeout and take the whole batch down with it. Use the async submit-then-webhook shape; the recipe is in how to fix n8n timeout errors when processing video.

For tracking, a spreadsheet genuinely works as the queue table. One row per file, a status column, an output URL column. We laid that out in Google Sheets is a fine video processing queue with n8n.

Python: submit, record, retry the stragglers

Node, PHP, and Python all look the same here because the API is plain REST. The idempotency trick is what matters: key each job by the input filename so a re-run doesn't double-encode.

import os, json, requests

API = "https://api.ffmpeg-micro.com/jobs"
KEY = os.environ["FFMPEG_MICRO_API_KEY"]
done = {json.loads(l)["input"] for l in open("submitted.jsonl")} if os.path.exists("submitted.jsonl") else set()

with open("submitted.jsonl", "a") as log:
    for name in sorted(os.listdir("input")):
        if name in done:
            continue
        r = requests.post(API,
            headers={"Authorization": f"Bearer {KEY}"},
            json={"input_url": f"https://cdn.example.com/in/{name}",
                  "operation": "transcode"})
        log.write(json.dumps({"input": name, "status": r.status_code,
                              "body": r.json()}) + "\n")
        log.flush()

Run it twice and the second run submits nothing. Kill it at file 190 and the next run starts at 190. That's the whole feature, and it's four lines of it.

Common pitfalls

When not to use a batch API

Be honest about the cases where this is the wrong tool:

FAQ

How do I transcode multiple videos at once with an API?

Submit one job per video in a loop, then wait on webhooks rather than blocking on each response. The API's queue runs them concurrently, so 50 files finish in close to the time of the slowest single file, not the sum of all fifty.

What does batch video processing cost?

FFmpeg Micro is usage-based after a free tier, so a batch costs the sum of its individual jobs with no idle server charges between runs. Current rates are on the pricing page, and /pricing/compare puts it next to the alternatives. The cost that's easy to miss with self-hosting is the instance you keep warm between batches.

Can I batch process a folder of videos without installing FFmpeg?

Yes. The API runs a managed FFmpeg toolkit, so your code only makes HTTP requests. That's the point for serverless environments where you can't ship a 70 MB binary, like Supabase Edge Functions or Vercel with Next.js.

How do I know when a batch is done?

Count terminal statuses against your submitted job log. Give each job a webhook_url and increment a counter or flip a row's status on each callback; the batch is done when the completed plus failed count equals the submitted count. Polling works too, but webhooks mean you're not making 340 status requests a minute.

Can an AI agent kick off a batch transcode?

Yes, through the MCP server. Claude or another agent can call the same job endpoints as a tool, which is useful when the file list itself comes from an agent step rather than a folder listing.

Grab an API key and run one file through it before you point it at all 340. Sign up free: https://www.ffmpeg-micro.com/auth/signup