FFmpeg Batch Processing: How to Process Hundreds of Videos
You have 500 videos that need transcoding. Maybe you are resizing them for different platforms. Maybe you are adding watermarks before distributing to clients. Maybe you just need to convert everything from MOV to MP4 because someone on your team insists on recording with QuickTime.
Whatever the reason, you are not going to sit there and run ffmpeg 500 times by hand.
This guide covers three approaches to batch processing with FFmpeg, from basic bash loops to parallel execution to API-based pipelines. Pick the one that matches your scale.
The Bash Loop (Simple, Slow)
The most common approach. Loop through a directory and run ffmpeg on each file:
for file in input/*.mp4; do
ffmpeg -i "$file" -vf scale=1280:720 -c:v libx264 -crf 23 \
"output/$(basename "$file")"
done
This works. It processes one video at a time, sequentially. For 10 files, fine. For 500 files on a machine with 16 cores, you are using 6% of your available compute.
The other problem: error handling. If file 247 fails because it has a corrupt header, the loop either stops (if you used set -e) or silently skips it and you find out three hours later when the output directory has 499 files instead of 500.
GNU Parallel (Fast, Fragile)
GNU parallel lets you run multiple ffmpeg processes simultaneously:
find input/ -name "*.mp4" | parallel -j 4 \
ffmpeg -i {} -vf scale=1280:720 -c:v libx264 -crf 23 \
output/{/}
The -j 4 flag runs 4 jobs concurrently. On a machine with 8 cores, this is a reasonable default. Push it higher and you risk running out of memory, because each ffmpeg process loads the entire input file into memory for decoding.
Parallel processing introduces new failure modes:
- Memory pressure. Four concurrent transcodes of 4K video can easily consume 16GB+ of RAM.
- Disk I/O bottleneck. Reading and writing 4 video streams simultaneously saturates most SSDs.
- Partial failures. If 2 of 4 parallel jobs fail, your output directory has a mix of completed and incomplete files with no clear indication of which is which.
You can add logging, retry logic, and health checks. But at that point you are building a job queue, not running a bash script.
xargs for Quick Parallel Runs
If you do not have GNU parallel installed, xargs works for simple cases:
find input/ -name "*.mp4" -print0 | xargs -0 -P 4 -I {} \
ffmpeg -i {} -vf scale=1280:720 -c:v libx264 -crf 23 \
output/$(basename {})
Same parallel execution, fewer features. No progress tracking, no retry, no job logging. Fine for a one-off batch, not for a production pipeline.
The API Approach (Scale Without Infrastructure)
At some point, the bash script approach breaks down. Maybe you need to process thousands of videos. Maybe you need to run jobs from a web application. Maybe you are tired of debugging memory issues on your local machine.
A video processing API handles the infrastructure for you. You send a request, the API processes the video on its own servers, and you get the result back.
With FFmpeg Micro, a batch job looks like this:
const videos = ["video1.mp4", "video2.mp4", "video3.mp4"];
const jobs = await Promise.all(
videos.map(video =>
fetch("https://api.ffmpeg-micro.com/v1/jobs", {
method: "POST",
headers: {
"Authorization": "Bearer YOUR_API_KEY",
"Content-Type": "application/json"
},
body: JSON.stringify({
input: video,
output_format: "mp4",
options: { scale: "1280:720", codec: "libx264", crf: 23 }
})
})
)
);
No memory management. No disk I/O tuning. No infrastructure to maintain. The API auto-scales based on your workload.
When to Use Each Approach
| Approach | Best For | Limitations |
|---|---|---|
| Bash loop | Under 50 videos, simple transforms | Slow, sequential, no error recovery |
| GNU parallel | 50-500 videos, local processing | Memory limits, needs monitoring, fragile |
| API | Any volume, production pipelines | Requires network, per-job cost |
The decision usually comes down to frequency. If you batch process videos once a quarter, a bash script is fine. If you do it daily or it is part of a product workflow, the infrastructure overhead of managing ffmpeg locally costs more than an API.
Cost Comparison
Processing 1,000 one-minute videos (1080p to 720p transcode):
| Method | Time | Cost |
|---|---|---|
| Bash loop (single core) | ~8 hours | $0 (your machine) |
| GNU parallel (4 cores) | ~2 hours | $0 (your machine) |
| Cloud VM (c5.2xlarge) | ~1 hour | ~$3.40 (on-demand) |
| FFmpeg Micro API | ~30 min | ~$25 (pay per job) |
The API costs more per run. But factor in the engineering time to set up parallel processing, handle failures, manage a cloud VM, and maintain the pipeline over months. At $150/hour engineering time, the API pays for itself after the first batch.
Getting Started
If you are processing under 50 videos occasionally, start with the bash loop. If you need parallel processing, try GNU parallel with conservative concurrency (-j 2 to start).
If you are building a product that processes user-uploaded videos, or you need to batch process at scale without managing infrastructure, try FFmpeg Micro. Sign up for free and run your first job in under a minute.
Ready to process videos at scale?
Start using FFmpeg Micro's simple API today. No infrastructure required.
Get Started Free