Mastering Video Subtitles with FFmpeg: Soft Subs vs. Hard Subs – Everything You Need to Know!

When we're using video translation software like pyVideoTrans, the final step of synthesizing the video often presents a choice: "embed soft subtitles" or "embed hard subtitles." What's the difference between these two methods? What are their respective pros and cons? And how can we use the powerful FFmpeg tool to achieve them?

Don't worry, these notes will explain everything in the simplest, most understandable way!

Two Ways to Embed Subtitles: Soft vs. Hard

Imagine subtitles as a "text overlay" for your video. This overlay can be applied in two ways:

1. Soft Subtitles: Think of these like a jacket you can put on or take off at any time. The subtitle data and the video picture are stored separately but are packaged together within the same video file (like an MP4 or MKV file). When viewers play the video, they can freely choose whether to display the subtitles. If the video has subtitles in multiple languages, they can even switch between them.
2. Hard Subtitles (Burned-in Subtitles): This is like printing a text design directly onto a T-shirt, making it part of the fabric. The subtitle's pixel information is directly "drawn" onto each frame of the video, becoming a part of the video image itself. Once created, these subtitles are always visible and cannot be turned off or changed.

FFmpeg Command in Action: Let's Get Hands-On!

FFmpeg is an incredibly powerful open-source multimedia processing tool, and we can use it to add both types of subtitles to our videos. Let's look at two specific command examples. Assume we have a video file 1.mp4 and a subtitle file (in SRT or ASS format).

Scenario 1: Embedding Soft Subtitles (Viewer-controllable, SRT字幕 recommended)

If you want viewers to be able to control the subtitle display themselves, or if you're providing subtitles in multiple languages for them to choose from, then soft subtitles are your go-to option.

FFmpeg Command:

ffmpeg -i 1.mp4 -i 1.srt -c:v copy -c:s mov_text -metadata:s:s:0 language=chn -movflags +faststart out_soft_subs.mp4

Command Breakdown, Line by Line:

• ffmpeg: The boss is here! This tells the computer we're going to use FFmpeg.
• -i 1.mp4: The first input file, which is our original video.
• -i 1.srt: The second input file, which is our SRT format subtitle file. (SRT is a common plain-text subtitle format).
• -c:v copy: -c:v refers to the video codec (codec for video). copy means "direct copy," so the video stream is not re-encoded. The advantage is that it's super fast, and there's no loss in video quality!
• -c:s mov_text: -c:s refers to the subtitle codec (codec for subtitles). mov_text is a text-based subtitle format supported by MP4 files. We convert the SRT subtitles into this format and embed them into the MP4.
• -metadata:s:s:0 language=chn: This line gives the subtitles an "ID card."
  • -metadata: Sets metadata.
  • :s:s:0: s stands for stream. The first s indicates a subtitle stream, and 0 means the first subtitle stream (if you embed multiple, they'll be 1, 2, etc.).
  • language=chn: Tells the player that this subtitle track is in Chinese ("chn" is the international standard code for Chinese). This way, the player can correctly display "Chinese" in the subtitle selection menu.
• -movflags +faststart: This is a small optimization for MP4 files. It moves some important "index information" of the video to the beginning of the file. This allows the video to start playing faster when streamed online (e.g., watching a video on a webpage), improving the user experience.
• out_soft_subs.mp4: The output filename. Our processed video with soft subtitles will be named this.

What does this command do?

It takes the video stream from 1.mp4 as is, then converts the 1.srt subtitle file into mov_text format and embeds it into the final out_soft_subs.mp4 file as a separate subtitle track.

Summary of Soft Subtitle Characteristics:

• Pros:
  • Viewers can freely choose to show/hide subtitles or switch between different language tracks.
  • Fast processing time and lossless video quality (because the video is copy'd).
  • Convenient file management, as video and subtitles are bundled in one file.
• Cons:
  • Compatibility Issues: While most modern players (like VLC, PotPlayer, mobile players) support them well, they might not display on some older devices or in certain playback environments (especially when a web browser plays a video directly using the <video> tag without JavaScript libraries), or they might require manual setup by the user.
  • Limited Styling: SRT subtitles themselves have basic styling. The final display appearance depends more on the player's rendering capabilities.

Scenario 2: Embedding Hard Subtitles (Subtitles "welded" onto the picture, ASS recommended to preserve styling)

If you want the subtitles to be visible no matter what, or if your subtitle file (like an ASS format file) contains a lot of cool styling (fonts, colors, positioning, animations, etc.) that you don't want to lose, then hard subtitles are your choice.

FFmpeg Command:

ffmpeg -i 1.mp4 -c:v libx264 -vf subtitles=1.ass -movflags +faststart out_hard_subs.mp4

Command Breakdown, In Detail:

• ffmpeg -i 1.mp4: Same as above, specifies the input video.
• -c:v libx264: -c:v is still the video codec. But this time, we're using libx264. This is a very popular, high-quality H.264 video encoder. Note: It's no longer copy, which means the video will be re-encoded!
• -vf subtitles=1.ass: This is the core part!
  • -vf: Indicates we're going to use a video filter.
  • subtitles=1.ass: subtitles is the name of the filter we're using. It will read the 1.ass subtitle file (ASS format supports rich styling) and "draw" the subtitle content onto each frame of the video.
• -movflags +faststart: Same as before, for web playback optimization.
• out_hard_subs.mp4: The output filename for the video with hard subtitles.

What does this command do?

It first reads the 1.mp4 video, then invokes the subtitles filter to render and overlay the content (including all styling) from the 1.ass subtitle file onto the video frames. Finally, it uses the libx264 encoder to re-encode this new video stream (with the subtitles now part of the image) and saves it as out_hard_subs.mp4.

Summary of Hard Subtitle Characteristics:

• Pros:
  • Excellent Compatibility: As long as the player can play the video, the subtitles will definitely be visible. No need to worry about whether the player supports subtitle features. Especially suitable for direct playback on web pages or on devices with simple functionality.
  • Perfect Style Preservation: If you're using an advanced subtitle format like ASS, hard subtitles can perfectly preserve all fonts, colors, positions, dynamic effects, etc. What you see is what you get!
• Cons:
  • Subtitles Cannot Be Turned Off or Changed: The subtitles are now part of the picture; viewers can't hide them.
  • Video Needs Re-encoding: This means:
    • Slower Processing Time: Re-encoding takes much more time than a direct copy.
    • Potential Quality Loss: Re-encoding will always result in some quality loss, unless you set a very high bitrate (which, in turn, makes the file larger).
  • Difficult to Modify: Once burned in, changing the subtitle content or style is very troublesome, basically requiring you to re-render the video.

Soft Subs vs. Hard Subs: Which One Should I Choose?

Now that you understand their differences and pros/cons, the choice becomes clearer:

• Do you prioritize flexibility and lossless video quality, and is your target audience likely to use modern players? -> Choose soft subtitles.

  • Example: Creating tutorial videos where you want users to be able to turn English/Chinese subtitles on or off; sharing a movie with friends who can watch it with VLC or similar players.
  • pyvideotrans would operate similarly to the first command when the user chooses to embed "soft subtitles."

• Do you prioritize maximum compatibility, want to ensure subtitles are always visible, or are complex subtitle styles very important? -> Choose hard subtitles.

  • Example: Creating a promotional video that needs to be played on various devices (including old TVs, projectors); your video subtitles have special layouts and animation effects; or you simply want to ensure everyone sees the subtitles when they open the video, regardless of whether they know how to configure their player.
  • pyvideotrans would operate similarly to the second command when the user chooses to embed "hard subtitles."

A Small Addendum: About External Subtitles

There's actually another common form of subtitles called External Subtitles. This is when the subtitle file (e.g., movie.srt) and the video file (e.g., movie.mp4) are two separate files, usually kept in the same folder. The player will automatically load them or allow the user to load them manually.

• Pros: Super easy to edit subtitles (you can change an SRT file with a simple text editor), and compatibility is generally very good.
• Cons: It's easy to forget to include the subtitle file when sharing, or the filenames might not match, preventing them from loading. Managing multiple subtitle files can also be cumbersome.

The "embedded soft subtitles" we discussed today aim to solve the distribution inconvenience of external subtitles by packaging the subtitles within the video file itself.

Conclusion

Through these notes, I believe you now have a clearer understanding of the principles and methods for handling soft and hard subtitles with FFmpeg:

Feature	Embedded Soft Subtitles	Hard Subtitles (Burned-in)
Core Command	-c:v copy (video no re-encode), -c:s mov_text	-c:v libx264 (video re-encode), -vf subtitles
Viewer Control	Can be turned on/off, switchable	Cannot be turned off, always displayed
Video Quality	Lossless (due to copy)	Potentially lossy (due to re-encoding)
Processing Speed	Fast	Slow
Style Preservation	Depends on player, basic styling	Perfect preservation (especially with ASS)
Compatibility	Good, but some environments may not support	Excellent, subtitles visible if video plays

In tools like pyvideotrans, understanding the difference between these two types of subtitles can help you make the best choice based on your needs, ensuring your video work is presented to your audience in the most suitable way.