[FFmpeg-user] Hardcoding subtitles

[Stefano Sabatini]

On date Wednesday 2013-07-03 16:44:30 +0200, Oliver Fromme wrote:
> I'm trying to "hardcode" forced subtitles into a video, so they are
> always displayed, independent of the player's subtitle support.
> In the end I have been successful, but it was kind of an adventure.
> I'm trying to describe the problems that I encountered, maybe this
> can help to make ffmpeg better.
> 
> My ffmpeg version is a recent git snapshot, just a few days old.
> 
> These are my input files:  Video and audio are in an MPEG2 PS file
> that comes from a DVD (via "mplayer -dumpstream").  The subtitles
> are in VOBSUB format (I think ffmpeg calls this "dvdsub"), i.e.
> .idx + .sub files.  These have been retrieved from the same DVD
> with "mencoder -vobsubout".
> 
> The first problem was that ffmpeg didn't want to read the VOBSUB
> files directly, no matter what I tried.  It seemed to be unable
> to detect the format automatically, and I was unable to find out
> how to specify it on the command line.  I probably tried more
> than 20 variantions, without success.  However, it's not a big
> deal:  I just wrapped the VOBSUB stream into an MKV container
> (using mkvmerge), and ffmpeg happily accepted it.  So, my input
> files are now the MPEG2 PS file (same as before) and an MKV file
> containing just the VOBSUB subtitle stream.
> 
> According to the excellent manual page, the overlay filter can
> be used for hardcoding subtitles, so I tried that.  The filter
> specification was simple:
> 
> -filter_complex "[0:v][1:s]overlay"
> 
> where file 0 is the MPEG2 PS and file 1 is the MKV file with the
> VOBSUB stream.  If it matters (I don't think it does), here's
> the complete command line:
> 
> ffmpeg1 -loglevel warning -stats -i dvdstream-3.mpg -i subtitles-3-1.mkv \
>         -filter_complex "[0:v][1:s]overlay" \
>         -c:v libx264 -preset veryslow -tune film -crf 18 -level 4.1 \
>         -codec:a copy -map 0:a -sn dvdstream-3-vid.mkv
> 
> Basically, that command worked and created a video with hardcoded
> subtitles.  However, there was one small problem:  The duration
> of the last subtitle was "infinite", i.e. the very last subtitle
> stayed on the screen until end of the video.  It was visible for
> several minutes including the whole end credits of the movie.
> Not nice.  So I went back to the manual page and re-read the
> section about the overlay filter.
> 
> I noticed the "repeatlast" option of this filter, but the last
> sentence said something like "A value of 0 ... is enabled by
> default", so I thought that 0 was the default.  But I didn't
> look closely enough ...  Upon further inspection, it's just the
> other way around.  Of course that's my mistake, but maybe the
> manual page could be made a little clearer?  Something like
> "The default value is 1" would probably be clear enough.

Changed.

> Anyway, I tried to disable that flag:
> 
> -filter_complex "[0:v][1:s]overlay=repeatlast=0"
> 
> It did make a difference, but not what I expected:  Now the last
> subtitle didn't appear at all!  I think this is a bug, although
> I wouldn't rule out the possibility that I did something wrong
> again.  I tried with a different DVD, but same result:  When
> repeatlast=0 is specified, the last VOBSUB subtitle is omitted
> completely.

If this is a bug, we need a sample and a command to reproduce the
error. Please consider to file a bug report on trac.

> 
> So, now I pondered over a workaround for that problem.
> I skimmed over some examples in the manual page that gave me an
> Idea:  I could try to use the gte() function to forcefully end
> the last subtitle at the desired time stamp.
> 
> First I used mplayer to look for the exact position where the
> last subtitle was supposed to end.  In this case that was at
> t = 3375 (that's 56 minutes and 15 seconds).  I reverted the
> "repeatlast" flag to the default so the last subtitle was
> visible, and used gte() to cancel it at the proper time:
> 

> -filter_complex "[0:v][1:s]overlay=x='if(gt(t,3375),NAN,x)'"

The problem is that x is NAN when evaluating this expression, so the
result is always NAN. Or in other words, recursive definitions are not
supported (and probably meaningless).

> 
> The idee was:  If the time is beyond 3375, set the x value of
> the subtitle frames to NAN (effectively disabling them),
> otherwise keep them at their original x value.  At least I
> thought that's what the x variable meant.


> 
> Obviously I was wrong, because the resulting video did not
> contain *any* subtitles at all.  No subtitles anywhere.
> Either I misunderstood the description of the x variable in
> the manual page (and in this case the manual page should be
> clarified), or there's another bug.  In my next attempt I
> replaced the x variable with 0, which should be the default
> for x anyway:
> 
> -filter_complex "[0:v][1:s]overlay=x='if(gt(t,3375),NAN,0)'"
> 
> Finally, that produced the expected result, and I'm happy now.

Even better, you could use the enable option (check timeline section
in the manual) to avoid x/NAN hacks:
enable=gt(t,3375)

[...]