[FFmpeg-user] Hardcoding subtitles

[Paul B Mahol]

On 7/3/13, Oliver Fromme <oliver at fromme.com> wrote:
> I'm trying to "hardcode" forced subtitles into a video, so they are
> always displayed, independent of the player's subtitle support.
> In the end I have been successful, but it was kind of an adventure.
> I'm trying to describe the problems that I encountered, maybe this
> can help to make ffmpeg better.
>
> My ffmpeg version is a recent git snapshot, just a few days old.
>
> These are my input files:  Video and audio are in an MPEG2 PS file
> that comes from a DVD (via "mplayer -dumpstream").  The subtitles
> are in VOBSUB format (I think ffmpeg calls this "dvdsub"), i.e.
> .idx + .sub files.  These have been retrieved from the same DVD
> with "mencoder -vobsubout".
>
> The first problem was that ffmpeg didn't want to read the VOBSUB
> files directly, no matter what I tried.  It seemed to be unable
> to detect the format automatically, and I was unable to find out
> how to specify it on the command line.  I probably tried more
> than 20 variantions, without success.  However, it's not a big
> deal:  I just wrapped the VOBSUB stream into an MKV container

Yes, it is big deal, if you want bug fixed, provide sample and open bug report.

You don't need to upload full file. Just enough so bug can be reproduced.

> (using mkvmerge), and ffmpeg happily accepted it.  So, my input
> files are now the MPEG2 PS file (same as before) and an MKV file
> containing just the VOBSUB subtitle stream.
>
> According to the excellent manual page, the overlay filter can
> be used for hardcoding subtitles, so I tried that.  The filter
> specification was simple:
>
> -filter_complex "[0:v][1:s]overlay"
>
> where file 0 is the MPEG2 PS and file 1 is the MKV file with the
> VOBSUB stream.  If it matters (I don't think it does), here's
> the complete command line:
>
> ffmpeg1 -loglevel warning -stats -i dvdstream-3.mpg -i subtitles-3-1.mkv \
>         -filter_complex "[0:v][1:s]overlay" \
>         -c:v libx264 -preset veryslow -tune film -crf 18 -level 4.1 \
>         -codec:a copy -map 0:a -sn dvdstream-3-vid.mkv
>
> Basically, that command worked and created a video with hardcoded
> subtitles.  However, there was one small problem:  The duration
> of the last subtitle was "infinite", i.e. the very last subtitle
> stayed on the screen until end of the video.  It was visible for
> several minutes including the whole end credits of the movie.
> Not nice.  So I went back to the manual page and re-read the
> section about the overlay filter.
>
> I noticed the "repeatlast" option of this filter, but the last
> sentence said something like "A value of 0 ... is enabled by
> default", so I thought that 0 was the default.  But I didn't
> look closely enough ...  Upon further inspection, it's just the
> other way around.  Of course that's my mistake, but maybe the
> manual page could be made a little clearer?  Something like
> "The default value is 1" would probably be clear enough.
>
> Anyway, I tried to disable that flag:
>
> -filter_complex "[0:v][1:s]overlay=repeatlast=0"
>
> It did make a difference, but not what I expected:  Now the last
> subtitle didn't appear at all!  I think this is a bug, although
> I wouldn't rule out the possibility that I did something wrong
> again.  I tried with a different DVD, but same result:  When
> repeatlast=0 is specified, the last VOBSUB subtitle is omitted
> completely.
>
> So, now I pondered over a workaround for that problem.
> I skimmed over some examples in the manual page that gave me an
> Idea:  I could try to use the gte() function to forcefully end
> the last subtitle at the desired time stamp.
>
> First I used mplayer to look for the exact position where the
> last subtitle was supposed to end.  In this case that was at
> t = 3375 (that's 56 minutes and 15 seconds).  I reverted the
> "repeatlast" flag to the default so the last subtitle was
> visible, and used gte() to cancel it at the proper time:
>
> -filter_complex "[0:v][1:s]overlay=x='if(gt(t,3375),NAN,x)'"
>
> The idee was:  If the time is beyond 3375, set the x value of
> the subtitle frames to NAN (effectively disabling them),
> otherwise keep them at their original x value.  At least I
> thought that's what the x variable meant.
>
> Obviously I was wrong, because the resulting video did not
> contain *any* subtitles at all.  No subtitles anywhere.
> Either I misunderstood the description of the x variable in
> the manual page (and in this case the manual page should be
> clarified), or there's another bug.  In my next attempt I
> replaced the x variable with 0, which should be the default
> for x anyway:
>
> -filter_complex "[0:v][1:s]overlay=x='if(gt(t,3375),NAN,0)'"
>
> Finally, that produced the expected result, and I'm happy now.
>
> Having to find out the ending time stamp for the last subtitle
> is a little cumbersome and can't be done automatically (I don't
> know a CLI tool that's able to display durations or ending
> time stamps from VOBSUB streams).  Therefore I would appreciate
> if the bug with repeatlast=0 could be fixed, so my workaround
> won't be necessary anymore.
>
> Unfortunately I can't easily upload sample files, because the
> full movie is too large, and I don't have appropriate software
> to cut a short clip from it that also retains the subtitles
> while keeping them in sync with the movie.