[FFmpeg-user] Hardcoding subtitles

[Oliver Fromme]

I'm trying to "hardcode" forced subtitles into a video, so they are
always displayed, independent of the player's subtitle support.
In the end I have been successful, but it was kind of an adventure.
I'm trying to describe the problems that I encountered, maybe this
can help to make ffmpeg better.

My ffmpeg version is a recent git snapshot, just a few days old.

These are my input files:  Video and audio are in an MPEG2 PS file
that comes from a DVD (via "mplayer -dumpstream").  The subtitles
are in VOBSUB format (I think ffmpeg calls this "dvdsub"), i.e.
.idx + .sub files.  These have been retrieved from the same DVD
with "mencoder -vobsubout".

The first problem was that ffmpeg didn't want to read the VOBSUB
files directly, no matter what I tried.  It seemed to be unable
to detect the format automatically, and I was unable to find out
how to specify it on the command line.  I probably tried more
than 20 variantions, without success.  However, it's not a big
deal:  I just wrapped the VOBSUB stream into an MKV container
(using mkvmerge), and ffmpeg happily accepted it.  So, my input
files are now the MPEG2 PS file (same as before) and an MKV file
containing just the VOBSUB subtitle stream.

According to the excellent manual page, the overlay filter can
be used for hardcoding subtitles, so I tried that.  The filter
specification was simple:

-filter_complex "[0:v][1:s]overlay"

where file 0 is the MPEG2 PS and file 1 is the MKV file with the
VOBSUB stream.  If it matters (I don't think it does), here's
the complete command line:

ffmpeg1 -loglevel warning -stats -i dvdstream-3.mpg -i subtitles-3-1.mkv \
        -filter_complex "[0:v][1:s]overlay" \
        -c:v libx264 -preset veryslow -tune film -crf 18 -level 4.1 \
        -codec:a copy -map 0:a -sn dvdstream-3-vid.mkv

Basically, that command worked and created a video with hardcoded
subtitles.  However, there was one small problem:  The duration
of the last subtitle was "infinite", i.e. the very last subtitle
stayed on the screen until end of the video.  It was visible for
several minutes including the whole end credits of the movie.
Not nice.  So I went back to the manual page and re-read the
section about the overlay filter.

I noticed the "repeatlast" option of this filter, but the last
sentence said something like "A value of 0 ... is enabled by
default", so I thought that 0 was the default.  But I didn't
look closely enough ...  Upon further inspection, it's just the
other way around.  Of course that's my mistake, but maybe the
manual page could be made a little clearer?  Something like
"The default value is 1" would probably be clear enough.

Anyway, I tried to disable that flag:

-filter_complex "[0:v][1:s]overlay=repeatlast=0"

It did make a difference, but not what I expected:  Now the last
subtitle didn't appear at all!  I think this is a bug, although
I wouldn't rule out the possibility that I did something wrong
again.  I tried with a different DVD, but same result:  When
repeatlast=0 is specified, the last VOBSUB subtitle is omitted
completely.

So, now I pondered over a workaround for that problem.
I skimmed over some examples in the manual page that gave me an
Idea:  I could try to use the gte() function to forcefully end
the last subtitle at the desired time stamp.

First I used mplayer to look for the exact position where the
last subtitle was supposed to end.  In this case that was at
t = 3375 (that's 56 minutes and 15 seconds).  I reverted the
"repeatlast" flag to the default so the last subtitle was
visible, and used gte() to cancel it at the proper time:

-filter_complex "[0:v][1:s]overlay=x='if(gt(t,3375),NAN,x)'"

The idee was:  If the time is beyond 3375, set the x value of
the subtitle frames to NAN (effectively disabling them),
otherwise keep them at their original x value.  At least I
thought that's what the x variable meant.

Obviously I was wrong, because the resulting video did not
contain *any* subtitles at all.  No subtitles anywhere.
Either I misunderstood the description of the x variable in
the manual page (and in this case the manual page should be
clarified), or there's another bug.  In my next attempt I
replaced the x variable with 0, which should be the default
for x anyway:

-filter_complex "[0:v][1:s]overlay=x='if(gt(t,3375),NAN,0)'"

Finally, that produced the expected result, and I'm happy now.

Having to find out the ending time stamp for the last subtitle
is a little cumbersome and can't be done automatically (I don't
know a CLI tool that's able to display durations or ending
time stamps from VOBSUB streams).  Therefore I would appreciate
if the bug with repeatlast=0 could be fixed, so my workaround
won't be necessary anymore.

Unfortunately I can't easily upload sample files, because the
full movie is too large, and I don't have appropriate software
to cut a short clip from it that also retains the subtitles
while keeping them in sync with the movie.

Best regards
   Oliver

--