[FFmpeg-devel] [PATCH] DVB Subtitles Fix

JULIAN GARDNER joolzg at btinternet.com
Wed May 4 23:21:04 CEST 2011





----- Original Message -----
> From: Ivan Kalvachev <ikalvachev at gmail.com>
> To: FFmpeg development discussions and patches <ffmpeg-devel at ffmpeg.org>
> Cc: 
> Sent: Wednesday, 4 May 2011, 21:32
> Subject: Re: [FFmpeg-devel] [PATCH] DVB Subtitles Fix
> 
> On 5/4/11, Michael Niedermayer <michaelni at gmx.at> wrote:
>>  On Wed, May 04, 2011 at 07:18:12PM +0100, JULIAN GARDNER wrote:
>>> 
>>> 
>>> 
>>> 
>>>  ----- Original Message -----
>>>  > From: Michael Niedermayer <michaelni at gmx.at>
>>>  > To: FFmpeg development discussions and patches 
> <ffmpeg-devel at ffmpeg.org>
>>>  > Cc:
>>>  > Sent: Wednesday, 4 May 2011, 18:25
>>>  > Subject: Re: [FFmpeg-devel] [PATCH] DVB Subtitles Fix
>>>  >
>>>  > On Wed, May 04, 2011 at 04:11:33PM +0100, JULIAN GARDNER wrote:
>>>  >>
>>>  >>
>>>  >>
>>>  >>
>>>  >>  ----- Original Message -----
>>>  >>  > From: Michael Niedermayer <michaelni at gmx.at>
>>>  >>  > To: FFmpeg development discussions and patches
>>>  > <ffmpeg-devel at ffmpeg.org>
>>>  >>  > Cc:
>>>  >>  > Sent: Wednesday, 4 May 2011, 15:14
>>>  >>  > Subject: Re: [FFmpeg-devel] [PATCH] DVB Subtitles Fix
>>>  >>  >
>>>  >>  > On Tue, Apr 26, 2011 at 11:34:10PM +0100, JULIAN GARDNER 
> wrote:
>>>  >>  >>  Ok first patch to fix the decoding and re-encoding 
> of DVB
>>>  > Subtitles.
>>>  >>  >>
>>>  >>  >>  I went through and validated against the spec, my 
> own engine and
>>>  > fixed the
>>>  >>  > bits that were failing or wrong.
>>>  >>  >
>>>  >>  > please explain how to reproduce the bugs that your patch 
> fixes
>>>  >>  >
>>>  >>  > Also please explain more elaborately what and why each 
> change is
>>>  >> made.
>>>  >>  >
>>>  >>
>>>  >>  I use LIVE and RECORDED streams from various sources which i 
> have
>>>  >> collected
>>>  > over the past years, these are put through ffmpeg and then the 
> output
>>>  > stream is
>>>  > played out in VLC and a couple of my own receivers which are used 
> in
>>>  > professional cable networks.
>>>  >
>>>  > understood, but does this patch fix any problem with these ffmpeg
>>>  > generated streams?
>>>  > Iam asking because id like to reproduce the problem and the fix.
>>>  >
>>>  >
>>>  it fixes all the problems with the encoded streams, also it allows 
> FFMPEG
>>>  to process a file correctly without bailing out with an error.
>>> 
>>>  I can upload a couple of streams if you want for you to see the 
> problems.
>> 
>>  thanks, yes this would help me.
>> 
>>  if they are small you can upload to our issue tracker
>>  http://ffmpeg.org/trac/ffmpeg
>> 
>> 
>>> 
>>>  >>
>>>  >>  >
>>>  >>  >>
>>>  >>  >>  I have tested this on recorded streams and live 
> streams and found
>>>  > no
>>>  >>  > problems except for one which i will try and explain 
> about.
>>>  >>  >>
>>>  >>  >>  Here in the UK some of our channels use a live 
> speech convertor
>>>  > where you
>>>  >>  > see the words appear as they are spoken, now this works 
> fine but the
>>>  >>
>>>  > way the
>>>  >>  > decode/encode works a lot more space is taken by this 
> encoder for
>>>  >> the
>>>  > subsequent
>>>  >>  > encoded stream. In one test the incoming packets made up 
> around 3k
>>>  >> of
>>>  > data, but
>>>  >>  > the amount of data in the output stream was around 15k.
>>>  >>  >>
>>>  >>  >>  joolz
>>>  >>  >
>>>  >>  >>   ffmpeg.c               |   11 -
>>>  >>  >>   libavcodec/dvbsub.c    |  103 ++++++++---------
>>>  >>  >>   libavcodec/dvbsubdec.c |  296
>>>  >>  > +++++++++++++++++++++++++++++--------------------
>>>  >>  >>   libavcodec/utils.c     |    3
>>>  >>  >>   4 files changed, 233 insertions(+), 180 
> deletions(-)
>>>  >>  >>  d22a3a04cf764a0b25138652ad82fe484311405f  
> dvbsubs.diff
>>>  >>  >>  diff --git a/ffmpeg.c b/ffmpeg.c
>>>  >>  >>  index 2c69608..e88bcfc 100644
>>>  >>  >>  --- a/ffmpeg.c
>>>  >>  >>  +++ b/ffmpeg.c
>>>  >>  >>  @@ -1092,15 +1092,8 @@ static void
>>>  > do_subtitle_out(AVFormatContext *s,
>>>  >>  >>           subtitle_out = 
> av_malloc(subtitle_out_max_size);
>>>  >>  >>       }
>>>  >>  >>
>>>  >>  >>  -    /* Note: DVB subtitle need one packet to draw 
> them and one
>>>  > other
>>>  >>  >>  -       packet to clear them */
>>>  >>  >>  -    /* XXX: signal it in the codec context ? */
>>>  >>  >>  -    if (enc->codec_id == CODEC_ID_DVB_SUBTITLE)
>>>  >>  >>  -        nb = 2;
>>>  >>  >>  -    else
>>>  >>  >>  -        nb = 1;
>>>  >>  >>  -
>>>  >>  >>  -    for(i = 0; i < nb; i++) {
>>>  >>  >>  +    i = 0;
>>>  >>  >>  +    {
>>>  >>  >>           sub->pts = av_rescale_q(pts,
>>>  > ist->st->time_base,
>>>  >>  > AV_TIME_BASE_Q);
>>>  >>  >>           // start_display_time is required to be 0
>>>  >>  >>           sub->pts              +=
>>>  >>  > av_rescale_q(sub->start_display_time, (AVRational){1, 
> 1000},
>>>  > AV_TIME_BASE_Q);
>>>  >>  >
>>>  >>  > why is this changed?
>>>  >>  >
>>>  >>
>>>  >>  Because only one stream is needed to process DVB Subtitles, i 
> could
>>>  >> not
>>>  > work out why you needed this. The NOTE is wrong regarding needed 
> one to
>>>  > draw and
>>>  > one to remove
>>>  >>
>>>  >>  What you will get from the DVB Stream is a packet with NO 
> regions,
>>>  >> this is
>>>  > the one that removes the subtitles from the picture, as per the 
> spec,
>>>  > there is a
>>>  > timeout value, which in the ffmpeg engine is always set to 30 
> seconds,
>>>  > this
>>>  > would cause the receiving unit to switch off the display after 30
>>>  > seconds of no
>>>  > valid dvb stream.
>>>  >
>>>  > Lets assume something generates subtitles, lets say it generates 
> one
>>>  > displaying "hello world" at start time 0:10 and end time 
> 0:50
>>>  > If i understand you correctly this needs a packet muxed with time 
> 0:10
>>>  > and a empty clear packet muxed at time 0:50
>>>  >
>>>  > And this loop generates these 2 packets from 1 subtitle with start 
> and
>>>  > end times.
>>>  >
>>>  > You remove it and i dont see how this could still work afterwards
>>>  > maybe iam missing something ?
>>> 
>>>  The subtitles dont have an end time per se, what you have is a display
>>>  time and a timeout value, this is incase of a stream problem or 
> corruption
> 
>>>  for a missing packet.
>>> 
>>>  now what happens is in you example you would get a display packet with
>>>  regions/objects/cluts etc and a timeout of 60, then around 50 seconds 
> into
>>>  the stream you would get a packet which has no objects/regions/cluts. 
> This
>>>  packet would turn the subs off, now if this packet was corrupted then 
> the
>>>  timeout would take care of the removal.
>> 
>>  I understand that, but this is only so when the source is dvb subs too
>>  it can be something else.a different subtitle format for example.
> 
> This nb variable doesn't have anything with the numbers of streams.
> 
> It ensures that in case of DVB output it would write 2 packets.
> The first one with the subtitles and the second one with no
> regions/segments/cluts.
> It does call avcodec_encode_subtitles() twice, so it does get
> different packets (I hope).
> It is kind of running the whole function twice in a row.
> 
> It seems that the dvb encoder have variable "hide_state" that always
> inverts itself when called, so every second encoded packet would be
> "hide" packet. And It seems that removing this variable is part of
> your patch.
> 
> So I guess the question is. If the source (input) subtitles are of the
> type that have both valid start and end time, how do you output 2
> packets (one to show and one to hide) ?
> Well, I guess this is mostly problem for the time when we have
> ass/ssa->dvb rendering.

i think your all missing the point, there is no such thing as an off in dvb subtitles. what you have is a set of display/regions/cluts and objects that make up a subtitle frame. If there are not object/cluts/regions then in theory you have an off frame. it does not always work that you have a on followed by an off. If you look at the bbc_small.ts i uploaded you can see that it draws 2 regions, 1 is a full width region with a single object, the 2nd line is made up of multiple objects as the new text is added.

So in this case you get 6 packets before we get a new region whch will cause a full redraw, no where in this stream is an "off" packet, so the inverting of the hide_state is NOT CORRECT and is not the correct way of doing dvb subtitles.

Also you will have noticed that i made changes to how the resetting and clearing of the cluts/objects etc are performed only as to only reset when the main display mode changes.

ass/ssa -> dvb subtitles

Well i did post a message 2 weeks ago asking about is, no replys, as to if there was some way to do what you mentioned ass/ssa to dvb subtitles. In this case all that needs to be done is that when you get the off time from the input you send to the encoder a 0 region subtitle block, the dvb subtitles encoder will then output an "off" packet.

joolz


More information about the ffmpeg-devel mailing list