[FFmpeg-devel] [PATCH] Implemented arrange feature to convert from low timebase formats to high timebase

[Roman Arutyunyan]

2012/10/17 Nicolas George <nicolas.george at normalesup.org>

> Le sextidi 26 vendémiaire, an CCXXI, Roman Arutyunyan a écrit :
> > This patch rearranges frame timestamps in muxer.
> >
> > The problem: When remuxing from low-timebase-resolution formats (like
> RTMP with 1KHz time base)
> > to high-resolution formats (like MPEG-TS with 90KHz) frame timestamps
> are multiplied by
> > resolution factor (*90 in this case). That significantly increases
> round-off error. If you
> > have two audio frames next to one another that kind of remuxing may
> result in your second frame's
> > timestamp be too far from where it should be as a result of round-off
> error multiplication.
>
> Have you discussed that particular problem somewhere (users mailing-list,
> bug tracker)? It looks to me like a framerate detection failure, and I am
> not sure that tweaking the timestamps randomly is the correct method of
> fixing it.
>

That's my experience with my project (nginx-rtmp-module). It works for me.
I wish we could discuss the problem here.  Maybe there will be better
solutions.
100% correct solution is impossible here because precise timestamps are lost
after rounding.

Timestamps are not actually tweaked but in most cases are auto-generated
like with '-async -1'. That's not random tweaking. BTW '-async -1' also
works
pretty good until first gap in audio stream.


>
> > While I have not found a PC software sensitive to that kind of error
> Apple iPhone is very
> > much sensitive producing crackles at audio frame boundaries after such
> conversion (I convert
> > RTMP to HLS).
> >
> > The patch intoduces '-arrange[:stream_specifier] arrange_threshold'
> option, where arrange_threshold
> > is max timestamp divergence in microseconds. When the argument is
> nonzero and current frame dts
> > is within arrange_threshold distance from muxer stream dts then packet
> dts is reset making
> > current frame receive the currect auto-generated dts from muxer. If new
> packet dts differs
> > too much then it's not changed in any way (that usually means a gap in
> audio stream).
> >
> > The feature makes sence when applied to audio stream.
> >
> > ---
> >  ffmpeg.h             |    2 ++
> >  ffmpeg_opt.c         |    4 ++++
> >  libavcodec/avcodec.h |    7 +++++++
> >  libavformat/utils.c  |    8 ++++++++
>
> The flag is stored in the AVCodecContext, the work is done in libavformat
> but the setting is controlled by the command-line tool. On the whole, it
> does not look right.
>

I'm agree. However AVCodecContext is easily accessed from output stream
structure
that's why I used it. I'll try to fix that.


> If the problem is specific to the ffmpeg command-line tool, then, assuming
> this is the right way of fixing it, then the flag should be in the
> OutputStream structure and the work done by ffmpeg.c in write_frame().
>

OK. Thanks for the clue. I'll fix that too.


>
> >  4 files changed, 21 insertions(+), 0 deletions(-)
> >
> > diff --git a/ffmpeg.h b/ffmpeg.h
> > index 56f8dfc..cadbea4 100644
> > --- a/ffmpeg.h
> > +++ b/ffmpeg.h
> > @@ -167,6 +167,8 @@ typedef struct OptionsContext {
> >      int        nb_pass;
> >      SpecifierOpt *passlogfiles;
> >      int        nb_passlogfiles;
> > +    SpecifierOpt *arrange;
> > +    int        nb_arrange;
> >  } OptionsContext;
> >
> >  typedef struct InputFilter {
> > diff --git a/ffmpeg_opt.c b/ffmpeg_opt.c
> > index 2c5fbfb..a50a9cf 100644
> > --- a/ffmpeg_opt.c
> > +++ b/ffmpeg_opt.c
> > @@ -994,6 +994,8 @@ static OutputStream
> *new_output_stream(OptionsContext *o, AVFormatContext *oc, e
> >          st->codec->global_quality = FF_QP2LAMBDA * qscale;
> >      }
> >
> > +    MATCH_PER_STREAM_OPT(arrange, i64, st->codec->arrange_threshold,
> oc, st);
> > +
> >      if (oc->oformat->flags & AVFMT_GLOBALHEADER)
> >          st->codec->flags |= CODEC_FLAG_GLOBAL_HEADER;
> >
> > @@ -2343,6 +2345,8 @@ const OptionDef options[] = {
> >          "extract an attachment into a file", "filename" },
> >      { "debug_ts",       OPT_BOOL | OPT_EXPERT,                       {
> &debug_ts },
> >          "print timestamp debugging info" },
> > +    { "arrange",        HAS_ARG | OPT_INT64 | OPT_SPEC,              {
> .off = OFFSET(arrange) },
> > +        "arrange closely spaced frames" },
>
> Both the name of the option and its description seem way too vague. A more
> specific name and an entry in the documentation would be in order.
>

Agreed. I've spent a lot for time thinking of a better name. Maybe there's
somebody around
who can suggest one? However that might depend on the solution we will
finally come to.


>
> Regards,
>
> --
>   Nicolas George
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.12 (GNU/Linux)
>
> iEYEARECAAYFAlB+u80ACgkQsGPZlzblTJOsjACeKQt6avaoJnLBsBhnDzVmvp1U
> 8M8An0cmB1KhOVL1rCCSg72s5SdghJjs
> =yLcd
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>
>