[FFmpeg-devel] [PATCH] flvdec: option for dropping negative CTS frames Initial frames with negative pts can produce video/audio desynchronization when a decoder is not prepared to handle negative pts. For example: QSV transcoding from RTMP Wowza server

[wm4]

On Wed,  5 Apr 2017 14:29:30 -0300
felipe at astroza.cl wrote:

> From: Felipe Astroza <felipe at astroza.cl>
> 
> Signed-off-by: Felipe Astroza <felipe at astroza.cl>
> ---
>  libavformat/flvdec.c | 14 +++++++++++---
>  1 file changed, 11 insertions(+), 3 deletions(-)
> 
> diff --git a/libavformat/flvdec.c b/libavformat/flvdec.c
> index 3959a36..1556fe0 100644
> --- a/libavformat/flvdec.c
> +++ b/libavformat/flvdec.c
> @@ -44,6 +44,7 @@
>  typedef struct FLVContext {
>      const AVClass *class; ///< Class for private options.
>      int trust_metadata;   ///< configure streams according onMetaData
> +    int drop_negative_cts;///< drop frames if cts is negative
>      int wrong_dts;        ///< wrong dts due to negative cts
>      uint8_t *new_extradata[FLV_STREAM_TYPE_NB];
>      int new_extradata_size[FLV_STREAM_TYPE_NB];
> @@ -1139,10 +1140,16 @@ retry_duration:
>              int32_t cts = (avio_rb24(s->pb) + 0xff800000) ^ 0xff800000;
>              pts = dts + cts;
>              if (cts < 0) { // dts might be wrong
> -                if (!flv->wrong_dts)
> +                if (flv->drop_negative_cts) {
>                      av_log(s, AV_LOG_WARNING,
> -                        "Negative cts, previous timestamps might be wrong.\n");
> -                flv->wrong_dts = 1;
> +                            "Negative cts, frames will be dropped.\n");
> +                    dts = pts = AV_NOPTS_VALUE;
> +                } else {
> +                    if (!flv->wrong_dts)
> +                        av_log(s, AV_LOG_WARNING,
> +                            "Negative cts, previous timestamps might be wrong.\n");
> +                    flv->wrong_dts = 1;
> +                }
>              } else if (FFABS(dts - pts) > 1000*60*15) {
>                  av_log(s, AV_LOG_WARNING,
>                         "invalid timestamps %"PRId64" %"PRId64"\n", dts, pts);
> @@ -1253,6 +1260,7 @@ static int flv_read_seek(AVFormatContext *s, int stream_index,
>  #define VD AV_OPT_FLAG_VIDEO_PARAM | AV_OPT_FLAG_DECODING_PARAM
>  static const AVOption options[] = {
>      { "flv_metadata", "Allocate streams according to the onMetaData array", OFFSET(trust_metadata), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, VD },
> +    { "flv_drop_negative_cts", "Drop frames with negative composition timestamp", OFFSET(drop_negative_cts), AV_OPT_TYPE_BOOL, { .i64 = 0 }, 0, 1, VD },
>      { "missing_streams", "", OFFSET(missing_streams), AV_OPT_TYPE_INT, { .i64 = 0 }, 0, 0xFF, VD | AV_OPT_FLAG_EXPORT | AV_OPT_FLAG_READONLY },
>      { NULL }
>  };

This seems all kind of wrong. You don't add a hack to a single demuxer
just because a single decoder can't handle unusual things in "some"
files. You don't add it as option either. (If this is a "fix my problem
the easiest way" hack, you should probably keep it in your own ffmpeg
branch.)

Why do the negative CTS happen, and what are their semantics?

Is this just the audio delay?