[Libav-user] Overlay filter has incorrect frame rate

[Brian C. Wiles]

On 07/27/2018 10:10 AM, Philippe Gorley wrote:
> On 2018-07-27 10:38 AM, Brian C. Wiles wrote:
>> Hi,
>>
>>
>>    I am trying to programmatically overlay one video onto another, and
>> the top (overlaid) video plays at half it's normal frame rate.  If I use
>> the ffmpeg command, it works fine. >
>>
>>    I started with the doc/examples/transcoding.c example and changed the
>> filter initialization.  Here's how I create the filters:
>>
>>
>> static int init_filters(char *source, char *source_type)
>> {
>>      char filter_spec[2000];
>>      unsigned int i;
>>      int ret;
>>      filter_ctx = av_malloc_array(ifmt_ctx->nb_streams,
>> sizeof(*filter_ctx));
>>      if (!filter_ctx)
>>          return AVERROR(ENOMEM);
>>
>>      for (i = 0; i < ifmt_ctx->nb_streams; i++)
>>      {
>>          filter_ctx[i].buffersrc_ctx = NULL;
>>          filter_ctx[i].buffersink_ctx = NULL;
>>          filter_ctx[i].filter_graph = NULL;
>>          if (!(ifmt_ctx->streams[i]->codecpar->codec_type ==
>>                    AVMEDIA_TYPE_AUDIO ||
>>                ifmt_ctx->streams[i]->codecpar->codec_type ==
>> AVMEDIA_TYPE_VIDEO))
>>              continue;
>>
>>          if (ifmt_ctx->streams[i]->codecpar->codec_type ==
>> AVMEDIA_TYPE_VIDEO)
>>          {
>>              sprintf(
>>                  filter_spec,
>>                  "movie='anim.mp4', "
>>                  "scale=w='min(250\\, iw*3/2):h=-1' [wm]; "
>>                  "[in][wm]
>> overlay=x=main_w-overlay_w-50:y='main_h-overlay_h-50'"
>>                  ":enable='between(t,5,10)' [out]");
>>          }
>>          else
>>          {
>>              strcpy(filter_spec,
>>                     "anull"); /* passthrough (dummy) filter for audio */
>>          }
>>
>>          ret = init_filter(&filter_ctx[i], stream_ctx[i].dec_ctx,
>>                            stream_ctx[i].enc_ctx, filter_spec);
>>          if (ret)
>>              return ret;
>>      }
>>      return 0;
>> }
>>
>>    I have tried using the fps and setpts filters, but that seems like a
>> hack to get it to honor the actual frame rate in the file.  Let me know
>> if you need more info.  I appreciate any help someone can give me. 
>> Thanks!
>
> Overlay takes frames from each input in timestamp order. It likely
> overlays two frames when they have matching timestamps. So the fps and
> setpts filters are necessary if the 2 inputs have different
> framerates/starting points.
>
> The ffmpeg tool likely takes care of this for you.
>
>>
>>
>> -Brian
>>
> Cheers,
>
  That's what I wondered.  So, if I specify the right fps and setpts
combinations, it should work fine?  I thought the overlay filter was
supposed to add and drop frames as needed.  Also, do I specify the frame
rate of the overlaid video or the bottom one?  Thanks!