[Libav-user] How to handle video muxing to file when real frame rate varies from the specified frame rate

[Nuno Santos]

Alex,

Today I have got somewhere I haven’t been before but still having a problem. 

If producing half of the frames, it seems to work. Of course is not perfect but 15 frames instead of 30 fps has to have some noticeable harsh effect. However, when producing all the frames it is supposed to produce, the program stops with the message:

get_video_frame - timestamp 236 pts 7
get_video_frame - timestamp 301 pts 9
get_video_frame - timestamp 308 pts 9
get_video_frame - timestamp 342 pts 10
int write_frame - pts 3584 dts 2560
get_video_frame - timestamp 376 pts 11
get_video_frame - timestamp 408 pts 12
int write_frame - pts 4608 dts 3584
get_video_frame - timestamp 452 pts 13
int write_frame - pts 4608 dts 3584

[mp4 @ 0x7fb86a98cc00] Application provided invalid, non monotonically increasing dts to muxer in stream 0: 3584 >= 3584


This is the relevant code:

AVFrame* get_video_frame(int timestamp, OutputStream *ost, const QImage &image)
{
    ...
    ost->frame->pts = (timestamp*_fps)/1000;
    // print timestamp and frame pts to console
    return ost->frame;
}

int write_frame(AVFormatContext *fmt_ctx, AVCodecContext *c, AVStream *st, AVFrame *frame)
{
    int ret = avcodec_send_frame(c, frame);

    if (ret<0) exit(1);

    while (ret >= 0)
    {
        AVPacket pkt = { 0 };

        ret = avcodec_receive_packet(c, &pkt);

        if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF)
        {
            break;
        }
        else if (ret<0)
        {
            fprintf(stderr, "Error encoding a frame: %s\n", av_err2str(ret));
            exit(1);
        }

        pkt.stream_index = st->index;

        if (pkt.pts != AV_NOPTS_VALUE)
            pkt.pts = av_rescale_q(pkt.pts, c->time_base, st->time_base);

        if (pkt.dts != AV_NOPTS_VALUE)
            pkt.dts = av_rescale_q(pkt.dts, c->time_base, st->time_base);

	// print packet pts and packet dts to console
        ret = av_interleaved_write_frame(fmt_ctx, &pkt);
        av_packet_unref(&pkt);

        if (ret < 0) exit(1);
    }

    return ret == AVERROR_EOF ? 1 : 0;
}

What I missing?

Thank you very much!

Regards,

Nuno

> On 9 Sep 2021, at 11:34, Nuno Santos <nuno.santos at imaginando.pt> wrote:
> 
> Hi,
> 
> I’m still finding the solution to this problem.
> 
> I have tried to summarise the problem the best way I could in a stack overflow question, with code snippets:
> 
> https://stackoverflow.com/questions/69109263/how-to-compensate-frame-rate-underrun-while-muxing-video-to-mp4-container-with-l <https://stackoverflow.com/questions/69109263/how-to-compensate-frame-rate-underrun-while-muxing-video-to-mp4-container-with-l>
> 
> Lib-av is definitively a complex topic with a lot of background knowledge and history that we can’t simply catch up in a snap. 
> 
> Thanks for your help and comprehension
> 
> Best regards,
> 
> Nuno
> 
>> On 5 Sep 2021, at 19:55, Nuno Santos <nuno.santos at imaginando.pt <mailto:nuno.santos at imaginando.pt>> wrote:
>> 
>> Alex,
>> 
>> Thanks for your reply. 
>> 
>> I’m still puzzled how to do this.
>> 
>> I’ve just tried to do this in the get_video_frame of muxing.c example
>> 
>> ost->frame->pkt_dts = av_rescale_q(timestamp, AVRational{1, 1000}, ost->st->time_base);
>> ost->frame->pts = ost->frame->pkt_dts;
>> 
>> But the program crashes even more quickly:
>> 
>> [mp4 @ 0x7fc9a6242600] pts (817920) < dts (841472) in stream 0
>> 
>> 
>> There is another place in the muxing.c example that uses a rescale function, av_packet_rescale_ts actually, on the write_frame function. But this function is used for both video and audio frames writing. Since the problem with frame producing is only for video, I would say that the rescale must be on the video side only, so it kind makes sense doing this adjustment on the get_video_frame function.
>> 
>> Any tips? 
>> 
>> Thanks!
>> 
>> Regards,
>> 
>> Nuno
>> 
>>> On 2 Sep 2021, at 22:06, Alexandr Kasyan <a.kasyan at ntechlab.com <mailto:a.kasyan at ntechlab.com>> wrote:
>>> 
>>> Nuno, 
>>> 
>>> I was too wordy. Assign dts, make dts = pts, they come together in the case.
>>> Is it getting better or do you still need a better way to calculate pts?
>>> 
>>> Regards, Alex
>>> _______________________________________________
>>> Libav-user mailing list
>>> Libav-user at ffmpeg.org <mailto:Libav-user at ffmpeg.org>
>>> https://ffmpeg.org/mailman/listinfo/libav-user
>>> 
>>> To unsubscribe, visit link above, or email
>>> libav-user-request at ffmpeg.org with subject "unsubscribe".
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://ffmpeg.org/pipermail/libav-user/attachments/20210910/a3124c78/attachment.htm>