[FFmpeg-user] How to dynamically drop frames as required to keep transcoding real time?

[Adrian Cable]

Mark,

> Sorry, I thought XXX referred to the output -- -vf fps=fps=XXX is *output* frame rate. To the best
> of my knowledge, the input frame rate of VFR video can't be specified.

I’m referring to XXX as the maximum *output* frame rate that the pipeline can support at a given time, while keeping up with the input.

> If input FR rises above the CFR output, the 'fps' filter will drop input frames in order to maintain
> CFR output.

Sure! But this isn’t what I’m looking for. If the actual output FR drops below the FR set (by -r XXX or -vf fps=fps=XXX etc.), the encoder will not drop frames. Instead the delay between input and output will just increase forever. I am looking for a way to make the encoder drop frames if the actual output FR drops below the FR set. For example: if I set -r 15 but (due to CPU limits) the encoder can only manage to produce 6 fps, right now it won’t drop frames, and as a result won’t keep up with the input. I want to make it drop frames if needed so it keeps up with the input.

-Adrian


From: ffmpeg-user <ffmpeg-user-bounces at ffmpeg.org> on behalf of Mark Filipak (ffmpeg) <markfilipak at bog.us>
Date: Wednesday, April 7, 2021 at 2:49 PM
To: ffmpeg-user at ffmpeg.org <ffmpeg-user at ffmpeg.org>
Subject: Re: [FFmpeg-user] How to dynamically drop frames as required to keep transcoding real time?
On 2021-04-07 10:10, Adrian Cable wrote:
> Mark – sorry if I wasn’t clear in my original email. The problem I am describing is real, not theoretical. I have enormous respect for the time of everyone on this list, and would not have emailed if my problem could be solved with “-vsync cfr” or anything as straightforward and as well documented.
>
> XXX *will not* be constant and *will not* be independent of the input frame rate. This is unfortunately the problem I am trying to solve.

Sorry, I thought XXX referred to the output -- -vf fps=fps=XXX is *output* frame rate. To the best
of my knowledge, the input frame rate of VFR video can't be specified.

> XXX will not be constant because the CPU load on the device (due to other running processes) varies over time. If other processes are loading the CPU, there will be less CPU available for encoding, so XXX will be lower. ...

If the input FR drops below the CFR output, the 'fps' filter will repeat input frames in order to
maintain CFR output.

>... If the CPU load on the device (due to other running processes) is low, XXX will be higher. ...

If input FR rises above the CFR output, the 'fps' filter will drop input frames in order to maintain
CFR output.

>... And, because CPU load (due to other running processes) may change during the transcode, XXX will vary during the transcode.

It seems to me that what you want is to somehow maintain picture rate regardless of frame rate. I'm
sorry if I misunderstand, but I don't think I can help you. Sorry. I suggest you look into variable
resolution encoding in order to maintain constant picture rate (but at lower resolution if/when
input FR exceeds output FR). I don't know how to do that.

Good hunting,
Mark.

> XXX will not be independent of the input frame rate, because decoding the input frames consumes CPU and memory bandwidth, which leaves less available for the re-encoding part. If the input frame rate is 60 fps, for example, XXX will be lower than if the input frame rate were 5 fps.
>
> -Adrian
>
>
> From: ffmpeg-user <ffmpeg-user-bounces at ffmpeg.org> on behalf of Mark Filipak (ffmpeg) <markfilipak at bog.us>
> Date: Wednesday, April 7, 2021 at 6:43 AM
> To: ffmpeg-user at ffmpeg.org <ffmpeg-user at ffmpeg.org>
> Subject: Re: [FFmpeg-user] How to dynamically drop frames as required to keep transcoding real time?
> On 2021-04-05 21:02, Adrian Cable wrote:
>> Hi,
>> I have a question that I thought should have an ‘easy’ answer – I have spent a lot of time searching and I cannot find it, so thought I would ask here.
>>
>> My application sounds simple. I have an input RTSP stream, which I need to transcode and output as an RTP stream. Command line is very simple, something like:
>>
>> ./ffmpeg -i rtsp://… -vf scale=1024:-1 -c:v libx264 -f rtp rtp://…
>>
>> Now, ffmpeg is running on a CPU-constrained platform, and it may be that, in order for the transcoding to ‘keep up’ with the input, the output framerate needs to be less than the input framerate, which is totally fine. The problem/question is: how do I get ffmpeg to drop frames as required to keep the pipeline running real time?
>>
>> I can’t simply ‘guess’ a sustainable output framerate and set using -r XXX or -vf fps=fps=XXX, because the properties of the input can change at any time, so any XXX won’t be constant.
>
> XXX *will* be constant and independent of the input frame rate.
> "Convert the video to specified constant frame rate by duplicating or dropping frames as necessary."
>
> I hope this helps,
> Mark.
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
>


--
Mars? We can't live there! Mars is fatal: no magnetosphere to deflect cosmic rays. A tropical
sunshade orbiting Earth would work. It's the only way. We need to start construction now. Forget Mars.
_______________________________________________
ffmpeg-user mailing list
ffmpeg-user at ffmpeg.org
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".