[FFmpeg-user] How to dynamically drop frames as required to keep transcoding real time?

[Mark Filipak (ffmpeg)]

On 2021-04-07 10:10, Adrian Cable wrote:
> Mark – sorry if I wasn’t clear in my original email. The problem I am describing is real, not theoretical. I have enormous respect for the time of everyone on this list, and would not have emailed if my problem could be solved with “-vsync cfr” or anything as straightforward and as well documented.
> 
> XXX *will not* be constant and *will not* be independent of the input frame rate. This is unfortunately the problem I am trying to solve.

Sorry, I thought XXX referred to the output -- -vf fps=fps=XXX is *output* frame rate. To the best 
of my knowledge, the input frame rate of VFR video can't be specified.

> XXX will not be constant because the CPU load on the device (due to other running processes) varies over time. If other processes are loading the CPU, there will be less CPU available for encoding, so XXX will be lower. ...

If the input FR drops below the CFR output, the 'fps' filter will repeat input frames in order to 
maintain CFR output.

>... If the CPU load on the device (due to other running processes) is low, XXX will be higher. ...

If input FR rises above the CFR output, the 'fps' filter will drop input frames in order to maintain 
CFR output.

>... And, because CPU load (due to other running processes) may change during the transcode, XXX will vary during the transcode.

It seems to me that what you want is to somehow maintain picture rate regardless of frame rate. I'm 
sorry if I misunderstand, but I don't think I can help you. Sorry. I suggest you look into variable 
resolution encoding in order to maintain constant picture rate (but at lower resolution if/when 
input FR exceeds output FR). I don't know how to do that.

Good hunting,
Mark.

> XXX will not be independent of the input frame rate, because decoding the input frames consumes CPU and memory bandwidth, which leaves less available for the re-encoding part. If the input frame rate is 60 fps, for example, XXX will be lower than if the input frame rate were 5 fps.
> 
> -Adrian
> 
> 
> From: ffmpeg-user <ffmpeg-user-bounces at ffmpeg.org> on behalf of Mark Filipak (ffmpeg) <markfilipak at bog.us>
> Date: Wednesday, April 7, 2021 at 6:43 AM
> To: ffmpeg-user at ffmpeg.org <ffmpeg-user at ffmpeg.org>
> Subject: Re: [FFmpeg-user] How to dynamically drop frames as required to keep transcoding real time?
> On 2021-04-05 21:02, Adrian Cable wrote:
>> Hi,
>> I have a question that I thought should have an ‘easy’ answer – I have spent a lot of time searching and I cannot find it, so thought I would ask here.
>>
>> My application sounds simple. I have an input RTSP stream, which I need to transcode and output as an RTP stream. Command line is very simple, something like:
>>
>> ./ffmpeg -i rtsp://… -vf scale=1024:-1 -c:v libx264 -f rtp rtp://…
>>
>> Now, ffmpeg is running on a CPU-constrained platform, and it may be that, in order for the transcoding to ‘keep up’ with the input, the output framerate needs to be less than the input framerate, which is totally fine. The problem/question is: how do I get ffmpeg to drop frames as required to keep the pipeline running real time?
>>
>> I can’t simply ‘guess’ a sustainable output framerate and set using -r XXX or -vf fps=fps=XXX, because the properties of the input can change at any time, so any XXX won’t be constant.
> 
> XXX *will* be constant and independent of the input frame rate.
> "Convert the video to specified constant frame rate by duplicating or dropping frames as necessary."
> 
> I hope this helps,
> Mark.
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> 
> To unsubscribe, visit link above, or email
> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
> _______________________________________________
> ffmpeg-user mailing list
> ffmpeg-user at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
> 
> To unsubscribe, visit link above, or email
> ffmpeg-user-request at ffmpeg.org with subject "unsubscribe".
> 


-- 
Mars? We can't live there! Mars is fatal: no magnetosphere to deflect cosmic rays. A tropical 
sunshade orbiting Earth would work. It's the only way. We need to start construction now. Forget Mars.