[FFmpeg-devel] [PATCH] avcodec/omx: fix xFramerate calculation

[Aman Gupta]

From: Aman Gupta <aman at tmm1.net>

Integer overflow in the Q16 framerate calculation was sending
invalid values to the OMX encoder.

On the RPI4, this manifested as bitrate controls being ignored
on video streams with 60000/1001 framerates. Video streams with
30000/1001 framerates were not affected.

Signed-off-by: Aman Gupta <aman at tmm1.net>
---
 libavcodec/omx.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/libavcodec/omx.c b/libavcodec/omx.c
index 837f5df666..c745860242 100644
--- a/libavcodec/omx.c
+++ b/libavcodec/omx.c
@@ -473,9 +473,9 @@ static av_cold int omx_component_init(AVCodecContext *avctx, const char *role)
     in_port_params.format.video.nFrameWidth  = avctx->width;
     in_port_params.format.video.nFrameHeight = avctx->height;
     if (avctx->framerate.den > 0 && avctx->framerate.num > 0)
-        in_port_params.format.video.xFramerate = (1 << 16) * avctx->framerate.num / avctx->framerate.den;
+        in_port_params.format.video.xFramerate = (int64_t)(1 << 16) * avctx->framerate.num / avctx->framerate.den;
     else
-        in_port_params.format.video.xFramerate = (1 << 16) * avctx->time_base.den / avctx->time_base.num;
+        in_port_params.format.video.xFramerate = (int64_t)(1 << 16) * avctx->time_base.den / avctx->time_base.num;
 
     err = OMX_SetParameter(s->handle, OMX_IndexParamPortDefinition, &in_port_params);
     CHECK(err);
-- 
2.20.1