[FFmpeg-devel] [PATCH] libavcodec/libfdk-aacenc: Scale VBR mode with FF_QP2LAMBDA

[Timo Rothenpieler]

On 22.02.2023 03:12, JonHGee wrote:
> libavcodec/libfdk-aacenc: VBR mode currently does not account for scaling when using -aq options with libfdk, resulting in clamping to vbr mode 5 whenever a value >0 is provided.  Adjusting for the scaling factor for proper VBR support.
> 
> ---
>   libavcodec/libfdk-aacenc.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/libavcodec/libfdk-aacenc.c b/libavcodec/libfdk-aacenc.c
> index 54549de473..da211baf51 100644
> --- a/libavcodec/libfdk-aacenc.c
> +++ b/libavcodec/libfdk-aacenc.c
> @@ -230,7 +230,7 @@ static av_cold int aac_encode_init(AVCodecContext *avctx)
>       }
>   
>       if (avctx->flags & AV_CODEC_FLAG_QSCALE || s->vbr) {
> -        int mode = s->vbr ? s->vbr : avctx->global_quality;
> +        int mode = s->vbr ? s->vbr : avctx->global_quality / FF_QP2LAMBDA;
>           if (mode <  1 || mode > 5) {
>               av_log(avctx, AV_LOG_WARNING,
>                      "VBR quality %d out of range, should be 1-5\n", mode);

Won't this break every existing command line and API client that has 
passed a value according to the current scale?

Also, what binds stronger here?
It this "(s->vbr ? s->vbr : avctx->global_quality) / FF_QP2LAMBDA" or 
"s->vbr ? s->vbr : (avctx->global_quality / FF_QP2LAMBDA)"?

In any case, this does not look correct to me.
Where would the sudden multiplication with FF_QP2LAMBDA come from in the 
first place?