[FFmpeg-devel] [PATCH] avcodec/libjxlenc: avoid failure when level 10 is required

[Leo Izen]

If a level 10 codestream is required, JxlEncoderSetBasicInfo
will fail as it verifies the codestream level restrictions.
However, there's no way for the library to provide the info
on what codestream level is actually required until the BasicInfo
struct is already set. Thus, we work around this problem by
setting the codestream level to 10 in order to prevent
JxlEncoderSetBasicInfo from failing, and then we later query
the required codestream level.

Signed-off-by: Leo Izen <leo.izen at gmail.com>
---
 libavcodec/libjxlenc.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/libavcodec/libjxlenc.c b/libavcodec/libjxlenc.c
index 6a948cc3ae..b427d5d1a7 100644
--- a/libavcodec/libjxlenc.c
+++ b/libavcodec/libjxlenc.c
@@ -257,11 +257,25 @@ static int libjxl_encode_frame(AVCodecContext *avctx, AVPacket *pkt, const AVFra
         return ret;
     }
 
+    /*
+     * We set all codestreams to level 10 here first, because
+     * JxlEncoderSetBasicInfo fails if level10 is required, but there is no
+     * way to determine if level10 is required until after the basic info is
+     * already set (a catch-22). To work around this problem, we set the level
+     * to level10 immediately, populate the basic info, and then query the
+     * level that is actually required.
+     *
+     * Return value is not checked because on a fresh encoder instance this
+     * cannot fail.
+     */
+    JxlEncoderSetCodestreamLevel(ctx->encoder, 10);
+
     /* populate the basic info settings */
     JxlEncoderInitBasicInfo(&info);
     jxl_fmt.num_channels = pix_desc->nb_components;
     info.xsize = frame->width;
     info.ysize = frame->height;
+    /* num_extra_channels includes the alpha channel */
     info.num_extra_channels = (jxl_fmt.num_channels + 1) % 2;
     info.num_color_channels = jxl_fmt.num_channels - info.num_extra_channels;
     info.bits_per_sample = av_get_bits_per_pixel(pix_desc) / jxl_fmt.num_channels;
-- 
2.36.1