[FFmpeg-devel] [PATCH] avcodec/av1dec: Always set ret before goto end

[Andreas Rheinhardt]

Hendrik Leppkes:
> On Thu, May 2, 2024 at 10:22 AM Andreas Rheinhardt
> <andreas.rheinhardt at outlook.com> wrote:
>>
>> Before 0f8763fbea4e8816cd54c2a481d4c048fec58394, av1_frame_ref()
>> and update_reference_list() could fail and therefore needed to
>> be checked, which incidentally set ret. This is no longer happening,
>> leading to a potential use of an uninitialized value which is
>> also the subject of Coverity ticket #1596605.
>>
>> Fix this by always setting ret before goto end; do not return
>> some random ancient value.
>>
>> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt at outlook.com>
>> ---
>>  libavcodec/av1dec.c | 12 ++++++++++--
>>  1 file changed, 10 insertions(+), 2 deletions(-)
>>
>> diff --git a/libavcodec/av1dec.c b/libavcodec/av1dec.c
>> index 79a30a114d..c3f255a29a 100644
>> --- a/libavcodec/av1dec.c
>> +++ b/libavcodec/av1dec.c
>> @@ -1335,6 +1335,12 @@ static int av1_receive_frame_internal(AVCodecContext *avctx, AVFrame *frame)
>>                      ret = set_output_frame(avctx, frame);
>>                      if (ret < 0)
>>                          av_log(avctx, AV_LOG_ERROR, "Set output frame error.\n");
>> +                } else {
>> +                    // CBS checks for us that the frame to be shown actually existed
>> +                    // in the bitstream; if it doesn't it could be e.g. due to
>> +                    // skip_frame setting. Return EAGAIN to indicate that we are
>> +                    // currently unable to produce output.
>> +                    ret = AVERROR(EAGAIN);
>>                  }
>>
> 
> In the vein of this comment, set_output_frame will also return 0
> without returning a frame in some cases - eg. with multiple layers.
> Should this equally return EAGAIN rather than claiming success without
> a frame?
> 

Thanks for pointing this out. There is a translation 0->AVERROR(EAGAIN)
at the end of this function if the output frame is unset. Maybe this
commit should rather set ret to 0 and rely on this (as it was before
0f8763fbea)? Let's hear what James says about this.

- Andreas