[FFmpeg-devel] [PATCH] decode: don't treat hwaccel->alloc_frame as always failing

[James Almer]

On 5/16/2020 8:41 PM, comex wrote:
> Fixes regression caused by a1133db30ef07896afd96f067e5c51531a4e85ab.
> 
> On my Mac, the bug seems to make `mpv --hwdec=auto` always fail with:
> 
> [ffmpeg/video] h264: get_buffer() failed
> [ffmpeg] Assertion src->f->buf[0] failed at src/libavcodec/h264_picture.c:70
> 
> Signed-off-by: comex <comexk at gmail.com>
> ---
>  libavcodec/decode.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/libavcodec/decode.c b/libavcodec/decode.c
> index 48a61d5419..3fdfb551e3 100644
> --- a/libavcodec/decode.c
> +++ b/libavcodec/decode.c
> @@ -1889,7 +1889,8 @@ int ff_get_buffer(AVCodecContext *avctx, AVFrame *frame, int flags)
>      if (hwaccel) {
>          if (hwaccel->alloc_frame) {
>              ret = hwaccel->alloc_frame(avctx, frame);
> -            goto fail;
> +            if (ret < 0)
> +                goto fail;

This is meant to skip the get_buffer2() call. The old goto end removed
in the commit you quoted ensured that was the case, but it was mistaken
for a failure path.

Either an end label is re-added immediately after the
ff_attach_decode_data() call and used here, or the videotoolbox hwaccel
is changed to not require an AVHWAccel.alloc_frame() implementation to
being with.

>          }
>      } else
>          avctx->sw_pix_fmt = avctx->pix_fmt;
>