[FFmpeg-devel] [PATCH 1/1] libavcodec/h264: Handle ENOMEM error when decoding slice nal unit.

[Asaf Kave]

Hello all,
Long time pass, can somebody look at this?
Thanks



On Wed, Jul 17, 2019 at 11:35 AM Asaf Kave <kaveasaf at gmail.com> wrote:

> Populate the real error when alloc_picture failed, during decoding h264
> slice.
> When this error occurred, need to break the decoding process of the rest
> nal's, and need to rise this error back to the user, to indicate that there
> is decoder memory issue.
> ---
>  libavcodec/h264_slice.c | 5 +++--
>  libavcodec/h264dec.c    | 4 ++++
>  2 files changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/libavcodec/h264_slice.c b/libavcodec/h264_slice.c
> index 5ceee107a0..76353fe85b 100644
> --- a/libavcodec/h264_slice.c
> +++ b/libavcodec/h264_slice.c
> @@ -1644,9 +1644,10 @@ static int h264_field_start(H264Context *h, const
> H264SliceContext *sl,
>      }
>
>      if (!FIELD_PICTURE(h) || h->first_field) {
> -        if (h264_frame_start(h) < 0) {
> +        ret = h264_frame_start(h);
> +        if (ret < 0) {
>              h->first_field = 0;
> -            return AVERROR_INVALIDDATA;
> +            return ret;
>          }
>      } else {
>          int field = h->picture_structure == PICT_BOTTOM_FIELD;
> diff --git a/libavcodec/h264dec.c b/libavcodec/h264dec.c
> index 8d1bd16a8e..0a4d4b8c62 100644
> --- a/libavcodec/h264dec.c
> +++ b/libavcodec/h264dec.c
> @@ -754,6 +754,10 @@ static int decode_nal_units(H264Context *h, const
> uint8_t *buf, int buf_size)
>
>          if (err < 0) {
>              av_log(h->avctx, AV_LOG_ERROR, "decode_slice_header error\n");
> +            if (err == AVERROR(ENOMEM)) {
> +               ret = err;
> +               goto end;
> +            }
>          }
>      }
>
> --
> 2.17.1
>
>