[FFmpeg-devel] [PATCH] avcodec/h264: remove redundant and bogus get_format call

[wm4]

The AVCodecContext.get_format callback is not only used for pixel format
negotiation with the API user, but also for hwaccel init. For the
latter, it's required that some codec parameters, in particular the
codec profile, are set when the callback is invoked.

This patch removes a get_format invocation where this is not guaranteed.
The codec parameters, including the profile, are really set further
below. (The same code path that sets the profile also calls get_format
properly too.)

This just happened to work by coincidence in most cases. For example, if
the API user just copied or reused the AVStream's AVCodecContext when
decoding, the profile would be set properly. But in some cases it
fails., such as with the sample WolfensteinTwitch.mp4 on the samples
server.

Remove the redundant get_format call. Apparently it serves no purpose
anymore, although it is possible that this was different at the time it
was added in commit ffd77f94a26be22b8ead3178ceec3ed39e68abc5.

This fixes hwaccel usage for API users which do not set the profile
when setting up the AVCodecContext (which is allowed).
---
 libavcodec/h264_slice.c | 14 --------------
 1 file changed, 14 deletions(-)

diff --git a/libavcodec/h264_slice.c b/libavcodec/h264_slice.c
index a346ccb..1272319 100644
--- a/libavcodec/h264_slice.c
+++ b/libavcodec/h264_slice.c
@@ -1129,17 +1129,6 @@ fail:
     return ret;
 }
 
-static enum AVPixelFormat non_j_pixfmt(enum AVPixelFormat a)
-{
-    switch (a) {
-    case AV_PIX_FMT_YUVJ420P: return AV_PIX_FMT_YUV420P;
-    case AV_PIX_FMT_YUVJ422P: return AV_PIX_FMT_YUV422P;
-    case AV_PIX_FMT_YUVJ444P: return AV_PIX_FMT_YUV444P;
-    default:
-        return a;
-    }
-}
-
 /**
  * Decode a slice header.
  * This will (re)intialize the decoder and call h264_frame_start() as needed.
@@ -1323,9 +1312,6 @@ int ff_h264_decode_slice_header(H264Context *h, H264SliceContext *sl)
                      || h->mb_width  != h->sps.mb_width
                      || h->mb_height != h->sps.mb_height * (2 - h->sps.frame_mbs_only_flag)
                     ));
-    if (h->avctx->pix_fmt == AV_PIX_FMT_NONE
-        || (non_j_pixfmt(h->avctx->pix_fmt) != non_j_pixfmt(get_pixel_format(h, 0))))
-        must_reinit = 1;
 
     if (first_slice && av_cmp_q(h->sps.sar, h->avctx->sample_aspect_ratio))
         must_reinit = 1;
-- 
2.5.1