[FFmpeg-cvslog] h264: check the pixel format directly and force a reinit on mismatches.

[Michael Niedermayer]

ffmpeg | branch: release/1.1 | Michael Niedermayer <michaelni at gmx.at> | Tue Jan 29 04:17:48 2013 +0100| [3ef1538121fa6daeb1767510f1d4ae2c306c9fec] | committer: Michael Niedermayer

h264: check the pixel format directly and force a reinit on mismatches.

The existing checks are insufficient to detect a pixel format
changes in case of some damaged streams.
Fixes inconsistency and later out of array accesses

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni at gmx.at>
(cherry picked from commit 11c99c78bafa77f679a1a3ba06ad00984b9a4cae)

Signed-off-by: Michael Niedermayer <michaelni at gmx.at>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=3ef1538121fa6daeb1767510f1d4ae2c306c9fec
---

 libavcodec/h264.c |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/libavcodec/h264.c b/libavcodec/h264.c
index 7ae778a..1152258 100644
--- a/libavcodec/h264.c
+++ b/libavcodec/h264.c
@@ -2765,7 +2765,8 @@ static int decode_slice_header(H264Context *h, H264Context *h0)
                      || s->avctx->bits_per_raw_sample != h->sps.bit_depth_luma
                      || h->cur_chroma_format_idc != h->sps.chroma_format_idc
                      || av_cmp_q(h->sps.sar, s->avctx->sample_aspect_ratio)));
-
+    if (h0->s.avctx->pix_fmt != get_pixel_format(h0))
+        must_reinit = 1;
 
     s->mb_width  = h->sps.mb_width;
     s->mb_height = h->sps.mb_height * (2 - h->sps.frame_mbs_only_flag);