[FFmpeg-devel] [FFmpeg-cvslog] h264: check the pixel format directly and force a reinit on mismatches.

[Michael Niedermayer]

On Sun, Feb 17, 2013 at 08:03:39PM +0100, Guillaume POIRIER wrote:
> Moin-moin,
> 
> On Tue, Jan 29, 2013 at 4:58 AM, Michael Niedermayer <git at videolan.org> wrote:
> > ffmpeg | branch: master | Michael Niedermayer <michaelni at gmx.at> | Tue Jan 29 04:17:48 2013 +0100| [11c99c78bafa77f679a1a3ba06ad00984b9a4cae] | committer: Michael Niedermayer
> >
> > h264: check the pixel format directly and force a reinit on mismatches.
> >
> > The existing checks are insufficient to detect a pixel format
> > changes in case of some damaged streams.
> > Fixes inconsistency and later out of array accesses
> 
> This fix breaks Hardware-accelerated decoding on OSX through VDA.
> 
> I get these kind of errors:
> A:   1.0 V:   0.0 A-V:  1.025 ct:  0.000   0/  0 ??% ??% ??,?% 218 0
> [h264_vda @ 0x10a0b91a0]Reinit context to 1920x1088, pix_fmt: 81
> [h264_vda @ 0x10a0b91a0]Missing reference picture, default is 0
> [h264_vda @ 0x10a0b91a0]decode_slice_header error
> [h264_vda @ 0x10a0b91a0]Reinit context to 1920x1088, pix_fmt: 81
> [h264_vda @ 0x10a0b91a0]reference picture missing during reorder
> [h264_vda @ 0x10a0b91a0]reference picture missing during reorder
> [h264_vda @ 0x10a0b91a0]reference picture missing during reorder
> [h264_vda @ 0x10a0b91a0]reference picture missing during reorder
> [h264_vda @ 0x10a0b91a0]Missing reference picture, default is 0
> [h264_vda @ 0x10a0b91a0]decode_slice_header error
> 
> Reverting just that commit restores H.264 decoding via VDA.
> 
> So what can I do to have the best of both worlds, that is, VDA
> decoding and the pixel format changes issue fixed by this commit?

please try:
From 8257f29ea56214c8eb97c655a2644283e2271817 Mon Sep 17 00:00:00 2001
From: Michael Niedermayer <michaelni at gmx.at>
Date: Sun, 17 Feb 2013 20:41:16 +0100
Subject: [PATCH] h264: avoid calling get_format() multiple times

Some applications do not like that.

Signed-off-by: Michael Niedermayer <michaelni at gmx.at>
---
 libavcodec/h264.c |   10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/libavcodec/h264.c b/libavcodec/h264.c
index 89fc222..c14b95a 100644
--- a/libavcodec/h264.c
+++ b/libavcodec/h264.c
@@ -2931,11 +2931,17 @@ static enum PixelFormat get_pixel_format(H264Context *h)
             return h->avctx->color_range == AVCOL_RANGE_JPEG ? AV_PIX_FMT_YUVJ422P
                                                              : AV_PIX_FMT_YUV422P;
         } else {
-            return h->avctx->get_format(h->avctx, h->avctx->codec->pix_fmts ?
+            int i;
+            const enum AVPixelFormat * fmt = h->avctx->codec->pix_fmts ?
                                         h->avctx->codec->pix_fmts :
                                         h->avctx->color_range == AVCOL_RANGE_JPEG ?
                                         hwaccel_pixfmt_list_h264_jpeg_420 :
-                                        ff_hwaccel_pixfmt_list_420);
+                                        ff_hwaccel_pixfmt_list_420;
+
+            for (i=0; fmt[i] != AV_PIX_FMT_NONE; i++)
+                if (fmt[i] == h->avctx->pix_fmt)
+                    return fmt[i];
+            return h->avctx->get_format(h->avctx, fmt);
         }
         break;
     default:
--
1.7.9.5

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

I do not agree with what you have to say, but I'll defend to the death your
right to say it. -- Voltaire
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20130217/5017c820/attachment.asc>