[FFmpeg-devel] [PATCH]Fix H264 VDPAU after r25243

[Carl Eugen Hoyos]

On Tuesday 05 October 2010 01:07:25 pm Jason Garrett-Glaser wrote:
> On Tue, Oct 5, 2010 at 2:44 AM, Carl Eugen Hoyos <cehoyos at ag.or.at> wrote:
> > Hi!
> >
> > H264 VDPAU currently crashes for many streams.
> > I don't know if av_image_copy() is supposed to work for PIX_FMT_*_VDPAU,
> > but attached patch fixes the crashes.
> >
> > Please comment, Carl Eugen
> 
> Hmm, is that code even meaningful for VDPAU?  I thought the GPU did
> its own memory/frame management.

I am unable to answer this question, but attached is a version that also fixes 
the crash and is probably more correct.

Carl Eugen
-------------- next part --------------
Index: ffmpeg/libavcodec/h264.c
===================================================================
--- ffmpeg/libavcodec/h264.c	(revision 25340)
+++ ffmpeg/libavcodec/h264.c	(working copy)
@@ -1917,7 +1917,7 @@
                 if (prev) {
                     av_image_copy(h->short_ref[0]->data, h->short_ref[0]->linesize,
                                   (const uint8_t**)prev->data, prev->linesize,
-                                  PIX_FMT_YUV420P, s->mb_width*16, s->mb_height*16);
+                                  s->avctx->pix_fmt, s->mb_width*16, s->mb_height*16);
                     h->short_ref[0]->poc = prev->poc+2;
                 }
                 h->short_ref[0]->frame_num = h->prev_frame_num;