[FFmpeg-devel] [PATCH]Fix H264 VDPAU after r25243

[Michael Niedermayer]

On Tue, Oct 05, 2010 at 01:18:28PM +0200, Carl Eugen Hoyos wrote:
> On Tuesday 05 October 2010 01:07:25 pm Jason Garrett-Glaser wrote:
> > On Tue, Oct 5, 2010 at 2:44 AM, Carl Eugen Hoyos <cehoyos at ag.or.at> wrote:
> > > Hi!
> > >
> > > H264 VDPAU currently crashes for many streams.
> > > I don't know if av_image_copy() is supposed to work for PIX_FMT_*_VDPAU,
> > > but attached patch fixes the crashes.
> > >
> > > Please comment, Carl Eugen
> > 
> > Hmm, is that code even meaningful for VDPAU?  I thought the GPU did
> > its own memory/frame management.
> 
> I am unable to answer this question, but attached is a version that also fixes 
> the crash and is probably more correct.
> 
> Carl Eugen

>  h264.c |    2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> dfc024bf388bb204f2cefc1375c5fbab84d5ea0b  patchh264vdpau.diff
> Index: ffmpeg/libavcodec/h264.c
> ===================================================================
> --- ffmpeg/libavcodec/h264.c	(revision 25340)
> +++ ffmpeg/libavcodec/h264.c	(working copy)
> @@ -1917,7 +1917,7 @@
>                  if (prev) {
>                      av_image_copy(h->short_ref[0]->data, h->short_ref[0]->linesize,
>                                    (const uint8_t**)prev->data, prev->linesize,
> -                                  PIX_FMT_YUV420P, s->mb_width*16, s->mb_height*16);
> +                                  s->avctx->pix_fmt, s->mb_width*16, s->mb_height*16);
>                      h->short_ref[0]->poc = prev->poc+2;

while not correct for VDPAU this change still looks correct to me

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Breaking DRM is a little like attempting to break through a door even
though the window is wide open and the only thing in the house is a bunch
of things you dont want and which you would get tomorrow for free anyway
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20101005/469aa2bb/attachment.pgp>