[MPlayer-users] VDPAU errors, flickering black video display on aspect change in MPEG2 TS

[Reimar Döffinger]

On Sun, Dec 08, 2013 at 03:18:28PM +0100, Ilja Sekler wrote:
> Am 08.12.2013 14:52, schrieb Reimar Döffinger:
> >--- a/libvo/vo_vdpau.c
> >+++ b/libvo/vo_vdpau.c
> >@@ -697,6 +697,9 @@ static int config(uint32_t width, uint32_t height, uint32_t d_width,
> >          vo_x11_create_vo_window(&vinfo, vo_dx, vo_dy, d_width, d_height,
> >                                  flags, CopyFromParent, "vdpau", title);
> >          resize();
> >+        if (IMGFMT_IS_VDPAU(image_format)
> >+            && !create_vdp_decoder(image_format, vid_width, vid_height, 2))
> >+            return -1;
> >          return 0;
> >      }
> >      image_format = format;
> 
> The patch fixes this particular problem and brings back the issue with
> multithreaded VDPAU decoding for h264 as described in
> 
> http://lists.mplayerhq.hu/pipermail/mplayer-users/2013-August/086498.html

A more brute-force hack then, with the advantage that it will
only affect buggy drivers:
--- a/libvo/vo_vdpau.c
+++ b/libvo/vo_vdpau.c
@@ -1031,6 +1031,12 @@ static int draw_slice(uint8_t *image[], int stride[], int w, int h,
         return VO_FALSE;
 
     vdp_st = vdp_decoder_render(decoder, rndr->render_state->surface, rndr->info, rndr->bitstream_buffers_used, rndr->bitstream_buffers);
+    if (vdp_st == VDP_STATUS_INVALID_SIZE) {
+        // reinit to work around a NVidia bug on aspect change
+        if (!create_vdp_decoder(image_format, vid_width, vid_height, max_refs))
+            return VO_FALSE;
+        vdp_st = vdp_decoder_render(decoder, rndr->render_state->surface, rndr->info, rndr->bitstream_buffers_used, rndr->bitstream_buffers);
+    }
     CHECK_ST_WARNING("Failed VDPAU decoder rendering");
     return VO_TRUE;
 }