[MPlayer-dev-eng] [PATCH] mplayer has broken Letterboxing with xv Video Output on i3wm

[Tobias Girstmair]

Hi all,

I'd like to propose the following patch, which fixes a problem with
tiling window managers. The patch should apply on both the 1.5 release,
as well as todays svn snapshot; happy to resubmit if there are problems.

Background:
   i3wm is a tiling window manager, meaning it force-resizes windows to
   fit into a grid. mplayer's xv backend doesn't draw letterboxes unless
   fullscreen is enabled, since it doesn't anticipate that its requested
   window size is overridden. This causes other windows' contents to
   appear in the mplayer window when switching workspaces, since the
   regions outside the video are not redrawn.

Solution:
   Remove the `if ( vo_fs )` gate in `void vo_xv_draw_colorkey(...)` in
   libvo/x11_common.c:2361.
   Since this function assumed the window is screen sized, it now draws
   oversized boxes. I modified it to use window width/height instead, so
   it only draw the pixels needed.

Steps to Reproduce:
   - use i3wm or another tiling window manager
   - play a video with mplayer -vo xv $FILE
   - switch workspaces back and forth
   - notice that the empty space around the video is now corrupted.


Thanks,
	tobi


Index: libvo/x11_common.c
===================================================================
--- MPlayer-1.5-orig/libvo/x11_common.c 2021-01-23 19:22:09.000000000 +0100
+++ MPlayer-1.5/libvo/x11_common.c      2022-05-06 23:36:32.977106863 +0200
@@ -2356,28 +2356,28 @@
                      w, h );
    }

-  /* draw black bars if needed */
    /* TODO! move this to vo_x11_clearwindow_part() */
-  if ( vo_fs )
+  /* draw left/right black bars if needed */
+  if (w < vo_dwidth)
    {
      XSetForeground( mDisplay, vo_gc, 0 );
-    /* making non-overlap fills, requires 8 checks instead of 4 */
-    if ( y > 0 )
-      XFillRectangle( mDisplay, vo_window, vo_gc,
-                      0, 0,
-                      vo_screenwidth, y);
-    if (x > 0)
-      XFillRectangle( mDisplay, vo_window, vo_gc,
-                      0, 0,
-                      x, vo_screenheight);
-    if (x + w < vo_screenwidth)
-      XFillRectangle( mDisplay, vo_window, vo_gc,
-                      x + w, 0,
-                      vo_screenwidth, vo_screenheight);
-    if (y + h < vo_screenheight)
-      XFillRectangle( mDisplay, vo_window, vo_gc,
-                      0, y + h,
-                      vo_screenwidth, vo_screenheight);
+    XFillRectangle( mDisplay, vo_window, vo_gc,
+                    0, 0,
+                    x, vo_dheight);
+    XFillRectangle( mDisplay, vo_window, vo_gc,
+                    x + w, 0,
+                    vo_dwidth, vo_dheight);
+  }
+  /* draw top/bottom black bars if needed */
+  if (h < vo_dheight)
+  {
+    XSetForeground( mDisplay, vo_gc, 0 );
+    XFillRectangle( mDisplay, vo_window, vo_gc,
+                    0, 0,
+                    vo_dwidth, y);
+    XFillRectangle( mDisplay, vo_window, vo_gc,
+                    0, y + h,
+                    vo_dwidth, vo_dheight);
    }
  }