[FFmpeg-user] x11grab segfault when display is closed

[Moritz Barsnick]

On Tue, Jun 26, 2018 at 00:35:19 +0200, Carl Eugen Hoyos wrote:
> > # ./ffmpeg -an -f x11grab -framerate 10 -video_size 1280x1024 -i :13
> > /tmp/cap.mp4
> > However, if ffmpeg is still running when the display (:13 in the above
> > command) is closed, ffmpeg segfaults.
> 
> Please provide backtrace, disassembly and register dump to
> allow us to debug the issue you see.

Without providing that (too lazy, and I'm not the original poster), I
can tell you that I can reporduce this, and it happens in this code in
libavdevice/xcbgrab.c:xcbgrab_read_packet():

	    if (c->follow_mouse || c->draw_mouse) {
	        pc  = xcb_query_pointer(c->conn, c->screen->root);
	        gc  = xcb_get_geometry(c->conn, c->screen->root);
	        p   = xcb_query_pointer_reply(c->conn, pc, NULL);
	        geo = xcb_get_geometry_reply(c->conn, gc, NULL);
	    }
	
	    if (c->follow_mouse && p->same_screen)
	        xcbgrab_reposition(s, p, geo);
	
	    if (c->show_region)
	        xcbgrab_update_region(s);
	
	#if CONFIG_LIBXCB_SHM
	    if (c->has_shm && xcbgrab_frame_shm(s, pkt) < 0)
	        c->has_shm = 0;
	#endif
	    if (!c->has_shm)
	        ret = xcbgrab_frame(s, pkt);
	
	#if CONFIG_LIBXCB_XFIXES
	    if (ret >= 0 && c->draw_mouse && p->same_screen)
	        xcbgrab_draw_mouse(s, pkt, p, geo);
	#endif

I get a crash at "if (ret >= 0 && c->draw_mouse && p->same_screen)"
because p is NULL. Presumably, all those xcb_*() functions at the top
of my quote can return NULL if the display is no longer accessible:

(gdb) p p
$2 = (xcb_query_pointer_reply_t *) 0x0
(gdb) p geo
$3 = (xcb_get_geometry_reply_t *) 0x0
(gdb) p pc
$4 = {sequence = 1247}
(gdb) p gc
$5 = {sequence = 1248}

The xcbgrab code probably needs to detect this and somehow generate
EOF.

(To reproduce:
 Terminal 1:
 $ Xvfb :1 -nolisten tcp -screen 0 800x600x24
 Terminal 2:
 $ ffmpeg -f x11grab -i :1 -f null -
 or rather
 $ gdb -ex r --args ffmpeg_g -f x11grab -i :1 -f null -
Then terminate Xvfb while ffmpeg is running.)

Cheers,
Moritz