[FFmpeg-devel] [WIP] XComposite window capture demuxer (Linux)

[Marton Balint]

On Mon, 18 May 2020, Emanuele Oriani wrote:

> Hi Marton/ffmpeg devs/all,
>
> Haven't seen any response to the proposed XComposite windows capture.
>
> As per below the reasoning for adding this code to libav*/ffmpeg is that 
> seems to be better in quality than x11grab, although it uses more CPU in 
> its current implementation (uses OpenGL and PBO - optional - to fetch 
> XComposite windows).
>
> Please let me know if it's still of interest and/or you have further 
> technical feedback.
> I've posted some performance tests in the thread below.
>
> Here links of two captures (one with x11grab, the other with xcompgrab) 
> and the sources:
>
> https://send.firefox.com/download/df859689f4cc6429/#EN6w9upyIMHqO7n_1hMTaQ
> https://github.com/Emanem/replayer/blob/master/src/xcompgrab.c
>
> Thanks in advance for your time and feedback,

Thanks. I am not familiar with the details of opengl/compositing 
capturing, but here are a few questions coming into my mind. Not only for 
you, but for other developers as well:

- Is there a way to keep the captured textures as an 
OpenGL/VAAPI/NVenc/Vulkan object, so the frames can work as some kind of 
hwaccel frames? Can this improve performance?

- What can be the reason between the quality/smoothness difference of 
x11grab and the Compositor capturer?

As for the code, I only took a quick look, here are some comments

- probably GCC atomics should be replaced by <stdatomic.h> atomic 
functions.

- Unless there is a reason, I think av_malloc/av_free is preferred for 
memory alloc functions.

- Maybe some openGL glue can be shared with libavdevice/opengl_enc.c? Take 
a look, I am not sure, because that was implemented with SDL and cross 
platform capabilities in mind, but since your capture device is only for 
linux (or not?), then maybe it makes more sense to keep it separete?

- Can the Xlib functions replaced with xcb? I kind of think that would be 
preferable if it is a sensible thing to do.

And sorry, I cannot say how useful this would be, maybe now is the time 
for people to speak up if somebody is particularly against adding this for 
any reason.

Regards,
Marton