[FFmpeg-user] How to cope with slow to open v4l2 devices

[LANGLOIS Olivier PIS -EXT]

> -----Original Message-----
> From: ffmpeg-devel-bounces at ffmpeg.org [mailto:ffmpeg-devel-
> bounces at ffmpeg.org] On Behalf Of LANGLOIS Olivier PIS -EXT
> Sent: Monday, March 17, 2014 5:47 PM
> To: ffmpeg-user at ffmpeg.org; ffmpeg-devel at ffmpeg.org
> Subject: [FFmpeg-devel] How to cope with slow to open v4l2 devices
>
> My v4l2 input is very slow (it is measured in seconds) to open and my other
> video input is a live feed produced by a process that will just bail out if
> ffmpeg is not able to consume the raw stream in real-time.
>
> The whole thing choke right at the beginning when ffmpeg opens the v4l2
> device.
> My live video producer determine that ffmpeg is too slow and close its side
> of the pipe.
> ALSA generates buffer overruns.
>
> What is the usual strategy to deal with several live inputs with different
> initialization times?
>
> /dev/video0 is a old Logitech webcam. I'm not ruling the possibility to buy a
> faster device if it can solve my problem.
> I also have full control over the live feed video producer I am just not sure
> how to modify it to cope with the problem.
> (https://github.com/lano1106/glcs)
>
> I could make it drop frames but then how will it discern between a one time
> initialization slow down vs a permanent problem where the encoder is just
> too slow to process the stream in realtime?
>
> I do not like too much this idea as I wonder how I will ensure synchronization
> between the different streams.
>
> 2. If I was reordering the input declaration order from the slowest device to
> the fastest on the cmdline, could it help?
>
> 3. Can ffmpeg report back when its initialization is done and when it is ready
> to accept data by socket and/or pipe?
>

I have digged a little bit more the question. initializing a usb v4l2 video capture device is taking ~300ms.

in libavdevice/v4l2.c:

The call to v4l2_ioctl(s->fd, VIDIOC_STREAMON, &type)

takes about 150 ms and by looking into the driver code, there is nothing that can be done to it as most of the time is spent inside usb_set_interface() which exchanges a couple of usb control messages. USB is slow.

Next, inside avformat_find_stream_info(), it takes about 8000 calls to v4l2_read_packet() and another 150 ms before the function returns something else than EAGAIN.

The solution that I have found finally is to increase the stdin buffer size to over 100MB so it can store about 15 frames 1080P to amortize the capture delay due to the webcam initialization.

You can see the result at with:

ffmpeg -nostats -y \
 -f rawvideo -video_size 1920x1080 -pixel_format bgra -framerate 30 -i /dev/stdin \
 -f alsa -acodec pcm_s16le -ar 44100 -ac 2 -i loop_capture \
 -f alsa -acodec pcm_s16le -ar 32000 -ac 2 -i hw:3,0 \
 -f v4l2 -input_format yuyv422 -video_size 320x240 -framerate 30 -i /dev/video0 \
 -filter_complex "overlay;amix" \
 -c:a libfdk_aac -profile:a aac_low -b:a 128k -ar 44100 \
 -c:v libx264 -preset veryfast -profile:v main -level 4.1 -pix_fmt yuv420p \
 -x264opts keyint=60:bframes=2 -maxrate 6000k -bufsize 12000k -shortest out.mkv

http://youtu.be/jVBYTrcC21Q

It is not perfect. Audio is bit off sync and suspect that this is due to the 300 ms capture delay. My next experiment will be to preinitialize the webcam and send its output to a local ffserver hoping that the live http stream will be faster to initialize than v4l2 webcam itself.

Please ignore the confidentiality notice below.
It is automatically added without my consent.


________________________________
CONFIDENTIALITY : This e-mail and any attachments are confidential and may be privileged. If you are not a named recipient, please notify the sender immediately and do not disclose the contents to another person, use it for any purpose or store or copy the information in any medium.