[FFmpeg-user] How to cope with slow to open v4l2 devices

[LANGLOIS Olivier PIS -EXT]

Hi,

My v4l2 input is very slow (it is measured in seconds) to open and my other video input is a live feed produced by a process that will just bail out if ffmpeg is not able to consume the raw stream in real-time.

The whole thing choke right at the beginning when ffmpeg opens the v4l2 device.
My live video producer determine that ffmpeg is too slow and close its side of the pipe.
ALSA generates buffer overruns.

What is the usual strategy to deal with several live inputs with different initialization times?

/dev/video0 is a old Logitech webcam. I'm not ruling the possibility to buy a faster device if it can solve my problem.
I also have full control over the live feed video producer I am just not sure how to modify it to cope with the problem.
(https://github.com/lano1106/glcs)

I could make it drop frames but then how will it discern between a one time initialization slow down vs a permanent problem where the encoder is just too slow to process the stream in realtime?

I do not like too much this idea as I wonder how I will ensure synchronization between the different streams.

2. If I was reordering the input declaration order from the slowest device to the fastest on the cmdline, could it help?

3. Can ffmpeg report back when its initialization is done and when it is ready to accept data by socket and/or pipe?

Here is my super ffmpeg cmdline:

ffmpeg -nostats -y \
 -f rawvideo -video_size 1920x1080 -pixel_format bgra -framerate 30 -i /dev/stdin \
 -f v4l2 -input_format mjpeg -video_size 320x240 -framerate 30 -i /dev/video0 \
 -f alsa -acodec pcm_s16le -ar 44100 -ac 2 -i loop_capture \
 -f alsa -acodec pcm_s16le -ar 44100 -ac 2 -i sysdefault:1 \
 -filter_complex "overlay;[3:a]lowpass=f=4000[lp_out];[2:a][lp_out]amix" \
 -c:a libfdk_aac -profile:a aac_low -b:a 128k -ar 44100 \
 -c:v libx264 -preset superfast -profile:v main -level 4.1 -pix_fmt yuv420p \
 -x264opts keyint=60:bframes=2 -maxrate 6000k -bufsize 12000k -shortest out.mkv

Something that I missed a lot during my debugging was timestamps in the log msgs. I did create a small & dirty patch. I do not know if there would be interest from the devs to add it....:

--- ffmpeg-2.1.4/libavutil/log.c.orig   2014-03-17 16:41:42.157039924 -0400
+++ ffmpeg-2.1.4/libavutil/log.c        2014-03-17 16:45:01.177223972 -0400
@@ -32,6 +32,7 @@
 #if HAVE_IO_H
 #include <io.h>
 #endif
+#include <time.h>
 #include <stdarg.h>
 #include <stdlib.h>
 #include "avutil.h"
@@ -226,6 +227,7 @@
     char line[LINE_SZ];
     static int is_atty;
     int type[2];
+    struct timespec ts;

     if (level > av_log_level)
         return;
@@ -253,6 +255,8 @@
         count = 0;
     }
     strcpy(prev, line);
+    clock_gettime(CLOCK_MONOTONIC, &ts);
+    fprintf(stderr, "[%d.%03d] ", ts.tv_sec, ts.tv_nsec/1000000);
     sanitize(part[0].str);
     colored_fputs(type[0], part[0].str);
     sanitize(part[1].str);

With this patch, I have the following output (with a different webcam but this demonstrate that v4l2 devices are very slow to start):

ffmpeg -y -loglevel debug -f v4l2 -input_format raw -input_format yuyv422 -video_size 320x240 -framerate 30 -i /dev/video0 -c:v libx264 -preset ultrafast out.mkv

[23626.939] Opening an input file: /dev/video0.
[23626.978] [video4linux2,v4l2 @ 0x955b300] fd:3 capabilities:84000001
[23626.979] [video4linux2,v4l2 @ 0x955b300] Current input_channel: 0, input_name: Camera 1, input_std: 0
[23626.981] [video4linux2,v4l2 @ 0x955b300] Setting time per frame to 1/30
[23627.726] [video4linux2,v4l2 @ 0x955b300] All info found
[23627.726] Input #0, video4linux2,v4l2, from '/dev/video0':
[23627.727]   Duration: [23627.727] N/A[23627.727] , start: [23627.727] 23627.691989[23627.727] , bitrate: [23627.727] 36864 kb/s[23627.727]
[23627.727]     Stream #0:0[23627.728] , 1, 1/1000000[23627.728] : Video: rawvideo (YUY2 / 0x32595559), yuyv422, 320x240, 1/1000000, 36864 kb/s[23627.728] , 30 fps[23627.728] , 30 tbr[23627.728] , 1000k tbn[23627.728] , 1000k tbc[23627.728]
[23627.728] Successfully opened the file.


Please ignore the confidentiality notice below.
It is automatically added without my consent.



________________________________
CONFIDENTIALITY : This e-mail and any attachments are confidential and may be privileged. If you are not a named recipient, please notify the sender immediately and do not disclose the contents to another person, use it for any purpose or store or copy the information in any medium.