[FFmpeg-user] USB Camera ffplay real-time and save to file

[Hans Carlson]

I have a cheap USB pipe inspection camera and I want to see the output 
(via ffplay) real-time, but also record (via ffmpeg).  I seem to be able 
to do both separately, but can't get them both to work at the same time 
using the tee muxer.  Well, it does work, but the ffplay output isn't 
real-time, there's maybe 1/2 to 1 second delay between the time I move the 
camera and when I see the results in the ffplay window.

Is there any way to use ffplay and ffmpeg at the same time such that the 
ffplay output is real-time and the video is recorded?

I don't care what output format to use for the saved file other than I 
want to save as much detail as possible.  I can post process it later if 
necessary.  This is why I'm using "copy".  I also don't care which output 
file type to use, so if there's something better than matroska that's 
fine.  Likewise for the input format for ffplay, I'm only using "nut" 
because I found an example that used "nut" and it seems to work.

I've included the full output for the "tee" version (below), but can 
provide output for the separate ffmpeg and ffplay commands if necessary.

Here's some details on the camera:

   $ v4l2-ctl --list-devices
   SheKar Camera: SheKar Camera (usb-0000:00:1a.0-1.2.4):
 	/dev/video3
 	/dev/video4

   Integrated Camera: Integrated C (usb-0000:00:1a.0-1.6):
 	/dev/video0
 	/dev/video1

   $ ffmpeg -hide_banner -f v4l2 -list_formats all -i /dev/video3
   [video4linux2,v4l2 @ 0x3ef6c80] Compressed:       mjpeg :          Motion-JPEG : 1280x720 640x480 1600x1200 1600x900 1024x768 800x600 320x240
   [video4linux2,v4l2 @ 0x3ef6c80] Raw       :     yuyv422 :           YUYV 4:2:2 : 1280x720 640x480 1600x1200 1600x900 1024x768 800x600 320x240
   /dev/video3: Immediate exit requested

If I use ffplay by itself, I can see the camera just fine and the video 
response is very fast.  In other words, if I move the camera, the ffplay 
video image responds real-time (or very, very close to real-time).

   $ ffplay -f v4l2 -input_format mjpeg -framerate 30 -video_size 1280x720 -i /dev/video3

If I use ffmpeg by itself, I can record the video, but obviously I can't 
see the video real-time.

   $ ffmpeg -f v4l2 -input_format mjpeg -framerate 30 -video_size 1280x720 -i /dev/video3 -map 0 -c:v copy output.mkv

When I combine ffmpeg and ffplay using the "tee" muxer, it works, but the 
ffplay output is delayed by 1/2 to 1 second.

   $ ffmpeg -f v4l2 -input_format mjpeg -framerate 30 -video_size 1280x720 -i /dev/video3 -map 0 -c:v copy -f tee "output.mkv|[f=nut]pipe:" | ffplay pipe:
   ffplay version N-99076-gb0203fa72b Copyright (c) 2003-2020 the FFmpeg developers
     built with gcc 9 (GCC)
     configuration: --enable-gpl --enable-version3 --enable-nonfree --enable-static --disable-shared --disable-debug --enable-libass --enable-libfdk-aac --enable-libfontconfig --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-openssl --enable-libbluray --enable-libcdio --enable-libtesseract
     libavutil      56. 58.100 / 56. 58.100
     libavcodec     58.105.100 / 58.105.100
     libavformat    58. 54.100 / 58. 54.100
     libavdevice    58. 11.101 / 58. 11.101
   ffmpeg version N-99076-gb0203fa72b  libavfilter     7. 87.100 /  7. 87.100
    Copyright (c) 2000-2020 the FFmpeg developers
     libswscale      5.  8.100 /  5.  8.100
     built with gcc 9 (GCC)
     libswresample   3.  8.100 /  3.  8.100
     configuration: --enable-gpl --enable-version3 --enable-nonfree --enable-static --disable-shared --disable-debug --enable-libass --enable-libfdk-aac --enable-libfontconfig --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-openssl --enable-libbluray --enable-libcdio --enable-libtesseract
     libpostproc    55.  8.100 / 55.  8.100
     libavutil      56. 58.100 / 56. 58.100
     libavcodec     58.105.100 / 58.105.100
     libavformat    58. 54.100 / 58. 54.100
     libavdevice    58. 11.101 / 58. 11.101
     libavfilter     7. 87.100 /  7. 87.100
     libswscale      5.  8.100 /  5.  8.100
     libswresample   3.  8.100 /  3.  8.100
     libpostproc    55.  8.100 / 55.  8.100
   [mjpeg @ 0x2c77800] unable to decode APP fields: Invalid data found when processing input
   Input #0, video4linux2,v4l2, from '/dev/video3':
     Duration: N/A, start: 1887217.220789, bitrate: N/A
       Stream #0:0: Video: mjpeg (Baseline), yuvj422p(pc, bt470bg/unknown/unknown), 1280x720, 30 fps, 30 tbr, 1000k tbn, 1000k tbc
   Output #0, tee, to 'output.mkv|[f=nut]pipe:': 0KB sq=    0B f=0/0
     Metadata:
       encoder         : Lavf58.54.100
       Stream #0:0: Video: mjpeg (Baseline), yuvj422p(pc, bt470bg/unknown/unknown), 1280x720, q=2-31, 30 fps, 30 tbr, 1000k tbn, 1000k tbc
   Stream mapping:
     Stream #0:0 -> #0:0 (copy)
   Press [q] to stop, [?] for help
   [mjpeg @ 0x7fdf24003bc0] unable to decode APP fields: Invalid data found when processing input
   Input #0, nut, from 'pipe:':aq=    0KB vq=    0KB sq=    0B f=0/0   ed=1.11x
     Metadata:
       encoder         : Lavf58.54.100
     Duration: N/A, start: 0.000000, bitrate: N/A
       Stream #0:0: Video: mjpeg (Baseline) (MJPG / 0x47504A4D), yuvj422p(pc, bt470bg/unknown/unknown), 1280x720, 30 fps, 30 tbr, 1000k tbn, 1000k tbc
   [mjpeg @ 0x7fdf24005840] unable to decode APP fields: Invalid data found when processing input
   [swscaler @ 0x7fdf1c1f4cc0] deprecated pixel format used, make sure you did set range correctly
   [mjpeg @ 0x7fdf24005840] unable to decode APP fields: Invalid data found when processing input
       Last message repeated 2 times
   [mjpeg @ 0x7fdf24005840] unable to decode APP fields: Invalid data found when processing input
     ... (repeats several times/sec)
   [mjpeg @ 0x7fdf24005840] unable to decode APP fields: Invalid data found when processing input
   frame=  229 fps= 25 q=-1.0 Lsize=N/A time=00:00:09.32 bitrate=N/A speed=1.01x
   video:25291kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
   Exiting normally, received signal 2.