[FFmpeg-user] Average N frames of RTSP video for better snapshot quality?

[Steven Kan]

> On Mar 8, 2022, at 10:32 AM, Michael Koch <astroelectronic at t-online.de> wrote:
> 
> Am 08.03.2022 um 19:09 schrieb Steven Kan:
>> After 7.5 years of waiting, my banana plant is finally flowering! I want to do a time-lapse capture of the flowering and fruiting process. Due to its location, the easiest way for me to get a camera out there is to use a little WyzeCam v3 with the RTSP firmware and the Wyze lamp socket. Unfortunately the WyzeCam doesn’t (yet) have a externally accessible JPG snapshot feature, so I have a cron job set up to:
>> 
>> ./ffmpeg -rtsp_transport tcp -i rtsp://anonymous:password@$IPAddress/live -frames:v 1 $outfile
>> 
>> every hour. The results are OK, but not fantastic:
>> 
>> https://www.kan.org/pictures/BananaTimeLapseFirstImage.jpg <https://www.kan.org/pictures/BananaTimeLapseFirstImage.jpg>
>> 
>> Is there a way to tell ffmpeg to collect N frames of video and output one single averaged image to improve the SNR? Even if there’s some wind, the flower stalk shouldn’t be moving much.
>> 
>> I tried:
>> 
>> ./ffmpeg -rtsp_transport tcp -i rtsp://anonymous:password@192.168.1.39/live -frames:v 10 ~/BananaLapse/MultiFrame%03d.jpg
>> 
>> and that results in N JPGs. I suppose I could have a second ffmpeg command that averages those 10 JPGs, but can this all be done in one pass? Thanks!
> 
> You can use the "tmix" filter before you extract the images from the video.
> 
> Michael

Thanks! Can I get a little help on the syntax? Right now it’s still expecting to output multiple images:

./ffmpeg -rtsp_transport tcp -i rtsp://anonymous:password@192.168.1.39/live -frames:v 10 -vf tmix=frames=10:weights="1" ~/BananaLapse/MultiFrame.jpg
ffmpeg version N-102535-g6ff2aba088-tessus  https://evermeet.cx/ffmpeg/  Copyright (c) 2000-2021 the FFmpeg developers
  built with Apple clang version 11.0.0 (clang-1100.0.33.17)
  configuration: --cc=/usr/bin/clang --prefix=/opt/ffmpeg --extra-version=tessus --enable-avisynth --enable-fontconfig --enable-gpl --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libfreetype --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvmaf --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-version3 --pkg-config-flags=--static --disable-ffplay
  libavutil      57.  0.100 / 57.  0.100
  libavcodec     59.  1.100 / 59.  1.100
  libavformat    59.  2.100 / 59.  2.100
  libavdevice    59.  0.100 / 59.  0.100
  libavfilter     8.  0.101 /  8.  0.101
  libswscale      6.  0.100 /  6.  0.100
  libswresample   4.  0.100 /  4.  0.100
  libpostproc    56.  0.100 / 56.  0.100
Guessed Channel Layout for Input Stream #0.1 : mono
Input #0, rtsp, from 'rtsp://anonymous:password@192.168.1.39/live':
  Metadata:
    title           : Session streamed by the WYZE Media Server
    comment         : live
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: h264 (Main), yuv420p(tv, bt709, progressive), 1920x1080, 20 fps, 20 tbr, 90k tbn
  Stream #0:1: Audio: pcm_alaw, 16000 Hz, mono, s16, 128 kb/s
File '/Users/steven/BananaLapse/MultiFrame.jpg' already exists. Overwrite? [y/N] y
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (native))
Press [q] to stop, [?] for help
[swscaler @ 0x10e418000] deprecated pixel format used, make sure you did set range correctly
Output #0, image2, to '/Users/steven/BananaLapse/MultiFrame.jpg':
  Metadata:
    title           : Session streamed by the WYZE Media Server
    comment         : live
    encoder         : Lavf59.2.100
  Stream #0:0: Video: mjpeg, yuvj420p(pc, progressive), 1920x1080, q=2-31, 200 kb/s, 20 fps, 20 tbn
    Metadata:
      encoder         : Lavc59.1.100 mjpeg
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A
[image2 @ 0x7fa809c0adc0] Could not get frame filename number 2 from pattern '/Users/steven/BananaLapse/MultiFrame.jpg'. Use '-frames:v 1' for a single image, or '-update' option, or use a pattern such as %03d within the filename.
av_interleaved_write_frame(): Invalid argument
[image2 @ 0x7fa809c0adc0] Could not get frame filename number 2 from pattern '/Users/steven/BananaLapse/MultiFrame.jpg'. Use '-frames:v 1' for a single image, or '-update' option, or use a pattern such as %03d within the filename.
av_interleaved_write_frame(): Invalid argument
[image2 @ 0x7fa809c0adc0] Could not get frame filename number 2 from pattern '/Users/steven/BananaLapse/MultiFrame.jpg'. Use '-frames:v 1' for a single image, or '-update' option, or use a pattern such as %03d within the filename.
av_interleaved_write_frame(): Invalid argument
[image2 @ 0x7fa809c0adc0] Could not get frame filename number 2 from pattern '/Users/steven/BananaLapse/MultiFrame.jpg'. Use '-frames:v 1' for a single image, or '-update' option, or use a pattern such as %03d within the filename.
av_interleaved_write_frame(): Invalid argument
[image2 @ 0x7fa809c0adc0] Could not get frame filename number 2 from pattern '/Users/steven/BananaLapse/MultiFrame.jpg'. Use '-frames:v 1' for a single image, or '-update' option, or use a pattern such as %03d within the filename.
av_interleaved_write_frame(): Invalid argument
[image2 @ 0x7fa809c0adc0] Could not get frame filename number 2 from pattern '/Users/steven/BananaLapse/MultiFrame.jpg'. Use '-frames:v 1' for a single image, or '-update' option, or use a pattern such as %03d within the filename.
av_interleaved_write_frame(): Invalid argument
[image2 @ 0x7fa809c0adc0] Could not get frame filename number 2 from pattern '/Users/steven/BananaLapse/MultiFrame.jpg'. Use '-frames:v 1' for a single image, or '-update' option, or use a pattern such as %03d within the filename.
av_interleaved_write_frame(): Invalid argument
[image2 @ 0x7fa809c0adc0] Could not get frame filename number 2 from pattern '/Users/steven/BananaLapse/MultiFrame.jpg'. Use '-frames:v 1' for a single image, or '-update' option, or use a pattern such as %03d within the filename.
av_interleaved_write_frame(): Invalid argument
[image2 @ 0x7fa809c0adc0] Could not get frame filename number 2 from pattern '/Users/steven/BananaLapse/MultiFrame.jpg'. Use '-frames:v 1' for a single image, or '-update' option, or use a pattern such as %03d within the filename.
av_interleaved_write_frame(): Invalid argument
frame=   10 fps=0.0 q=24.8 Lsize=N/A time=00:00:00.50 bitrate=N/A dup=10 drop=1 speed=2.27x    
video:1549kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Conversion failed!