[FFmpeg-user] Lag when streaming video+audio from RPi Zero W with Logitech C920

[Emanuele]

I've been trying to set up a baby monitor with a Raspberry Pi Zero
and a Logitech C920 webcam. It does work with VLC (cvlc) but it lags too
much and gets worse over time. 

So I am playing around with ffmpeg and
I am getting some better results. This is what I've done so far. 

First
I set the webcam to output h264 1080p natively (the Pi Zero W can't
afford to do any transcoding). 

v4l2-ctl
--set-fmt-video=width=1920,height=1080,pixelformat=1 

Now, if I stream
audio only with 

ffmpeg 
 -f alsa 
 -i hw:1,0 
 -vn 
 -flags
+global_header 
 -acodec aac 
 -ac 1 
 -ar 16000 
 -ab 16k 
 -f rtp
rtp://192.168.0.10:5002 [1] > audio.sdp 

It works great and the lag is
about 1 second (definitely acceptable). 

If I stream video only with


ffmpeg 
 -f v4l2 
 -vcodec h264 
 -i /dev/video0 
 -an 
 -vcodec copy

 -pix_fmt yuv420p 
 -r 30 
 -b:v 512k 
 -flags +global_header 
 -f rtp
rtp://192.168.0.10:5000 [2] > video.sdp 

same result, very little lag
(for some reason the first -vcodec is necessary to force the webcam to
output h264). 

However, when I stream both with 

ffmpeg 
 -f v4l2 

-vcodec h264 
 -i /dev/video0 
 -f alsa 
 -i hw:1,0 
 -an 
 -preset
ultrafast 
 -tune zerolatency 
 -vcodec copy 
 -pix_fmt yuv420p 
 -r 30

 -b:v 512k 
 -flags +global_header 
 -f rtp rtp://192.168.0.10:5000 [3]

 -vn 
 -flags +global_header 
 -acodec aac 
 -ac 1 
 -ar 16000 
 -ab
16k 
 -f rtp rtp://192.168.0.10:5002 [4] > both.sdp 

The lag ramps up
to ~10 seconds and audio and video are out of sync. Does anybody know
why? 

I've tried UDP and TCP instead of RTP but then the lag is ALWAYS
high, even with audio/video only. 

Any suggestion is much appreciated.


P.S. On the client side (MacOS) I'm receiving with 

ffplay
-protocol_whitelist file,rtp,udp -i file.sdp
  


Con Tiscali Mobile Smart 30 4G hai minuti illimitati, 100 SMS e 30 Giga in 4G a soli 8,99€ al mese. http://tisca.li/smart30