[FFmpeg-user] Problem with combining a video and an audio stream from USB device

[FHoevi]

Dear all,

I hope you could provide a hint to my problem: I have two USB devices
attached to an RPi, both show up as usual as /dev/video0. Here's some
additional info coming from two command line inputs:

 

Device 1, video only (attached to an RPi4):

"ffmpeg -f v4l2 -list_formats all -i /dev/video0" reports

[video4linux2,v4l2 @ 0xe5e1c0] Compressed:       mjpeg :
Motion-JPEG : 1280x720 640x480 320x240

 

"v4l2-ctl --list-formats-ext" reports

ioctl: VIDIOC_ENUM_FMT

        Type: Video Capture

 

        [0]: 'MJPG' (Motion-JPEG, compressed)

                Size: Discrete 1280x720

                        Interval: Stepwise 0.033s - 0.033s with step 0.000s
(30.000-30.000 fps)

                Size: Discrete 640x480

                        Interval: Stepwise 0.033s - 0.033s with step 0.000s
(30.000-30.000 fps)

                Size: Discrete 320x240

                        Interval: Stepwise 0.033s - 0.033s with step 0.000s
(30.000-30.000 fps)

 

Does work: ffmpeg -f v4l2 -i /dev/video0 -vcodec h264_omx -preset ultrafast
-tune zerolatency -g 300 -b:v 1M -mpegts_service_type
advanced_codec_digital_hdtv -f mpegts udp://OtherMachine:Port?pkt_size=1316

 

Device 2, video and audio (attached to an RPi3, but does not work either on
the RPi4):

"ffmpeg -f v4l2 -list_formats all -i /dev/video0" reports

[video4linux2,v4l2 @ 0x2c41210] Compressed:       mjpeg :
Motion-JPEG : 1920x1080 1280x720

 

"v4l2-ctl --list-formats-ext" reports

ioctl: VIDIOC_ENUM_FMT

               Index       : 0

               Type        : Video Capture

               Pixel Format: 'MJPG' (compressed)

               Name        : Motion-JPEG

                              Size: Discrete 1920x1080

                                            Interval: Discrete 0.033s
(30.000 fps)

                                            Interval: Discrete 0.067s
(15.000 fps)

                              Size: Discrete 1280x720

                                            Interval: Discrete 0.033s
(30.000 fps)

                                            Interval: Discrete 0.067s
(15.000 fps)

 

After quite some tedious work and way too many hours I got this running:

 

Video only: ffmpeg -f v4l2 -input_format mjpeg -i /dev/video0 -c:v copy
-preset ultrafast -tune zerolatency -g 300 -f matroska
udp://OtherMachine:Port?pkt_size=1316

Does not work at all: ffmpeg -f v4l2 -input_format mjpeg -i /dev/video0 -c:v
copy -preset ultrafast -tune zerolatency -g 300 -f mpegts
udp://OtherMachine:Port?pkt_size=1316, on "OtherMachine" I do see that there
is an incoming data stream via VLC, but it could not be digested properly.

Audio only: ffmpeg -f alsa -thread_queue_size 1024 -i plughw:1 -c:a mp2 -ac
2 -ar 44100 -preset ultrafast -tune zerolatency -b:a 128K -f mpegts
udp://OtherMachine:Port?pkt_size=1316

 

But this does not work either:

ffmpeg -f v4l2 -input_format mjpeg -i /dev/video0 -f alsa -thread_queue_size
1024 -i plughw:1 -c:v copy -c:a mp2 -ac 2 -ar 44100 -preset ultrafast -tune
zerolatency -g 300  -b:a 128K -f mpegts
udp://OtherMachine:Port?pkt_size=1316

 

Could you please provide a hint on how to get these two streams for device 2
working together? Both of them come from the same hardware/device, my guess
is that the MJPG video stream is somehow not fully compliant with the mpegts
standard (like it is for device 1) since it works with matroska, but not
with mpegts. Could that be? What needs to be done in that case?

 

Thanks in advance and kind regards

FHoevi