[FFmpeg-user] cant stream audio and video at the same time

[Gisbert Haas]

Hi,

as I tried to explain, on the link you provided I learned about the v4l2 
timestamps option "-ts abs" I'm now using:

Linux Mint Laptop:

ffmpeg -f alsa -i default -f v4l2 -input_format yuv420p -video_size 
1280x720 -ts abs -i /dev/video0 -c:v libx264 -b:v 8M -c:a aac -f mpegts  
udp://230.10.1.1:5700


Raspberry Pi 4, RasbianOS 32 bit

ffmpeg -f alsa -i default -f v4l2 -input_format yuv420p -video_size 
1280x720 -ts abs -i /dev/video0 -c:v h264_v4l2m2m -b:v 8M -c:a aac -f 
mpegts  udp://230.10.1.1:5700

only difference between the two is the video output format due to 
different architecture.


On Linux Laptop the start_time is now in sync (with -ts abs):

Input #0, alsa, from 'default':
   Duration: N/A, start: 1609881042.036804, bitrate: 1536 kb/s
     Stream #0:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s

Input #1, video4linux2,v4l2, from '/dev/video0':
   Duration: N/A, start: 1609881042.395996, bitrate: 147456 kb/s
     Stream #1:0: Video: rawvideo (YUY2 / 0x32595559), yuyv422, 
1280x720, 147456 kb/s, 10 fps, 10 tbr, 1000k tbn, 1000k tbc


On the Raspberry Pi even with the -ts abs set, the start_time did not 
change:

Input #0, alsa, from 'hw:2,0':
   Duration: N/A, start: 1609881274.125326, bitrate: 1536 kb/s
     Stream #0:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s

Input #1, video4linux2,v4l2, from '/dev/video0':
   Duration: N/A, start: 0.000000, bitrate: 884736 kb/s
     Stream #1:0: Video: rawvideo (YUY2 / 0x32595559), yuyv422, 
1280x720, 884736 kb/s, 60 fps, 60 tbr, 1000k tbn, 1000k tbc

instead I'm getting an error:

[video4linux2,v4l2 @ 0x23d0440] Unknown timestamps


with using option -ts mono2abs the time stamps are not quite the same:

Input #0, alsa, from 'default':
   Duration: N/A, start: 1609881635.974239, bitrate: 1536 kb/s
     Stream #0:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s
[video4linux2,v4l2 @ 0x1028850] Detected monotonic timestamps, converting
Input #1, video4linux2,v4l2, from '/dev/video0':
   Duration: N/A, start: 1609851045.581055, bitrate: 294912 kb/s
     Stream #1:0: Video: rawvideo (YUY2 / 0x32595559), yuyv422, 640x480, 
294912 kb/s, 60 fps, 60 tbr, 1000k tbn, 1000k tbc


I hope with the -ts option I'm using the right option as I cant find 
anything else on v4l2 would modify the start_time


Thanks a lot






On 1/5/21 8:58 PM, Carl Eugen Hoyos wrote:
> CAUTION: This email originated from outside of the organization. Do 
> not click links or open attachments unless you recognize the sender 
> and know the content is safe.
>
>
>>
>> Am 05.01.2021 um 16:05 schrieb Gisbert Haas <gisbert.haas at myvinn.com 
>> <mailto:gisbert.haas at myvinn.com>>:
>>
>> Input #0, alsa, from 'hw:2,0':
>>   Duration: N/A, start: 1609858998.181873, bitrate: 1536 kb/s
>>     Stream #0:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s
>> [video4linux2,v4l2 @ 0x32cd6e0] Dequeued v4l2 buffer contains 
>> corrupted data (0 bytes).
>> Input #1, video4linux2,v4l2, from '/dev/video0':
>>   Duration: N/A, start: 0.000000, bitrate: 884736 kb/s
>
> These streams cannot get synchronised because of the different start 
> times.
> See https://ffmpeg.org/ffmpeg-devices.html#Options-18
>
> Please find out what top-posting means and avoid it here.
> And please do not reply in private to emails sent to a mailing list.
>
> Carl Eugen