[Libav-user] Muxing sparse media streams

[Nicolas George]

Le tridi 3 floréal, an CCXXIV, Anton Yabchinskiy a écrit :
> I'm trying to mux two streams (1280x720 H.264 video and 22050 Hz, mono
> PCM audio) into an AVI file. The problem is that stream data may have
> arbitrary gaps, and on input I have raw video frames/blocks of audio
> data with associated POSIX timestamps (the time at which video/audio
> was captured, in microseconds).
> 
> So, it may look something like this (where V is video data and A is
> corresponding audio data):
> 
> | VVVV  VVV   |
> |  AAA AA AAAA|
> 
> On playback there must be silence instead of missing audio data, and
> whatever (maybe, last valid frame) in place of missing video
> frames. Also, regions where were no video and no audio at the same
> time should be skipped and not muxed to the output (but I'm leaving
> this apart for now).
> 
> What could be a proper way to mux such data?

It depends on the player. libavformat will let you mux anything provided you
set consistent timestamps, but players will take the timestamps into account
in various ways.

I suspect a lot of players, when showing audio and video, sync on audio.
Therefore you will probably need to explicitly insert silence at the places
you want it.

As for the video, it should work as is, but you have to be sure the format
you chose is capable of doing variable frame rate.

Regards,

-- 
  Nicolas George
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/libav-user/attachments/20160421/41f000dd/attachment.sig>