[Ffmpeg-devel] Interleaving audio and video

[Baptiste Coudurier]

Michael Niedermayer wrote:
> Hi
> 
> On Fri, Feb 02, 2007 at 12:34:10PM +0100, Baptiste Coudurier wrote:
>> Michael Niedermayer wrote:
>>> Hi
>>>
>>> On Thu, Feb 01, 2007 at 09:57:15AM +0100, Baptiste Coudurier wrote:
>>>> Michael Niedermayer wrote:
>>>>> Hi
>>>>>
>>>>> On Wed, Jan 31, 2007 at 08:10:28PM -0800, Roman Shaposhnik wrote:
>>>>>> While implementing an idea suggested by Michael for restructuring DV 
>>>>>> audio muxer I've come to realize that DV muxer currently expects a 
>>>>>> caller to take care of interleaving audio and video properly based
>>>>>> on PTS valued. Now we know that ffmpeg.c does it, but is it fair
>>>>>> to require that kind of behaviour in general of all users of 
>>>>>> libavformat ?
>>>>>>
>>>>>> Or to put it differently, do I have to care about caller giving 
>>>>>> me the following pattern of DV audio/video packets:
>>>>> see av_interleaved_write_frame() (the docs above it)
>>>>> and AVOutputFormat.interleave_packet()
>>>>>
>>>> Would it be same to force using interleave_packet for some formats ?
>>>>
>>>> DV,GXF,MXF always require audio to be specially packed, and therefore
>>>> won't support passing 48000 samples packet like MOV can contain.
>>> interleave_packet() wasnt designed to split and merge packets which seems
>>> to be what you want?
>> well, when I coded gxf muxer, which needs 65536 samples per packet, you
>> suggested me to use interlave_packet, I assumed that it was the correct
>> way to interleave samples.
> 
> hmm, that must have been a missunderstanding, sorry, anyway i see no
> serious problem with using interleave_packet() to chopup and merge PCM
> packets its just not designed for that
> 
> 
>>> and about huge PCM packets, these should be never generated (=the demuxer
>>> should split them) the reason is simply memory requiremnt, audio buffering
>>> issues for playback and seeking granularity if the pcm is streamcopied
>>> if 48k samples is huge or not is another question, 10mb audio packets
>>> definitly are to large though and yes files which such audio packets
>>> exist
>> Maybe lavf need AVPacketizer ?
>>
>> MXF specs enforce pcm audio samples packets at 48khz to
>> 2000/1920/1601,2,1 for 24/25/29.97 fps, and so on, which ensures
>> perfect(almost for NTSC) sync.
>>
>> That should be also perfect for other formats, there is no framer for
>> pcm atm, what do you suggests ?
>>
>> Add option to ffmpeg.c ? Set enc->frame_size to 1920 or so, though that
>> will perturb mov muxer for example which expect frame_size to 1 for pcm.
> 
> if mxf/gxf are the only containers havig such odd limitations then the code
> to deal with it should be in them (its already there i assume and so its
> the least work and seems simplest)
> 
> also keep in mind that in reality audio and video come from different
> hardware and their sampling intervals are not synchronized so the idea
> of X audio samples per video sample is broken by design, its not a
> problem in reality as audio can be resampled video frames can be droped
> or duplicated but if your goal is good quality then such a limitation is
> unaccpetable, iam really curious why formats like that are being designed
> while much simpler, more advanced and less retarded formats already 
> exist

Which ones if you would name them ? Im curious, and audio framing is
always a serious problem in every workflow.

-- 
Baptiste COUDURIER                              GnuPG Key Id: 0x5C1ABAAA
SMARTJOG S.A.                                    http://www.smartjog.com
Key fingerprint                 8D77134D20CC9220201FC5DB0AC9325C5C1ABAAA
Phone: +33 1 49966312