[FFmpeg-devel] [RFC] libavfilter audio API and related issues

[S.N. Hemanth Meenakshisundaram]

On 06/15/2010 04:00 PM, Stefano Sabatini wrote:
> On date Sunday 2010-06-13 19:20:55 -0700, S.N. Hemanth Meenakshisundaram encoded:
>>>
>>> Hi All,
>>>
>>> Attached is a rough draft of the audio API. [...]
>>>        
>> Hi All,
>>
>> I now have a working audio filter framework. [...]
>> Here are the changes and comments incorporated:
>>
>> 1. AVFilterSamplesRef now has a pts field as suggested. I can't find
>> a pos field for audio in existing code so I haven't added that yet.
>>      
> Maybe pts should be moved to AVFilterBuffer, same for pos, see also
> the Michael comment.
>    

Will do.


>> [...]
>>
>> Some questions:
>>
>> 1. Audio decode frame uses parameters like num_channels, sample_fmt
>> etc from 'is->audio_st->codec' (the audio codec context). That is
>> also where I read these parameters and put them in the audio buffers
>> given to the filter chain. So if these parameters change between
>> audio frames, will the new values reflect in the codec context?
>>      
> Do you mean if they should be set again in the encoder? I don't have a
> deep knowledge of lavc so I cannot say if this is effectively
> supported, I remember that changing encoding parameter on the fly is
> not supported (in that case you need to re-set the encoder).
>    

I was thinking about what the decoder does when decoding audio streams 
that have varying number of channels etc. For example, Michael mentioned 
earlier that there are streams of TV shows where the audio goes mono 
during ad segments. When decoding such a stream, will the decoder update 
the new channel_layout/number of channels parameter etc in the codec 
context so that I can read and populate into the SamplesRef structures?

>    
>> 2. I can't find a variable indicating whether audio data is planar
>> or packed. Is there one? I am assuming it is always packed right
>> now. How can I find out whether a frame returned by
>> audio_decode_frame is planar or packed?
>>      
> I suppose they're always packed, but I really don't remember, that's
> for sure this should be clearly documented. Also I'd like to move the
> sample definitions to lavu (samplefmt.h/sampledesc.h?), like it is
> done for the pixel formats, this would avoid a compile-time dependency
> of lavfi in lavc.
>    

I will add this and the channel layout descriptions to lavu.

>
>    
>> 3. The audio decoder's codec context has two variables num_channels
>> and channel_layout. num_channels is the one currently used by
>> audio_decode_frame when calculating number of samples in a frame
>> etc. Will the decoders all populate channel_layout as well
>> correctly?
>>      
> Channel layout has been introduced recently, so expect some problem
> with it (it may be left unset), but again I leave to reply someone
> with a better lavc knowledge.
>
>    
>> 4. Currently, audio_decode_frame always outputs frames with a 16-bit
>> sample size. If the codec output is in a different format, it calls
>> a conversion function. I guess the plan is to eventually remove this
>> and replace it with an output filter that can output in any required
>> sample format?
>>      
> Yes.
>    
>

I'll fix the code as per comments and start off with i. volume filter 
(for testing the framework some more) and ii. resample filter (since it 
seems a fundamental operation required in many places).

Cheers,
Hemanth