[FFmpeg-devel] MPEG-PS demuxer index memory usage

[Michel Bardiaux]

Michael Niedermayer a ?crit :
> On Thu, Jan 03, 2008 at 10:48:49PM +0000, Paul Kelly wrote:
>> Hello
>> I'm using libavformat to demux a continuous stream of MPEG2-PS data and am 
>> running into the problem that memory usage steadily increases over time. 
>> The problem is not observed when demuxing an MPEG2-TS stream.
>>
>> After a bit of digging around I discovered the problem is caused by the 
>> timestamp indexing in the PS demuxer - specifically, the calls to 
>> av_add_index_entry() in mpegps_read_pes_header() in libavformat/mpeg.c. 
>> All I'm doing is transcoding the stream to a different output format and I 
>> don't need to be able to perform seeking but there doesn't seem to be any 
>> way to disable the index. (I guess the memory occupied by the index isn't 
>> a problem if a fixed-size file is being demuxed, but in my case I am 
>> reading data from a hardware MPEG encoder card and splitting the output 
>> into separate files and the process is required to run indefinitely - the 
>> index quickly grows to an unwieldy size.)
>>
>> As far as I can see the flag AVFMT_GENERIC_INDEX can be turned off to stop 
>> indexing if generic indexing is used (perhaps that's a non-standard usage 
>> though) - but is there no way to turn off the indexing in the MPEG-PS 
>> demuxer?
>>
>> Might it be a good idea to add another flag to turn off the 
>> demuxer-specific indexing, and make individual demuxers respect this? A 
>> general catch-all way of disabling indexing (or specifying that seeking 
>> isn't required) might be more elegant though.
>>
>> I can get over the immediate problem by simply commenting out the line 
>> calling av_add_index_entry() in libavformat/mpeg.c, but would like to help 
>> get a better solution into libavformat if I can.
> 
> Disabling it with a flag is surely interresting. But i think there are better
> solutions.
> One for example would be a max_index_size. And when thats reached index
> entries would be pseudo randomly droped. That would limit the used memory and
> still speed up seeking.
> 
Another possibility (not exclusive):

if(!url_is_streamed(s->pb)) av_add_index_entry(...)

After all, if the input is streamed, the index is unlikely to be of any use!

Season greetings,
-- 
Michel Bardiaux
R&D Director
T +32 [0] 2 790 29 41
F +32 [0] 2 790 29 02
E mailto:mbardiaux at mediaxim.be

Mediaxim NV/SA
Vorstlaan 191 Boulevard du Souverain
Brussel 1160 Bruxelles
http://www.mediaxim.com/