[FFmpeg-devel] MPEG-PS demuxer index memory usage

[Michael Niedermayer]

On Fri, Jan 04, 2008 at 06:11:35PM +0000, Paul Kelly wrote:
> On Fri, 4 Jan 2008, Michael Niedermayer wrote:
>
>> On Fri, Jan 04, 2008 at 05:50:01PM +0100, Michel Bardiaux wrote:
>>> Michael Niedermayer a ?crit :
>>>> On Thu, Jan 03, 2008 at 10:48:49PM +0000, Paul Kelly wrote:
> [...]
>>>>> Might it be a good idea to add another flag to turn off the
>>>>> demuxer-specific indexing, and make individual demuxers respect this? A
>>>>> general catch-all way of disabling indexing (or specifying that seeking
>>>>> isn't required) might be more elegant though.
>>>>>
>>>>> I can get over the immediate problem by simply commenting out the line
>>>>> calling av_add_index_entry() in libavformat/mpeg.c, but would like to 
>>>>> help
>>>>> get a better solution into libavformat if I can.
>>>>
>>>> Disabling it with a flag is surely interresting. But i think there are 
>>>> better
>>>> solutions.
>>>> One for example would be a max_index_size. And when thats reached index
>>>> entries would be pseudo randomly droped. That would limit the used 
>>>> memory and
>>>> still speed up seeking.
>>>>
>>> Another possibility (not exclusive):
>>>
>>> if(!url_is_streamed(s->pb)) av_add_index_entry(...)
>>>
>>> After all, if the input is streamed, the index is unlikely to be of any 
>>> use!
>>
>> patch welcome
>
> Wouldn't it be best to make whatever changes are decided on to 
> av_add_index_entry() directly, to avoid code duplication adding it to the 
> individual demuxers that call av_add_index_entry() themselves?
>
> I'm currently working on a patch for the max_index_size suggestion as I 
> thought it sounded like a good versatile solution. Am I correct in thinking 
> the max_index_size member should be added to the AVStream struct, seeing 
> there is a separate index for each stream? I notice though in 
> AVFormatContext there is "int index_built;" which looks vaguely relevant 
> but doesn't seem to be used by anything at all.
>
> While looking at av_add_index_entry() in depth I realised the current size 
> *is* actually limited, to UINT_MAX bytes but in general I guess you'd 
> probably run out of memory before hitting that limit. To keep the same 
> behaviour by default though AVStream->max_index_size could be initialised 
> to UINT_MAX. Perhaps it could also be automatically set to 0 if a streamed 
> source is being used??
>
> I also need to think of a good way of picking a random index entry to be 
> removed, but will look into it.

One example, drop every 2nd entry if the limit is reached.

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Observe your enemies, for they first find out your faults. -- Antisthenes
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20080104/df6e662e/attachment.pgp>