[FFmpeg-devel] MPEG-PS demuxer index memory usage

[Paul Kelly]

On Fri, 4 Jan 2008, Michael Niedermayer wrote:
> On Fri, Jan 04, 2008 at 06:11:35PM +0000, Paul Kelly wrote:
>>> On Fri, Jan 04, 2008 at 05:50:01PM +0100, Michel Bardiaux wrote:
>>>> Michael Niedermayer a ?crit :
>>>>> On Thu, Jan 03, 2008 at 10:48:49PM +0000, Paul Kelly wrote:
>> [...]
>>>>>> Might it be a good idea to add another flag to turn off the
>>>>>> demuxer-specific indexing, and make individual demuxers respect this? A
>>>>>> general catch-all way of disabling indexing (or specifying that seeking
>>>>>> isn't required) might be more elegant though.
>>>>>>
>>>>> Disabling it with a flag is surely interresting. But i think there are
>>>>> better
>>>>> solutions.
>>>>> One for example would be a max_index_size. And when thats reached index
>>>>> entries would be pseudo randomly droped. That would limit the used
>>>>> memory and
>>>>> still speed up seeking.
>>>>>
>>>> Another possibility (not exclusive):
>>>>
>>>> if(!url_is_streamed(s->pb)) av_add_index_entry(...)
>>>>
>>
>> I'm currently working on a patch for the max_index_size suggestion as I
>> thought it sounded like a good versatile solution. Am I correct in thinking
>> the max_index_size member should be added to the AVStream struct, seeing
>> there is a separate index for each stream? I notice though in
>> AVFormatContext there is "int index_built;" which looks vaguely relevant
>> but doesn't seem to be used by anything at all.
>>
>> While looking at av_add_index_entry() in depth I realised the current size
>> *is* actually limited, to UINT_MAX bytes but in general I guess you'd
>> probably run out of memory before hitting that limit. To keep the same
>> behaviour by default though AVStream->max_index_size could be initialised
>> to UINT_MAX. Perhaps it could also be automatically set to 0 if a streamed
>> source is being used??
>>
>> I also need to think of a good way of picking a random index entry to be
>> removed, but will look into it.
>
> One example, drop every 2nd entry if the limit is reached.

OK, first attempt at a patch is attached. Is the general idea OK? I'm not 
sure where the check for !url_is_streamed() should go so I haven't 
included it - possibly in av_new_stream()?

Paul
-------------- next part --------------
A non-text attachment was scrubbed...
Name: max_index_size.diff
Type: text/x-diff
Size: 1761 bytes
Desc: 
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20080104/2a9aad7b/attachment.diff>