[FFmpeg-devel] MPEG-PS demuxer index memory usage

[Michael Niedermayer]

On Fri, Jan 04, 2008 at 05:50:01PM +0100, Michel Bardiaux wrote:
> Michael Niedermayer a ?crit :
> > On Thu, Jan 03, 2008 at 10:48:49PM +0000, Paul Kelly wrote:
> >> Hello
> >> I'm using libavformat to demux a continuous stream of MPEG2-PS data and am 
> >> running into the problem that memory usage steadily increases over time. 
> >> The problem is not observed when demuxing an MPEG2-TS stream.
> >>
> >> After a bit of digging around I discovered the problem is caused by the 
> >> timestamp indexing in the PS demuxer - specifically, the calls to 
> >> av_add_index_entry() in mpegps_read_pes_header() in libavformat/mpeg.c. 
> >> All I'm doing is transcoding the stream to a different output format and I 
> >> don't need to be able to perform seeking but there doesn't seem to be any 
> >> way to disable the index. (I guess the memory occupied by the index isn't 
> >> a problem if a fixed-size file is being demuxed, but in my case I am 
> >> reading data from a hardware MPEG encoder card and splitting the output 
> >> into separate files and the process is required to run indefinitely - the 
> >> index quickly grows to an unwieldy size.)
> >>
> >> As far as I can see the flag AVFMT_GENERIC_INDEX can be turned off to stop 
> >> indexing if generic indexing is used (perhaps that's a non-standard usage 
> >> though) - but is there no way to turn off the indexing in the MPEG-PS 
> >> demuxer?
> >>
> >> Might it be a good idea to add another flag to turn off the 
> >> demuxer-specific indexing, and make individual demuxers respect this? A 
> >> general catch-all way of disabling indexing (or specifying that seeking 
> >> isn't required) might be more elegant though.
> >>
> >> I can get over the immediate problem by simply commenting out the line 
> >> calling av_add_index_entry() in libavformat/mpeg.c, but would like to help 
> >> get a better solution into libavformat if I can.
> > 
> > Disabling it with a flag is surely interresting. But i think there are better
> > solutions.
> > One for example would be a max_index_size. And when thats reached index
> > entries would be pseudo randomly droped. That would limit the used memory and
> > still speed up seeking.
> > 
> Another possibility (not exclusive):
> 
> if(!url_is_streamed(s->pb)) av_add_index_entry(...)
> 
> After all, if the input is streamed, the index is unlikely to be of any use!

patch welcome

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

I have often repented speaking, but never of holding my tongue.
-- Xenocrates
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20080104/09f9d2ee/attachment.pgp>