[MPlayer-dev-eng] Serious memory exhaustion when playing H264 by ffh264vda decoder

[Reimar Döffinger]

On Tue, Nov 19, 2013 at 08:54:32AM +0800, Xidorn Quan wrote:
> On Tue, Nov 19, 2013 at 5:38 AM, Reimar Döffinger
> <Reimar.Doeffinger at gmx.de> wrote:
> > On 18.11.2013, at 02:04, Xidorn Quan <quanxunzhen at gmail.com> wrote:
> >> @reimar, I reviewed your commit r36422, the one mentioned by Zongyao.
> >> I found that such code (overrides buffer functions when do_dr1 is
> >> false) didn't exist in the old code at all, so I think there might be
> >> some consideration. What did you think then? Besides, I think we have
> >> to avoid any change in buffer functions (and any parameters might have
> >> been used by avcodec) after avcodec_open2. I don't have an idea what
> >> to do now since the code in vd_ffmpeg seems to be fragile. Any
> >> suggestion?
> >
> > I think mostly that h264_vda should not expose its internal functions in the avctx.
> > Next I think a good workaround would be not to touch these fields if CODEC_CAP_DR1 isn't set.
> > Lastly I don't think the code is particularly fragile. If the code used the values set before open that should still be fine, it just will cause a few additional warnings.
> > The problem is that h264_vda seems to override the values, that is IMO an API violation, the documentation says that the user sets them, and nothing about libavcodec setting them.
> > This would break things even if some user just _calls_ avctx.get_buffer themselves.
> 
> OK, that sounds reasonable. I'll fix the problem on the ffmpeg side.

Thanks, I think that would be good, but I changed the MPlayer code to avoid the
issue from that side as well (untested though).