[MPlayer-dev-eng] Serious memory exhaustion when playing H264 by ffh264vda decoder

[Xidorn Quan]

On Tue, Nov 19, 2013 at 5:38 AM, Reimar Döffinger
<Reimar.Doeffinger at gmx.de> wrote:
> On 18.11.2013, at 02:04, Xidorn Quan <quanxunzhen at gmail.com> wrote:
>> @reimar, I reviewed your commit r36422, the one mentioned by Zongyao.
>> I found that such code (overrides buffer functions when do_dr1 is
>> false) didn't exist in the old code at all, so I think there might be
>> some consideration. What did you think then? Besides, I think we have
>> to avoid any change in buffer functions (and any parameters might have
>> been used by avcodec) after avcodec_open2. I don't have an idea what
>> to do now since the code in vd_ffmpeg seems to be fragile. Any
>> suggestion?
>
> I think mostly that h264_vda should not expose its internal functions in the avctx.
> Next I think a good workaround would be not to touch these fields if CODEC_CAP_DR1 isn't set.
> Lastly I don't think the code is particularly fragile. If the code used the values set before open that should still be fine, it just will cause a few additional warnings.
> The problem is that h264_vda seems to override the values, that is IMO an API violation, the documentation says that the user sets them, and nothing about libavcodec setting them.
> This would break things even if some user just _calls_ avctx.get_buffer themselves.

OK, that sounds reasonable. I'll fix the problem on the ffmpeg side.