[FFmpeg-devel] [PATCH] Unbreak av_malloc_max(0) API/ABI

[Joakim Tjernlund]

On Tue, 2020-10-13 at 14:03 -0300, James Almer wrote:
> 
> 
> On 10/13/2020 11:24 AM, Derek Buitenhuis wrote:
> > On 13/10/2020 15:19, Joakim Tjernlund wrote:
> > > For now just fixing av_malloc_max(0) will do, av_max_malloc2() etc. is beyond this patch.
> > 
> > As Ronald mentioned, if you're going to fix the ABI/API, you should actually
> > fix every use of that ABI/API, not just the one you care about (0).
> > 
> > So max values of 1-31 should also be handled.
> 
> I personally don't think there's anything to fix, or if there is, it's
> not this. For starters av_max_alloc() was never changed. It was how
> av_malloc() handled the max_alloc_size variable. And then, the old
> behavior was both buggy (0 to 31 equaling SIZE_MAX or slightly smaller)
> and undocumented. I could accept reverting it for the 4.3 branch for the
> sake of ABI compatibility, but after a sobump 0-31 should not wrap
> around or be replaced by something arbitrary like INT_MAX.

Is av_max_alloc(0) useful ? I mean, can you do useful stuff with ffmpeg if max size is truly 0 ?
If not, you could consider 0 mean set it to system maxmimum(as my first patch did)

> 
> Chromium already fixed their code by requesting a SIZE_MAX limit
> (meaning effectively unlimited), which is what they wanted to begin
> with, and others should follow.

Yes, but how long will that take ? the change is already out and reverting it fully serves little purpose.
You could apply my patch and then revert it at a later time when you break API anyway.

   Jocke