[FFmpeg-devel] [PATCH v4] Unbreak av_malloc_max(0) API/ABI

[Joakim Tjernlund]

Pretty please ?

 Jocke

On Thu, 2020-10-29 at 13:46 +0000, Joakim Tjernlund wrote:
> 
> Ping ..
> 
> On Tue, 2020-10-27 at 08:48 +0000, Joakim Tjernlund wrote:
> > 
> > Yet a ping ...
> > 
> > This is a simple technical patch, it just needs a policy decision.
> > Can I have one ?
> > 
> >    Jocke
> > 
> > On Thu, 2020-10-22 at 14:17 +0200, Joakim Tjernlund wrote:
> > > Ping ?
> > > 
> > >  Jocke
> > > 
> > > On Fri, 2020-10-16 at 10:57 +0200, Joakim Tjernlund wrote:
> > > > From https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugs.chromium.org%2Fp%2Fchromium%2Fissues%2Fdetail%3Fid%3D1095962&data=04%7C01%7Cjoakim.tjernlund%40infinera.com%7C099e7246a65a480ea5d808d87c111026%7C285643de5f5b4b03a1530ae2dc8aaf77%7C1%7C0%7C637395760010458386%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=u%2Fr4rM4fQkwS%2Buj9D%2BF%2FJsxy1nUJBWRnrNQdn71kdhw%3D&reserved=0
> > > > ----------------------------
> > > > This seems to be caused by the custom handling of "av_max_alloc(0)" in
> > > > Chromium's ffmpeg fork to mean unlimited (added in [1]).
> > > > 
> > > > Upstream ffmpeg doesn't treat 0 as a special value; versions before 4.3 seemingly worked
> > > > because 32 was subtracted from max_alloc_size (set to 0 by Chromium) resulting in an
> > > > integer underflow, making the effective limit be SIZE_MAX - 31.
> > > > 
> > > > Now that the above underflow doesn't happen, the tab just crashes. The upstream change
> > > > for no longer subtracting 32 from max_alloc_size was included in ffmpeg 4.3. [2]
> > > > 
> > > > [1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fchromium-review.googlesource.com%2Fc%2Fchromium%2Fthird_party%2Fffmpeg%2F%2B%2F73563&data=04%7C01%7Cjoakim.tjernlund%40infinera.com%7C099e7246a65a480ea5d808d87c111026%7C285643de5f5b4b03a1530ae2dc8aaf77%7C1%7C0%7C637395760010458386%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=qLA%2Bda2ge5hh0AZx6xKfOBPMD3iDkfiJdiyij2fB6y0%3D&reserved=0
> > > > [2] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FFFmpeg%2FFFmpeg%2Fcommit%2F731c77589841&data=04%7C01%7Cjoakim.tjernlund%40infinera.com%7C099e7246a65a480ea5d808d87c111026%7C285643de5f5b4b03a1530ae2dc8aaf77%7C1%7C0%7C637395760010458386%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=jLpWoZ%2BFPIMr8Sw56eyypQre65T9tJEzGLgRSquTUPI%3D&reserved=0
> > > > ---------------------------
> > > > 
> > > > Restore av_malloc_max(0) to MAX_INT fixing MS Teams, Discord older chromium etc.
> > > > 
> > > > Signed-off-by: Joakim Tjernlund <joakim.tjernlund at infinera.com>
> > > > ---
> > > > 
> > > >  v2: Cover the full API range 0-31
> > > > 
> > > >  v3: Closer compat with < 4.3 ffmpeg
> > > > 
> > > >  v4: Adjust size accoriding to Andreas Rheinhardt comments
> > > > 
> > > >  libavutil/mem.c | 2 ++
> > > >  1 file changed, 2 insertions(+)
> > > > 
> > > > diff --git a/libavutil/mem.c b/libavutil/mem.c
> > > > index cfb6d8a..44870a9 100644
> > > > --- a/libavutil/mem.c
> > > > +++ b/libavutil/mem.c
> > > > @@ -71,6 +71,8 @@ void  free(void *ptr);
> > > >  static size_t max_alloc_size= INT_MAX;
> > > > 
> > > > 
> > > >  void av_max_alloc(size_t max){
> > > > +    if (max < 32)
> > > > +        max = SIZE_MAX - 32 + max; /* be compatible to older(< 4.3) versions */
> > > >      max_alloc_size = max;
> > > >  }
> > > > 
> > > > 
> > > 
> >