[FFmpeg-devel] [avformat] Prevent undefined shift with wrap_bits > 63.

[Dale Curtis]

On Mon, Nov 20, 2017 at 12:40 PM, Michael Niedermayer <
michael at niedermayer.cc> wrote:

> On Mon, Nov 20, 2017 at 12:05:05PM -0800, Dale Curtis wrote:
> > On Sat, Nov 18, 2017 at 2:44 AM, Michael Niedermayer
> <michael at niedermayer.cc
> > > wrote:
> > >
> > > this would skip the code for wrap_bits >= 63, this does not look
> > > correct
> > >
> >
> > Why do you think that's incorrect? The max int64_t value is 1 << 63 and 2
> > << 63 == 1 << 64 ?
>
> I think its incorrect because the change would completely skip the loop
> but a wrap of "1<<64" would still need pts to be filled in.
>

There's no way this code can work with a wrap_bits > 63.  2<<64 > uint64_t.
The best we can do over my change is to make wrap_bits == 63 work by
changing the line from 2LL << (wrap_bits - 1) to 2ULL << (wrap_bits -
1). av_compare_mod takes a uint64_t in this position so that should be
okay. How does that sound?

- dale