[FFmpeg-devel] [PATCH] avcodec/mpegvideo: remove redundant workaround to recalculate last nonzero coefficient

[Ramiro Polla]

On Thu, Aug 29, 2024 at 9:13 PM Michael Niedermayer
<michael at niedermayer.cc> wrote:
> On Wed, Aug 28, 2024 at 02:14:33PM +0200, Ramiro Polla wrote:
> > On Thu, Aug 22, 2024 at 1:24 AM Ramiro Polla <ramiro.polla at gmail.com> wrote:
> > > The x86 optimized dct_quantize only calculates the last nonzero
> > > coefficient correctly if the zigzag scan order is used. For the
> > > alternate scan order, this value is incorrect.
> > >
> > > To work around this, the dct_unquantize functions process the entire
> > > block if the alternate scan order is used.
> > >
> > > But a second workaround (bb198e198ab) was added that recalculates the
> > > last nonzero coefficient after dct_quantize is called if the alternate
> > > scan order is used.
> > >
> > > This commit removes the first workaround, which became redundant.
> > > ---
> > >  libavcodec/mpegvideo.c     | 9 +++------
> > >  libavcodec/x86/mpegvideo.c | 6 ++----
> > >  2 files changed, 5 insertions(+), 10 deletions(-)
> >
> > Michael, could you please check if my analysis and the changes are correct?
>
> If you tested it and it works it has to be correct.
> This would result in significant errors if it was wrong

Thanks. I had manually tested by removing or leaving in place the
first and/or second workarounds, and checking that it failed without
both workarounds and worked with either workarounds. Fate passed, but
fate doesn't test alternate scan order. That seemed good enough, so I
pushed it.

Ramiro