[FFmpeg-devel] [PATCH 5/5] libavcodec/jpeg2000dec.c: Remove log2_chroma check in pixel format selection

[Gautam Ramakrishnan]

On Tue, Jun 23, 2020 at 2:55 AM Carl Eugen Hoyos <ceffmpeg at gmail.com> wrote:
>
> Am Mo., 22. Juni 2020 um 04:57 Uhr schrieb Gautam Ramakrishnan
> <gautamramk at gmail.com>:
> >
> > On Mon, Jun 22, 2020 at 1:54 AM Carl Eugen Hoyos <ceffmpeg at gmail.com> wrote:
> > >
> > > Am So., 21. Juni 2020 um 21:11 Uhr schrieb <gautamramk at gmail.com>:
> > > >
> > > > From: Gautam Ramakrishnan <gautamramk at gmail.com>
> > > >
> > > > The log2_chroma_wh is derived from the sample separations of the
> > > > codestream if the file is a j2k codestream. Not sure if sample
> > > > separation is same is subsampling and whether using sample
> > > > separation values from the codestream to determine pixel format.
> > >
> > > What would get fixed by this change?
> > >
> > The p1_01.j2k image was not getting recognized by the native
> > decoder due to this condition.
>
> In any case, this was missing from the commit message.
>
> > It would now get recognized. If this patch is fine,
>
> I wanted to suggest to add the following two lines after
> the calls to pix_fmt_guess():
>     if (s->avctx->pix_fmt == AV_PIX_FMT_NONE && ncomponents == 1)
>         s->avctx->pix_fmt = AV_PIX_FMT_GRAY8;
>
I had tried this for testing initially. I placed this inside the
if (i == possible_fmts_nb) check. It seemed to work correctly.
I could possibly resend with this also, but I did not know this would
be a good solution.
> But p1_01.j2k does not get decoded with the change either here.
>
> > I would preferably remove this check at all places.
>
> I thought the check is needed but if fuzzing does not produce
> invalid memory access for you, it may be ok.
>
I'll run the fuzzer again carefully.
> Carl Eugen
> _______________________________________________




-- 
-------------
Gautam |