[FFmpeg-devel] [PATCH] avfilter/vf_scale: set RGB to always be full range

[Jan Ekström]

On Wed, Sep 16, 2020 at 11:38 PM Michael Niedermayer
<michael at niedermayer.cc> wrote:
>
> On Wed, Sep 16, 2020 at 12:16:43AM +0300, Jan Ekström wrote:
> > This value - while it looks like the actual range of the content -
> > is nothing but the internal value of swscale.
> >
> > Thus, if we have RGB content, force the value to 1. Swscale will
> > ignore it, but at least the value of the output AVFrame will now
> > properly be "full range" instead of "limited range", as it is right
> > now.
> > ---
> >  libavfilter/vf_scale.c | 17 +++++++++++++++++
> >  1 file changed, 17 insertions(+)
> >
> > diff --git a/libavfilter/vf_scale.c b/libavfilter/vf_scale.c
> > index 58eee96744..12df27edf4 100644
> > --- a/libavfilter/vf_scale.c
> > +++ b/libavfilter/vf_scale.c
> [...]
> > @@ -768,7 +777,15 @@ scale:
> >          else if (in_range != AVCOL_RANGE_UNSPECIFIED)
> >              in_full  = (in_range == AVCOL_RANGE_JPEG);
> >          if (scale->out_range != AVCOL_RANGE_UNSPECIFIED)
> > +            // note: this can be silently overridden by
> > +            //       sws_setColorspaceDetails for non-YCbCr formats
> >              out_full = (scale->out_range == AVCOL_RANGE_JPEG);
> > +        else if (out_desc->flags & AV_PIX_FMT_FLAG_RGB)
> > +            // the range values received from swscale are its internal
> > +            // identifiers, and will not be nonzero for any sort of RGB.
> > +            // thus, if we do not set it to nonzero ourselves, this will
> > +            // be incorrect.
> > +            out_full = 1;
>
> Does anything document the meaning of range or AVCOL_RANGE for RGB ?
> The enum is documented as "MPEG vs JPEG YUV range."
>
> What is limited range RGB ? what would the ranges of R,G,B be and where
> is that specified ?
>
> That said, changing it to full range for RGB as done by this patch is
> probably ok if it helps some code (for example allows simplifications)
>

I am not here to say that we would have to support limited range RGB,
which I mostly only see mentioned as the PC graphics output for
devices such as video projectors.

The problem is just that vf_scale took that internal swscale flag and
attempted to interpret that as full|limited range flag as-is.

Anyways, posted a v2 which actually verifies the requested flags
against the actually configured ones, while keeping the translation
between swscale internal value and "is this stuff going to be full
range or not".

Possibly simpler to follow/read than this one :) , even though clearly
more complex.

Jan