[FFmpeg-devel] Controlling range conversions in the "scale" filter

Steinar Apalnes steipal at gmail.com
Sun Sep 5 13:00:37 EEST 2021 

Hi,
It seems to be a bug (maybe by design) when trying to control how the
video/color ranges are treated in the "scale" filter.
Basically what I need to specify is that the range should NOT be converted
when input range == output range, for example:
in_range=pc:out_range=pc or
in_range=tv:out_range=tv

In my head there should be NO range conversion in either commands.
However, this does not seem to be the case. If you look at the test
commands below and their output you will see that a range conversion has
taken place even though input range and output range are set to be the
exact same. The only parameters I change are the scale width and height.
I'm using a late august 2021 build.

Command:
ffmpeg -f lavfi -i smptebars=duration=1:size=720x576:rate=1 -vf
"signalstats,metadata=print:key=lavfi.signalstats.YMIN,metadata=print:key=lavfi.signalstats.YMAX,scale=w=1920:h=1080:in_color_matrix=bt709:out_color_matrix=bt709:in_range=pc:out_range=pc,signalstats,metadata=print:key=lavfi.signalstats.YMIN,metadata=print:key=lavfi.signalstats.YMAX"
-f null -

Raw FFmpeg output:
ffmpeg version N-103405-g1930a85e83 Copyright (c) 2000-2021 the FFmpeg
developers
  built with gcc 10.3.0 (Rev2, Built by MSYS2 project)
  configuration:  --disable-static --enable-shared --pkg-config=pkgconf
--cc='ccache gcc' --cxx='ccache g++' --disable-autodetect --enable-amf
--enable-bzlib --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2
--enable-iconv --enable-lzma --enable-nvenc --enable-zlib --enable-sdl2
--enable-ffnvcodec --enable-nvdec --enable-cuda-llvm --enable-libmp3lame
--enable-libopus --enable-libvorbis --enable-libx264 --enable-libx265
--enable-libdav1d --enable-libaom --disable-debug --enable-fontconfig
--enable-libass --enable-libbluray --enable-libfreetype --enable-libmfx
--enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb
--enable-libopenjpeg --enable-libsnappy --enable-libsoxr --enable-libspeex
--enable-libtheora --enable-libtwolame --enable-libvidstab
--enable-libvo-amrwbenc --enable-libwebp --enable-libxml2 --enable-libzimg
--enable-libshine --enable-gpl --enable-avisynth --enable-libxvid
--enable-libopenmpt --enable-version3 --enable-libsrt --enable-libgsm
--enable-libvmaf --enable-libsvtav1 --enable-mbedtls
--extra-cflags=-DLIBTWOLAME_STATIC --extra-libs=-lstdc++
--extra-cflags=-DLIBXML_STATIC --extra-libs=-liconv --disable-w32threads
--shlibdir=/local64/bin-video
  libavutil      57.  4.101 / 57.  4.101
  libavcodec     59.  6.100 / 59.  6.100
  libavformat    59.  4.102 / 59.  4.102
  libavdevice    59.  0.101 / 59.  0.101
  libavfilter     8.  4.100 /  8.  4.100
  libswscale      6.  0.100 /  6.  0.100
  libswresample   4.  0.100 /  4.  0.100
  libpostproc    56.  0.100 / 56.  0.100
Input #0, lavfi, from 'smptebars=duration=1:size=720x576:rate=1':
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 720x576 [SAR
1:1 DAR 5:4], 1 tbr, 1 tbn
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
[Parsed_metadata_1 @ 00000298dcb6c7c0] frame:0    pts:0       pts_time:0
[Parsed_metadata_1 @ 00000298dcb6c7c0] lavfi.signalstats.YMIN=7
[Parsed_metadata_2 @ 00000298dcb6bcc0] frame:0    pts:0       pts_time:0
[Parsed_metadata_2 @ 00000298dcb6bcc0] lavfi.signalstats.YMAX=235
[swscaler @ 00000298dcb7fec0] Warning: data is not aligned! This can lead
to a speed loss
[Parsed_metadata_5 @ 00000298dcb6b0c0] frame:0    pts:0       pts_time:0
[Parsed_metadata_5 @ 00000298dcb6b0c0] lavfi.signalstats.YMIN=0
[Parsed_metadata_6 @ 00000298dcb6b4c0] frame:0    pts:0       pts_time:0
[Parsed_metadata_6 @ 00000298dcb6b4c0] lavfi.signalstats.YMAX=255
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf59.4.102
  Stream #0:0: Video: wrapped_avframe, yuv420p(pc, progressive), 1920x1080
[SAR 45:64 DAR 5:4], q=2-31, 200 kb/s, 1 fps, 1 tbn
    Metadata:
      encoder         : Lavc59.6.100 wrapped_avframe
frame=    1 fps=0.0 q=-0.0 Lsize=N/A time=00:00:01.00 bitrate=N/A
speed=20.1x
video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB
muxing overhead: unknown

If we can trust the signalstats filter, YMIN has gone from 7 to 0, YMAX has
gone from 235 to 255

Furthermore, it's not consistent:

Command:
ffmpeg -f lavfi -i smptebars=duration=1:size=720x576:rate=1 -vf
"signalstats,metadata=print:key=lavfi.signalstats.YMIN,metadata=print:key=lavfi.signalstats.YMAX,scale=w=1280:h=720:in_color_matrix=bt709:out_color_matrix=bt709:in_range=pc:out_range=pc,signalstats,metadata=print:key=lavfi.signalstats.YMIN,metadata=print:key=lavfi.signalstats.YMAX"
-f null -

Washed FFmpeg output:
[Parsed_metadata_1 @ 000001ab9f8dc9c0] frame:0    pts:0       pts_time:0
[Parsed_metadata_1 @ 000001ab9f8dc9c0] lavfi.signalstats.YMIN=7
[Parsed_metadata_2 @ 000001ab9f8db3c0] frame:0    pts:0       pts_time:0
[Parsed_metadata_2 @ 000001ab9f8db3c0] lavfi.signalstats.YMAX=235
[swscaler @ 000001ab9f8efec0] Warning: data is not aligned! This can lead
to a speed loss
[Parsed_metadata_5 @ 000001ab9f8dc4c0] frame:0    pts:0       pts_time:0
[Parsed_metadata_5 @ 000001ab9f8dc4c0] lavfi.signalstats.YMIN=5
[Parsed_metadata_6 @ 000001ab9f8db5c0] frame:0    pts:0       pts_time:0
[Parsed_metadata_6 @ 000001ab9f8db5c0] lavfi.signalstats.YMAX=255

In this case YMIN has gone from 7 to 5, YMAX has gone from 235 to 255

Here are a few more:
ffmpeg -f lavfi -i smptebars=duration=1:size=720x576:rate=1 -vf
"signalstats,metadata=print:key=lavfi.signalstats.YMIN,metadata=print:key=lavfi.signalstats.YMAX,scale=w=720:h=720:in_color_matrix=bt709:out_color_matrix=bt709:in_range=pc:out_range=pc,signalstats,metadata=print:key=lavfi.signalstats.YMIN,metadata=print:key=lavfi.signalstats.YMAX"
-f null -

[Parsed_metadata_1 @ 000001fb530bb9c0] frame:0    pts:0       pts_time:0
[Parsed_metadata_1 @ 000001fb530bb9c0] lavfi.signalstats.YMIN=7
[Parsed_metadata_2 @ 000001fb530bb0c0] frame:0    pts:0       pts_time:0
[Parsed_metadata_2 @ 000001fb530bb0c0] lavfi.signalstats.YMAX=235
[swscaler @ 000001fb530cfe80] Warning: data is not aligned! This can lead
to a speed loss
[Parsed_metadata_5 @ 000001fb530bb2c0] frame:0    pts:0       pts_time:0
[Parsed_metadata_5 @ 000001fb530bb2c0] lavfi.signalstats.YMIN=7
[Parsed_metadata_6 @ 000001fb530bbbc0] frame:0    pts:0       pts_time:0
[Parsed_metadata_6 @ 000001fb530bbbc0] lavfi.signalstats.YMAX=243

ffmpeg -f lavfi -i smptebars=duration=1:size=720x576:rate=1 -vf
"signalstats,metadata=print:key=lavfi.signalstats.YMIN,metadata=print:key=lavfi.signalstats.YMAX,scale=w=720:h=719:in_color_matrix=bt709:out_color_matrix=bt709:in_range=pc:out_range=pc,signalstats,metadata=print:key=lavfi.signalstats.YMIN,metadata=print:key=lavfi.signalstats.YMAX"
-f null -

[Parsed_metadata_1 @ 00000229dd9bc2c0] frame:0    pts:0       pts_time:0
[Parsed_metadata_1 @ 00000229dd9bc2c0] lavfi.signalstats.YMIN=7
[Parsed_metadata_2 @ 00000229dd9bc0c0] frame:0    pts:0       pts_time:0
[Parsed_metadata_2 @ 00000229dd9bc0c0] lavfi.signalstats.YMAX=235
[swscaler @ 00000229dd9cfe80] Warning: data is not aligned! This can lead
to a speed loss
[Parsed_metadata_5 @ 00000229dd9bb6c0] frame:0    pts:0       pts_time:0
[Parsed_metadata_5 @ 00000229dd9bb6c0] lavfi.signalstats.YMIN=4
[Parsed_metadata_6 @ 00000229dd9bafc0] frame:0    pts:0       pts_time:0
[Parsed_metadata_6 @ 00000229dd9bafc0] lavfi.signalstats.YMAX=251


So again, you can see how the behavior is difficult to control and it seems
to be related to the actual scaling of the image. So I'm hoping someone can
shed light on this behavior.


Thanks

-steinar

More information about the ffmpeg-devel mailing list