[FFmpeg-user] SSIM filter showing (small) differences in identical(?) streams
pdr0
pdr0 at shaw.ca
Sat Feb 27 05:52:54 EET 2021
Ian Pilcher wrote
> I am trying to understand how the SSIM and VMAF filters work, with an
> eye to finding the "best" compression settings for a video which will
> be composed from a series of TIFF images. Unfortunately, I'm stuck at
> the beginning, as I can't get the SSIM filter to behave as expected.
>
> ./source contains the original sequence of images.
>
> $ tiffinfo source/000000.tif
> TIFF Directory at offset 0x473108 (4665608)
> Image Width: 1440 Image Length: 1080
> Bits/Sample: 8
> Sample Format: unsigned integer
> Compression Scheme: None
> Photometric Interpretation: RGB color
> Samples/Pixel: 3
> Rows/Strip: 1
> Planar Configuration: single image plane
>
> I attempt to create a lossless video of the first minute.
>
> $ ffmpeg -start_number 0 -framerate 60000/1001 -i source/%06d.tif -t
> 00:01:00 -c:v huffyuv lossless.mkv
>
> The result appears reasonable.
>
> $ mediainfo lossless.mkv
> General
> Unique ID :
> 235140899628261703308032414639716345340
> (0xB0E67D6EF6B78362D9BCF9EA3080A5FC)
> Complete name : lossless.mkv
> Format : Matroska
> Format version : Version 4
> File size : 8.54 GiB
> Duration : 59 s 994 ms
> Overall bit rate : 1 223 Mb/s
> Writing application : Lavf58.45.100
> Writing library : Lavf58.45.100
> ErrorDetectionType : Per level 1
>
> Video
> ID : 1
> Format : HuffYUV
> Format version : Version 2
> Codec ID : V_MS/VFW/FOURCC / HFYU
> Duration : 59 s 994 ms
> Bit rate : 1 199 Mb/s
> Width : 1 440 pixels
> Height : 1 080 pixels
> Display aspect ratio : 4:3
> Frame rate mode : Constant
> Frame rate : 59.940 FPS
> Color space : RGB
> Bit depth : 8 bits
> Scan type : Progressive
> Bits/(Pixel*Frame) : 12.860
> Stream size : 8.37 GiB (98%)
> Writing library : Lavc58.91.100 huffyuv
> Default : Yes
> Forced : No
>
> Now let's see what the SSIM filter says.
>
> $ ffmpeg -i lossless.mkv -start_number 0 -framerate 60000/1001 -i
> source/%06d.tif -t 00:01:00 -filter_complex ssim -f null -
> ...
> Input #0, matroska,webm, from 'lossless.mkv':
> Metadata:
> ENCODER : Lavf58.45.100
> Duration: 00:00:59.99, start: 0.000000, bitrate: 1223104 kb/s
> Stream #0:0: Video: huffyuv (HFYU / 0x55594648), bgr0, 1440x1080,
> 59.94 fps, 59.94 tbr, 1k tbn, 1k tbc (default)
> Metadata:
> ENCODER : Lavc58.91.100 huffyuv
> DURATION : 00:00:59.994000000
> Input #1, image2, from 'source/%06d.tif':
> Duration: 01:47:16.00, start: 0.000000, bitrate: N/A
> Stream #1:0: Video: tiff, rgb24, 1440x1080, 59.94 tbr, 59.94 tbn,
> 59.94 tbc
> ...
> [Parsed_ssim_0 @ 0x55fb09c738c0] not matching timebases found between
> first input: 1/1000 and second input 1001/60000, results may be incorrect!
> ...
> [Parsed_ssim_0 @ 0x55fb09c738c0] SSIM R:0.833774 (7.793009) G:0.835401
> (7.835723) B:0.831058 (7.722615) All:0.833411 (7.783532)
>
> That's not what I expected. My understanding is that the R, G, B, and
> All values should all be "1.000000 (inf)".
>
> The "not matching timebases" warning is the obvious thing to look at.
> After much searching, I came upon the -video_track_timescale option, but
> it seems to only take an integer, and 60 is not the same as 59.94, so it
> seems that I simply can't directly compare a video stream with a non-
> integer framerate to an image sequence.
>
> As a workaround, I tried extracting the lossless video frames as a
> separate image sequence.
>
> $ ffmpeg -i lossless.mkv -start_number 0 lossless/%06d.png
>
> This created the expected sequence of image files (000000.png -
> 003595.png). Since I have both the "source" and the "lossless" streams
> as images sequences, I can use ImageMagick to compare them.
>
> $ for I in `seq -w 0 3595` ; do compare -metric AE source/00${I}.tif
> lossless/00${I}.png /tmp/diff.png 2>/dev/null || echo $I ; done
>
> This produces no output, indicating that ImageMagick thinks that the
> TIFF files in ./source and the PNG files in ./lossless contain
> completely identical image data. What does the SSIM filter say?
>
> $ ffmpeg -framerate 60000/1001 -start_number 0 -i lossless/%06d.png
> -framerate 60000/1001 -start_number 0 -i source/%06d.tif -t 00:01:00
> -filter_complex ssim -f null -
> ...
> Input #0, image2, from 'lossless/%06d.png':
> Duration: 00:00:59.99, start: 0.000000, bitrate: N/A
> Stream #0:0: Video: png, rgb24(pc), 1440x1080, 59.94 fps, 59.94
> tbr, 59.94 tbn, 59.94 tbc
> Input #1, image2, from 'source/%06d.tif':
> Duration: 01:47:16.00, start: 0.000000, bitrate: N/A
> Stream #1:0: Video: tiff, rgb24, 1440x1080, 59.94 tbr, 59.94 tbn,
> 59.94 tbc
> ...
> [Parsed_ssim_0 @ 0x556c7e948980] SSIM R:0.999775 (36.484195)
> G:0.999777 (36.508857) B:0.999774 (36.451744) All:0.999775 (36.481536)
>
> Close, but not the "1.000000 (inf)" that I would expect for identical
> streams. I suppose that it may be related to the "rgb24(pc)" vs.
> rgb24 pixel formats; it's hard to be sure, as Google isn't turning up
> anything that describes what "rgb24(pc)" actually means.
>
> Does anyone know what the heck is going on here? Am I doing something
> wrong?
Different timebase means there is potential to compare different frames.
Use settb
ffmpeg -framerate 60000/1001 -i lossless.mkv -start_number 0 -framerate
60000/1001 -i source/%06d.tif -t 00:01:00 -lavfi
"[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]ssim"
-f null -
For the 2nd image sequence png vs. tiff, one is rgb24(pc) , one is rgb24 .
There might be some metadata that is skewing the results. Need more info
--
Sent from: http://ffmpeg-users.933282.n4.nabble.com/
More information about the ffmpeg-user
mailing list