[FFmpeg-user] SSIM filter showing (small) differences in identical(?) streams

pdr0 pdr0 at shaw.ca
Sat Feb 27 05:52:54 EET 2021


Ian Pilcher wrote
> I am trying to understand how the SSIM and VMAF filters work, with an
> eye to finding the "best" compression settings for a video which will
> be composed from a series of TIFF images.  Unfortunately, I'm stuck at
> the beginning, as I can't get the SSIM filter to behave as expected.
> 
> ./source contains the original sequence of images.
> 
>   $ tiffinfo source/000000.tif
>   TIFF Directory at offset 0x473108 (4665608)
>     Image Width: 1440 Image Length: 1080
>     Bits/Sample: 8
>     Sample Format: unsigned integer
>     Compression Scheme: None
>     Photometric Interpretation: RGB color
>     Samples/Pixel: 3
>     Rows/Strip: 1
>     Planar Configuration: single image plane
> 
> I attempt to create a lossless video of the first minute.
> 
>   $ ffmpeg -start_number 0 -framerate 60000/1001 -i source/%06d.tif -t 
> 00:01:00 -c:v huffyuv lossless.mkv
> 
> The result appears reasonable.
> 
>   $ mediainfo lossless.mkv
>   General
>   Unique ID                                : 
> 235140899628261703308032414639716345340
> (0xB0E67D6EF6B78362D9BCF9EA3080A5FC)
>   Complete name                            : lossless.mkv
>   Format                                   : Matroska
>   Format version                           : Version 4
>   File size                                : 8.54 GiB
>   Duration                                 : 59 s 994 ms
>   Overall bit rate                         : 1 223 Mb/s
>   Writing application                      : Lavf58.45.100
>   Writing library                          : Lavf58.45.100
>   ErrorDetectionType                       : Per level 1
> 
>   Video
>   ID                                       : 1
>   Format                                   : HuffYUV
>   Format version                           : Version 2
>   Codec ID                                 : V_MS/VFW/FOURCC / HFYU
>   Duration                                 : 59 s 994 ms
>   Bit rate                                 : 1 199 Mb/s
>   Width                                    : 1 440 pixels
>   Height                                   : 1 080 pixels
>   Display aspect ratio                     : 4:3
>   Frame rate mode                          : Constant
>   Frame rate                               : 59.940 FPS
>   Color space                              : RGB
>   Bit depth                                : 8 bits
>   Scan type                                : Progressive
>   Bits/(Pixel*Frame)                       : 12.860
>   Stream size                              : 8.37 GiB (98%)
>   Writing library                          : Lavc58.91.100 huffyuv
>   Default                                  : Yes
>   Forced                                   : No
> 
> Now let's see what the SSIM filter says.
> 
>   $ ffmpeg -i lossless.mkv -start_number 0 -framerate 60000/1001 -i 
> source/%06d.tif -t 00:01:00 -filter_complex ssim -f null -
>   ...
>   Input #0, matroska,webm, from 'lossless.mkv':
>     Metadata:
>       ENCODER         : Lavf58.45.100
>     Duration: 00:00:59.99, start: 0.000000, bitrate: 1223104 kb/s
>       Stream #0:0: Video: huffyuv (HFYU / 0x55594648), bgr0, 1440x1080, 
> 59.94 fps, 59.94 tbr, 1k tbn, 1k tbc (default)
>       Metadata:
>         ENCODER         : Lavc58.91.100 huffyuv
>         DURATION        : 00:00:59.994000000
>   Input #1, image2, from 'source/%06d.tif':
>     Duration: 01:47:16.00, start: 0.000000, bitrate: N/A
>       Stream #1:0: Video: tiff, rgb24, 1440x1080, 59.94 tbr, 59.94 tbn, 
> 59.94 tbc
>   ...
>   [Parsed_ssim_0 @ 0x55fb09c738c0] not matching timebases found between 
> first input: 1/1000 and second input 1001/60000, results may be incorrect!
>   ...
>   [Parsed_ssim_0 @ 0x55fb09c738c0] SSIM R:0.833774 (7.793009) G:0.835401 
> (7.835723) B:0.831058 (7.722615) All:0.833411 (7.783532)
> 
> That's not what I expected.  My understanding is that the R, G, B, and
> All values should all be "1.000000 (inf)".
> 
> The "not matching timebases" warning is the obvious thing to look at.
> After much searching, I came upon the -video_track_timescale option, but
> it seems to only take an integer, and 60 is not the same as 59.94, so it
> seems that I simply can't directly compare a video stream with a non-
> integer framerate to an image sequence.
> 
> As a workaround, I tried extracting the lossless video frames as a
> separate image sequence.
> 
>   $ ffmpeg -i lossless.mkv -start_number 0 lossless/%06d.png
> 
> This created the expected sequence of image files (000000.png -
> 003595.png).  Since I have both the "source" and the "lossless" streams
> as images sequences, I can use ImageMagick to compare them.
> 
>   $ for I in `seq -w 0 3595` ; do compare -metric AE source/00${I}.tif 
> lossless/00${I}.png /tmp/diff.png 2>/dev/null || echo $I ; done
> 
> This produces no output, indicating that ImageMagick thinks that the
> TIFF files in ./source and the PNG files in ./lossless contain
> completely identical image data.  What does the SSIM filter say?
> 
>   $ ffmpeg -framerate 60000/1001 -start_number 0 -i lossless/%06d.png 
> -framerate 60000/1001 -start_number 0 -i source/%06d.tif -t 00:01:00 
> -filter_complex ssim -f null -
>   ...
>   Input #0, image2, from 'lossless/%06d.png':
>     Duration: 00:00:59.99, start: 0.000000, bitrate: N/A
>       Stream #0:0: Video: png, rgb24(pc), 1440x1080, 59.94 fps, 59.94 
> tbr, 59.94 tbn, 59.94 tbc
>   Input #1, image2, from 'source/%06d.tif':
>     Duration: 01:47:16.00, start: 0.000000, bitrate: N/A
>       Stream #1:0: Video: tiff, rgb24, 1440x1080, 59.94 tbr, 59.94 tbn, 
> 59.94 tbc
>   ...
>   [Parsed_ssim_0 @ 0x556c7e948980] SSIM R:0.999775 (36.484195) 
> G:0.999777 (36.508857) B:0.999774 (36.451744) All:0.999775 (36.481536)
> 
> Close, but not the "1.000000 (inf)" that I would expect for identical
> streams.  I suppose that it may be related to the "rgb24(pc)" vs.
> rgb24 pixel formats; it's hard to be sure, as Google isn't turning up
> anything that describes what "rgb24(pc)" actually means.
> 
> Does anyone know what the heck is going on here?  Am I doing something
> wrong?


Different timebase means there is potential to compare different frames. 

Use settb

ffmpeg -framerate 60000/1001 -i lossless.mkv -start_number 0 -framerate
60000/1001 -i source/%06d.tif -t 00:01:00 -lavfi 
"[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]ssim"
-f null -



For the 2nd image sequence png vs. tiff, one is rgb24(pc) , one is rgb24 .
There might be some metadata that is skewing the results. Need more info



--
Sent from: http://ffmpeg-users.933282.n4.nabble.com/


More information about the ffmpeg-user mailing list