[Ffmpeg-devel] Distorted output when sequence_display_extension is present

[Måns Rullgård]

Nico Sabbi <nicola_sabbi at fastwebnet.it> writes:

> Hi,
> this file: ftp://www.mplayerhq.hu/MPlayer/samples/MPEG-VOB/winnicki.mpg
> is a 640x480 mpeg2 sample encoded with a sequence_display_extension
> in which horizontal_display_size=640 and vertical_display_size=240.

So it seems, but why?

> libavcodec assigns SAR=0.5 (the corresponding DAR is 0.67), which
> results in a very badly distored picture.

It does indeed.

> If in mpeg12.c I modify the assignment of avctx->sample_aspect_ratio
> in terms of
> (AVRational){s->width, s->height}
> instead of
> (AVRational){s1->pan_scan.width, s1->pan_scan.height}
> the output is no more distorted.
>
> This problems is visibile in other samples that I have but that I
> can't distribute.
>
> IIUC the parameters in the sequence_display_extension (at least in
> this case) define a region to crop (640x240) but lavc doesn't seem
> to be cropping it.

The decoder shouldn't crop it.  The display_horizontal_size and
display_vertical_size values are only to be used when actually
displaying the video.  Here's what the spec says:

  display_horizontal_size - See display_vertical_size.

  display_vertical_size - display_horizontal_size and
  display_vertical_size together define a rectangle which may be
  considered as the "intended display's" active region. If this
  rectangle is smaller than the encoded frame size, then the display
  process may be expected to display only a portion of the encoded
  frame. Conversely if the display rectangle is larger than the
  encoded frame size, then the display process may be expected to
  display the reconstructed frames on a portion of the display device
  rather than on the whole display device.

  display_horizontal_size shall be in the same units as
  horizontal_size (samples of the encoded frames).

  display_vertical_size shall be in the same units as vertical_size
  (lines of the encoded frames).

  display_horizontal_size and display_vertical_size do not affect the
  decoding process but may be used by the display process that is not
  standardised in this Specification.

> Is this an incorrect stream or a bug in libavcodec?

Another quote from the spec:

  aspect_ratio_information either specifies that the "Sample Aspect
  Ratio" (SAR) of the reconstructed frame is 1,0 (square samples) or
  alternatively it gives the "Display Aspect Ratio" (DAR).

    - If sequence_display_extension() is not present, then it is
      intended that the entire reconstructed frame is intended to be
      mapped to the entire active region of the display. The sample
      aspect ratio may be calculated as follows:
                                    
        SAR = DAR * horizontal_size / vertical_size

      NOTE 1 - In this case horizontal_size and vertical_size are
      constrained by the SAR of the source and the DAR selected.

    - If sequence_display_extension() is present then the sample
      aspect ratio may be calculated as follows:
                     
        SAR = DAR * display_horizontal_size / display_vertical_size

This is obviously nonsense.  Those divisions have to be reversed to
make any sense at all.  With that correction, the standard appears to
imply that the DAR specified in the sequence header refers to the
displayed picture, i.e. display_horizontal_size x display_vertical_size.

In our case, this means that the 640x480 video is intended to be
cropped (where to cut would be specified in a picture display
extension, of which there are none in your stream) and displayed on a
640x240 output device with aspect ratio 4/3.  Such a device would have
a SAR of 1/2.

I would say lavc is correct in some sense.

It appears that the stream was created by something called "MPEG Video
Wizard (12/2003)" from Womble Multimedia, Inc.  Whatever this is, it
seems to have some pretty strange ideas about video.

-- 
M?ns Rullg?rd
mru at inprovide.com