[MPlayer-dev-eng] [PATCH] fix aspect ratio calculations broken by new av_cmp_q

[Andrew Wason]

av_cmp_q() changed in r25338/25340 ffmpeg - it can now return INT_MIN
if one of the rationals is 0/0

av_cmp_q used to return:
 * @return 0 if a==b, 1 if a>b and -1 if a<b
but now returns:
 * @return 0 if a==b, 1 if a>b, -1 if a<b, and INT_MIN if one of the
 * values is of the form 0/0

So statements like "if (av_cmp_q(a,b))" behave differently now if one
rational is 0/0

This breaks the display aspect ratio in mplayer for a number of
samples because libmpcodecs/vd_ffmpeg.c initializes
last_sample_aspect_ratio to 0/0 and we end up resetting sh->aspect to
0 in libmpcodecs/vd_ffmpeg.c:init_vo() when last_sample_aspect_ratio
is used in av_cmp_q

Attached patch initializes last_sample_aspect_ratio before using it to
avoid this.

Sample uploaded to MPlayer/incoming/aspect/last_sample_aspect_ratio.mov
This plays at 1280x1080 without the patch, and 1920x1080 after (which
is also what it played at prior to ffmpeg r25338)

Andrew
-------------- next part --------------
Index: libmpcodecs/vd_ffmpeg.c
===================================================================
--- libmpcodecs/vd_ffmpeg.c	(revision 32511)
+++ libmpcodecs/vd_ffmpeg.c	(working copy)
@@ -536,6 +536,9 @@
     float aspect= av_q2d(avctx->sample_aspect_ratio) * avctx->width / avctx->height;
     int width, height;
 
+    if (ctx->last_sample_aspect_ratio.den == 0)
+        ctx->last_sample_aspect_ratio = avctx->sample_aspect_ratio;
+
     width = avctx->width;
     height = avctx->height;