[FFmpeg-cvslog] avfilter/vf_scale: do aspect ratio and scale factor compensation together

[Michael Niedermayer]

ffmpeg | branch: master | Michael Niedermayer <michaelni at gmx.at> | Sat Jan 25 16:48:13 2014 +0100| [1e48c39ece3ec790dad9a0e308ebe1a77b2f08ba] | committer: Michael Niedermayer

avfilter/vf_scale: do aspect ratio and scale factor compensation together

Fixes rounding error

Signed-off-by: Michael Niedermayer <michaelni at gmx.at>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=1e48c39ece3ec790dad9a0e308ebe1a77b2f08ba
---

 libavfilter/vf_scale.c |   10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

diff --git a/libavfilter/vf_scale.c b/libavfilter/vf_scale.c
index 945ed12..bb5cd39 100644
--- a/libavfilter/vf_scale.c
+++ b/libavfilter/vf_scale.c
@@ -295,16 +295,14 @@ static int config_props(AVFilterLink *outlink)
         w = inlink->w;
     if (!(h = scale->h))
         h = inlink->h;
-    if (w == -1)
-        w = av_rescale(h, inlink->w, inlink->h);
-    if (h == -1)
-        h = av_rescale(w, inlink->h, inlink->w);
 
     /* Make sure that the result is divisible by the factor we determined
      * earlier. If no factor was set, it is nothing will happen as the default
      * factor is 1 */
-    w = (w / factor_w) * factor_w;
-    h = (h / factor_h) * factor_h;
+    if (w == -1)
+        w = av_rescale(h, inlink->w, inlink->h * factor_w) * factor_w;
+    if (h == -1)
+        h = av_rescale(w, inlink->h, inlink->w * factor_h) * factor_h;
 
     /* Note that force_original_aspect_ratio may overwrite the previous set
      * dimensions so that it is not divisible by the set factors anymore. */