[FFmpeg-cvslog] swr/rematrix: Fix handling of AV_CH_LAYOUT_STEREO_DOWNMIX output

[Michael Niedermayer]

ffmpeg | branch: release/1.2 | Michael Niedermayer <michaelni at gmx.at> | Tue Aug 20 16:56:51 2013 +0200| [c55a09a8b65f88d748daa885b4829019a081040c] | committer: Carl Eugen Hoyos

swr/rematrix: Fix handling of AV_CH_LAYOUT_STEREO_DOWNMIX output

Fixes Ticket2859

Note, testcases related to the downmix channels are welcome.
(id like to make sure this is working correctly now, as obviously it didnt
 work before ...)

Signed-off-by: Michael Niedermayer <michaelni at gmx.at>
(cherry picked from commit c56d4dab039b352961cca298d753b04e2f2fd990)

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=c55a09a8b65f88d748daa885b4829019a081040c
---

 libswresample/rematrix.c |    8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/libswresample/rematrix.c b/libswresample/rematrix.c
index cfe0a1b..541f46a 100644
--- a/libswresample/rematrix.c
+++ b/libswresample/rematrix.c
@@ -78,9 +78,6 @@ static int even(int64_t layout){
 }
 
 static int clean_layout(SwrContext *s, int64_t layout){
-    if((layout & AV_CH_LAYOUT_STEREO_DOWNMIX) == AV_CH_LAYOUT_STEREO_DOWNMIX)
-        return AV_CH_LAYOUT_STEREO;
-
     if(layout && layout != AV_CH_FRONT_CENTER && !(layout&(layout-1))) {
         char buf[128];
         av_get_channel_layout_string(buf, sizeof(buf), -1, layout);
@@ -120,6 +117,11 @@ av_cold static int auto_matrix(SwrContext *s)
     in_ch_layout = clean_layout(s, s->in_ch_layout);
     out_ch_layout = clean_layout(s, s->out_ch_layout);
 
+    if(   out_ch_layout == AV_CH_LAYOUT_STEREO_DOWNMIX
+       && (in_ch_layout & AV_CH_LAYOUT_STEREO_DOWNMIX) == 0
+    )
+        out_ch_layout = AV_CH_LAYOUT_STEREO;
+
     if(!sane_layout(in_ch_layout)){
         av_get_channel_layout_string(buf, sizeof(buf), -1, s->in_ch_layout);
         av_log(s, AV_LOG_ERROR, "Input channel layout '%s' is not supported\n", buf);