[FFmpeg-cvslog] AAC encoder: Fix application of M/S with PNS

[Claudio Freire]

ffmpeg | branch: master | Claudio Freire <klaussfreire at gmail.com> | Thu Nov 26 03:27:06 2015 -0300| [fc36d852ee3413f7cd00ce531ba985925fa7a749] | committer: Claudio Freire

AAC encoder: Fix application of M/S with PNS

When both M/S coding and PNS are enabled, scalefactors
and coding books would be mistakenly clobbered when setting
the M/S flag on PNS'd bands. The flag needs to be set to
signal the generation of correlated noise, but the scalefactors,
coefficients and the coding books need to be kept intact.

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=fc36d852ee3413f7cd00ce531ba985925fa7a749
---

 libavcodec/aaccoder.c |    4 ++--
 libavcodec/aacenc.c   |    8 +++++++-
 2 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/libavcodec/aaccoder.c b/libavcodec/aaccoder.c
index fd9785e..2337784 100644
--- a/libavcodec/aaccoder.c
+++ b/libavcodec/aaccoder.c
@@ -887,8 +887,8 @@ static void search_for_ms(AACEncContext *s, ChannelElement *cpe)
                     }
                     cpe->ms_mask[w*16+g] = dist2 <= dist1 && B1 < B0;
                     if (cpe->ms_mask[w*16+g]) {
-                        /* Setting the M/S mask is useful with I/S, but only the flag */
-                        if (!cpe->is_mask[w*16+g]) {
+                        /* Setting the M/S mask is useful with I/S or PNS, but only the flag */
+                        if (!cpe->is_mask[w*16+g] && sce0->band_type[w*16+g] != NOISE_BT && sce1->band_type[w*16+g] != NOISE_BT) {
                             sce0->sf_idx[w*16+g] = mididx;
                             sce1->sf_idx[w*16+g] = sididx;
                             sce0->band_type[w*16+g] = midcb;
diff --git a/libavcodec/aacenc.c b/libavcodec/aacenc.c
index 796038c..ca6c9dd 100644
--- a/libavcodec/aacenc.c
+++ b/libavcodec/aacenc.c
@@ -292,7 +292,13 @@ static void apply_mid_side_stereo(ChannelElement *cpe)
         for (w2 =  0; w2 < ics->group_len[w]; w2++) {
             int start = (w+w2) * 128;
             for (g = 0; g < ics->num_swb; g++) {
-                if (!cpe->ms_mask[w*16 + g] && !cpe->is_mask[w*16 + g]) {
+                /* ms_mask can be used for other purposes in PNS and I/S,
+                 * so must not apply M/S if any band uses either, even if
+                 * ms_mask is set.
+                 */
+                if (!cpe->ms_mask[w*16 + g] || cpe->is_mask[w*16 + g]
+                    || cpe->ch[0].band_type[w*16 + g] == NOISE_BT
+                    || cpe->ch[1].band_type[w*16 + g] == NOISE_BT) {
                     start += ics->swb_sizes[g];
                     continue;
                 }