[FFmpeg-cvslog] avcodec/audiotoolboxdec: Properly fill out_format

[Christopher Snowhill]

ffmpeg | branch: master | Christopher Snowhill <kode54 at gmail.com> | Thu Dec 30 21:23:09 2021 -0800| [944b8c96c90ae050606e638f77a15d6292209b91] | committer: James Almer

avcodec/audiotoolboxdec: Properly fill out_format

Monterey needs mBytesPerFrame and mBytesPerPacket to be set, and I'm
surprised this didn't break any previous system versions.

Fixes bug #9564: Cannot decode xHE-AAC with audiotoolbox (aac_at) on
Mac OS Monterey. Fixes likely bug that none of the AudioToolbox
decoders work on Monterey.

Signed-off-by: Christopher Snowhill <kode54 at gmail.com>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=944b8c96c90ae050606e638f77a15d6292209b91
---

 libavcodec/audiotoolboxdec.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/libavcodec/audiotoolboxdec.c b/libavcodec/audiotoolboxdec.c
index b8d0e868f7..daccf4abf8 100644
--- a/libavcodec/audiotoolboxdec.c
+++ b/libavcodec/audiotoolboxdec.c
@@ -375,6 +375,11 @@ static av_cold int ffat_create_decoder(AVCodecContext *avctx,
     avctx->ch_layout.order       = AV_CHANNEL_ORDER_UNSPEC;
     avctx->ch_layout.nb_channels = out_format.mChannelsPerFrame = in_format.mChannelsPerFrame;
 
+    out_format.mBytesPerFrame =
+        out_format.mChannelsPerFrame * (out_format.mBitsPerChannel / 8);
+    out_format.mBytesPerPacket =
+        out_format.mBytesPerFrame * out_format.mFramesPerPacket;
+
     if (avctx->codec_id == AV_CODEC_ID_ADPCM_IMA_QT)
         in_format.mFramesPerPacket = 64;