[FFmpeg-devel] [PATCH] latmenc: Check for LOAS sync word

[James Almer]

On 15/01/13 10:34 PM, Michael Niedermayer wrote:
> you can check extradata
> as a sideeffect it would fix the crash
> with the right (well wrong) data the current patch not fix the crash

How about the attached patch? (It still needs reindentation)
If there's no extradata then the packet is either LOAS or ADTS, and it should be
used if it's the former or discarded if it's anything else.

This also prevents potential misdetection of raw packets as ADTS packets, since 
the check was on the first 3 bytes as well and without checking if there was any
extradata available.

Regards.
-------------- next part --------------
>From dc95954687b24faf19284fa39d5cb34ce5863741 Mon Sep 17 00:00:00 2001
From: James Almer <jamrial at gmail.com>
Date: Wed, 16 Jan 2013 05:21:55 -0300
Subject: [PATCH] latmenc: Check for LOAS sync word

Write the packet unaltered if found.

Fixes ticket #1917

Signed-off-by: James Almer <jamrial at gmail.com>
---
 libavformat/latmenc.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/libavformat/latmenc.c b/libavformat/latmenc.c
index 233eab8..d6bdf62 100644
--- a/libavformat/latmenc.c
+++ b/libavformat/latmenc.c
@@ -152,8 +152,14 @@ static int latm_write_packet(AVFormatContext *s, AVPacket *pkt)
     if (s->streams[0]->codec->codec_id == AV_CODEC_ID_AAC_LATM)
         return ff_raw_write_packet(s, pkt);
 
-    if (pkt->size > 2 && pkt->data[0] == 0xff && (pkt->data[1] >> 4) == 0xf) {
+    if (!s->streams[0]->codec->extradata) {
+        if (pkt->size > 2 && pkt->data[0] == 0x56 && (pkt->data[1] >> 4) == 0xe &&
+            (AV_RB16(pkt->data + 1) & 0x1FFF) + 3 == pkt->size)
+            return ff_raw_write_packet(s, pkt);
+
+    if (pkt->size > 2 && pkt->data[0] == 0xff && (pkt->data[1] >> 4) == 0xf)
         av_log(s, AV_LOG_ERROR, "ADTS header detected - ADTS will not be incorrectly muxed into LATM\n");
+
         return AVERROR_INVALIDDATA;
     }
     if (pkt->size > 0x1fff)
-- 
1.8.0.msysgit.0