[FFmpeg-devel] Playback issue on Sony TVs caused by btrt box

Mon Mar 21 11:10:28 EET 2022

Ping... thanks!

-----Original Message-----
From: Eran Kornblau 
Sent: Sunday, 6 March 2022 11:41
To: FFmpeg development discussions and patches <ffmpeg-devel at ffmpeg.org>
Subject: RE: [FFmpeg-devel] Playback issue on Sony TVs caused by btrt box

> 
> 
> > On Tue, Feb 22, 2022 at 9:55 AM Marton Balint <cus at passwd.hu> wrote:
> > >
> > > On Mon, 21 Feb 2022, Eran Kornblau wrote:
> > >
> > > >> On Mon, Feb 21, 2022 at 5:07 PM Eran Kornblau <eran.kornblau at kaltura.com> wrote:
> > > >>>
> > > >>> Hi all,
> > > >>>
> > > >>> We've recently upgraded our ffmpeg version, and we got a playback issue on some Sony TV models that are playing HBBTV/DASH+DRM - video plays fine, audio doesn't play at all.
> > > >>> Listing here some of the affected models (not pasting all, the list is long...) - KDL-32W600D, KDL-40W650D, KDL-48W650D, KDL-43W750D, KDL-49W750D, KDL-55W650D.
> > > >>>
> > > >>> After some investigation, we found the cause was the addition 
> > > >>> of the 'btrt' atom to the mp4 – 
> > > >>> https://eur02.safelinks.protection.outlook.com/?url=https%3A%2
> > > >>> F%
> > > >>> 2F
> > > >>> gith
> > > >>> ub.com%2FFFmpeg%2FFFmpeg%2Fcommit%2F3838e8fc210aa09a9f9058506c
> > > >>> 0c
> > > >>> e8
> > > >>> 0b6a
> > > >>> d9b9c3&data=04%7C01%7C%7Cc96aade7b4354115844508d9f5665925%
> > > >>> 7C
> > > >>> 0c
> > > >>> 5037
> > > >>> 48de3f4e2597e26819d53a42b6%7C1%7C0%7C637810642206022682%7CUnkn
> > > >>> ow
> > > >>> n%
> > > >>> 7CTW
> > > >>> FpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwi
> > > >>> LC
> > > >>> JX
> > > >>> VCI6
> > > >>> Mn0%3D%7C3000&sdata=Na7ZxDQTby8Xq9CjB51lJyP1IOMdwLqoRLNODU
> > > >>> C4
> > > >>> nX
> > > >>> 0%3D
> > > >>> &reserved=0 The TV decoder expects to get the sinf atom 
> > > >>> right after esds, and doesn't properly handle the btrt atom in 
> > > >>> between (our packager adds the sinf atom at the end of the original stsd entry that was read from the mp4 file).
> > > >>>
> > > >>> Since, in my understanding, the btrt atom was added mostly for reporting reasons, IMHO, it should be a config option - off by default.
> > > >>> I would happily submit a patch for it, but sending this first, in case there are any concerns/objections.
> > > >>>
> > > >>
> > > >> In case the use case was unknown, the primary reason for adding 
> > > >> this was to utilize this box to inform a media server of an 
> > > >> incoming live stream's bit rate, since the overall bit rate 
> > > >> cannot be calculated for something that isn't done yet 
> > > >> (https://eur02.safelinks.protection.outlook.com/?url=https%3A%2
> > > >> F%
> > > >> 2F
> > > >> dashif-documents.azurewebsites.net%2FIngest%2Fmaster%2FDASH-IF-
> > > >> In
> > > >> ge
> > > >> st.html&data=04%7C01%7C%7Cd324fb1cfab4426c652908d9f5f4e1f1%
> > > >> 7C
> > > >> 0c
> > > >> 503748de3f4e2597e26819d53a42b6%7C1%7C1%7C637811254375585151%7CU
> > > >> nk
> > > >> no
> > > >> wn%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik
> > > >> 1h
> > > >> aW
> > > >> wiLCJXVCI6Mn0%3D%7C3000&sdata=75Ua9DE4HzBSOXD6eWQ%2B422ss3Y
> > > >> Ya
> > > >> 7m
> > > >> 94impyQd7%2Bkg%3D&reserved=0 as an example of one such use 
> > > >> case). This being an alternative to nonstandard things such as ISML manifests.
> > > >>
> > > >> Additionally, since it seemed to be specified at the end of the given structures, I added it at the end of these given boxes. Apparently thus it made your live patching of that box no longer compatible with these parsers, since you just append your required things to the end of it. Am I understanding things correctly?
> > > >>
> > > >
> > > > Our code adds the sinf box after the existing stsd entry, it is correct also when btrt is present, as an evidence to this - playback works fine on all other devices.
> > > > The problem, in my understanding, is in the decoder present on 
> > > > the affected Sony TV models - it seems the decoder is making an assumption that the sinf box should come right after the esds box, while AFAIK, the spec does not mandate any order. Verbatim response we got from Sony: "btrt box in the audio chunk is causing the TV decoder to fail. TV decoder is expecting sinf box after esds box.".
> > > >
> > > > So, even though it seems the TV decoder is the culprit, since the btrt atom can cause such failures, IMHO it should be omitted by default.
> > > > Users who need the benefits this atom provides, will be able to enable this functionality, of course.
> > >
> > > Ability to turn it off is OK, changing default to not write it 
> > > does not seem right.
> > >
> > > Also you are poiting at btrt atom, but in fact sony said: "TV 
> > > decoder expecting sinf box after sdds box", so the proper fix 
> > > seems to be to change the order of the atoms, and add a big fat 
> > > warning to the code that because of some sony TV models, sinf box 
> > > must be immediately after esds box. Is this doable?
> > >
> > 
> > My understanding (feel free to correct me if I understood it
> > incorrectly) is that they are patching the written box structures afterwards, and that is now failing. So what libavformat itself is writing out in movenc actually works.
> > 
> > Which is what the expected result should be, since I think this code has been in movenc since release 4.4, I think? I would have expected that if movenc's output was not working by itself that we would have issues spawning regarding that.
> > 
> 
> Correct, the DRM encryption is applied by our packager (https://github.com/kaltura/nginx-vod-module), not by ffmpeg, and it adds `sinf` after all existing atoms in the `stsd` entry.
> According to the code in movenc, had the encryption been done by ffmpeg, the sinf atom would have been written before `btrt`, I guess that should have worked (I didn't try...).
> Changing our code to insert the `sinf` right after `esds` is of course possible too, just a bit annoying, because we currently treat the original `stsd` entry as opaque, and just append to it.
> 
> From our perspective, it was working for years, and the addition of `btrt` broke it, that is the main reason I suggested to change the default...
> But anyway, I understand you want to keep it on by default, so will submit a patch to only add the ability to disable it.
> 

Following some delay... attaching the patch.

I copied the implementation of `write_tmcd` - the new flag supports forcing btrt (for mode != mp4), as well as disabling it.

Thank you!

Eran

> Thanks,
> 
> Eran
> 
> > Jan
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel at ffmpeg.org
> > https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fff
> > mp 
> > eg.org%2Fmailman%2Flistinfo%2Fffmpeg-devel&data=04%7C01%7C%7Cd32
> > 4f
> > b1cfab4426c652908d9f5f4e1f1%7C0c503748de3f4e2597e26819d53a42b6%7C1%7
> > C1 
> > %7C637811254375585151%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLC
> > JQ 
> > IjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=XA1%2FPD
> > 2A
> > M98XB%2FYfg82u58Op5CTQT0hJIonF%2BgK8qlE%3D&reserved=0
> > 
> > To unsubscribe, visit link above, or email ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
> >
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-movenc-add-write_btrt-option.patch
Type: application/octet-stream
Size: 4913 bytes
Desc: 0001-movenc-add-write_btrt-option.patch
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20220321/55dfe05f/attachment.obj>