[FFmpeg-devel] Add Mediacodec audio decoders support

[Cosmin Stejerean]

> On Jul 17, 2024, at 5:39 PM, Paul B Mahol <onemda at gmail.com> wrote:
> 
> On Tue, Jul 16, 2024 at 5:56 PM Cosmin Stejerean via ffmpeg-devel <
> ffmpeg-devel at ffmpeg.org> wrote:
> 
>> 
>> 
>>> On Jul 16, 2024, at 4:58 PM, Zhao Zhili <quinkblack at foxmail.com> wrote:
>>> 
>>> 
>>> 
>>>> On Jul 16, 2024, at 21:20, Matthieu Bouron <matthieu.bouron at gmail.com>
>> wrote:
>>>> 
>>>> On Wed, Jul 10, 2024 at 6:31 PM Matthieu Bouron
>>>> <matthieu.bouron at gmail.com> wrote:
>>>>> 
>>>>> On Wed, Jul 10, 2024 at 4:04 PM Zhao Zhili <quinkblack at foxmail.com>
>> wrote:
>>>>>> 
>>>>>> 
>>>>>>> On Jun 12, 2024, at 21:42, Matthieu Bouron <
>> matthieu.bouron at gmail.com> wrote:
>>>>>>> 
>>>>>>> Hello,
>>>>>>> 
>>>>>>> This patchset adds Mediacodec audio decoders support. Currently,
>> only AAC, AMR,
>>>>>>> MP3, FLAC, VORBIS and OPUS are supported.
>>>>>>> 
>>>>>>> This is mainly useful to avoid shipping Android builds of FFmpeg
>> that are
>>>>>>> subjects to licensing/patents (due to AAC and AMR).
>>>>>> 
>>>>>> I’m not keen on put OS audio decoder/encoder wrapper into FFmpeg.
>> They don’t bring
>>>>>> new features, they don’t improve performance. I know these type of
>> wrapper exist in current
>>>>>> project, but I’m not sure if it’s a good idea to add more.
>>>>> 
>>>>> I agree that on the technical side it doesn't bring new features nor
>>>>> performance improvements. It's all about avoiding licensing/patents
>>>>> issues with AAC and AMR here.
>>>>> In this specific case we already have the wrapper infrastructure, the
>>>>> audio part only needs small adjustments to work.
>>>>> Moreover, if that helps, I can reduce the scope of the patch to AAC
>>>>> and AMR only and get rid of mp3/flac/vorbis/opus support. What do you
>>>>> think ?
>>>> 
>>>> Ping.
>>>> 
>>>> IMHO, this benefits users wanting to ship an Android app that relies
>>>> on FFmpeg upstream in countries that are subject to AAC/AMR licensing.
>>>> While I agree that it's not great from a purely technical pov since we
>>>> already have better native decoders, it will allow the use of FFmpeg
>>>> in such situation without the need to use or create another FFmpeg
>>>> fork dedicated to Android. Plus, as I said above, we already have the
>>>> wrapper and the additional code to make it work for audio is
>>>> relatively small and scoped. Restricting the wrapper to AAC/AMR seems
>>>> like a good compromise to me.
>>> 
>>> Sounds reasonable, but I’d like to get more opinions. cc TC.
>> 
>> To add another data point, the platform decoders might also be more secure
>> due to sandboxing. I believe as of Android Q the software decoders provided
>> by MediaCodec have been moved to run within a constrained sandbox.
>> 
> 
> "May".
> Need citations and exact repeatable scientific proof.
> 

Citation https://android-developers.googleblog.com/2019/05/queue-hardening-enhancements.html#:~:text=In%20Android%20Q%2C%20we%20moved,components%20into%20less%20privileged%20sandboxes.

See the section titled "A Constrained Sandbox for Software Codecs"

- Cosmin