[FFmpeg-devel] Add Mediacodec audio decoders support

[Cosmin Stejerean]

> On Jul 16, 2024, at 4:58 PM, Zhao Zhili <quinkblack at foxmail.com> wrote:
> 
> 
> 
>> On Jul 16, 2024, at 21:20, Matthieu Bouron <matthieu.bouron at gmail.com> wrote:
>> 
>> On Wed, Jul 10, 2024 at 6:31 PM Matthieu Bouron
>> <matthieu.bouron at gmail.com> wrote:
>>> 
>>> On Wed, Jul 10, 2024 at 4:04 PM Zhao Zhili <quinkblack at foxmail.com> wrote:
>>>> 
>>>> 
>>>>> On Jun 12, 2024, at 21:42, Matthieu Bouron <matthieu.bouron at gmail.com> wrote:
>>>>> 
>>>>> Hello,
>>>>> 
>>>>> This patchset adds Mediacodec audio decoders support. Currently, only AAC, AMR,
>>>>> MP3, FLAC, VORBIS and OPUS are supported.
>>>>> 
>>>>> This is mainly useful to avoid shipping Android builds of FFmpeg that are
>>>>> subjects to licensing/patents (due to AAC and AMR).
>>>> 
>>>> I’m not keen on put OS audio decoder/encoder wrapper into FFmpeg. They don’t bring
>>>> new features, they don’t improve performance. I know these type of wrapper exist in current
>>>> project, but I’m not sure if it’s a good idea to add more.
>>> 
>>> I agree that on the technical side it doesn't bring new features nor
>>> performance improvements. It's all about avoiding licensing/patents
>>> issues with AAC and AMR here.
>>> In this specific case we already have the wrapper infrastructure, the
>>> audio part only needs small adjustments to work.
>>> Moreover, if that helps, I can reduce the scope of the patch to AAC
>>> and AMR only and get rid of mp3/flac/vorbis/opus support. What do you
>>> think ?
>> 
>> Ping.
>> 
>> IMHO, this benefits users wanting to ship an Android app that relies
>> on FFmpeg upstream in countries that are subject to AAC/AMR licensing.
>> While I agree that it's not great from a purely technical pov since we
>> already have better native decoders, it will allow the use of FFmpeg
>> in such situation without the need to use or create another FFmpeg
>> fork dedicated to Android. Plus, as I said above, we already have the
>> wrapper and the additional code to make it work for audio is
>> relatively small and scoped. Restricting the wrapper to AAC/AMR seems
>> like a good compromise to me.
> 
> Sounds reasonable, but I’d like to get more opinions. cc TC.

To add another data point, the platform decoders might also be more secure due to sandboxing. I believe as of Android Q the software decoders provided by MediaCodec have been moved to run within a constrained sandbox.

- Cosmin