[FFmpeg-devel] Add Mediacodec audio decoders support

[Zhao Zhili]

> On Jul 16, 2024, at 23:48, Cosmin Stejerean via ffmpeg-devel <ffmpeg-devel at ffmpeg.org> wrote:
> 
> 
> 
>> On Jul 16, 2024, at 4:58 PM, Zhao Zhili <quinkblack at foxmail.com> wrote:
>> 
>> 
>> 
>>> On Jul 16, 2024, at 21:20, Matthieu Bouron <matthieu.bouron at gmail.com> wrote:
>>> 
>>> On Wed, Jul 10, 2024 at 6:31 PM Matthieu Bouron
>>> <matthieu.bouron at gmail.com> wrote:
>>>> 
>>>> On Wed, Jul 10, 2024 at 4:04 PM Zhao Zhili <quinkblack at foxmail.com> wrote:
>>>>> 
>>>>> 
>>>>>> On Jun 12, 2024, at 21:42, Matthieu Bouron <matthieu.bouron at gmail.com> wrote:
>>>>>> 
>>>>>> Hello,
>>>>>> 
>>>>>> This patchset adds Mediacodec audio decoders support. Currently, only AAC, AMR,
>>>>>> MP3, FLAC, VORBIS and OPUS are supported.
>>>>>> 
>>>>>> This is mainly useful to avoid shipping Android builds of FFmpeg that are
>>>>>> subjects to licensing/patents (due to AAC and AMR).
>>>>> 
>>>>> I’m not keen on put OS audio decoder/encoder wrapper into FFmpeg. They don’t bring
>>>>> new features, they don’t improve performance. I know these type of wrapper exist in current
>>>>> project, but I’m not sure if it’s a good idea to add more.
>>>> 
>>>> I agree that on the technical side it doesn't bring new features nor
>>>> performance improvements. It's all about avoiding licensing/patents
>>>> issues with AAC and AMR here.
>>>> In this specific case we already have the wrapper infrastructure, the
>>>> audio part only needs small adjustments to work.
>>>> Moreover, if that helps, I can reduce the scope of the patch to AAC
>>>> and AMR only and get rid of mp3/flac/vorbis/opus support. What do you
>>>> think ?
>>> 
>>> Ping.
>>> 
>>> IMHO, this benefits users wanting to ship an Android app that relies
>>> on FFmpeg upstream in countries that are subject to AAC/AMR licensing.
>>> While I agree that it's not great from a purely technical pov since we
>>> already have better native decoders, it will allow the use of FFmpeg
>>> in such situation without the need to use or create another FFmpeg
>>> fork dedicated to Android. Plus, as I said above, we already have the
>>> wrapper and the additional code to make it work for audio is
>>> relatively small and scoped. Restricting the wrapper to AAC/AMR seems
>>> like a good compromise to me.
>> 
>> Sounds reasonable, but I’d like to get more opinions. cc TC.
> 
> To add another data point, the platform decoders might also be more secure due to sandboxing. I believe as of Android Q the software decoders provided by MediaCodec have been moved to run within a constrained sandbox.

I don’t think what Android says “secure” apply to use. The sandbox means not to
break the OS and leak information from one APP to another APP, because mediacodec
service is shared by all APP. An APP with FFmpeg inside and use software decoder/encoder
don’t have such issues since the APP don’t share decoders/encoders with other APP.

> 
> - Cosmin
> 
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".