[FFmpeg-devel] Add Mediacodec audio decoders support

[Martin Storsjö]

On Wed, 17 Jul 2024, Anton Khirnov wrote:

> Quoting Cosmin Stejerean via ffmpeg-devel (2024-07-16 22:14:19)
>> 
>> 
>> > On Jul 16, 2024, at 8:24 PM, Rémi Denis-Courmont <remi at remlab.net> wrote:
>> > 
>> > Le tiistaina 16. heinäkuuta 2024, 18.48.06 EEST Cosmin Stejerean via ffmpeg-
>> > devel a écrit :
>> >> To add another data point, the platform decoders might also be more secure
>> >> due to sandboxing. I believe as of Android Q the software decoders provided
>> >> by MediaCodec have been moved to run within a constrained sandbox.
>> > 
>> > Platform decoders are in all likelihood strictly less secure than software 
>> > decoders. Software decoders will run in a user-space sandboxed within their 
>> > respective application. Platform decoders will run in a more privileged system 
>> > service, with direct access to a kernel driver in EL1, through that to the 
>> > firmware running on the video DSP.
>> > 
>> > More performant and energy-efficient. But also way way less secure.
>> > 
>> > The only viewpoint whence this is more secure, is the content publisher's: 
>> > this model enables DRM with hardware pass-through (but that does not even 
>> > apply if you use FFmpeg as the front end).
>> > 
>> 
>> Platform provided *software* decoders should be more secure than bundled software decoders due to the sandboxing of software decoders in recent versions of Android.
>
> If that is such an important feature to someone then it is not
> inconceivable to implement some sort of sandboxing inside avcodec.
>
> I'm not a big fan of the argument "we should provide passthrough to
> proprietary decoders because they are more secure".

My 2 cents on this matter: I don't care much about the arguments about 
more secure or less secure here - I don't see that as affecting the 
decision either way.

We generally don't add wrappings of third party proprietary 
encoders/decoders.

But for codecs shipped as part of the OS, I don't see an issue with us 
allowing accessing those codecs. It's not like we're favouring any 
specific third party, we're just facilitating access to whatever is 
already there.

And especially in this case, we already have the general code for 
accessing the MediaCodec API for codecs on Android - I don't have a 
problem with extending this to a few more codecs, as that's not much more 
than just providing mappings to the codec identifiers.

Similarly, we already allow decoding and encoding of a long range of 
codecs via AudioToolbox on Apple platforms - I don't see MediaCodec on 
Android as any different than that.

// Martin