[Libav-user] Hardware decoding on iOS devices ...... VideoToolbox .. how to know?

[JP May]

Howdy,

we’re doing a project

- using C with ffmpeg compiled in,
- and this is running on iOS devices.

There seems to be a number of problems with ensuring that hardware decoding is used.

In one example, we have a file with three streams muxed,

- an 8 bit YUV stream
- a 10 bit YUV stream
- a data stream

Here’s the two video streams:

Stream #0:0[0x100], 21, 1/90000: Video: hevc (Main), 1 reference frame ([36][0][0][0] / 0x0024), yuv420p(tv), 1536x1024 [SAR 1:1 DAR 3:2], 0/1, 30 fps, 300 tbr, 90k tbn, 30 tbc

Stream #0:1[0x101], 21, 1/90000: Video: hevc (Main 10), 1 reference frame ([36][0][0][0] / 0x0024), yuv420p10le(tv), 1536x1024 [SAR 1:1 DAR 3:2], 0/1, 30 fps, 300 tbr, 90k tbn, 30 tbc


MY QUESTION IS THIS:

Is there a way to ABSOLUTELY KNOW whether hardware or software decoding was used?

In an iOS setting, after decoding there are TWO possibilities (we know about):

	(result 1) data[0] is NULL, and in data[3] you can find an AVFrame

secondly

	(result 2) data[0] data[1] data[2] hold the Y, U, V planar data, data[3] is null



My question is simply: How do you ABSOLUTELY KNOW whether hardware or software decoding was used?

Is it ABSOLUTELY the case that result-1 means it was hardware decoded?

Is it ABSOLUTELY the case that result-2 means it was software decoded?




Here are some more details of this mystery for anyone with the time:

With a brand-new XR iPhone, with today’s latest OS,

it appears to NOT do hardware decoding on the 8bit channel -   you get

		“result 2” for the 8bit channel
		“result 1” for the 10bit channel.

There would seem to be no reason a new XR would do
 - hardware decoding for a 10but YUV channel, but
 - software decoding for an 8bit YUV channel

in contrast, with for example an older iPhone8 it depends on the OS. 

Sometimes you do get ”result 2”/"result 2” 
and you sometimes get ”result 1”/"result 2” 

(So, with the new fast hardware you get ”software”/”hardware" whereas with certain old phones you get ”hardware”/“hardware”)

Also, even when a device is giving ”result 2”/"result 2” , it is often suspiciously slow rate.

(We are not even drawing to screen - just decoding.)

Further, there seems to be no reason at all that a given device would do hardware on a 10bit channel and software on an 8bit channel.

Again my basic question is:

is there absolutely a way to know whether hardware or software decoding was used?

Looking at these two possibilities:

	(result 1) data[0] is NULL, and in data[3] you can find an AVFrame

secondly

	(result 2) data[0] data[1] data[2] hold the Y, U, V planar data, data[3] is null

is it >> ABSOLUTELY << the case that if you get an AVFrame in data[3] it was definitely hardware decoding?

Conversely, is it >> ABSOLUTELY << the case that YUV in [1, 2, 3] means it was software decoding?


Any help greatly appreciated !

It is a real mystery currently!



















-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ffmpeg.org/pipermail/libav-user/attachments/20190124/7e39893a/attachment.html>