[FFmpeg-devel] How to do (HEVC) decoder fallback?

[John Cox]

Hi

I have an HEVC decoder built from the standard ffmpeg hevc decoder.  It
has been heavily optimised for the Raspberry Pi and uses the support
processors (QPU & VPU) of that chip to achieve plausible speed (on a Pi3
it can normally decode 10Mbit/sec 30fps 8-bit 4:2:0 1080p and has a
decent go at 10-bit 1080p but you will need some overclock to get
reliable 30fps)

It only supports 8 & 10bit, 4:2:0 HEVC with a max width of 2048, and
ouputs frames in a somewhat odd Broadcom format (sand) which doesn't fit
any of the existing FFmpeg models as it is arranged in 128 byte wide
vertical stripes rather than any sort of planar format.  I also have a
few functions that deal with sand conversion to raw 420 for conformance
testing.

What I want to do is to add this in such a way that ffmpeg will use it
if the incoming stream is one it can deal with but will fall back to the
standard hevc decoder if it can't.

I've looked at the h/w accel route, but at first sight (I'll admit to
becoming quite confused here) that appears to (a) want the hwaccel to
produce the same format frames as the base deecoder would (which it
doesn't) and (b) to use the same vps/sps/pps processing as the base
decoder (and I've modified that a bit).  What I would really like is for
there to be some sort of fallback route for software decoders that share
the same AVCodecID s.t. if one fails init then the next one is tried but
that doesn't seem to be possible with the current setup.  Am I missing
something?

As it stands the code is built into the main hevc decoder code with a
lot of ifdefs & if (rpi_enable), but I think it would be better off in
its own decoder.  If you want to look at the current state of the art
then you can find it in https://github.com/jc-kynesim/rpi-ffmpeg.git on
branch test/wpp_1 - I do have a separated decoder version but I'd like
to find out how I should integrate it before I commit it.

Many thanks

John Cox