[FFmpeg-devel] FFmpeg MT and HWACCEL mess

[Reimar Döffinger]

On 27 Mar 2011, at 21:29, Michael Niedermayer <michaelni at gmx.at> wrote:
> On Sun, Mar 27, 2011 at 08:37:26PM +0200, Reimar Döffinger wrote:
>> On Sun, Mar 27, 2011 at 07:59:35PM +0200, Michael Niedermayer wrote:
>>> On Sun, Mar 27, 2011 at 02:40:58PM +0200, Reimar Döffinger wrote:
>>>> I just had to realize that due all the new thread code in avcodec_open
>>>> making the HWACCEL stuff anything than a messy, broken hack has become
>>>> even harder.
>>>> Since it both initializes the threads and skips calling init, that means
>>>> whether to use threads or not has to be decided during init.
>>>> However we do not have any data available during init, so we have no chance
>>>> of testing whether the hardware supports the given input, nor can we use
>>>> get_format to decide which if any hwaccel to use.
>>>> This also pretty much puts a block on implementing a codec what would
>>>> automatically decide between hwaccel with readback and software decoding.
>>>> Any ideas or should I just finally take the hint and give up on FFmpeg
>>>> supporting hardware decoding in a way that isn't a PITA?
>>> 
>>> How would you sugest that ffmpeg should handle it?
>> 
>> I've said from the beginning that I consider the whole CODEC_CAP thing
>> a misadventure born of laziness.
>> However my early attempts to change it got no real reaction and more
>> and more code is added that makes things more and more brittle and
>> thus harder to actually improve anything.
> 
> 
> Let me just make sure i understand you correctly as ive been unable
> to find much with a search for reimar & CODEC_CAP
> You want a single codec that automatically uses hw/sw/threads as
> is best for the video at hand ?

In the longer term probably yes. In the shorter term I want a single codec where there already existing get_format can be used to select what to do (perfectly fine to keep the current system in addition though).
As an extension to that, I think the codec should then do also the necessary setup like disabling threading if hwaccel is selected, or disabling reordering in case of XvMC etc.
There are many other things to do, however just the functionality of the codec to say: here's all the formats we can decode this specific file to (hwaccel or not), please chose one is something I consider a first step.

> 
>>> And a decoder that switches between hw and multiframe-threaded software
>>> can be done with a codec that switches between 2 codec instances.
>>> It still has the problem that the 2 variants have different delay
>>> (multiframe software has higher delay) and this should cause problems
>>> with switching independant of implementation
>> 
>> First, there is no real problem in adding delay, even if it is probably
>> not a good idea.
>> However I don't care so much about runtime switching, just the case
>> "I want it to use the hardware when it is available and not when it is
>> not" working without 100 lines that exist for no other reason than a
>> bad API would be great!
> 
> we could have a h264auto codec that tries to open the various h264
> decoders in order of fastest to slowest until one succeeds.

The disadvantage is that it leaves the application no choice.
E.g. an application might run on a systems that supports both VA-API and VDPAU and it too might support both, but e.g. it might always want to use VDPAU because it is better tested.
FFmpeg's opinion of what is best is absolutely not helpful.
Thus IMO unless you can offer all options up-front it seems likely to me that would be the typical "dead-end" API. You start writing your program with it and once it gets nor complex you wish its developers to hell because you have to rip it out again.

> 
>> For example how is an application even supposed to find that hardware-accelerated
>> codec?
>> Just trawling through the list and looking for both a matching codec id and
>> CODEC_CAP works, as does hardcoding the name but both sure aren't a sensible way!
> 
> a avcodec_find_decoder2(id, cap)
> could be added
> 
> maybe you could suggest a API that would be easier to use for you?

Well, as said, get_format presents a list of what it can provide, I can select it.
I guess it can be done by a wrapper-codec and it might be simple, but to me that feels like a cheap excuse to avoid cleaning up the decoder code (as in initializing threading long before we actually need it etc.).
Now the crystalHD stuff is something I haven't really thought about, but it is simpler since for a well-programmer user it should be transparent so we should be able to get away with FFmpeg choosing on its own + a "force off" flag or so.
>