[FFmpeg-devel] [PATCH] avutils/hwcontext: When deriving a hwdevice, search for existing device in both directions

[Soft Works]

> -----Original Message-----
> From: ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> On Behalf Of
> Hendrik Leppkes
> Sent: Saturday, 7 August 2021 08:52
> To: FFmpeg development discussions and patches <ffmpeg-
> devel at ffmpeg.org>
> Subject: Re: [FFmpeg-devel] [PATCH] avutils/hwcontext: When deriving a
> hwdevice, search for existing device in both directions
> 
> On Sat, Aug 7, 2021 at 3:46 AM Soft Works <softworkz at hotmail.com> wrote:
> >
> > The test /libavutil/tests/hwdevice checks that when deriving a device
> > from a source device and then deriving back to the type of the source
> > device, the result is matching the original source device, i.e. the
> > derivation mechanism doesn't create a new device in this case.
> >
> > Previously, this test was usually passed, but only due to two
> > different kind of flaws:
> >
> > 1. The test covers only a single level of derivation (and back)
> >
> > It derives device Y from device X and then Y back to the type of X and
> > checks whether the result matches X.
> >
> > What it doesn't check for, are longer chains of derivation like:
> >
> > CUDA1 > OpenCL2 > CUDA3 and then back to OpenCL4
> >
> > In that case, the second derivation returns the first device (CUDA3 ==
> > CUDA1), but when deriving OpenCL4, hwcontext.c was creating a new
> > OpenCL4 context instead of returning OpenCL2, because there was no
> > link from CUDA1 to OpenCL2 (only backwards from OpenCL2 to CUDA1)
> >
> > If the test would check for two levels of derivation, it would have
> > failed.
> >
> > This patch fixes those (yet untested) cases by introducing forward
> > references (derived_device) in addition to the existing back
> > references (source_device).
> >
> 
> I already see one problem here, if you have so many derived cases
> happening, its also feasible to assume one source was used to derive more
> then one device, which this cannot track, and in fact looks like the code
> would just override and leak a reference.
> 
> How common would it be that such complex derived chains happen, when
> you say even this one is untested?

I'm aware of what you are pointing at, but I couldn't easily construct a case
where this would have an impact.

What we need to consider here is that this is about derivation of HW device
contexts - not frame contexts.

In case of frame contexts, we can have pretty long chains of derivation between
decoder, filters [0-n] and encoder, where we always have a separate HW frames
context in-between. For device contexts, it's instead about creating and 
maintaining just a single one for each device. 

From a viewpoint of functionality, it's probably only a theoretical issue, but 
it might rarely happen that it gets overwritten. 
Even though it might appear to be overkill, but for a perfect solution, I'd need
to maintain an array of derived_devices.

Shall I do it?

softworkz