[FFmpeg-devel] [PATCH] add phqm filter and img_hash

[Paul B Mahol]

On 11/1/19, Christopher Kennedy <ckennedy at ellation.com> wrote:
> On Wed, Oct 30, 2019 at 10:57 AM Paul B Mahol <onemda at gmail.com> wrote:
>>
>> On 10/30/19, Christopher Kennedy <ckennedy at ellation.com> wrote:
>> > On Wed, Oct 30, 2019 at 10:07 AM Paul B Mahol <onemda at gmail.com> wrote:
>> >>
>> >> On 10/30/19, Christopher Kennedy <ckennedy at ellation.com> wrote:
>> >> > On Sat, Oct 26, 2019 at 9:15 AM Paul B Mahol <onemda at gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> On 10/26/19, Christopher Kennedy <ckennedy at ellation.com> wrote:
>> >> >> > On Sat, Oct 26, 2019 at 8:22 AM Paul B Mahol <onemda at gmail.com>
>> >> >> > wrote:
>> >> >> >>
>> >> >> >> On 10/26/19, Christopher Kennedy <ckennedy at ellation.com> wrote:
>> >> >> >> > This is a reference/encode comparison filter with two files
>> >> >> >> > input
>> >> >> >> > like
>> >> >> >> > the psnr or vmaf filter.
>> >> >> >> > So it is completely different and uses the C++ OpenCV API
>> >> >> >> > since
>> >> >> >> > this
>> >> >> >> > img_hash library is not in the C API.
>> >> >> >> > It's unique to what the OCV filter does, and has more research
>> >> >> >> > implications from my talk at Demuxed 2019.
>> >> >> >>
>> >> >> >> I do not see how that is relevant.
>> >> >> >>
>> >> >> >> There should be generic opencv filter which could do this above
>> >> >> >> in
>> >> >> >> generic way, and not by adding yet another filter that uses only
>> >> >> >> some
>> >> >> >> part of opencv.
>> >> >> >
>> >> >> > Is it really possible to do framesync() operations like dual
>> >> >> > input
>> >> >> > filters
>> >> >> > like psnr/vmaf and also handle input/output rendering of the
>> >> >> > frames
>> >> >> > too?
>> >> >> > This sounds odd to me but I would love to understand how this is
>> >> >> > possible.
>> >> >>
>> >> >> framesync is nothing special, its just used a lot, there are video
>> >> >> filters that do not use it and still operate on multiple inputs.
>> >> >>
>> >> >> >
>> >> >> > The C OpenCV API is not recommended so this does the OpenCV part
>> >> >> > in C++ which allows it to be fully utilized and supported. So
>> >> >> > that
>> >> >> > seems
>> >> >> > better to me than using the API OpenCV won't really support and
>> >> >> > doesn't
>> >> >> > allow usage of img_hash (it is NOT in the C API of OpenCV,
>> >> >> > impossible
>> >> >> > to
>> >> >> > use).
>> >> >> >
>> >> >> > The OpenCV C++ img_hash library is the fastest implementation and
>> >> >> > does
>> >> >> > work best in OpenCV. So implementing this in C directly isn't a
>> >> >> > task
>> >> >> > I
>> >> >> > believe
>> >> >> > is good to do.
>> >> >>
>> >> >> I do not care how generic opencv filter is done, it can be C++ just
>> >> >> fine.
>> >> >>
>> >> >> >
>> >> >> > So should the current OpenCV stuff be merged into this filter, is
>> >> >> > that
>> >> >> > possible?
>> >> >>
>> >> >> Current opencv stuff in libavfilter is pretty dead and not
>> >> >> maintained
>> >> >> at
>> >> >> all.
>> >> >> So I'm not really fond of adding yet another opencv filter that not
>> >> >> gonna be maintained.
>> >> >
>> >> > Well I would maintain it, but maybe I should instead run a Fork of
>> >> > FFmpeg
>> >> > with
>> >> > C++ and OpenCV properly done.
>> >> >
>> >> > Yes the current OCV stuff is dead and not supported by OpenCV or
>> >> > maintained
>> >> > it sounds like.
>> >> >
>> >> > So if we can't put C++ into FFmpeg (yet we already have with
>> >> > decklink
>> >> > filters,
>> >> > so that is unfair to say).
>> >> >
>> >> > And we can't switch to use OpenCV C++ yet the C API is dead and not
>> >> > supported.
>> >>
>> >> Who said that?
>> >> C++ module filter in libavfilter dealing with latest OpenCV and well
>> >> maintained is always welcome.
>> >
>> >
>> > So I should be moving the functionality of the current ocv filter into
>> > a
>> > C++ file for filters to use like phqm, the current ocv transforms?
>> >
>> > Is this a good plan, separate the C++ from the vf_ filter as I have
>> > done
>> > with img_hash.cpp (yet make it a more generic set of OpenCV C++
>> > helper functions to plug into C vf_filters which can be separated by
>> > function yet unified in code used from the C++ base code)?
>>
>> IMHO filter file can be cpp file without issues. And even if not use
>> ff_ prefixed functions,
>
> Okay how about this layout...
>
> libavfilter/opencv.cpp (all OpenCV code, using C++ API, helper
> functions for use by the filters)
>
> libafilter/vf_libopencv.c   current filter, but uses opencv.cpp
> functions for OpenCV code
>
> libavfilter/vf_phqm.c       uses OpenCV functions from opencv.cpp

This also should be in libopencv filter.

>
> Since OpenCV ranges in varying functions we may need, and doing
> it as C++ cleanly separately in one place sharing with the C filters seems
> nice.
>
> I agree the duplication of OpenCV in C and C++ APIs seems bad, and
> using just the C++
> API is what I see as best according to the OpenCV devs statements
> against using the C API.
>
>
> Is that acceptable or is a better layout suggested?
>
> Thanks,
> Christopher
>
>>
>> >
>> > Wanting to clarify exactly what is acceptable so I can do it and not
>> > waste
>> > time / effort doing stuff that isn't right.
>> >
>
>> >> >
>> >
>