[FFmpeg-devel] A few filter questions

[Clément Bœsch]

On Thu, Jul 17, 2014 at 12:33:41PM +0200, Gerion Entrup wrote:
> Good day,
> 
> I'm currently working on a video signature filter for ffmpeg. This allows you to 
> fingerprint videos.

Oh, nice.

> This fingerprint is built up of 9mb/s of bits or 2-3 mb/s bits compressed.
> 
> In this context a few questions come into my mind:
> - Should I print this whole bitstream to stdout/stderr at the end? Is it maybe 
> a better choice to made an own stream out of this. But which type of stream 
> this is?

How does the fingerprint looks like? Could it make sense as a gray video
output fractal, or maybe some kind of audio signal?

Also, you still have the string metadata possibility (git grep SET_META
libavfilter).

>   (btw, the video signature algorithm needs 90 following frames, so I can 
> theoretically write every 90 frames something somewhere.)

Do you cache all these frames or just update your caches/stats & drop
them?

> - If I print the whole bitstream to stdout/stderr (my current implementation), 
> is there a possibility to use this later in an external program? The only 
> other globally analyze filter I found is volumedetect. This filter at the end 
> prints per print_stats the calculated results to the console. Is there a 
> possibility within the API for an external program to use these values or do I 
> have to grep the output?

stdout/stderr really isn't a good thing. Using metadata is way better
because you can output them from ffprobe, and parse them according to
various outputs (XML, CSV, JSON, ...).

Another solution I can now think of is to simply pass an output file as
option to the filter. That's typically how we do the 2-pass thing with
vidstab filter.

[...]
> Another thing that came into my mind: Can filter force other filters to go into 
> the filterchain? I see it, when I force GREY_8 only in my filter, it 
> automatically enables the scale filter, too.

Some filter are inserted automatically for conversion & constraints, but
that's not decided by the filters but the framework itself.

>                                              The reason I asked is the lookup 
> for my filter. Currently my filter analyzes a video and then produces a lot of 
> numbers. To compare two videos and decide, wheather they match or not, these 
> numbers has to be compared. I see three possibilities:
> 1. Write an VV->V filter. Reimplement (copy) the code from the V->V signature 
> filter and give a boolean as output (match or match not).
> 2. Take the V->V filter and write a python (or whatever) script that fetch the 
> output and calculates then the rest.
> 3. Write an VV->V filter, but enforce, that the normal signature filter is 
> executed first to both streams, use the result and then calculate the matching 
> type. Unfortunately I have no idea, how to do this and whether it is possible 
> at all. Can you give me an advice?
> 

So if you output a file in the filter itself:
  ffmpeg -i video   -vf fingerprint=video.sig -f null -
  ffmpeg -i another -vf fingerprint=video.sig:check=1 -f null -

Or if you save the signature "stream" in a video (in gray8 for instance):
  ffmpeg -i video   -vf fingerprint -c:v ffv1 sig.nut
  ffmpeg -i another -i sig.nut -vf '[0][1] fingerprint=mode=check' -f null -

The 2nd method is "better" because it doesn't require file handling in the
library, and it also allows stuff like using a diff filter (if you also
apply fingerprint - not with mode=check - on `another`)

Am I understanding right your wondering?

> The last possibility also would allow something like twopass volume 
> normalisation. Currently there is a volumedetect and volume filter. To 
> normalize once could run volumedetect, then fetch the output, and put the 
> values into the volume filter, but I currently don't see a way to do this 
> automatically directly in ffmpeg.

Check tools/normalize.py, it's using ebur128 and the metadata system.

> 
> (Once the filter is in a good state, I will try to bring it upstream.)
> 

Cool

> Best,
> Gerion
> 

-- 
Clément B.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 473 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20140717/47c93b71/attachment.asc>