[FFmpeg-devel] [PATCH] add signature filter for MPEG7 video signature

[Gerion Entrup]

On Montag, 21. März 2016 11:53:27 CET Thilo Borgmann wrote:
> Am 21.03.16 um 00:14 schrieb Gerion Entrup:
> > On Sonntag, 20. März 2016 17:01:17 CET Thilo Borgmann wrote:
> >>> On Sun, Mar 20, 2016 at 12:00:13PM +0100, Gerion Entrup wrote:
> >> [...]
> >> 
> >>> This filter does not implement all features of MPEG7. Missing features:
> >>> 
> >>> - binary output
> >>> - compression of signature files
> >> 
> >> I assume these features are optional?
> > 
> > Compression is optional (could be set as flag in the binary
> > representation). I have not found, whether binary output is optional.
> > 
> > It is definitely possible to only work with the XML-Files.
> 
> Of course, but having an unspecified XML output is almost useless if binary
> output is not optional. So I think it is crucial to know what the spec says
> about output.
The spec defines the XML output my filter do atm and specifies a binary output 
additional.
> 
> >>> - work only on (cropped) parts of the video
> >> 
> >> How useful is this then? Has fingerprint computed only on (cropped) parts
> >> of the video any value outside of FFmpeg itself - does this comply to
> >> the spec so that it can be compared with any other software generating
> >> it?
> > 
> > To clarify, the filter does not crop anything. The standard defines an
> > optional cropping to, I guess, concentrate on specific video parts (this
> > is not implemented). Assuming someone is recording a monitor, then e.g.
> > the unrelated part of the video could be cropped out. Beside that, the
> > signature itself is invariant to cropping up to a certain limit.
> > 
> > The cropping values (upper left and bottom right position are specified in
> > the xml, so another software could either crop the same way or compare
> > only with the cropped input.
> > (The fact, that ffmpeg has a cropping filter, would make such a feature
> > some kind of redundant.)
> 
> If I understand it correctly, the filter should not crop the image but only
> use the pixel information within the specified area or the whole image.
> Making it a filter option is useful, because the fingerprint of a part of
> the image can be used in a filter chain continuing with the entire image
> (no actual crop is required).
Of course it would be great, if the filter would support it. It needs some
modification to the summed area table. Once the binary output is ready, I will
try to do it.

> 
> > The XML is standard compliant.
> 
> So XML output is compliant to the spec? Or is the XML itself just valid XML?
The XML output is compliant to the spec. The whole format is specified there.

> > The signature is not bitexact. 3-4 (ternary)
> > values in the framesignature differ from the signature of the sample
> > files, but the conformence tests [1] allow up to 15 ternaryerrors.
> 
> Bitexact compared to what?
The institute, where I write the filter, owns the sample files mentioned in the 
doc together with the correspondent binary and XML signatures (so I could 
compare it).

> Does it allow up to 15 ternary errors for assume two inputs are equal enough
> to be the same image or does it state that the fingerprint itself may
> differ for 15 ternary errors for the very same image?
The 15 tenary errors are valid between the sample signature and the 
reimplementation for the same sample.

But ok, you seem to be right, the binary representation seems to be necessary. 
I quote (out of the linked doc):
"The number of dimensions whose ternary values differ between the test and the 
reference video signatures shall be less than or equal to 15 out of 380, if 
the FrameConfidence values of both the test and the reference video signatures 
are greater than or equal to 4. The ternary values of the frame signature 
shall be decoded from the binary representation according to Table E.1."

At the moment the XML output of my filter comparing to the given XML output of 
the sample file fulfills the condition (except of the last sentence of course).

I'll try to implement the binary output as well, controlled by a flag.

> 
> > I'm not exactly sure
> > why the difference occur, I guess, this are floating point issues, because
> > my implementation calculates the values with a slightly different method
> > than the reference software. Currently, I don't have the sample files
> > anymore. If there is interest, I will try to get them again.
> > 
> > [1] https://www.itscj.ipsj.or.jp/sc29/open/29view/29n11696t.doc
> 
> Being compliant to the reference output is crucial, otherwise we calculate
> just some fingerprint but not the MPEG7 fingerprint... don't you think?
The fingerprint is definitly useful, given the other program could parse the XML
specified in the spec.

> 
> -Thilo
> 
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel