[FFmpeg-devel] Mixture of homographies video stabilization

[Michael Niedermayer]

On Mon, Aug 06, 2018 at 03:49:00PM +0100, Matthew Lai wrote:
> Sorry the matrices are actually 3x3, and most of the time will be spent on
> solving a 8x9 * 9x1 = 0 systems. And then all the feature points will have
> to be multiplied by the 9x1 matrix reshaped into 3x3.
> 
> The math is here:
> https://www.uio.no/studier/emner/matnat/its/UNIK4690/v16/forelesninger/lecture_4_3-estimating-homographies-from-feature-correspondences.pdf
> 

> Multiplying 3x3 matrices is trivial, but SVD isn't. SVD is the standard
> approach to decompose a homography matrix. I have not looked into
> alternative approaches.

Iam a bit puzzled.
IIUC SVD is the slowest part of the filter from what you write above and we
have no evidence that it is needed. And you did not look at alternatives ? ;)

Quoting "Calibration-Free Rolling Shutter Removal" from
   Matthias Grundmann1,2           Vivek Kwatra1      Daniel Castro2         Irfan Essa1,2
...
    Aggregating all linear constraints Ax for each feature match
    (x, y) yields an homogenous linear system, which can be
    solved for under the constraint ||h||2 = 1 using the SVD
    of A. Alternatively, the system can be transformed into a
    homogenous system by explicitly setting the bottom right
    element of each homography to 1, i.e. hk (3, 3) = 1 for all k,
    which is a reasonable choice for video, as the small inter-
    frame motions are virtually free of degenerated cases.

this sounds like theres a very easy way to avoid
SVD.
from what i remember the main point of
SVD is its ability to work with unstable / degenerate cases.
But, in which case is such a degenrate case a good estimation for the
motion/transform ?    
    

> 
> Cool! Will see re. ME. It's likely to be the performance-limiting part, and
> it would be nice to run that on GPU.

just a quick reference point, ME for encoding is many times real time.
for example a 720x576 video encodes here with 1227 fps on a many years old
box. That includes alot of other things like decoding and all the bitstream
encoding and transforms, not just ME. And this is 100% on the CPU

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

I have never wished to cater to the crowd; for what I know they do not
approve, and what they approve I do not know. -- Epicurus
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20180807/e7559edb/attachment.sig>