[FFmpeg-devel] [GSoC] Motion Interpolation

[Michael Niedermayer]

On Sat, Aug 13, 2016 at 12:18:56PM +0000, Davinder Singh wrote:
> On Thu, Aug 11, 2016 at 12:10 AM Davinder Singh <ds.mudhar at gmail.com> wrote:
> 
> > [...]
> >
> > latest changes:
> https://github.com/dsmudhar/FFmpeg/blob/dev/libavfilter/vf_minterpolate.c
> uses shared motion estimation code now, added options, improved vsbmc
> i tried to make filter options as flexible as possible so that multiple
> algorithms are be supported.
> 
> @Ronald:
> have a look:
> https://github.com/dsmudhar/FFmpeg/blob/dev/libavfilter/motion_estimation.c
> i think if penalty factor can be moved into cost function, motion
> estimation can be shared with encoders. we can start work on this after
> GSoC?
> 
> TODO:
> frame border motion estimation.
> add scene change threshold. roughness check doesn't work so well and
> introduce artifacts.
> add docs.
> 
> 
> > here's another idea: dynamic block size selection for MC-FRUC
> > since it's not video encoding, using 16x16 block with fixed search window
> > may not work same for all resolution videos. what if we automatic resize
> > block depending on resolution? like if 16x16, P=20 works fine for 1280x720
> > video, we can scale it according to width, e.g for 1920x1080 which 1.5x
> > 1280, we use 24x24 block and also scale P accordingly? i haven't tested it
> > yet though.
> >
> 
> i tested this. quality was improved with 1080p but not with smaller
> resolution.
> 
> I tried to scale best settings of 720p. UMH. 1080p same video.
> scale nothing: mb16 p18
> stddev:    1.16 PSNR: 46.80 MAXDIFF:  197 bytes:1085529600/1073088000
> scale search window: mb16, p27
> stddev:    1.21 PSNR: 46.47 MAXDIFF:  193 bytes:1085529600/1073088000
> scale both: mb24 p18
> stddev:    1.14 PSNR: 46.93 MAXDIFF:  181 bytes:1085529600/1073088000
> 
> ESA
> mb16 p16:
> stddev:    1.18 PSNR: 46.65 MAXDIFF:  181 bytes:1085529600/1073088000
> mb24 p24:
> stddev:    1.16 PSNR: 46.77 MAXDIFF:  181 bytes:1085529600/1073088000
> 
> 640p ESA
> m16 p16:
> stddev:    1.01 PSNR: 47.97 MAXDIFF:  160 bytes:119577600/118540800
> scale p: mb16 p8:
> stddev:    1.02 PSNR: 47.95 MAXDIFF:  148 bytes:119577600/118540800
> scale both: m8 p8:
> stddev:    1.05 PSNR: 47.63 MAXDIFF:  187 bytes:119577600/118540800
> 
> i think quality can be further improved, generated test window weights were
> not perfect.
> should i keep this feature? since block-size won't be log2 int, that will
> break vsbmc which use quadtree division for smaller blocks.
> 
> 
> > [1]: JVT-F017.pdf by Z Chen <http://akuvian.org/src/x264/JVT-F017.pdf.gz>
> >
> 
> 
> On Thu, Aug 11, 2016 at 9:09 PM Paul B Mahol <onemda at gmail.com> wrote:
> 
> > Could you please squash your commits and attach patches that add
> > vf_mestimate
> > and vf_minterpolate filters?
> >
> 
> patch attached.

>  doc/filters.texi                |   25 
>  libavfilter/Makefile            |    2 
>  libavfilter/allfilters.c        |    2 
>  libavfilter/motion_estimation.c |  451 +++++++++++++
>  libavfilter/motion_estimation.h |   76 ++
>  libavfilter/vf_mestimate.c      |  376 +++++++++++
>  libavfilter/vf_minterpolate.c   | 1332 ++++++++++++++++++++++++++++++++++++++++
>  7 files changed, 2264 insertions(+)
> c09f35d9a5d564abb4dc3227872fd757c493ccb0  0001-added-motion-estimation-and-interpolation-filters.patch
> From 19fa6a20f017bf4712ca8a63126e59552efdeb9c Mon Sep 17 00:00:00 2001
> From: dsmudhar <ds.mudhar at gmail.com>
> Date: Sat, 2 Apr 2016 23:33:17 +0530
> Subject: [PATCH] added motion estimation and interpolation filters

this builds with some compiler warnings, please fix them

here gcc produces:
libavfilter/vf_mestimate.c: In function ‘filter_frame’:
libavfilter/vf_mestimate.c:147:15: warning: unused variable ‘mv_y’ [-Wunused-variable]
libavfilter/vf_mestimate.c:147:9: warning: unused variable ‘mv_x’ [-Wunused-variable]
libavfilter/vf_minterpolate.c:1166:22: warning: "VSBMC_T1" is not defined [-Wundef]
libavfilter/vf_minterpolate.c: In function ‘search_mv’:
libavfilter/vf_minterpolate.c:417:15: warning: unused variable ‘mb_i’ [-Wunused-variable]
libavfilter/vf_minterpolate.c: In function ‘bilateral_me’:
libavfilter/vf_minterpolate.c:476:9: warning: unused variable ‘count’ [-Wunused-variable]
libavfilter/vf_minterpolate.c:475:9: warning: unused variable ‘changed’ [-Wunused-variable]
libavfilter/vf_minterpolate.c:474:15: warning: unused variable ‘mv_y’ [-Wunused-variable]
libavfilter/vf_minterpolate.c:474:9: warning: unused variable ‘mv_x’ [-Wunused-variable]
libavfilter/vf_minterpolate.c: In function ‘var_size_bme’:
libavfilter/vf_minterpolate.c:563:5: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
libavfilter/vf_minterpolate.c:563:15: warning: unused variable ‘T’ [-Wunused-variable]
libavfilter/vf_minterpolate.c: In function ‘inject_frame’:
libavfilter/vf_minterpolate.c:700:23: warning: variable ‘frame’ set but not used [-Wunused-but-set-variable]
libavfilter/vf_minterpolate.c: In function ‘interpolate’:
libavfilter/vf_minterpolate.c:1179:13: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
libavfilter/vf_minterpolate.c:1089:39: warning: unused variable ‘dy’ [-Wunused-variable]
libavfilter/vf_minterpolate.c:1089:35: warning: unused variable ‘dx’ [-Wunused-variable]
libavfilter/vf_minterpolate.c: At top level:
libavfilter/vf_minterpolate.c:353:17: warning: ‘get_sbad2’ defined but not used [-Wunused-function]
libavfilter/vf_minterpolate.c:378:17: warning: ‘search_mv_esa’ defined but not used [-Wunused-function]


[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

No great genius has ever existed without some touch of madness. -- Aristotle
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20160814/81bd1d18/attachment.sig>