[FFmpeg-devel] [DISCUSSION] Motion Estimation API/Library

[Davinder Singh]

Hello everyone,

As we've been planning since forever (
https://github.com/FFmpeg/FFmpeg/blob/master/libavcodec/snow.h#L182,
http://ffmpeg.org/pipermail/ffmpeg-devel/2016-July/197095.html) we need
Motion Estimation code that could be shared in codecs and motion filters.

The idea is to make the Motion Estimation idependent of Encoders more
specifically - AVCodecContext.
So, I’ve moved motion estimation and me_cmp code to a new location -
libmotion <https://github.com/dsmudhar/FFmpeg/tree/gsoc17/libmotion>. I
think it’s a good idea to make a new lib instead moving it to
libavutil (as discussed
previously <http://ffmpeg.org/pipermail/ffmpeg-devel/2016-July/197161.html>).
That way we can make it independent of everything else in FFmpeg.
Everything will be accessed through single context AVMotionEstContext, and
also be initialized through it. This API could also be used by researchers
to test new codecs or FRUC. And additionally, we can implement best Optical
Flow methods: http://vision.middlebury.edu/flow/eval/results/results-i1.php in
the future which are not available in OpenCV.

GOALS:
1. Make me_cmp_func()
<https://github.com/dsmudhar/FFmpeg/blob/gsoc17/libmotion/me_cmp.h#L48>
independent of MpegEncContext and AVCodecContext. To accomplish this, we
can move following from MpegEncContext to MotionEstContext or[*]
MECmpContext (as these are used in cmp fxn in me_cmp.c
<https://github.com/dsmudhar/FFmpeg/blob/gsoc17/libmotion/me_cmp.c>):
    int mb_intra;
    PixblockDSPContext pdsp;
    FDCTDSPContext fdsp;
    IDCTDSPContext idsp;
    int (*fast_dct_quantize)();
    void (*dct_unquantize_inter)();
    void (*dct_unquantize_intra)();
    int block_last_index[12];
    int qscale;
    ScanTable intra_scantable;
    int ac_esc_length;
    uint8_t *intra_ac_vlc_length;
    uint8_t *intra_ac_vlc_last_length;
    uint8_t *inter_ac_vlc_length;
    uint8_t *inter_ac_vlc_last_length;
    uint8_t *luma_dc_vlc_length;
    MECmpContext mecc; [*]

As pointed by michaelni earlier, these are mostly more used by Encoders
than ME. If libavcodec is more appropriate place for some of these (e.g.
pdsp, fdsp) and these are also used by few cmp functions, like
FF_CMP_BIT, FF_CMP_RD, FF_CMP_DCT, FF_CMP_PSNR,
we can move these to specific encoders and set function address in Context
before using them. Or we can register cmp functions via by calling API
while initializing FFmpeg/Options. Only generic functions will be part of
libmotion.

2. Initialization of MECmpContext at one place (like init pdsp and dct if
moved in MECmpContext).
3. Make motion_est.c
<https://github.com/dsmudhar/FFmpeg/blob/gsoc17/libmotion/motion_est.c>
code independent of MpegEncContext. for this a lot of things need to be
moved from MpegEncContext to MotionEstContext. MpegEncContext is used at
lot of places. Please suggest your approach on this. Generic ME should
surely be seperated.

Please share your views on this.

Regards,
-- 
Davinder Singh