[FFmpeg-devel] [PATCH 2/2] avcodec/motion_est: fix penalty_factor for b frames

[Ramiro Polla]

On Mon, Mar 23, 2020 at 10:42 PM Michael Niedermayer
<michael at niedermayer.cc> wrote:
> On Sun, Mar 22, 2020 at 04:55:25PM +0100, Ramiro Polla wrote:
> > In ff_estimate_b_frame_motion(), penalty_factor would be used before
> > being initialized in estimate_motion_b(). Also, the initialization
> > would happen more than once unnecessarily.
> > ---
> >  libavcodec/motion_est.c                  | 15 ++++++++-------
> >  tests/ref/vsynth/vsynth2-mpeg2-422       |  6 +++---
> >  tests/ref/vsynth/vsynth2-mpeg2-ivlc-qprd |  6 +++---
> >  tests/ref/vsynth/vsynth2-mpeg4-adap      |  6 +++---
> >  4 files changed, 17 insertions(+), 16 deletions(-)
> >
> > diff --git a/libavcodec/motion_est.c b/libavcodec/motion_est.c
> > index 02c75fd470..1feb46cec3 100644
> > --- a/libavcodec/motion_est.c
> > +++ b/libavcodec/motion_est.c
> > @@ -1123,9 +1123,6 @@ static int estimate_motion_b(MpegEncContext *s, int mb_x, int mb_y,
> >      uint8_t * const mv_penalty= c->mv_penalty[f_code] + MAX_DMV;
> >      int mv_scale;
> >
> > -    c->penalty_factor    = get_penalty_factor(s->lambda, s->lambda2, c->avctx->me_cmp);
> > -    c->sub_penalty_factor= get_penalty_factor(s->lambda, s->lambda2, c->avctx->me_sub_cmp);
> > -    c->mb_penalty_factor = get_penalty_factor(s->lambda, s->lambda2, c->avctx->mb_cmp);
> >      c->current_mv_penalty= mv_penalty;
> >
> >      get_limits(s, 16*mb_x, 16*mb_y);
>
>
>
> > @@ -1491,7 +1488,6 @@ void ff_estimate_b_frame_motion(MpegEncContext * s,
> >                               int mb_x, int mb_y)
> >  {
> >      MotionEstContext * const c= &s->me;
> > -    const int penalty_factor= c->mb_penalty_factor;
> >      int fmin, bmin, dmin, fbmin, bimin, fimin;
> >      int type=0;
> >      const int xy = mb_y*s->mb_stride + mb_x;
> > @@ -1517,18 +1513,23 @@ void ff_estimate_b_frame_motion(MpegEncContext * s,
> >          dmin= direct_search(s, mb_x, mb_y);
> >      else
> >          dmin= INT_MAX;
> > +
> > +    c->penalty_factor    = get_penalty_factor(s->lambda, s->lambda2, c->avctx->me_cmp);
> > +    c->sub_penalty_factor= get_penalty_factor(s->lambda, s->lambda2, c->avctx->me_sub_cmp);
> > +    c->mb_penalty_factor = get_penalty_factor(s->lambda, s->lambda2, c->avctx->mb_cmp);
>
> If mb_penalty_factor isnt correct in this before this maybe isnt enough
> as the direct_search() uses mb_penalty_factor

Fixed.

New patch attached.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-avcodec-motion_est-fix-penalty_factor-for-b-frames.patch
Type: text/x-patch
Size: 8171 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20200324/47bcaa1d/attachment.bin>