[FFmpeg-devel] [PATCH] start code detection for hardware decode in mpeg4/divx streams

[Michael Niedermayer]

On Thu, Oct 18, 2012 at 09:24:11PM +0530, anuj mittal wrote:
> On Wed, Oct 17, 2012 at 4:26 PM, anuj mittal <am.devel at gmail.com> wrote:
> > Hello
> >
> > I've some AVIs with divx bitstream packed frames. When I play these
> > file using hardware decode VAAPI, I notice frame drops.
> >
> > It looks like that the logic to calculate the end of the frame (while
> > calculating the slice size to be sent to hardware) isn't correct for
> > hardware decode. If I have a B frame packed with a P, it just sends
> > the size of P+B to the hardware. Because of this, the packed B frames
> > do not get decoded at all.
> >
> > The attached patch tries to fix this behaviour. The code will now
> > check for the start code while calculating the value of "end". If the
> > start of next frame has been detected, it will update s->mb_y.
> >
> >
> 
> Does this look ok? Thank you.
> 
> Anuj

>  h263dec.c    |   11 ++++++++++-
>  ituh263dec.c |   14 ++++++++++++--
>  2 files changed, 22 insertions(+), 3 deletions(-)
> b098443d3108d1d133b562088791aaa8bbdf6826  0001-Start-code-frame-handling-while-decoding-using-hardw.patch
> From c654d9a2aca9f5f20c2949f18ce584eb363c47e5 Mon Sep 17 00:00:00 2001
> From: Anuj Mittal <am.devel at gmail.com>
> Date: Wed, 17 Oct 2012 16:08:42 +0530
> Subject: [PATCH] Start code frame handling while decoding using hardware.
> 
> ---
>  libavcodec/h263dec.c    |   11 ++++++++++-
>  libavcodec/ituh263dec.c |   14 ++++++++++++--
>  2 files changed, 22 insertions(+), 3 deletions(-)
> 
> diff --git a/libavcodec/h263dec.c b/libavcodec/h263dec.c
> index 7141d49..4602f8d 100644
> --- a/libavcodec/h263dec.c
> +++ b/libavcodec/h263dec.c
> @@ -169,10 +169,19 @@ static int decode_slice(MpegEncContext *s){
>      ff_set_qscale(s, s->qscale);
>  
>      if (s->avctx->hwaccel) {
> +        int ret;
> +
>          const uint8_t *start= s->gb.buffer + get_bits_count(&s->gb)/8;
>          const uint8_t *end  = ff_h263_find_resync_marker(s, start + 1, s->gb.buffer_end);
>          skip_bits_long(&s->gb, 8*(end - start));
> -        return s->avctx->hwaccel->decode_slice(s->avctx, start, end - start);
> +        ret = s->avctx->hwaccel->decode_slice(s->avctx, start, end - start);
> +
> +        /* Next frame detected */
> +        if (!end[0] && !end[1] && (end[2] == 1)){

these can access out of array values

    
> +            s->mb_y = s->mb_height;
> +        }
> +
> +        return ret;
>      }
>  
>      if(s->partitioned_frame){
> diff --git a/libavcodec/ituh263dec.c b/libavcodec/ituh263dec.c
> index 6d43233..f4470dc 100644
> --- a/libavcodec/ituh263dec.c
> +++ b/libavcodec/ituh263dec.c
> @@ -217,11 +217,21 @@ const uint8_t *ff_h263_find_resync_marker(MpegEncContext *s, const uint8_t *av_r
>          int prefix_len = ff_mpeg4_get_video_packet_prefix_length(s);
>          for(;p<end; p+=2){
>              if(!*p){
> -                if      (!p[-1] && ((p[1] >> (23-prefix_len)) == 1)) return p - 1;
> -                else if (!p[ 1] && ((p[2] >> (23-prefix_len)) == 1)) return p;
> +                if      (!p[-1] && (((p[1] >> (23-prefix_len)) == 1) || (p[1] == 1))) return p - 1;
> +                else if (!p[ 1] && (((p[2] >> (23-prefix_len)) == 1) || (p[2] == 1))) return p;

this isnt a resync marker, so it does something else than what the
name of the function would suggest

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Old school: Use the lowest level language in which you can solve the problem
            conveniently.
New school: Use the highest level language in which the latest supercomputer
            can solve the problem without the user falling asleep waiting.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20121018/e9b37a6e/attachment.asc>