[Libav-user] decoding mp4 questions

[wm4]

On Sat, 14 Jun 2014 12:49:17 +0100
Adam Billyard <adambillyard at googlemail.com> wrote:

> 
> On 14 Jun 2014, at 12:05, wm4 <nfxjfg at googlemail.com> wrote:
> 
> > On Sat, 14 Jun 2014 11:40:03 +0100
> > Adam Billyard <adambillyard at googlemail.com> wrote:
> > 
> >> Hi All,
> >> 
> >> 
> >> All the examples I can find of using swscale to convert an AVFrame from YUV to RGB, use the *CodecContext* width and height:
> >> 
> >> eg
> >> 
> >> sws_getContext(pCodecCtx->width, pCodecCtx->height, 
> >>                        pCodecCtx->pix_fmt, 
> >>                         dst_width, dst_height, PIX_FMT_RGB24, SWS_BILINEAR, 
> >>                        NULL, NULL, NULL);
> > 
> > All ffmpeg examples I've seen were either outdated or broken. Nowadays
> > you should just use the AVFrame parameters.
> 
> So true - and sad that such a great engineering effort lacks even basic up to date examples.
> 
> > 
> >> 
> >> But when I decode a 1024x768 mp4 movie, the CodecContext->width is 8 and height 1024..  Is that expected?
> > 
> > No, that seems unusual. Normally the codec context should have the
> > parameters of the last AVFrame returned, I think. Maybe something fishy
> > is going on.
> 
> Yes, thats what I expected - and if I use the last Frame dimensions all looks fine (modulo the issue below).  Bit worrying where CodecCtx is copying that width=8 from..  I began wondering whether its actually a slice, but avcodec_decode_video2() is claiming it has a "framefinished".

A slice? No, that should never happen.

> > 
> >> So I have been using:
> >> 
> >> sws_getContext(frame->width, frame->height, pCodecCtx->pix_fmt,
> >> 			 dst_width, dst_height, PIX_FMT_RGB24, SWS_BILINEAR,
> >>                         NULL, NULL, NULL);
> >> 
> >> 
> >> ie after decoding an AVFrame, I use its (now known) size to get my sws context.  Is this correct?
> > 
> > Yes. Also, sws_getContext is deprecated.
> 
> Oh!  So I thought that was the new way over img_convert()!
> 
> What should I be using to convert a AVFrame of YUV to RGB (and resize)?  And any reference source to look at?  I do particular want a CPU-based solution not one relying on GPU.

libswscale is the library to use, but the APIs to allocate the context
are a bit chaotic. You can either try sws_getCachedContext(), or
sws_alloc_context()+the other setup functions.

But as long as it exists, sws_getContext() should work fine and not
cause corrupted output.

> > 
> >> 
> >> Lastly, I find this works fine for .mov and .avi containers, but for .mp4 movies, it all runs but looks like I'm just getting chroma with extra crud of missing/stale macroblocks..
> >> 
> >> Video of me pointing at an iPad screen looks like this:
> >> http://i614.photobucket.com/albums/tt229/ElektraGlide/ScreenShot2014-06-14at113519_zps126c6638.png
> >> 
> >> Any ideas?
> > 
> > No. Does playback with ffplay work? Also, mov and mp4 are the same
> > format.
> 
> They all play fine with VLC, ffplay etc.  I'm confident the streams are valid.
> 
> However, I just noted this log message when starting to stream:
> 
> Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/Users/xxxxxxxxx.mp4':
>   Metadata:
>     major_brand     : mp42
>     minor_version   : 0
>     compatible_brands: mp42isom
>     creation_time   : 2014-01-24 17:23:18
>   Duration: 00:02:49.17, start: 0.000000, bitrate: 1149 kb/s
>     Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 1024x768, 1144 kb/s, 30 fps, 30 tbr, 3k tbn, 60 tbc
>     Metadata:
>       creation_time   : 2014-01-24 17:23:18
>     Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 2 kb/s
>     Metadata:
>       creation_time   : 2014-01-24 17:23:18
> found video codec for format: 768x1024 using yuvj420p
> [h264 @ 0x59c0600] Missing reference picture, default is 0
> [h264 @ 0x59c0600] decode_slice_header error
> [h264 @ 0x59c0600] concealing 3072 DC, 3072 AC, 3072 MV errors in P frame
> [h264 @ 0x5ad5a00] reference picture missing during reorder
> [h264 @ 0x5ad5a00] Missing reference picture, default is 65540
> [h264 @ 0x5939c00] mmco: unref short failure
> [h264 @ 0x5939c00] reference picture missing during reorder
> [h264 @ 0x5939c00] Missing reference picture, default is 66820
> [h264 @ 0x593a400] mmco: unref short failure
> [h264 @ 0x593a400] number of reference frames (0+6) exceeds max (5; probably corrupt input), discarding one
> 
> So ffmpeg is missing the initial I-frame - hence all the B/P deltas from there on make a colorful mess.
> 
> I tried rewinding the stream before I start, but it makes no difference.
> 
> Is there anything in the dump info that indicates a problem?

Sorry, can't help. I assume ffplay also shows corrupt frames in the
beginning, and then at some point recovers. You should see the same
with your code (as long as it actually reads new packets and feeds them
to the decoder correctly). You can try to use valgrind to ensure that
nothing reads out of bounds memory or that no double frees are
occurring - these can quite easily happen with these APIs.