[FFmpeg-devel] video corruption using h264 hw decoders

[Joakim Plate]

On Tue, 1 Nov 2011 03:06:06 +0100, Michael Niedermayer wrote:

> On Sun, Oct 30, 2011 at 11:32:11PM +0100, Joakim Plate wrote:
>> [5 quoted lines suppressed]
> 
> If this works and is tested with videos that change resolution then
> iam fine with it.
> 
> [...]

Okey, done some testing with this. Problem is we don't know we are hw
decoding when avcodec_set_dimensions is called the first time, thus we'd
start by initing wrong size. This is around line 2700 ish in h264.c. This
is what we tried (against older ffmpeg so not really interesting:
http://pastebin.com/G6hZ0PpA)

I noticed a bug there looking at it it. It's using avctx->width/height to
check for changes compared to sps info. But width/height can have been
modified by low_res (not that that is supported). But i think the check
should be against coded_width/height.

This brings me to an alternate idea. What about never cropping
coded_width/coded_height, only apply cropping to width/height parameters.

What do you think?


Ps. The reason we can not recover macroblock size by alignment is that we
don't know if we should align to 16 or 32 for height, which is defined by:
sps.frame_mbs_only_flag that is not exported.

/Joakim