[Ffmpeg-devel] few remarks for h264 decoder

Gábor Kovács picard
Sat Dec 31 00:34:31 CET 2005

>> pred_direct_motion() uses 8bit width fill_rectangle() with ref[0],ref[1] 
>> which can be negative numbers causing trouble with val*0x0101 in 
>> fill_rectangle()
> val is unsigned!

Example ref[0] is -1. When it's passed to uint32_t val, it will become 0xffffffff.
(uint16_t) (0xffffffff * 0x0101) = 0xfeff  (which in this case means one of the 
ref_cache value is PART_NOT_AVAILABLE instead of LIST_NOT_USED)

>> I didn't check it throughly but mmx h264_qpel4_hv_lowpass macro uses
>> approximation (comment sais ((a-b)/4-b)/4). I know a proper solution
>> needs 32bit range, but this is why God created "pmaddwd" :) I think the
>> performance loss needed for accurate calculation would be minimal.
> i dont see why you claim this to be an approximation, please elaborate /
> proof your claim

input:  abccba
what it does: (((a-b)/4-b)/4+c+32)>>6

reformulated (not bothering for a moment with 16bit limit)

what it should be:

So there are two problems. One is the factor of c (16 vs 20). 
Other is the loss of precision (lower bits) using the /4 right shifts.

> do you have any sample files which are decoded differently in lavc then
> with the reference sw due to any of the possible issues you posted? these
> would be quite usefull if anyone wants to investigate&fix these

In general I was testing these, but not directly ffmpeg...

bye, Picard

More information about the ffmpeg-devel mailing list