[Ffmpeg-devel] few remarks for h264 decoder

[Gábor Kovács]

>> pred_direct_motion() uses 8bit width fill_rectangle() with ref[0],ref[1] 
>> which can be negative numbers causing trouble with val*0x0101 in 
>> fill_rectangle()
> 
> val is unsigned!

Example ref[0] is -1. When it's passed to uint32_t val, it will become 0xffffffff.
(uint16_t) (0xffffffff * 0x0101) = 0xfeff  (which in this case means one of the 
ref_cache value is PART_NOT_AVAILABLE instead of LIST_NOT_USED)

>> I didn't check it throughly but mmx h264_qpel4_hv_lowpass macro uses
>> approximation (comment sais ((a-b)/4-b)/4). I know a proper solution
>> needs 32bit range, but this is why God created "pmaddwd" :) I think the
>> performance loss needed for accurate calculation would be minimal.
> 
> i dont see why you claim this to be an approximation, please elaborate /
> proof your claim

input:  abccba
what it does: (((a-b)/4-b)/4+c+32)>>6

reformulated (not bothering for a moment with 16bit limit)
((a-b)-b*4+c*16+32*16)>>(6+4)
(a-5*b+c*16+512)>>10

what it should be:
(a-5*b+c*20+512)>>10 

So there are two problems. One is the factor of c (16 vs 20). 
Other is the loss of precision (lower bits) using the /4 right shifts.

> do you have any sample files which are decoded differently in lavc then
> with the reference sw due to any of the possible issues you posted? these
> would be quite usefull if anyone wants to investigate&fix these

In general I was testing these, but not directly ffmpeg...
http://ftp3.itu.ch/av-arch/jvt-site/draft_conformance/

bye, Picard