[FFmpeg-devel] H.264 audit - first try to match MBAFF(/PAFF) modifications to the spec.

Ulrich Habel espero7757
Tue Jul 17 22:13:08 CEST 2007


Hello together,

this is my first try to do an audit to check h264.c and companion files for 
conformity against the spec in respect to modification to be done to support
MBAFF(/PAFF).

1) the first document trys to find positions in the 14496-10 document related 
to MBAFF - like macroblock positioning, filtering and many others ...

Furthermore it is an audit check file. Every item conforming to the spec
is checkmarked, any open issues are to be done. At the moment I found
59 items to be checked against, 14 of them are already doing it right (IMHO).
Another 20 seems to need some tweaks to follow the spec. The big "rest"
is work in progress.

  ISO-IEC-14496-10_MBAFF_for-libavcodec_20070715.txt


2) the second document shows the names of attributes used in the 14496-10
compared to the working attributes within the h264.c code.

  ffmpeg_libavcodec_h264.txt


All documents are "an open book" and are work in progress. If you find it 
usefull, use it. I will go on to modify the code patch by patch to get the
MBAFF thing working.

Cheers
  Uli

PS: like to get my rig running, right?   ;-)
-------------- next part --------------
=====================================================================================
International Standard ISO/IEC 14496-10 (Advanced Video Coding) in reference to MBAFF
=====================================================================================
									 Ulrich Habel
									   2007-07-15

        AUDIT REPORT



 ------ Implementation Status: (*) implemented, (-) don't care, (>) to be done
/   ---	                 Type: (M)andatory, (V)erification of stream, (I)nformational
|  /
|  |

IS T	page	chapter		statements
-------------------------------------------------------------------------------------

-  I	xviii	0.6.2		progressive-scan vs. interlaced-scan, coding fields vs. frames:

				1a) progressive-scan: fields share the same capture time
				1b) interlaced-scan: fields are separated in capture time
		
				2a) fields are coded separately or 
				2b) fields are coded together as frame
		
				3a) Remark: 1a) typically uses 2b)
				3b) Remark: 1b) choses one of 2a) and 2b)


-  I	  2	3.16		bottom field

-  I	  2	3.17		bottom macroblock (of a macroblock pair)

-  I	  3	3.25		coded field

-  I	  3	3.26		coded frame

-  I	  4	3.48		field

-  I	  4	3.49		field macroblock

-  I	  4	3.50		field macroblock pair

-  I	  4	3.51		field scan

-  I	  4	3.53		frame

-  I	  4	3.54		frame macroblock

-  I	  4	3.55		frame macroblock pair

-  I	  5	3.76		macroblock-adaptive frame/field decoding

-  I	  5	3.77		macroblock address

-  I	  5	3.78		macroblock location

-  I	  5	3.79		macroblock pair

-  I	  6	3.96		opposite parity

-  I	  6	3.100		parity

-  I	  6	3.102		picture

-  I	  7	3.128		run

-  I	  7	3.129		sample aspect ratio

-  I	  8	3.135		skipped macroblock

-  I	  8	3.136		slice

-  I	  8	3.145		sub-macroblock

-  I	  8	3.146		sub-macroblock partition

-  I	  9	3.151		top field

-  I	  9	3.152		top macroblock (of a macroblock pair)

-  I	  9	3.158		zig-zag scan


-  I	  9	4.12		MB: Macroblock

-  I	  9	4.13		MBAFF: Macroblock-Adaptive Frame-Field Coding


!  M	 12	(5-7)	        process InverseRasterScan(a, b, c, d, e)						               @see 7.4.3
				INFO only: parameters are
				a: macroblock address (mbAddr / (1 + MbaffFrameFlag))
				b: macroblock width
				c: macroblock width * (1 + MbaffFrameFlag)
				d: picture width (PicWidthInSamplesL)
				e: (0) calculate x position / (1) calculate y position


-  I	 14f	6.1	        Bitstream formats: NAL unit stream and byte stream

!  M	 15	6.2(1)		SubWidthC               := f( chroma_format_idc )					               @see 7.3.2.1
				SubHeightC		:= f( chroma_format_idc )

!  M	 19	6.2(2)		MbWidthC                := f( SubWidthC )
				MbHeightC		:= f( SubHeightC )

-  I	 20	6.3	        1) Non-BAFF: Slice is sequence of macroblocks / BFAFF: Slice is sequence of macroblock pairs
				2) MBAFF: a macroblock pair consist of two macroblocks

>  M	 21	6.4.1		Picture location (x, y) := f ( macroblock address (mbAddr), MbaffFrameFlag )
				(6-3) ... (6-10)

				y correction for MbaffFrameFlag == 1 && currMbFrameFlag == 1:		
				y := (y of InverseRasterScan(...)) + (mbAddr is Odd) * 16

				y correction for MbaffFrameFlag == 1 && currMbFrameFlag == 0:
				y := (y of InverseRasterScan(...)) + (mbAddr is Odd)

>  M	 24	6.4.6		Inhibit process for MbaffFrameFlag == 1							               @see 6.4.7

>  M	 24	6.4.7		Add process for MbaffFrameFlag == 1							               @see 6.4.6
				Derivation process for neighbouring macroblock addresses and their availability in MBAFF frames

>  M	 30f	6.4.9ff		When MbaffFrameFlag == 0: do (@see 6.4.9.1), (@see 6.4.6)
				When MbaffFrameFlag == 1: do (@see 6.4.9.2), (@see 6.4.7)
				currMbFrameFlag        := f( current MB is frame MB )
				mbIsTopMbFlag          := f( CurrMbAddr is even )
				mbAddrXFrameFlag       := f( mbAddrX is frame MB )
   
-  I	 36	7.3.1		NAL unit syntax

!  M	 37	7.3.2.1(1)	chroma_format_idc         when profile_idc in desired state

*  M	 37	7.3.2.1(2)	offset_for_top_to_bottom_field

*  M     38     7.3.2.1(3)	pic_width_in_mbs_minus1

*  M	 38	7.3.2.1(4)	pic_height_in_map_units_minus1

*  M	 38	7.3.2.1(5)	frame_mbs_only_flag

*  M	 38	7.3.2.1(6)	mb_adaptive_frame_field_flag

*  M	 39	7.3.2.1(7)	direct_8x8_inference_flag

*  M	 39	7.3.2.1(8)	frame_cropping_flag

*  M	 39	7.3.2.1(9)	frame_crop_left_offset

*  M	 39	7.3.2.1(10)	frame_crop_right_offset

*  M	 39	7.3.2.1(11)	frame_crop_top_offset

*  M	 39	7.3.2.1(12)	frame_crop_bottom_offset

*  M	 39	7.3.2.2		entropy_coding_mode_flag

!  M	 44	7.3.3(1)	first_mb_in_slice

?  M	 44	7.3.3(2)	slice_type

*  M	 44	7.3.3(3)	frame_num

?  M	 44	7.3.3(4)	field_pic_flag            when frame_mbs_only_flag in desired state

?  M	 44	7.3.3(5)	bottom_field_flag

*  M	 44	7.3.3(6)	delta_pic_order_cnt_bottom

?  M	 48	7.3.4(1)	mb_skip_run               when slice_type, entropy_coding_mode_flag in desired state

?  M	 48	7.3.4(2)	mb_skip_flag              when slice_type, entropy_coding_mode_flag in desired state

!  M	 48	7.3.4(3)	prevMbSkipped          := f( mb_skip_run, mb_skip_flag )
!				CurrMbAddr             := f( MbaffFrameFlag, first_mb_in_slice, procedure NextMbAddress())             @see 7.4.3
!				mb_field_decoding_flag    when prevMbSkipped, MbaffFrameFlag, (CurrMbAddr is even)

** M	 67f	7.4.2.1(1)	PicWidthInMbs          := f( pic_width_in_mbs_minus1 )					               @see 7.3.2.1
*-				PicWidthInSamplesL     := f( PicWidthInMbs )
?-				PicWidthInSamplesC     := f( PicWidthInMbs, MbWidthC )					               @see 7.3.2.1, @see 6.2
*-				PicHeightInMapUnits    := f( pic_height_in_map_units_minus1 )					       @see 7.3.2.1
--				PicSizeInMapUnits      := f( PicWidthInMbs, PicHeightInMapUnits )
?*				FrameHeightInMbs       := f( frame_mbs_only_flag, PicHeightInMapUnits )			               @see 7.3.2.1

-  I	 68	7.4.2.1(2)	mb_adaptive_frame_field_flag (MBAFF) explaination  ( --> MBAFF / "PAFF")

-  V	 68	7.4.2.1(3)	direct_8x8_inference_flag influence on B_Skip, B_Direct_16x16, B_Direct_8x8                            @see 7.3.2.1

!  M	 68	7.4.2.1(4)	CropUnitX	       := f( SubWidthC )						               @see 6.2
!				CropUnitY	       := f( frame_mbs_only_flag, SubHeightC )					       @see 7.3.2.1, @see 6.2
!				Crop pos. horizontal   := f( frame_crop_left_offset, frame_crop_right_offset, CropUnitX )	       @see 7.3.2.1
!				Crop pos. vertical     := f( frame_crop_top_offset, frame_crop_bottom_offset, CropUnitY )	       @see 7.3.2.1

-  V	 74	7.4.2.9.1	check slice_id range related to MbaffFrameFlag

-  I	 75	7.4.3(1)	first MbAddr of slice  := f( first_mb_in_slice, MbaffFrameFlag )			               @see 7.3.4, @see 7.4.3

-  M	 77f	7.4.3(2)	field_pic_flag explaination
!				MbaffFrameFlag         := f( mb_adaptive_frame_field_flag, field_pic_flag )		               @see 7.3.2.1, @see 7.3.3
!				PicHeightInMbs	       := f( field_pic_flag, FrameHeightInMbs )					       @see 7.3.3, @see 7.4.2.1
!				PicHeightInSamplesL    := f( PicHeightInMbs )
!				PicHeightInSamplesC    := f( PicHeightInMbs, MbHeightC )				               @see 6.2
!				PicSizeInMbs	       := f( PicWidthInMbs, PicHeightInMbs )					       @see 7.4.2.1
!				MaxPicNum	       := f( field_pic_flag, MaxFrameNum )					       @see 7.3.3, @see ???
!				CurrPicNum	       := f( field_pic_flag, frame_num )					       @see 7.3.3

-  I	 78	7.4.3(3)	bottom_field_flag explaination

-  V	 78	7.4.3(4)	check delta_pic_order_cnt_bottom
				check delta_pic_order_cnt[0|1]

-  V	 80	7.4.3(5)	check num_ref_idx_10_active_minus1 related to field_pic_flag and MbaffFrameFlag
				check num_ref_idx_11_active_minus1 related to field_pic_flag and MbaffFrameFlag

-  I	 86	7.4.4		mb_skip_flag explaination
				mb_field_decoding_flag explaination

!  M	 86	7.4.4		mb_field_decoding_flag prediction := f( mb_field_decoding_flagN[left|up] )			               @see 7.3.4

-  V	 94	7.4.5.1		check ref_idx_10[mbPartIdx] related to mb_field_decoding_flag, MbaffFrameFlag
				check ref_idx_11[mbPartIdx] related to mb_field_decoding_flag, MbaffFrameFlag
   
-  V	 96	7.4.5.2		check ref_idx_10[mbPartIdx] related to mb_field_decoding_flag, MbaffFrameFlag
				check ref_idx_11[mbPartIdx] related to mb_field_decoding_flag, MbaffFrameFlag

-  I	 99ff	8.2.1		TopFieldOrderCnt
				BottomFieldOrderCnt

?  M	137	8.3.5		dy		       := f( MbaffFrameFlag, (current macroblock is field macroblock) )  when mb_type of desired type
																       @see 7.4.3

>  M	141	8.4.1.2		( refIdxL[0|1],
				  mvL[0|1],
				  subMvCnt,
				  predFlagL[0|1] )     := f( direct_8x8_inference_flag, mbPartIdx, subMbPartIdx, mb_type )             @see 7.3.2.1, @see 7.4.2.1

>  M	142	8.4.1.2.1	( colPic,
				  mbAddrCol,
				  mvCol,
				  refIdxCol,
				  vertMvScale,
				  PicCodingStruct )    := f( field_pic_flag, mb_field_decoding_flag, mb_adaptive_frame_field_flag )
				subMbPartIdx           := f( mbPartIdx ) when (direct_8x8_inference_flag == 1)			       @see 7.3.2.1

>  M	146	8.4.1.2.3	( refIdxL[0|1],
				  refIdxL0Frm,
				  mvL[0|1],
				  predFlagL[0|1] )     := f( macroblock is field macroblock, mbPartIdx, subMbPartIdx )	               @see ???

>  M	151	8.4.1.4		mvCLX		       := f( field coding mode, mvLX, refIdxLX, SubWidthC, SubHeightC )	               @see ???, @see 6.2

>  M	157	8.4.2.2.2	refPicHeightEffectiveC := f( mb_field_decoding_flag, refPicHeightEffectiveC, PicHeightInSamplesC, MbaffFrameFlag and mb_field_decoding_flag
																       @see 7.4.3

>  M	160	8.4.2.3.2(1)	pic[0|1]               := f( field_pic_flag, (parity and current macroblock), refIdxL[0|1], RefPicList[0|1][] )
																       @see ???

>  M	160f	8.4.2.3.2(2)	refIdxL[0|1]WP	       := f( (current MB is field MB), MbaffFrameFlag, RefIdxL[0|1] )		       @see 7.4.3

>  M	166	8.5.5		4x4 block scans zig-zag and field scan

>  M	167f	8.5.6		8x8 block scans zig-zag and field scan

>  M	178f	8.5.12		( S'L, S'C )           := f( (current MB is field MB), MbaffFrameFlag )			               @see 7.4.3

>  M	185	8.7(1)		fieldModeMbFlag        := f( (current MB is field MB), field_pic_flag, MbaffFrameFlag )		       @see 7.4.3

>  M	185	8.7(2)		filterLeftMbEdgeFlag   := f( disable_deblocking_filter_idc, mbAddrA, MbaffFrameFlag, CurrMbAddr, PicWidthInMbs )
				filterTopMbEdgeFlag    := f( disable_deblocking_filter_idc, mbAddrB, MbaffFrameFlag, CurrMbAddr, PicWidthInMbs )
																       @see 7.4.3

>  M	188	8.7.1		dy                     := f( fieldModeFilterFlag, MbaffFrameFlag )				       @see 7.4.3

>  M	190	8.7.2.1		bS                     := f( mixedModeEdgeFlag, verticalEdgeFlag, MbaffFrameFlag, macroblock pairs )   @see 7.4.3

>  M	232	9.3.3.1.1.1	ctxIdxInc              := f( mb_field_decoding_flag for mb_skip_flag, MbaffFrameFlag, N MB is frame MB )
																       @see 7.4.3
  
>  M	232	9.3.3.1.1.2	ctxIdxInc              := f( mb_field_decoding_flag for mb_field_decoding_flag, MbaffFrameFlag, N MB is frame MB )
																       @see 7.4.3

>  M	235	9.3.3.1.1.6	refIdxZeroFlagN	       := f( (current MB is field MB), MbaffFrameFlag, N MB is field MB )              @see 7.4.3


-  I	255	A.2.1		frame_mbs_only_flag constraints

>  M	258	A.3.1		fR		       := f( picture is frame / field )

>  M	260	A.3.2		fR		       := f( picture is frame / field )

    
-  I	269	B.1.1		Byte stream NAL unit syntax

-  I	270	B.2		Byte stream NAL unit decoding process


-  I	294	D.2.2(1)	pic_struct - SEI picture timing and field_pic_flag, bottom_field_flag, fixed_frame_rate_flag

-  I	295	D.2.2(2)	ct_type for progressive / interlaced

-  I	301	D.2.8		original_field_pic_flag, original_bottom_field_flag

-  I	301	D.2.9(1)	target_bottom_field_flag

-  I	302	D.2.9(2)	spare_bottom_field_flag[i]



AUDIT RESULTS:
-------------

14 OK
39 OPEN OR IN WORK  (of 53 positions in total)



<EOF>
-------------- next part --------------
==========================
ffmpeg, libavcodec, h264.c
==========================


AVCodec h264_decoder = {}

-->

static int decode_frame(AVCodecContext *avctx, void *data, int *data_size, uint8_t *buf, int buf_size) {}

-->

static int decode_nal_units(H264Context *h, uint8_t *buf, int buf_size){}

-->

1)
static uint8_t *decode_nal(H264Context *h, uint8_t *src, int *dst_length, int *consumed, int length){}

2)
static inline int decode_seq_parameter_set(H264Context *h){}

  --> 

  static inline int decode_vui_parameters(H264Context *h, SPS *sps){}

    -->

    inline void decode_hrd_parameters(H264Context *h, SPS *sps){}



===


ISO/IEC										ffmpeg
-------------------------------------------------------------------------------------------------------------
 37	7.3.2.1		profile_idc						sps->profile_idc
 37	7.3.2.1		level_idc						sps->level_idc
 37	7.3.2.1		qqpprime_y_zero_transform_bypass_flag			sps->transform_bypass
 37	7.3.2.1		scaling_list(...)					sps->scaling_matrix4, sps->scaling_matrix8
 37	7.3.2.1		log2_max_frame_num_minus4 + 4				sps->log2_max_frame_num
 37	7.3.2.1		pic_order_cnt_type					sps->poc_type
 37	7.3.2.1		log2_max_pic_order_cnt_lsb_minus4 + 4			sps->log2_max_poc_lsb
 37	7.3.2.1		delta_pic_order_always_zero_flag			sps->delta_pic_order_always_zero_flag
 37	7.3.2.1		offset_for_non_ref_pic					sps->offset_for_non_ref_pic
 37	7.3.2.1		offset_for_top_to_bottom_field				sps->offset_for_top_to_bottom_field
 37	7.3.2.1		num_ref_frames_in_pic_order_cnt_style			sps->poc_cycle_length
 37	7.3.2.1		offset_for_ref_frame[i]					sps->offset_for_ref_frame[i]
 37	7.3.2.1		num_ref_frames						sps->ref_frame_count
 37	7.3.2.1		gaps_in_frame_num_allowed_flag				sps->gaps_in_frame_num_allowed_flag
 38	7.3.2.1		pic_width_in_mbs_minus1 + 1				sps->mb_width
 38	7.3.2.1		pic_height_in_map_units_minus1 + 1			sps->mb_height                                !!! CHECK !!!   (map != mbs)
 38	7.3.2.1		frame_mbs_only_flag					sps->frame_mbs_only_flag
 38	7.3.2.1		mb_adaptive_frame_field_flag				sps->mb_aff
 38	7.3.2.1		direct_8x8_inference_flag				sps->direct_8x8_inference_flag
 38	7.3.2.1		frame_cropping_flag					sps->crop
 38	7.3.2.1		frame_crop_left_offset					sps->crop_left
 38	7.3.2.1		frame_crop_right_offset					sps->crop_right
 38	7.3.2.1		frame_crop_top_offset					sps->crop_top
 38	7.3.2.1		frame_crop_bottom_offset				sps->crop_bottom
 44	7.3.3		slice_type						h->slice_type_fixed, h->slice_type
 44	7.3.3		pic_parameter_set_id					h->pps = map[ <-- ], h->sps = map [ <-- ]
 44	7.3.3		frame_num						h->frame_num
 44     7.3.3		0							h->mb_mbaff
 44     7.3.3		field_pic_flag == false					h->mb_aff_frame := sps.mb_aff
 44	7.3.3		frame_mbs_only_flag, field_pic_flag			s->picture_structure = f( <-- )
 48	7.3.4		
320	E.1.1		sar_width						sps->sar.num
320	E.1.1		sar_height						sps->sar.den
320	E.1.1		timing_info_present_flag				sps->timing_info_present_flag
320	E.1.1		num_units_in_tick					sps->num_units_in_tick
320	E.1.1		time_scale						sps->time_scale
320	E.1.1		fixed_frame_rate_flag					sps->fixed_frame_rate_flag
321	E.1.1		bitstream_restriction_flag				sps->bitstream_restriction_flag				
321	E.1.1		num_reorder_frames					sps->num_reorder_frames




TODO:
pixel_aspect[aspect_ratio_idc]			expand Amd.1:2007 aspect ratios in array and array bound test

// TODO: HRD in VUI in SPS not taken since now
//bit_rate_scale					to be included for calculation
//cpb_size_scale					to be included for calculation

first_mb_in_slice					to be included for calculation



<EOF>




More information about the ffmpeg-devel mailing list