[FFmpeg-devel] [PATCH 0/3] add vvc raw demuxer

Mon Dec 14 15:31:51 EET 2020

Hi Mark,
I have almost done the cbs for sps, pps, and slice header. I will start to
implement the parser.
Few questions for you:
1. We need over-read some nals to detect the frame boundaries. But those
nals may switch/replace sps/pps. Do we need to use an output cbs and get
frames from the output cbs?
2. We can't handle an incompleted nal in current cbs.
https://github.com/FFmpeg/FFmpeg/blob/03c8fe49ea3f2a2444607e541dff15a1ccd7f0c2/libavcodec/h2645_parse.c#L437,
do we have a plan to fix it? What's your suggstion for the frame split?
3. How to well test the cbs?

On Sun, Nov 22, 2020 at 10:49 AM Nuo Mi <nuomi2021 at gmail.com> wrote:

> Thanks for the replies.
>
> On Sun, Nov 22, 2020 at 5:53 AM James Almer <jamrial at gmail.com> wrote:
>
>> On 11/21/2020 2:18 PM, Mark Thompson wrote:
>> > On 21/11/2020 02:19, James Almer wrote:
>> >> On 11/20/2020 10:55 PM, Nuo Mi wrote:
>> >>> Hi Mark, James and all,
>> >>> There are two ways to implement the parser.
>> >>> 1. use cbs like av1 parser.
>> >>> 2. follow h264/h265 parser method.
>> >>> Which is your preferred?
>> >>>
>> >>> thanks
>> >>
>> >> A CBS module for VVC is needed regardless of how the parser is
>> >> ultimately implemented (There will be other uses, like bitstream
>> >> filters), so might as well use it for it, like with AV1.
>> >
>> > +1.
>> >
>> > Attached is some initial investigation I did recently into CBS
>> > implementation.  The non-CBS parts are all hacks to get a raw stream
>> fed
>> > into trace_headers, and can be ignored for non-test purposes.
>> >
>> > My thoughts emerging from that:
>> >
>> > * The commonality with H.26[45] is sufficient that it's probably best
>> to
>> > extend that into CBS H.26[456].
>> >    * All of the styling is the same, so doing that avoids duplicating a
>> > lot of templating code.
>> >    * I haven't done that here because it was easier to hack in by
>> > starting with a new file.
>> >    * Separately I've been looking at merging the implementation of more
>> > common elements of H.26[45], and that will apply here too (SEI in
>> > particular, where H.274 matches in parts as well).
>
> Fully agreed with this. I  did some homework after James replied to me.
> See attachment.
> The code is far behind complete, it's similar to your code and aligned
> with James's suggestion.
> Seems we need pretty large work to enable the h266 cbs. Is it possible to
> open some branch on GitHub, so I can offload your work?
>
>
>
> >
>> > * There is /a lot/ of boilerplate needed to get even a minimal version
>> > working.
>> >    * The parameter sets are even bigger and more complex than they were
>> > in H.265 (and the VUI that you want for metadata is still right at the
>> > end of the SPS).
>> >    * The clean specification is very appreciated - I haven't found any
>> > nasty interdependencies yet, though I haven't read all of it in detail.
>> >    * Given the huge surface area for errors here, it may not be a good
>> > idea to include it until we have the full conformance test suite (like
>> > H.265.1) to test it with.
>>
>  The vvc is finalized, it supposes will not have big syntax changes.
> I download my conformance clips here
> https://www.itu.int/wftp3/av-arch/jvet-site/bitstream_exchange/VVC/draft_conformance/draft5/
>
> >
>> > * It seemed like the right choice on naming was to use H.266 as the
>> name
>> > everywhere and always refer to that:
>> >    * It stays consistent with the most common name for H.264 (still the
>> > most-used codec).
>> >    * It's a formal standard, unlike the JVET drafts.
>> >    * Anyone can download the full specification for free
>> > (<https://www.itu.int/rec/T-REC-H.266-202008-I/en>), unlike the
>> ISO/IEC
>> > 23090-3 version.
>>
>
>  thanks for the link, better than my JVET docuemnet, Let me use it.
>
>
>> The CBS module i absolutely agree should be named h266. Same with
>> h2645_parse, which should be renamed to h26x_parse once adapted to also
>> split H266 NALUs.
>> But for the decoder and parser, it will depend on how the codec turns
>> out to be best known as. Take for example h265, which is called HEVC
>> everywhere, and unlike h264 where AVC is seemingly not used as much.
>> Right now, VVC seems to get more relevant hits on google than h266.
>>
>> And we can add two codec ids for it, like h265/HEVC (Or rather one, and
>> a #define alias).
>>
>> >
>> > I'm not working on this right now, but I'll probably come back to it in
>> > the not-too-distant future.
>> >
>> > So, any thoughts welcome - given the scope of this we should probably
>> > agree roughly what the intended structure will be before writing much
>> more.
>>
>> My opinion, judging by current name usage in most news articles and
>> announcements:
>>
>> h26x_parse
>> cbs_h26x (using h26x_parse for NAL splitting)
>> cbs_h266_syntax_template
>> h266_metadata_bsf (Using CBS)
>> vvc_parser (Using CBS for parameter set and frame header parsing. Raw
>> bitstream merging is separate)
>> vvc_decoder (Maybe using CBS?
>
>
> Native vvc decoder is a huge task. Before this, maybe we can integrate the
> vvdec(https://github.com/fraunhoferhhi/vvdec) to verify the parser and
> cbs.
> vvdec optimized for x86. it's about 37 fps at 1080P on my 4 years old i7.
>
>
>
>
>
>
>
>
> AV_CODEC_ID_VVC
>> #define AV_CODEC_ID_H266 AV_CODEC_ID_VVC
>>
>> >
>> > Thanks,
>> >
>> > - Mark
>> >
>> > _______________________________________________
>> > ffmpeg-devel mailing list
>> > ffmpeg-devel at ffmpeg.org
>> > https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>> >
>> > To unsubscribe, visit link above, or email
>> > ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>> >
>>
>> _______________________________________________
>> ffmpeg-devel mailing list
>> ffmpeg-devel at ffmpeg.org
>> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>>
>> To unsubscribe, visit link above, or email
>> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".
>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cbs_wip.patch
Type: application/octet-stream
Size: 164015 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20201214/84f8291e/attachment.obj>