[Libav-user] HEVC 2/3 of the frame is green
srinivas gudumasu
srinu.try at gmail.com
Thu Mar 30 17:17:02 EEST 2023
Hi,
Refer https://www.rfc-editor.org/rfc/rfc7798.html clause 4.4.3 (Fragmentation
Unit).
When you see the PayloadHdr type as 49 it's a Fragmentation Unit RTP
packet. The FU header has two fields S and E. S bit is set for the
first Fragmentation
Unit of a NAL Unit and E bit is set for the last Fragmentation Unit of a
NAL Unit.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PayloadHdr (Type=49) | FU header | DONL (cond) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-|
| DONL (cond) | |
|-+-+-+-+-+-+-+-+ |
| FU payload |
| |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| :...OPTIONAL RTP padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+---------------+
|0|1|2|3|4|5|6|7|
+-+-+-+-+-+-+-+-+
|S|E| FuType |
+---------------+
Figure 10: The Structure of FU Header
Another way of detecting the last packet of a frame is using a Marker
bit. M bit is set to 1, for the last packet of the access unit,
carried in the current RTP stream.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P|X| CC |M| PT | sequence number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| timestamp |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| synchronization source (SSRC) identifier |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| contributing source (CSRC) identifiers |
| .... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Thanks and Regards,
Srinivas G
On Wed, Mar 29, 2023 at 6:28 AM Michael Wood <mornwood23 at gmail.com> wrote:
>
> On Tue, Mar 28, 2023 at 7:53 PM Michael Wood <mornwood23 at gmail.com> wrote:
>
>> Yeah that is correct,
>>
>> I'm still unable to fix it, i'm trying to figure out what could cause the
>> frame to not be properly decoded.
>>
>> Best,
>> Milos
>>
>> On Sat, Mar 25, 2023 at 2:48 PM Strahinja Radman <dr.strashni at gmail.com>
>> wrote:
>>
>>>
>>>
>>> On 25. 3. 2023., at 13:26, Michael Wood <mornwood23 at gmail.com> wrote:
>>>
>>> Hi Strahinja,
>>>
>>> Thanks for the help,
>>>
>>> I'm wondering if I need to update extradata (VPS,SPS,PPS) after initial
>>> setup of the AVCodecContext from the initial value coming from RTP ?
>>>
>>> Best,
>>> Milos
>>>
>>> On Sat, Mar 25, 2023 at 11:25 AM Strahinja Radman <dr.strashni at gmail.com>
>>> wrote:
>>>
>>>>
>>>>
>>>> On Sat, 25 Mar 2023 at 11:09, Michael Wood <mornwood23 at gmail.com>
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Fri, Mar 24, 2023 at 1:40 PM Strahinja Radman <
>>>>> dr.strashni at gmail.com> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On 24. 3. 2023., at 10:59, Michael Wood <mornwood23 at gmail.com> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Mar 24, 2023 at 10:28 AM Strahinja Radman <
>>>>>> dr.strashni at gmail.com> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 23. 3. 2023., at 20:19, Michael Wood <mornwood23 at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Mar 23, 2023 at 8:06 PM Strahinja Radman <
>>>>>>> dr.strashni at gmail.com> wrote:
>>>>>>>
>>>>>>>
>>>>>>> On Thu, 23 Mar 2023 at 20:05, Michael Wood <mornwood23 at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Mar 23, 2023 at 6:20 PM Michael Wood <mornwood23 at gmail.com>
>>>>>>> wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> I'm having a weird issue with the decoding of H265 4K stream where
>>>>>>> 1/3 of the frame is rendered normally and the other 2/3 is in green (
>>>>>>> https://ibb.co/C8yFygm). This only happens if i try to decode 4K
>>>>>>> stream, for non 4K streams everything works as expected.
>>>>>>>
>>>>>>> I'm using live555 library to get stream from RTSP server after which
>>>>>>> packets are sent to the ffmpeg to be decoded and later rendered
>>>>>>>
>>>>>>> DESCRIBE response contains all required VPS,SPS,PPS fields that are
>>>>>>> passed as extra data to the codec context.
>>>>>>>
>>>>>>> This is what is returned from DESCRIBE:
>>>>>>>
>>>>>>> v=0
>>>>>>> o=- 1679598602413481 1679598602413481 IN IP4 192.168.1.212
>>>>>>> s=Media Presentation
>>>>>>> e=NONE
>>>>>>> b=AS:5050
>>>>>>> t=0 0
>>>>>>> a=control:rtsp://
>>>>>>> 192.168.1.212/Streaming/Channels/101/?transportmode=unicast&profile=Profile_1
>>>>>>> m=video 0 RTP/AVP 96
>>>>>>> c=IN IP4 0.0.0.0
>>>>>>> b=AS:5000
>>>>>>> a=recvonly
>>>>>>> a=x-dimensions:3840,2160
>>>>>>> a=control:rtsp://
>>>>>>> 192.168.1.212/Streaming/Channels/101/trackID=1?transportmode=unicast&profile=Profile_1
>>>>>>> a=rtpmap:96 H265/90000
>>>>>>> a=fmtp:96 sprop-vps=QAEMA///AWAAAAMAgAAAAwAAAwCWAAC94Eg=;
>>>>>>> sprop-sps=QgEDAWAAAAMAgAAAAwAAAwCWAACgAeAgAhx/jb33am7uS/YC3AQEBBAAAD6AAALuByHe5RAAmRgACsPgAEyMAAVh8EACZGAAKw+AATIwABWHwg==;
>>>>>>> sprop-pps=RAHBcrCcCgFGZA==
>>>>>>>
>>>>>>> a=Media_header:MEDIAINFO=494D4B48010300000400050000000000000000000000000081000000000000000000000000000000;
>>>>>>> a=appversion:1.0
>>>>>>>
>>>>>>> When trying to decode packets i'm getting this output from hevc, the
>>>>>>> ffplay/VLC works normally with the incoming stream.
>>>>>>>
>>>>>>> [hevc @ 0x14ac15600] nal_unit_type: 32(VPS), nuh_layer_id: 0,
>>>>>>> temporal_id: 0
>>>>>>> [hevc @ 0x14ac15600] nal_unit_type: 33(SPS), nuh_layer_id: 0,
>>>>>>> temporal_id: 0
>>>>>>> [hevc @ 0x14ac15600] nal_unit_type: 34(PPS), nuh_layer_id: 0,
>>>>>>> temporal_id: 0
>>>>>>> [hevc @ 0x14ac15600] Decoding VPS
>>>>>>> [hevc @ 0x14ac15600] Main profile bitstream
>>>>>>> [hevc @ 0x14ac15600] Decoding SPS
>>>>>>> [hevc @ 0x14ac15600] Main profile bitstream
>>>>>>> [hevc @ 0x14ac15600] Decoding VUI
>>>>>>> [hevc @ 0x14ac15600] Decoding PPS
>>>>>>> [hevc @ 0x14ac15600] nal_unit_type: 32(VPS), nuh_layer_id: 0,
>>>>>>> temporal_id: 0
>>>>>>> [hevc @ 0x14ac15600] Decoding VPS
>>>>>>> [hevc @ 0x14ac15600] Main profile bitstream
>>>>>>> [hevc @ 0x14ac15600] nal_unit_type: 33(SPS), nuh_layer_id: 0,
>>>>>>> temporal_id: 0
>>>>>>> [hevc @ 0x14ac15600] Decoding SPS
>>>>>>> [hevc @ 0x14ac15600] Main profile bitstream
>>>>>>> [hevc @ 0x14ac15600] Decoding VUI
>>>>>>> [hevc @ 0x14ac15600] nal_unit_type: 34(PPS), nuh_layer_id: 0,
>>>>>>> temporal_id: 0
>>>>>>> [hevc @ 0x14ac15600] Decoding PPS
>>>>>>> [hevc @ 0x14ac15600] nal_unit_type: 19(IDR_W_RADL), nuh_layer_id: 0,
>>>>>>> temporal_id: 0
>>>>>>> [hevc @ 0x14ac15600] Format yuvj420p chosen by get_format().
>>>>>>> [hevc @ 0x14ac15600] Output frame with POC 0.
>>>>>>> [hevc @ 0x14ac15600] nal_unit_type: 19(IDR_W_RADL), nuh_layer_id: 0,
>>>>>>> temporal_id: 0
>>>>>>> [hevc @ 0x14ac15600] First slice in a frame missing.
>>>>>>> [hevc @ 0x14ac15600] nal_unit_type: 19(IDR_W_RADL), nuh_layer_id: 0,
>>>>>>> temporal_id: 0
>>>>>>> [hevc @ 0x14ac15600] First slice in a frame missing.
>>>>>>> [hevc @ 0x14ac15600] nal_unit_type: 32(VPS), nuh_layer_id: 0,
>>>>>>> temporal_id: 0
>>>>>>> [hevc @ 0x14ac15600] Decoding VPS
>>>>>>> [hevc @ 0x14ac15600] Main profile bitstream
>>>>>>> [hevc @ 0x14ac15600] nal_unit_type: 33(SPS), nuh_layer_id: 0,
>>>>>>> temporal_id: 0
>>>>>>> [hevc @ 0x14ac15600] Decoding SPS
>>>>>>> [hevc @ 0x14ac15600] Main profile bitstream
>>>>>>> [hevc @ 0x14ac15600] Decoding VUI
>>>>>>> [hevc @ 0x14ac15600] nal_unit_type: 34(PPS), nuh_layer_id: 0,
>>>>>>> temporal_id: 0
>>>>>>> [hevc @ 0x14ac15600] Decoding PPS
>>>>>>> [hevc @ 0x14ac15600] nal_unit_type: 19(IDR_W_RADL), nuh_layer_id: 0,
>>>>>>> temporal_id: 0
>>>>>>> [hevc @ 0x14ac15600] Output frame with POC 0.
>>>>>>> [hevc @ 0x14ac15600] nal_unit_type: 19(IDR_W_RADL), nuh_layer_id: 0,
>>>>>>> temporal_id: 0
>>>>>>> [hevc @ 0x14ac15600] First slice in a frame missing.
>>>>>>> [hevc @ 0x14ac15600] nal_unit_type: 19(IDR_W_RADL), nuh_layer_id: 0,
>>>>>>> temporal_id: 0
>>>>>>> [hevc @ 0x14ac15600] First slice in a frame missing.
>>>>>>> [hevc @ 0x14ac15600] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0,
>>>>>>> temporal_id: 2
>>>>>>> [hevc @ 0x14ac15600] Output frame with POC 1.
>>>>>>> [hevc @ 0x14ac15600] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0,
>>>>>>> temporal_id: 2
>>>>>>> [hevc @ 0x14ac15600] First slice in a frame missing.
>>>>>>> [hevc @ 0x14ac15600] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0,
>>>>>>> temporal_id: 2
>>>>>>> [hevc @ 0x14ac15600] First slice in a frame missing.
>>>>>>> [hevc @ 0x14ac15600] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0,
>>>>>>> temporal_id: 2
>>>>>>> [hevc @ 0x14ac15600] Output frame with POC 2.
>>>>>>> [hevc @ 0x14ac15600] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0,
>>>>>>> temporal_id: 2
>>>>>>> [hevc @ 0x14ac15600] First slice in a frame missing.
>>>>>>> [hevc @ 0x14ac15600] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0,
>>>>>>> temporal_id: 2
>>>>>>> [hevc @ 0x14ac15600] First slice in a frame missing.
>>>>>>>
>>>>>>> If I write the stream to the file and play that file either through
>>>>>>> ffplay or VLC the frames are rendered normally.
>>>>>>>
>>>>>>> Any suggestion would be welcomed.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Milos
>>>>>>>
>>>>>>> Hey Strahinja,
>>>>>>>
>>>>>>> Thanks for the answer,
>>>>>>>
>>>>>>> Can you provide more info on how to do it ?
>>>>>>>
>>>>>>> Best,
>>>>>>> Milos
>>>>>>> _______________________________________________
>>>>>>> Libav-user mailing list
>>>>>>> Libav-user at ffmpeg.org
>>>>>>> https://ffmpeg.org/mailman/listinfo/libav-user
>>>>>>>
>>>>>>> To unsubscribe, visit link above, or email
>>>>>>> libav-user-request at ffmpeg.org with subject "unsubscribe".
>>>>>>> Hey Milos,
>>>>>>>
>>>>>>> In AVFrame structure, check width, height and stride variables.
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> Regards
>>>>>>> Strahinja Radman
>>>>>>> _______________________________________________
>>>>>>> Libav-user mailing list
>>>>>>> Libav-user at ffmpeg.org
>>>>>>> https://ffmpeg.org/mailman/listinfo/libav-user
>>>>>>>
>>>>>>> To unsubscribe, visit link above, or email
>>>>>>> libav-user-request at ffmpeg.org with subject "unsubscribe".
>>>>>>>
>>>>>>> Hey Strahinja,
>>>>>>>
>>>>>>> It seems that both width & height are set correctly. I can't
>>>>>>> find stride, i don't see any reference to it in the AVFrame header.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Milos
>>>>>>> _______________________________________________
>>>>>>> Libav-user mailing list
>>>>>>> Libav-user at ffmpeg.org
>>>>>>> https://ffmpeg.org/mailman/listinfo/libav-user
>>>>>>>
>>>>>>> To unsubscribe, visit link above, or email
>>>>>>> libav-user-request at ffmpeg.org with subject "unsubscribe”.
>>>>>>>
>>>>>>>
>>>>>>> Hey Milos,
>>>>>>>
>>>>>>> What does a parameter linesize say?
>>>>>>>
>>>>>>> AVFrame->linesize[0]
>>>>>>>
>>>>>>> Kind regards,
>>>>>>> Strahinja
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> Hi Strahinja,
>>>>>>
>>>>>> The linesize looks good and corresponds to frame size
>>>>>>
>>>>>> frame->linesize[0] = 3840
>>>>>> frame->linesize[1] = 1920
>>>>>> frame->linesize[2] = 1920
>>>>>>
>>>>>> Best,
>>>>>> Milos
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Hey Milos,
>>>>>>
>>>>>> Can you use this method to save one frame to the PGM file and check
>>>>>> its contents right after you receive an AVFrame
>>>>>>
>>>>>> static void pgm_save(unsigned char* buf, int wrap, int xsize, int
>>>>>> ysize, const char* filename)
>>>>>> {
>>>>>> FILE* f;
>>>>>> int i;
>>>>>> f = fopen(filename, "w");
>>>>>> fprintf(f, "P5\n%d %d\n%d\n", xsize, ysize, 255);
>>>>>> for (i = 0; i < ysize; i++)
>>>>>> fwrite(buf + i * wrap, 1, xsize, f);
>>>>>> fclose(f);
>>>>>> }
>>>>>>
>>>>>> buf is frame->data[0]
>>>>>> wrap is frame->linesize[0]
>>>>>> xsize is width and ysize is height
>>>>>>
>>>>>> Kind regards,
>>>>>> Strahinja
>>>>>>
>>>>>>
>>>>> Hi Strahinja,
>>>>>
>>>>> I'm getting the same result with saved picture, here is the link to
>>>>> the output (
>>>>> https://drive.google.com/file/d/1zEABl32ehVu-gnPdqjWciAb7im18VZIf/view?usp=sharing
>>>>> )
>>>>>
>>>>> Best,
>>>>> Milos
>>>>>
>>>>>
>>>>
>>>> Hey Milos,
>>>>
>>>> One thing to check is if live555 is configured correctly. Check if
>>>> every decoder field is set properly and contains expected values. Sometimes
>>>> stream properties can change during the decoding process and that can
>>>> affect the output.
>>>>
>>>> Kind regards,
>>>> Strahinja
>>>>
>>>>> --
>>>>
>>>> Regards
>>>> Strahinja Radman
>>>>
>>>
>>> Hey Milos,
>>>
>>> That shouldnt be necessary, that information should be used while
>>> configuring the decoder context.
>>>
>>> Kind regards,
>>> Strahinja
>>>
>>>
>
> Update:
>
> So I've managed to figure out what is happening, the NAL units I'm
> receiving from the live555 library are actually fragments of a single
> picture. Technically I need to combine those fragments into a single packet
> and send it to ffmpeg for decoding.
>
> Any suggestions on how to recognize which fragments belong to a single
> packet ?
>
> Best,
> Milos
> _______________________________________________
> Libav-user mailing list
> Libav-user at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/libav-user
>
> To unsubscribe, visit link above, or email
> libav-user-request at ffmpeg.org with subject "unsubscribe".
>
--
Thanks and Regards,
Srinivas G
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://ffmpeg.org/pipermail/libav-user/attachments/20230330/33bb6a06/attachment.htm>
More information about the Libav-user
mailing list