[FFmpeg-user] Conversion Failed When Converting HEVC RTP Stream to MPEG-TS

[Assaf Inbal]

Hey all,

I'm having issues with storing an RTP stream with HEVC/AAC MPEG-TS
chunks. The command I'm using is:
> ffmpeg -protocol_whitelist file,udp,rtp -i /path/to/sdp -map 0 -c copy -segment_time 5 -break_non_keyframes 1 -individual_header_trailer 0 -f segment  /path/to/output/recording_%d.ts

What I've noticed is that if FFmpeg receives an IDR, i.e., SPS/PPS/VPS
+ I frame, as the very first packets I get the following errors:
> [segment @ 0x556451a37d00] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
> [segment @ 0x556451a37d00] packet stream:0 pts:NOPTS pts_time:NOPTS dts:0 dts_time:0 duration_time:0.02 is_key:1 frame:0
> [segment @ 0x556451a37d00] segment:'/path/to/output/recording__0.ts' starts with packet stream:0 pts:NOPTS pts_time:NOPTS frame:0
> [mpegts @ 0x556451a52f00] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
> [mpegts @ 0x556451a52f00] first pts value must be set
> av_interleaved_write_frame(): Invalid data found when processing input
> No more output streams to write to, finishing.
> Conversion failed!

As can be seen from the message above, it seems the first frame
doesn't have PTS nor DTS defined and that causes it to exit.
If, however, FFmpeg receives the packets mid-stream and needs to wait
for an IDR to start, then everything works as expected.

I've tried to debug it and noticed that the RTP parser does take the
timestamp from the RTP header and sets it accordingly for each part of
the frame but later on, after the RTP payloads are combined to a
single AVpacket, the timestamps are not set for the first frame. The
following frames seem to be set properly. In the second case where
FFmpeg is started mid-stream, the first frame it receives is partial
and so later dropped and the following (P) frames have their correct
timestamp and MPEG-TS doesn't complain about them.
I believe the problem lies somewhere around the `ff_fetch_timestamp()`
where it starts off with setting PTS to AV_NOPTS_VALUE and doesn't set
a different one afterwards. If I apply the following patch, it seems
to fix this use-case but it breaks some unit tests so I'm not sure
this is the correct solution:
> diff --git a/libavcodec/parser.c b/libavcodec/parser.c
> index a63f532c48..ff0aed09d2 100644
> --- a/libavcodec/parser.c
> +++ b/libavcodec/parser.c
> @@ -99,7 +99,7 @@ void ff_fetch_timestamp(AVCodecParserContext *s, int off, int remove, int fuzzy)
>      for (i = 0; i < AV_PARSER_PTS_NB; i++) {
>          if (s->cur_offset + off >= s->cur_frame_offset[i] &&
>              (s->frame_offset < s->cur_frame_offset[i] ||
> -             (!s->frame_offset && !s->next_frame_offset)) && // first field/frame
> +             (!s->frame_offset && s->next_frame_offset == -1)) && // first field/frame
>              // check disabled since MPEG-TS does not send complete PES packets
>              /*s->next_frame_offset + off <*/  s->cur_frame_end[i]){
>

Does anyone have any idea why this is happening and can point me in
the right direction for overcoming this issue?

Thanks in advance,
Assaf