[Libav-user] Using Libav to make HLS clips from H264

[Carl Eugen Hoyos]

2017-11-20 17:50 GMT+01:00 Tyler Brooks <tylerjbrooks at gmail.com>:
> I am using a Hi35xx camera processor from HiSilicon.  It is an Arm9 with a
> video pipeline bolted on the side.  At one end of the pipeline is the CMOS
> sensor.  At the other end is a H264 encoder.  When I turn on the pipeline,
> the encoder outputs H264 NAL packets like this:
>
>     frame0: <SPS>,<PPS>,<SEI>,<key frame>
>     frame1: <delta frame>
>     frame2: <delta frame>
>     ...
>     frameN: <delta frame>
>     frameN+1: <SPS>,<PPS>,<SEI><key frame>
>     frameN+2: <delta frame>
>     frameN+3: <delta frame>
>     ...
>     etc.
>
> I am turning that into HLS clips by doing the following (pseudo code for
> clarity) :
>
>     av_register_all();
>     avformat_network_init();
>
>     avformat_alloc_output_context2(&ctx_out, NULL, "hls", "./foo.m3u8");
>
>     strm_out = avformat_new_stream(ctx_out, NULL);
>
>     codec_out = strm_out->codecpar;
>     codec_out->codec_id = AV_CODEC_ID_H264;
>     codec_out->codec_type = AVMEDIA_TYPE_VIDEO;
>     codec_out->width = encoder_width;
>     codec_out->height = encoder_height;
>     codec_out->bit_rate = encoder_bitrate;
>     codec_out->codec_tag = 0;
>
>     avformat_write_header(ctx_out, NULL);
>
>     while(get_packet_from_pipeline_encoder(&encoder_packet)) {
>       AVPacket pkt;
>       av_init_packet(&pkt);
>       pkt.stream_index = 0;
>
>       pkt.dts = AV_NOPTS_VALUE;
>       pkt.pts = AV_NOPTS_VALUE;
>       pkt.duration = (1000000/FRAMERATE);    // frame rate in microseconds
>
>       pkt.data = encoder_packet.data;
>       pkt.size = encoder_packet.size;
>
>       if (is_keyframe(&encoder_packet)) {
>         pkt.flags |= AV_PKT_FLAG_KEY;
>       }
>
>       av_write_frame(ctx_out, &pkt);
>     }
>
>     av_write_trailer(ctx_out);
>     avformat_free_context(ctx_out);
>
> This seems to work fine except that the resulting HLS frame rate is not
> right.  Of course, this happens because I am not setting the pts/dts stuff
> correctly and ffmpeg lets me know that.  So I have two quetions:
>
>  1. Am I going about this right?
>  2. How can I set the pts/dts stuff correctly?
>
> The encoder is giving me packets and I am submitting them as frames.

> Those `<SPS>, <PPS> and <SEI>` packets are really out of band
> data and don't really have a timestamp.

If the encoder does not tell you the timestamps of the encoded
frames, how are you - unrelated to FFmpeg ! - supposed to
write files, no matter the format?

Carl Eugen