[FFmpeg-user] how to handle missing pts values to keep audio sync (mpegts->mp4)

[Patrick Fischer]

>

Hello!

 

 
 

This is my first mail on this list. So i hope i didn't got the wrong mailinglist.

 

 
 

Im writing an program which received a mpeg ts udp stream and convert it to mp4 on the fly.

 

Sometimes i have a problem with async audio.

 

 
 

[mpegts @ 0x35fd580] PES packet size mismatch
[mpeg2video @ 0x3604200] skipped MB in I frame at 25 32
[mpeg2video @ 0x3604200] skipped MB in I frame at 1 33
[mpeg2video @ 0x3604200] ac-tex damaged at 0 34
[mpeg2video @ 0x3604200] ac-tex damaged at 0 35
[mpeg2video @ 0x3604200] Warning MVs not available
[mpeg2video @ 0x3604200] concealing 176 DC, 176 AC, 176 MV errors in I frame
[mp2 @ 0x3649fa0] Header missing
[err]failed to decode audio packet

 

 
 

I use udp so it is possible to loose a packages. I have to handle it.

 

 
 

my video_push and audio_push are appended.

 

I dump out the PTS before i push it to the encoder.

 

So you can see that 67 PTS Values are missing.

 

With a 25fps video it is a delay of 2,68 sec. That's exactly what i see if i play the mp4 file via vlc.
 I have tried to skip some video frames, but din't have any effect.
 How to bring back sync audio?
 skip audio or video frames?
 recalc pts values?
 add dummy frames?
 all together?

 

 
 

push audio frame to 689152 (673)
push audio frame to 690176 (674)
[mpegts @ 0x2163580] PES packet size mismatch
push video frame to 392
[mpeg2video @ 0x216a200] skipped MB in I frame at 25 32
[mpeg2video @ 0x216a200] skipped MB in I frame at 1 33
[mpeg2video @ 0x216a200] ac-tex damaged at 0 34
[mpeg2video @ 0x216a200] ac-tex damaged at 0 35
[mpeg2video @ 0x216a200] Warning MVs not available
[mpeg2video @ 0x216a200] concealing 176 DC, 176 AC, 176 MV errors in I frame
push video frame to 393
push video frame to 394<----------------------------- 1 PTS Values is missing!!!!!!!
push video frame to 396
push video frame to 397
[mp2 @ 0x21affa0] Header missing
[err]failed to decode audio packet
push audio frame to 691200 (675)
push audio frame to 692224 (676)
push audio frame to 693248 (677)
push audio frame to 694272 (678)
push audio frame to 695296 (679)
push audio frame to 696320 (680)
push audio frame to 697344 (681)
push audio frame to 698368 (682)
push audio frame to 699392 (683)
push audio frame to 700416 (684)
push audio frame to 701440 (685)
push audio frame to 702464 (686)
push audio frame to 703488 (687)
push audio frame to 704512 (688)
push audio frame to 705536 (689)
push audio frame to 706560 (690)
push audio frame to 707584 (691)
push audio frame to 708608 (692)
push video frame to 398
push video frame to 465                           <----------------------------- 67 PTS Values are missing!!!!!!!
push video frame to 466
 
 

 
 

 
 

 
 

my video_push look something like that:

 

int enc_audio_frame (my_enc_t *enc,AVFrame *audio_frame){

  if ( enc->first_video_pts == AV_NOPTS_VALUE)
        return 0;

  if (enc->first_audio_pts == AV_NOPTS_VALUE) {
        enc->first_audio_pts = audio_frame->pts;
  }

  if (audio_frame->pts != AV_NOPTS_VALUE) {
      audio_frame->pts = av_rescale_q(audio_frame->pts - enc->first_audio_pts, my_decoder_output.a_time_base, enc->conf.a_time_base);
  }
  AVPacket *enc_packet = (AVPacket *) malloc (sizeof (AVPacket));
  av_init_packet(enc_packet);
  enc_packet->size = 0;
  enc_packet->data=NULL;
  int packet_done = 0;
  if ( avcodec_encode_audio2 (enc->audio_codec_ctx,enc_packet,audio_frame,&packet_done) < 0) {
      fprintf (stderr,"[err]%s failed to encode audio samples\n", __FUNCTION__);
      return -1;
  }
  if ( packet_done ) {
    enc_packet->stream_index= 1;
    fprintf (stderr,"push audio frame to %"PRId64" (%"PRId64")\n", audio_frame->pts, audio_frame->pts/1024);
    enc->push(enc->push_priv,enc_packet);
  }
  av_free_packet(enc_packet);
  av_free (enc_packet);

  return 0;

 

}

 

 
 

my audio_push look something like that:

 

int enc_audio_frame (my_enc_t *enc,AVFrame *audio_frame){

  if ( enc->first_video_pts == AV_NOPTS_VALUE)
        return 0;

  if (enc->first_audio_pts == AV_NOPTS_VALUE) {
        enc->first_audio_pts = audio_frame->pts;
  }

  if (audio_frame->pts != AV_NOPTS_VALUE) {
      audio_frame->pts = av_rescale_q(audio_frame->pts - enc->first_audio_pts, my_decoder_output.a_time_base, enc->conf.a_time_base);
  }
  AVPacket *enc_packet = (AVPacket *) malloc (sizeof (AVPacket));
  av_init_packet(enc_packet);
  enc_packet->size = 0;
  enc_packet->data=NULL;
  int packet_done = 0;
  if ( avcodec_encode_audio2 (enc->audio_codec_ctx,enc_packet,audio_frame,&packet_done) < 0) {
      fprintf (stderr,"[err]%s failed to encode audio samples\n", __FUNCTION__);
      return -1;
  }
  if ( packet_done ) {
    enc_packet->stream_index= 1;
    fprintf (stderr,"push audio frame to %"PRId64" (%"PRId64")\n", audio_frame->pts, audio_frame->pts/1024);

 

    enc->push(enc->push_priv,enc_packet);
  }
  av_free_packet(enc_packet);
  av_free (enc_packet);

  return 0;