[MPlayer-dev-eng] pts audio sync/-tv audio/video interleaving

[Charles Henrich]

> if you know that you don't need dropped/dupped frames, and you can provide
> always synced a-v, just set correction values to zero, by disabling code
> setting them to nonzero...
> 
> anyway it shouldn't desync unless something is wrong in the demuxer.

So Arpi, from your point of view (bugs excluded of course), if the demuxer
does (essentially):

Assuming audio is set correctly, if packets are added to the stream when
requested with proper PTS values, it should sync them together correctly, with
or without -mc 0, correct?

If you could look at these relevant code sections and see if you see anything
obviously wrong that would be cool...

AUDIO SETUP:

    funcs->control(tvh->priv, TVI_CONTROL_AUD_SET_SAMPLERATE,
                  &tv_param_audiorate);

    sh_audio = new_sh_audio(demuxer, 0);
    sh_audio->wf = (WAVEFORMATEX *)malloc(sizeof(WAVEFORMATEX));

    sh_audio->format = sh_audio_format;
    sh_audio->sample_format = audio_format;

    funcs->control(tvh->priv, TVI_CONTROL_AUD_GET_SAMPLERATE,
                   &sh_audio->samplerate);
    funcs->control(tvh->priv, TVI_CONTROL_AUD_GET_SAMPLESIZE,
                   &sh_audio->samplesize);
    funcs->control(tvh->priv, TVI_CONTROL_AUD_GET_CHANNELS,
                   &sh_audio->channels);

    sh_audio->wf->nBlockAlign = sh_audio->wf->nAvgBytesPerSec;
    demuxer->audio->sh = sh_audio;
    sh_audio->ds = demuxer->audio;
    demuxer->audio->id = 0;

    sh_audio->o_bps = sh_audio->samplerate * sh_audio->samplesize *
                      sh_audio->channels;
    sh_audio->i_bps = sh_audio->samplerate * sh_audio->samplesize *
                      sh_audio->channels;

    // uncompressed PCM format
    sh_audio->wf->wFormatTag = sh_audio->format;
    sh_audio->wf->nChannels = sh_audio->channels;
    sh_audio->wf->wBitsPerSample = sh_audio->samplesize*8;
    sh_audio->wf->nSamplesPerSec = sh_audio->samplerate;
    sh_audio->wf->nAvgBytesPerSec = sh_audio->wf->nChannels *
                                    sh_audio->samplesize *
                                    sh_audio->samplerate;

AUDIO READ DATA (grab_audio_frame returns exactly len bytes each call, which
is i_bps/fps) :

    if (ds==demux->audio && tv_param_noaudio == 0 &&
        tvh->functions->control(tvh->priv,
                                TVI_CONTROL_IS_AUDIO, 0) == TVI_CONTROL_TRUE)
        {
        len = tvh->functions->get_audio_framesize(tvh->priv);

        do {
            tvh->seq++;

            dp=new_demux_packet(len);
            aframeswaiting=tvh->functions->grab_audio_frame(tvh->priv,
                                                            dp->buffer,len);
            dp->pts=tvh->seq*1.0/sh_video->fps;
            dp->pos=tvh->seq*len;
            ds_add_packet(demux->audio,dp);

           } while (aframeswaiting > 0);

        }

VIDEO READ DATA (len = 0 when no frame available, go back to processing audio,
if I read all the available frames here, it desync's faster, if in this chunk
of code I explicitly pause the process, and wait for a new frame to be
painted, then we stay syncronized, but CPU idle some 50% of the time
(Depending on tunables).  Currently the bt848 code sets up SIGUSR1 to be
called everytime a frame is ready, when ready the frame is copied into a ring
buffer, and PTS time stamp stuck on the frame.  I'd be happy to do
gettimeofday() for this instead of frame counts, if someone could suggest any
way in which to do a gettimeofday() for audio..  Or Ohhhhh!!  Wait, the audio
data can be any size right?  So If I return all audio data in one chunk
(instead of framed) and slap the timeofday() stamp on the last byte read, will
it work it backwards properly?  I guess is PTS beginning of frame or end of
frame?.. Will experiment with this):

    if (ds==demux->video && tvh->functions->control(tvh->priv,
                            TVI_CONTROL_IS_VIDEO, 0) == TVI_CONTROL_TRUE)
        {
        len = tvh->functions->get_video_framesize(tvh->priv);

        if(len == 0) return 1;

        dp=new_demux_packet(len);

        cframe=tvh->functions->grab_video_frame(tvh->priv, dp->buffer, len);

        dp->pos=cframe*len;
        dp->pts=cframe/sh_video->fps;
        ds_add_packet(demux->video,dp);
        }

This is the meat of the syncronization code, anyone who can spot any glaring
errors, Im all ears!  Thanks!

-Crh

       Charles Henrich                                   henrich at msu.edu

                       http://www.sigbus.com:81/~henrich