[MPlayer-dev-eng] [PATCH] Network synchronized playback using UDP

[Uoti Urpala]

On Thu, 2009-03-26 at 17:17 -0700, Jason Holt wrote:
> > while True:
> >    closebehind = master.pts - THRESHOLD < self.pts <= master.pts
> >    # If nonblocking we just check whether we already received a new UDP
> > packet
> >    master.update_pts(blocking=not closebehind)
> >    if (abs(self.pts - master.pts) > THRESHOLD):
> >        self.seek(master.pts)
> >        break
> >    if (self.pts <= master.pts):
> >        break
> 
> 
> Let's say the master and slave are at 10 seconds, with a 10fps video and
> threshold of 0.15s.  The master seeks back to 8 seconds.  The slave calls
> seek, and let's say that lands us at 5, because that's the nearest keyframe
> before the requested position.

Using a threshold of 0.15 s is a bad choice if you expect keyframes to
be seconds apart. The threshold should be a value where we expect
pausing or decoding fast to match seeking as syncing methods, so it
should be of about the same magnitude as keyframe interval, probably a
second or a few.

> !closebehind, so we block.  Now the master is at 8.1, so we seek again and
> we're still at 5.  That continues until the master passes the next keyframe
> on the slave, at which point we hopefully jump to within THRESHOLD of the
> master.  If we don't, then the client ends up showing the frozen keyframe
> nearest behind the master until we get lucky, which might be never if the
> frames are spaced just right.

How do you imagine that "might be never" case to occur? I don't see how
it could be caused by any frame spacing as long as a couple of basic
sanity assumptions hold. Assume a threshold value of 2 seconds, that
seeking to a later target position will never result in going to an
earlier keyframe or vice versa, and that seeking to exactly the time of
a keyframe will go to that keyframe. Now
1) If the slave ever seeks ahead of the master then it should stay at
that position until it is exactly in sync (it can't seek to any other
position, and it won't advance in frames before either).
2) If the slave seeks exactly to the last position from master then the
"closebehind" condition is true on the next iteration and the slave will
decode at full speed without blocking until it is exactly in sync with
the master (no new seek should be triggered after the first one, as it
should take at least 2 seconds before the master can send a new
timestamp that differs by over 2 seconds from the last one).
3) Suppose the slave seeks behind the master. Now if it ever after that
does a seek which goes to a position which is <= its _own_ previous
position then it will have to do that every time until the master nears
a new keyframe, and then either 1) or at least 2) has to occur. If it
never seeks backward of its own position then it's always either seeking
forward or decoding ahead without blocking, and has to catch up with the
master unless it's just too slow to decode in realtime (doing a seek
after blocking every frame to stay a constant number of frames behind
the master is not possible, since so much seeking without advancing
faster would require all frames to be keyframes and then you'd hit 2).

> I tried yours and a couple of other approaches, and I'm pretty sure the one
> below is the most correct (and simplest).  A key observation for me was that

Did you see some practical problem with my suggestion? If so what
threshold value did you use? If you used the 0.15 above then that's not
a good choice (though I think it shouldn't really break things as long
as decoding a frame takes significantly less, or demuxer seeks are not
quite accurate - that may happen depending on which container format you
use).


> Here's the central logic for the slaves:

>       if (my_position < udp_master_position + udp_timing_tolerance) {
>         // play immediately
>       } else {
>         float timing_error, absolute_timing_error;
>         float master_position = wait_for_udp(udp_port);
> 
>         if (master_position == -1.0) {
>           // udp timed out.  use the old value
>         } else {
>           udp_master_position = master_position;
>         }
> 
>         // timing_error < 0 means we're behind the master
>         timing_error = mpctx->sh_video->pts - udp_master_position;
>         absolute_timing_error = (timing_error < 0) ? -timing_error:
> timing_error;
> 
>         if (absolute_timing_error > udp_timing_tolerance ) {
>           abs_seek_pos=SEEK_ABSOLUTE;
>           rel_seek_secs=master_position;
>         }
>       }

This looks inferior to my suggestion, as you don't guarantee getting the
slaves exactly in sync unless you use some infinitesimal
udp_timing_tolerance, and if you do that then you need to rely more on
demuxer seek behavior to get sync in practice.

> Updated patch attached.

Wasn't really.