[MPlayer-dev-eng] Proposed patch for allowing some jitter in mencoder (improved sync handling)
Matthias Hopf
mat at mshopf.de
Fri Feb 24 18:46:09 CET 2006
Hi all,
sorry for crossposting, but this patch needs testing badly. I would
suggest to post test results on mencoder-users, and only discussion
about the code on mplayer-dev-eng.
This is an experimental patch I've been using for the last couple of
weeks. It certainly helps me getting rid of duplicated and skipped
frames in weired streams (especially MPEG transport streams), as long as
their number is leveling out, and in the end it should get perfectly
(i.e. within 1 frame) synchronized audio even for really broken streams.
As it is experimental it is *not* in a state that can be considered for
inclusion. But I'd like to see your comments.
Issues with the current patch:
1) I'd like to change the involved synchronization algorithm once more.
Expect the user controllable parameters to change once more as well,
and that would need re-testing as well.
2) Right now streams with jitter always smaller than <min> will not be
changed, e.g. if <min> is larger than 1 and the stream has a constant
offset of 1 frame. Relates to 1).
3) A-V output figures will be more or less meaningless and close to 0,
as I combined v_timer_corr and v_pts_corr. I added two interesting
jitter values (current and accumulated squared) which are actually
closer to being really informative. Now what to do with A-V output,
nuke it? Change it? Keep it for compatibility's sake? Print out
current jitter there?
4) Combining of v_timer_corr and v_pts_corr has been done in the least
intrusive way. In the very end I hope to nuke one of them. However,
if someone can tell me why and how these should be handled
differently, I'd be glad to update my patch.
5) The behavior of the old algorithm can *somehow* be emulated with
-allowjitter=min=2,max=2,delay=0 (except for not having the
quantization issue any more, so it should be better even in this
case), however the default values (min=1.5,max=2.5,delay=40) try to
get even better synchronization at hopefully lower jitter generation
at the same time.
I somehow explained how the jitter correction works in the manual page
(-allowjitter), tell me if this is not sufficient or misunderstanding.
I can only update German and English.
I would especially be interested whether this patch helps with
previously problematic streams (and if it does with which parameter
set), and whether it breaks any streams that had been encoded fine
before.
Thanks
Matthias
--
Matthias Hopf <mhopf at suse.de> __ __ __
Maxfeldstr. 5 / 90409 Nuernberg (_ | | (_ |__ mat at mshopf.de
Phone +49-911-74053-715 __) |_| __) |__ labs www.mshopf.de
-------------- next part --------------
Index: cfg-mencoder.h
===================================================================
RCS file: /cvsroot/mplayer/main/cfg-mencoder.h,v
retrieving revision 1.107
diff -u -p -u -r1.107 cfg-mencoder.h
--- cfg-mencoder.h 19 Feb 2006 09:34:36 -0000 1.107
+++ cfg-mencoder.h 24 Feb 2006 17:04:07 -0000
@@ -197,6 +197,13 @@ m_option_t of_conf[]={
extern float avi_aspect_override; /* defined in libmpdemux/muxer_avi.c */
extern int write_odml; /* defined in libmpdemux/muxer_avi.c */
+m_option_t jitter_conf[]={
+ {"min", &jitter_min, CONF_TYPE_FLOAT, CONF_MIN, 0.5, 0, NULL},
+ {"max", &jitter_max, CONF_TYPE_FLOAT, CONF_MIN, 1.5, 0, NULL},
+ {"delay", &jitter_delay, CONF_TYPE_FLOAT, CONF_MIN, 0.0, 0, NULL},
+ {NULL, NULL, 0, 0, 0, 0, NULL}
+};
+
m_option_t mencoder_opts[]={
/* name, pointer, type, flags, min, max */
@@ -214,6 +221,9 @@ m_option_t mencoder_opts[]={
{"ofps", &force_ofps, CONF_TYPE_FLOAT, CONF_MIN|CONF_GLOBAL, 0, 0, NULL},
{"o", &out_filename, CONF_TYPE_STRING, CONF_GLOBAL, 0, 0, NULL},
+ // maximum allowed jitter
+ {"allowjitter", jitter_conf, CONF_TYPE_SUBCONFIG, 0, 0, 0, NULL},
+
// limit number of skippable frames after a non-skipped one
{"skiplimit", &skip_limit, CONF_TYPE_INT, 0, 0, 0, NULL},
{"noskiplimit", &skip_limit, CONF_TYPE_FLAG, 0, 0, -1, NULL},
Index: mencoder.c
===================================================================
RCS file: /cvsroot/mplayer/main/mencoder.c,v
retrieving revision 1.342
diff -u -p -u -r1.342 mencoder.c
--- mencoder.c 24 Feb 2006 15:43:20 -0000 1.342
+++ mencoder.c 24 Feb 2006 17:04:07 -0000
@@ -24,6 +24,7 @@
#include <stdlib.h>
#include <string.h>
#include <signal.h>
+#include <math.h>
#include "config.h"
#ifdef __MINGW32__
@@ -153,6 +154,11 @@ static float default_max_pts_correction=
static float max_pts_correction=0;//default_max_pts_correction;
static float c_total=0;
+// To test: 1.5/2.5/25 1.1/2.0/100 or even 1.5/2.5/50
+static float jitter_min=1.5;
+static float jitter_max=2.5;
+static float jitter_delay=40;
+
static float audio_preload=0.5;
static float audio_delay_fix=0.0;
static float audio_delay=0.0;
@@ -389,6 +395,8 @@ off_t muxer_f_size=0;
double v_pts_corr=0;
double v_timer_corr=0;
+double v_jitter_sum=0;
+int skip_flag=0; // >0=skip <0=duplicate
m_entry_t* filelist = NULL;
char* filename=NULL;
@@ -1087,7 +1095,7 @@ while(!at_eof){
int blit_frame=0;
float a_pts=0;
float v_pts=0;
- int skip_flag=0; // 1=skip -1=duplicate
+ double v_delta=0;
if((end_at_type == END_AT_SIZE && end_at <= ftello(muxer_f)) ||
(end_at_type == END_AT_TIME && end_at < mux_v->timer))
@@ -1277,31 +1285,39 @@ if(demuxer2){ // 3-pass encoding, read c
// check frame duplicate/drop:
//printf("\r### %5.3f ###\n",v_timer_corr);
-float mux_frametime = (float)mux_v->h.dwScale/mux_v->h.dwRate;
-
-if (v_timer_corr >= mux_frametime && (skip_limit<0 || skip_flag < skip_limit)) {
- v_timer_corr-=mux_frametime;
- ++skip_flag; // skip
-}
-while (v_timer_corr <= -mux_frametime && (skip_limit<0 || -skip_flag < skip_limit)) {
- v_timer_corr+=mux_frametime;
- --skip_flag; // dup
-}
-
-// either v_pts_corr is big, more than 2 times framerate, then we follow its advice,
-// or, it cancels out v_timer_corr, in which case be happy and do nothing.
-
-while ((v_pts_corr <= -mux_frametime && skip_flag > 0) || (v_pts_corr <= -2*mux_frametime)) {
- v_pts_corr+=mux_frametime;
- --skip_flag; // dup
-}
-if ((v_pts_corr >= mux_frametime && skip_flag < 0) || (v_pts_corr >= 2*mux_frametime)) {
- if (skip_flag<=0) { // we can't skip more than 1 frame now
- v_pts_corr-=mux_frametime;
- ++skip_flag; // skip
- }
-}
+double mux_frametime = (double)mux_v->h.dwScale/mux_v->h.dwRate;
+// v_pts_corr had been used to update skip_flag, which is only a quantized
+// version of adding this to v_timer_corr if there was a skip/dup pending
+// - so we do it non-quantized here
+// TODO: nuke v_pts_corr altogether?!?
+#define COPYSGN(x,y) ((x)>=0?(y):-(y))
+#define ABS(x) COPYSGN(x,x)
+
+ v_timer_corr += v_pts_corr;
+ v_pts_corr = 0;
+ v_delta = v_timer_corr/mux_frametime;
+ if ((v_delta >=0 && v_jitter_sum >= 0) ||
+ (v_delta <=0 && v_jitter_sum <= 0)) {
+ v_jitter_sum += ABS(v_delta)*v_delta;
+ if (ABS(v_delta) > jitter_max) {
+ double v_delta_trunc = COPYSGN(v_delta, floor (ABS(v_delta) - 0.5));
+// double v_delta_trunc = trunc (v_delta);
+ skip_flag += (int) v_delta_trunc;
+ v_timer_corr -= v_delta_trunc * mux_frametime;
+ v_jitter_sum = COPYSGN (v_delta, jitter_delay);
+ } else if (ABS(v_delta) > jitter_min &&
+ ABS(v_jitter_sum) >= jitter_delay) {
+ skip_flag += COPYSGN (v_delta, 1);
+ v_timer_corr -= COPYSGN (v_delta, mux_frametime);
+ v_jitter_sum -= COPYSGN (v_delta, jitter_min*2); // jitter_delay/2? jitter_max?
+ }
+ } else {
+ if (ABS(v_jitter_sum) > jitter_min && ABS(v_delta) < jitter_min)
+ v_jitter_sum /= 2;
+ else
+ v_jitter_sum = 0;
+ }
} // demuxer2
ptimer_start = GetTimerMS();
@@ -1408,12 +1424,13 @@ if(sh_audio && !demuxer2){
// sh_video->timer-=x;
c_total+=x;
v_pts_corr+=x;
+//fprintf (stderr, "\nAV_delay %5.3f x %5.3f max_pts_correction %5.3f v_pts_corr %5.3f v_timer_corr %5.3f\n", AV_delay, x, max_pts_correction, v_pts_corr, v_timer_corr);
+/* printf("\nA:%6.3f V:%6.3f A-V:%7.3f oAV:%7.3f diff:%7.3f ct:%7.3f vpc:%7.3f \n",
+ a_pts,v_pts,a_pts-v_pts,
+ (float)(mux_a->timer-mux_v->timer),
+ AV_delay, c_total, v_pts_corr ); */
}
-// printf("A:%6.1f V:%6.1f A-V:%7.3f oAV:%7.3f diff:%7.3f ct:%7.3f vpc:%7.3f \r",
-// a_pts,v_pts,a_pts-v_pts,
-// (float)(mux_a->timer-mux_v->timer),
-// AV_delay, c_total, v_pts_corr );
// printf("V:%6.1f \r", d_video->pts );
#if 0
@@ -1445,7 +1462,7 @@ if(sh_audio && !demuxer2){
#else
if(!quiet) {
if(verbose>0) {
- mp_msg(MSGT_AVSYNC,MSGL_STATUS,"Pos:%6.1fs %6df (%2d%%) %3dfps Trem:%4dmin %3dmb A-V:%5.3f [%d:%d] A/Vms %d/%d D/B/S %d/%d/%d \r",
+ mp_msg(MSGT_AVSYNC,MSGL_STATUS,"Pos:%6.1fs %6df (%2d%%) %3dfps Trem:%4dmin %3dmb A-V:%5.3f [%d:%d] j %4.2f/%4.2f A/Vms %d/%d D/B/S %d/%d/%d \r",
mux_v->timer, decoded_frameno, (int)(p*100),
(t>1) ? (int)(decoded_frameno/t+0.5) : 0,
(p>0.001) ? (int)((t/p-t)/60) : 0,
@@ -1453,18 +1470,20 @@ if(sh_audio && !demuxer2){
v_pts_corr,
(mux_v->timer>1) ? (int)(mux_v->size/mux_v->timer/125) : 0,
(mux_a && mux_a->timer>1) ? (int)(mux_a->size/mux_a->timer/125) : 0,
+ v_delta, v_jitter_sum,
audiorate/audiosamples, videorate/videosamples,
duplicatedframes, badframes, skippedframes
);
} else
- mp_msg(MSGT_AVSYNC,MSGL_STATUS,"Pos:%6.1fs %6df (%2d%%) %5.2ffps Trem:%4dmin %3dmb A-V:%5.3f [%d:%d]\r",
+ mp_msg(MSGT_AVSYNC,MSGL_STATUS,"Pos:%6.1fs %6df (%2d%%) %5.2ffps Trem:%4dmin %3dmb A-V:%5.3f [%d:%d] jitter %4.2f/%4.2f \r",
mux_v->timer, decoded_frameno, (int)(p*100),
(t>1) ? (float)(decoded_frameno/t) : 0,
(p>0.001) ? (int)((t/p-t)/60) : 0,
(p>0.001) ? (int)(ftello(muxer_f)/p/1024/1024) : 0,
v_pts_corr,
(mux_v->timer>1) ? (int)(mux_v->size/mux_v->timer/125) : 0,
- (mux_a && mux_a->timer>1) ? (int)(mux_a->size/mux_a->timer/125) : 0
+ (mux_a && mux_a->timer>1) ? (int)(mux_a->size/mux_a->timer/125) : 0,
+ v_delta, v_jitter_sum
);
}
#endif
Index: DOCS/man/de/mplayer.1
===================================================================
RCS file: /cvsroot/mplayer/main/DOCS/man/de/mplayer.1,v
retrieving revision 1.208
diff -u -p -u -r1.208 mplayer.1
--- DOCS/man/de/mplayer.1 5 Feb 2006 14:48:02 -0000 1.208
+++ DOCS/man/de/mplayer.1 24 Feb 2006 17:04:07 -0000
@@ -6480,6 +6480,24 @@ allgemeine Kommentare ?ber den Film
.RE
.
.TP
+.B \-allowjitter <option1:option2:...>
+Gibt Parameter f?r die Jitter-Korrektur an. Jitter ist die variierende
+Zeitstempeldifferenz f?r zusammengeh?rige Video- und Audio-Frames und
+wird hier in Anzahl Frames gemessen.
+.sp 1
+Die verf?gbaren Optionen sind:
+.RSs
+.IPs min=<value>
+Minimaler ben?tigter Jitter f?r Framedrops/-skips (Standard ist 1.5)
+.IPs max=<value>
+Maximal erlaubter Jitter - ein ?berschreiten f?hrt sofort zu
+Framedrops/-skips (Standard ist 2.5)
+.IPs delay=<value>
+Aufaddierter quadrierter Jitter der f?r Framedrops/-skips mindestens
+ben?tigt wird (Standard ist 40)
+.RE
+.
+.TP
.B \-noautoexpand
F?ge den Filter expand nicht automatisch in die Filterkette von MEncoder ein.
N?tzlich, um zu kontrollieren, an welcher Stelle der Filterkette die Untertitel
Index: DOCS/man/en/mplayer.1
===================================================================
RCS file: /cvsroot/mplayer/main/DOCS/man/en/mplayer.1,v
retrieving revision 1.1232
diff -u -p -u -r1.1232 mplayer.1
--- DOCS/man/en/mplayer.1 20 Feb 2006 23:50:40 -0000 1.1232
+++ DOCS/man/en/mplayer.1 24 Feb 2006 17:04:08 -0000
@@ -6195,6 +6195,23 @@ general comments about the work
.RE
.
.TP
+.B \-allowjitter <option1:option2:...>
+Specify parameters for jitter correction. Jitter is the varying time stamp
+difference for connected video and audio frames and measured in fractional
+numbers of frames here.
+.sp 1
+Available options are:
+.RSs
+.IPs min=<value>
+minimum jitter necessary for drops/skips (default is 1.5)
+.IPs max=<value>
+maximum allowed jitter - overshooting results in immediate drops/skips
+(default is 2.5)
+.IPs delay=<value>
+accumulated squared jitter necessary for drops/skips (default is 40)
+.RE
+.
+.TP
.B \-noautoexpand
Do not automatically insert the expand filter into the MEncoder filter chain.
Useful to control at which point of the filter chain subtitles are rendered
More information about the MPlayer-dev-eng
mailing list