[rtmpdump] [PATCH] Add option --realtime (-R) to rtmpdump to disable the BUFX hack

Ulrik Dickow udickow at gmail.com
Wed Jul 25 19:08:16 CEST 2012


Hello Howard Chu, Steven Penny, Jindřich Makovička and others,

The attached patch enables the user to seek/resume and easily get a
fully valid output file from servers where the BUFX Pause/Unpause hack
makes the server jump backwards in time.  It is a third solution to the
problem described in these two threads:

  [rtmpdump] RTMPDump failing to record stream from GeorgiaTech Server
  http://lists.mplayerhq.hu/pipermail/rtmpdump/2012-May/001985.html

  [rtmpdump] Crude fix rtmpdump repeats, "jumps" timestamps backwards
  http://lists.mplayerhq.hu/pipermail/rtmpdump/2012-May/001947.html

Before getting deeper into my own patch, I'll comment on those two threads.

In the former thread, Steven Penny offers this solution:

> Use -v, brotha

Yes, the --live/-v option also avoided the jump-problem in my case, but
then rtmpdump ignored my --start option:

  WARNING: Can't seek in a live stream, ignoring --start option

The source and docs say all over the place that seek/resume in live
streams is not allowed, so I won't argue against that.  Keep it so.

In the latter thread, Jindřich Makovička offers a patch that silently
drops incoming packets with timestamps smaller than the last one saved
(for audio and video separately).  This solution doesn't sacrifice the
ability to seek/resume.  But when the jumps backward are large and
frequent, it may increase the download time and consumed bandwidth quite
a lot.  In my case by a factor 3 to 4 on both parameters, compared with
my own patch.  Additionally, the packet-dropping solution might give
small glitches in the final output.  It's probably a good idea to add
such a packet-dropping patch anyway, but I think it should at least once
warn the user that a backwards jump has happened -- and if my patch is
added too, it can suggest that the user retries the download with that
new switch turned on instead.

The essential change in my own patch is this in rtmpdump.c:

     /* Try to keep the stream moving if it pauses on us */
  -  if (!bLiveStream && !(protocol & RTMP_FEATURE_HTTP))
  +  if (!bLiveStream && !bRealtimeStream && !(protocol &
RTMP_FEATURE_HTTP))
       rtmp.Link.lFlags |= RTMP_LF_BUFX;

The rest of the patch is just setting and documenting the new option
properly.  "Realtime" is not a perfect name, better suggestions are
welcome.  In my case, it actually downloaded 100 seconds of video+audio
in only 41.5 seconds, i.e. almost 2.5 times faster than realtime:

  $ date;\time rtmpdump ... --start 713 --stop 813 -R -o ...;date
  ons jul 25 14:46:52 CEST 2012
  RTMPDump 2.4.git20120724.g7689787
  (c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
  Connecting ...
  INFO: Connected...
  Starting download at: 0.000 kB
    in approximately realtime (disabled BUFX speedup hack)
  For duration: 100.000 sec
  [...]
  INFO:   displayWidth          1024.00
  INFO:   displayHeight         576.00
  INFO:   framerate             25.00
  INFO:   moovposition          321937506.00
  INFO:   duration              1860.03
  17201.495 kB / 813.63 sec (43.7%)
  Download may be incomplete (downloaded about 43.70%), try resuming
  0.16user 0.43system 0:41.50elapsed 1%CPU (0avgtext+0avgdata
4660maxresident)k
  0inputs+34408outputs (0major+1921minor)pagefaults 0swaps
  ons jul 25 14:47:33 CEST 2012

(My patch doesn't change VERSION; I only did that in the make(1) command).

Without the --realtime/-R switch, it still defaults to using the BUFX
hack.  Tested right before the above test with the same rtmpdump and
command line, except for deleting '-R', it then made the first jump
backwards already at timestamp 720.64, then proceeded until timestamp
776.34, where it jumped all the way back to close to ts 721 again.  Soon
after, I Ctrl-C'ed it.

Five days earlier, using
rtmpdump-2.4-0.1.20110811gitc58cfb3e.fc16.x86_64, it happened to jump in
smaller steps, much more frequently, making the download from ts 704 to
822 (118 seconds) last 150 seconds and result in a huge and jumpy file.

I tested my patch on Fedora 16 x86_64 Linux, adding only '-j8
VERSION=...' to the make(1) command line.  Only tested initial seek
(--start), not resuming a download.

Finally I should mention that the post by KSV here was a great help,
crucially mentioning the "RTMP_LF_BUFX hack":

  http://stream-recorder.com/forum/jumps-back-footage-t12927.html

Regards,

-- 
Ulrik Dickow
-------------- next part --------------
From 7689787f1adc8d6f63e61e39572f6a66d54a614c Mon Sep 17 00:00:00 2001
From: Ulrik Dickow <u.dickow at gmail.com>
Date: Tue, 24 Jul 2012 17:17:26 +0200
Subject: [PATCH] Add option --realtime (-R) to rtmpdump to disable the BUFX
 hack

---
 rtmpdump.1      |    8 +++++++-
 rtmpdump.1.html |   12 +++++++++++-
 rtmpdump.c      |   19 ++++++++++++++-----
 3 files changed, 32 insertions(+), 7 deletions(-)

diff --git a/rtmpdump.1 b/rtmpdump.1
index 0d9de8d..7bb5328 100644
--- a/rtmpdump.1
+++ b/rtmpdump.1
@@ -1,4 +1,4 @@
-.TH RTMPDUMP 1 "2011-07-20" "RTMPDump v2.4"
+.TH RTMPDUMP 1 "2012-07-24" "RTMPDump v2.4"
 .\" Copyright 2011 Howard Chu.
 .\" Copying permitted according to the GNU General Public License V2.
 .SH NAME
@@ -177,6 +177,12 @@ live streams is possible.
 Name of live stream to subscribe to. Defaults to
 .IR playpath .
 .TP
+.B \-\-realtime	\-R
+Download approximately in realtime, without attempting to speed up via
+Pause/Unpause commands ("the BUFX hack").
+Useful for servers that jump backwards in time at the Unpause command.
+Resuming and seeking in realtime streams is still possible.
+.TP
 .B \-\-resume		\-e
 Resume an incomplete RTMP download.
 .TP
diff --git a/rtmpdump.1.html b/rtmpdump.1.html
index 826f722..4c39b35 100644
--- a/rtmpdump.1.html
+++ b/rtmpdump.1.html
@@ -6,7 +6,7 @@
 <tr><td>RTMPDUMP(1)<td align="center"><td align="right">RTMPDUMP(1)
 </thead>
 <tfoot>
-<tr><td>RTMPDump v2.4<td align="center">2011-07-20<td align="right">RTMPDUMP(1)
+<tr><td>RTMPDump v2.4<td align="center">2012-07-24<td align="right">RTMPDUMP(1)
 </tfoot>
 <tbody><tr><td colspan="3"><br><br><ul>
 <!-- Copyright 2011 Howard Chu.
@@ -34,6 +34,7 @@ rtmpdump − RTMP streaming media client
 [<b>−y</b><i> playpath</i>]
 [<b>−Y</b>]
 [<b>−v</b>]
+[<b>−R</b>]
 [<b>−d</b><i> subscription</i>]
 [<b>−e</b>]
 [<b>−k</b><i> skip</i>]
@@ -218,6 +219,15 @@ Name of live stream to subscribe to. Defaults to
 </dl>
 <p>
 <dl compact><dt>
+<b>−−realtime −R</b>
+<dd>
+Download approximately in realtime, without attempting to speed up via
+Pause/Unpause commands ("the BUFX hack").
+Useful for servers that jump backwards in time at the Unpause command.
+Resuming and seeking in realtime streams is still possible.
+</dl>
+<p>
+<dl compact><dt>
 <b>−−resume −e</b>
 <dd>
 Resume an incomplete RTMP download.
diff --git a/rtmpdump.c b/rtmpdump.c
index 34bfdba..e52f7d4 100644
--- a/rtmpdump.c
+++ b/rtmpdump.c
@@ -441,7 +441,7 @@ GetLastKeyframe(FILE * file,	// output file [in]
 
 int
 Download(RTMP * rtmp,		// connected RTMP object
-	 FILE * file, uint32_t dSeek, uint32_t dStopOffset, double duration, int bResume, char *metaHeader, uint32_t nMetaHeaderSize, char *initialFrame, int initialFrameType, uint32_t nInitialFrameSize, int nSkipKeyFrames, int bStdoutMode, int bLiveStream, int bHashes, int bOverrideBufferTime, uint32_t bufferTime, double *percent)	// percentage downloaded [out]
+	 FILE * file, uint32_t dSeek, uint32_t dStopOffset, double duration, int bResume, char *metaHeader, uint32_t nMetaHeaderSize, char *initialFrame, int initialFrameType, uint32_t nInitialFrameSize, int nSkipKeyFrames, int bStdoutMode, int bLiveStream, int bRealtimeStream, int bHashes, int bOverrideBufferTime, uint32_t bufferTime, double *percent)	// percentage downloaded [out]
 {
   int32_t now, lastUpdate;
   int bufferSize = 64 * 1024;
@@ -492,6 +492,8 @@ Download(RTMP * rtmp,		// connected RTMP object
 		    bResume ? "Resuming" : "Starting",
 		    (double) size / 1024.0);
 	}
+      if (bRealtimeStream)
+	RTMP_LogPrintf("  in approximately realtime (disabled BUFX speedup hack)\n");
     }
 
   if (dStopOffset > 0)
@@ -682,6 +684,8 @@ void usage(char *prog)
 	  RTMP_LogPrintf
 	    ("--subscribe|-d string   Stream name to subscribe to (otherwise defaults to playpath if live is specifed)\n");
 	  RTMP_LogPrintf
+	    ("--realtime|-R           Don't attempt to speed up download via the Pause/Unpause BUFX hack\n");
+	  RTMP_LogPrintf
 	    ("--flv|-o string         FLV output file name, if the file name is - print stream to stdout\n");
 	  RTMP_LogPrintf
 	    ("--resume|-e             Resume a partial RTMP download\n");
@@ -748,6 +752,7 @@ main(int argc, char **argv)
   int protocol = RTMP_PROTOCOL_UNDEFINED;
   int retries = 0;
   int bLiveStream = FALSE;	// is it a live stream? then we can't seek/resume
+  int bRealtimeStream = FALSE;  // If true, disable the BUFX hack (be patient)
   int bHashes = FALSE;		// display byte counters not hashes by default
 
   long int timeout = DEF_TIMEOUT;	// timeout connection after 120 seconds
@@ -832,6 +837,7 @@ main(int argc, char **argv)
 #endif
     {"flashVer", 1, NULL, 'f'},
     {"live", 0, NULL, 'v'},
+    {"realtime", 0, NULL, 'R'},
     {"flv", 1, NULL, 'o'},
     {"resume", 0, NULL, 'e'},
     {"timeout", 1, NULL, 'm'},
@@ -851,7 +857,7 @@ main(int argc, char **argv)
 
   while ((opt =
 	  getopt_long(argc, argv,
-		      "hVveqzr:s:t:p:a:b:f:o:u:C:n:c:l:y:Ym:k:d:A:B:T:w:x:W:X:S:#j:",
+		      "hVveqzRr:s:t:p:a:b:f:o:u:C:n:c:l:y:Ym:k:d:A:B:T:w:x:W:X:S:#j:",
 		      longopts, NULL)) != -1)
     {
       switch (opt)
@@ -936,6 +942,9 @@ main(int argc, char **argv)
 	case 'v':
 	  bLiveStream = TRUE;	// no seeking or resuming possible!
 	  break;
+	case 'R':
+	  bRealtimeStream = TRUE; // seeking and resuming is still possible
+	  break;
 	case 'd':
 	  STR2AVAL(subscribepath, optarg);
 	  break;
@@ -1181,7 +1190,7 @@ main(int argc, char **argv)
 		   &flashVer, &subscribepath, &usherToken, dSeek, dStopOffset, bLiveStream, timeout);
 
   /* Try to keep the stream moving if it pauses on us */
-  if (!bLiveStream && !(protocol & RTMP_FEATURE_HTTP))
+  if (!bLiveStream && !bRealtimeStream && !(protocol & RTMP_FEATURE_HTTP))
     rtmp.Link.lFlags |= RTMP_LF_BUFX;
 
   off_t size = 0;
@@ -1348,8 +1357,8 @@ main(int argc, char **argv)
 
       nStatus = Download(&rtmp, file, dSeek, dStopOffset, duration, bResume,
 			 metaHeader, nMetaHeaderSize, initialFrame,
-			 initialFrameType, nInitialFrameSize,
-			 nSkipKeyFrames, bStdoutMode, bLiveStream, bHashes,
+			 initialFrameType, nInitialFrameSize, nSkipKeyFrames,
+			 bStdoutMode, bLiveStream, bRealtimeStream, bHashes,
 			 bOverrideBufferTime, bufferTime, &percent);
       free(initialFrame);
       initialFrame = NULL;
-- 
1.7.7.6


More information about the rtmpdump mailing list