January 2015 - rtmpdump - lists.mplayerhq.hu

Re: [rtmpdump] Patch: improve recovery by scanning forward following packet headers to find last valid packet offset
by Peter Pickford 12 Feb '15

12 Feb '15

Hi Martin. Thanks for your feedback. rtmpdump is not the easiest of code for me to understand, its been quite a while since I did any c for work. The patch isn't the best c in the world, or even logic, but it did fix the resumes though and if its of any use please send an an improved patch. My approach was to insert some code in the middle to do what I needed with minimum change to the rest. Apologies if my perverse way of thinking about the offsets is a bit perturbing but it was the best I could of think of to explain it. The idea is to find the last 'valid' tag, the last tag header where the pretag field is accessible. and setup tsize so the the existing code would scan backwards from there to the correct seekable tag and resume the download from there. Once it becomes necessary to scan forward from the beginning, it would be better to just scan forward recording the last seekable tag rather that the forwards and back nonsense but that was more invasive and given my lack of understanding of rtmpdump I though the least change the better. It all seems reminiscent I what we used to call relative programming putting in the odd +- 1 till it works. I wonder how difficult it would be to port rtmpdump to go there are already some go rtmp libraries, If I were to invest much more time into rtmpdump that is probably the way I would go. The problem(s) are that you cant look backwards if the last tag isn't correct and the is nothing after the last tag. I have sent a revised version that checks the current tag's previous tag size against the previous tags payloadsize and stops moving forward in the file if they don't match. It also skips that last couple of tags in the file, perhaps that is overkill. Unfortunately I only just saw your post, to many dummy emails and no forwarding etc, so I didn't incorporate your recommendations in the previous post I have removed the necessary seek in the attached patch. I wonder if some more validation would help like checking the flags are correct and the payload size is sensible (not sure what the largest sensible size would be but I suspect its a lot less than 2^24 (or 32bits for the payloadsize). Once that is done the scanning backwards code could just be deleted. There is also a problem with the pause resume functionality when downloading from the bbc sometimes it works fine sometime you get garbage, perhaps if the pause resume failure can be detected switching to a resume strategy would speed up download progress. Some responses inline. Thanks Peter > I was just looking at this code (considering a resume function for a > HDS downloader), so I thought I would try and understand your patch as > well. It looks like this patch is intended to help resume a file with > an incomplete FLV tag at the end. Some comments below; hope they are > useful. > On 5 November 2014 00:04, Peter Pickford <rtmpdump(a)netremedies.ca> wrote: > > diff --git a/rtmpdump.c b/rtmpdump.c > > index 13741a7..5a69395 100644 > > --- a/rtmpdump.c > > +++ b/rtmpdump.c > > @@ -303,8 +303,29 @@ GetLastKeyframe(FILE * file, // output file [in] > > // find the last seekable frame > > off_t tsize = 0; > This initial value no longer used. > > uint32_t prevTagSize = 0; > > + uint32_t headerSize = 0; > > + uint32_t payloadSize = 0; > > + off_t offset = 0; > > + off_t lastGoodTag = 0; > > > > // go through the file and find the last video keyframe > > + fseek(file, 5, SEEK_SET); > This seek shouldn’t be necessary because the flags byte at offset 4 > was just read a few lines up. > True fseek to 4 and a byte is read making position 5. Removed > > > + if (fread(buffer, 1, 4, file) != 4) { > > + RTMP_Log(RTMP_LOGERROR, "Couldn't read headerSize from file!"); > > + return RD_FAILED; > > + } > > + headerSize = AMF_DecodeInt32(buffer); > > + offset = headerSize; > > + while (offset < size) { > What if headerSize >= size? The lastGoodTag = 0 initial value isn’t > going to help. If that's the case nothing is going to help the file is likely totally empty. Wouldn't this be trapped as "Unexpected start of file, error in tag sizes, couldn't arrive at prevTagSize=0" > > + fseeko(file, offset + 5, SEEK_SET); > > + if (fread(buffer, 1, 4, file) != 4) { > The fourth byte isn’t used is it? This alarmed me until I realized you > were still only decoding 3 bytes further down. Yes not that clear. New version reads more of the tag header. > > + break; > > + } > > + payloadSize = AMF_DecodeInt24(buffer); > > + lastGoodTag = offset; > What if the file is truncated in the middle of this “last good tag”, > and some of the payload is missing? Maybe you should also check that > the end offset (below) is within the total file size. (Unless you > meant this to point to the first bad tag.) Wouldn't the reading backward code look at the prevtag and seek backwards. It more of the last reachable prevtag pointer so that the original code can seek backwards rather than the last good tag. last discovered prevtag? I though something similar and tried to record the last but one tag header that was found. The read problem turned out to be that the last few tag headers have corrupted prevtag values! So I probably should strip that out out but then I have to test again working trumps elegant for me here as testing isn't that easy for me the downloads don't always fail. Its all a bit too much +4 +15 etc for my liking prevtag and payload not the same values, not the easiest to understand for me. > > + offset += payloadSize + 15; // the back pointer the tag header > > and the payload > > + } > > + tsize = size - lastGoodTag - 4; // set tsize to point to offset > > from end of file of the header of last good tag > Previously “tsize” was the offset of the _end_ of the last tag from > EOF (i.e. zero). Now I think you are making it the offset of the > _start_ of the “last good tag” (i.e. end of 2nd last good tag). > > do > > { > > int xread;

3 2

librtmp performance and high latency
by ctetrick＠satx.rr.com 26 Jan '15

26 Jan '15

Hi, I'm using a windows build of ffmpeg from zeranoe which uses librtmp. There is a long standing problem that I'm trying to fix. There is a case on the ffmpeg bug tracker: https://trac.ffmpeg.org/ticket/1604 And a related thread on the forum: http://ffmpeg.zeranoe.com/forum/viewtopic.php?f=7&t=657&sid=276f96e2c4ff9f6… I've posted on both. The basic problem is that RTMP_SendPacket() writes to the socket in 128 byte chunks, and when the destination server has a long RTT the stream stalls. I've tested with a simple fix - there is code there to buffer the chuncks for HTTP, but if the buffering is enabled for all sends, the whole packet is written to the socket, and the stream works much better. I'd really like to get this or something like it into a patch. I think it may fix this problem. Anybody have any feedback on this? Thanks, Cary

3 3

Re: [rtmpdump] Building rtmpsuck with MSVC toolchain
by Gunnar 23 Jan '15

23 Jan '15

no direct help, just indirect, the hope there are solutions for you "somewhere out there" - MSVC compile, wanted the same some years ago for easier debugging. Found what was claimed to be needed but I dont have links handy. Basically a script and header file for (re- or cross- or overlay)definition of integer,etc types. However, never got to test it (bc found the solution below) - secrete token for authorization, I also run into that for the famous (public) site. C-SPAN.org. couldnt get it right myself, luckily someone else had done it correctly. Involved a 3-4 step procedure of using a bunch of tools for searching for a keyword in the decoded swf flash file, recoding it and adding that to the arguments of the rtmpdump command found by rtmpsrv (which I modified to print the arguments to a batch file, etc) Might have been contribution by K-S-V?? The "security token" was something like plain ASCII "public secret" (hehe) and what to add to the rtmpdump args was (still is) "T4#up_Ko!", just to give examples on what they look like. I also struggled with rtmpsuck which somehow got the security token right but cldnt do the handshake (or any other thing) correctly. Seems C-SPAN server introduced new version AMF3 commands INFO stuff,etc which caused these problems. (all changed at the same time, server upgrade?, when security token was introduced and it actually might be just a default server system setting, left there by mistake?) Gunnar PS at that same time C-SPAN live changed from rtmp to AdobeHDS F4F protocol (K-S-V php script OK except for error handling). C-SPAN-live supposedly now demands "cable subscription login" but actually doesnt, just the right http links for getting the manifest file for the F4F packets. Seems this is where streaming is now going, leaving rtmp behind? on 1/21/2015 1:00 PM, rtmpdump-request(a)mplayerhq.hu wrote: > > Would I be correct in thinking that rtmpsuck only supports RTMP and RTMPE connections and not RTMPS ones? > > If so... What us a good way to discover the arguments being passed to the server? > > My work revolves around connecting to an RTMPS server which exposes two kinds of streams... Public ones that require no auth, and private one. This private one require a valid session token which used to be discoverable after logging in with a valid account, though something seems to have changed (something they acknowledge, just wont me too any specifics). > > Amy suggestions? > > Thanks, > Brendan

1 0

rtmpsuck and RTMPS
by Brendan Grant 17 Jan '15

17 Jan '15

Would I be correct in thinking that rtmpsuck only supports RTMP and RTMPE connections and not RTMPS ones? If so... What us a good way to discover the arguments being passed to the server? My work revolves around connecting to an RTMPS server which exposes two kinds of streams... Public ones that require no auth, and private one. This private one require a valid session token which used to be discoverable after logging in with a valid account, though something seems to have changed (something they acknowledge, just wont me too any specifics). Amy suggestions? Thanks, Brendan

1 0

fw 4gb file bugreport
by compn 14 Jan '15

14 Jan '15

mail was sent to projects(a)mplayerhq.hu , so forwarding it here. -- Hello, When I using youtube-dl to download some video from website,I found that When the video file is very large like 4.48GB. RtmpDump will report error "ERROR: Unexpected start of file, error in tag sizes, couldn't arrive at prevTagSize=0" So I download the source file,try and try again finally the project works, and then check and check again ... At last I figure out what's wrong with the program. In the last version seems trying to fix the large file problem using ftello instead of ftell .But the fseek function remains the same ,when execute fseek(file, 0, SEEK_END); size = ftello(file); Actually ftello(file) returns 0 to size.Finally I figure out the bug is that replace all fseek with fseeko to deal with large file problem. Thank you for your team proving this wonderful software stack. //sunsocool //fseek(file, 0, SEEK_END); fseeko(file, 0, SEEK_END); size = ftello(file); //fseek(file, 4, SEEK_SET); fseeko(file, 4, SEEK_SET); ... // go through the file and find the last video keyframe do { int xread; skipkeyframe: if (size - tsize < 13) { RTMP_Log(RTMP_LOGERROR, "Unexpected start of file, error in tag sizes, couldn't arrive at prevTagSize=0 size(%lld) - tsize(%lld)< 13 ", size, tsize); return RD_FAILED; } --

1 0

Problem accessing the DNS
by YIRAN LI 07 Jan '15

07 Jan '15

Hi guys, I'm a new comer and it will be my first question here. I'm totally new to rtmp and wanted to stream from my computer to Youtube live streaming. I can do it using ffmpeg command line tool, like: ffmpeg -i input file xx xx xx -f flv rtmp://xxxx Now I've started to write a sample c++ program using librtmp. (librtmp was compiled using "make SYS=mingw CRYPTO=" ) the URL used in the program is the URL to which ffmpeg could successfully streamed. So I think it's correct. As you can see, I had a "Problem accessing the DNS", the program was run on Windows and I've allowed it to bypass the firewall. Hope someone can help me with it. Great thanks

2 1

Building rtmpsuck with MSVC toolchain
by Brendan Grant 04 Jan '15

04 Jan '15

Any chance someone has an Makefile or two for building rtmpsuck under the MSVC toolchain? I'm trying to get to the bottom of why I cannot get an ffmpeg build to connect to a certain (what I believe to be an) RTMPS stream for interop reasons (bringing the stream to other mobile platforms), so I've recently been trying to get rtmpsuck to build under my preferred toolchain (MSVC) under MinGW (because doing so in VS 2013 seems a bridge too far) as the existing 2 year old binary from http://rtmpdump.mplayerhq.hu/ fails to handshake (the same issue I seem to be having with full ffmpeg in my own wrapping app)... and being able to directly debug against it in my preferred tools would be useful to try to get a few steps further. While I know there have been discussions in past over VC lacking C99 support, and so preventing building under the MSVC toolchain, that does seem to be something that's been supported for a while now (ffmpeg is buildable under MSVS 2013 for example)... so I must ask... does anyone have a functioning makefile to build the rtmpdump set of programs (including rtmpsuck) under MinGW & the MSVC toolchain? When last I dug into this issue a number of months back it was quite clear that the makefile was pretty hard coded for non MSVC systems, but that a few tweaks of CC and expected o/obj files could improve things... I seem to have not saved my attempts from then. Thanks, Brendan

3 2