[rtmpdump] Handshake fails when "_DEBUG" defined

[Skaarj NaPali]

Hi,

There is a bug in "static bool HandShake(RTMP * r, bool FP9HandShake)"
which triggers only when "_DEBUG" is defined. Certain RTMP servers are
rejecting the handshake when the client signature data is all zero.
So, when connecting to such a server without using an encrypted
connection and without using the "FP9" handshake and when "_DEBUG" is
defined, the server will disconnect. The server simply closes the
TCP/IP connection after the handshake.

The problem can get reproduced with this short clip:
rtmpdump --rtmp
rtmpt://pssimp7lsfs.fplive.net:80/pssimp7ls/geo_d_at_ch/Clip_01_Stiefvater_und_Sohn_SDfull25
--flv out.flv

RTMPDump v2.1
(c) 2009 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
DEBUG: Protocol : RTMPT
DEBUG: Hostname : pssimp7lsfs.fplive.net
DEBUG: Port     : 80
DEBUG: Playpath : Clip_01_Stiefvater_und_Sohn_SDfull25
DEBUG: tcUrl    : rtmpt://pssimp7lsfs.fplive.net:80/pssimp7ls/geo_d_at_ch
DEBUG: swfUrl   : (null)
DEBUG: pageUrl  : (null)
DEBUG: app      : pssimp7ls/geo_d_at_ch
DEBUG: auth     : (null)
DEBUG: subscribepath : (null)
DEBUG: flashVer : LNX 10,0,22,87
DEBUG: live     : no
DEBUG: timeout  : 120 sec
DEBUG: Setting buffer time to: 36000000ms
Connecting ...
DEBUG: RTMP_Connect, ... connected, handshaking
DEBUG: Clientsig:
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00...
...
DEBUG: HandShake: Type Answer   : 03
DEBUG: HandShake: Server Uptime : 1855384364
DEBUG: HandShake: FMS Version   : 3.0.4.1
DEBUG: Server signature:
6E 96 EB 2C 03 00 04 01 92 D9 53 A7 A3 4D D1...
...
DEBUG: HandShake: Sending handshake response:
6E 96 EB 2C 00 00 00 00 92 D9 53 A7 A3 4D D1...
...
DEBUG: HandShake: 2nd handshake:
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00...
...
DEBUG: HandShake: Handshaking finished....
DEBUG: RTMP_Connect, handshaked
DEBUG: Invoking connect
INFO: Connected...
ERROR: RTMP_ReadPacket, failed to read RTMP packet header
DEBUG: Closing connection.
---------------------------------------

The problem could get fixed by removing the "#ifdef _DEBUG" from lines
304-310 from "handshake.h"

Current:

  /* generate random data */
#ifdef _DEBUG
  memset(clientsig+8, 0, RTMP_SIG_SIZE-8);
#else
  ip = (int32_t *)(clientsig+8);
  for (i = 2; i < RTMP_SIG_SIZE/4; i++)
    *ip++ = rand();
#endif

New:

  /* generate random data */
  ip = (int32_t *)(clientsig+8);
  for (i = 2; i < RTMP_SIG_SIZE/4; i++)
    *ip++ = rand();

The same problem applies most likely also for "static bool
SHandShake(RTMP * r)" as the code there is very similar.

KR