[rtmpdump] 64 bit crash fix (undefined union assignment)

[Kirill Zorin]

Good news, everyone!

According to ISO C, assigning (i.e. "a = b;") one union to another only has to copy the "active" member of the union, that is, the last-accessed one. This means that if we assign unions with different active members, the behaviour is undefined because the wrong bytes will be copied; indeed, it crashes on (some!) 64-bit builds for this reason. The attached patch fixes the problem, and as far as I can tell I've fixed it in all relevant locations in librtmp.

It should be noted that it would be nice if we had some kind of discipline with respect to this in the future, considering that the addition of a union to any existing struct we assign will generate the same problem. This is made worse by the fact that this is the kind of stuff that you can't grep for and have to inspect manually, and that the side effects never manifest at the exact "wrong" location (i.e. the assignment itself), but instead further down the line as the program hopelessly tries to access pointers that were "half-copied", etc. To illustrate, this particular problem took about an hour to find; an hour I could have been wasting instead.

K

-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-fixed-undefined-behaviour-due-to-union-assignment.patch
Type: application/octet-stream
Size: 1488 bytes
Desc: not available
URL: <http://lists.mplayerhq.hu/pipermail/rtmpdump/attachments/20110930/6f8c5590/attachment.obj>