[FFmpeg-devel] [PATCH] xtea: fix CBC mode when src=dst

[Giorgio Vazzana]

Hello,

the documentation for the function:

void av_xtea_crypt(struct AVXTEA *ctx, uint8_t *dst, const uint8_t *src,
                   int count, uint8_t *iv, int decrypt);

states that src can be equal to dst. In this case, when using CBC
mode, the decryption will fail, because for a block different from the
first one the wrong initialization vector is used. The correct
initialization vector should be the ciphertext of the previous block,
which we need to save before it's overwritten. Wikipedia has a nice
picture that should make this clear:
https://en.wikipedia.org/wiki/Block_cipher_modes_of_operation#Cipher-block_chaining_.28CBC.29

I've written a small program that demonstrates the issue. A patch like
the one attached should be enough to fix the problem. Please comment.

Regards,

Giorgio Vazzana
-------------- next part --------------
A non-text attachment was scrubbed...
Name: avutil_xtea_test.c
Type: text/x-csrc
Size: 1055 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20120930/021e6b70/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-xtea-fix-CBC-mode-when-src-dst.patch
Type: application/octet-stream
Size: 1345 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20120930/021e6b70/attachment.obj>