[Libav-user] VDPAU wierd error

[Carl Eugen Hoyos]

2016-09-07 17:58 GMT+02:00 Mihai Chindea <mihai.chindea at uti.eu.com>:
> i keep getting this error "The linesize 1280 cannot be represented as uint32" when I call
> av_hwframe_transfer_data(), apparently it's source is in libavuil\hwcontext_vdpau.c:307.
>
> is this condition "if (dst->linesize[i] < 0 || (uint64_t)dst->linesize > UINT32_MAX) " desired ?
> shouldn't it be "if(dst->linesize[i] < 0 || (uint64_t)dst->linesize[ 0 ] > UINT32_MAX)" ??

Feel free to send a patch made with git format-patch to the development
mailing list, I don't think this code was ever tested.

Carl Eugen