[FFmpeg-devel] flashsv error due to negative linesize value - not an intentional cross post!

[Jason Askew]

I've definately isolated the issue to copy_region_enc.

static int copy_region_enc(uint8_t *sptr, uint8_t *dptr,
        int dx, int dy, int h, int w, int stride, uint8_t *pfptr) {

    int i,j;
    uint8_t *nsptr;
    uint8_t *npfptr;
    int diff = 0;

    for (i = dx+h; i > dx; i--) {
        nsptr = sptr+(i*stride)+dy*3;
        npfptr = pfptr+(i*stride)+dy*3;
        for (j=0 ; j<w*3 ; j++) {
            diff |=npfptr[j]^nsptr[j];
            dptr[j] = nsptr[j];
        }
        dptr += w*3;
    }
    if (diff)
        return 1;
    return 0;
}


int stride is set to linesize[0].  So, when linesize is negative, it
throws this function off, I believe due to:

        nsptr = sptr+(i*stride)+dy*3;
        npfptr = pfptr+(i*stride)+dy*3;

since stride is negative, these pointers end up being pointed to the
wrong memory.

Is there a good code to look at to see an example of negative linesize
and how to move thru frames with a negative linesize value?  Would
that put me on the right track?

Thanks.