[Libav-user] avcodec_encode_video2() crashes after encoding a few pictures

[Arvind Raman]

I am developing an application using the FFmpeg libraries and am facing a crash issue that I am finding extremely hard to debug. My printf logs suggest that crash happens within the avcodec_encode_video2() function but I am not sure what is causing the crash. 

Just for debugging purpose, I created the following infinite loop on the "encode" thread. The loop iterates several times and crashes somewhere in between the 350th and the 400th iteration. The crash is not deterministic. Please note there are video decode (libavcodec) and demux (libavformat) threads running in parallel that are interrupting the avcodec_encode_video2() function. Could this be a problem.

Do you spot anything that might be causing the crash? I am using the x.264 encoder. The only error log that I get on the terminal is "Segmentation fault (core dumped)" so I am finding it hard to proceed. Is there anyway to get x.264 to print where the crash might be happening. 

Its quite likely that this might be a silly coding mistake from my end, but just wanted to get a quick check done from you all and receive your suggestions.

while(1)
{
    av_init_packet(&encode_packet);
    frame->pts += 3750;

    frame->data[0] = frame_buf;
    frame->data[1] = frame->data[0] + (frame->linesize[0] * encoder_ctx->height);
    frame->data[2] = frame->data[1] + (frame->linesize[1] * (encoder_ctx->height / 2));
    encode_status = avcodec_encode_video2(encoder_ctx, &encode_packet, frame, &got_packet);

    av_free_packet(&encode_packet);
}

I tried running valgrind as well but that didn't throw any illegal memory access and it throws the following error. Could this be causing the crash? What is this error about? I saw similar error messages elsewhere as well, but don't think I found any solution there.


vex amd64->IR: unhandled instruction bytes: 0x66 0xF 0x38 0x41 0xC0 0x66 0xF 0x7E
==10933== valgrind: Unrecognised instruction at address 0x51393bc.
==10933==    at 0x51393BC: ??? (in /usr/local/lib/libx264.so.133)
==10933==    by 0x2000200020001F: ???
==10933== Your program just tried to execute an instruction that Valgrind
==10933== did not recognise.  There are two possible reasons for this.
==10933== 1. Your program has a bug and erroneously jumped to a non-code
==10933==    location.  If you are running Memcheck and you just saw a
==10933==    warning about a bad jump, it's probably your program's fault.
==10933== 2. The instruction is legitimate but Valgrind doesn't handle it,
==10933==    i.e. it's Valgrind's fault.  If you think this is the case or
==10933==    you are not sure, please let us know and we'll try to fix it.
==10933== Either way, Valgrind will now raise a SIGILL signal which will
==10933== probably kill your program.
==10933== 
==10933== Process terminating with default action of signal 4 (SIGILL)
==10933==  Illegal opcode at address 0x51393BC
==10933==    at 0x51393BC: ??? (in /usr/local/lib/libx264.so.133)
==10933==    by 0x2000200020001F: ???

Any help would be greatly appreciated!!!

Best regards
Arvind