Fixed it, my bad, used the same packet for decoding & encoding. av_free_packet(&packet); av_init_packet(&packet); ret = avcodec_encode_video2(pCodecCtxOut, &packet, frame, &got_output); fixes it.