[Libav-user] MediaCodec H.264 Issue

[John Laxson]

Hello,

I am working to get the h.264 mediacodec interface working on Android, but the decoded frames are degraded visually coming out of the codec and looking for some assistance on tracking it down.

As a baseline, I can handle and decode this data [1] with the mp4 demuxer and software h.264 decoder just fine.  I have plumbed getting the android JNI environment to FFmpeg, and successfully initialized the h264_mediacodec codec, which decodes without returning errors.  The resulting frames [2] (vs reference [3] with software decoder) show smearing with motion, I’m not sure how one might characterize the issue.

To isolate the issue, I built a sample app that decodes the same file using only the Android API components and it decodes correctly.  Logging the API calls made to MediaCodec using Java vs FFmpeg, it seems that FFmpeg supplies packets that are shorter by usually 1-2 bytes.  I tracked this to the h264_mp4toannexb bitstream filter.  If I bypass that, the packets fed to MediaCodec are identical, though no frames are emitted by MediaCodec, I presume FFmpeg configures MediaCodec differently.  Will investigate this next.

Does anyone have MediaCodec working well in this scenario?  Is there any magic needed on the ffmpeg api side to configure things correctly?  This is on Android 12 with FFmpeg 5.0.1 on a Samsung S20.

Thanks,
John

[1] (365MB!) https://storage.googleapis.com/ffmpeg-mediacodec-issue/video.mp4
[2] https://storage.googleapis.com/ffmpeg-mediacodec-issue/damaged_frame.jpg
[2] https://storage.googleapis.com/ffmpeg-mediacodec-issue/software_frame.jpg