[Libav-user] Frame decoding and sw_scale timings for HD video

[Don Moir]

> Hi,
>
> I'm working on application which uses both ffmpeg and ffms (wrapper on
> ffmpeg) for decoding video file.
> The decoding audio and video is then played on user interface.
> For low resolution video till 640x360p both audio and video works fine and
> frame rate is maintained as per source file.
> However when video frame size increases 1280x720p, I'm not able to achieve
> desire frame rate and since audio is synchronized to video, audio quality is
> decreasing which lots of breaks.
> After analyzing the timings of critical function in my filter chain I found
> that, though av_read_frame and avcodec_decode_video2 takes almost same time
> for both 640x360 and 1280x720, there is considerable difference for
> sws_scale() function.
> Please someone suggest is there a way to reduce computation time of
> sws_scale.
> Audio decoding is taking constant time for both 640x360p and 1280x720p
> video. Though I'm working to further reduce its timing.
>
> Also if someone has some other ideas on how to reduce overall time, to
> achieve desire frame rate without compromising on audio quality please let
> me know.

What image system are you using for the final display ? What is the PixFormat of the video ?

You should decode and output audio in separate thread. In case video decoding is held up, this will allow audio to sound fine.

Beyond this you may need to do frame dropping. Most will implement this by not displaying a frame. This generally does not work 
since the bottle neck may be in decoding, etc. Other forms of frame dropping get a bit tricky and you need to be careful.