[Libav-user] sws_scale has weird behavior when not resizing

[Alexandre Millette]

Hi again.

TL;DR: use the SWS_ACCURATE_RND flag.

I have solve my issue (or rather found a workaround) and thought it 
might help some people:

 > Also I am not sure what would be the proper way to handle this 
situation.
 > Sure I can round the width of the frame to the nearest greater 
multiple of
 > 8 and add one to it, but I'm afraid that in some situation it may 
bust the
 > line size and cause an exception. Plus it doesn't seem right to use such
 > magic numbers..

The problem only occurs when sws_scale does no resizing, that is when 
the source and destination sizes are the same. By looking at the 
sws_getContext source code, I have found that one flag, 
SWS_ACCURATE_RND, among other things, force the function to not use it's 
no-resizing algorithm. So we can prevent the issue by setting this flag.
I am not quite sure of the exact usage of this flag, but I know VLC used 
to have it set for quality issues (that could be related to my own 
issue) but have since removed it from some builds (like their android's) 
because of performance problems. I have personally seen no difference in 
performance with the flag, but I have still decided to use it only when 
need be, that is when the width and height are not multiples of 8.

Here's a glimpse of my code now:

AVCodecContext* mpCodecCtx;
AVFrame* lpFrame;
AVFrame* lpFrameRGB;
static struct SwsContext* img_convert_ctx = 0;
// ...
// mpCodecCtx is opened normally
// lpFrame is allocated and returned by avcodec_decode_video2
// lpFrameRGB is allocated and filled by avpicture_fill
int w = mpCodecCtx->width;
int h = mpCodecCtx->height;
int SwsFlags = SWS_FAST_BILINEAR;
if ((w & 0x7) || (h & 0x7))
   SwsFlags |= SWS_ACCURATE_RND;
img_convert_ctx = sws_getContext(w, h, PIX_FMT_YUV420P,
  w, h, PIX_FMT_RGB32, SwsFlags, NULL, NULL, NULL);
int ret = sws_scale(img_convert_ctx, lpFrame->data,
  lpFrame->linesize, 0, h,
  lpFrameRGB->data, lpFrameRGB->linesize);

On a somewhat related note, can the sws_scale behavior that I was seeing 
be considered a bug?
If so, how should I open a ticket for it to be fix?

Alex M