[FFmpeg-devel] [RFC] Negative Bitrate Assertion Error

[David Czech]

Hello, I am trying to fix a crash (for Google
Code-In<http://www.google-melange.com/gci/task/show/google/gci2010/videolan/t129169175566>)
I found while fuzz testing ffmpeg: found
here<https://roundup.ffmpeg.org/issue2426>.
I have tested this crash on ubuntu's packaged ffmpeg and the latest version
of ffmpeg from svn trunk.

 I am new to ffmpeg's source code, but I do have some theories and
information about the bug. As seen in the backtrace, we have a call to
av_rescale() from av_estimate_timings_from_bit_rate. The ic->bit_rate passed
to av_rescale is negative which causes the assertion error later on.
The question is, should rescale handle negative values or if bitrate isn't
supposed to be negative, where should the bitrate value be validated?

By the way, I've noticed that av_rescale should call av_rescale_rnd, with
the exact same arguments, but it doesn't seem like that is happening (?).

#4  0x08446e87 in av_rescale_rnd (a=549859968, b=1200000,
c=-1875801792, rnd=AV_ROUND_ZERO) at libavutil/mathematics.c:79
#5  0x084471ef in av_rescale (a=0, b=0, c=-4294967296) at
libavutil/mathematics.c:130


OK, Thank-you for any replies, insight, comments.