[MPlayer-users] bad code (UMR) in mp3lib/layer2.c causing signal 11 in mplayer

[Nilmoni Deb]

Hi,
	I have discovered some UMR (uninitialized memory read) thats causing 
mplayer/ffmpeg current cvs to give sig11 just before starting to play an
avi movie. Last part of the output is:

==========================================================================
Opening audio decoder: [mp3lib] MPEG layer-2, layer-3
dec_audio: Allocating 4608 + 65536 = 70144 bytes for output buffer
mp3lib: made decode tables with MMX optimization
mp3lib: using 3DNow!Ex optimized decore!
MP3lib: init layer2&3 finished, tables done


MPlayer interrupted by signal 11 in module: init_audio_codec
- MPlayer crashed by bad usage of CPU/FPU/RAM. Recompile MPlayer with
--enable-debug and make a 'gdb' backtrace and disassembly. For details,
see DOCS/bugreports.html section 5.b.
- MPlayer crashed. This shouldn't happen. It can be a bug in the MPlayer
code _or_ in your drivers _or_ in your gcc version. If you think it's
MPlayer's fault, please read DOCS/bugreports.html and follow instructions
there. We can't and won't help unless you provide these informations when
reporting a possible bug.
DEMUXER: freeing demuxer at 0x842de20
vo: x11 uninit called but X11 not inited..



Further investigation revealed the problem to be in the file 
mp3lib/layer2.c, in the function II_select_table as shown below:

    242 static void II_select_table(struct frame *fr)
    243 {
    244   static int translate[3][2][16] =
    245    { { { 0,2,2,2,2,2,2,0,0,0,1,1,1,1,1,0 } ,
    246        { 0,2,2,0,0,0,1,1,1,1,1,1,1,1,1,0 } } ,
    247      { { 0,2,2,2,2,2,2,0,0,0,0,0,0,0,0,0 } ,
    248        { 0,2,2,0,0,0,0,0,0,0,0,0,0,0,0,0 } } ,
    249      { { 0,3,3,3,3,3,3,0,0,0,1,1,1,1,1,0 } ,
    250        { 0,3,3,0,0,0,1,1,1,1,1,1,1,1,1,0 } } };
    251
    252   int table,sblim;
    253   static struct al_table *tables[5] =
    254        { alloc_0, alloc_1, alloc_2, alloc_3 , alloc_4 };
    255   static int sblims[5] = { 27 , 30 , 8, 12 , 30 };
    256
    257   if(fr->lsf)
    258     table = 4;
    259   else
    260     table =
translate[fr->sampling_frequency][2-fr->stereo][fr->bitrate_
index];
    261   sblim = sblims[table];
    262
    263   printf("\t::::: mp3lib/layer2.c II_select_table table=%d\n",
table);
    264   fr->alloc      = tables[table];
    265   fr->II_sblimit = sblim;
    266 }


The printf statement (added by me) at line 263 shows table = 30 which is
bad since table should have values in range [0,4].

Further debugging showed that fr->lsf=0 which made table get its value
from translate. But fr->sampling_frequency = 3 when its correct range is
[0,2] !! Obviously, a series of UMR ultimately causes a sig 11.

Now, this ugly hack made the sig 11 go away: replace line 260 with ->

  {
  if (fr->sampling_frequency>2) table=4; else
    table =
translate[fr->sampling_frequency][2-fr->stereo][fr->bitrate_index];
  }

Obviously, this is not a solution. What do the developers suggest ?

thanks
- Nil