[FFmpeg-devel] [PATCH 20/39] avcodec/h261dec: Don't initialize unused part of RLTable

[Andreas Rheinhardt]

Anton Khirnov:
> Quoting Andreas Rheinhardt (2020-12-10 12:16:38)
>> The H.261 decoder only uses an RLTable's VLC table, yet it also
>> initializes its index_run, max_level and max_run. This commit stops
>> doing so; it will also simplify making this decoder init-threadsafe,
>> as the H.261 decoder and encoder now initialize disjoint parts of their
>> common RLTable.
> 
> Does it then make sense to keep this RLTable common?
> 
I presume you want to know whether the RLTable structure should be split
into smaller structures?

To answer this question, let's examine what parts are used for each RLTable.

Initializing an RLTable's VLCs uses all three static tables (table_vlc,
table_run and table_level); initializing the rest only uses table_run
and table_level. get_rl_index (only used in encoders) uses index_run and
max_level, which is initialized by ff_rl_init).

ff_h261_rl_tcoeff:
The H.261 encoder uses table_vlc and get_rl_index (thereby also using
index_run and max_level); the H.261 decoder uses only the first VLC.
max_run is completely unused.

ff_rl_mpeg1/ff_rl_mpeg2:
Only the first VLC table is used by the various decoders using them; the
encoder uses max_level, index_run and table_vlc. max_run is unused.

ff_rl_speedhq:
The decoder uses the first VLC table; the encoder uses table_vlc as well
as max_level and index_run. max_run is unused.

ff_h263_rl_inter:
ituh263dec.c only uses the first VLC table; mpeg4videodec.c uses all the
VLC tables as well as max_run and max_level. ituh263enc.c uses
index_run, max_level as well as table_vlc. mpeg4videoenc.c uses
table_vlc, max_level, max_run as well as index_run.

ff_rl_intra_aic:
ituh263dec.c only uses the first VLC; ituh263enc uses table_vlc,
index_run, max_level. max_run is unused.

ff_rvlc_rl_intra/inter:
Only used in mpeg4videodec.c. For ff_rvlc_rl_intra, only the first VLC
is used; for ff_rvlc_rl_inter, all VLCs are potentially used. max_level,
max_run and index_run are completely unused (notice that I did not
remove the calls to ff_rl_init for these RLTables because I actually do
not have a sample to confirm that these are unused).

ff_rl_table
msmpeg4dec.c uses the VLCs (only the first VLC for 0..2) as well as
max_run and max_level; msmpeg4enc.c uses table_vlc, index_run, max_level
and max_run.

ff_mpeg4_rl_intra
mpeg4videodec.c uses the first VLC as well as max_run and max_level.
mpeg4videoenc.c uses table_vlc as well as index_run, max_level and max_run.

So seven of the ten RLTables (I am counting the ff_rl_tables as one)
don't use max_run at all; and seven of the ten RLTables (but not the
same seven) only use the first VLC. It is possible to remove rl_vlc from
RLTable and make it separate; the code in mpeg4videodec.c and
msmpeg4dec.c (the only places where VLCs other than the first are used)
are already compatible with this. Yet the other things belong together
in the same struct.

- Andreas

PS: It seems you overlooked patches 7-13.