[Libav-user] Building FFmpeg libraries with AddressSanitizer

[Patrick Mihelich]

Hi all,

I tried to build from 4.4.1 using --toolchain=gcc-asan, but I hit a problem
linking libswscale. I wonder if someone here understands the problem, or
has otherwise been able to build recent versions of FFmpeg with ASan
enabled?

My system:
$ uname -mov
#1 SMP PREEMPT Debian 5.16.12-1~bpo11+1 (2022-03-08) x86_64 GNU/Linux
$ gcc --version
gcc (Debian 10.2.1-6) 10.2.1 20210110

Configured with:
$ ./configure --prefix=/some/local/path --disable-static --enable-shared
--enable-rpath --disable-doc --disable-programs --toolchain=gcc-asan

libavutil built successfully, but libswscale failed to link:

$ make V=1
...
gcc -shared -Wl,-soname,libswscale.so.5 -Wl,-Bsymbolic
-Wl,--version-script,libswscale/libswscale.ver -Llibavcodec -Llibavdevice
-Llibavfilter -Llibavformat -Llibavresample -Llibavutil -Llibpostproc
-Llibswscale -Llibswresample -fsanitize=address  -Wl,--as-needed
-Wl,-z,noexecstack -Wl,--warn-common
-Wl,-rpath-link=:libpostproc:libswresample:libswscale:libavfilter:libavdevice:libavformat:libavcodec:libavutil:libavresample
-Wl,-rpath,/usr/local/companion-libs/ffmpeg/v1002/lib -o
libswscale/libswscale.so.5 libswscale/alphablend.o libswscale/gamma.o
libswscale/hscale.o libswscale/hscale_fast_bilinear.o libswscale/input.o
libswscale/log2_tab.o libswscale/options.o libswscale/output.o
libswscale/rgb2rgb.o libswscale/slice.o libswscale/swscale.o
libswscale/swscale_unscaled.o libswscale/utils.o libswscale/vscale.o
libswscale/x86/hscale_fast_bilinear_simd.o libswscale/x86/input.o
libswscale/x86/output.o libswscale/x86/rgb2rgb.o libswscale/x86/rgb_2_rgb.o
libswscale/x86/scale.o libswscale/x86/swscale.o libswscale/x86/yuv2rgb.o
libswscale/x86/yuv2yuvX.o libswscale/x86/yuv_2_rgb.o libswscale/yuv2rgb.o
-lavutil -lm -pthread -lvdpau -lX11 -lm
/usr/bin/ld: libswscale/x86/rgb2rgb.o: warning: relocation against
`__odr_asan.mask24r' in read-only section `.text'
/usr/bin/ld: libswscale/x86/rgb2rgb.o: relocation R_X86_64_PC32 against
undefined symbol `__odr_asan.mask24l' can not be used when making a shared
object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
make: *** [ffbuild/library.mak:103: libswscale/libswscale.so.5] Error 1

However, libswscale/x86/rgb2rgb.o was already compiled with -fPIC:

gcc -I. -I./ -D_ISOC99_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
-D_POSIX_C_SOURCE=200112 -D_XOPEN_SOURCE=600 -DPIC -DZLIB_CONST
-DHAVE_AV_CONFIG_H -DBUILDING_swscale -fsanitize=address   -std=c11
-fomit-frame-pointer -fPIC -pthread   -g -Wdeclaration-after-statement
-Wall -Wdisabled-optimization -Wpointer-arith -Wredundant-decls
-Wwrite-strings -Wtype-limits -Wundef -Wmissing-prototypes
-Wno-pointer-to-int-cast -Wstrict-prototypes -Wempty-body -Wno-parentheses
-Wno-switch -Wno-format-zero-length -Wno-pointer-sign
-Wno-unused-const-variable -Wno-bool-operation -Wno-char-subscripts -O3
-fno-math-errno -fno-signed-zeros -fno-tree-vectorize
-Werror=format-security -Werror=implicit-function-declaration
-Werror=missing-prototypes -Werror=return-type -Werror=vla -Wformat
-fdiagnostics-color=auto -Wno-maybe-uninitialized   -MMD -MF
libswscale/x86/rgb2rgb.d -MT libswscale/x86/rgb2rgb.o -c -o
libswscale/x86/rgb2rgb.o libswscale/x86/rgb2rgb.c

The object file has relocations for various __asan_* symbols, but not for
any __odr_asan:

$ readelf --relocs libswscale/x86/rgb2rgb.o | egrep '(GOT|PLT|JU?MP_SLOT)'
| grep asan | cut -d " " -f9 | sort | uniq -c
      1 __asan_init
      1 __asan_register_g[...]
    132 __asan_report_load1
     12 __asan_report_load2
     12 __asan_report_load4
    134 __asan_report_store1
     20 __asan_report_store2
      4 __asan_report_store4
     76 __asan_report_store8
      1 __asan_unregister[...]
      1 __asan_version_mi[...]

Thanks,
Patrick
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://ffmpeg.org/pipermail/libav-user/attachments/20220419/a483c83c/attachment.htm>