[FFmpeg-devel] [PATCH 2/2] configure: instruct MSVC 2015 to properly process UTF-8 string literals

[Hendrik Leppkes]

On Sat, Feb 4, 2017 at 5:50 AM, Matt Oliver <protogonoi at gmail.com> wrote:
> On 4 February 2017 at 02:32, Hendrik Leppkes <h.leppkes at gmail.com> wrote:
>
>> On Fri, Feb 3, 2017 at 3:05 PM, James Almer <jamrial at gmail.com> wrote:
>> > On 2/3/2017 5:41 AM, Hendrik Leppkes wrote:
>> >> Without the /UTF-8 switch, the MSVC compiler treats all files as in the
>> >> system codepage, instead of in UTF-8, which causes UTF-8 string literals
>> >> to be interpreted wrong.
>> >>
>> >> This switch was only introduced in VS2015 Update 2, and any earlier
>> >> versions do not have an equivalent solution.
>> >>
>> >> Fixes fate-sub-scc on MSVC 2015+
>> >> ---
>> >>  configure | 3 +++
>> >>  1 file changed, 3 insertions(+)
>> >>
>> >> diff --git a/configure b/configure
>> >> index d3d652f0f4..231cc3eca7 100755
>> >> --- a/configure
>> >> +++ b/configure
>> >> @@ -6327,6 +6327,9 @@ EOF
>> >>      # Issue has been fixed in MSVC v19.00.24218.
>> >>      check_cpp_condition windows.h "_MSC_FULL_VER >= 190024218" ||
>> >>          check_cflags -d2SSAOptimizer-
>> >> +    # enable utf-8 source processing on VS2015 U2 and newer
>> >> +    check_cpp_condition windows.h "_MSC_FULL_VER >= 190023918" &&
>> >> +        add_cflags -utf-8
>> >
>> > Probably better use check_cflags, just in case.
>> >
>>
>> check_cflags doesn't work, since most wrong options just cause it to
>> emit a warning but not error out (although confusingly some do error
>> out, like the d2 option above, since the d2 prefix directly targets
>> the c2 compiler stage, and unknown options there error instead of
>> warn).
>> Thats the whole reason I added a version check in the first place
>> instead of solely using check_cflags with it.
>>
>> I mean, no real harm to use check_cflags together with the version
>> check, but since it doesn't do anything, I figured I would save the
>> extra check and a few forks.
>
>
> Is there any possibility to also find a fix for this on older msvc
> versions? If you direct me to the specific lines of code that are
> mis-compiling/executing ill have a look.
> In the mean time this patch looks good to me.

For this particular fate test, ccaption_dec.c line 73 onwards has
various unicode escape sequences, which don't get interpreted
properly.

The one generic workaround is encoding byte sequences directly, which
is really ugly and code-changes are usually hard to justify.
Another MSVC workaround is an undocumented compiler pragma, which
works in 2013, but not in 2012 (#pragma
execution_character_set("utf-8")), but adding pragmas to files is also
not something you can just easily hide away without it becoming ugly.

- Hendrik