develooper Front page | perl.perl5.porters | Postings from July 2015

RE: [perl #125714] Adding support for VS2015 (VC14)

Thread Previous | Thread Next
From:
Steve Hay
Date:
July 30, 2015 18:10
Subject:
RE: [perl #125714] Adding support for VS2015 (VC14)
Message ID:
C69C5F337BA04D49984AA7C86FFB6E773A0AA4DE@UKMAIL03.planit.group
On 29 July 2015 at 13:39, bulk88 via RT <perlbug-followup@perl.org> wrote:
> On Wed Jul 29 00:51:26 2015, shay wrote:
>  ----------------------------
> @@ -1074,6 +1087,17 @@ config.w32 : $(CFGSH_TMPL)
>         @echo.>>$@
>         @echo #ifndef _config_h_footer_>>$@
>         @echo #define _config_h_footer_>>$@
> +.IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE"
> +       @echo #undef FILE_ptr>>$@
> +       @echo #define FILE_ptr(fp) PERLIO_FILE_ptr(fp)>>$@
> +       @echo #undef FILE_cnt>>$@
> +       @echo #define FILE_cnt(fp) PERLIO_FILE_cnt(fp)>>$@
> +       @echo #undef FILE_base>>$@
> +       @echo #define FILE_base(fp) PERLIO_FILE_base(fp)>>$@
> +       @echo #undef FILE_bufsiz>>$@
> +       @echo #define FILE_bufsiz(fp) (PERLIO_FILE_cnt(fp) + PERLIO_FILE_ptr(fp) - PERLIO_FILE_base(fp))>>$@
> +       @echo #define I_STDBOOL>>$@
> +.ENDIF
>         @echo #undef Off_t>>$@
>         @echo #undef LSEEKSIZE>>$@
>         @echo #undef Off_t_size>>$@
> ----------------------------------------------
> This is overcomplicated. This should be done in win32.h #if _MSC_VER >= 1400. Why isn't PERLIO_FILE_base/etc used on every Win32 CC config including GCC and then PERLIO_FILE_base is implement one way or the other in win32.h? Those "+       @echo #undef FILE_ptr>>$@" lines are slow, each one is its own cmd.exe process launched by the make tool during the build process. It is best if there are less of them. This slowness note is from the Win32 parallel building project (see its ticket or prior ML threads).

I will come back to this later since it's a separate issue from getting things building with VS2015. The whole business of all these echo lines needs sorting out better somehow, not just these new lines.


>
> ----------------------------------------------
> @@ -563,6 +567,10 @@ CXX_FLAG   = -TP -EHsc
>
>  LIBC           = msvcrt.lib
>
> +!IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE"
> +LIBC           = $(LIBC) vcruntime.lib ucrt.lib
> +!ENDIF
> +
> ----------------------------------------------
>
> Spiritually, ucrt.lib/ucrtbase.dll is the MS libc now. VC 2015 msvcrt.lib only contains static link code for things like CPU probing and math ops. I am slightly slightly slightly afraid that lib won't be -e/-f-able anymore. This can break toolchain stuff, like when you stuff double quotes to fix a spaces in path on cmd line problem in a file path to fix something EUMM/MB/CB-ish, and now every breaks since it isn't a valid file path anymore. Maybe 2 of the 3 libs need to be put in $Config{libs} where oldnames.lib, kernel32.lib, user32.lib, gdi32.lib, etc are. I'd say ucrt.lib is the file to do probe for symbol names if you have some code that scans (not test compiles) "the libc" for what posix/c std lib symbols/functions it implements.

Done in the v3 patch attached.


>
> ------------------------------------------------
> @@ -603,6 +611,11 @@ OPTIMIZE   += -fp:precise
>  DEFINES                += -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE
>  .ENDIF
>
> +# Likewise for deprecated Winsock APIs in VC++ 14.0 for now.
> +.IF "$(CCTYPE)" == "MSVC140" || "$(CCTYPE)" == "MSVC140FREE"
> +DEFINES                += -D_WINSOCK_DEPRECATED_NO_WARNINGS
> +.ENDIF
> +
>  # In VS 2005 (VC++ 8.0) Microsoft changes time_t from 32-bit to
>  # 64-bit, even in 32-bit mode.  It also provides the _USE_32BIT_TIME_T
>  # preprocessor option to revert back to the old functionality for
> ------------------------------------------------
>
> There are now 3 huge "no deprecate" -Ds on the command line. There will probably be a new no-deprecate for every version of Visual C. These should really go into win32.h. Unix perl builds have atmost 2 or 3 -Ds, not 10 or 15 -Ds. This might require reorganization of perl.h since the first CC .h is sys/types.h around line 670 in perl.h. config.h looks like the only place to the defines before sys/types.h. Some experiments need to be done if those no-deprecate defines can be moved to win32.h.

I will also return to this since it is again separate from VS2015 support.


>
>> Also attached is my second attempt: This does surprisingly well with
>> the quick-and-dirty approach of simply casting a FILE* to the
>> appropriate new CRT struct (__crt_stdio_stream_data) and accessing the
>> members of that. It might even suffice, rather than going to the
>> hassle of writing accessors and probably needing to separate getters
>> from setters?
>
> It looks like __crt_stdio_stream_data is allocated by the plain c calloc_base() function. That means msize can be used on it. msize is used for the pioinfo/osfhnd code for VC 2005 already since the struct repeatdly changed sizes during VC 2005's lifetime. On 32 bits, __crt_stdio_stream_data is supposedly 0x38 bytes long, IDK what it is on 64. You can write an assert/DEBUGGING comparing the msize() to sizeof() as a sanity check.
>
> I support the casting+macro solution. It makes the best machine code.
>
> It can also be argued that since VC 2015 is so "new", msize should always be used because an automatic OS update can change ucrtbase.dll at any time. IDK if ucrtbase.dll is VC redist package maintained (very rare to change), Service Pack/OS upgrade only changed (medium risk) or windows updates maintained (high risk of change). Unlike the osfhnd code, where perl is only interested in the first member of the struct, with __crt_stdio_stream_data it is interested in the middle members. If MS changes the order of the members under us, we are screwed.
>
> MS did change the order to move int _cnt between pointer "_ptr" and pointer "_base" to now sitting after _base http://www.nntp.perl.org/group/perl.perl5.porters/2015/05/msg227727.html when they created the UCRT. If there is a non-DEBUGGING/always msize done on a __crt_stdio_stream_data * at start up, the only thing msize check will do is abort the perl process on startup, since we can't know what the struct will be in the future. You will only get back a working perl by downgrading the OS/uninstall a hotfix, or upgrade to the next perl maint release. At this point it is probably unlikely the struct members will be rearranged, only new ones added to the end. That doesn't affect us in reaching the old members. My final opinion is the msize should be DEBUGGING only, plus Ruby isn't doing it https://github.com/ruby/ruby/commit/3446537f1a8f6a0dbc2b27a0f25af13c7f61abf8  (IDK if Ruby has a ticket/discussion for that commit or not).

Done in the v3 patch attached. 


>
>> However, the build currently fails to link due to __pioinfo apparently
>> no longer being present in the new CRT (or else I'm linking the wrong
>> libs?):
>>
>> win32.obj : error LNK2001: unresolved external symbol __imp____pioinfo
>>
>> This was added (actually restored and modified from earlier ripped out
>> code) by:
>>
>> http://perl5.git.perl.org/perl.git/commit/b47a847f6284f6f98ad7509cf77a4aeb802d8fce
>>
>> As that commit notes, "If __pioinfo isn't exported anymore, the Perl
>> build will break." So this was always known to be an area of danger
>> and impending breakage; the question is: What do we do about it now
>> that it's happened?
>>
>> bulk88: Do you have any ideas on the latter point?
>
> File a bug ticket with MS at https://connect.microsoft.com/VisualStudio/ to get __pioinfo from ucrtbase.dll.
>
> In the mean time the easy but not best performance solution for is for Perl on VC 2015 to only support static UCRT from libucrt.lib since the libc/mscrt/ucrt symbols like _pioinfo can't hide from the linker when static linking.

Done in the v3 patch attached, but I'm not very keen on this. Perl has always used the CRT DLL, which makes me very uneasy about switching to LIBC. It does get it building, though, but now crashes on startup trying to set environ[0] to NULL in S_init_postdump_symbols().

I briefly tried #defining NO_ENVIRON_ARRAY to temporarily duck that issue but the build failed because Perl_my_setenv() is then not provided by util.c, but is apparently still required by mg.c (at least)... :-/

Is this problem due to the switch from dynamic to static CRT? Or just something new in VC14 anyway?


>
> Ruby has the same exact problem as we do https://github.com/ruby/ruby/blob/trunk/win32/win32.c#L2241 IDK how they are dealing with it or they are currently broken on VC 2015.
>
> Old python used pioinfo https://github.com/python/cpython/blob/2.7/Modules/posixmodule.c#L532 they stopped using for VC 2015 https://github.com/python/cpython/blob/master/Modules/posixmodule.c#L1068  https://bugs.python.org/issue23524 , I dont like their solution of swapping the callback everywhere. They were reaching inside the ioinfo struct to check if its allocated/open FD or not, without triggering MS CRT's invalid param handler/exception/whatever. Python doesn't have the "sockets in libc FDs on win32" problem Perl and Ruby has, python sockets live in a class by themselves so they aren't FDs https://rt.perl.org/Ticket/Display.html?id=118127#txn-1357875 .
>
> The other 2 solutions I have in my head on how to get ucrtbase.dll's _pioinfo are, ummm, crazy, but would work. So any point in mentioning them or will I scare the children?
>

Since I'm not keen on the static CRT change, what are the other options that you see here?

Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About