develooper Front page | perl.perl5.porters | Postings from March 2019

[perl #133959] 70bd6bc82b breaks multiple tests on threadedFreeBSD-13

Thread Previous
From:
James E Keenan
Date:
March 26, 2019 04:05
Subject:
[perl #133959] 70bd6bc82b breaks multiple tests on threadedFreeBSD-13
Message ID:
rt-4.0.24-14638-1553573134-493.133959-75-0@perl.org
# New Ticket Created by  James E Keenan 
# Please include the string:  [perl #133959]
# in the subject line of all future correspondence about this issue. 
# <URL: https://rt.perl.org/Ticket/Display.html?id=133959 >


This is the second of two tickets reporting significant test failures in 
Perl 5 blead when built with threads on FreeBSD-13-CURRENT.  (The first 
was https://rt.perl.org/Ticket/Display.html?id=133958.)

Commit 70bd6bc82b breaks multiple tests on FreeBSD-13-CURRENT when perl 
is built with threads.

Bisection pointed to:

#####
commit 70bd6bc82ba64c1d197d3ec823f43c4a454b2920
Author:     Karl Williamson <khw@cpan.org>
AuthorDate: Wed Mar 20 22:59:39 2019 -0600
Commit:     Karl Williamson <khw@cpan.org>
CommitDate: Thu Mar 21 10:50:01 2019 -0600

locale.c: Don't try to recreate the LC_ALL C locale

On threaded perls, we create a locale object for LC_ALL "C"
early in the startup phase. When the user asks for that
locale, we can just switch to it instead of trying to
create a new one.

Doing the creation worked, but ended up with a memory leak.
My guess, and its only a guess, is that it's a bug in glibc
newlocale.c, in which it does an early return, not doing
proper cleanup, when it discovers it can re-use an existing
locale without needing to create a new one.

The reason I think its a glibc bug is that the sample
one-liner sent to me

PERL_DESTRUCT_LEVEL=2 valgrind --leak-check=full ./perl -DLv -Ilib 
-e'require POSIX;POSIX::setlocale(&POSIX::LC_ALL, "C");' 2>&1 | more

produced a stack output of where the leaked memory had been
allocated. I put a print immediately after that line, and
prints at the points where things get freed. Every
allocation was matched by an attempt to free it. But
clearly at least one failed. freelocale() returns void, so
can't be checked for failing.

Anyway, it's better to try not to create a new locale when
we already have an existing one, and doing so, as this
commit does, causes the leak to go away.

No tests are added, as there are plenty of similar tests
already in the suite, and they all should have been
leaking.

#####

Result of make test_harness:

#####
Test Summary Report
-------------------
run/switches.t                                                   (Wstat: 
0 Tests: 137 Failed: 3)
   Failed tests:  121, 123, 129
re/regexp_qr_embed_thr.t                                         (Wstat: 
139 Tests: 9 Failed: 0)
   Non-zero wait status: 139
   Parse errors: No plan found in TAP output
op/threads-dirh.t                                                (Wstat: 
138 Tests: 3 Failed: 1)
   Failed test:  1
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 6 tests but ran 3.
op/threads.t                                                     (Wstat: 
139 Tests: 9 Failed: 1)
   Failed test:  1
   Non-zero wait status: 139
   Parse errors: Bad plan.  You planned 30 tests but ran 9.
../cpan/DB_File/t/db-threads.t                                   (Wstat: 
138 Tests: 4 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 7 tests but ran 4.
../cpan/Module-Metadata/t/extract-version.t                      (Wstat: 
0 Tests: 224 Failed: 0)
   TODO passed:   7, 11, 15, 19, 23, 27, 31, 35, 39, 43, 47
                 51, 55, 59, 63, 67, 71, 75, 79, 83, 87
                 91, 95, 99, 106, 110, 114, 118, 122, 126
                 130, 134, 138, 142, 149, 153, 157, 161
                 165, 169, 173, 177, 181, 185, 195, 199
                 203
../cpan/Test-Simple/t/Legacy/threads.t                           (Wstat: 
139 Tests: 0 Failed: 0)
   Non-zero wait status: 139
   Parse errors: Bad plan.  You planned 6 tests but ran 0.
../cpan/Test-Simple/t/Test2/acceptance/try_it_threads.t          (Wstat: 
138 Tests: 0 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 6 tests but ran 0.
../cpan/Test-Simple/t/Test2/modules/API/Instance.t               (Wstat: 
138 Tests: 40 Failed: 0)
   Non-zero wait status: 138
   Parse errors: No plan found in TAP output
../dist/threads-shared/t/av_refs.t                               (Wstat: 
138 Tests: 5 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 14 tests but ran 5.
../dist/threads-shared/t/av_simple.t                             (Wstat: 
138 Tests: 14 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 47 tests but ran 14.
../dist/threads-shared/t/blessed.t                               (Wstat: 
138 Tests: 25 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 37 tests but ran 25.
../dist/threads-shared/t/clone.t                                 (Wstat: 
138 Tests: 6 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 40 tests but ran 6.
../dist/threads-shared/t/cond.t                                  (Wstat: 
138 Tests: 5 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 32 tests but ran 5.
../dist/threads-shared/t/hv_refs.t                               (Wstat: 
138 Tests: 5 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 20 tests but ran 5.
../dist/threads-shared/t/hv_simple.t                             (Wstat: 
138 Tests: 2 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 16 tests but ran 2.
../dist/threads-shared/t/object.t                                (Wstat: 
138 Tests: 16 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 28 tests but ran 16.
../dist/threads-shared/t/object2.t                               (Wstat: 
138 Tests: 131 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 133 tests but ran 131.
../dist/threads-shared/t/shared_attr.t                           (Wstat: 
138 Tests: 4 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 101 tests but ran 4.
../dist/threads-shared/t/sv_refs.t                               (Wstat: 
138 Tests: 7 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 21 tests but ran 7.
../dist/threads-shared/t/sv_simple.t                             (Wstat: 
138 Tests: 8 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 11 tests but ran 8.
../dist/threads-shared/t/wait.t                                  (Wstat: 
138 Tests: 11 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 91 tests but ran 11.
../dist/threads-shared/t/waithires.t                             (Wstat: 
138 Tests: 8 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 63 tests but ran 8.
../dist/threads/t/basic.t                                        (Wstat: 
138 Tests: 3 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 34 tests but ran 3.
../dist/threads/t/blocks.t                                       (Wstat: 
138 Tests: 0 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 5 tests but ran 0.
../dist/threads/t/context.t                                      (Wstat: 
138 Tests: 3 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 31 tests but ran 3.
../dist/Thread-Queue/t/01_basic.t                                (Wstat: 
138 Tests: 2 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 81 tests but ran 2.
../dist/threads/t/end.t                                          (Wstat: 
138 Tests: 2 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 6 tests but ran 2.
../dist/threads/t/err.t                                          (Wstat: 
138 Tests: 4 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 10 tests but ran 4.
../dist/Thread-Queue/t/05_extract.t                              (Wstat: 
138 Tests: 8 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 20 tests but ran 8.
../dist/threads/t/exit.t                                         (Wstat: 
139 Tests: 5 Failed: 0)
   Non-zero wait status: 139
   Parse errors: Bad plan.  You planned 18 tests but ran 5.
../dist/Thread-Queue/t/06_insert.t                               (Wstat: 
139 Tests: 3 Failed: 0)
   Non-zero wait status: 139
   Parse errors: Bad plan.  You planned 16 tests but ran 3.
../dist/Thread-Queue/t/09_ended.t                                (Wstat: 
139 Tests: 20 Failed: 0)
   Non-zero wait status: 139
   Parse errors: Bad plan.  You planned 60 tests but ran 20.
../dist/threads/t/free.t                                         (Wstat: 
139 Tests: 1 Failed: 0)
   Non-zero wait status: 139
   Parse errors: Bad plan.  You planned 29 tests but ran 1.
../dist/Thread-Queue/t/10_timed.t                                (Wstat: 
139 Tests: 13 Failed: 0)
   Non-zero wait status: 139
   Parse errors: Bad plan.  You planned 19 tests but ran 13.
../dist/threads/t/join.t                                         (Wstat: 
138 Tests: 2 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 20 tests but ran 2.
../dist/Thread-Queue/t/11_limit.t                                (Wstat: 
139 Tests: 10 Failed: 0)
   Non-zero wait status: 139
   Parse errors: Bad plan.  You planned 13 tests but ran 10.
../dist/threads/t/kill.t                                         (Wstat: 
139 Tests: 2 Failed: 0)
   Non-zero wait status: 139
   Parse errors: Bad plan.  You planned 18 tests but ran 2.
../dist/threads/t/libc.t                                         (Wstat: 
139 Tests: 1 Failed: 0)
   Non-zero wait status: 139
   Parse errors: Bad plan.  You planned 11 tests but ran 1.
../dist/threads/t/list.t                                         (Wstat: 
138 Tests: 4 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 15 tests but ran 4.
../dist/threads/t/problems.t                                     (Wstat: 
138 Tests: 5 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 10 tests but ran 5.
../dist/threads/t/stack.t                                        (Wstat: 
138 Tests: 10 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 18 tests but ran 10.
../dist/threads/t/state.t                                        (Wstat: 
138 Tests: 25 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 59 tests but ran 25.
../dist/threads/t/stress_cv.t                                    (Wstat: 
138 Tests: 3 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 61 tests but ran 3.
../dist/threads/t/stress_re.t                                    (Wstat: 
139 Tests: 3 Failed: 0)
   Non-zero wait status: 139
   Parse errors: Bad plan.  You planned 61 tests but ran 3.
../dist/threads/t/stress_string.t                                (Wstat: 
138 Tests: 3 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 61 tests but ran 3.
../dist/threads/t/thread.t                                       (Wstat: 
138 Tests: 4 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 35 tests but ran 4.
../ext/File-Glob/t/threads.t                                     (Wstat: 
139 Tests: 1 Failed: 0)
   Non-zero wait status: 139
   Parse errors: No plan found in TAP output
../ext/Hash-Util-FieldHash/t/03_class.t                          (Wstat: 
139 Tests: 3 Failed: 0)
   Non-zero wait status: 139
   Parse errors: Bad plan.  You planned 7 tests but ran 3.
../ext/Hash-Util-FieldHash/t/04_thread.t                         (Wstat: 
139 Tests: 18 Failed: 0)
   Non-zero wait status: 139
   Parse errors: Bad plan.  You planned 30 tests but ran 18.
../ext/PerlIO-scalar/t/scalar.t                                  (Wstat: 
138 Tests: 83 Failed: 0)
   Non-zero wait status: 138
   Parse errors: Bad plan.  You planned 123 tests but ran 83.
../ext/XS-APItest/t/keyword_plugin_threads.t                     (Wstat: 
0 Tests: 1 Failed: 1)
   Failed test:  1
../lib/Thread.t                                                  (Wstat: 
139 Tests: 7 Failed: 0)
   Non-zero wait status: 139
   Parse errors: Bad plan.  You planned 13 tests but ran 7.
Files=2652, Tests=1228043, 1390 wallclock secs (138.41 usr 30.43 sys + 
855.43 cusr 170.59 csys = 1194.86 CPU)
Result: FAIL
*** [test_harness] Error code 6

make: stopped in /home/jkeenan/gitwork/perl
1 error
#####

Many of the failures above with "bad plan" are segfaults.  Example:

#####
[perl] $ ./perl -Ilib -Icpan/Test-Simple/lib 
cpan/Test-Simple/t/Legacy/threads.t
1..6
Segmentation fault (core dumped)
[perl] $ ./perl -Ilib -Icpan/Test-Simple/lib 
cpan/Test-Simple/t/Test2/acceptance/try_it_threads.t
1..6
Bus error (core dumped)
[perl] $ ./perl -Ilib -Icpan/Test-Simple/lib 
cpan/Test-Simple/t/Test2/modules/API/Instance.t
ok 1 - Just to get things initialized.
...
ok 40 - No errors
Segmentation fault (core dumped)
#####


See also Carlos Guevara's smoke-test results, e.g.,

http://perl5.test-smoke.org/report/83046

Thank you very much.
Jim Keenan


#####
Summary of my perl5 (revision 5 version 29 subversion 10) configuration:
   Commit id: 70bd6bc82ba64c1d197d3ec823f43c4a454b2920
   Platform:
     osname=freebsd
     osvers=13.0-current
     archname=amd64-freebsd-thread-multi
     uname='freebsd perl-reporter-05 13.0-current freebsd 13.0-current 
r340361 generic amd64 '
     config_args='-des -Dusedevel -Duseithreads -Doptimize=-O2 -pipe 
-fstack-protector -fno-strict-aliasing'
     hint=recommended
     useposix=true
     d_sigaction=define
     useithreads=define
     usemultiplicity=define
     use64bitint=define
     use64bitall=define
     uselongdouble=undef
     usemymalloc=n
     default_inc_excludes_dot=define
     bincompat5005=undef
   Compiler:
     cc='cc'
     ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H 
-fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include 
-D_FORTIFY_SOURCE=2'
     optimize='-O2 -pipe -fstack-protector -fno-strict-aliasing'
     cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H 
-fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
     ccversion=''
     gccversion='4.2.1 Compatible FreeBSD Clang 6.0.1 
(tags/RELEASE_601/final 335540)'
     gccosandvers=''
     intsize=4
     longsize=8
     ptrsize=8
     doublesize=8
     byteorder=12345678
     doublekind=3
     d_longlong=define
     longlongsize=8
     d_longdbl=define
     longdblsize=16
     longdblkind=3
     ivtype='long'
     ivsize=8
     nvtype='double'
     nvsize=8
     Off_t='off_t'
     lseeksize=8
     alignbytes=8
     prototype=define
   Linker and Libraries:
     ld='cc'
     ldflags ='-pthread -Wl,-E  -fstack-protector-strong -L/usr/local/lib'
     libpth=/usr/lib /usr/local/lib /usr/lib/clang/6.0.1/lib /usr/lib
     libs=-lpthread -lgdbm -ldl -lm -lcrypt -lutil
     perllibs=-lpthread -ldl -lm -lcrypt -lutil
     libc=
     so=so
     useshrplib=false
     libperl=libperl.a
     gnulibc_version=''
   Dynamic Linking:
     dlsrc=dl_dlopen.xs
     dlext=so
     d_dlsymun=undef
     ccdlflags=' '
     cccdlflags='-DPIC -fPIC'
     lddlflags='-shared  -L/usr/local/lib -fstack-protector-strong'


Characteristics of this binary (from libperl):
   Compile-time options:
     HAS_TIMES
     MULTIPLICITY
     PERLIO_LAYERS
     PERL_COPY_ON_WRITE
     PERL_DONT_CREATE_GVSV
     PERL_IMPLICIT_CONTEXT
     PERL_MALLOC_WRAP
     PERL_OP_PARENT
     PERL_PRESERVE_IVUV
     PERL_USE_DEVEL
     USE_64_BIT_ALL
     USE_64_BIT_INT
     USE_ITHREADS
     USE_LARGE_FILES
     USE_LOCALE
     USE_LOCALE_COLLATE
     USE_LOCALE_CTYPE
     USE_LOCALE_NUMERIC
     USE_LOCALE_TIME
     USE_PERLIO
     USE_PERL_ATOF
     USE_REENTRANT_API
     USE_THREAD_SAFE_LOCALE
   Built under freebsd
   Compiled at Mar 25 2019 23:22:07
   %ENV:
     PERL2DIR="/home/jkeenan/gitwork/perl2"
     PERL_WORKDIR="/home/jkeenan/gitwork/perl"
   @INC:
     lib
     /usr/local/lib/perl5/site_perl/5.29.10/amd64-freebsd-thread-multi
     /usr/local/lib/perl5/site_perl/5.29.10
     /usr/local/lib/perl5/5.29.10/amd64-freebsd-thread-multi
     /usr/local/lib/perl5/5.29.10

#####


Thread Previous


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About