develooper Front page | perl.perl5.porters | Postings from April 2019

[perl #133958] cpan/Test-Simple/t/Legacy/More.t: frequent failuresin non-threaded builds on FreeBSD-13

Thread Next
From:
James E Keenan via RT
Date:
April 2, 2019 21:31
Subject:
[perl #133958] cpan/Test-Simple/t/Legacy/More.t: frequent failuresin non-threaded builds on FreeBSD-13
Message ID:
rt-4.0.24-636-1554240676-1159.133958-15-0@perl.org
On Thu, 28 Mar 2019 15:15:50 GMT, jkeenan@pobox.com wrote:
> On 3/28/19 10:16 AM, Karl Williamson wrote:
> > On 3/28/19 6:21 AM, James E Keenan wrote:
> >> On 3/27/19 11:15 PM, karl williamson via RT wrote:
> >>> On 3/25/19 6:45 PM, James E Keenan (via RT) wrote:
> >>>> # New Ticket Created by  James E Keenan
> >>>> # Please include the string:  [perl #133958]
> >>>> # in the subject line of all future correspondence about this issue.
> >>>> # <URL: https://rt.perl.org/Ticket/Display.html?id=133958 >
> >>>>
> >>>>
> >>>> We are experiencing massive test failures in our smoke-testing of the
> >>>> Perl 5 core distribution on FreeBSD-13-CURRENT.  I am filing two
> >>>> separate bug tickets for these failures, since one failure appeared 
> >>>> long
> >>>> before the others.  This is the first of the two tickets.
> >>>>
> >>>> cpan/Test-Simple/t/Legacy/More.t is repeatedly failing during
> >>>> smoke-testing of non-threaded builds on FreeBSD-13.  See, e.g.:
> >>>>
> >>>> http://perl5.test-smoke.org/report/82470
> >>>> http://perl5.test-smoke.org/report/82946
> >>>>
> >>>> Test failures:
> >>>> ~~ ../cpan/Test-Simple/t/Legacy/More.t ......................... FAILED
> >>>> 54 Non-zero exit status: 1
> >>>>      [stdio] -Dcc=clang++ -Duse64bitall
> >>>>      [stdio] -Dcc=clang++
> >>>>      [stdio] -Dcc=clang++ DEBUGGING
> >>>>
> >>>> We only have one regular smoke-testing rig set up for FreeBSD-13, that
> >>>> of course being one of Carlos Guevara's rigs.  These failures:
> >>>>
> >>>> i. Do not seem to occur on threaded builds.
> >>>>
> >>>> ii. Seem to occur on most, but not all, non-threaded builds.
> >>>>
> >>>> iii. Appear to have first reported during a smoke-test in
> >>>> http://perl5.test-smoke.org/report/79958, which suggests that the
> >>>> failures were introduced in, or shortly before commit
> >>>> https://perl5.git.perl.org/perl.git/commitdiff/8021814b48.  That commit
> >>>> is in a deleted branch: smoke-me/khw-vlb.
> >>>>
> >>>> iv. Both Carlos and I have attempted to reproduce this failure manually
> >>>> -- but it only seems to crop up during smoke-testing.
> >>>>
> >>>> v. Here's the end of the file which is failing:
> >>>>
> >>>> #####
> >>>> $ tail cpan/Test-Simple/t/Legacy/More.t
> >>>>             );
> >>>>
> >>>>
> >>>> # rt.cpan.org 53469  is_deeply with regexes
> >>>> is_deeply( qr/a/, qr/a/, "same regex" );
> >>>>
> >>>>
> >>>> # These two tests must remain at the end.
> >>>> is( $@, $Err,               '$@ untouched' );
> >>>> cmp_ok( $!, '==', $Errno,   '$! untouched' );
> >>>> #####
> >>>>
> >>>> Thank you very much.
> >>>> Jim Keenan
> >>>>
> >>>> perl perl perl
> >>>>
> >>>>
> >>>
> >>>
> >>> Have you tried manually running this with address sanitizer?
> >>>
> >>
> >> 1. I myself have never run address sanitizer.  How would I do that?
> > 
> > Instructions are in perlhacktips.  It's just some Configure options to 
> > clang, which I believe you're already using.  Then you run the test suite.
> >>
> >> 2. Even if I did that, how would that address the problem that we've 
> >> only been able to observe this problem during smoke-testing?
> > 
> > There are two likely causes for something showing up only in smoke 
> > testing that I can think of off-hand.  One is timing, with lots of 
> > parallel jobs.  
> 
> I should note that in this environment $TEST_JOBS is set to 2.  So 
> (assuming I understand the test code correctly), only handy00.t and 
> handy01.t really matter.  I tried running those two in parallel by:
> 
> $> cd t; ./perl harness ../ext/XS-APItest/t/handy00.t 
> ../ext/XS-APItest/t/handy01.t; cd -
> 
> I could not reproduce the problem.  But it seems to occur regularly 
> inside Test-Smoke's harness.
> 
> > And the other is that there is a read of uninitialized 
> > memory.  Address sanitizer will catch this latter cause.  This is 
> > similar to valgrind on Linux.  
> 
> I will try to test with valgrind today or tomorrow.
> 

I mistyped.  I meant to say that I would try to build a perl with AddressSanitizer.  See below.

> 
> > I have run valgrind on the test suite 
> > recently with no errors beyond the usual two in CPAN (and yes there is a 
> > ticket for these open)
> >>

My efforts to build and test a perl at blead with AddressSanitizer in this FreeBSD-13 were largely unsuccessful.

After struggling with the syntax of the ./Configure invocation, I was eventually able to get 'make' to complete successfully.  See attachment with name starting with 'perl_V'.

However, I could not run 'make test_harness' successfully.  Normally, in these VMs I have $TEST_JOBS set to 2.  However, 'make test_harness' was 'Killed' in the t/re/*.t tests.  See attachment starting with 'test_harness'.

I then set TEST_JOBS=1 and ran 'make test' -- so no tests were running in parallel.  I got only a little farther on.  See attachment starting with 'make_test'.

Now, I concede:  (a) I've never previously attempted to build with AddressSanitizer; (b) This VM may be too small to run a perl built with AddressSanitizer.  But Carlos's FreeBSD-13 smoker, while not large, is larger than mine -- and we're still getting the failures on non-threaded builds in t/Legacy/More.t which are the subject of this RT.  See, e.g., http://perl5.test-smoke.org/report/83974.

Thank you very much.

-- 
James E Keenan (jkeenan@cpan.org)

---
via perlbug:  queue: perl5 status: open
https://rt.perl.org/Ticket/Display.html?id=133958

Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About