[perl #127885] [patches] enhance bench.pl to test same perl underdifferent options/args

# New Ticket Created by  Jim Cromie 
# Please include the string:  [perl #127885]
# in the subject line of all future correspondence about this issue. 
# <URL: https://rt.perl.org/Ticket/Display.html?id=127885 >


This is a bug report for perl from jim.cromie@gmail.com,
generated with the help of perlbug 1.40 running under perl 5.24.0.


-----------------------------------------------------------------
[Please describe your issue here]

Porting/bench.pl has 2 interrelated shortcomings:

1- it stores cachegrind results using {$testname}{$perlname}.
   This means that if the same $perlname is used 2x, the 2nd results
   overwrite the 1st.

2- it cannot pass different options/args to each Perl-Under-Test.

This patchset corrects those problems, allowing to test the same perl
under different runtime conditions, as caused by the use of different
-M<foo> etc options.  It was motivated by my desire to test the
runtime cost of using -DPERL_TRACE_OPS.

example 0:

   $] perl Porting/bench.pl --jobs=2 --verbose \
        --tests=/loop::for::my/ \
        --perlargs='-MDevel::Peek' \
        -- \
        perl5.24.0=+off      -e'"BEGIN{runops_debug(0)}"' \
        perl5.24.0=+traceops -e'"BEGIN{runops_debug(1)}"'


example 1:

   $] perl Porting/bench.pl --jobs=2 \
        --tests=/loop::for::my/ \
        -- perl5.18.2=+strict -Mstrict perl5.18.0=+dumper -MData::Dumper

yields:
    ...
    loop::for::my_array4
    empty for loop with my var and 4 integer array

           perl5.18.2+strict perl5.18.0+dumper
           ----------------- -----------------
        Ir            100.00            114.20
        Dr            100.00            119.35
        Dw            100.00            135.03
      COND            100.00             99.71

example 2:

   $] perl Porting/bench.pl --jobs=2 \
        --tests=/loop::for::my/ \
        -- \
        perl-A=+bare perl-A=+workload -DpsltocPmfr \
        perl-B=+bare perl-B=+workload -DpsltocPmfr

That wont actually work, cuz the -D<options> cause extra output which
the cachegrind output parser doesnt tolerate.  I havent tried
redirecting the PUT (perl-under-test) output elsewhere, it might fix
this.

example 3:

   $] perl Porting/bench.pl --jobs=2 \
       --verbose --debug --tests=/loop::for::pkg_/ \
       -- \
       perl5.23.9=+bare \
       perl5.23.9=+full=PERL_MEM_LOG=3mst -MData::Dumper

This uses the extended [=label=(ENV-name-value-pairs)] to add
PERL_MEM_LOG=3mst to the 2nd PUT's run-time environment.  Im not
entirely happy with the syntax of the construct, but the per-PUT %ENV
is needed for test scenarios like this one.

the patches:

0001-Porting-bench.pl-verbose-assist-on-tests-failure.patch
0002-prep-grind-run-to-accept-args-from-PUTs.patch
0003-Porting-bench.pl-protect-against-data-loss.patch
0004-Porting-bench.pl-allow-per-PUT-perl-under-test-optio.patch
0005-store-data-using-unique-label-not-perl-exe-which-may.patch
0006-grind_print-adjust-for-labels.patch
0007-add-pod-for-PUT.patch
0008-s-results-res_puts-in-process_puts.patch
0009-add-label-feature-which-appends-the-label-to-perl-na.patch
0010-provide-per-PUT-environment.patch

they are also available at
https://github.com/jimc/perl/tree/jimc/bench

They are still WIP; they extend the use-cases as I intended, but I
havent seen some of the expected differences (example 1,3 particularly),
and may have botched some of the internal keying.


[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
    category=core
    severity=low
---
Site configuration information for perl 5.24.0:

Configured by jimc at Thu Apr  7 11:37:42 MDT 2016.

Summary of my perl5 (revision 5 version 23 subversion 10) configuration:
  Commit id: 9357b2a3717073932fefec424b81a3c63a35e388
  Platform:
    osname=linux, osvers=4.6.0-rc1-p1, archname=i686-linux-thread-multi
    uname='linux popeye 4.6.0-rc1-p1 #88 smp sun mar 27 13:07:04 mdt 2016
i686 i686 i386 gnulinux '
    config_args='-des -Dusedevel -Dusethreads -DDEBUGGING=both
-Accflags=-DPERL_TRACE_OPS -DPERL_MEM_LOG'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    use64bitint=undef, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DPERL_TRACE_OPS
-DPERL_MEM_LOG -fwrapv -DDEBUGGING -fno-strict-aliasing -pipe
-fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2',
    optimize='-O2 -g',
    cppflags='-D_REENTRANT -D_GNU_SOURCE -DPERL_TRACE_OPS -DPERL_MEM_LOG
-fwrapv -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector-strong
-I/usr/local/include'
    ccversion='', gccversion='5.3.1 20151207 (Red Hat 5.3.1-2)',
gccosandvers=''
    intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234,
doublekind=3
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12,
longdblkind=3
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
    alignbytes=4, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/local/lib /usr/lib /lib /usr/lib64
    libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc
-lgdbm_compat
    perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
    libc=libc-2.22.so, so=so, useshrplib=false, libperl=libperl.a
    gnulibc_version='2.22'
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E'
    cccdlflags='-fPIC', lddlflags='-shared -O2 -g -L/usr/local/lib
-fstack-protector-strong'

Locally applied patches:
    RC0

---
@INC for perl 5.24.0:
    lib
    /home/jimc/perl5/lib/perl5/i686-linux-thread-multi
    /home/jimc/perl5/lib/perl5
    /home/jimc/perl5/lib/perl5/i686-linux-thread-multi
    /home/jimc/perl5/lib/perl5
    /usr/local/lib/perl5/site_perl/5.23.10/i686-linux-thread-multi
    /usr/local/lib/perl5/site_perl/5.23.10
    /usr/local/lib/perl5/5.23.10/i686-linux-thread-multi
    /usr/local/lib/perl5/5.23.10
    /usr/local/lib/perl5/site_perl
    .

---
Environment for perl 5.24.0:
    HOME=/home/jimc
    LANG=en_US.utf8
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)

PATH=/home/jimc/perl5/bin:/home/jimc/perl5/bin:/usr/lib/qt-3.3/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/jimc/.local/bin:/home/jimc/bin
    PERL5LIB=/home/jimc/perl5/lib/perl5:/home/jimc/perl5/lib/perl5
    PERL_BADLANG (unset)
    PERL_LOCAL_LIB_ROOT=/home/jimc/perl5:/home/jimc/perl5
    PERL_MB_OPT=--install_base "/home/jimc/perl5"
    PERL_MM_OPT=INSTALL_BASE=/home/jimc/perl5
    SHELL=/bin/bash

• [perl #127885] [patches] enhance bench.pl to test same perl underdifferent options/args by Jim Cromie
• Re: [perl #127885] [patches] enhance bench.pl to test same perlunder different options/args by Dave Mitchell
• Re: [perl #127885] [patches] enhance bench.pl to test same perlunder different options/args by Dave Mitchell
• Re: [perl #127885] [patches] enhance bench.pl to test same perlunder different options/args by Aristotle Pagaltzis
• Re: [perl #127885] [patches] enhance bench.pl to test same perlunder different options/args by Jim Cromie