[perl #112198] startup valgrind error on OS X

# New Ticket Created by  Nicholas Clark 
# Please include the string:  [perl #112198]
# in the subject line of all future correspondence about this issue. 
# <URL: https://rt.perl.org:443/rt3/Ticket/Display.html?id=112198 >


This is a bug report for perl from nick@ccl4.org,
generated with the help of perlbug 1.39 running under perl 5.15.9.


-----------------------------------------------------------------
[Please describe your issue here]

I'm tempted to report this as a bug, then reject it for the reasons described,
so that the report should show up in searches.

For blead, on OS X Snow Leopard, I'm seeing this:

$ valgrind --dsymutil=yes ./perl -e0
==60650== Memcheck, a memory error detector
==60650== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==60650== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==60650== Command: ./perl -e0
==60650== 
--60650-- run: /usr/bin/dsymutil "./perl"
==60650== Invalid read of size 8
==60650==    at 0x7FFFFFE00A71: ???
==60650==    by 0x1001AD94F: __inline_memmove_chk (_string.h:69)
==60650==    by 0x1001D8471: Perl_sv_setpvn (sv.c:4499)
==60650==    by 0x1001F9BE4: Perl_newSVpv (sv.c:8405)
==60650==    by 0x10004F036: S_init_postdump_symbols (perl.c:4200)
==60650==    by 0x10003EC32: S_parse_body (perl.c:2144)
==60650==    by 0x10003D7AC: perl_parse (perl.c:1633)
==60650==    by 0x1000013C6: main (perlmain.c:118)
==60650==  Address 0x1009e5e08 is 136 bytes inside a block of size 142 alloc'd
==60650==    at 0x10041F5CF: malloc (vg_replace_malloc.c:266)
==60650==    by 0x10012C81E: Perl_safesysmalloc (util.c:100)
==60650==    by 0x1001352A8: Perl_my_setenv (util.c:2067)
==60650==    by 0x10003D3EF: perl_parse (perl.c:1578)
==60650==    by 0x1000013C6: main (perlmain.c:118)
==60650== 
==60650== 
==60650== HEAP SUMMARY:
==60650==     in use at exit: 107,497 bytes in 609 blocks
==60650==   total heap usage: 728 allocs, 119 frees, 134,992 bytes allocated
==60650== 
==60650== LEAK SUMMARY:
==60650==    definitely lost: 0 bytes in 0 blocks
==60650==    indirectly lost: 0 bytes in 0 blocks
==60650==      possibly lost: 0 bytes in 0 blocks
==60650==    still reachable: 107,409 bytes in 608 blocks
==60650==         suppressed: 88 bytes in 1 blocks
==60650== Rerun with --leak-check=full to see details of leaked memory
==60650== 
==60650== For counts of detected and suppressed errors, rerun with: -v
==60650== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)

This is with valgrind 3.7.0, built by macports. The reported error is from
this call to newSVpv in perl.c:

#if defined(MSDOS) && !defined(DJGPP)
	    *s = '\0';
	    (void)strupr(old_var);
	    *s = '=';
#endif
	    sv = newSVpv(s+1, 0);
	    (void)hv_store(hv, old_var, s - old_var, sv, 0);
	    if (env_is_not_environ)
	        mg_set(sv);
	  }
      }
#endif /* USE_ENVIRON_ARRAY */

which looks to be copying the environment. After quite a bit of grovelling,
this seems to be very sensitive to the precise shape of the environment, and
happens to trigger on my PATH, which *is* 136 bytes long (so 137 with a
trailing NUL or a newline):

$ echo $PATH | wc
       1       1     137

If I copy that to a variable TEST it triggers on that instead, and no error
from PATH


The code in question is deep in this bit of the system string headers:

#define memmove(dest, src, len)					\
  ((__darwin_obsz0 (dest) != (size_t) -1)				\
   ? __builtin___memmove_chk (dest, src, len, __darwin_obsz0 (dest))	\
   : __inline_memmove_chk (dest, src, len))

static __inline void *
__inline_memmove_chk (void *__dest, const void *__src, size_t __len)
{
  return __builtin___memmove_chk (__dest, __src, __len, __darwin_obsz0(__dest));
}


[which looks to be buggy in that I *think* both halves of the ternary are the
same code.]

In turn, __darwin_obsz0 is this:

#define __darwin_obsz0(object) __builtin_object_size (object, 0)


So, what's buggy? After a lot of trying to get my head around the above
routines, I *think* there are no (functional) bugs in the system headers.
They seem to be using the gcc builtins correctly.

I also don't think that there's a bug in perl - if I call *printf() on the
value valgrind doesn't generate an error, and I can call strlen() quite
safely on it without running off the end. So I don't think it's our fault.

What I *think* is happening is an unfortunate clash of assumptions between
valgrind and the OS. Look closely at this:

==60650== Invalid read of size 8
==60650==  Address 0x1009e5e08 is 136 bytes inside a block of size 142 alloc'd

142 is not a multiple of 4, let alone 8. And that's an error about a read
beyond the end, not a use of uninitialised memory.

Whilst 142 is undoubtedly legal, this is not what one would expect from a block
returned from malloc(). So my hunch is that the OS/toolchain code assumes that
all memory is *mapped* up to the next 8 byte boundary, and so it's safe to
always read in 8 byte increments. (And is not buggy because it never acts on
bytes beyond the end of actual length, bytes which are undefined). Whereas
valgrind assumes that nothing is making these sort of assumptions.
(And I don't know x86_64 assembler, and I'm not about to learn to go grovelling
further. Single stepping compiled C code isn't fun)

So I don't think it's our bug. But I'm not sure whose it is, or if it's sort
of neither-but-both.

But I thought I should report this as if I don't, I'm sure at some point
someone will (correctly) be worried by what they see and report it.

Nicholas Clark

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
    category=core
    severity=low
---
Site configuration information for perl 5.15.9:

Configured by nick at Mon Apr  2 11:54:43 BST 2012.

Summary of my perl5 (revision 5 version 15 subversion 9) configuration:
  Commit id: 8928fb824a6d653a67f3b20a68ef9606292dbf15
  Platform:
    osname=darwin, osvers=10.8.0, archname=darwin-2level
    uname='darwin mouse-mill.local 10.8.0 darwin kernel version 10.8.0: tue jun 7 16:33:36 pdt 2011; root:xnu-1504.15.3~1release_i386 i386 '
    config_args='-Dusedevel=y -Dcc=ccache gcc -Dld=gcc -Ubincompat5005 -Uinstallusrbinperl -Dcf_email=nick@ccl4.org -Dperladmin=nick@ccl4.org -Dinc_version_list=  -Dinc_version_list_init=0 -Dstatic_ext=Fcntl -DDEBUGGING -Doptimize=-g -Uusethreads -Uuse64bitall -Uuselongdouble -Uusemymalloc -Duseperlio -Dprefix=~/Sandpit/snap-v5.15.9-65-g8928fb8 -Dusevendorprefix -Dvendorprefix=~/Sandpit/vendor -Uuserelocatableinc -Uuseshrplib -de -Accccflags=-fcatch-undefined-behavior -Accflags=-DNO_MATHOMS -Umad'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=undef, usemultiplicity=undef
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=undef, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='ccache gcc', ccflags ='-fno-common -DPERL_DARWIN -no-cpp-precomp -DNO_MATHOMS -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector -I/opt/local/include',
    optimize='-g',
    cppflags='-no-cpp-precomp -fno-common -DPERL_DARWIN -no-cpp-precomp -DNO_MATHOMS -DDEBUGGING -fno-strict-aliasing -pipe -fstack-protector -I/opt/local/include'
    ccversion='', gccversion='4.2.1 (Apple Inc. build 5666) (dot 3)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc', ldflags =' -fstack-protector -L/usr/local/lib -L/opt/local/lib'
    libpth=/usr/local/lib /opt/local/lib /usr/lib
    libs=-lgdbm -ldbm -ldl -lm -lutil -lc
    perllibs=-ldl -lm -lutil -lc
    libc=, so=dylib, useshrplib=false, libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags=' -bundle -undefined dynamic_lookup -L/usr/local/lib -L/opt/local/lib -fstack-protector'

Locally applied patches:
    

---
@INC for perl 5.15.9:
    lib
    /Users/nick/Sandpit/snap-v5.15.9-65-g8928fb8/lib/perl5/site_perl/5.15.9/darwin-2level
    /Users/nick/Sandpit/snap-v5.15.9-65-g8928fb8/lib/perl5/site_perl/5.15.9
    /Users/nick/Sandpit/vendor/lib/perl5/vendor_perl/5.15.9/darwin-2level
    /Users/nick/Sandpit/vendor/lib/perl5/vendor_perl/5.15.9
    /Users/nick/Sandpit/snap-v5.15.9-65-g8928fb8/lib/perl5/5.15.9/darwin-2level
    /Users/nick/Sandpit/snap-v5.15.9-65-g8928fb8/lib/perl5/5.15.9
    .

---
Environment for perl 5.15.9:
    DYLD_LIBRARY_PATH (unset)
    HOME=/Users/nick
    LANG=en_GB.ISO8859-1
    LANGUAGE (unset)
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/opt/local/bin:/opt/local/sbin:/Users/nick/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin:/usr/local/sbin:/sbin:/usr/sbin
    PERL_BADLANG (unset)
    SHELL=/bin/bash

• [perl #112198] startup valgrind error on OS X by Nicholas Clark
• Re: [perl #112198] startup valgrind error on OS X by Tony Cook
• Re: [perl #112198] startup valgrind error on OS X by Nicholas Clark