develooper Front page | perl.perl5.porters | Postings from April 2003

1GB RAM limit for perl5.8.0 -Dusemymalloc under linux?

Thread Next
From:
Alberto Accomazzi
Date:
April 3, 2003 05:19
Subject:
1GB RAM limit for perl5.8.0 -Dusemymalloc under linux?
Message ID:
3E8B0FE0.80901@cfa.harvard.edu
I have recently recompiled perl 5.8.0 under linux/intel (redhat 7.3 + 
latest patches) with the -Dusemymalloc flag in order to get around a bug 
in libc-2.2.5's malloc affecting allocations of large hashes as detailed 
in http://archive.develooper.com/perl5-porters@perl.org/msg88107.html

Unfortunately the resulting binary seems to be limited to accessing 
approximately 900MB of RAM, at least from the tests I ran.  Has anybody 
seen this and is there a way around it?  My system has 3GB RAM (plus 
swap) and the memory limit per process (in user space) is 2.5GB, which I 
am able to access using the perl binary compiled with libc's malloc.

TIA,

-- Alberto


$ /usr/local/perl5.8.0-mymalloc/bin/perl5.8.0 -e '$| = 1; while (1) { 
$n++; push(@a, 1..1048576); printf("%3d - ", $n); system("grep ^VmSize 
/proc/$$/status") }'
   1 - VmSize:      53024 kB
   2 - VmSize:      74656 kB
[...]
  31 - VmSize:     688564 kB
  32 - VmSize:     709136 kB
Out of memory during "large" request for 268439552 bytes, total sbrk() 
is 723216384 bytes at -e line 1.


$ /usr/local/bin/perl5.8.0 -e '$| = 1; while (1) { $n++; push(@a, 
1..1048576); printf("%3d - ", $n); system("grep ^VmSize /proc/$$/status") }'
   1 - VmSize:      65012 kB
   2 - VmSize:      93932 kB
[...]
126 - VmSize:    2655960 kB
127 - VmSize:    2673368 kB
Out of memory!

$ uname -a
Linux adsfife 2.4.18-24.7.xsmp #1 SMP Fri Jan 31 07:28:32 EST 2003 i686 
unknown

$ cat /proc/meminfo
         total:    used:    free:  shared: buffers:  cached:
Mem:  3172843520 410439680 2762403840        0  1654784 254332928
Swap: 8587182080 40198144 8546983936
MemTotal:      3098480 kB
MemFree:       2697660 kB
MemShared:           0 kB
Buffers:          1616 kB
Cached:         230552 kB
SwapCached:      17820 kB
Active:          36704 kB
Inact_dirty:    165832 kB
Inact_clean:     71624 kB
Inact_target:    54832 kB
HighTotal:     2227648 kB
HighFree:      1959432 kB
LowTotal:       870832 kB
LowFree:        738228 kB
SwapTotal:     8385920 kB
SwapFree:      8346664 kB
Committed_AS:   205292 kB


=======================================================================
$ /usr/local/bin/perl5.8.0 -V
Summary of my perl5 (revision 5.0 version 8 subversion 0) configuration:
   Platform:
     osname=linux, osvers=2.4.18-24.7.xsmp, archname=i686-linux
     uname='linux adsfife 2.4.18-24.7.xsmp #1 smp fri jan 31 07:28:32 
est 2003 i686 unknown '
     config_args='-Dprefix=/usr/local -des'
     hint=recommended, useposix=true, d_sigaction=define
     usethreads=undef use5005threads=undef useithreads=undef 
usemultiplicity=undef
     useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
     use64bitint=undef use64bitall=undef uselongdouble=undef
     usemymalloc=n, bincompat5005=undef
   Compiler:
     cc='cc', ccflags ='-fno-strict-aliasing -I/usr/local/include 
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
     optimize='-O2',
     cppflags='-fno-strict-aliasing -I/usr/local/include 
-I/usr/include/gdbm'
     ccversion='', gccversion='2.96 20000731 (Red Hat Linux 7.3 
2.96-113)', gccosandvers=''
     intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
     d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
     ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', 
lseeksize=8
     alignbytes=4, prototype=define
   Linker and Libraries:
     ld='cc', ldflags =' -L/usr/local/lib'
     libpth=/usr/local/lib /lib /usr/lib
     libs=-lnsl -lndbm -lgdbm -ldl -lm -lc -lcrypt -lutil
     perllibs=-lnsl -ldl -lm -lc -lcrypt -lutil
     libc=/lib/libc-2.2.5.so, so=so, useshrplib=false, libperl=libperl.a
     gnulibc_version='2.2.5'
   Dynamic Linking:
     dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic'
     cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'


Characteristics of this binary (from libperl):
   Compile-time options: USE_LARGE_FILES
   Built under linux
   Compiled at Apr  1 2003 16:35:28
   @INC:
     /usr/local/lib/perl5/5.8.0/i686-linux
     /usr/local/lib/perl5/5.8.0
     /usr/local/lib/perl5/site_perl/5.8.0/i686-linux
     /usr/local/lib/perl5/site_perl/5.8.0
     /usr/local/lib/perl5/site_perl/5.6.1
     /usr/local/lib/perl5/site_perl
     .

========================================================================
$ /usr/local/perl5.8.0-mymalloc/bin/perl5.8.0 -V
Summary of my perl5 (revision 5.0 version 8 subversion 0) configuration:
   Platform:
     osname=linux, osvers=2.4.18-24.7.xsmp, archname=i686-linux
     uname='linux adsfife 2.4.18-24.7.xsmp #1 smp fri jan 31 07:28:32 
est 2003 i686 unknown '
     config_args='-Dusemymalloc -Dprefix=/usr/local/perl5.8.0-mymalloc -des'
     hint=recommended, useposix=true, d_sigaction=define
     usethreads=undef use5005threads=undef useithreads=undef 
usemultiplicity=undef
     useperlio=define d_sfio=undef uselargefiles=define usesocks=undef
     use64bitint=undef use64bitall=undef uselongdouble=undef
     usemymalloc=y, bincompat5005=undef
   Compiler:
     cc='cc', ccflags ='-fno-strict-aliasing -I/usr/local/include 
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -I/usr/include/gdbm',
     optimize='-O2',
     cppflags='-fno-strict-aliasing -I/usr/local/include 
-I/usr/include/gdbm'
     ccversion='', gccversion='2.96 20000731 (Red Hat Linux 7.3 
2.96-113)', gccosandvers=''
     intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
     d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
     ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', 
lseeksize=8
     alignbytes=4, prototype=define
   Linker and Libraries:
     ld='cc', ldflags =' -L/usr/local/lib'
     libpth=/usr/local/lib /lib /usr/lib
     libs=-lnsl -lndbm -lgdbm -ldl -lm -lc -lcrypt -lutil
     perllibs=-lnsl -ldl -lm -lc -lcrypt -lutil
     libc=/lib/libc-2.2.5.so, so=so, useshrplib=false, libperl=libperl.a
     gnulibc_version='2.2.5'
   Dynamic Linking:
     dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic'
     cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'


Characteristics of this binary (from libperl):
   Compile-time options: USE_LARGE_FILES
   Built under linux
   Compiled at Apr  2 2003 10:20:40
   @INC:
     /usr/local/perl5.8.0-mymalloc/lib/5.8.0/i686-linux
     /usr/local/perl5.8.0-mymalloc/lib/5.8.0
     /usr/local/perl5.8.0-mymalloc/lib/site_perl/5.8.0/i686-linux
     /usr/local/perl5.8.0-mymalloc/lib/site_perl/5.8.0
     /usr/local/perl5.8.0-mymalloc/lib/site_perl
     .


****************************************************************************
Alberto Accomazzi
NASA Astrophysics Data System                     http://adswww.harvard.edu
Harvard-Smithsonian Center for Astrophysics      http://cfa-www.harvard.edu
60 Garden Street, MS 83, Cambridge, MA 02138 USA
****************************************************************************


Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About