develooper Front page | perl.perl5.porters | Postings from May 2004

RE: [perl #29873] seg fault w/v5.8.4 on IBM RS/6k when populating large hash

Thread Previous | Thread Next
From:
al.danial
Date:
May 26, 2004 10:04
Subject:
RE: [perl #29873] seg fault w/v5.8.4 on IBM RS/6k when populating large hash
Message ID:
48C9AB115A3DEB44AA5CA4B56BB6E4B502EEB655@MBSP02.services.sp.trw.com
Merijn, thanks for looking into the problem so quickly.
I neglected to mention that the other machines on which
the program runs all have much less memory than the 8 GB
on the IBM.  The Linux box has (only) 1 GB RAM and has 
no problems with the code.

Here is my rough estimate of the memory that would be
taken by the scalar data within the hash of hash of lists:

    3 *        # number of sets
    300000 *   # number of ID's
    7 *        # number of rand's
   (9*4        # number of int's per scalar variable (??)
               # times number of bytes per int plus
    + 8)       # number of bytes for the rand double 

  = 277 MB

I have no idea how much additional space is needed by the hash 
tables but even if it is double the amount above, the code 
should be able to (and does on Linux) run comfortably within 
1 GB.          -- Al

 -+- -+- -+- -+- -+- -+- -+- -+- -+- -+- -+- -+- -+- -+-
Albert Danial  al.danial@ngc.com  310-813-9516 fax: -4467
NGST R10/1328 One Space Park, Redondo Bch., CA 90278, USA 



=> -----Original Message-----
=> From: H. Merijn Brand via RT [mailto:perlbug-followup@perl.org]
=> Sent: Wednesday, May 26, 2004 1:50 AM
=> To: Danial, Al
=> Subject: Re: [perl #29873] seg fault w/v5.8.4 on IBM RS/6k when
=> populating large hash 
=> 
=> 
=> On Wed 26 May 2004 02:41, "al.danial@ngc.com (via RT)" 
=> <perlbug-followup@perl.org> wrote:
=> > # New Ticket Created by  al.danial@ngc.com 
=> > # Please include the string:  [perl #29873]
=> > # in the subject line of all future correspondence about 
=> this issue. 
=> > # <URL: http://rt.perl.org:80/rt3/Ticket/Display.html?id=29873 >
=> > 
=> > 
=> > This is a bug report for perl from al.danial@ngc.com,
=> > generated with the help of perlbug 1.35 running under perl v5.8.4.
=> > 
=> > 
=> > -----------------------------------------------------------------
=> > This program causes a seg fault on an IBM RS/6000 with AIX 4.3.3.
=> > 
=> >     #!/usr/local/perl-5.8.4/bin/perl -w
=> >     my $upper_limit;
=> >     $upper_limit = 275000;  # works
=> >     $upper_limit = 300000;  # fails
=> >     my %data = ();
=> >     foreach my $set (qw( A B C )) {
=> >         foreach my $id (1..$upper_limit) {
=> >             $data{$set}{$id} = [ rand, rand, rand, rand, 
=> rand, rand, rand, ];
=> >         }
=> >     }
=> 
=> That fully depends on your system configuration. On my 
=> system, the first limit
=> also causes problems on both recent and older perl's:
=> 
=> i2:/tmp 103 > perl test.pl
=> Out of memory!
=> Exit 1
=> i2:/tmp 104 >
=> 
=> i2:/tmp 104 > perl5.6.1 test.pl
=> Out of memory!
=> Exit 1
=> i2:/tmp 105 >
=> 
=> No seg faults though. Mahor difference between my perl's and 
=> yours is that
=> myine are built with vac, and your's are built with gcc.
=> 
=> > The failure happens with Perl version 5.6.1 on the IBM as well.
=> > 
=> > The workstation has 8 GB of memory so I don't think it is because
=> > the code is using too much space.  The above program works fine
=> > on Linux, Sun, and SGI workstations using Perl 5.8.0.
=> > 
=> > 
=> > [Please do not change anything below this line]
=> > -----------------------------------------------------------------
=> > ---
=> > Flags:
=> >     category=core
=> >     severity=high
=> > ---
=> > Site configuration information for perl v5.8.4:
=> > 
=> > Configured by adanial at Thu May 20 11:47:39 PDT 2004.
=> > 
=> > Summary of my perl5 (revision 5 version 8 subversion 4) 
=> configuration:
=> >   Platform:
=> >     osname=aix, osvers=4.3.3.0, archname=aix
=> >     uname='aix mohr 3 4 00041bcf4c00 '
=> >     config_args='-Dprefix=/usr/local/perl-5.8.4 -Dcc=gcc -de'
=> >     hint=recommended, useposix=true, d_sigaction=define
=> >     usethreads=undef use5005threads=undef 
=> useithreads=undef usemultiplicity=undef
=> >     useperlio=define d_sfio=undef uselargefiles=define 
=> usesocks=undef
=> >     use64bitint=undef use64bitall=undef uselongdouble=undef
=> >     usemymalloc=n, bincompat5005=undef
=> >   Compiler:
=> >     cc='gcc', ccflags ='-D_ALL_SOURCE -D_ANSI_C_SOURCE 
=> -D_POSIX_SOURCE -DUSE_NATIVE_DLOPEN -fno-strict-aliasing 
=> -I/usr/local/include -D_LARGE_FILES',
=> >     optimize='-O',
=> >     cppflags='-D_ALL_SOURCE -D_ANSI_C_SOURCE 
=> -D_POSIX_SOURCE -DUSE_NATIVE_DLOPEN -fno-strict-aliasing 
=> -I/usr/local/include'
=> >     ccversion='', gccversion='3.3.3', gccosandvers='aix4.3.3.0'
=> >     intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=4321
=> >     d_longlong=define, longlongsize=8, d_longdbl=define, 
=> longdblsize=8
=> >     ivtype='long', ivsize=4, nvtype='double', nvsize=8, 
=> Off_t='off_t', lseeksize=8
=> >     alignbytes=8, prototype=define
=> >   Linker and Libraries:
=> >     ld='gcc', ldflags =' -Wl,-brtl -L/usr/local/lib -Wl,-b32'
=> >     libpth=/usr/local/lib /lib /usr/lib /usr/ccs/lib
=> >     libs=-lbind -lnsl -lgdbm -ldbm -ldb -ldl -lld -lm 
=> -lcrypt -lc -lbsd
=> >     perllibs=-lbind -lnsl -ldl -lld -lm -lcrypt -lc -lbsd
=> >     libc=/lib/libc.a, so=a, useshrplib=false, libperl=libperl.a
=> >     gnulibc_version=''
=> >   Dynamic Linking:
=> >     dlsrc=dl_aix.xs, dlext=so, d_dlsymun=undef, 
=> ccdlflags='-Xlinker 
=> -bE:/usr/local/perl-5.8.4/lib/5.8.4/aix/CORE/perl.exp'
=> >     cccdlflags=' ', lddlflags='  -Wl,-bhalt:4 -Wl,-bM:SRE 
=> -Wl,-bI:$(PERL_INC)/perl.exp -Wl,-bE:$(BASEEXT).exp 
=> -Wl,-bnoentry -lc -L/usr/local/lib'
=> > 
=> > Locally applied patches:
=> >     
=> > 
=> > ---
=> > @INC for perl v5.8.4:
=> >     /usr/local/perl-5.8.4/lib/5.8.4/aix
=> >     /usr/local/perl-5.8.4/lib/5.8.4
=> >     /usr/local/perl-5.8.4/lib/site_perl/5.8.4/aix
=> >     /usr/local/perl-5.8.4/lib/site_perl/5.8.4
=> >     /usr/local/perl-5.8.4/lib/site_perl
=> >     .
=> > 
=> > ---
=> > Environment for perl v5.8.4:
=> >     HOME=/home/adanial
=> >     LANG=en_US
=> >     LANGUAGE (unset)
=> >     LC__FASTMSG=true
=> >     
=> LD_LIBRARY_PATH=/opt/platform/lsf/4.2/aix4/lib:/usr/local/kma
=> tplot-0.2.2/lib
=> >     LIBPATH=/usr/lib::/opt/matlab53/extern/lib/ibm_rs
=> >     LOGDIR (unset)
=> >     
=> PATH=/opt/platform/lsf/4.2/aix4/bin:/opt/platform/lsf/4.2/aix
4/etc:/opt/IDEAS7/ideas/ms7/bin:/home/adanial/bin:/bin:/usr/bin:/usr/ccs/bin:/usr/ucb:/usr/bin/X11:/usr/=> dt/bin:/usr/local/bin:.:/opt/sysnoise/5.5/bin:/opt/matlab53/b
in:/opt/langtools/bin:/opt/linux/IBMJava2-13/bin
=> >     PERL_BADLANG (unset)
=> >     SHELL=/bin/bash
=> > 
=> 
=> -- 
=> H.Merijn Brand        Amsterdam Perl Mongers 
(http://amsterdam.pm.org/)
using perl-5.6.1, 5.8.3, & 5.9.x, and 809 on  HP-UX 10.20 & 11.00, 11i,
   AIX 4.3, SuSE 9.0, and Win2k.           http://www.cmve.net/~merijn/
http://archives.develooper.com/daily-build@perl.org/   perl-qa@perl.org
send smoke reports to: smokers-reports@perl.org, QA: http://qa.perl.org



Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About