develooper Front page | perl.perl5.porters | Postings from July 2019

Re: [perl #134305] Memory Leak in perl 5.28 and 5.30

Thread Previous | Thread Next
From:
Alexander Shikov
Date:
July 27, 2019 04:36
Subject:
Re: [perl #134305] Memory Leak in perl 5.28 and 5.30
Message ID:
20190726064533.GA23762@crete.itcons.net.ua
On Fri, Jul 26, 2019 at 11:50:04 +0800, 郭樂聰 wrote:
> Hello,
> 
>  Interesting case, looks like it is a problem with locale implementation. I
> am
> looking into locale.c to see if I can find anything. Can you try to run your
> perl script with LC_ALL=C and see if the memory leak problem persists?
> Thanks a lot.
> 
> Best regards,
> Baggio

Hello!

Dear Baggio,

yes, you're right. With env LC_ALL=C script executes without leak:

>top -b | grep perl
20006 minotaur      1  81    0    19M  9716K RUN      1:22  26.95% perl5.30.0
20014 minotaur      1  52    0    19M  9724K RUN      0:44  25.00% perl5.30.0
19991 root          1  52    0    19M  9720K nanslp   0:44  23.97% perl5.30.0

O_o

> (via RT) <perlbug-followup@perl.org> 於 2019年7月26日 週五 上午6:50寫道:
> 
> > # New Ticket Created by
> > # Please include the string:  [perl #134305]
> > # in the subject line of all future correspondence about this issue.
> > # <URL: https://rt.perl.org/Ticket/Display.html?id=134305 >
> >
> >
> >
> > This is a bug report for perl from a.shikov@dtel-ix.net,
> > generated with the help of perlbug 1.41 running under perl 5.28.1.
> >
> >
> > -----------------------------------------------------------------
> > [Please describe your issue here]
> >
> > Hello!
> >
> > Recently we migrated some Perl scripts from old to another server.
> > Both servers - FreeBSD.
> > On the old server there were 11.2-RELEASE, Perl 5.24.
> > On the new server there are  12.0-RELEASE, Perl 5.28.2.
> >
> > On a new server we've noticed an intensive memory leak. After investigation
> > the trouble we've found that memory leak is caused by Sys::Syslog and can
> > be
> > reproduced by simple script:
> >
> > ---------------------------------------------------------------------------
> > #!/usr/local/bin/perl -w
> >
> > use strict;
> > use Sys::Syslog qw(:standard :macros);
> >
> > openlog('syslog-demo', 'ndelay', LOG_LOCAL2);
> >
> > my $c = 0;
> >
> > my @chars = ("A".."Z", "a".."z", "0".."9", "." , "-", "_" );
> >
> > while ($c++ <= 10000) {
> >         my $string = '';
> >         $string .= $chars[rand @chars] for 1..32;
> >         for (1..512) {
> >                 syslog(LOG_DEBUG, "DEBUG LOG %d/%d RANDOM STRING: %s", $c,
> > $_, $string );
> >         }
> >         sleep 1;
> > }
> > ---------------------------------------------------------------------------
> >
> > Add the line below to syslog.conf to save log to file:
> > local2.*                                        /var/log/yarpd.log
> >
> > Then start script and check out the log file with tail -f.
> >
> > Right after script is started 'top' shows that memory utilization starts
> > to grow rapidly:
> >
> > last pid: 44500;  load averages:  1.12,  1.19,  1.22          up
> > 21+05:23:33  15:21:28
> > 105 processes: 2 running, 103 sleeping
> > CPU: 43.2% user,  0.0% nice,  7.2% system,  0.4% interrupt, 49.2% idle
> > Mem: 427M Active, 189M Inact, 6452K Laundry, 448M Wired, 200M Buf, 892M
> > Free
> > Swap: 4096M Total, 222M Used, 3874M Free, 5% Inuse
> >
> >   PID USERNAME    THR PRI NICE   SIZE    RES STATE    C   TIME    WCPU
> > COMMAND
> > 45209 root          1  22    0    19M  8956K nanslp   0   0:00   1.11% perl
> > [...]
> >
> > 45209 root          1  22    0   196M   186M nanslp   1   0:04   4.50% perl
> > [...]
> >
> > 45209 root          1  22    0   266M   258M nanslp   0   0:05   4.64% perl
> > [...]
> >
> > 45209 root          1  22    0   541M   534M nanslp   0   0:11   2.29% perl
> >
> > Tested on physical machine, on virtual machine - the situation is the same
> > on both.
> > It just depends on the number of messages per second. 512 was enough for
> > VDS to
> > trigger the problem, on physical machine I tested it with 9000.
> >
> > The problem is not with Sys::Syslog module, because with Unix::Syslog the
> > problem is the same.
> >
> > After that I tested this script with three major versions of Perl: 5.26.3,
> > 5.28.1, 5.28.2 and 5.30.0
> > on my desktop with FreeBSD 12.0-RELEASE.
> >
> > Only with 5.26.3 the script does not eats memory. The rest three versions
> > have this
> > problem. It looks like in 5.28 there is a kind of bug causing memory leak
> > if there is
> > intensive I/O operation invoked, for example, by generating of big number
> > of syslog messages.
> >
> > Any suggestions are kindly appreciated. Thanks!
> >
> > [Please do not change anything below this line]
> > -----------------------------------------------------------------
> > ---
> > Flags:
> >     category=core
> >     severity=high
> > ---
> > Site configuration information for perl 5.28.1:
> >
> > Configured by mat at Thu Nov 29 21:03:47 EET 2018.
> >
> > Summary of my perl5 (revision 5 version 28 subversion 1) configuration:
> >
> >   Platform:
> >     osname=freebsd
> >     osvers=12.0-release-p1
> >     archname=amd64-freebsd-thread-multi
> >     uname='freebsd crete.itcons.net.ua 12.0-release-p1 freebsd
> > 12.0-release-p1 r342298 generic amd64 '
> >     config_args='-sde -Dprefix=/usr/local -Dlibperl=libperl.so.5.28.1
> > -Darchlib=/usr/local/lib/perl5/5.28/mach
> > -Dprivlib=/usr/local/lib/perl5/5.28
> > -Dman3dir=/usr/local/lib/perl5/5.28/perl/man/man3
> > -Dman1dir=/usr/local/lib/perl5/5.28/perl/man/man1
> > -Dsitearch=/usr/local/lib/perl5/site_perl/mach/5.28
> > -Dsitelib=/usr/local/lib/perl5/site_perl -Dscriptdir=/usr/local/bin
> > -Dsiteman3dir=/usr/local/lib/perl5/site_perl/man/man3
> > -Dsiteman1dir=/usr/local/lib/perl5/site_perl/man/man1 -Ui_malloc -Ui_iconv
> > -Uinstallusrbinperl -Dusenm=n -Dcc=cc -Duseshrplib -Dinc_version_list=none
> > -Dcf_by=mat -Dcf_email=mat@FreeBSD.org -Dcf_time=Thu Nov 29 21:03:47 EET
> > 2018 -Alddlflags=-L/usr/ports/lang/perl5.28/work/perl-5.28.1
> > -L/usr/local/lib/perl5/5.28/mach/CORE -lperl
> > -Dshrpldflags=$(LDDLFLAGS:N-L/usr/ports/lang/perl5.28/work/perl-5.28.1:N-L/usr/local/lib/perl5/5.28/mach/CORE:N-lperl)
> > -Wl,-soname,$(LIBPERL:R)
> > -Doptimize=-O2 -pipe  -fstack-protector -fno-strict-aliasing
> > -Ddefault_inc_excludes_dot=n -Uusedtrace -Ui_gdbm -Dusemultiplicity=y
> > -Dusemymalloc=n -Dusesitecustomize -Dusethreads=y'
> >     hint=recommended
> >     useposix=true
> >     d_sigaction=define
> >     useithreads=define
> >     usemultiplicity=define
> >     use64bitint=define
> >     use64bitall=define
> >     uselongdouble=undef
> >     usemymalloc=n
> >     default_inc_excludes_dot=undef
> >     bincompat5005=undef
> >   Compiler:
> >     cc='cc'
> >     ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing
> > -pipe -fstack-protector-strong -I/usr/local/include -D_FORTIFY_SOURCE=2'
> >     optimize='-O2 -pipe -fstack-protector -fno-strict-aliasing '
> >     cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing
> > -pipe -fstack-protector-strong -I/usr/local/include'
> >     ccversion=''
> >     gccversion='4.2.1 Compatible FreeBSD Clang 6.0.1
> > (tags/RELEASE_601/final 335540)'
> >     gccosandvers=''
> >     intsize=4
> >     longsize=8
> >     ptrsize=8
> >     doublesize=8
> >     byteorder=12345678
> >     doublekind=3
> >     d_longlong=define
> >     longlongsize=8
> >     d_longdbl=define
> >     longdblsize=16
> >     longdblkind=3
> >     ivtype='long'
> >     ivsize=8
> >     nvtype='double'
> >     nvsize=8
> >     Off_t='off_t'
> >     lseeksize=8
> >     alignbytes=8
> >     prototype=define
> >   Linker and Libraries:
> >     ld='cc'
> >     ldflags ='-pthread -Wl,-E  -fstack-protector-strong -L/usr/local/lib'
> >     libpth=/usr/lib /usr/local/lib /usr/lib/clang/6.0.1/lib /usr/lib
> >     libs=-lpthread -lm -lcrypt -lutil
> >     perllibs=-lpthread -lm -lcrypt -lutil
> >     libc=
> >     so=so
> >     useshrplib=true
> >     libperl=libperl.so.5.28.1
> >     gnulibc_version=''
> >   Dynamic Linking:
> >     dlsrc=dl_dlopen.xs
> >     dlext=so
> >     d_dlsymun=undef
> >     ccdlflags='  -Wl,-R/usr/local/lib/perl5/5.28/mach/CORE'
> >     cccdlflags='-DPIC -fPIC'
> >     lddlflags='-shared  -L/usr/local/lib/perl5/5.28/mach/CORE -lperl
> > -L/usr/local/lib -fstack-protector-strong'
> >
> >
> > ---
> > @INC for perl 5.28.1:
> >     /usr/local/lib/perl5/site_perl/mach/5.28
> >     /usr/local/lib/perl5/site_perl
> >     /usr/local/lib/perl5/5.28/mach
> >     /usr/local/lib/perl5/5.28
> >
> > ---
> > Environment for perl 5.28.1:
> >     HOME=/home/minotaur
> >     LANG=uk_UA.UTF-8
> >     LANGUAGE (unset)
> >     LC_COLLATE=C
> >     LD_LIBRARY_PATH (unset)
> >     LOGDIR (unset)
> >
> > PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin:/home/minotaur/bin
> >     PERLDOC_PAGER=less -sR
> >     PERL_BADLANG (unset)
> >     SHELL=/bin/tcsh
> >
> >

-- 
Alexander Shikov
Technical Staff, Digital Telecom IX
Tel.: +380 44 201 14 07
Mob.: +380 50 410 30 57
http://dtel-ix.net/

Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About