develooper Front page | perl.perl5.porters | Postings from July 2019

Re: [perl #134305] Memory Leak in perl 5.28 and 5.30

Thread Previous | Thread Next
From:
=?UTF-8?B?6YOt5qiC6IGw?=
Date:
July 26, 2019 03:50
Subject:
Re: [perl #134305] Memory Leak in perl 5.28 and 5.30
Message ID:
CAD9X63FY+ikEVGBR_TWwneJ+QZYEjdbC_D67inZKTfbr+Bx0HQ@mail.gmail.com
Hello,

 Interesting case, looks like it is a problem with locale implementation. I
am
looking into locale.c to see if I can find anything. Can you try to run your
perl script with LC_ALL=C and see if the memory leak problem persists?
Thanks a lot.

Best regards,
Baggio

(via RT) <perlbug-followup@perl.org> 於 2019年7月26日 週五 上午6:50寫道:

> # New Ticket Created by
> # Please include the string:  [perl #134305]
> # in the subject line of all future correspondence about this issue.
> # <URL: https://rt.perl.org/Ticket/Display.html?id=134305 >
>
>
>
> This is a bug report for perl from a.shikov@dtel-ix.net,
> generated with the help of perlbug 1.41 running under perl 5.28.1.
>
>
> -----------------------------------------------------------------
> [Please describe your issue here]
>
> Hello!
>
> Recently we migrated some Perl scripts from old to another server.
> Both servers - FreeBSD.
> On the old server there were 11.2-RELEASE, Perl 5.24.
> On the new server there are  12.0-RELEASE, Perl 5.28.2.
>
> On a new server we've noticed an intensive memory leak. After investigation
> the trouble we've found that memory leak is caused by Sys::Syslog and can
> be
> reproduced by simple script:
>
> ---------------------------------------------------------------------------
> #!/usr/local/bin/perl -w
>
> use strict;
> use Sys::Syslog qw(:standard :macros);
>
> openlog('syslog-demo', 'ndelay', LOG_LOCAL2);
>
> my $c = 0;
>
> my @chars = ("A".."Z", "a".."z", "0".."9", "." , "-", "_" );
>
> while ($c++ <= 10000) {
>         my $string = '';
>         $string .= $chars[rand @chars] for 1..32;
>         for (1..512) {
>                 syslog(LOG_DEBUG, "DEBUG LOG %d/%d RANDOM STRING: %s", $c,
> $_, $string );
>         }
>         sleep 1;
> }
> ---------------------------------------------------------------------------
>
> Add the line below to syslog.conf to save log to file:
> local2.*                                        /var/log/yarpd.log
>
> Then start script and check out the log file with tail -f.
>
> Right after script is started 'top' shows that memory utilization starts
> to grow rapidly:
>
> last pid: 44500;  load averages:  1.12,  1.19,  1.22          up
> 21+05:23:33  15:21:28
> 105 processes: 2 running, 103 sleeping
> CPU: 43.2% user,  0.0% nice,  7.2% system,  0.4% interrupt, 49.2% idle
> Mem: 427M Active, 189M Inact, 6452K Laundry, 448M Wired, 200M Buf, 892M
> Free
> Swap: 4096M Total, 222M Used, 3874M Free, 5% Inuse
>
>   PID USERNAME    THR PRI NICE   SIZE    RES STATE    C   TIME    WCPU
> COMMAND
> 45209 root          1  22    0    19M  8956K nanslp   0   0:00   1.11% perl
> [...]
>
> 45209 root          1  22    0   196M   186M nanslp   1   0:04   4.50% perl
> [...]
>
> 45209 root          1  22    0   266M   258M nanslp   0   0:05   4.64% perl
> [...]
>
> 45209 root          1  22    0   541M   534M nanslp   0   0:11   2.29% perl
>
> Tested on physical machine, on virtual machine - the situation is the same
> on both.
> It just depends on the number of messages per second. 512 was enough for
> VDS to
> trigger the problem, on physical machine I tested it with 9000.
>
> The problem is not with Sys::Syslog module, because with Unix::Syslog the
> problem is the same.
>
> After that I tested this script with three major versions of Perl: 5.26.3,
> 5.28.1, 5.28.2 and 5.30.0
> on my desktop with FreeBSD 12.0-RELEASE.
>
> Only with 5.26.3 the script does not eats memory. The rest three versions
> have this
> problem. It looks like in 5.28 there is a kind of bug causing memory leak
> if there is
> intensive I/O operation invoked, for example, by generating of big number
> of syslog messages.
>
> Any suggestions are kindly appreciated. Thanks!
>
> [Please do not change anything below this line]
> -----------------------------------------------------------------
> ---
> Flags:
>     category=core
>     severity=high
> ---
> Site configuration information for perl 5.28.1:
>
> Configured by mat at Thu Nov 29 21:03:47 EET 2018.
>
> Summary of my perl5 (revision 5 version 28 subversion 1) configuration:
>
>   Platform:
>     osname=freebsd
>     osvers=12.0-release-p1
>     archname=amd64-freebsd-thread-multi
>     uname='freebsd crete.itcons.net.ua 12.0-release-p1 freebsd
> 12.0-release-p1 r342298 generic amd64 '
>     config_args='-sde -Dprefix=/usr/local -Dlibperl=libperl.so.5.28.1
> -Darchlib=/usr/local/lib/perl5/5.28/mach
> -Dprivlib=/usr/local/lib/perl5/5.28
> -Dman3dir=/usr/local/lib/perl5/5.28/perl/man/man3
> -Dman1dir=/usr/local/lib/perl5/5.28/perl/man/man1
> -Dsitearch=/usr/local/lib/perl5/site_perl/mach/5.28
> -Dsitelib=/usr/local/lib/perl5/site_perl -Dscriptdir=/usr/local/bin
> -Dsiteman3dir=/usr/local/lib/perl5/site_perl/man/man3
> -Dsiteman1dir=/usr/local/lib/perl5/site_perl/man/man1 -Ui_malloc -Ui_iconv
> -Uinstallusrbinperl -Dusenm=n -Dcc=cc -Duseshrplib -Dinc_version_list=none
> -Dcf_by=mat -Dcf_email=mat@FreeBSD.org -Dcf_time=Thu Nov 29 21:03:47 EET
> 2018 -Alddlflags=-L/usr/ports/lang/perl5.28/work/perl-5.28.1
> -L/usr/local/lib/perl5/5.28/mach/CORE -lperl
> -Dshrpldflags=$(LDDLFLAGS:N-L/usr/ports/lang/perl5.28/work/perl-5.28.1:N-L/usr/local/lib/perl5/5.28/mach/CORE:N-lperl)
> -Wl,-soname,$(LIBPERL:R)
> -Doptimize=-O2 -pipe  -fstack-protector -fno-strict-aliasing
> -Ddefault_inc_excludes_dot=n -Uusedtrace -Ui_gdbm -Dusemultiplicity=y
> -Dusemymalloc=n -Dusesitecustomize -Dusethreads=y'
>     hint=recommended
>     useposix=true
>     d_sigaction=define
>     useithreads=define
>     usemultiplicity=define
>     use64bitint=define
>     use64bitall=define
>     uselongdouble=undef
>     usemymalloc=n
>     default_inc_excludes_dot=undef
>     bincompat5005=undef
>   Compiler:
>     cc='cc'
>     ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing
> -pipe -fstack-protector-strong -I/usr/local/include -D_FORTIFY_SOURCE=2'
>     optimize='-O2 -pipe -fstack-protector -fno-strict-aliasing '
>     cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing
> -pipe -fstack-protector-strong -I/usr/local/include'
>     ccversion=''
>     gccversion='4.2.1 Compatible FreeBSD Clang 6.0.1
> (tags/RELEASE_601/final 335540)'
>     gccosandvers=''
>     intsize=4
>     longsize=8
>     ptrsize=8
>     doublesize=8
>     byteorder=12345678
>     doublekind=3
>     d_longlong=define
>     longlongsize=8
>     d_longdbl=define
>     longdblsize=16
>     longdblkind=3
>     ivtype='long'
>     ivsize=8
>     nvtype='double'
>     nvsize=8
>     Off_t='off_t'
>     lseeksize=8
>     alignbytes=8
>     prototype=define
>   Linker and Libraries:
>     ld='cc'
>     ldflags ='-pthread -Wl,-E  -fstack-protector-strong -L/usr/local/lib'
>     libpth=/usr/lib /usr/local/lib /usr/lib/clang/6.0.1/lib /usr/lib
>     libs=-lpthread -lm -lcrypt -lutil
>     perllibs=-lpthread -lm -lcrypt -lutil
>     libc=
>     so=so
>     useshrplib=true
>     libperl=libperl.so.5.28.1
>     gnulibc_version=''
>   Dynamic Linking:
>     dlsrc=dl_dlopen.xs
>     dlext=so
>     d_dlsymun=undef
>     ccdlflags='  -Wl,-R/usr/local/lib/perl5/5.28/mach/CORE'
>     cccdlflags='-DPIC -fPIC'
>     lddlflags='-shared  -L/usr/local/lib/perl5/5.28/mach/CORE -lperl
> -L/usr/local/lib -fstack-protector-strong'
>
>
> ---
> @INC for perl 5.28.1:
>     /usr/local/lib/perl5/site_perl/mach/5.28
>     /usr/local/lib/perl5/site_perl
>     /usr/local/lib/perl5/5.28/mach
>     /usr/local/lib/perl5/5.28
>
> ---
> Environment for perl 5.28.1:
>     HOME=/home/minotaur
>     LANG=uk_UA.UTF-8
>     LANGUAGE (unset)
>     LC_COLLATE=C
>     LD_LIBRARY_PATH (unset)
>     LOGDIR (unset)
>
> PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin:/home/minotaur/bin
>     PERLDOC_PAGER=less -sR
>     PERL_BADLANG (unset)
>     SHELL=/bin/tcsh
>
>

Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About