develooper Front page | perl.perl5.porters | Postings from July 2019

[perl #134305] Memory Leak in perl 5.28 and 5.30

Thread Previous | Thread Next
From:
perlbug-followup
Date:
July 25, 2019 22:50
Subject:
[perl #134305] Memory Leak in perl 5.28 and 5.30
Message ID:
rt-4.0.24-32080-1563902566-392.134305-75-0@perl.org
# New Ticket Created by   
# Please include the string:  [perl #134305]
# in the subject line of all future correspondence about this issue. 
# <URL: https://rt.perl.org/Ticket/Display.html?id=134305 >



This is a bug report for perl from a.shikov@dtel-ix.net,
generated with the help of perlbug 1.41 running under perl 5.28.1.


-----------------------------------------------------------------
[Please describe your issue here]

Hello!

Recently we migrated some Perl scripts from old to another server.
Both servers - FreeBSD.
On the old server there were 11.2-RELEASE, Perl 5.24.
On the new server there are  12.0-RELEASE, Perl 5.28.2.

On a new server we've noticed an intensive memory leak. After investigation
the trouble we've found that memory leak is caused by Sys::Syslog and can be
reproduced by simple script:

---------------------------------------------------------------------------
#!/usr/local/bin/perl -w

use strict;
use Sys::Syslog qw(:standard :macros);

openlog('syslog-demo', 'ndelay', LOG_LOCAL2);

my $c = 0;

my @chars = ("A".."Z", "a".."z", "0".."9", "." , "-", "_" );

while ($c++ <= 10000) {
	my $string = '';
	$string .= $chars[rand @chars] for 1..32;
	for (1..512) {
		syslog(LOG_DEBUG, "DEBUG LOG %d/%d RANDOM STRING: %s", $c, $_, $string );
	}
	sleep 1;
}
---------------------------------------------------------------------------

Add the line below to syslog.conf to save log to file:
local2.*					/var/log/yarpd.log

Then start script and check out the log file with tail -f.

Right after script is started 'top' shows that memory utilization starts to grow rapidly:

last pid: 44500;  load averages:  1.12,  1.19,	1.22	      up 21+05:23:33  15:21:28
105 processes: 2 running, 103 sleeping
CPU: 43.2% user,  0.0% nice,  7.2% system,  0.4% interrupt, 49.2% idle
Mem: 427M Active, 189M Inact, 6452K Laundry, 448M Wired, 200M Buf, 892M Free
Swap: 4096M Total, 222M Used, 3874M Free, 5% Inuse

  PID USERNAME	  THR PRI NICE	 SIZE	 RES STATE    C   TIME	  WCPU COMMAND
45209 root	    1  22    0	  19M  8956K nanslp   0   0:00	 1.11% perl
[...]

45209 root	    1  22    0	 196M	186M nanslp   1   0:04	 4.50% perl
[...]

45209 root	    1  22    0	 266M	258M nanslp   0   0:05	 4.64% perl
[...]

45209 root	    1  22    0	 541M	534M nanslp   0   0:11	 2.29% perl

Tested on physical machine, on virtual machine - the situation is the same on both.
It just depends on the number of messages per second. 512 was enough for VDS to
trigger the problem, on physical machine I tested it with 9000.

The problem is not with Sys::Syslog module, because with Unix::Syslog the problem is the same.

After that I tested this script with three major versions of Perl: 5.26.3, 5.28.1, 5.28.2 and 5.30.0
on my desktop with FreeBSD 12.0-RELEASE.

Only with 5.26.3 the script does not eats memory. The rest three versions have this
problem. It looks like in 5.28 there is a kind of bug causing memory leak if there is 
intensive I/O operation invoked, for example, by generating of big number of syslog messages.

Any suggestions are kindly appreciated. Thanks!

[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
    category=core
    severity=high
---
Site configuration information for perl 5.28.1:

Configured by mat at Thu Nov 29 21:03:47 EET 2018.

Summary of my perl5 (revision 5 version 28 subversion 1) configuration:
   
  Platform:
    osname=freebsd
    osvers=12.0-release-p1
    archname=amd64-freebsd-thread-multi
    uname='freebsd crete.itcons.net.ua 12.0-release-p1 freebsd 12.0-release-p1 r342298 generic amd64 '
    config_args='-sde -Dprefix=/usr/local -Dlibperl=libperl.so.5.28.1 -Darchlib=/usr/local/lib/perl5/5.28/mach -Dprivlib=/usr/local/lib/perl5/5.28 -Dman3dir=/usr/local/lib/perl5/5.28/perl/man/man3 -Dman1dir=/usr/local/lib/perl5/5.28/perl/man/man1 -Dsitearch=/usr/local/lib/perl5/site_perl/mach/5.28 -Dsitelib=/usr/local/lib/perl5/site_perl -Dscriptdir=/usr/local/bin -Dsiteman3dir=/usr/local/lib/perl5/site_perl/man/man3 -Dsiteman1dir=/usr/local/lib/perl5/site_perl/man/man1 -Ui_malloc -Ui_iconv -Uinstallusrbinperl -Dusenm=n -Dcc=cc -Duseshrplib -Dinc_version_list=none -Dcf_by=mat -Dcf_email=mat@FreeBSD.org -Dcf_time=Thu Nov 29 21:03:47 EET 2018 -Alddlflags=-L/usr/ports/lang/perl5.28/work/perl-5.28.1 -L/usr/local/lib/perl5/5.28/mach/CORE -lperl -Dshrpldflags=$(LDDLFLAGS:N-L/usr/ports/lang/perl5.28/work/perl-5.28.1:N-L/usr/local/lib/perl5/5.28/mach/CORE:N-lperl) -Wl,-soname,$(LIBPERL:R)
-Doptimize=-O2 -pipe  -fstack-protector -fno-strict-aliasing  -Ddefault_inc_excludes_dot=n -Uusedtrace -Ui_gdbm -Dusemultiplicity=y -Dusemymalloc=n -Dusesitecustomize -Dusethreads=y'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=undef
    bincompat5005=undef
  Compiler:
    cc='cc'
    ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_FORTIFY_SOURCE=2'
    optimize='-O2 -pipe -fstack-protector -fno-strict-aliasing '
    cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
    ccversion=''
    gccversion='4.2.1 Compatible FreeBSD Clang 6.0.1 (tags/RELEASE_601/final 335540)'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='cc'
    ldflags ='-pthread -Wl,-E  -fstack-protector-strong -L/usr/local/lib'
    libpth=/usr/lib /usr/local/lib /usr/lib/clang/6.0.1/lib /usr/lib
    libs=-lpthread -lm -lcrypt -lutil
    perllibs=-lpthread -lm -lcrypt -lutil
    libc=
    so=so
    useshrplib=true
    libperl=libperl.so.5.28.1
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='  -Wl,-R/usr/local/lib/perl5/5.28/mach/CORE'
    cccdlflags='-DPIC -fPIC'
    lddlflags='-shared	-L/usr/local/lib/perl5/5.28/mach/CORE -lperl -L/usr/local/lib -fstack-protector-strong'


---
@INC for perl 5.28.1:
    /usr/local/lib/perl5/site_perl/mach/5.28
    /usr/local/lib/perl5/site_perl
    /usr/local/lib/perl5/5.28/mach
    /usr/local/lib/perl5/5.28

---
Environment for perl 5.28.1:
    HOME=/home/minotaur
    LANG=uk_UA.UTF-8
    LANGUAGE (unset)
    LC_COLLATE=C
    LD_LIBRARY_PATH (unset)
    LOGDIR (unset)
    PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin:/home/minotaur/bin
    PERLDOC_PAGER=less -sR
    PERL_BADLANG (unset)
    SHELL=/bin/tcsh


Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About