develooper Front page | perl.perl5.porters | Postings from February 2018

[perl #132892] Possibly memory leak when matching strings usingpre-compiled regexes stored in hash key (perl >= v5.26)

Thread Next
From:
perlbug-followup
Date:
February 22, 2018 04:39
Subject:
[perl #132892] Possibly memory leak when matching strings usingpre-compiled regexes stored in hash key (perl >= v5.26)
Message ID:
rt-4.0.24-2770-1519228160-1065.132892-75-0@perl.org
# New Ticket Created by   
# Please include the string:  [perl #132892]
# in the subject line of all future correspondence about this issue. 
# <URL: https://rt.perl.org/Ticket/Display.html?id=132892 >


Hello guys,

I have a long running perl script which allows the user configure the
program behavior with a hash that got pre-compiled regexps
as keys and the corresponding callback code as the value like below.
Recently I have upgraded to perl v5.26 and the memory
usage grows since then, so I have fall back to perl v5.24.3 as a temporary
solution. What I observed before fallback is that the
memory grows when I try to iterates the hash keys and matches them against
some text.

%action = (

  qr/^regex here/ => sub { print "match" },

);


I have written a small test for this problem, you can find it in the
attached perlbug.tar.xz. 20180221-mem-leak-poc.pl is the prof of
concept code. It will output the platform and perl version as the first
line, and make use of the ps command to retrieve the vsz and
rss of the perl interpreter before each iteration. The test result of
different platform are stored in the rslt/ sub-directory for your reference.
According to the results, with perl <v5.26, the size in memory remain
unchanged; While the memory usage keeps increasing when the
script is run with perl >=v5.26. I have limited the number of iteration,
the memory usage grows further if you increases the number of iteration.

|-- 20180221-mem-leak-poc.pl
`-- rslt
    |-- centos6-p5.10.1.log
    |-- centos6-p5.24.3.log
    |-- centos6-p5.26.1.log
    |-- centos6-p5.27.8.log
    |-- dfbsd-p5.24.3.log
    |-- dfbsd-p5.26.1.log
    |-- freebsd-p5.24.3.log
    `-- freebsd-p5.26.1.log

According to my empirical result, this problem occurs when using perl newer
than v5.26, the first line of perl -v of the affected perl
interpreters below.

> This is perl 5, version 26, subversion 1 (v5.26.1) built for amd64-freebsd
> This is perl 5, version 27, subversion 8 (v5.27.8) built for x86_64-linux
> This is perl 5, version 26, subversion 1 (v5.26.1) built for x86_64-linux

This is perl 5, version 26, subversion 1 (v5.26.1) built for
> x86_64-dragonfly


I have traced the problem a little bit and I *guess* the problem is in the
function S_concat_pat of regcomp.c and the new SVs
copied out are immortal, so every time my program iterate through the keys
of hash to match against things, it creates more
copies of the regexes. I can try to dig deeper when I am free to build the
debug versions of perl, but that would be great if this
case can be taken care by the professionals here. Thanks a lot.
​

​
Best regards,
Baggio

Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About