Front page | perl.perl5.porters |
Postings from October 2003
At 16:11 +0000 10/20/03, Jack Steadman (via RT) wrote:
>There are a couple of bug reports out there for shared array memory problems,
>but I've got a major memory leak that I think is related to a leak with shared
>hashes. It seems that after a reference is assigned to a shared
>hash it simply
>won't go out of scope. Try this:
>
>#!/usr/bin/perl
>
>use strict;
>use threads;
>use threads::shared;
>use Thread::Queue;
>
>my $top = &share({});
>$top->{mid} = &share({});
>
>my $i = 0;
>my $state = 1;
>while (1) {
>
> ## this can be any array/hash/object reference
> my $queue = new Thread::Queue;
>
> $top->{mid}->{$i} = $queue;
>
> $i++;
> if ($i > 10) {
> $i = 0;
> }
>}
Confirmed on Mac OS X as well.
>Running this will cause the program's memory usage to balloon to 100MB+ within
>seconds. Incidentally, the following code causes the program to
>hang at the point
>of the first delete while monopolizing the CPU:
>
>#!/usr/bin/perl
>
>use strict;
>use threads;
>use threads::shared;
>use Thread::Queue;
>
>my $top = &share({});
>$top->{mid} = &share({});
>
>my $i = 0;
>my $state = 1;
>while (1) {
>
> my $queue = new Thread::Queue;
>
> if ($state) {
> print "adding key $i\n";
> $top->{mid}->{$i} = $queue;
> } else {
> print "deleting key $i\n";
> delete $top->{mid}->{$i};
> }
>
> $i++;
> if ($i > 10) {
> $i = 0;
> $state = ($state) ? 0 : 1;
> }
>}
On Mac OS X this segfaults after saying "deleting key 0".
I don't know how performance bound you are, but both programs work ok
when you used forks.pm (using an unchanged program, by specifying
"-Mforks -Mforks::shared" on the command line). So this might be a
temporary alternative that would allow you to keep your source
unchanged and not have to worry about having to restart because it is
eating memory.
Of course, forks.pm is slower than threads.pm as the communication
between threads is handled by a TCP/IP connection, involving
Freeze/Thaw operations, etc. So that means more CPU usage in
general and more latency in communication between threads. But of
course, you save a lot of memory because the OS's COW can do its work.
Liz