develooper Front page | perl.perl5.porters | Postings from July 2010

Re: Patch to make string-append on win32 100 times faster

Thread Previous | Thread Next
July 31, 2010 05:33
Re: Patch to make string-append on win32 100 times faster
Message ID:
From: Jim Cromie <>
Date: 31.07.2010 06:37
> On Fri, Jul 30, 2010 at 3:26 PM,  <> wrote:
>> From: Marvin Humphrey <>
>> Date: 30.07.2010 17:34
>>>> * Why the "+10"?  Why have a extra minimum at all?  Why "10" instead
>>>> of another number?  Given your examples, for a large string the 10 is
>>>> a trivial fraction of the total and for a small string, the efficiency
>>>> either doesn't matter or the string quickly grows into a large string
>>>> anyway.
>>> That's similar to my analysis.  I think it's more useful to round up to a
>>> multiple of the pointer size for small values.
>> If I understand correctly (which I'm not certain about), this is done anyway
>> right afterwards in
>> newlen = PERL_STRLEN_ROUNDUP(newlen);
>> Wolfram
> what happens if you presize the variable 1st ?
> then assign an empty string,
> then start appending the additions.
> this will avoid all the sv_grow calls,
> all the reallocs, and just use the one-time preallocated memory.
> this avoids all the mods to the heuristics youre proposing
> (modest tho they might be), and has no unintended effects
> on other benchmarkable use cases.
There are many cases when you don't know the the final size of your 
string. In that case you would need to invent a strategy similar to what 
I proposed. Take my original script:

$ps_text = qx( pdftops some_pdf_file.pdf - );

The PDF file is about 2MB. I know it will grow during PDF -> Postscript 
conversion. But it's hard to estimate how much it will grow. If I play 
safe, I will likely waste much more memory than some well designed 
perl-internal strategy.
Also, any perl compiled to use the internal malloc/realloc will "round 
up to a certain geometric progression anyway" (Ilya Zakharevich). If you 
look at Marvin Humphrey's post, you'll see that such a geometric growth 
strategy (although with different details concerning acceptable memory 
overhead etc.) is very common.


Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About