Front page | perl.perl5.porters |
Postings from August 2010
On Aug 17, 2010, at 10:09 AM, Wolfram Humann wrote: > -------- Original Message -------- > Subject: Re: Patch to make string-append on win32 100 times faster > From: Jan Dubois <jand@activestate.com> > To: 'David Golden' <xdaveg@gmail.com>, 'Reini Urban' <rurban@x-ray.at> > Date: 17.08.2010 01:09 >> On Mon, 16 Aug 2010, David Golden wrote: >> >>> I think I'm with Reini on this one. I'd rather keep it Windows-only until a >>> performance problem is documented elsewhere. >>> >> >> It's been documented that the patch improves performance on FreeBSD by a >> *factor* of 160 for one particular test case, as well as a factor of 4 >> on Linux. In the meantime Wolfram has constructed another testcase where >> the patch improves performance on Linux by a factor of 45 (my system) to >> 230 (Wolfram's system). >> >> Granted, these are worst-case samples, but at least the first test >> program was derived from a real-world application: appending lots of >> small strings to an ever-growing large buffer. > I think that even my worst-case Linux test case is not totally out-of-the-world. It occurs whenever a large number of strings grows simultaneously by small increments. One application that comes to mind is pivoting (exchanging rows and columns of) a large text-based table: loop over the lines, splitting each row and appending the chunks to the respective column's string; finally print the column strings as new rows when the complete table is read. FWIW, I have several pieces of important business logic that add packed integers to hash values for millions of iterations. Seems like a worst case to me. Having said that, I've been moving these cases to use a temporary file, basically using the file system as a hash. This in the end, proved to work a lot faster and scaled much better, way beyond the physical RAM that was available. LizThread Previous | Thread Next