Faster Perl_keyword

As of change 23876 Perl_keyword, called by the tokeniser, should be about 20%
faster. (Not that I expect that this will show up measurably by timing perl as
a whole). ( http://public.activestate.com/cgi-bin/perlbrowse?patch=23876 )

I hacked Perl_keyword to record everything it ever called:

I32
Perl_keyword(pTHX_ register char *d, I32 len)
{
  if (len < 256)
  {
    int fd = open (INSTALL_PREFIX_EXP "/keywordlog", O_CREAT|O_WRONLY|O_APPEND, S_IRUSR|S_IWUSR);
    if (fd) {
      unsigned char buffer[257];
      *buffer = len;
      memcpy(buffer+1,d,len);
      buffer[len+1] = '\0';
      write(fd, buffer, len+2);
      close(fd);
    }
  }
  else
  {
    int fd = open (INSTALL_PREFIX_EXP "/longkeywordlog", O_CREAT|O_WRONLY|O_APPEND, S_IRUSR|S_IWUSR);
    if (fd) {
      write(fd, d, len+1);
      close(fd);
    }
  }

then used this data to feed into a subclass of ExtUtils::Constant to generate
a new lookup table weighted to match the more common keywords earlier. (see
http://public.activestate.com/cgi-bin/perlbrowse?file=perl_keyword.pl&blame=1 )

Benchmarking was done with the attached program linked against a file
containing each Perl_keyword (with the warning call at 'elseif' converted to
a call to the dummy function stuff() to stop it being optimised away).

So now you can log your own preferred set of perl keywords, and regenerate
toke.c to be fractionally faster for whatever perl you used. :-)
Although I doubt that this will make a difference, as I'm assuming that my
135M of input data consisting of a lot of CPAN was typical enough.

Nicholas Clark

• Faster Perl_keyword by Nicholas Clark
• Re: Faster Perl_keyword by Marcus Holland-Moritz
• Re: Faster Perl_keyword by Nicholas Clark
• Re: Faster Perl_keyword by Marcus Holland-Moritz