[ID 20000427.006] Mis-tokenizing of some blocks when run throughintrospective loader

This is a bug report for perl from mdc@desktop.com,
generated with the help of perlbug 1.28 running under perl v5.6.0.


-----------------------------------------------------------------
[Please enter your report here]

Problem: Blocks whose last expression is a non-semicolon-terminated
list value on a separate line from the closing curly brace are
improperly rejected when run through an introspective-loader/source-
generator team.

I'm totally stumped by this one--at first I thought something in the
lexer was behaving differently with source filters in effect, but
testing with Filter::cpp (which doesn't alter the test package) seems
to work fine.

(A quick check with Deparse confirmed my belief that 'sub foo { (0) }'
is precisely equivalent to 'sub foo { 0; }', which is as it should
be.)

Here's the testcase (derived from Ken Fox's loader demo script):

#!/usr/bin/perl -w

BEGIN {
  my $Internal = [ 'package Internal;',
		   '',
		   'sub foo {',
		   '  (0)',
		   '}',
		   '',
		   'sub testing {',
		   '  print "Internal.pm loaded ok\n";',
		   '}',
		   '',
		   '1;' ];
  
  my @External = @$Internal;
  foreach (@External) { s/Internal/External/g; }
  open EXT, ">/tmp/External.pm";
  print EXT join("\n", @External), "\n";
  close EXT;

  sub generator {
      my($maxlen, $state) = @_;
      $_ = shift @{$state->{-text}};
      if (defined $_) {
          $_ .= "\n";
          print "[generated] line $state->{-line}: $_";
          ++$state->{-line};
          return 1;
      }
      return 0;
  }

  sub introspective_loader {
    my $self = shift;
    return(\&generator, { -text => $Internal,
			  -line => 1 });
  }

  @INC = ( '/tmp', \&introspective_loader );
}

require External;
External::testing();

require Internal;
Internal::testing();

__END__


[Please do not change anything below this line]
-----------------------------------------------------------------
---
Flags:
    category=core
    severity=medium
---
Site configuration information for perl v5.6.0:

Configured by mdc at Thu Apr 27 15:11:52 PDT 2000.

Summary of my perl5 (revision 5.0 version 6 subversion 0) configuration:
  Platform:
    osname=linux, osvers=2.2.5-15, archname=i686-linux
    uname='linux mcontois-unix.jumpdata.com 2.2.5-15 #1 mon apr 19 23:00:46 edt 1999 i686 unknown '
    config_args='-d'
    hint=recommended, useposix=true, d_sigaction=define
    usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef
    useperlio=undef d_sfio=undef uselargefiles=define 
    use64bitint=undef use64bitall=undef uselongdouble=undef usesocks=undef
  Compiler:
    cc='cc', optimize='-O2', gccversion=egcs-2.91.66 19990314/Linux (egcs-1.1.2 release)
    cppflags='-fno-strict-aliasing -I/usr/local/include'
    ccflags ='-fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
    stdchar='char', d_stdstdio=define, usevfork=false
    intsize=4, longsize=4, ptrsize=4, doublesize=8
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
    ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=4, usemymalloc=n, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -L/usr/local/lib'
    libpth=/usr/local/lib /lib /usr/lib
    libs=-lnsl -lndbm -lgdbm -ldb -ldl -lm -lc -lposix -lcrypt
    libc=/lib/libc-2.1.1.so, so=so, useshrplib=false, libperl=libperl.a
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic'
    cccdlflags='-fpic', lddlflags='-shared -L/usr/local/lib'

Locally applied patches:
    

---
@INC for perl v5.6.0:
    /usr/local/lib/perl5/5.6.0/i686-linux
    /usr/local/lib/perl5/5.6.0
    /usr/local/lib/perl5/site_perl/5.6.0/i686-linux
    /usr/local/lib/perl5/site_perl/5.6.0
    /usr/local/lib/perl5/site_perl
    .

---
Environment for perl v5.6.0:
    HOME=/u/mdc
    LANG (unset)
    LANGUAGE (unset)
    LD_LIBRARY_PATH=/dt/vendor/lib:/dt/server/lib
    LOGDIR (unset)
    PATH=/usr/bin:/u/mdc/bin:/dt/vendor/bin:/dt/server/bin:/usr/bin:/bin:/usr/local/bin:/usr/X11R6/bin:/usr/X11R6/bin
    PERL_BADLANG (unset)
    SHELL=/bin/bash

• [ID 20000427.006] Mis-tokenizing of some blocks when run throughintrospective loader by Mark Contois