[perl #45667] m/[#]/x treated inconsistently

On Fri Sep 28 08:07:29 2007, nicholas wrote:
> On Mon, Sep 24, 2007 at 01:52:17PM +0100, Dave Mitchell wrote:
> > On Mon, Sep 24, 2007 at 10:05:13AM +0200, demerphq wrote:
> > > On 9/23/07, Dave Mitchell <davem@iabyn.com> wrote:
> > > > On Sat, Sep 22, 2007 at 08:05:59AM -0700, Chris Dolan wrote:
> > > > > These should output the same value, but instead the first one
> emits
> > > > > "f" and the second one undef.
> > > > >
> > > > >        my $qr = qr/(f)/;
> > > > >        print "#f" =~ m/[#](f)/x;
> > > > >        print "#f" =~ m/[#]$qr/x;
> > > > >
> > > > > This is more revealing:
> > > > >
> > > > >        print qr/[#]$qr/x;
> > > > >
> > > > > yields:
> > > > >
> > > > >        (?x-ism:[#]$qr
> > > > >        )
> > > > >
> > > > > So, it appears that [#] is disabling interpolation for
> variables that
> > > > > follow, but then still treating the content as regexp.
> Clearly, it's
> > > > > not *just* treating it as a comment, or the remaining qr/[/
> would be
> > > > > a syntax error.
> > > >
> > > > Hmm, looks like the tokenizer isn't smart enough to know to
> ignore #'s
> > > > when within a character class. S_scan_const() needs to be made
> even more
> > > > scary than it already is. Probably a post 5.10.0 job.
> > >
> > > It is not clear to me that this is a bug actually. The
> documentation
> > > states that # must be escaped in an /x pattern afair.
> >
> > But it also says that # isn't special when in a character class:
> >
> >     $ ./perl -Dr -e'/[#]a
> >     b/x'
> >     Compiling REx "[#]a%nb"
> >     rarest char # at 0
> >     Final program:
> >        1: EXACT <#ab> (5)
> >        5: END (0)
> >
> >     $ perl -le'print "ok" if "#" =~ /[#]/x'
> >     ok
> >     $
> 
> Do we have an agreed TODO test? (or tests)

Agreed?  Probably not.  I’ve been thinking about this issue lately,
specifically in conjunction with (but not limited to) (?[...]) regexp sets.

/x mode allows variables to be commented out:

$ ./perl -Ilib -le '$x="\n"; print "a" =~ / a # $x blah blah /x'
1

But not always:

$ ./perl -Ilib -Xle '$# = "#"; $x="\n"; print / [a] $# $x blah blah /x
|| nope'
nope

Regexp sets are supposed to imply /x, but variables are not commented out:

$ ./perl -Xe '$x="\n"; /(?[ [a] # $x blah blah' -e'])/'
Unexpected character in regex; marked by <-- HERE in m/(?[ [a] # 
 b <-- HERE lah blah
])/ at -e line 1.

Remove the variable, and it is fine:

$  ./perl -Xe '$x="\n"; /(?[ [a] # blah blah' -e '])/'
$

Or add /x; that is also fine:

$ ./perl -Xe '$x="\n"; /(?[ [a] # $x blah blah' -e'])/x'
$

So this is a bigger can of worms than anyone thought.  Perhaps the real
fix is for the regexp engine to call back to the parser when it sees
what it thinks is a variable.  Dave Mitchell is not going to like what I
am suggesting. :-)

Also, I wonder whether this is at all fixable in the end.  I suspect
however hard we try we will just end up fixing a few cases and breaking
a few others, because the rules are ultimately contradictory.

-- 

Father Chrysostomos


---
via perlbug:  queue: perl5 status: open
https://rt.perl.org:443/rt3/Ticket/Display.html?id=45667

• [perl #45667] m/[#]/x treated inconsistently by Father Chrysostomos via RT
• Re: [perl #45667] m/[#]/x treated inconsistently by Dave Mitchell