perl.perl5.changes

[perl.git] branch blead, updated. v5.19.0-142-gdfe12d6 by James Keenan

2013-05-23T01:40:49Z

From: James Keenan In perl.git, the branch blead has been updated

- Log -----------------------------------------------------------------
commit dfe12d64f471217d8004892c05035d320449ae24
Author: James E Keenan
Date: Thu May 23 03:26:55 2013 +0200

Provide alternate email address for jkeenan.
-----------------------------------------------------------------------

Summary of changes:
Porting/checkAUTHORS.pl | 3 ++-
1 files changed, 2 insertions(+), 1 deletions(-)

diff --git a/Porting/checkAUTHORS.pl b/Porting/checkAUTHORS.pl
index 237a307..c1e108b 100755
--- a/Porting/checkAUTHORS.pl
+++ b/Porting/checkAUTHORS.pl
@@ -659,6 +659,8 @@ jhannah\100mutationgrid.com jay\100jays.net
+ jhannah\100omnihotels.com
jidanni\100jidanni.org jidanni\100hoffa.dreamhost.com
jjore\100cpan.org twists\100gmail.com
+jkeenan\100cpan.org jkeen\100verizon.net
++ jkeenan\100dromedary-001.ams6.corp.booking.com
jns\100integration-house.com jns\100gellyfish.com
+ gellyfish\100gellyfish.com
john\100atlantech.com john\100titanic.atlantech.com
@@ -919,4 +921,3 @@ wolfgang.laun\100alcatel.at wolfgang.laun\100chello.at
wolfsage\100gmail.com mhorsfall\100darmstadtium.(none)
yath\100yath.de yath-perlbug\100yath.de

-jkeen@verizon.net jkeenan@cpan.org

--
Perl5 Master Repository

[perl.git] branch blead, updated. v5.19.0-141-g3b89f54 by James Keenan

2013-05-23T01:08:42Z

From: James Keenan In perl.git, the branch blead has been updated

- Log -----------------------------------------------------------------
commit 3b89f54e7729300a9c982327779d9077e60d586d
Author: James Keenan
Date: Thu May 23 03:06:54 2013 +0200

Correct known_pod_issues.dat.
-----------------------------------------------------------------------

Summary of changes:
t/porting/known_pod_issues.dat | 5 -----
1 files changed, 0 insertions(+), 5 deletions(-)

diff --git a/t/porting/known_pod_issues.dat b/t/porting/known_pod_issues.dat
index ffef833..b175a81 100644
--- a/t/porting/known_pod_issues.dat
+++ b/t/porting/known_pod_issues.dat
@@ -296,7 +296,6 @@ pod/perltru64.pod ? Should you be using F<...> or maybe L<...> instead of 1
pod/perltru64.pod Verbatim line length including indents exceeds 79 by 4
pod/perlunifaq.pod empty section in previous paragraph 1
pod/perluniprops.pod =item type mismatch 6
-pod/perlutil.pod Apparent internal link is missing its forward slash 2
pod/perlvms.pod ? Should you be using F<...> or maybe L<...> instead of 1
pod/perlvms.pod Verbatim line length including indents exceeds 79 by 2
pod/perlwin32.pod Verbatim line length including indents exceeds 79 by 12
@@ -311,10 +310,6 @@ porting/todo.pod Verbatim line length including indents exceeds 79 by 7
symbian/perlutil.pod Verbatim line length including indents exceeds 79 by 4
utils/c2ph Verbatim line length including indents exceeds 79 by 44
utils/pod2html Pod NAME already used 1
-utils/prove Apparent broken link 2
-utils/prove Pod NAME already used 1
-utils/prove There is more than one target 1
-utils/prove Verbatim line length including indents exceeds 79 by 3
vms/ext/filespec.pm Verbatim line length including indents exceeds 79 by 1
x2p/a2p.pod empty section in previous paragraph 2
lib/benchmark.pm Verbatim line length including indents exceeds 79 by 4

--
Perl5 Master Repository

[perl.git] branch blead, updated. v5.19.0-140-gd21c5a5 by James Keenan

2013-05-23T00:34:03Z

From: James Keenan In perl.git, the branch blead has been updated

- Log -----------------------------------------------------------------
commit d21c5a51cf8cd457ab9c037060defa51692199a3
Author: David Steinbrunner
Date: Tue May 21 06:47:33 2013 -0400

typo fix for vms script
-----------------------------------------------------------------------

Summary of changes:
vms/mms2make.pl | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/vms/mms2make.pl b/vms/mms2make.pl
index 7795e4a..1706c51 100644
--- a/vms/mms2make.pl
+++ b/vms/mms2make.pl
@@ -14,7 +14,7 @@
# we invert top of stack at a .else
# we pop at a .endif
# we deselect any other line if $conditions[0] is 0
-# I'm being very lazy - push a 1 at start, then dont need to check for
+# I'm being very lazy - push a 1 at start, then don't need to check for
# an empty @conditions [assume nesting in descrip.mms is correct]
# 2.1 26-Feb-1995 Charles Bailey bailey@newman.upenn.edu
# - handle MMS macros generated by MakeMaker

--
Perl5 Master Repository

[perl.git] branch blead, updated. v5.19.0-139-gb6a6e95 by James Keenan

2013-05-23T00:01:03Z

From: James Keenan In perl.git, the branch blead has been updated

- Log -----------------------------------------------------------------
commit b6a6e9564d193cb709231fad4f4816f447263147
Author: David Steinbrunner
Date: Tue May 21 06:47:05 2013 -0400

typo fixes for regen scripts
-----------------------------------------------------------------------

Summary of changes:
regen/mk_invlists.pl | 4 ++--
regen/reentr.pl | 4 ++--
regen/regcharclass.pl | 12 ++++++------
regen/regen_lib.pl | 2 +-
4 files changed, 11 insertions(+), 11 deletions(-)

diff --git a/regen/mk_invlists.pl b/regen/mk_invlists.pl
index 843d6e7..67b6e41 100644
--- a/regen/mk_invlists.pl
+++ b/regen/mk_invlists.pl
@@ -118,7 +118,7 @@ for my $i (0 .. @$folds_ref - 1) {
next unless ref $folds_ref->[$i]; # Skip single-char folds
push @has_multi_char_fold, $cp_ref->[$i];

- # Add to the the non-finals list each code point that is in a non-final
+ # Add to the non-finals list each code point that is in a non-final
# position
for my $j (0 .. @{$folds_ref->[$i]} - 2) {
push @is_non_final_fold, $folds_ref->[$i][$j]
@@ -204,7 +204,7 @@ for my $prop (qw(
# that crosses the 255/256 boundary if it is one that matches the
# property. For example, in the Word property, there is a range of code
# points that start at U+00F8 and goes through U+02C1. Instead of
- # artifically cutting that off at 256 because 256 is the first code point
+ # artificially cutting that off at 256 because 256 is the first code point
# above Latin1, we let the range go to its natural ending. That gives us
# extra information with no added space taken. But if the range that
# crosses the boundary is one that doesn't match the property, we don't
diff --git a/regen/reentr.pl b/regen/reentr.pl
index 899e83b..c5e7129 100644
--- a/regen/reentr.pl
+++ b/regen/reentr.pl
@@ -177,7 +177,7 @@ my %seend; # the type of this function's "D"
my %seenm; # all the types
my %seenu; # the length of the argument list of this function

-while () { # Read in the protypes.
+while () { # Read in the protoypes.
next if /^\s+$/;
chomp;
my ($func, $hdr, $type, @p) = split(/\s*\|\s*/, $_, -1);
@@ -587,7 +587,7 @@ EOF
my $GENFUNC = uc $genfunc;
my $D = ifprotomatch($FUNC, grep {/D/} @p);
my $d = $seend{$func};
- $d =~ s/\*$//; # snip: we need need the base type.
+ $d =~ s/\*$//; # snip: we need the base type.
push @struct, < $seent{$func} _${genfunc}_struct;
# if $D
diff --git a/regen/regcharclass.pl b/regen/regcharclass.pl
index f5cf315..976582a 100755
--- a/regen/regcharclass.pl
+++ b/regen/regcharclass.pl
@@ -71,7 +71,7 @@ that C contains at least one character.
=item C

Check to see if the string matches a given codepoint (hypothetically a
-U32). The condition is constructed as as to "break out" as early as
+U32). The condition is constructed as to "break out" as early as
possible if the codepoint is out of range of the condition.

IOW:
@@ -307,7 +307,7 @@ sub __cond_join {
# Each string is then stored in the 'strs' subhash as a hash record
# made up of the results of __uni_latin1, using the keynames
# 'low','latin1','utf8', as well as the synthesized 'LATIN1', 'high', and
-# 'UTF8' which hold a merge of 'low' and their lowercase equivelents.
+# 'UTF8' which hold a merge of 'low' and their lowercase equivalents.
#
# Size data is tracked per type in the 'size' subhash.
#
@@ -470,7 +470,7 @@ sub _optree {
$else= 0 unless defined $else;
$depth= 0 unless defined $depth;

- # if we have an emptry string as a key it means we are in an
+ # if we have an empty string as a key it means we are in an
# accepting state and unless we can match further on should
# return the value of the '' key.
if (exists $trie->{''} ) {
@@ -491,7 +491,7 @@ sub _optree {
# it means we are an accepting state (end of sequence).
my @conds= sort { $a <=> $b } grep { length $_ } keys %$trie;

- # if we havent any keys there is no further we can match and we
+ # if we haven't any keys there is no further we can match and we
# can return the "else" value.
return $else if !@conds;

@@ -935,7 +935,7 @@ sub _cond_as_str {
my @masks;
if (@ranges > 1) {

- # See if the entire set shares optimizable characterstics, and if so,
+ # See if the entire set shares optimizable characteristics, and if so,
# return the optimization. We delay checking for this on sets with
# just a single range, as there may be better optimizations available
# in that case.
@@ -1198,7 +1198,7 @@ sub make_macro {
return $self->render( $optree, ($type =~ /^cp/) ? 1 : 0, \%opts, $def_fmt );
}

-# if we arent being used as a module (highly likely) then process
+# if we aren't being used as a module (highly likely) then process
# the __DATA__ below and produce macros in regcharclass.h
# if an argument is provided to the script then it is assumed to
# be the path of the file to output to, if the arg is '-' outputs
diff --git a/regen/regen_lib.pl b/regen/regen_lib.pl
index a575c97..3abec2f 100644
--- a/regen/regen_lib.pl
+++ b/regen/regen_lib.pl
@@ -76,7 +76,7 @@ sub close_and_rename {
warn "changed '$name' to '$final_name'\n" if $Verbose > 0;
push @Changed, $final_name unless $Verbose < 0;

- # Some dosish systems can't rename over an existing file:
+ # Some DOSish systems can't rename over an existing file:
safer_unlink $final_name;
chmod 0600, $name if $Needs_Write;
rename $name, $final_name or die "renaming $name to $final_name: $!";

--
Perl5 Master Repository

[perl.git] branch blead, updated. v5.19.0-138-g2effe01 by James Keenan

2013-05-22T23:04:35Z

From: James Keenan In perl.git, the branch blead has been updated

- Log -----------------------------------------------------------------
commit 2effe01f18007a1b28e11f29b319f2b5cd0ae4df
Author: David Steinbrunner
Date: Tue May 21 06:46:18 2013 -0400

typo fixes for root level scripts

Add David Steinbrunner to AUTHORS.
Update pod issues database.
-----------------------------------------------------------------------

Summary of changes:
AUTHORS | 1 +
autodoc.pl | 2 +-
configpm | 2 +-
install_lib.pl | 2 +-
installperl | 2 +-
make_ext.pl | 2 +-
make_patchnum.pl | 2 +-
makedef.pl | 2 +-
regen.pl | 2 +-
t/porting/known_pod_issues.dat | 5 +++++
write_buildcustomize.pl | 2 +-
11 files changed, 15 insertions(+), 9 deletions(-)

diff --git a/AUTHORS b/AUTHORS
index 892539d..c446fbc 100644
--- a/AUTHORS
+++ b/AUTHORS
@@ -312,6 +312,7 @@ David Nicol
David R. Favor
David Sparks
David Starks-Browning
+David Steinbrunner
David Sundstrom
David M. Syzdek
David Wheeler
diff --git a/autodoc.pl b/autodoc.pl
index 59dc6f0..e000d5a 100644
--- a/autodoc.pl
+++ b/autodoc.pl
@@ -14,7 +14,7 @@
# MANIFEST and *.[ch].
#
# This script is normally invoked as part of 'make all', but is also
-# called from from regen.pl.
+# called from regen.pl.
#
# '=head1' are the only headings looked for. If the next line after the
# heading begins with a word character, it is considered to be the first line
diff --git a/configpm b/configpm
index 33137c6..cfcc467 100755
--- a/configpm
+++ b/configpm
@@ -384,7 +384,7 @@ $fetch_string .= <<'EOT';
EOT

} else {
- # We only have ' delimted.
+ # We only have ' delimited.

$fetch_string .= <<'EOT';
return undef unless $Config_SH_expanded =~ /\n$key=\'(.*?)\'\n/s;
diff --git a/install_lib.pl b/install_lib.pl
index ae8ba0a..308af70 100644
--- a/install_lib.pl
+++ b/install_lib.pl
@@ -16,7 +16,7 @@ BEGIN {
# prefixes.
#
# So we set $^X to pretend that we're the already installed perl, so
- # Config.pm doesits ... expansion off that location.
+ # Config.pm does its ... expansion off that location.

my $location = $Config{initialinstalllocation};
die <<'OS' unless defined $location;
diff --git a/installperl b/installperl
index 0e590e1..ac1c770 100755
--- a/installperl
+++ b/installperl
@@ -671,7 +671,7 @@ sub installlib {
# this, as $_ becomes a pathname, and so still works. However, it's not
# obvious that $_ is needed later, and hence $_ must not be modified.

- # Also, many of the regex exlusion tests below are now superfluous, as the
+ # Also, many of the regex exclusion tests below are now superfluous, as the
# files in question are either no longer in blead, or now in ext/, dist/ or
# cpan/ and not copied into lib/

diff --git a/make_ext.pl b/make_ext.pl
index 3254628..799ab8b 100644
--- a/make_ext.pl
+++ b/make_ext.pl
@@ -43,7 +43,7 @@ my $ext_dirs_re = '(?:' . join('|', @ext_dirs) . ')';
# Mostly because they still not ported to specified platform.
#
# If any extensions are listed with a '+' char then only those
-# extensions will be built, but only if they arent countermanded
+# extensions will be built, but only if they aren't countermanded
# by an '!ext' and are appropriate to the type of building being done.

# It may be deleted in a later release of perl so try to
diff --git a/make_patchnum.pl b/make_patchnum.pl
index 77d4a4e..3f857b5 100644
--- a/make_patchnum.pl
+++ b/make_patchnum.pl
@@ -178,7 +178,7 @@ git_unpushed='$unpushed_commit_list'";
$commit_title ||= "Commit id:";
}

-# we extract the filename out of the warning header, so dont mess with that
+# we extract the filename out of the warning header, so don't mess with that
write_files(<<"EOF_HEADER", <<"EOF_CONFIG");
/**************************************************************************
* WARNING: 'git_version.h' is automatically generated by make_patchnum.pl
diff --git a/makedef.pl b/makedef.pl
index d2c5f5e..ae3b510 100644
--- a/makedef.pl
+++ b/makedef.pl
@@ -627,7 +627,7 @@ if ($ARGS{PLATFORM} eq 'netware') {
}

if ($define{'USE_PERLIO'}) {
- # Export the symols that make up the PerlIO abstraction, regardless
+ # Export the symbols that make up the PerlIO abstraction, regardless
# of its implementation - read from a file
push @syms, 'perlio.sym';

diff --git a/regen.pl b/regen.pl
index 62dc820..1b3ae5e 100644
--- a/regen.pl
+++ b/regen.pl
@@ -1,6 +1,6 @@
#!/usr/bin/perl -w
#
-# regen.pl - a wrapper that runs all *.pl scripts to to autogenerate files
+# regen.pl - a wrapper that runs all *.pl scripts to autogenerate files

require 5.004; # keep this compatible, an old perl is all we may have before
# we build the new one
diff --git a/t/porting/known_pod_issues.dat b/t/porting/known_pod_issues.dat
index b175a81..ffef833 100644
--- a/t/porting/known_pod_issues.dat
+++ b/t/porting/known_pod_issues.dat
@@ -296,6 +296,7 @@ pod/perltru64.pod ? Should you be using F<...> or maybe L<...> instead of 1
pod/perltru64.pod Verbatim line length including indents exceeds 79 by 4
pod/perlunifaq.pod empty section in previous paragraph 1
pod/perluniprops.pod =item type mismatch 6
+pod/perlutil.pod Apparent internal link is missing its forward slash 2
pod/perlvms.pod ? Should you be using F<...> or maybe L<...> instead of 1
pod/perlvms.pod Verbatim line length including indents exceeds 79 by 2
pod/perlwin32.pod Verbatim line length including indents exceeds 79 by 12
@@ -310,6 +311,10 @@ porting/todo.pod Verbatim line length including indents exceeds 79 by 7
symbian/perlutil.pod Verbatim line length including indents exceeds 79 by 4
utils/c2ph Verbatim line length including indents exceeds 79 by 44
utils/pod2html Pod NAME already used 1
+utils/prove Apparent broken link 2
+utils/prove Pod NAME already used 1
+utils/prove There is more than one target 1
+utils/prove Verbatim line length including indents exceeds 79 by 3
vms/ext/filespec.pm Verbatim line length including indents exceeds 79 by 1
x2p/a2p.pod empty section in previous paragraph 2
lib/benchmark.pm Verbatim line length including indents exceeds 79 by 4
diff --git a/write_buildcustomize.pl b/write_buildcustomize.pl
index 3d773a5..018e60e 100644
--- a/write_buildcustomize.pl
+++ b/write_buildcustomize.pl
@@ -34,7 +34,7 @@ my @toolchain = qw(cpan/AutoLoader/lib
# Used only in ExtUtils::Liblist::Kid::_win32_ext()
push @toolchain, 'cpan/Text-ParseWords/lib' if $^O eq 'MSWin32';

-# lib must be last, as the the toolchain modules write themselves into it
+# lib must be last, as the toolchain modules write themselves into it
# as they build, and it's important that @INC order ensures that the partially
# written files are always masked by the complete versions.

--
Perl5 Master Repository

[perl.git] branch blead, updated. v5.19.0-137-g5364049 by Karl Williamson

2013-05-22T19:31:55Z

From: Karl Williamson In perl.git, the branch blead has been updated

- Log -----------------------------------------------------------------
commit 5364049c28608557a7832b748edaf27ce5262d89
Author: Karl Williamson
Date: Wed May 22 13:28:07 2013 -0600

perldiag: Describe \g syntax error

Commit 779fedd7c3021f013726c8f53cb9e66c54637ebf added a warning,
which is nearly identical to an existing one, but didn't change the
description of the existing one to encompass both. This rectifies that.
-----------------------------------------------------------------------

Summary of changes:
pod/perldiag.pod | 6 ++++--
1 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/pod/perldiag.pod b/pod/perldiag.pod
index 5ceafdf..13ffbee 100644
--- a/pod/perldiag.pod
+++ b/pod/perldiag.pod
@@ -5677,8 +5677,10 @@ instead of:

=item Unterminated \g{...} pattern in regex; marked by <-- HERE in m/%s/

-(F) You missed a close brace on a \g{..} pattern (group reference) in
-a regular expression. Fix the pattern and retry.
+(F) In a regular expression, you had a C<\g> that wasn't followed by a
+proper group reference. In the case of C<\g{>, the closing brace is
+missing; otherwise the C<\g> must be followed by an integer. Fix the
+pattern and retry.

=item Unterminated <> operator

--
Perl5 Master Repository

[perl.git] branch blead, updated. v5.19.0-136-g779fedd by Karl Williamson

2013-05-22T19:06:41Z

From: Karl Williamson In perl.git, the branch blead has been updated

- Log -----------------------------------------------------------------
commit 779fedd7c3021f013726c8f53cb9e66c54637ebf
Author: Karl Williamson
Date: Wed Mar 20 11:42:18 2013 -0600

regcomp.c: Actually emit proper warning

Before this commit, /\g/ raised the wrong warning
Reference to invalid group 0
This rearranges the code so that the proper warning is emitted.
Unterminated \g... pattern
-----------------------------------------------------------------------

Summary of changes:
pod/perldiag.pod | 2 ++
regcomp.c | 12 ++++++++----
t/porting/diag.t | 1 -
t/re/reg_mesg.t | 1 +
4 files changed, 11 insertions(+), 5 deletions(-)

diff --git a/pod/perldiag.pod b/pod/perldiag.pod
index ec4812c..5ceafdf 100644
--- a/pod/perldiag.pod
+++ b/pod/perldiag.pod
@@ -5673,6 +5673,8 @@ instead of:

<<"foo"

+=item Unterminated \g... pattern in regex; marked by <-- HERE in m/%s/
+
=item Unterminated \g{...} pattern in regex; marked by <-- HERE in m/%s/

(F) You missed a close brace on a \g{..} pattern (group reference) in
diff --git a/regcomp.c b/regcomp.c
index 05d15a4..7293a57 100644
--- a/regcomp.c
+++ b/regcomp.c
@@ -10673,8 +10673,14 @@ tryagain:
goto parse_named_seq;
} }
num = atoi(RExC_parse);
- if (isg && num == 0)
- vFAIL("Reference to invalid group 0");
+ if (isg && num == 0) {
+ if (*RExC_parse == '0') {
+ vFAIL("Reference to invalid group 0");
+ }
+ else {
+ vFAIL("Unterminated \\g... pattern");
+ }
+ }
if (isrel) {
num = RExC_npar - num;
if (num < 1)
@@ -10687,8 +10693,6 @@ tryagain:
char * const parse_start = RExC_parse - 1; /* MJD */
while (isDIGIT(*RExC_parse))
RExC_parse++;
- if (parse_start == RExC_parse - 1)
- vFAIL("Unterminated \\g... pattern");
if (hasbrace) {
if (*RExC_parse != '}')
vFAIL("Unterminated \\g{...} pattern");
diff --git a/t/porting/diag.t b/t/porting/diag.t
index c5dcbee..920646a 100644
--- a/t/porting/diag.t
+++ b/t/porting/diag.t
@@ -603,7 +603,6 @@ Unknown Unicode option value %d
Unrecognized character %s; marked by <-- HERE after %s<-- HERE near column %d
Unstable directory path, current directory changed unexpectedly
Unterminated compressed integer in unpack
-Unterminated \g... pattern in regex; marked by <-- HERE in m/%s/
Usage: CODE(0x%x)(%s)
Usage: %s(%s)
Usage: %s::%s(%s)
diff --git a/t/re/reg_mesg.t b/t/re/reg_mesg.t
index 537283c..ca5c8df 100644
--- a/t/re/reg_mesg.t
+++ b/t/re/reg_mesg.t
@@ -104,6 +104,7 @@ my @death =

'/(x)\2/' => 'Reference to nonexistent group {#} m/(x)\2{#}/',

+ '/\g/' => 'Unterminated \g... pattern {#} m/\g{#}/',
'/\g{1/' => 'Unterminated \g{...} pattern {#} m/\g{1{#}/',

'my $m = "\\\"; $m =~ $m', => 'Trailing \ in regex m/\/',

--
Perl5 Master Repository

[perl.git] branch smoke-me/davem/enable_cow, created. v5.19.0-131-g2ba1403 by Dave Mitchell

2013-05-22T16:24:07Z

From: Dave Mitchell In perl.git, the branch smoke-me/davem/enable_cow has been created

at 2ba14035dacbc515fc44a7ebcd54bea3c87fec05 (commit)

- Log -----------------------------------------------------------------
commit 2ba14035dacbc515fc44a7ebcd54bea3c87fec05
Author: David Mitchell
Date: Wed May 22 17:13:53 2013 +0100

perl -V displayed PERL_NEW_COPY_ON_WRITE twice

I didn't realise that the perl -V list of defines is now split between
perl.h and perl.c and so added it to one (thinking it was missing) when it
was already in the other).

M perl.h

commit 043d863a0a5fb0c62bd3462573b9ce9af2c20094
Author: David Mitchell
Date: Wed May 22 16:38:29 2013 +0100

re-enable Copy-on-Write by default.

COW was first introduced (and enabled by default) in 5.17.7.
It was disabled by default in 5.17.10, because it was though to have too
many rough edges for the 5.18.0 release.

By re-enabling it now, early in the 5.19.x release cycle, hopefully it
will be ready for production use by 5.20.

This commit mainly reverts 9f351b45f4 and e1fd41328c (with modifications),
then updates perldelta.

M perl.h
M pod/perldelta.pod
M pod/perlre.pod
M pod/perlreref.pod
M pod/perlretut.pod
M pod/perlvar.pod
M t/re/pat_rt_report.t
-----------------------------------------------------------------------

--
Perl5 Master Repository

[perl.git] branch blead, updated. v5.19.0-135-g1fdd5e5 by Karl Williamson

2013-05-22T16:13:20Z

From: Karl Williamson In perl.git, the branch blead has been updated

- Log -----------------------------------------------------------------
commit 1fdd5e539a93d9e9573e769f06c0f3d3c3d7e3ac
Author: Karl Williamson
Date: Tue May 21 20:34:47 2013 -0600

Make Unicode::UCD::search_invlist() available

This commit documents this function, removing the initial underscore
from its name. (And it hardens input checking.)

M ext/XS-APItest/t/handy.t
M lib/Unicode/UCD.pm
M lib/Unicode/UCD.t
M pod/perldelta.pod

commit ad5f730f426dcac62c9dc19bda79e1586ec5f135
Author: Karl Williamson
Date: Tue May 21 19:41:44 2013 -0600

Unicode::UCD Clarifications in pod

There are no "missing" values in inversion maps; there is a default
value returned for each one. So change the example variables' names.
Plus another sentence rewording for clarity.

M lib/Unicode/UCD.pm

commit e2ce70ff8deec39f04109eef037018788bacae10
Author: Karl Williamson
Date: Mon May 20 12:13:22 2013 -0600

Unicode::UCD: Move function in file.

This is in preparation for making this function public, and it should be
listed in the pod later than it otherwise would be.

M lib/Unicode/UCD.pm

commit 5b2b197795133a17faa5d8ac3b648776925c57d7
Author: Karl Williamson
Date: Mon May 20 10:41:01 2013 -0600

Unicode::UCD: Correct wrong pod info

This was erroneous. Extra clarifications are also added.

M lib/Unicode/UCD.pm
-----------------------------------------------------------------------

Summary of changes:
ext/XS-APItest/t/handy.t | 4 +-
lib/Unicode/UCD.pm | 233 +++++++++++++++++++++++++++++-----------------
lib/Unicode/UCD.t | 9 ++
pod/perldelta.pod | 7 ++
4 files changed, 167 insertions(+), 86 deletions(-)

diff --git a/ext/XS-APItest/t/handy.t b/ext/XS-APItest/t/handy.t
index 84ffb02..3a8abc9 100644
--- a/ext/XS-APItest/t/handy.t
+++ b/ext/XS-APItest/t/handy.t
@@ -98,7 +98,7 @@ foreach my $name (sort keys %properties) {
my $i = utf8::native_to_unicode($j);
my $function = uc($name);

- my $matches = Unicode::UCD::_search_invlist(\@invlist, $i);
+ my $matches = Unicode::UCD::search_invlist(\@invlist, $i);
if (! defined $matches) {
$matches = 0;
}
@@ -260,7 +260,7 @@ foreach my $name (sort keys %to_properties) {
my $i = utf8::native_to_unicode($j);
my $function = $name;

- my $index = Unicode::UCD::_search_invlist(\@{$list_ref}, $j);
+ my $index = Unicode::UCD::search_invlist(\@{$list_ref}, $j);

my $ret;
my $char_name = charnames::viacode($j) // "No name";
diff --git a/lib/Unicode/UCD.pm b/lib/Unicode/UCD.pm
index 9c3dd7c..eeaf2c3 100644
--- a/lib/Unicode/UCD.pm
+++ b/lib/Unicode/UCD.pm
@@ -5,7 +5,7 @@ use warnings;
no warnings 'surrogate'; # surrogates can be inputs to this
use charnames ();

-our $VERSION = '0.51';
+our $VERSION = '0.52';

require Exporter;

@@ -24,6 +24,7 @@ our @EXPORT_OK = qw(charinfo
prop_value_aliases
prop_invlist
prop_invmap
+ search_invlist
MAX_CP
);

@@ -80,6 +81,9 @@ Unicode::UCD - Unicode character database
my ($list_ref, $map_ref, $format, $missing)
= prop_invmap("General Category");

+ use Unicode::UCD 'search_invlist';
+ my $index = search_invlist(\@invlist, $code_point);
+
use Unicode::UCD 'compexcl';
my $compexcl = compexcl($codepoint);

@@ -2124,6 +2128,9 @@ code points that have the property-value:
C does not know about any user-defined nor Perl internal-only
properties, and will return C if called with one of those.

+The L function is provided for finding a code point within
+an inversion list.
+
=cut

# User-defined properties could be handled with some changes to utf8_heavy.pl;
@@ -2261,71 +2268,12 @@ sub prop_invlist ($;$) {
return @invlist;
}

-sub _search_invlist {
- # Find the range in the inversion list which contains a code point; that
- # is, find i such that l[i] <= code_point < l[i+1]. Returns undef if no
- # such i.
-
- # If this is ever made public, could use to speed up .t specials. Would
- # need to use code point argument, as in other functions in this pm
-
- my $list_ref = shift;
- my $code_point = shift;
- # Verify non-neg numeric XXX
-
- my $max_element = @$list_ref - 1;
-
- # Return undef if list is empty or requested item is before the first element.
- return if $max_element < 0;
- return if $code_point < $list_ref->[0];
-
- # Short cut something at the far-end of the table. This also allows us to
- # refer to element [$i+1] without fear of being out-of-bounds in the loop
- # below.
- return $max_element if $code_point >= $list_ref->[$max_element];
-
- use integer; # want integer division
-
- my $i = $max_element / 2;
-
- my $lower = 0;
- my $upper = $max_element;
- while (1) {
-
- if ($code_point >= $list_ref->[$i]) {
-
- # Here we have met the lower constraint. We can quit if we
- # also meet the upper one.
- last if $code_point < $list_ref->[$i+1];
-
- $lower = $i; # Still too low.
-
- }
- else {
-
- # Here, $code_point < $list_ref[$i], so look lower down.
- $upper = $i;
- }
-
- # Split search domain in half to try again.
- my $temp = ($upper + $lower) / 2;
-
- # No point in continuing unless $i changes for next time
- # in the loop.
- return $i if $temp == $i;
- $i = $temp;
- } # End of while loop
-
- # Here we have found the offset
- return $i;
-}
-
=pod

=head2 B

use Unicode::UCD 'prop_invmap';
- my ($list_ref, $map_ref, $format, $missing)
+ my ($list_ref, $map_ref, $format, $default)
= prop_invmap("General Category");

C is used to get the complete mapping definition for a property,
@@ -2352,8 +2300,11 @@ It is a fatal error to call this function except in list context.

In addition to the the two arrays that form the inversion map, C
returns two other values; one is a scalar that gives some details as to the
-format of the entries of the map array; the other is used for specialized
-purposes, described at the end of this section.
+format of the entries of the map array; the other is a default value, useful
+in maps whose format name begins with the letter C<"a">, as described
+L; and for specialized purposes, such as
+converting to another data structure, described at the end of this main
+section.

This means that C returns a 4 element list. For example,

@@ -2413,7 +2364,8 @@ that, instead of treating these as unassigned Unicode code points, the value
for this range should be C. If you wish, you can change the returned
arrays accordingly.

-The maps are almost always simple scalars that should be interpreted as-is.
+The maps for almost all properties are simple scalars that should be
+interpreted as-is.
These values are those given in the Unicode-supplied data files, which may be
inconsistent as to capitalization and as to which synonym for a property-value
is given. The results may be normalized by using the L
@@ -2508,7 +2460,7 @@ is like C<"s"> in that all the map array elements are scalars, but here they are
restricted to all being integers, and some have to be adjusted (hence the name
C<"a">) to get the correct result. For example, in:

- my ($uppers_ranges_ref, $uppers_maps_ref, $format)
+ my ($uppers_ranges_ref, $uppers_maps_ref, $format, $default)
= prop_invmap("Simple_Uppercase_Mapping");

the returned arrays look like this:
@@ -2521,30 +2473,32 @@ the returned arrays look like this:
182 0
...

+and C<$default> is 0.
+
Let's start with the second line. It says that the uppercase of code point 97
is 65; or C == "A". But the line is for the entire range of code
-points 97 through 122. To get the mapping for any code point in a range, you
-take the offset it has from the beginning code point of the range, and add
+points 97 through 122. To get the mapping for any code point in this range,
+you take the offset it has from the beginning code point of the range, and add
that to the mapping for that first code point. So, the mapping for 122 ("z")
is derived by taking the offset of 122 from 97 (=25) and adding that to 65,
yielding 90 ("z"). Likewise for everything in between.

-The first line works the same way. The first map in a range is always the
-correct value for its code point (because the adjustment is 0). Thus the
-C is just itself. Also, C is also itself, as the
-adjustment is 0+1-0 .. C is 96.
-
Requiring this simple adjustment allows the returned arrays to be
significantly smaller than otherwise, up to a factor of 10, speeding up
searching through them.

+Ranges that map to C<$default>, C<"0">, behave somewhat differently. For
+these, each code point maps to itself. So, in the first line in the example,
+S> is 0, S> is 1, ..
+S> is 96.
+
=item B>

means that some of the map array elements have the form given by C<"a">, and
the rest are ordered lists of code points.
For example, in:

- my ($uppers_ranges_ref, $uppers_maps_ref, $format)
+ my ($uppers_ranges_ref, $uppers_maps_ref, $format, $default)
= prop_invmap("Uppercase_Mapping");

the returned arrays look like this:
@@ -2571,6 +2525,9 @@ CAPITAL LETTER N).
No adjustments are needed to entries that are references to arrays; each such
entry will have exactly one element in its range, so the offset is always 0.

+The fourth (index [3]) element (C<$default>) in the list returned for this
+format is 0.
+
=item B>

This is like C<"a">, but some elements are the empty string, and should not be
@@ -2600,6 +2557,9 @@ represents 0+1-0 = 1; ... code point 0x39, (DIGIT NINE), represents 0+9-0 = 9;
(ARABIC-INDIC DIGIT ZERO), represents 0; ... 0x07C1 (NKO DIGIT ONE),
represents 0+1-0 = 1 ...

+The fourth (index [3]) element (C<$default>) in the list returned for this
+format is the empty string.
+
=item B>

is a combination of the C<"al"> type and the C<"ae"> type. Some of
@@ -2617,6 +2577,9 @@ An example slice is:
0x00B0 0
...

+The fourth (index [3]) element (C<$default>) in the list returned for this
+format is 0.
+
=item B>

means that all the elements of the map array are either rational numbers or
@@ -2656,6 +2619,9 @@ C<"ar">.
0x660 0 ARABIC-INDIC DIGIT ZERO .. NINE
0x66A "NaN"

+The fourth (index [3]) element (C<$default>) in the list returned for this
+format is C<"NaN">.
+
=item B>

means the Name property. All the elements of the map array are simple
@@ -2701,6 +2667,9 @@ L.
Note that the mapping is the one that is specified in the Unicode data files,
and to get the final decomposition, it may need to be applied recursively.

+The fourth (index [3]) element (C<$default>) in the list returned for this
+format is 0.
+
=back

Note that a format begins with the letter "a" if and only the property it is
@@ -2713,29 +2682,31 @@ which is an integer. That is, it must match the regular expression:
Further, the first element in a range never needs adjustment, as the
adjustment would be just adding 0.

-A binary search can be used to quickly find a code point in the inversion
-list, and hence its corresponding mapping.
+A binary search such as that provided by L, can be used to
+quickly find a code point in the inversion list, and hence its corresponding
+mapping.

-The final element (index [3], assigned to C<$default> in the "block" example) in
-the four element list returned by this function may be useful for applications
+The final, fourth element (index [3], assigned to C<$default> in the "block"
+example) in the four element list returned by this function is used with the
+C<"a"> format types; it may also be useful for applications
that wish to convert the returned inversion map data structure into some
other, such as a hash. It gives the mapping that most code points map to
under the property. If you establish the convention that any code point not
explicitly listed in your data structure maps to this value, you can
potentially make your data structure much smaller. As you construct your data
structure from the one returned by this function, simply ignore those ranges
-that map to this value, generally called the "default" value. For example, to
+that map to this value. For example, to
convert to the data structure searchable by L, you can follow
this recipe for properties that don't require adjustments:

- my ($list_ref, $map_ref, $format, $missing) = prop_invmap($property);
+ my ($list_ref, $map_ref, $format, $default) = prop_invmap($property);
my @range_list;

# Look at each element in the list, but the -2 is needed because we
# look at $i+1 in the loop, and the final element is guaranteed to map
- # to $missing by prop_invmap(), so we would skip it anyway.
+ # to $default by prop_invmap(), so we would skip it anyway.
for my $i (0 .. @$list_ref - 2) {
- next if $map_ref->[$i] eq $missing;
+ next if $map_ref->[$i] eq $default;
push @range_list, [ $list_ref->[$i],
$list_ref->[$i+1],
$map_ref->[$i]
@@ -2745,13 +2716,13 @@ this recipe for properties that don't require adjustments:
print charinrange(\@range_list, $code_point), "\n";

With this, C will return C if its input code point maps
-to C<$missing>. You can avoid this by omitting the C statement, and adding
+to C<$default>. You can avoid this by omitting the C statement, and adding
a line after the loop to handle the final element of the inversion map.

Similarly, this recipe can be used for properties that do require adjustments:

for my $i (0 .. @$list_ref - 2) {
- next if $map_ref->[$i] eq $missing;
+ next if $map_ref->[$i] eq $default;

# prop_invmap() guarantees that if the mapping is to an array, the
# range has just one element, so no need to worry about adjustments.
@@ -3411,7 +3382,7 @@ RETRY:
}

# Find the range that the override applies to.
- my $i = _search_invlist(\@invlist, $cp);
+ my $i = search_invlist(\@invlist, $cp);
if ($cp < $invlist[$i] || $cp >= $invlist[$i + 1]) {
croak __PACKAGE__, "::prop_invmap: wrong_range, cp=$cp; i=$i, current=$invlist[$i]; next=$invlist[$i + 1]"
}
@@ -3520,6 +3491,100 @@ RETRY:
return (\@invlist, \@invmap, $format, $missing);
}

+sub search_invlist {
+
+=pod
+
+=head2 B
+
+ use Unicode::UCD qw(prop_invmap prop_invlist);
+ use Unicode::UCD 'search_invlist';
+
+ my @invlist = prop_invlist($property_name);
+ print $code_point, ((search_invlist(\@invlist, $code_point) // -1) % 2)
+ ? " isn't"
+ : " is",
+ " in $property_name\n";
+
+ my ($blocks_ranges_ref, $blocks_map_ref) = prop_invmap("Block");
+ my $index = search_invlist($blocks_ranges_ref, $code_point);
+ print "$code_point is in block ", $blocks_map_ref->[$index], "\n";
+
+C is used to search an inversion list returned by
+C or C for a particular L.
+C is returned if the code point is not found in the inversion list
+(this happens only when it is not a legal L, or is less +than the list's first element). A warning is raised in the first instance. + +Otherwise, it returns the index into the list of the range that contains the +code point.; that is, find C such that + + list[i]<= code_point < list[i+1]. + +As explained in L, whether a code point is in the list or not +depends on if the index is even (in) or odd (not in). And as explained in +L, the index is used with the returned parallel array to find +the mapping. + +=cut + + + my $list_ref = shift; + my $input_code_point = shift; + my $code_point = _getcode($input_code_point); + + if (! defined $code_point) { + carp __PACKAGE__, "::search_invlist: unknown code '$input_code_point'"; + return; + } + + my $max_element = @$list_ref - 1; + + # Return undef if list is empty or requested item is before the first element. + return if $max_element < 0; + return if $code_point < $list_ref->[0]; + + # Short cut something at the far-end of the table. This also allows us to + # refer to element [$i+1] without fear of being out-of-bounds in the loop + # below. + return $max_element if $code_point >= $list_ref->[$max_element]; + + use integer; # want integer division + + my $i = $max_element / 2; + + my $lower = 0; + my $upper = $max_element; + while (1) { + + if ($code_point >= $list_ref->[$i]) { + + # Here we have met the lower constraint. We can quit if we + # also meet the upper one. + last if $code_point < $list_ref->[$i+1]; + + $lower = $i; # Still too low. + + } + else { + + # Here, $code_point < $list_ref[$i], so look lower down. + $upper = $i; + } + + # Split search domain in half to try again. + my $temp = ($upper + $lower) / 2; + + # No point in continuing unless $i changes for next time + # in the loop. + return $i if $temp == $i; + $i = $temp; + } # End of while loop + + # Here we have found the offset + return $i; +} + =head2 Unicode::UCD::UnicodeVersion This returns the version of the Unicode Character Database, in other words, the diff --git a/lib/Unicode/UCD.t b/lib/Unicode/UCD.t index e070def..4b0c227 100644 --- a/lib/Unicode/UCD.t +++ b/lib/Unicode/UCD.t @@ -2026,5 +2026,14 @@ foreach my $prop (keys %props) { pass("prop_invmap('$mod_prop')"); } +# A few tests of search_invlist +use Unicode::UCD qw(search_invlist); + +my ($scripts_ranges_ref, $scripts_map_ref) = prop_invmap("Script"); +my $index = search_invlist($scripts_ranges_ref, 0x390); +is($scripts_map_ref->[$index], "Greek", "U+0390 is Greek"); +my @alpha_invlist = prop_invlist("Alpha"); +is(search_invlist(\@alpha_invlist, ord("\t")), undef, "search_invlist returns undef for code points before first one on the list"); + ok($/ eq $input_record_separator, "The record separator didn't get overridden"); done_testing(); diff --git a/pod/perldelta.pod b/pod/perldelta.pod index cb7ab03..431e811 100644 --- a/pod/perldelta.pod +++ b/pod/perldelta.pod @@ -126,6 +126,13 @@ memory used by 3.26 and earlier. C is always propagated to a test's C<@INC>, even under C<-T>. +=item * + +Unicode::UCD has been upgraded from version 0.51 to 0.52. + +A function, L is now available to do +search an inversion list or map for a code point. + =back =head2 Removed Modules and Pragmata -- Perl5 Master Repository
[perl.git] branch blead, updated. v5.19.0-131-g2248d90 by Ricardo Signes 2013-05-22T13:33:33Z From: Ricardo Signes In perl.git, the branch blead has been updated - Log ----------------------------------------------------------------- commit 2248d90cae57123bbd57bed3759d290369a37652 Author: Ricardo Signes Date: Wed May 22 09:33:16 2013 -0400 remove a spurious space ----------------------------------------------------------------------- Summary of changes: pod/perlfunc.pod | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pod/perlfunc.pod b/pod/perlfunc.pod index 08b9df9..9540a01 100644 --- a/pod/perlfunc.pod +++ b/pod/perlfunc.pod @@ -9197,7 +9197,7 @@ These flow-control keywords are documented in L. =item when These flow-control keywords related to the experimental switch feature are -documented in L . +documented in L. =back -- Perl5 Master Repository [perl.git] branch blead, updated. v5.19.0-130-g2073970 by Nicholas Clark 2013-05-22T11:59:20Z From: Nicholas Clark In perl.git, the branch blead has been updated - Log ----------------------------------------------------------------- commit 2073970fee26667acdf365b19220c8184fa8b18d Author: Nicholas Clark Date: Wed May 22 13:56:50 2013 +0200 Some more comments describing the slab allocator. These are based on the description of commit 8be227ab5eaa23f2, but provide explanations near to the relevant C code. ----------------------------------------------------------------------- Summary of changes: op.c | 17 ++++++++++++++++- 1 files changed, 16 insertions(+), 1 deletions(-) diff --git a/op.c b/op.c index 3bdbdf6..96a06b1 100644 --- a/op.c +++ b/op.c @@ -165,11 +165,23 @@ Perl_Slab_Alloc(pTHX_ size_t sz) OP *o; size_t opsz, space; + /* We only allocate ops from the slab during subroutine compilation. + We find the slab via PL_compcv, hence that must be non-NULL. It could + also be pointing to a subroutine which is now fully set up (CvROOT() + pointing to the top of the optree for that sub), or a subroutine + which isn't using the slab allocator. If our sanity checks aren't met, + don't use a slab, but allocate the OP directly from the heap. */ if (!PL_compcv || CvROOT(PL_compcv) || (CvSTART(PL_compcv) && !CvSLABBED(PL_compcv))) return PerlMemShared_calloc(1, sz); - if (!CvSTART(PL_compcv)) { /* sneak it in here */ + /* While the subroutine is under construction, the slabs are accessed via + CvSTART(), to avoid needing to expand PVCV by one pointer for something + unneeded at runtime. Once a subroutine is constructed, the slabs are + accessed via CvROOT(). So if CvSTART() is NULL, no slab has been + allocated yet. See the commit message for 8be227ab5eaa23f2 for more + details. */ + if (!CvSTART(PL_compcv)) { CvSTART(PL_compcv) = (OP *)(slab = S_new_slab(aTHX_ PERL_SLAB_SIZE)); CvSLABBED_on(PL_compcv); @@ -180,6 +192,9 @@ Perl_Slab_Alloc(pTHX_ size_t sz) opsz = SIZE_TO_PSIZE(sz); sz = opsz + OPSLOT_HEADER_P; + /* The slabs maintain a free list of OPs. In particular, constant folding + will free up OPs, so it makes sense to re-use them where possible. A + freed up slot is used in preference to a new allocation. */ if (slab->opslab_freed) { OP **too = &slab->opslab_freed; o = *too; -- Perl5 Master Repository [perl.git] branch smoke-me/nicholas/embed_lib, updated. v5.19.0-137-g19b4a09 by Nicholas Clark 2013-05-22T10:15:27Z From: Nicholas Clark In perl.git, the branch smoke-me/nicholas/embed_lib has been updated - Log ----------------------------------------------------------------- commit 19b4a091cf8836e58746afad403eff0c21e31277 Author: Nicholas Clark Date: Tue Apr 17 15:59:44 2012 +0200 Refactor autodoc.pl to read MANIFEST line by line, instead of slurping it. M autodoc.pl commit 8aae984e4f7549ef237c15d4a7f26d4f77feaace Author: Nicholas Clark Date: Tue Apr 17 15:47:13 2012 +0200 autodoc.pl now uses regen/embed_lib.pl to parse embed.fnc Previously it had its own custom parsing code for embed.fnc. M autodoc.pl commit ada7da9103425235b28b8223e09f4636c0087efe Author: Nicholas Clark Date: Sun Apr 8 19:44:11 2012 +0200 Refactor autodoc.pl to use regen_lib.pl to generate file headers & footers. This reduces duplication of code that generates '-*- buffer-read-only: t -*-', 'ex: set ro:' etc. M Makefile.SH M autodoc.pl commit 90f9998116d74c6d013ab5053c7f330a612b5026 Author: Nicholas Clark Date: Sun Apr 8 19:36:49 2012 +0200 In regen/regen_lib.pl, add 'Pod' as a third supported 'language'. Pod needs a commenting style distinct from C and Perl. (ie the empty string) M regen/regen_lib.pl commit 57cc0707c6dcba96c136512636ecb79f8519c08c Author: Nicholas Clark Date: Sun Apr 8 13:28:06 2012 +0200 t/porting/diag.t now uses regen/embed_lib.pl to parse embed.fnc Previously it had its own custom parsing code for embed.fnc. M t/porting/diag.t commit 6b23d0ababee7054192f3a575cebdffa07100dc4 Author: Nicholas Clark Date: Sun Apr 8 13:08:28 2012 +0200 In t/porting/diag.t, use TestInit to chdir and set up @INC. Also eliminate the $|=1, as it's really not necessary for tests that neither fork nor shell out to other processes. M t/porting/diag.t commit 69c2ee803e78866069c45e8f1c3092d2c6393de3 Author: Nicholas Clark Date: Sun Apr 8 19:56:47 2012 +0200 Avoid VMS running autodoc.pl twice for no reason. Previous descrip_mms.template carried separate rules to build perlintern.pod and perlapi.pod, each of which caused autodoc.pl to run. autdoc.pl builds both files when it runs, so this was duplicate work. M vms/descrip_mms.template commit 0067d1d8cb4534a239eecb84c47d09e095e76f5e Author: Nicholas Clark Date: Wed May 22 11:48:03 2013 +0200 Refactor the "check categories and severity" logic in diag.t Move the "todo" bailout as early as possible, before any work is done. Replace C [sic] with C, as this makes the intent much clearer. Use parenthesis for like() and is() as this is consistent with the rest of the test, and permits a simplification of the BEGIN handling. M t/porting/diag.t ----------------------------------------------------------------------- Summary of changes: Makefile.SH | 3 +- autodoc.pl | 72 ++++++++++++++------------------------------- regen/regen_lib.pl | 14 ++++++--- t/porting/diag.t | 64 ++++++++++++++-------------------------- vms/descrip_mms.template | 6 ++-- 5 files changed, 60 insertions(+), 99 deletions(-) diff --git a/Makefile.SH b/Makefile.SH index 7502235..a18dff0 100755 --- a/Makefile.SH +++ b/Makefile.SH @@ -1076,7 +1076,8 @@ pod/perltoc.pod: $(perltoc_pod_prereqs) $(PERL_EXE) $(ext) pod/buildtoc pod/perlapi.pod: pod/perlintern.pod -pod/perlintern.pod: $(MINIPERL_EXE) autodoc.pl embed.fnc +# Depending on $(nonxs_ext) gets us Text::Wrap built +pod/perlintern.pod: $(MINIPERL_EXE) autodoc.pl embed.fnc $(nonxs_ext) $(MINIPERL) autodoc.pl pod/perlmodlib.pod: $(MINIPERL_EXE) pod/perlmodlib.PL MANIFEST diff --git a/autodoc.pl b/autodoc.pl index 59dc6f0..dd0d8bb 100644 --- a/autodoc.pl +++ b/autodoc.pl @@ -25,6 +25,14 @@ use strict; +if (@ARGV) { + my $workdir = shift; + chdir $workdir + or die "Couldn't chdir to '$workdir': $!"; +} +require 'regen/regen_lib.pl'; +require 'regen/embed_lib.pl'; + # # See database of global and static function prototypes in embed.fnc # This is used to generate prototype headers under various configurations, @@ -244,17 +252,11 @@ removed without notice.\n\n$docs" if $flags =~ /x/; sub output { my ($podname, $header, $dochash, $missing, $footer) = @_; - my $filename = "pod/$podname.pod"; - open my $fh, '>', $filename or die "Can't open $filename: $!"; + my $fh = open_new("pod/$podname.pod", undef, + {by => "$0 extracting documentation", + from => 'the C source files'}); - print $fh <<"_EOH_", $header; --*- buffer-read-only: t -*- - -!!!!!!! DO NOT EDIT THIS FILE !!!!!!! -This file is built by $0 extracting documentation from the C source -files. - -_EOH_ + print $fh $header; my $key; # case insensitive sort, with fallback for determinacy @@ -306,41 +308,15 @@ _EOB_ } print $fh "=back\n\n"; } + print $fh $footer, "=cut\n"; -print $fh $footer, <<'_EOF_'; -=cut - - ex: set ro: -_EOF_ - - close $fh or die "Can't close $filename: $!"; + read_only_bottom_close_and_rename($fh); } -if (@ARGV) { - my $workdir = shift; - chdir $workdir - or die "Couldn't chdir to '$workdir': $!"; -} - -open IN, "embed.fnc" or die $!; - -while () { - chomp; - next if /^:/; - while (s|\\\s*$||) { - $_ .= ; - chomp; - } - s/\s+$//; - next if /^\s*(#|$)/; - - my ($flags, $retval, $func, @args) = split /\s*\|\s*/, $_; - - next unless $func; - - s/\b(NN|NULLOK)\b\s+//g for @args; - $func =~ s/\t//g; # clean up fields from embed.pl - $retval =~ s/\t//; +foreach (@{(setup_embed())[0]}) { + next if @$_ < 2; + my ($flags, $retval, $func, @args) = @$_; + s/\b(?:NN|NULLOK)\b\s+//g for @args; $funcflags{$func} = { flags => $flags, @@ -349,21 +325,19 @@ while () { }; } -my $file; # glob() picks up docs from extra .c or .h files that may be in unclean # development trees. -my $MANIFEST = do { - local ($/, *FH); - open FH, "MANIFEST" or die "Can't open MANIFEST: $!"; - ; -}; +open my $fh, '<', 'MANIFEST' + or die "Can't open MANIFEST: $!"; +while (my $line = <$fh>) { + next unless my ($file) = $line =~ /^(\S+\.[ch])\t/; -for $file (($MANIFEST =~ /^(\S+\.c)\t/gm), ($MANIFEST =~ /^(\S+\.h)\t/gm)) { open F, "< $file" or die "Cannot open $file for docs: $!\n"; $curheader = "Functions in file $file\n"; autodoc(\*F,$file); close F or die "Error closing $file: $!\n"; } +close $fh or die "Error whilst reading MANIFEST: $!"; for (sort keys %funcflags) { next unless $funcflags{$_}{flags} =~ /d/; diff --git a/regen/regen_lib.pl b/regen/regen_lib.pl index a575c97..4c13535 100644 --- a/regen/regen_lib.pl +++ b/regen/regen_lib.pl @@ -36,7 +36,8 @@ sub safer_unlink { sub open_new { my ($final_name, $mode, $header) = @_; my $name = $final_name . '-new'; - my $lang = $final_name =~ /\.(?:c|h|tab|act)$/ ? 'C' : 'Perl'; + my $lang = $final_name =~ /\.pod$/ ? 'Pod' : + $final_name =~ /\.(?:c|h|tab|act)$/ ? 'C' : 'Perl'; my $fh = gensym; if (!defined $mode or $mode eq '>') { if (-f $name) { @@ -82,11 +83,13 @@ sub close_and_rename { rename $name, $final_name or die "renaming $name to $final_name: $!"; } +my %lang_opener = (Perl => '# ', Pod => '', C => '/* '); + sub read_only_top { my %args = @_; die "Missing language argument" unless defined $args{lang}; die "Unknown language argument '$args{lang}'" - unless $args{lang} eq 'Perl' or $args{lang} eq 'C'; + unless exists $lang_opener{$args{lang}}; my $style = $args{style} ? " $args{style} " : ' '; my $raw = "-*- buffer-read-only: t -*-\n"; @@ -125,8 +128,9 @@ EOM $raw .= $args{final} if $args{final}; local $Text::Wrap::columns = 78; - my $cooked = $args{lang} eq 'Perl' - ? wrap('# ', '# ', $raw) . "\n" : wrap('/* ', $style, $raw) . " */\n\n"; + my $cooked = $args{lang} eq 'C' + ? wrap('/* ', $style, $raw) . " */\n\n" + : wrap($lang_opener{$args{lang}}, $lang_opener{$args{lang}}, $raw) . "\n"; $cooked =~ tr/\0/ /; # Don't break Larry's name etc $cooked =~ s/ +$//mg; # Remove all trailing spaces $cooked =~ s! \*/\n!$args{quote}!s if $args{quote}; @@ -152,7 +156,7 @@ sub read_only_bottom_close_and_rename { if (defined $lang && $lang eq 'Perl') { $comment =~ s/^/# /mg; - } else { + } elsif (!defined $lang or $lang ne 'Pod') { $comment =~ s/^/ * /mg; $comment =~ s! \* !/* !; $comment .= " */"; diff --git a/t/porting/diag.t b/t/porting/diag.t index c5dcbee..8698bf9 100644 --- a/t/porting/diag.t +++ b/t/porting/diag.t @@ -1,15 +1,15 @@ #!/usr/bin/perl -use warnings; -use strict; BEGIN { - chdir 't'; - require './test.pl'; + @INC = '..' if -f '../TestInit.pm'; } +use TestInit qw(T); # T is chdir to the top level -plan('no_plan'); +use warnings; +use strict; -$|=1; +require 't/test.pl'; +plan('no_plan'); # --make-exceptions-list outputs the list of strings that don't have # perldiag.pod entries to STDERR without TAP formatting, so they can @@ -19,37 +19,19 @@ $|=1; # Just add the documentation instead. my $make_exceptions_list = ($ARGV[0]||'') eq '--make-exceptions-list'; -chdir '..' or die "Can't chdir ..: $!"; -BEGIN { defined $ENV{PERL_UNICODE} and push @INC, "lib"; } +require 'regen/embed_lib.pl'; +# Look for functions that look like they could be diagnostic ones. my @functions; - -open my $func_fh, "<", "embed.fnc" or die "Can't open embed.fnc: $!"; - -# Look for functions in embed.fnc that look like they could be diagnostic ones. -while (<$func_fh>) { - chomp; - s/^\s+//; - while (s/\s*\\$//) { # Grab up all continuation lines, these end in \ - my $next = <$func_fh>; - $next =~ s/^\s+//; - chomp $next; - $_ .= $next; - } - next if /^:/; # Lines beginning with colon are comments. - next unless /\|/; # Lines without a vertical bar are something we can't deal - # with - my @fields = split /\s*\|\s*/; - next unless $fields[2] =~ /warn|err|(\b|_)die|croak/i; - push @functions, $fields[2]; - +foreach (@{(setup_embed())[0]}) { + next if @$_ < 2; + next unless $_->[2] =~ /warn|err|(\b|_)die|croak/i; # The flag p means that this function may have a 'Perl_' prefix # The flag s means that this function may have a 'S_' prefix - push @functions, "Perl_$fields[2]", if $fields[0] =~ /p/; - push @functions, "S_$fields[2]", if $fields[0] =~ /s/; -} - -close $func_fh; + push @functions, $_->[2]; + push @functions, 'Perl_' . $_->[2] if $_->[0] =~ /p/; + push @functions, 'S_' . $_->[2] if $_->[0] =~ /s/; +}; my $regcomp_re = "(?(?:ckWARN(?:\\d+)?reg\\w*|vWARN\\d+))"; my $function_re = join '|', @functions; @@ -401,24 +383,24 @@ sub check_message { # We found an actual valid entry in perldiag.pod for this error. pass($key); + return $ret + if $entries{$key}{cattodo}; + # Now check the category and severity # Cache our severity qr thingies - use 5.01; + use feature 'state'; state %qrs; my $qr = $qrs{$severity} ||= qr/$severity/; - return $ret - if $entries{$key}{cattodo}; - - like $entries{$key}{severity}, $qr, + like($entries{$key}{severity}, $qr, $severity =~ /\[/ ? "severity is one of $severity for $key" - : "severity is $severity for $key"; + : "severity is $severity for $key"); - is $entries{$key}{category}, $categories, + is($entries{$key}{category}, $categories, ($categories ? "categories are [$categories]" : "no category") - . " for $key"; + . " for $key"); } # Later, should start checking that the severity is correct, too. } elsif ($partial) { diff --git a/vms/descrip_mms.template b/vms/descrip_mms.template index af27c3c..bb5b00c 100644 --- a/vms/descrip_mms.template +++ b/vms/descrip_mms.template @@ -322,12 +322,12 @@ PERLDELTA_CURRENT = [.pod]perl5191delta.pod $(PERLDELTA_CURRENT) : [.pod]perldelta.pod Copy/NoConfirm/Log $(MMS$SOURCE) $(PERLDELTA_CURRENT) -[.pod]perlapi.pod : embed.fnc autodoc.pl $(MINIPERL_EXE) - $(MINIPERL) autodoc.pl - [.pod]perlmodlib.pod : [.pod]perlmodlib.pl MANIFEST $(MINIPERL_EXE) $(MINIPERL) [.pod]perlmodlib.pl "-q" +[.pod]perlapi.pod : [.pod]perlintern.pod + @ $(NOOP) + [.pod]perlintern.pod : embed.fnc autodoc.pl $(MINIPERL_EXE) $(MINIPERL) autodoc.pl -- Perl5 Master Repository [perl.git] branch smoke-me/nicholas/embed_lib, created. v5.19.0-129-gc5b6cc8 by Nicholas Clark 2013-05-22T10:15:20Z From: Nicholas Clark In perl.git, the branch smoke-me/nicholas/embed_lib has been created at c5b6cc8c74b4cfaa225d9702c4139816707e1999 (commit) - Log ----------------------------------------------------------------- commit c5b6cc8c74b4cfaa225d9702c4139816707e1999 Author: Nicholas Clark Date: Mon Apr 8 16:30:18 2013 +0200 Add comments describing the operation of S_scan_ident(). M toke.c commit 27ffb73a493be4fd4322b04f1bcbdabe6dff523b Author: Nicholas Clark Date: Mon Apr 8 16:27:19 2013 +0200 More tests for the "Ambiguous use of ... resolved ..." warning S_scan_ident(). In particular, test that the special case for reporting $# as @ works. M t/lib/warnings/toke commit 5fb91d484fee378b489da85e763d4ae7df3f2baa Author: Max Maischein Date: Tue May 21 21:40:21 2013 +0200 Fix typo in Porting/sync-with-cpan I will run `make test_porting` on the correct branch before submitting a patch. I will run `make test_porting` on the correct branch before submitting a patch. I will run `make test_porting` on the correct branch before submitting a patch. M Porting/sync-with-cpan commit a1450e8bf79c3c2d21815edb107c092b823221fd Author: Max Maischein Date: Tue May 21 19:09:47 2013 +0200 Eliminate external call to tar executable in favour of Archive::Tar Archive::Tar has ->extract_archive, which does Just That. This means we lose immediate support for bzip2 files, but not all incarnations of `tar` support it either. Signed-off-by: David Golden M Porting/sync-with-cpan commit b6956a94c03907bb423a113afad54a0e343f44cf Author: Chris 'BinGOs' Williams Date: Tue May 21 14:13:52 2013 +0100 Add new Module::CoreList::Utils to MANIFEST M MANIFEST commit eab82fd6486ac0971d2f1b8800432a3b8f0b315c Merge: b4b2fef 5730025 Author: Chris 'BinGOs' Williams Date: Tue May 21 13:32:53 2013 +0100 Merge branch 'smoke-me/coreutils' into blead commit b4b2fefd8bdd72b0f88245101f270e9dc39568f6 Author: Nicholas Clark Date: Wed Apr 24 17:01:43 2013 +0200 Remove 2 superfluous use lines from ExtUtils::CBuilder. use IO::File was added with the upgrade to 0.23 (commit c3fb68a339256eb3 in April 2008), but the use of IO::File in the code was eliminated by the upgrade to 0.2800 (commit 06e8058f27e4269b in Dec 2010), which replaced the code in question with use of File::Temp. The latter refactoring also added the use Data::Dumper; line, but did not add any code which uses Data::Dumper. M dist/ExtUtils-CBuilder/lib/ExtUtils/CBuilder/Base.pm commit fdf38e49aa470e60cb56fdd7a3a49f85da8ea2ce Author: Nicholas Clark Date: Wed Apr 10 13:44:07 2013 +0200 Refactor 3 tests so that they avoid clearing %:: via list assignment. For all 3, clearing %:: was intended to expose another bug, not to directly test the effects of clearing %:: Tested by building the 3 revisions that added the tests, and confirming that the revised test also triggers the bug that each fixed. M t/op/stash.t commit aee674b7db1a4a3fd0082f4247560d3c945d0f5c Author: Nicholas Clark Date: Fri Apr 5 12:12:53 2013 +0200 Test that caller does not SEGV when the current package is undefined. Prior to commit d4d03940c58a0177 this code would SEGV under ithreads. Test suggested by Brian Fraser. M t/op/caller.t commit ef4abe7673cc0b19ffd81c9a42f79de20a52ac87 Author: Nicholas Clark Date: Mon May 20 17:55:46 2013 +0200 Remove fakethr.h and eliminate all references to it and FAKE_THREADS fakethr.h and FAKE_THREADS were for a "green" threads implementation of 5005threads. 5005threads itself is long gone, and it's not clear that -DFAKE_THREADS *ever* built correctly. Certainly it did not work for the 5.005 release, and it did not work at the time of the commits for the initial checkin. The closest that it seems to have been to working is around commit c6ee37c52f2ca9e5 (Dec 1997), where the headers no longer contained errors, but perl.c failed to compile. M MANIFEST M configpm M dist/threads-shared/lib/threads/shared.pm M dist/threads-shared/shared.xs D fakethr.h M perl.h M vms/descrip_mms.template commit 31a53e79025eb5944553e671a615e60f384fe743 Merge: 9e7945e 8ca91d3 Author: Nicholas Clark Date: Tue May 21 07:49:29 2013 +0200 Merge the refactoring of x2p's Makefile to blead. commit 8ca91d39a4a96434bc909d710b033c8505e2cde3 Author: Nicholas Clark Date: Wed May 15 21:48:12 2013 +0200 Simplify x2p's Makefile. Eliminate plexe and plc, which relate to tests for the compiler and were added as part of commit 52cebf5efc9883c7 in May 1988. The compiler was removed by commit de1254415ffeb03b in Sept 2006. The files a2p.man and s2p.man were removed with Perl 5.000. The macros 'public', 'private' and 'util' mirror the structure of the top level Makefile.SH. However, x2p only ever used 'public', so simplify things by inlining the value of 'public' in the only place that uses it. 'addedbyconf' now only contains the value of 'plextract', so use the latter directly in the one place that had used 'addedbyconf'. M x2p/Makefile.SH commit e1da83eac92e1d2253d6b03218386b36cb4bb0af Author: Nicholas Clark Date: Wed May 15 21:34:21 2013 +0200 Remove rules to delete *.loadmap and *.prelmap files, generated on VM/ESA These rules to clean up properly on VM/ESA were added in Oct 1998 by perforce changes 1961, 1962 and 1963 in the confperl branch. VM/ESA support was removed by commit 043fec90e88a2e23 in Aug 2012. M Cross/Makefile-cross-SH M Makefile.SH M x2p/Makefile.SH commit 6105164f37e277ea0eb5dec4a432b86b49489be1 Author: Nicholas Clark Date: Wed May 15 21:26:59 2013 +0200 Add x2p/Makefile.SH to the *.SH files processed by the top level Makefile. This makes the top level Makefile responsible for deleting x2p/Makefile. With this, we can eliminate the 'sh' and 'shextract' from x2p's Makefile. M Makefile.SH M x2p/Makefile.SH commit 655e5c9e391de4eef96d3ad88908a53cddbf53e3 Author: Nicholas Clark Date: Wed May 15 21:17:31 2013 +0200 Remove the now unused 'shlist' Makefile targets. M Makefile.SH M x2p/Makefile.SH commit 9719c95bf27185afc59c14150d4f052f92217540 Author: Nicholas Clark Date: Wed May 15 20:57:37 2013 +0200 Programmatically generate dependencies for all *.SH files. Previously dependencies for Makefile, config.h and makedepend were explicitly coded into verbatim sections of Makefile.SH, with most of the others being generated by makedepend.SH M Makefile.SH M makedepend.SH commit fc40bd89f80e1d94b62f08a3ab57f39ea3333029 Author: Nicholas Clark Date: Wed May 15 20:25:47 2013 +0200 Add runtests.SH to the code that generates the targets sh and shextract. The generated Makefile is functionally identical. M Makefile.SH commit c0d271fb2cf4e67f892e8f1b575439d13b73a01f Author: Nicholas Clark Date: Wed May 15 17:14:50 2013 +0200 Eliminate x2p/cflags.SH. x2p's Makefile can use ../cflags x2p/Makefile is only used on *nix, so this should not present a problem. M MANIFEST M Porting/exec-bit.txt M x2p/Makefile.SH D x2p/cflags.SH commit 3ae7e562a2ff05c27052670c20b7ba3b7005bf7b Author: Nicholas Clark Date: Wed May 15 19:40:46 2013 +0200 Pass in to makedepend the name(s) of files that all objects depend on. M Makefile.SH M makedepend.SH M x2p/Makefile.SH commit 0ff64a71c493f41b23ddd052a465c3d40c3264f2 Author: Nicholas Clark Date: Wed May 15 16:54:40 2013 +0200 In the Makefile, generate the value of shextract programmatically. shextract lists the files generated by shell scripts named *.SH which need deleting. Instead of maintaining it by hand, generate it in Makefile.SH from the list of *.SH files. This indirectly eliminates the last reference to F. makedir.SH was eliminated by commit ebca06e95b02d1a8 and the reference to makedir.SH in Makefile.SH eliminated by commit 847471027aa42997 (both Sept 2009), but that commit missed that the reference to makedir a few lines later was related. M Makefile.SH commit 417de989d3cb17f83d8a6af00ff4b171a8682c8c Author: Nicholas Clark Date: Wed May 15 13:01:22 2013 +0200 Defer running makedepend in x2p/ until it's needed. We can run makedepend in x2p/ as part of the regular build process, which means that it can run as part of a parallel make. This gets a slight speedup (0.2s on this system), and increases flexibility on how x2p/Makefile is generated. M Makefile.SH commit 2fea0471858664126bb4c23b75f6bf15c262cd8b Author: Nicholas Clark Date: Wed May 15 12:49:44 2013 +0200 Add x2p/makefile to the dependencies for targets within x2p/ This will give us more flexibility in when and how we generate it. [x2p/makefile is x2p/GNUmakefile on OS X] M Makefile.SH commit d730c2986a70e4fa953cb2f352b7b5ebf41ea580 Author: Nicholas Clark Date: Wed May 15 12:33:05 2013 +0200 Remove vestigial *.SH code which can link the extracted file into ../ 4 of the *.SH files have code which links the extracted file into the parent directory if the current directory ends SH. This code dates from Perl 1. It seems that the intent was that the *.SH files could live in a SH/ directory, but that feature was neither used or mentioned in the documentation in Perl 1, and hasn't been used since. M Cross/Makefile-cross-SH M Makefile.SH M makedepend.SH M x2p/Makefile.SH commit 9e7945ee50c30a17a84968fb0c7e5c38223af6f9 Merge: 39ecab1 7df6ffa Author: Nicholas Clark Date: Mon May 20 21:42:33 2013 +0200 Merge the pack refactoring to blead. commit 7df6ffa3fb2efaa96f5e6001f1cc311ad9f8b185 Author: Nicholas Clark Date: Mon May 20 21:40:27 2013 +0200 Notes in perldelta for the tiny XS-visible changes of the pack refactoring. M pod/perldelta.pod commit 228e69a781c23b2599bbd89f1de29c212f10ea31 Author: Nicholas Clark Date: Wed May 8 10:40:07 2013 +0200 Annotate the "cold" paths in pp_pack.c with UNLIKELY(). The usual case is nice regular bytes in the host's nice regular order. M pp_pack.c commit 545872c5eddf1f00b9826a3d8d682387f1c5049d Author: Nicholas Clark Date: Tue May 7 18:25:53 2013 +0200 Eliminate the implementations of [hv]to[vh][ls] for mixed-endian systems. As pp_pack.c has had mixed-endian support removed, there is little point in keeping code in perl.h and util.c only needed for architectures that cannot be built. M perl.h M util.c commit 1cb43330cc18f2eee5ab1eb58e65991407f4b243 Author: Nicholas Clark Date: Tue May 7 18:17:39 2013 +0200 Eliminate Perl_my_swabn(), as it is now unused. It is not marked as part of the API, and no code on CPAN is using it. M embed.fnc M embed.h M proto.h M util.c commit 3a88beaa68dbb5bad93145daa0c829e0aeb40adb Author: Nicholas Clark Date: Tue May 7 17:39:42 2013 +0200 When endian-swapping in pack, simply copy the bytes in reverse order. This should restore support for big endian Crays. It doesn't support mixed-endian systems. M embed.fnc M pp_pack.c M proto.h commit aaec8192358ffb8080ad85754ea9aeca93b06c8b Author: Nicholas Clark Date: Tue May 7 16:22:08 2013 +0200 Pass 'needs_swap' as a parameter to SHIFT_BYTES. It feels wrong to have it as an implicit parameter sucked in via the textual expansion of the macro. Whilst it can be derived from the parameter 'datumtype', it seems that the C compiler generates a lot less efficient code that way. M pp_pack.c commit 20aa3a3813ca2068a61ea9582e99df57cc735e62 Author: Nicholas Clark Date: Tue May 7 15:15:54 2013 +0200 When endian-swapping in unpack, simply copy the bytes in reverse order. It is considerably simpler to re-order the bytes before reading them into the variable of the desired type, than to read into the variable and then need a specialised "reverse this integer" function for each size of integer. This should restore support for big endian Crays. It doesn't support mixed-endian systems. Support for mixed-endian systems can be restored (if needed) by re-ordering the bytes correctly at the locations which currently only know how to reverse the bytes. M pp_pack.c commit cd7e75121be11e7979933a8f7a47cd842a685157 Author: Nicholas Clark Date: Tue May 7 12:23:00 2013 +0200 Fold the implementation of DO_BO_UNPACK into SHIFT_BYTES(), its only user. M pp_pack.c commit 7285e3f43c94010f09aa1b7a3ccbdbe7de1aa3e8 Author: Nicholas Clark Date: Tue May 7 11:53:17 2013 +0200 Re-implement SHIFT16 and SHIFT32 as wrappers to SHIFT_BYTES. M pp_pack.c commit e396d2357dcec18c01b1cc78d9edc80a3aa4066d Author: Nicholas Clark Date: Tue May 7 10:08:53 2013 +0200 Refactor the two uses of SHIFT{16,32} not followed by DO_BO_UNPACK(). There is one place where SHIFT16() is not immediately followed by DO_BO_UNPACK() ('n!' and 'v!'), and one place where SHIFT32() is not immediately followed by DO_BO_UNPACK() ('N!' and 'V!'). Every other use of either macro is followed by DO_BO_UNPACK(). For the two locations that don't have DO_BO_UNPACK(), the modifiers '<' and '>' are illegal, hence DO_BO_UNPACK() will always be a no-op. Hence add DO_BO_UNPACK() to both (which permits refactoring to simplify things), and add assert()s that the assumptions are true. M pp_pack.c commit 2b4ad5699f46c7f63185ad4e7d1f61e16aaa73f6 Author: Nicholas Clark Date: Tue May 7 09:25:39 2013 +0200 Remove the now unused 'type' argument from the macros DO_BO_(UN)?PACK. M pp_pack.c commit e8966daba327099373354b8935491872e853160e Author: Nicholas Clark Date: Mon May 6 20:40:34 2013 +0200 As DO_BO_(UN)?PACK are now implemented with my_swabn(), simplify 'J' and 'j'. The conditionally compiled code to figure out the correct named integer size for UVs and IVs is no longer needed, as my_swabn() will swap UVs and IVs, whatever their length. M pp_pack.c M t/porting/diag.t commit 0c0fa9f456cec3d6fd9e033a5024063564ee1ca7 Author: Nicholas Clark Date: Mon May 6 19:58:10 2013 +0200 Merge the identical definitions of DO_BO_UNPACK and DO_BO_PACK. Delete the now-unused mixed-endian definitions of DO_BO_UNPACK and DO_BO_PACK, and the macro BO_CANT_DOIT, as they were the last two things using it. M pp_pack.c commit a1219b5e0bb6c311848c834f67e70ff7a19c6bf4 Author: Nicholas Clark Date: Mon May 6 19:41:10 2013 +0200 Swap byte order in DO_BO_(UN)?PACK based on a variable needs_swap. Add the macro NEEDS_SWAP to initialise needs_swap based on TYPE_ENDIANNESS(datumtype). This makes the two definitions of DO_BO_UNPACK identical, and the two definitions of DO_BO_PACK identical. This also makes building pp_pack.c on a mixed endian byteorder architecture a compile time error. The commit adds pointers on where to add code to re-instate support for such architectures. M pp_pack.c commit e8fda8c499f46bb801215ff1dbea51c1f3007a6c Author: Nicholas Clark Date: Mon May 6 18:43:26 2013 +0200 DO_BO_{UN,}PACK can use my_swabn() directly. This removes the only users of my_{be,le,h}to{be,le,h}n, so remove the definitions for those macros. M perl.h M pp_pack.c commit 7e38f754fea50c46f3571cd67982e061b3cf8304 Author: Nicholas Clark Date: Mon May 6 18:05:24 2013 +0200 Use DO_BO_{UN,}PACK in place of DO_BO_UNPACK_{N,PC} Also, eliminate the intermediate macros DO_BO_PACK_PTR and DO_BO_UNPACK_PTR. M pp_pack.c commit 79be8fb4ac8fa99573841ce07cea480399422f8f Author: Nicholas Clark Date: Mon May 6 17:16:18 2013 +0200 Eliminate my_{hto[bl]e,[bl]etoh}{16,32,64,s,i,l} as nothing now uses them. M embed.fnc M embed.h M perl.h M proto.h M util.c commit 68a0946fb4222dd6bcf5c2881bf6cd64c2c2e5ef Author: Nicholas Clark Date: Mon May 6 16:54:12 2013 +0200 In pp_pack.c, refactor DO_BO_(UN)?PACK to use my_letohn etc Previously they were using a my_letoh* function for the appropriate size. This commit probably breaks fixed 16 and 32 bit templates on big endian Crays where sizeof(short) and sizeof(int) are 8. A future commit will restore support. (Although it's not clear whether such Crays can still build blead cleanly, as we've not had any feedback since Boeing decommissioned their Cray.) M pp_pack.c commit 86c6fce076648c5abe5ab491fec9d3142d93bde1 Author: Nicholas Clark Date: Mon May 6 16:36:09 2013 +0200 In pp_pack.c, refactor DO_BO_(UN)?PACK_PTR to use my_letohn etc Previously they were casting the pointer to an integer value, calling a my_letoh* function for the appropriate size, and casting the return value back to a pointer. M pp_pack.c commit df609c5f9dbf07fc7a18075911f5eaefdeea3ffd Author: Nicholas Clark Date: Mon May 6 16:22:01 2013 +0200 In pp_pack.c, refactor DO_BO_(UN)?PACK_N to use sizeof() on the variable. Previously they were using sizeof() on the type argument passed to the macro. This change makes their implementation more like DO_BO_(UN)?PACK and DO_BO_(UN)?PACK_PTR. M pp_pack.c commit 3cc90e2e712615f2081743c38153954d60e3f0df Author: Nicholas Clark Date: Mon May 6 15:46:02 2013 +0200 Refactor macros in pp_pack.c, removing support for mixed-endian byte orders. The byte-order handling can be simplified considerably if we don't have to support mixed-endian systems. It's not clear whether Perl 5.000 even compiled on PDP-11s, let alone more recent versions. Support probably can be added back at the end of the refactoring, if someone has time and a PDP-11 to test on, but for the intermediate stages it is a lot easier not to need to think about such platforms. M pp_pack.c commit 691a44df41bbec11b634fc7bbf78a6eeb1f768a3 Author: Nicholas Clark Date: Mon May 6 14:40:04 2013 +0200 Eliminate the conditionally-compiled fallback functions for htonl etc. These are now only being used for mixed-endian platforms which do not provide their own htnol (etc) functions. Given that the fallbacks have been buggy since they were added in Perl 3.0, it's safe to conclude that no mixed-endian platforms were ever using these functions. It's also unclear why these functions were ever marked as 'A', part of the API. XS code can't call them directly, as it can't rely on them being compiled. Unsurprisingly, no code on CPAN references them. M embed.fnc M embed.h M perl.h M proto.h M util.c commit 66bb80e4324617840aa70c8af410bf76eedcbea1 Author: Nicholas Clark Date: Mon May 6 13:28:20 2013 +0200 Remove buggy loop-based byte swapping code. The irony is that the union-based code special-cased for little endian systems actually works everywhere, even on mixed-endian systems. M pod/perldiag.pod M util.c commit 36bb5e1d607385b423cc4360bce20f3043900790 Author: Nicholas Clark Date: Mon May 6 08:20:57 2013 +0200 Update the Win32 miniperl conditional compilation, for the htonl etc changes. As of commit 19253ae62cd13079 (Oct 2012), miniperl on Win32 avoids using Winsock. The win32_* wrappers for htonl etc had used the pre-processor macro MYSWAP to conditionally compile in the correct code. However, MYSWAP was defined as a side effect of using the htonl etc wrappers in util.c, which are no longer needed. Hence use the WIN32_NO_SOCKETS macro directly in win32sck.c for the correct conditional compilation. M win32/win32sck.c commit e4b03fedd1c92616345d5428c7b0d2981b0e00d8 Author: Nicholas Clark Date: Sat May 4 22:50:12 2013 +0200 Fallbacks for ntohl, ntohs, htonl and htons for little endian systems. The host byteorder agnostic functions in util.c are now only used on mixed endian systems. M perl.h commit 8e2d7c446d5ab31e6c8e661a3c17f0f0b199d525 Author: Nicholas Clark Date: Sat May 4 22:03:59 2013 +0200 Provide ntohl, ntohs, htonl and htons no-op macros on big endian systems. This means that there are always macros or functions for ntohl, ntohs, htonl and htons available, so eliminate use of HAS_NTOHL etc, and unconditionally compile the code that it was protecting. However, as code on CPAN is using these guard macros, define all of them in perl.h (Technically the 4 are not quite no-ops, as they truncate their values to 32 or 16 bits, to be consistent with the implementations for platforms which need re-ordering.) M perl.h M pp_pack.c M pp_sys.c commit 260c6fee6deba90b42b0734fbdea2fea1d69c9bc Author: Nicholas Clark Date: Sat May 4 21:30:27 2013 +0200 Provide vtohl, vtohs, htovl and htovs no-op macros on little endian systems. This means that there are always macros or functions for vtohl, vtohs, htovl and htovs available, so eliminate HAS_VTOHL etc, and unconditionally compile the code that it was protecting. grep.cpan.me shows that no code on CPAN uses any of these macros. (Technically the 4 are not quite no-ops, as they truncate their values to 32 or 16 bits, to be consistent with the implementations for platforms which need re-ordering.) M perl.h M pp_pack.c M util.c commit 5ae724797a32a9a98d51759460d87f419fdd5a45 Author: Nicholas Clark Date: Sat May 4 21:07:14 2013 +0200 vtohl, vtohs, htovl and htovs are no-ops on 64 bit little endian systems. Previously they were implemented as function calls on 64 bit little endian systems. Bit endian systems implemented them as byte-swapping macros. 32 little endian system didn't implement them at all. 32 and 64 bit little endian systems now behave identically. M perl.h commit fcdf154797de71462d8c8f5606482a86ed73943a Author: Nicholas Clark Date: Fri May 3 17:06:01 2013 +0200 Eliminate the definitions for DO_BO_PACK_P and DO_BO_UNPACK_P. Commit 07409e015252427f in April 2005 replaced all the uses of these two macros with DO_BO_PACK_PC and DO_BO_UNPACK_PC. M pp_pack.c commit f8e5a5db38586c726249f161f1fbb581d1738ca2 Author: Nicholas Clark Date: Fri May 3 15:52:38 2013 +0200 Eliminate pre-5.9.x conditional code for PERL_PACK_CAN_SHRIEKSIGN PERL_PACK_CAN_SHRIEKSIGN has been unconditionally defined for versions 5.9.x and greater, and undefined for 5.8.x. As we are never going to need to port changes back to maint-5.8 any more, eliminate all the 5.8.x related code and the macro that supports it. M packsizetables.c M pp_pack.c M regen/genpacksizetables.pl commit 5fa237154915dfcb787e0c5403e6260482a5fb3e Author: Nicholas Clark Date: Fri May 3 15:26:34 2013 +0200 Eliminate pre-5.9.x conditional code for PERL_PACK_CAN_BYTEORDER PERL_PACK_CAN_BYTEORDER has been unconditionally defined for versions 5.9.x and greater, and undefined for 5.8.x. As we are never going to need to port changes back to maint-5.8 any more, eliminate all the 5.8.x related code, and the macro that supports it. M pp_pack.c commit 39ecab1c690aee1a1fdeefb6c706448af5d2981f Merge: 6d5abc6 843b375 Author: Nicholas Clark Date: Mon May 20 21:12:01 2013 +0200 Merge the reworking of how genpacksizetables.pl is used. commit 843b37592a564e2e84c28a00619ac6c132c6dbc9 Author: Nicholas Clark Date: Mon Apr 29 12:06:39 2013 +0200 Add regen/genpacksizetables.pl to t/porting/regen.t M t/porting/regen.t commit e3f061af802f7db7c760d810c495d83763fff668 Author: Nicholas Clark Date: Mon Apr 29 11:54:19 2013 +0200 Move genpacksizetables.pl to regen/genpacksizetables.pl M MANIFEST D genpacksizetables.pl M packsizetables.c A regen/genpacksizetables.pl commit 6b05cb8115fc15fed0e6bf162a9cc797222572ee Author: Nicholas Clark Date: Mon Apr 29 11:47:30 2013 +0200 Refactor genpacksizetables.pl to use regen/regen_lib.pl M genpacksizetables.pl M packsizetables.c commit 298bc19c0935302faa4957de141245bb64c6b2df Author: Nicholas Clark Date: Mon Apr 29 11:23:11 2013 +0200 Extract the generated packprops array into packsizetables.c. The C source is the output from genpacksizetables.pl Previously it was pasted into pp_pack.c LocalWords: packprops M MANIFEST A packsizetables.c M pp_pack.c commit e2add64b70b6d5c55a6598257032bf0777c5976c Author: Nicholas Clark Date: Mon Apr 29 11:08:05 2013 +0200 Propagate a change from commit 1651fc447620d361 into genpacksizetables.pl. pp_pack.c contains a table generated by genpacksizetables.pl, pasted into the C source. The C source was updated by commit 1651fc447620d361 in April 2007, but the table used to generate the code was not. M genpacksizetables.pl commit d8a3828919c2d8a9e1344ba4d05795d37ed9cdb6 Author: Karl Williamson Date: Wed Mar 13 16:18:12 2013 -0600 genpacksizetables.pl: Correct comment typo M genpacksizetables.pl commit 6d5abc62848da0e110869e2b13c68bf498ff3a90 Author: Nicholas Clark Date: Wed Mar 6 16:27:08 2013 +0100 Change LIKELY() and UNLIKELY() to cast their first argument to a bool. This avoids problems such as pointers being compared with the literal value 1. Suggested by Zefram. Note that this is technically a change of behaviour, as the macro EXPECT(), which they are both both wrappers for, returns its value, so they will now be returning TRUE or FALSE, rather than the actual value passed to them. However, all uses in the core and on CPAN are only within if() statements, so this should not make any difference in practice. M perl.h ----------------------------------------------------------------------- -- Perl5 Master Repository [perl.git] branch nicholas/configpm, deleted. v5.17.0-612-g4cea661 by Nicholas Clark 2013-05-22T08:58:10Z From: Nicholas Clark In perl.git, the branch nicholas/configpm has been deleted was 4cea661a4dfd3760254ea67f29aef8e4094e7255 ----------------------------------------------------------------------- 4cea661a4dfd3760254ea67f29aef8e4094e7255 perlhacktips: add AddressSanitizer ----------------------------------------------------------------------- -- Perl5 Master Repository [perl.git] branch smoke-me/nicholas/S_scan_ident, deleted. v5.17.10-50-g5fd6a1f by Nicholas Clark 2013-05-22T08:54:27Z From: Nicholas Clark In perl.git, the branch smoke-me/nicholas/S_scan_ident has been deleted was 5fd6a1f52e77aaae4c3ed9b44061fca8731d857f ----------------------------------------------------------------------- 5fd6a1f52e77aaae4c3ed9b44061fca8731d857f Add comments describing the operation of S_scan_ident(). ----------------------------------------------------------------------- -- Perl5 Master Repository [perl.git] branch blead, updated. v5.19.0-129-gc5b6cc8 by Nicholas Clark 2013-05-22T08:54:19Z From: Nicholas Clark In perl.git, the branch blead has been updated - Log ----------------------------------------------------------------- commit c5b6cc8c74b4cfaa225d9702c4139816707e1999 Author: Nicholas Clark Date: Mon Apr 8 16:30:18 2013 +0200 Add comments describing the operation of S_scan_ident(). M toke.c commit 27ffb73a493be4fd4322b04f1bcbdabe6dff523b Author: Nicholas Clark Date: Mon Apr 8 16:27:19 2013 +0200 More tests for the "Ambiguous use of ... resolved ..." warning S_scan_ident(). In particular, test that the special case for reporting $# as @ works. M t/lib/warnings/toke ----------------------------------------------------------------------- Summary of changes: t/lib/warnings/toke | 15 +++++++++++++++ toke.c | 19 +++++++++++++++++++ 2 files changed, 34 insertions(+), 0 deletions(-) diff --git a/t/lib/warnings/toke b/t/lib/warnings/toke index 5ee3ad5..df2a0b4 100644 --- a/t/lib/warnings/toke +++ b/t/lib/warnings/toke @@ -425,10 +425,25 @@ EXPECT # toke.c use warnings 'ambiguous' ; $a = ${time} ; +$a = @{time} ; +$a = $#{time} ; # This one is special cased in toke.c +$a = %{time} ; +$a = *{time} ; +$a = defined &{time} ; # To avoid calling &::time no warnings 'ambiguous' ; $a = ${time} ; +$a = @{time} ; +$a = $#{time} ; # This one is special cased in toke.c +$a = %{time} ; +$a = *{time} ; +$a = defined &{time} ; # To avoid calling &::time EXPECT Ambiguous use of ${time} resolved to $time at - line 3. +Ambiguous use of @{time} resolved to @time at - line 4. +Ambiguous use of @{time} resolved to @time at - line 5. +Ambiguous use of %{time} resolved to %time at - line 6. +Ambiguous use of *{time} resolved to *time at - line 7. +Ambiguous use of &{time} resolved to &time at - line 8. ######## # toke.c use warnings 'ambiguous' ; diff --git a/toke.c b/toke.c index ff6308f..bca5d56 100644 --- a/toke.c +++ b/toke.c @@ -9300,6 +9300,8 @@ S_scan_ident(pTHX_ char *s, const char *send, char *dest, STRLEN destlen, I32 ck *d = '\0'; d = dest; if (*d) { + /* Either a digit variable, or parse_ident() found an identifier + (anything valid as a bareword), so job done and return. */ if (PL_lex_state != LEX_NORMAL) PL_lex_state = LEX_INTERPENDMAYBE; return s; @@ -9311,8 +9313,12 @@ S_scan_ident(pTHX_ char *s, const char *send, char *dest, STRLEN destlen, I32 ck || s[1] == '{' || strnEQ(s+1,"::",2)) ) { + /* Dereferencing a value in a scalar variable. + The alternatives are different syntaxes for a scalar variable. + Using ' as a leading package separator isn't allowed. :: is. */ return s; } + /* Handle the opening { of @{...}, &{...}, *{...}, %{...}, ${...} */ if (*s == '{') { bracket = s; s++; @@ -9339,20 +9345,29 @@ S_scan_ident(pTHX_ char *s, const char *send, char *dest, STRLEN destlen, I32 ck d[1] = '\0'; } } + /* Convert $^F, ${^F} and the ^F of ${^FOO} to control characters */ if (*d == '^' && *s && isCONTROLVAR(*s)) { *d = toCTRL(*s); s++; } + /* Warn about ambiguous code after unary operators if {...} notation isn't + used. There's no difference in ambiguity; it's merely a heuristic + about when not to warn. */ else if (ck_uni && !bracket) check_uni(); if (bracket) { + /* If we were processing {...} notation then... */ if (isIDFIRST_lazy_if(d,is_utf8)) { + /* if it starts as a valid identifier, assume that it is one. + (the later check for } being at the expected point will trap + cases where this doesn't pan out.) */ d += is_utf8 ? UTF8SKIP(d) : 1; parse_ident(&s, &d, e, 1, is_utf8); *d = '\0'; while (s < send && SPACE_OR_TAB(*s)) s++; if ((*s == '[' || (*s == '{' && strNE(dest, "sub")))) { + /* ${foo[0]} and ${foo{bar}} notation. */ if (ckWARN(WARN_AMBIGUOUS) && keyword(dest, d - dest, 0)) { const char * const brack = (const char *) @@ -9385,6 +9400,8 @@ S_scan_ident(pTHX_ char *s, const char *send, char *dest, STRLEN destlen, I32 ck while (s < send && SPACE_OR_TAB(*s)) s++; + /* Expect to find a closing } after consuming any trailing whitespace. + */ if (*s == '}') { s++; if (PL_lex_state == LEX_INTERPNORMAL && !PL_lex_brackets) { @@ -9407,6 +9424,8 @@ S_scan_ident(pTHX_ char *s, const char *send, char *dest, STRLEN destlen, I32 ck } } else { + /* Didn't find the closing } at the point we expected, so restore + state such that the next thing to process is the opening { and */ s = bracket; /* let the parser handle it */ *dest = '\0'; } -- Perl5 Master Repository [perl.git] branch blead, updated. v5.19.0-127-g5fb91d4 by Max Maischein 2013-05-21T19:41:24Z From: Max Maischein In perl.git, the branch blead has been updated - Log ----------------------------------------------------------------- commit 5fb91d484fee378b489da85e763d4ae7df3f2baa Author: Max Maischein Date: Tue May 21 21:40:21 2013 +0200 Fix typo in Porting/sync-with-cpan I will run `make test_porting` on the correct branch before submitting a patch. I will run `make test_porting` on the correct branch before submitting a patch. I will run `make test_porting` on the correct branch before submitting a patch. ----------------------------------------------------------------------- Summary of changes: Porting/sync-with-cpan | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/Porting/sync-with-cpan b/Porting/sync-with-cpan index ca3bedf..68e22ca 100755 --- a/Porting/sync-with-cpan +++ b/Porting/sync-with-cpan @@ -230,7 +230,7 @@ say "Cleaning out old directory"; system git => 'clean', '-dfxq', $pkg_dir; say "Unpacking $new_file"; -Archive::Tar->extract_archive( $newfile ); +Archive::Tar->extract_archive( $new_file ); (my $new_dir = $new_file) =~ s/\.tar\.gz//; # ensure 'make' will update all files -- Perl5 Master Repository [perl.git] branch blead, updated. v5.19.0-126-ga1450e8 by David Golden 2013-05-21T17:59:07Z From: David Golden In perl.git, the branch blead has been updated - Log ----------------------------------------------------------------- commit a1450e8bf79c3c2d21815edb107c092b823221fd Author: Max Maischein Date: Tue May 21 19:09:47 2013 +0200 Eliminate external call to tar executable in favour of Archive::Tar Archive::Tar has ->extract_archive, which does Just That. This means we lose immediate support for bzip2 files, but not all incarnations of `tar` support it either. Signed-off-by: David Golden ----------------------------------------------------------------------- Summary of changes: Porting/sync-with-cpan | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Porting/sync-with-cpan b/Porting/sync-with-cpan index b6ee246..ca3bedf 100755 --- a/Porting/sync-with-cpan +++ b/Porting/sync-with-cpan @@ -127,6 +127,7 @@ use 5.010; use strict; use warnings; use Getopt::Long; +use Archive::Tar; $| = 1; @@ -229,8 +230,8 @@ say "Cleaning out old directory"; system git => 'clean', '-dfxq', $pkg_dir; say "Unpacking $new_file"; +Archive::Tar->extract_archive( $newfile ); -system tar => 'xfz', $new_file; (my $new_dir = $new_file) =~ s/\.tar\.gz//; # ensure 'make' will update all files system('find', $new_dir, '-exec', 'touch', '{}', ';'); -- Perl5 Master Repository [perl.git] branch blead, updated. v5.19.0-125-gb6956a9 by Chris 'Bingos' Williams 2013-05-21T13:18:59Z From: Chris 'Bingos' Williams In perl.git, the branch blead has been updated - Log ----------------------------------------------------------------- commit b6956a94c03907bb423a113afad54a0e343f44cf Author: Chris 'BinGOs' Williams Date: Tue May 21 14:13:52 2013 +0100 Add new Module::CoreList::Utils to MANIFEST ----------------------------------------------------------------------- Summary of changes: MANIFEST | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/MANIFEST b/MANIFEST index 32b958c..72fcff9 100644 --- a/MANIFEST +++ b/MANIFEST @@ -3209,6 +3209,7 @@ dist/Module-CoreList/identify-dependencies A usage example for Module::CoreList dist/Module-CoreList/lib/Module/CoreList.pm Module::CoreList dist/Module-CoreList/lib/Module/CoreList.pod Module::CoreList dist/Module-CoreList/lib/Module/CoreList/TieHashDelta.pm Module::CoreList guts +dist/Module-CoreList/lib/Module/CoreList/Utils.pm Module::CoreList guts dist/Module-CoreList/Makefile.PL Module::CoreList dist/Module-CoreList/MANIFEST Module::CoreList dist/Module-CoreList/README Module::CoreList @@ -3216,6 +3217,7 @@ dist/Module-CoreList/t/corelist.t Module::CoreList tests dist/Module-CoreList/t/deprecated.t Module::CoreList tests dist/Module-CoreList/t/find_modules.t Module::CoreList tests dist/Module-CoreList/t/pod.t Module::CoreList tests +dist/Module-CoreList/t/utils.t Module::CoreList tests dist/Net-Ping/Changes Net::Ping dist/Net-Ping/lib/Net/Ping.pm Hello, anybody home? dist/Net-Ping/t/100_load.t Ping Net::Ping -- Perl5 Master Repository [perl.git] branch blead, updated. v5.19.0-124-geab82fd by Chris 'Bingos' Williams 2013-05-21T12:36:11Z From: Chris 'Bingos' Williams In perl.git, the branch blead has been updated - Log ----------------------------------------------------------------- commit eab82fd6486ac0971d2f1b8800432a3b8f0b315c Merge: b4b2fef 5730025 Author: Chris 'BinGOs' Williams Date: Tue May 21 13:32:53 2013 +0100 Merge branch 'smoke-me/coreutils' into blead ----------------------------------------------------------------------- Summary of changes: dist/Module-CoreList/lib/Module/CoreList/Utils.pm | 854 +++++++++++++++++++++ dist/Module-CoreList/t/utils.t | 23 + 2 files changed, 877 insertions(+), 0 deletions(-) create mode 100644 dist/Module-CoreList/lib/Module/CoreList/Utils.pm create mode 100644 dist/Module-CoreList/t/utils.t diff --git a/dist/Module-CoreList/lib/Module/CoreList/Utils.pm b/dist/Module-CoreList/lib/Module/CoreList/Utils.pm new file mode 100644 index 0000000..a04ef6c --- /dev/null +++ b/dist/Module-CoreList/lib/Module/CoreList/Utils.pm @@ -0,0 +1,854 @@ +package Module::CoreList::Utils; + +use strict; +use warnings; +use vars qw[$VERSION %utilities]; +use Module::CoreList; +use Module::CoreList::TieHashDelta; + +$VERSION = '2.91'; + +sub utilities { + my $perl = shift; + $perl = shift if eval { $perl->isa(__PACKAGE__) }; + return unless $perl or exists $utilities{$perl}; + return sort keys %{ $utilities{$perl} }; +} + +sub first_release_raw { + my $util = shift; + $util = shift if eval { $util->isa(__PACKAGE__) }; + #and scalar @_ and $_[0] =~ m#\A[a-zA-Z_][0-9a-zA-Z_]*(?:(::|')[0-9a-zA-Z_]+)*\z#; + my $version = shift; + + my @perls = $version + ? grep { exists $utilities{$_}{ $util } && + $utilities{$_}{ $util } ge $version } keys %utilities + : grep { exists $utilities{$_}{ $util } } keys %utilities; + + return grep { exists $Module::CoreList::released{$_} } @perls; +} + +sub first_release_by_date { + my @perls = &first_release_raw; + return unless @perls; + return (sort { $Module::CoreList::released{$a} cmp $Module::CoreList::released{$b} } @perls)[0]; +} + +sub first_release { + my @perls = &first_release_raw; + return unless @perls; + return (sort { $a cmp $b } @perls)[0]; +} + +sub removed_from { + my @perls = &removed_raw; + return shift @perls; +} + +sub removed_from_by_date { + my @perls = sort { $Module::CoreList::released{$a} cmp $Module::CoreList::released{$b} } &removed_raw; + return shift @perls; +} + +sub removed_raw { + my $util = shift; + $util = shift if eval { $util->isa(__PACKAGE__) }; + return unless my @perls = sort { $a cmp $b } first_release_raw($util); + @perls = grep { exists $Module::CoreList::released{$_} } @perls; + my $last = pop @perls; + my @removed = grep { $_ > $last } sort { $a cmp $b } keys %utilities; + return @removed; +} + +my %delta = ( + 5 => { + changed => { + 'a2p' => '1', + 'c2ph' => '1', + 'cppstdin' => '1', + 'find2perl' => '1', + 'pstruct' => '1', + 's2p' => '1', + }, + removed => { + } + }, + + 5.001 => { + delta_from => 5, + changed => { + 'h2xs' => '1', + }, + removed => { + } + }, + + 5.002 => { + delta_from => 5.001, + changed => { + 'h2ph' => '1', + 'perlbug' => '1', + 'perldoc' => '1', + 'pod2html' => '1', + 'pod2latex' => '1', + 'pod2man' => '1', + 'pod2text' => '1', + }, + removed => { + } + }, + + 5.00307 => { + delta_from => 5.002, + changed => { + 'pl2pm' => '1', + }, + removed => { + 'cppstdin' => 1, + 'pstruct' => 1, + } + }, + + 5.004 => { + delta_from => 5.00307, + changed => { + 'splain' => '1', + }, + removed => { + } + }, + + 5.005 => { + delta_from => 5.00405, + changed => { + 'perlcc' => '1', + }, + removed => { + } + }, + + 5.00503 => { + delta_from => 5.005, + changed => { + }, + removed => { + } + }, + + 5.00405 => { + delta_from => 5.004, + changed => { + }, + removed => { + } + }, + + 5.006 => { + delta_from => 5.00504, + changed => { + 'dprofpp' => '1', + 'pod2usage' => '1', + 'podchecker' => '1', + 'podselect' => '1', + 'pstruct' => '1', + }, + removed => { + } + }, + + 5.006001 => { + delta_from => 5.006, + changed => { + }, + removed => { + } + }, + + 5.007003 => { + delta_from => 5.006002, + changed => { + 'libnetcfg' => '1', + 'perlivp' => '1', + 'psed' => '1', + 'xsubpp' => '1', + }, + removed => { + } + }, + + 5.008 => { + delta_from => 5.007003, + changed => { + 'enc2xs' => '1', + 'piconv' => '1', + }, + removed => { + } + }, + + 5.008001 => { + delta_from => 5.008, + changed => { + 'cpan' => '1', + }, + removed => { + } + }, + + 5.009 => { + delta_from => 5.008009, + changed => { + }, + removed => { + 'corelist' => 1, + 'instmodsh' => 1, + 'prove' => 1, + } + }, + + 5.008002 => { + delta_from => 5.008001, + changed => { + }, + removed => { + } + }, + + 5.006002 => { + delta_from => 5.006001, + changed => { + }, + removed => { + } + }, + + 5.008003 => { + delta_from => 5.008002, + changed => { + 'instmodsh' => '1', + 'prove' => '1', + }, + removed => { + } + }, + + 5.00504 => { + delta_from => 5.00503, + changed => { + }, + removed => { + } + }, + + 5.009001 => { + delta_from => 5.009, + changed => { + 'instmodsh' => '1', + 'prove' => '1', + }, + removed => { + } + }, + + 5.008004 => { + delta_from => 5.008003, + changed => { + }, + removed => { + } + }, + + 5.008005 => { + delta_from => 5.008004, + changed => { + }, + removed => { + } + }, + + 5.008006 => { + delta_from => 5.008005, + changed => { + }, + removed => { + } + }, + + 5.009002 => { + delta_from => 5.009001, + changed => { + 'corelist' => '1', + }, + removed => { + } + }, + + 5.008007 => { + delta_from => 5.008006, + changed => { + }, + removed => { + } + }, + + 5.009003 => { + delta_from => 5.009002, + changed => { + 'ptar' => '1', + 'ptardiff' => '1', + 'shasum' => '1', + }, + removed => { + } + }, + + 5.008008 => { + delta_from => 5.008007, + changed => { + }, + removed => { + } + }, + + 5.009004 => { + delta_from => 5.009003, + changed => { + 'config_data' => '1', + }, + removed => { + } + }, + + 5.009005 => { + delta_from => 5.009004, + changed => { + 'cpan2dist' => '1', + 'cpanp' => '1', + 'cpanp-run-perl' => '1', + }, + removed => { + 'perlcc' => 1, + } + }, + + 5.010000 => { + delta_from => 5.009005, + changed => { + }, + removed => { + } + }, + + 5.008009 => { + delta_from => 5.008008, + changed => { + 'corelist' => '1', + }, + removed => { + } + }, + + 5.010001 => { + delta_from => 5.010000, + changed => { + }, + removed => { + } + }, + + 5.011 => { + delta_from => 5.010001, + changed => { + }, + removed => { + } + }, + + 5.011001 => { + delta_from => 5.011, + changed => { + }, + removed => { + } + }, + + 5.011002 => { + delta_from => 5.011001, + changed => { + 'perlthanks' => '1', + }, + removed => { + } + }, + + 5.011003 => { + delta_from => 5.011002, + changed => { + }, + removed => { + } + }, + + 5.011004 => { + delta_from => 5.011003, + changed => { + }, + removed => { + } + }, + + 5.011005 => { + delta_from => 5.011004, + changed => { + }, + removed => { + } + }, + + 5.012 => { + delta_from => 5.011005, + changed => { + }, + removed => { + } + }, + + 5.013 => { + delta_from => 5.012005, + changed => { + }, + removed => { + } + }, + + 5.012001 => { + delta_from => 5.012, + changed => { + }, + removed => { + } + }, + + 5.013001 => { + delta_from => 5.013, + changed => { + }, + removed => { + } + }, + + 5.013002 => { + delta_from => 5.013001, + changed => { + }, + removed => { + } + }, + + 5.013003 => { + delta_from => 5.013002, + changed => { + }, + removed => { + } + }, + + 5.013004 => { + delta_from => 5.013003, + changed => { + }, + removed => { + } + }, + + 5.012002 => { + delta_from => 5.012001, + changed => { + }, + removed => { + } + }, + + 5.013005 => { + delta_from => 5.013004, + changed => { + }, + removed => { + } + }, + + 5.013006 => { + delta_from => 5.013005, + changed => { + }, + removed => { + } + }, + + 5.013007 => { + delta_from => 5.013006, + changed => { + 'ptargrep' => '1', + }, + removed => { + } + }, + + 5.013008 => { + delta_from => 5.013007, + changed => { + }, + removed => { + } + }, + + 5.013009 => { + delta_from => 5.013008, + changed => { + 'json_pp' => '1', + }, + removed => { + } + }, + + 5.012003 => { + delta_from => 5.012002, + changed => { + }, + removed => { + } + }, + + 5.013010 => { + delta_from => 5.013009, + changed => { + }, + removed => { + } + }, + + 5.013011 => { + delta_from => 5.013010, + changed => { + }, + removed => { + } + }, + + 5.014 => { + delta_from => 5.013011, + changed => { + }, + removed => { + } + }, + + 5.014001 => { + delta_from => 5.014, + changed => { + }, + removed => { + } + }, + + 5.015 => { + delta_from => 5.014004, + changed => { + }, + removed => { + 'dprofpp' => 1, + } + }, + + 5.012004 => { + delta_from => 5.012003, + changed => { + }, + removed => { + } + }, + + 5.015001 => { + delta_from => 5.015, + changed => { + }, + removed => { + } + }, + + 5.015002 => { + delta_from => 5.015001, + changed => { + }, + removed => { + } + }, + + 5.015003 => { + delta_from => 5.015002, + changed => { + }, + removed => { + } + }, + + 5.014002 => { + delta_from => 5.014001, + changed => { + }, + removed => { + } + }, + + 5.015004 => { + delta_from => 5.015003, + changed => { + }, + removed => { + } + }, + + 5.015005 => { + delta_from => 5.015004, + changed => { + }, + removed => { + } + }, + + 5.015006 => { + delta_from => 5.015005, + changed => { + 'zipdetails' => '1', + }, + removed => { + } + }, + + 5.015007 => { + delta_from => 5.015006, + changed => { + }, + removed => { + } + }, + + 5.015008 => { + delta_from => 5.015007, + changed => { + }, + removed => { + } + }, + + 5.015009 => { + delta_from => 5.015008, + changed => { + }, + removed => { + } + }, + + 5.016 => { + delta_from => 5.015009, + changed => { + }, + removed => { + } + }, + + 5.017 => { + delta_from => 5.016003, + changed => { + }, + removed => { + } + }, + + 5.017001 => { + delta_from => 5.017, + changed => { + }, + removed => { + } + }, + + 5.017002 => { + delta_from => 5.017001, + changed => { + }, + removed => { + } + }, + + 5.016001 => { + delta_from => 5.016, + changed => { + }, + removed => { + } + }, + + 5.017003 => { + delta_from => 5.017002, + changed => { + }, + removed => { + } + }, + + 5.017004 => { + delta_from => 5.017003, + changed => { + }, + removed => { + } + }, + + 5.014003 => { + delta_from => 5.014002, + changed => { + }, + removed => { + } + }, + + 5.017005 => { + delta_from => 5.017004, + changed => { + }, + removed => { + } + }, + + 5.016002 => { + delta_from => 5.016001, + changed => { + }, + removed => { + } + }, + + 5.012005 => { + delta_from => 5.012004, + changed => { + }, + removed => { + } + }, + + 5.017006 => { + delta_from => 5.017005, + changed => { + }, + removed => { + } + }, + + 5.017007 => { + delta_from => 5.017006, + changed => { + }, + removed => { + } + }, + + 5.017008 => { + delta_from => 5.017007, + changed => { + }, + removed => { + } + }, + + 5.017009 => { + delta_from => 5.017008, + changed => { + }, + removed => { + } + }, + + 5.014004 => { + delta_from => 5.014003, + changed => { + }, + removed => { + } + }, + + 5.016003 => { + delta_from => 5.016002, + changed => { + }, + removed => { + } + }, + + 5.017010 => { + delta_from => 5.017009, + changed => { + }, + removed => { + } + }, + + 5.017011 => { + delta_from => 5.017010, + changed => { + }, + removed => { + } + }, + 5.018000 => { + delta_from => 5.017011, + changed => { + }, + removed => { + } + }, + 5.019000 => { + delta_from => 5.018000, + changed => { + }, + removed => { + 'cpan2dist' => '1', + 'cpanp' => '1', + 'cpanp-run-perl' => '1', + 'pod2latex' => '1', + } + }, +); + +for my $version (sort { $a <=> $b } keys %delta) { + my $data = $delta{$version}; + + tie %{$utilities{$version}}, 'Module::CoreList::TieHashDelta', + $data->{changed}, $data->{removed}, + $data->{delta_from} ? $utilities{$data->{delta_from}} : undef; +} + +# Create aliases with trailing zeros for $] use + +$utilities{'5.000'} = $utilities{5}; + +_create_aliases(\%utilities); + +sub _create_aliases { + my ($hash) = @_; + + for my $version (keys %$hash) { + next unless $version >= 5.010; + + my $padded = sprintf "%0.6f", $version; + + # If the version in string form isn't the same as the numeric version, + # alias it. + if ($padded ne $version && $version == $padded) { + $hash->{$padded} = $hash->{$version}; + } + } +} + +'foo'; diff --git a/dist/Module-CoreList/t/utils.t b/dist/Module-CoreList/t/utils.t new file mode 100644 index 0000000..4822495 --- /dev/null +++ b/dist/Module-CoreList/t/utils.t @@ -0,0 +1,23 @@ +use strict; +use warnings; +use Test::More tests => 9; + +BEGIN { require_ok('Module::CoreList::Utils'); } + +ok( defined $Module::CoreList::Utils::utilities{5}{a2p}, '5 had a2p' ); +is( Module::CoreList::Utils->first_release('a2p'), 5, 'a2p first released in 5' ); +is( Module::CoreList::Utils::first_release('a2p'), 5, 'a2p first released in 5' ); +is( Module::CoreList::Utils->first_release('corelist'), 5.008009, 'corelist with v5.8.9'); +is( Module::CoreList::Utils->first_release_by_date('corelist'), 5.009002, 'corelist with v5.9.2'); +is( Module::CoreList::Utils::first_release_by_date('corelist'), 5.009002, 'corelist with v5.9.2'); +{ + my @expected = qw(a2p c2ph cppstdin find2perl h2xs pstruct s2p); + { + my @foo = Module::CoreList::Utils->utilities(5.001); + is_deeply( \@foo, \@expected, '5.001 utils all present and correct' ); + } + { + my @foo = Module::CoreList::Utils::utilities(5.001); + is_deeply( \@foo, \@expected, '5.001 utils all present and correct' ); + } +} -- Perl5 Master Repository [perl.git] branch smoke-me/nicholas/ExtUtils-CBuilder-Base-prune, deleted. v5.17.11-22-g82adae2 by Nicholas Clark 2013-05-21T10:32:32Z From: Nicholas Clark In perl.git, the branch smoke-me/nicholas/ExtUtils-CBuilder-Base-prune has been deleted was 82adae220e83c4202eabe01f49eeb4844d271715 ----------------------------------------------------------------------- 82adae220e83c4202eabe01f49eeb4844d271715 Remove 2 superfluous use lines from ExtUtils::CBuilder. ----------------------------------------------------------------------- -- Perl5 Master Repository [perl.git] branch blead, updated. v5.19.0-113-gb4b2fef by Nicholas Clark 2013-05-21T10:32:26Z From: Nicholas Clark In perl.git, the branch blead has been updated - Log ----------------------------------------------------------------- commit b4b2fefd8bdd72b0f88245101f270e9dc39568f6 Author: Nicholas Clark Date: Wed Apr 24 17:01:43 2013 +0200 Remove 2 superfluous use lines from ExtUtils::CBuilder. use IO::File was added with the upgrade to 0.23 (commit c3fb68a339256eb3 in April 2008), but the use of IO::File in the code was eliminated by the upgrade to 0.2800 (commit 06e8058f27e4269b in Dec 2010), which replaced the code in question with use of File::Temp. The latter refactoring also added the use Data::Dumper; line, but did not add any code which uses Data::Dumper. ----------------------------------------------------------------------- Summary of changes: .../lib/ExtUtils/CBuilder/Base.pm | 4 +--- 1 files changed, 1 insertions(+), 3 deletions(-) diff --git a/dist/ExtUtils-CBuilder/lib/ExtUtils/CBuilder/Base.pm b/dist/ExtUtils-CBuilder/lib/ExtUtils/CBuilder/Base.pm index c76489b..479a787 100644 --- a/dist/ExtUtils-CBuilder/lib/ExtUtils/CBuilder/Base.pm +++ b/dist/ExtUtils-CBuilder/lib/ExtUtils/CBuilder/Base.pm @@ -6,13 +6,11 @@ use File::Basename; use Cwd (); use Config; use Text::ParseWords; -use IO::File; -use Data::Dumper;$Data::Dumper::Indent=1; use IPC::Cmd qw(can_run); use File::Temp qw(tempfile); use vars qw($VERSION); -$VERSION = '0.280209'; +$VERSION = '0.280210'; # More details about C/C++ compilers: # http://developers.sun.com/sunstudio/documentation/product/compiler.jsp -- Perl5 Master Repository [perl.git] branch blead, updated. v5.19.0-112-gfdf38e4 by Nicholas Clark 2013-05-21T10:19:42Z From: Nicholas Clark In perl.git, the branch blead has been updated - Log ----------------------------------------------------------------- commit fdf38e49aa470e60cb56fdd7a3a49f85da8ea2ce Author: Nicholas Clark Date: Wed Apr 10 13:44:07 2013 +0200 Refactor 3 tests so that they avoid clearing %:: via list assignment. For all 3, clearing %:: was intended to expose another bug, not to directly test the effects of clearing %:: Tested by building the 3 revisions that added the tests, and confirming that the revised test also triggers the bug that each fixed. ----------------------------------------------------------------------- Summary of changes: t/op/stash.t | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) diff --git a/t/op/stash.t b/t/op/stash.t index fd5450e..2681d47 100644 --- a/t/op/stash.t +++ b/t/op/stash.t @@ -11,7 +11,7 @@ plan( tests => 58 ); # Used to segfault (bug #15479) fresh_perl_like( - '%:: = ""', + 'delete $::{STDERR}; my %a = ""', qr/Odd number of elements in hash assignment at - line 1\./, { switches => [ '-w' ] }, 'delete $::{STDERR} and print a warning', @@ -59,14 +59,15 @@ package main; { local $ENV{PERL_DESTRUCT_LEVEL} = 2; fresh_perl_is( - 'package A; sub a { // }; %::=""', + 'package A::B; sub a { // }; %A::=""', '', '', ); # Variant of the above which creates an object that persists until global - # destruction. + # destruction, and triggers an assertion failure prior to change + # a420522db95b7762 fresh_perl_is( - 'use Exporter; package A; sub a { // }; %::=""', + 'use Exporter; package A; sub a { // }; delete $::{$_} for keys %::', '', '', ); -- Perl5 Master Repository [perl.git] branch smoke-me/nicholas/RT54044, deleted. v5.17.10-49-gcffb726 by Nicholas Clark 2013-05-21T10:19:40Z From: Nicholas Clark In perl.git, the branch smoke-me/nicholas/RT54044 has been deleted was cffb726fc69b7e9513c76a479db79979ff833be3 ----------------------------------------------------------------------- cffb726fc69b7e9513c76a479db79979ff833be3 Refactor 3 tests so that they avoid clearing %:: via list assignment. ----------------------------------------------------------------------- -- Perl5 Master Repository [perl.git] branch smoke-me/nicholas/caller-undef-package, deleted. v5.17.10-46-g4ac62a0 by Nicholas Clark 2013-05-21T09:58:35Z From: Nicholas Clark In perl.git, the branch smoke-me/nicholas/caller-undef-package has been deleted was 4ac62a04d6ee7e388129dca0977c6e908f4a43ff ----------------------------------------------------------------------- 4ac62a04d6ee7e388129dca0977c6e908f4a43ff Test that caller does not SEGV when the current package is undefined. ----------------------------------------------------------------------- -- Perl5 Master Repository [perl.git] branch blead, updated. v5.19.0-111-gaee674b by Nicholas Clark 2013-05-21T09:58:19Z From: Nicholas Clark In perl.git, the branch blead has been updated - Log ----------------------------------------------------------------- commit aee674b7db1a4a3fd0082f4247560d3c945d0f5c Author: Nicholas Clark Date: Fri Apr 5 12:12:53 2013 +0200 Test that caller does not SEGV when the current package is undefined. Prior to commit d4d03940c58a0177 this code would SEGV under ithreads. Test suggested by Brian Fraser. ----------------------------------------------------------------------- Summary of changes: t/op/caller.t | 10 +++++++++- 1 files changed, 9 insertions(+), 1 deletions(-) diff --git a/t/op/caller.t b/t/op/caller.t index efce8df..c37a6ed 100644 --- a/t/op/caller.t +++ b/t/op/caller.t @@ -5,7 +5,7 @@ BEGIN { chdir 't' if -d 't'; @INC = '../lib'; require './test.pl'; - plan( tests => 91 ); + plan( tests => 92 ); } my @c; @@ -297,6 +297,14 @@ is eval "s//< is $w, 1, 'value from (caller 0)[9] (bitmask) works in ${^WARNING_BITS}'; } +# This was fixed with commit d4d03940c58a0177, which fixed bug #78742 +fresh_perl_is <<'END', "__ANON__::doof\n", {}, +package foo; +BEGIN {undef %foo::} +sub doof { caller(0) } +print +(doof())[3]; +END + "caller should not SEGV when the current package is undefined"; $::testing_caller = 1; do './op/caller.pl' or die $@; -- Perl5 Master Repository [perl.git] branch smoke-me/nicholas/fakethr, deleted. v5.19.0-28-gb4c0512 by Nicholas Clark 2013-05-21T07:15:30Z From: Nicholas Clark In perl.git, the branch smoke-me/nicholas/fakethr has been deleted was b4c05129f48e705ff8e033db447ab1d4e22b832d ----------------------------------------------------------------------- b4c05129f48e705ff8e033db447ab1d4e22b832d Remove fakethr.h and eliminate all references to it and FAKE_THREADS ----------------------------------------------------------------------- -- Perl5 Master Repository [perl.git] branch blead, updated. v5.19.0-110-gef4abe7 by Nicholas Clark 2013-05-21T07:15:25Z From: Nicholas Clark In perl.git, the branch blead has been updated - Log ----------------------------------------------------------------- commit ef4abe7673cc0b19ffd81c9a42f79de20a52ac87 Author: Nicholas Clark Date: Mon May 20 17:55:46 2013 +0200 Remove fakethr.h and eliminate all references to it and FAKE_THREADS fakethr.h and FAKE_THREADS were for a "green" threads implementation of 5005threads. 5005threads itself is long gone, and it's not clear that -DFAKE_THREADS *ever* built correctly. Certainly it did not work for the 5.005 release, and it did not work at the time of the commits for the initial checkin. The closest that it seems to have been to working is around commit c6ee37c52f2ca9e5 (Dec 1997), where the headers no longer contained errors, but perl.c failed to compile. ----------------------------------------------------------------------- Summary of changes: MANIFEST | 1 - configpm | 1 - dist/threads-shared/lib/threads/shared.pm | 2 +- dist/threads-shared/shared.xs | 4 +- fakethr.h | 75 ----------------------------- perl.h | 9 +--- vms/descrip_mms.template | 13 +---- 7 files changed, 6 insertions(+), 99 deletions(-) delete mode 100644 fakethr.h diff --git a/MANIFEST b/MANIFEST index 5916b98..32b958c 100644 --- a/MANIFEST +++ b/MANIFEST @@ -3882,7 +3882,6 @@ ext/XS-Typemap/t/Typemap.t test that typemaps work ext/XS-Typemap/Typemap.pm XS::Typemap extension ext/XS-Typemap/Typemap.xs XS::Typemap extension fakesdio.h stdio in terms of PerlIO -fakethr.h Fake threads header feature.h Feature header form.h Public declarations for formats generate_uudmap.c Generate uudmap.h, the uuencode decoding map diff --git a/configpm b/configpm index 0be40f9..33137c6 100755 --- a/configpm +++ b/configpm @@ -82,7 +82,6 @@ my @header_files = qw(EXTERN.h INTERN.h XSUB.h av.h config.h cop.h cv.h regcomp.h regexp.h regnodes.h scope.h sv.h thread.h utf8.h util.h); -# No point in adding fakethr.h, as it no longer works push @header_files, $^O eq 'VMS' ? 'vmsish.h' : qw(dosish.h perliol.h time64.h unixish.h); diff --git a/dist/threads-shared/lib/threads/shared.pm b/dist/threads-shared/lib/threads/shared.pm index ecbc1fd..e5c3669 100644 --- a/dist/threads-shared/lib/threads/shared.pm +++ b/dist/threads-shared/lib/threads/shared.pm @@ -7,7 +7,7 @@ use warnings; use Scalar::Util qw(reftype refaddr blessed); -our $VERSION = '1.43'; +our $VERSION = '1.44'; my $XS_VERSION = $VERSION; $VERSION = eval $VERSION; diff --git a/dist/threads-shared/shared.xs b/dist/threads-shared/shared.xs index 3dccc39..d3e859d 100644 --- a/dist/threads-shared/shared.xs +++ b/dist/threads-shared/shared.xs @@ -612,7 +612,7 @@ S_abs_2_rel_milli(double abs) bool Perl_sharedsv_cond_timedwait(perl_cond *cond, perl_mutex *mut, double abs) { -#if defined(NETWARE) || defined(FAKE_THREADS) || defined(I_MACH_CTHREADS) +#if defined(NETWARE) || defined(I_MACH_CTHREADS) Perl_croak_nocontext("cond_timedwait not supported on this platform"); #else # ifdef WIN32 @@ -671,7 +671,7 @@ Perl_sharedsv_cond_timedwait(perl_cond *cond, perl_mutex *mut, double abs) return (got_it); # endif /* OS2 */ # endif /* WIN32 */ -#endif /* NETWARE || FAKE_THREADS || I_MACH_CTHREADS */ +#endif /* NETWARE || I_MACH_CTHREADS */ } diff --git a/fakethr.h b/fakethr.h deleted file mode 100644 index 643806d..0000000 --- a/fakethr.h +++ /dev/null @@ -1,75 +0,0 @@ -/* fakethr.h - * - * Copyright (C) 1999, by Larry Wall and others - * - * You may distribute under the terms of either the GNU General Public - * License or the Artistic License, as specified in the README file. - * - */ - -typedef int perl_mutex; -typedef int perl_key; - -typedef struct perl_thread *perl_os_thread; -/* With fake threads, thr is global(ish) so we don't need dTHR */ -#define dTHR extern int errno - -struct perl_wait_queue { - struct perl_thread * thread; - struct perl_wait_queue * next; -}; -typedef struct perl_wait_queue *perl_cond; - -/* Ask thread.h to include our per-thread extras */ -#define HAVE_THREAD_INTERN -struct thread_intern { - perl_os_thread next_run, prev_run; /* Linked list of runnable threads */ - perl_cond wait_queue; /* Wait queue that we are waiting on */ - IV private; /* Holds data across time slices */ - I32 savemark; /* Holds MARK for thread join values */ -}; - -#define init_thread_intern(t) \ - STMT_START { \ - t->self = (t); \ - (t)->i.next_run = (t)->i.prev_run = (t); \ - (t)->i.wait_queue = 0; \ - (t)->i.private = 0; \ - } STMT_END - -/* - * Note that SCHEDULE() is only callable from pp code (which - * must be expecting to be restarted). We'll have to do - * something a bit different for XS code. - */ - -#define SCHEDULE() return schedule(), PL_op - -#define MUTEX_LOCK(m) -#define MUTEX_UNLOCK(m) -#define MUTEX_INIT(m) -#define MUTEX_DESTROY(m) -#define COND_INIT(c) perl_cond_init(c) -#define COND_SIGNAL(c) perl_cond_signal(c) -#define COND_BROADCAST(c) perl_cond_broadcast(c) -#define COND_WAIT(c, m) \ - STMT_START { \ - perl_cond_wait(c); \ - SCHEDULE(); \ - } STMT_END -#define COND_DESTROY(c) - -#define THREAD_CREATE(t, f) f((t)) -#define THREAD_POST_CREATE(t) NOOP - -#define YIELD NOOP - -/* - * Local variables: - * c-indentation-style: bsd - * c-basic-offset: 4 - * indent-tabs-mode: nil - * End: - * - * ex: set ts=8 sts=4 sw=4 et: - */ diff --git a/perl.h b/perl.h index 87e7f0f..1f6e4e4 100644 --- a/perl.h +++ b/perl.h @@ -2670,9 +2670,6 @@ freeing any remaining Perl interpreters. # ifdef NETWARE # include # else -# ifdef FAKE_THREADS -# include "fakethr.h" -# else # ifdef WIN32 # include # else @@ -2699,8 +2696,7 @@ typedef pthread_key_t perl_key; # endif /* I_MACH_CTHREADS */ # endif /* OS2 */ # endif /* WIN32 */ -# endif /* FAKE_THREADS */ -#endif /* NETWARE */ +# endif /* NETWARE */ #endif /* USE_ITHREADS */ #if defined(WIN32) @@ -4513,9 +4509,6 @@ EXTCONST char PL_bincompat_options[] = # ifdef DEBUG_LEAKING_SCALARS_FORK_DUMP " DEBUG_LEAKING_SCALARS_FORK_DUMP" # endif -# ifdef FAKE_THREADS - " FAKE_THREADS" -# endif # ifdef FCRYPT " FCRYPT" # endif diff --git a/vms/descrip_mms.template b/vms/descrip_mms.template index 20da73c..af27c3c 100644 --- a/vms/descrip_mms.template +++ b/vms/descrip_mms.template @@ -151,14 +151,6 @@ THRLIBS2 = sys$share:cma$open_lib_shr/share|sys$share:cma$open_rtl/share .endif .endif -.ifdef FAKETHREADED -THREADDEF = -THREADH = fakethr.h -THREAD = THREAD -.else -THREADH = -.endif - .ifdef MTU .ifdef MTK MTHREADLINKFLAGS = /THREADS=(MULTIPLE_KERNEL,UPCALLS) @@ -255,9 +247,8 @@ h2 = iperlsys.h keywords.h mydtrace.h mg.h mg_vtable.h nostdio.h op.h h3 = op_reg_common.h opcode.h opnames.h overload.h pad.h parser.h patchlevel.h h4 = perl.h perlapi.h perlio.h perlsdio.h perlvars.h perly.h h5 = pp.h pp_proto.h proto.h regcomp.h regexp.h regnodes.h scope.h -h6 = sv.h thread.h utf8.h util.h vmsish.h warnings.h -h7 = xsub.h $(THREADH) -h = $(h0) $(h1) $(h2) $(h3) $(h4) $(h5) $(h6) $(h7) +h6 = sv.h thread.h utf8.h util.h vmsish.h warnings.h xsub.h +h = $(h0) $(h1) $(h2) $(h3) $(h4) $(h5) $(h6) acopt = $(ARCHCORE)perlshr_attr.opt $(ARCHCORE)$(DBG)perlshr_bld.opt ac = archcore_includes.ts $(acopt) -- Perl5 Master Repository [perl.git] branch blead, updated. v5.19.0-109-g31a53e7 by Nicholas Clark 2013-05-21T05:56:16Z From: Nicholas Clark In perl.git, the branch blead has been updated - Log ----------------------------------------------------------------- commit 31a53e79025eb5944553e671a615e60f384fe743 Merge: 9e7945e 8ca91d3 Author: Nicholas Clark Date: Tue May 21 07:49:29 2013 +0200 Merge the refactoring of x2p's Makefile to blead. commit 8ca91d39a4a96434bc909d710b033c8505e2cde3 Author: Nicholas Clark Date: Wed May 15 21:48:12 2013 +0200 Simplify x2p's Makefile. Eliminate plexe and plc, which relate to tests for the compiler and were added as part of commit 52cebf5efc9883c7 in May 1988. The compiler was removed by commit de1254415ffeb03b in Sept 2006. The files a2p.man and s2p.man were removed with Perl 5.000. The macros 'public', 'private' and 'util' mirror the structure of the top level Makefile.SH. However, x2p only ever used 'public', so simplify things by inlining the value of 'public' in the only place that uses it. 'addedbyconf' now only contains the value of 'plextract', so use the latter directly in the one place that had used 'addedbyconf'. M x2p/Makefile.SH commit e1da83eac92e1d2253d6b03218386b36cb4bb0af Author: Nicholas Clark Date: Wed May 15 21:34:21 2013 +0200 Remove rules to delete *.loadmap and *.prelmap files, generated on VM/ESA These rules to clean up properly on VM/ESA were added in Oct 1998 by perforce changes 1961, 1962 and 1963 in the confperl branch. VM/ESA support was removed by commit 043fec90e88a2e23 in Aug 2012. M Cross/Makefile-cross-SH M Makefile.SH M x2p/Makefile.SH commit 6105164f37e277ea0eb5dec4a432b86b49489be1 Author: Nicholas Clark Date: Wed May 15 21:26:59 2013 +0200 Add x2p/Makefile.SH to the *.SH files processed by the top level Makefile. This makes the top level Makefile responsible for deleting x2p/Makefile. With this, we can eliminate the 'sh' and 'shextract' from x2p's Makefile. M Makefile.SH M x2p/Makefile.SH commit 655e5c9e391de4eef96d3ad88908a53cddbf53e3 Author: Nicholas Clark Date: Wed May 15 21:17:31 2013 +0200 Remove the now unused 'shlist' Makefile targets. M Makefile.SH M x2p/Makefile.SH commit 9719c95bf27185afc59c14150d4f052f92217540 Author: Nicholas Clark Date: Wed May 15 20:57:37 2013 +0200 Programmatically generate dependencies for all *.SH files. Previously dependencies for Makefile, config.h and makedepend were explicitly coded into verbatim sections of Makefile.SH, with most of the others being generated by makedepend.SH M Makefile.SH M makedepend.SH commit fc40bd89f80e1d94b62f08a3ab57f39ea3333029 Author: Nicholas Clark Date: Wed May 15 20:25:47 2013 +0200 Add runtests.SH to the code that generates the targets sh and shextract. The generated Makefile is functionally identical. M Makefile.SH commit c0d271fb2cf4e67f892e8f1b575439d13b73a01f Author: Nicholas Clark Date: Wed May 15 17:14:50 2013 +0200 Eliminate x2p/cflags.SH. x2p's Makefile can use ../cflags x2p/Makefile is only used on *nix, so this should not present a problem. M MANIFEST M Porting/exec-bit.txt M x2p/Makefile.SH D x2p/cflags.SH commit 3ae7e562a2ff05c27052670c20b7ba3b7005bf7b Author: Nicholas Clark Date: Wed May 15 19:40:46 2013 +0200 Pass in to makedepend the name(s) of files that all objects depend on. M Makefile.SH M makedepend.SH M x2p/Makefile.SH commit 0ff64a71c493f41b23ddd052a465c3d40c3264f2 Author: Nicholas Clark Date: Wed May 15 16:54:40 2013 +0200 In the Makefile, generate the value of shextract programmatically. shextract lists the files generated by shell scripts named *.SH which need deleting. Instead of maintaining it by hand, generate it in Makefile.SH from the list of *.SH files. This indirectly eliminates the last reference to F. makedir.SH was eliminated by commit ebca06e95b02d1a8 and the reference to makedir.SH in Makefile.SH eliminated by commit 847471027aa42997 (both Sept 2009), but that commit missed that the reference to makedir a few lines later was related. M Makefile.SH commit 417de989d3cb17f83d8a6af00ff4b171a8682c8c Author: Nicholas Clark Date: Wed May 15 13:01:22 2013 +0200 Defer running makedepend in x2p/ until it's needed. We can run makedepend in x2p/ as part of the regular build process, which means that it can run as part of a parallel make. This gets a slight speedup (0.2s on this system), and increases flexibility on how x2p/Makefile is generated. M Makefile.SH commit 2fea0471858664126bb4c23b75f6bf15c262cd8b Author: Nicholas Clark Date: Wed May 15 12:49:44 2013 +0200 Add x2p/makefile to the dependencies for targets within x2p/ This will give us more flexibility in when and how we generate it. [x2p/makefile is x2p/GNUmakefile on OS X] M Makefile.SH commit d730c2986a70e4fa953cb2f352b7b5ebf41ea580 Author: Nicholas Clark Date: Wed May 15 12:33:05 2013 +0200 Remove vestigial *.SH code which can link the extracted file into ../ 4 of the *.SH files have code which links the extracted file into the parent directory if the current directory ends SH. This code dates from Perl 1. It seems that the intent was that the *.SH files could live in a SH/ directory, but that feature was neither used or mentioned in the documentation in Perl 1, and hasn't been used since. M Cross/Makefile-cross-SH M Makefile.SH M makedepend.SH M x2p/Makefile.SH ----------------------------------------------------------------------- Summary of changes: Cross/Makefile-cross-SH | 7 --- MANIFEST | 1 - Makefile.SH | 68 ++++++++++++++++--------------- Porting/exec-bit.txt | 1 - makedepend.SH | 39 +---------------- x2p/Makefile.SH | 35 ++------------- x2p/cflags.SH | 104 ----------------------------------------------- 7 files changed, 43 insertions(+), 212 deletions(-) delete mode 100755 x2p/cflags.SH diff --git a/Cross/Makefile-cross-SH b/Cross/Makefile-cross-SH index e5a14b4..e83cd4c 100644 --- a/Cross/Makefile-cross-SH +++ b/Cross/Makefile-cross-SH @@ -957,7 +957,6 @@ _mopup: -@test -f vms/README_vms.pod && rm -f vms/README_vms.pod -rm -f perl.exp ext.libs extra.pods uni.data opmini.o -rm -f perl.export perl.dll perl.libexp perl.map perl.def - -rm -f perl.loadmap miniperl.loadmap perl.prelmap miniperl.prelmap -rm -f perl.third lib*.so.perl.third perl.3log t/perl.third t/perl.3log -rm -f perl.pixie lib*.so.perl.pixie lib*.so.Addrs -rm -f perl.Addrs perl.Counts t/perl.Addrs t/perl.Counts *perl.xok @@ -1281,12 +1280,6 @@ ctags: !NO!SUBS! $eunicefix $Makefile -case `pwd` in -*SH) - $rm -f ../$Makefile - $ln $Makefile ../$Makefile - ;; -esac $rm -f $firstmakefile # Now do any special processing required before building. diff --git a/MANIFEST b/MANIFEST index 7115372..5916b98 100644 --- a/MANIFEST +++ b/MANIFEST @@ -5493,7 +5493,6 @@ x2p/a2p.h Global declarations x2p/a2p.pod Pod for awk to perl translator x2p/a2p.y A yacc grammar for awk x2p/a2py.c Awk compiler, sort of -x2p/cflags.SH A script that emits C compilation flags per file x2p/EXTERN.h Same as above x2p/find2perl.PL A find to perl translator x2p/hash.c Hashes again diff --git a/Makefile.SH b/Makefile.SH index ddb9be1..7502235 100755 --- a/Makefile.SH +++ b/Makefile.SH @@ -443,16 +443,27 @@ xconfig.h: config_h.SH Cross/config-\$(CROSS_NAME).sh ;; esac +SH_to_target() { + echo $@ | sed -e s/\\\.SH//g -e s/_/./g +} + +SH='Makefile.SH cflags.SH config_h.SH makedepend.SH myconfig.SH runtests.SH pod/Makefile.SH x2p/Makefile.SH' +shextract=`SH_to_target $SH` + ## In the following dollars and backticks do not need the extra backslash. -$spitshell >>$Makefile <<'!NO!SUBS!' +$spitshell >>$Makefile < -private = preplibrary $(CONFIGPM) $(CONFIGPOD) lib/ExtUtils/Miniperl.pm git_version.h lib/buildcustomize.pl +private = preplibrary \$(CONFIGPM) \$(CONFIGPOD) lib/ExtUtils/Miniperl.pm git_version.h lib/buildcustomize.pl # Files to be built with variable substitution before miniperl # is available. -sh = Makefile.SH cflags.SH config_h.SH makedepend.SH myconfig.SH pod/Makefile.SH +sh = $SH + +shextract = $shextract +!GROK!THIS! -shextract = Makefile cflags config.h makedepend makedir myconfig pod/Makefile +## In the following dollars and backticks do not need the extra backslash. +$spitshell >>$Makefile <<'!NO!SUBS!' addedbyconf = UU $(shextract) pstruct @@ -589,18 +600,21 @@ perl$(OBJ_EXT): git_version.h case "$usedl$static_cwd" in defineundef) util_deps='$(MINIPERL_EXE) $(CONFIGPM) lib/auto/Cwd/Cwd$(DLSUFFIX) FORCE' - x2p_deps='$(MINIPERL_EXE) $(CONFIGPM) $(dynamic_ext) FORCE' + x2p_deps="\$(MINIPERL_EXE) \$(CONFIGPM) \$(dynamic_ext) x2p/$firstmakefile FORCE" ;; definedefine) util_deps='$(PERL_EXE) $(CONFIGPM) FORCE' - x2p_deps='$(PERL_EXE) $(CONFIGPM) $(dynamic_ext) FORCE' + x2p_deps="\$(PERL_EXE) \$(CONFIGPM) \$(dynamic_ext) x2p/$firstmakefile FORCE" ;; *) util_deps='$(PERL_EXE) $(CONFIGPM) FORCE' - x2p_deps='$(PERL_EXE) $(CONFIGPM) FORCE' + x2p_deps="\$(PERL_EXE) \$(CONFIGPM) x2p/$firstmakefile FORCE" ;; esac $spitshell >>$Makefile <+x2p/$firstmakefile: makedepend + cd x2p; \$(MAKE) depend + translators: $x2p_deps @echo " "; echo " Making x2p stuff"; cd x2p; \$(LDLIBPTH) \$(MAKE) all @@ -1271,13 +1285,12 @@ _mopup: -@test -f vms/README_vms.pod && rm -f vms/README_vms.pod -rm -f perl.exp ext.libs $(generated_pods) uni.data opmini.o perlmini.o pod/roffitall -rm -f perl.export perl.dll perl.libexp perl.map perl.def - -rm -f perl.loadmap miniperl.loadmap perl.prelmap miniperl.prelmap -rm -f perl.third lib*.so.perl.third perl.3log t/perl.third t/perl.3log -rm -f perl.pixie lib*.so.perl.pixie lib*.so.Addrs -rm -f perl.Addrs perl.Counts t/perl.Addrs t/perl.Counts *perl.xok -rm -f cygwin.c libperl*.def libperl*.dll cygperl*.dll *.exe.stackdump -rm -f $(PERL_EXE) $(MINIPERL_EXE) $(LIBPERL) libperl.* microperl - -rm -f config.arch config.over $(DTRACE_H) runtests + -rm -f config.arch config.over $(DTRACE_H) # Do not 'make _tidy' directly. _tidy: @@ -1389,21 +1402,22 @@ MAKEDEPEND = Makefile makedepend $(FIRSTMAKEFILE): README $(MAKEDEPEND) $(MAKE) depend MAKEDEPEND= +!NO!SUBS! -config.h: config_h.SH config.sh - $(SHELL) config_h.SH +for f in $SH; do + file=`SH_to_target $f` + $spitshell >>$Makefile < -.PHONY: depend -depend: makedepend - sh ./makedepend MAKE=$(MAKE) - cd x2p; $(MAKE) depend +$file: $f config.sh + \$(SHELL) $f +!GROK!THIS! +done -# Cannot postpone this until $firstmakefile is ready ;-) -makedepend: makedepend.SH config.sh - sh ./makedepend.SH +$spitshell >>$Makefile <<'!NO!SUBS!' -runtests: runtests.SH config.sh - sh ./runtests.SH +.PHONY: depend +depend: makedepend + sh ./makedepend MAKE=$(MAKE) cflags .PHONY: test check test_prep test_prep_nodll test_prep_pre \ test_prep_reonly test_tty test-tty test_notty test-notty \ @@ -1576,7 +1590,7 @@ noknack: utilities nokfilenack: utilities $(RUN_PERL) -Ilib utils/perlbug -nok -s '(UNINSTALLED)' -F perl.nok -A -.PHONY: clist hlist shlist +.PHONY: clist hlist clist: $(c) echo $(c) | tr ' ' $(TRNL) >.clist @@ -1584,12 +1598,6 @@ clist: $(c) hlist: $(h) echo $(h) | tr ' ' $(TRNL) >.hlist -shlist: $(sh) - echo $(sh) | tr ' ' $(TRNL) >.shlist - -Makefile: Makefile.SH ./config.sh - $(SHELL) Makefile.SH - .PHONY: distcheck distcheck: FORCE perl '-MExtUtils::Manifest=&fullcheck' -e 'fullcheck()' @@ -1604,12 +1612,6 @@ ctags: !NO!SUBS! $eunicefix Makefile -case `pwd` in -*SH) - $rm -f ../Makefile - $ln Makefile ../Makefile - ;; -esac $rm -f $firstmakefile # Now do any special processing required before building. diff --git a/Porting/exec-bit.txt b/Porting/exec-bit.txt index e2f445b..0073e77 100644 --- a/Porting/exec-bit.txt +++ b/Porting/exec-bit.txt @@ -27,7 +27,6 @@ runtests.SH t/TEST vms/ext/filespec.t x2p/Makefile.SH -x2p/cflags.SH Porting/Maintainers.pl Porting/add-package.pl Porting/bisect.pl diff --git a/makedepend.SH b/makedepend.SH index 5669b8d..f992af3 100755 --- a/makedepend.SH +++ b/makedepend.SH @@ -39,7 +39,7 @@ mkdir .depending # This script should be called with # sh ./makedepend MAKE=$(MAKE) case "$1" in - MAKE=*) eval $1 ;; + MAKE=*) eval $1; shift ;; esac export PATH || (echo "OOPS, this isn't sh. Desperation time. I will feed myself to sh."; sh \$0; kill \$\$) @@ -206,35 +206,12 @@ for file in `$cat .clist`; do -e 's|\.c\.c|.c|' $uwinfix .cout .cerr| \ $uniq | $sort | $uniq >> .deptmp fi - echo "$filebase\$(OBJ_EXT): cflags" >> .deptmp + echo "$filebase\$(OBJ_EXT): $@" >> .deptmp done $sed <$mf >$mf.new -e '1,/^# AUTOMATICALLY/!d' -$MAKE shlist || ($echo "Searching for .SH files..."; \ - $echo *.SH | $tr ' ' $trnl | $egrep -v '\*' >.shlist) - -# Now extract the dependencies on makedepend.SH and Makefile.SH -# (they should reside in the main Makefile): -rm -f .shlist.old -mv .shlist .shlist.old -$egrep -v '^makedepend\.SH' <.shlist.old >.shlist -rm -f .shlist.old -mv .shlist .shlist.old -$egrep -v '^Makefile\.SH' <.shlist.old >.shlist -rm -f .shlist.old -mv .shlist .shlist.old -$egrep -v '^perl_exp\.SH' <.shlist.old >.shlist -rm -f .shlist.old -mv .shlist .shlist.old -$egrep -v '^config_h\.SH' <.shlist.old >.shlist -rm .shlist.old - if $test -s .deptmp; then - for file in `cat .shlist`; do - $echo `$expr X$file : 'X$.*$.SH'`: $file $TOP/config.sh \; \ - $sh $file >> .deptmp - done $echo "Updating $mf..." $echo "# If this runs make out of memory, delete /usr/include lines." \ >> $mf.new @@ -262,10 +239,6 @@ else $sed -f .hsed >> $mf.new <.deptmp $sed -n 's|h:#include "$.*$".*$|h: \1|p' | \ $sed -f .hsed >> $mf.new - for file in `$cat .shlist`; do - $echo `$expr X$file : 'X$.*$.SH'`: $file $TOP/config.sh \; \ - $sh $file >> $mf.new - done fi $rm -f $mf.old $cp $mf $mf.old @@ -273,15 +246,9 @@ $rm -f $mf $cp $mf.new $mf $rm $mf.new $echo "# WARNING: Put nothing here or make depend will gobble it up!" >> $mf -$rm -rf .deptmp UU .shlist .clist .hlist .hsed .cout .cerr +$rm -rf .deptmp UU .clist .hlist .hsed .cout .cerr rmdir .depending !NO!SUBS! $eunicefix makedepend chmod +x makedepend -case `pwd` in -*SH) - $rm -f ../makedepend - ln makedepend ../makedepend - ;; -esac diff --git a/x2p/Makefile.SH b/x2p/Makefile.SH index 3f16623..ae017b6 100755 --- a/x2p/Makefile.SH +++ b/x2p/Makefile.SH @@ -77,26 +77,10 @@ PERL = $perl cat >>Makefile <<'!NO!SUBS!' REALPERL = ../perl -CCCMD = `sh $(shellflags) cflags "optimize='$(OPTIMIZE)'" $@` - -public = a2p$(EXE_EXT) s2p find2perl - -private = - -manpages = a2p.man s2p.man - -util = - -sh = Makefile.SH cflags.SH -shextract = Makefile cflags +CCCMD = `sh $(shellflags) ../cflags "optimize='$(OPTIMIZE)'" $@` pl = find2perl.PL s2p.PL plextract = find2perl s2p -plexe = find2perl.exe s2p.exe -plc = find2perl.c s2p.c -plm = a2p.loadmap - -addedbyconf = $(shextract) $(plextract) h = EXTERN.h INTERN.h ../config.h ../handy.h hash.h a2p.h str.h util.h @@ -110,7 +94,7 @@ lintflags = -phbvxac .c$(OBJ_EXT): $(CCCMD) -DPERL_FOR_X2P $*.c -all: $(public) $(private) $(util) +all: a2p$(EXE_EXT) s2p find2perl @echo " " a2p$(EXE_EXT): $(obj) a2p$(OBJ_EXT) @@ -149,13 +133,13 @@ a2p$(OBJ_EXT): a2p.c a2py.c a2p.h EXTERN.h util.h INTERN.h \ $(CCCMD) a2p.c clean: - rm -f a2p$(EXE_EXT) psed *$(OBJ_EXT) $(plexe) $(plc) $(plm) + rm -f a2p$(EXE_EXT) psed *$(OBJ_EXT) $(plexe) $(plc) distclean: veryclean realclean: clean -rmdir .depending - rm -f core $(addedbyconf) all + rm -f core $(plextract) all rm -f $(FIRSTMAKEFILE) $(FIRSTMAKEFILE).old makefile.old veryclean: realclean @@ -170,7 +154,7 @@ lint: lint $(lintflags) $(defs) $(c) > a2p.fuzz depend: ../makedepend - sh ../makedepend MAKE=$(MAKE) + sh ../makedepend MAKE=$(MAKE) ../cflags clist: echo $(c) | tr ' ' $(TRNL) >.clist @@ -178,9 +162,6 @@ clist: hlist: echo $(h) | tr ' ' $(TRNL) >.hlist -shlist: - echo $(sh) | tr ' ' $(TRNL) >.shlist - $(plextract): $(RUN) $(PERL) -I../lib $@.PL @@ -194,10 +175,4 @@ $(obj): makedepend: depend !NO!SUBS! $eunicefix Makefile -case `pwd` in -*SH) - $rm -f ../Makefile - $ln Makefile ../Makefile - ;; -esac rm -f $firstmakefile diff --git a/x2p/cflags.SH b/x2p/cflags.SH deleted file mode 100755 index caeb336..0000000 --- a/x2p/cflags.SH +++ /dev/null @@ -1,104 +0,0 @@ -#!/bin/sh - -case $PERL_CONFIG_SH in -'') - if test -f config.sh; then TOP=.; - elif test -f ../config.sh; then TOP=..; - elif test -f ../../config.sh; then TOP=../..; - elif test -f ../../../config.sh; then TOP=../../..; - elif test -f ../../../../config.sh; then TOP=../../../..; - else - echo "Can't find config.sh."; exit 1 - fi - . $TOP/config.sh - ;; -esac -: This forces SH files to create target in same directory as SH file. -: This is so that make depend always knows where to find SH derivatives. -case "$0" in -*/cflags.SH) cd `expr X$0 : 'X$.*$/'` ;; -cflags.SH) ;; -*) case `pwd` in - */x2p) ;; - *) if test -d x2p; then cd x2p - else echo "Can't figure out where to write output."; exit 1 - fi;; - esac;; -esac -echo "Extracting x2p/cflags (with variable substitutions)" -: This section of the file will have variable substitutions done on it. -: Move anything that needs config subs from !NO!SUBS! section to !GROK!THIS!. -: Protect any dollar signs and backticks that you do not want interpreted -: by putting a backslash in front. You may delete these comments. -rm -f cflags -$spitshell >cflags <-!GROK!THIS! - -: In the following dollars and backticks do not need the extra backslash. -$spitshell >>cflags <<'!NO!SUBS!' -case $PERL_CONFIG_SH in -'') - if test -f config.sh; then TOP=.; - elif test -f ../config.sh; then TOP=..; - elif test -f ../../config.sh; then TOP=../..; - elif test -f ../../../config.sh; then TOP=../../..; - elif test -f ../../../../config.sh; then TOP=../../../..; - else - echo "Can't find config.sh."; exit 1 - fi - . $TOP/config.sh - ;; -esac - -case "X$1" in -Xoptimize=*|X"optimize=*") - eval "$1" - shift - ;; -esac - -also=': ' -case $# in -1) also='echo 1>&2 " CCCMD = "' -esac - -case $# in -0) set *.c; echo "The current C flags are:" ;; -esac - -set `echo "$* " | sed -e 's/\.[oc] / /g' -e 's/\.obj / /g' -e "s/\\$obj_ext / /g"` - -for file do - - case "$#" in - 1) ;; - *) echo $n " $file.c $c" ;; - esac - - : allow variables like str_cflags to be evaluated - - eval 'eval ${'"${file}_cflags"'-""}' - - : or customize here - - case "$file" in - a2p) ;; - a2py) ;; - hash) ;; - str) ;; - util) ;; - walk) ;; - *) ;; - esac - - ccflags="`echo $ccflags | sed -e 's/-DMULTIPLICITY//'`" - - echo "$cc -c $ccflags $optimize" - eval "$also "'"$cc -c $ccflags $optimize"' - - . $TOP/config.sh - -done -!NO!SUBS! -chmod 755 cflags -$eunicefix cflags -- Perl5 Master Repository [perl.git] branch smoke-me/nicholas/x2p-depend-later, deleted. v5.19.0-108-g8ca91d3 by Nicholas Clark 2013-05-21T05:56:03Z From: Nicholas Clark In perl.git, the branch smoke-me/nicholas/x2p-depend-later has been deleted was 8ca91d39a4a96434bc909d710b033c8505e2cde3 ----------------------------------------------------------------------- 8ca91d39a4a96434bc909d710b033c8505e2cde3 Simplify x2p's Makefile. ----------------------------------------------------------------------- -- Perl5 Master Repository [perl.git] branch khw-ebcdic, created. v5.19.0-259-g846bc9b by Karl Williamson 2013-05-21T04:24:24Z From: Karl Williamson In perl.git, the branch khw-ebcdic has been created at 846bc9bf1b056b943f4da698643a26146d7bb790 (commit) - Log ----------------------------------------------------------------- commit 846bc9bf1b056b943f4da698643a26146d7bb790 Author: Karl Williamson Date: Mon May 20 22:20:29 2013 -0600 lib/locale.t: Generalize to work on non-ASCII M lib/locale.t commit a6dc0886bc6232d9d0b71d2d562277b2feb15093 Author: Karl Williamson Date: Mon May 20 22:01:02 2013 -0600 ext/XS-APItest/t/hash.t: Generalize to run on non-ASCII platforms M ext/XS-APItest/t/hash.t commit 8332e10343f90f7c48aea19765c817b13b41a6cd Author: Karl Williamson Date: Mon May 20 21:59:48 2013 -0600 dist/Storable/t/code.t: Fixes to run under EBCDIC M dist/Storable/t/code.t commit 95da796bec07d59e19022d1d28039bc22d11ae2f Author: Karl Williamson Date: Mon May 20 21:58:26 2013 -0600 XXX CPAN, incomplete cpan/Pod-Simple/lib/Pod/Simple/BlackBox.pm: Generalize for non-ASCII platforms M cpan/Pod-Simple/lib/Pod/Simple/BlackBox.pm commit 5687c8ae95323d058890af5f2e1086e34420a618 Author: Karl Williamson Date: Mon May 20 21:54:01 2013 -0600 t/lib/feature/implicit: Generalize for non-ASCII platforms M t/lib/feature/implicit commit d504de60a392b1d23273aa424aeefef3375c2d09 Author: Karl Williamson Date: Sat May 4 13:29:15 2013 -0600 pp.c, regexec.c: Declare buffers large enough These three buffers are not declared with the proper size. There is a #define to use in these declarations, so use it. These matter only on EBCDIC platforms, where the one in pp.c prior to this commit could cause a buffer overrun there. The others shouldn't because what is being used is known (smaller) size. M pp.c M regcomp.c M regexec.c commit 34e1a92649a6ee5a3400a7c9d8b2ee46078505b3 Author: Karl Williamson Date: Sat May 4 13:27:19 2013 -0600 pp.c: Don'tdeclare array to large There is an existing #define that gives the correct size for this buffer. No need to calculate it (which actually gives a larger than needed value). M pp.c commit 3a45f684f9156aa1689875849cc594d9643513b5 Author: Karl Williamson Date: Tue Apr 23 18:58:54 2013 -0600 XXX experimental pp_pack.c: 'u' M pp_pack.c commit 7e2040ce5c9cb40813f58c7ef58b54f9636582ce Author: Karl Williamson Date: Mon Feb 25 17:25:08 2013 -0700 XXX CPAN Normalize This converts Unicode::Normalize to use the native tables that are used by Perl starting in XXX, while using the Unicode-ordered ones that were used before then. Another alternative would be to have mktables generate just these tables in Unicode ordering. M cpan/Unicode-Normalize/Normalize.xs commit 42944d11d58338d83121e6124e0298671f7465b2 Author: Karl Williamson Date: Mon Feb 25 17:22:55 2013 -0700 XXX CPAN prob wrong Collate This changes to implicity usenative code points. This is likely wrong, as the module comes with its own data, that are probably in terms of Unicode M cpan/Unicode-Collate/Collate.xs commit be3ff53cae307da719e6dbba0521f5e2d2269eec Author: Karl Williamson Date: Sat Apr 27 22:14:02 2013 -0600 utf8.c: Remove wrapper functions. Now that the Unicode data is stored in native character set order, it is rare to need to work with the Unicode order. Traditionally, the real work was done in functions that worked with the Unicode order, and wrapper functions (or macros) were used to translate to/from native. There are two groups of functions: one that translates from code point to UTF-8, and the other group goes the opposite direction. This commit changes the base function that translates from UTF-8 to code point to output native instead of Unicode. Those extremely rare instances where Unicode output is needed instead will have to hand-wrap calls to this function with a translation macro, as now described in the API pod. Prior to this, it was the other way, the native was wrapped, and the rare, strict Unicode wasn't. This eliminates a layer of function call overhead for a common case. The base function that translates from code point to UTF-8 retains its Unicode input, as that is more natural to process. However, it is de-emphasized in the pod, with the functionality description moved to the pod for a native input wrapper function. And, those wrappers are now macros in all cases; previously there was function call overhead sometimes. (Equivalent exported functions are retained, however, for XS code that uses the Perl_foo() form.) I had hoped to rebase this commit, squashing it with an earlier commit in this series, eliminating the use of a temporary function name change, but the work involved turns out to be large, with no real payoff. M embed.fnc M embed.h M mathoms.c M proto.h M utf8.c M utf8.h commit 533300b0d2ec1d67073a6ddc120ee9a77daa34ea Author: Karl Williamson Date: Tue Apr 30 09:13:35 2013 -0600 perlapi vis utf8.c: Nits M utf8.c commit 9de633f190be2dad88cb009787ad64815ae211fe Author: Karl Williamson Date: Tue Apr 30 08:04:45 2013 -0600 utf8.c: Move 2 functions to earlier in file This moves these two functions to be adjacent to the function they each call, thus keeping like things together. M utf8.c commit f3605b1fe8d59ef71f121517234da460de0d8d45 Author: Karl Williamson Date: Sat Apr 20 17:04:08 2013 -0600 gv.c: Add comment M gv.c commit bb386848fb07eb968f88c153affdb433e65774f9 Author: Karl Williamson Date: Fri Apr 19 13:58:12 2013 -0600 t/op/coreamp.t: Generalize for non-ASCII platfomrs M t/op/coreamp.t commit 741da65799930958059e70b7fabafc607de9dbe7 Author: Karl Williamson Date: Fri Apr 19 13:19:44 2013 -0600 XXX temporary lib/warnings.pm: Add debugging info M lib/warnings.pm commit 702f6b5da738d682d938bd1970d2fb48a10e0f5a Author: Karl Williamson Date: Fri Apr 19 13:18:20 2013 -0600 regcomp.c: Add missing (parens) to expression A pair of parentheses were missing leading to this 'if' not acting as intended. M regcomp.c commit fcbc87b965557d975f89d20692217eafad00076e Author: Karl Williamson Date: Wed Apr 17 21:49:10 2013 -0600 t/re/re_tests: Some tests are platform-specific M t/re/re_tests commit d5cdc6dec1420fe8aed92974d7ffe973480446bd Author: Karl Williamson Date: Wed Apr 17 21:47:41 2013 -0600 t/re/regexp.t: Add ability to skip depending on platform This adds the capability to specify that a test is to be done only on an ASCII platform, or only on an EBCDIC. M t/re/regexp.t commit 1bcd758400a5a2b7bf1179bbcd4693d17e409aad Author: Karl Williamson Date: Wed Apr 17 08:22:36 2013 -0600 t/io/crlf.t: Generalize for non-ASCII platforms M t/io/crlf.t commit 0563bec690cb5abe77f2e1e36e9887590b520b84 Author: Karl Williamson Date: Tue Apr 16 20:15:08 2013 -0600 unicode_constants.h: regened for ebcdic M unicode_constants.h commit 65475008b11479112c919973a5691217139d3bec Author: Karl Williamson Date: Tue Apr 16 15:49:06 2013 -0600 XXX finish up t/re/regexp.t: Generalize for non-ASCII platforms This adds code to the processing of the tests in t/re/re_tests to automatically convert from unicode to native character sets Add comment about circular tests XXX better commit message M t/re/regexp.t commit fe3d822aa365b4797cf124ffde55be1f1ff89896 Author: Karl Williamson Date: Tue Apr 16 12:13:07 2013 -0600 ext/B/t/b.t: Generalize for non-ASCII platforms M ext/B/t/b.t commit 5defc07f4405765741d6d9e2c94f94a49f2d42f2 Author: Karl Williamson Date: Tue Apr 16 12:02:26 2013 -0600 dist/Safe/t/safeutf8.t: Generalize to non-ASCII platform M dist/Safe/t/safeutf8.t commit fdef3f7077faa72228e58ba44b97f82aa3de362f Author: Karl Williamson Date: Tue Apr 16 11:50:04 2013 -0600 t/op/warn.t: Generalize for non-ASCII platforms M t/op/warn.t commit 628e13aaa6992585f9dafd96f1877f3da2dc20cf Author: Karl Williamson Date: Tue Apr 16 10:18:02 2013 -0600 re/reg_email.t: Generalize for non-ASCII platforms This replaces all the hard-coded hex character values. It uses the new (?[ ]) notation. I checked that the compiled regex matches the exact same code points as before these changes. M t/re/reg_email.t commit 9b580692be38f2db9fc7117f85750ea83268eb29 Author: Karl Williamson Date: Tue Apr 16 09:03:47 2013 -0600 dist/ExtUtils-Install/t/InstallWithMM.t: Skip if EBCDIC Because is uses JSON M dist/ExtUtils-Install/t/InstallWithMM.t commit 2aef090608ada5704da20d43994c6941f652b685 Author: Karl Williamson Date: Sun Apr 14 21:31:04 2013 -0600 XXX: t/lib/warnings/utf8: Experiment with malformed utf8 M t/lib/warnings/utf8 commit cb9a8eed5bf9dec65fe715e0dda7ab4a0e089e79 Author: Karl Williamson Date: Sat Apr 13 22:04:50 2013 -0600 XXX skip cpan tests M t/TEST commit 4489f3f5ffc31e4eb866518ddc564090fe497145 Author: Karl Williamson Date: Sat Apr 13 16:19:20 2013 -0600 ext/XS-APItest/t/svpeek.t: Generalize for non-ASCII platforms M ext/XS-APItest/t/svpeek.t commit 2cf059823672a6faa9e0a8a3cf631e35be604032 Author: Karl Williamson Date: Sat Apr 13 16:14:35 2013 -0600 ext/XS-APItest/t/svpv_magic.t: Generalize for non-ASCII platforms M ext/XS-APItest/t/svpv_magic.t commit 5953fbd2c6e85dc8d5620e495c93cd5f7c466489 Author: Karl Williamson Date: Sat Apr 13 15:54:37 2013 -0600 lib/DBM_Filter/t/encode.t: Generalize for non-ASCII platforms M lib/DBM_Filter/t/encode.t commit 648c06a431fc120aff99feb7fcab25d676490ca4 Author: Karl Williamson Date: Sat Apr 13 15:48:06 2013 -0600 XXX finish up lib/dumpvar.pl: Generalize for EBCDIC Has octal constants M lib/dumpvar.pl commit a7b5a6c7a84ca7a20fbf1aac804fa28d0514bc6a Author: Karl Williamson Date: Sat Apr 13 15:35:52 2013 -0600 XXX finish up lib/utf8.t: Generalize for non-ASCII platforms This includes choosing a different code point that has 3 bytes in both UTF-8 and UTF-EBCDIC, so that the pos numbers work for both. M lib/utf8.t commit 452c48fd89a0a2bede8d3a6685b9b142930dfd6a Author: Karl Williamson Date: Sat Apr 13 15:16:44 2013 -0600 t/uni/parser.t: Generalize for non-ASCII platforms M t/uni/parser.t commit df4e0817b6f574e096c1062534436a900b3cc2f7 Author: Karl Williamson Date: Sat Apr 13 14:41:46 2013 -0600 t/uni/method.t: Generalize for non-ASCII platforms I couldn't figure out a way to not use the hard-coded values M t/uni/method.t commit c7c2ed39a8bfd40a040e4a5bad72da2bf4bb9f18 Author: Karl Williamson Date: Sat Apr 13 14:26:09 2013 -0600 t/op/magic.t: Generalize for non-ASCII platforms M t/op/magic.t commit 18562296e0468f8dc1598420fa95c2dfb754cd61 Author: Karl Williamson Date: Sat Apr 13 13:36:41 2013 -0600 t/io/through.t: Generalize for non-ASCII platforms This uses hard-coded values for EBCDIC because of the shell issues M t/io/through.t commit ab569bc79b233f2a5971df237e32c7a9c6123449 Author: Karl Williamson Date: Sat Apr 13 13:16:00 2013 -0600 toke.c: Fix EBCDIC bugs with single char variable names Latin1 variable single character variable names should all be legal, but the test was not for non-ASCII, it was for variant characters. On EBCDIC platforms, this isn't the same as non-ASCII. The legal control character variable names are not the same as the C0 and DEL controls, but are \001 \037 minus those that traditionally match \s on ASCII platforms, plus \c?. M toke.c commit 643081f8b54c4cd3c641e942ca0cddcdb1460bce Author: Karl Williamson Date: Sat Apr 13 12:55:09 2013 -0600 toke.c: An EBCDIC fix toCTRL(0..31) yields a printing character. This is different from toCTRL(control) on EBCDIC machines. M toke.c commit 0378d3b35a8ad68ba56471f293873c5397e159d0 Author: Karl Williamson Date: Sat Apr 13 12:52:17 2013 -0600 XXX \c must be followed by printable This should be revised and included in 5.18, 5.19 depending on RFC outcome. M dquote_static.c commit d85d5c77618888b8f58da715d40d0f3535a3cbe0 Author: Karl Williamson Date: Sat Apr 13 11:41:04 2013 -0600 XXX temp toCTRL M dquote_static.c M ext/B/B.pm M handy.h M pod/perlebcdic.pod M t/op/chars.t commit 8381ec4d6e99cba0e8a72df7a8e129e91e8295cb Author: Karl Williamson Date: Sat Apr 13 09:18:41 2013 -0600 perlio.c: Generalize for EBCDIC This code had the hex constants for CARRIAGE RETURN and LINE FEED hard-coded in. It appears to me from the comments that '\r' and '\n' are not suitable to use instead. This commit changes the constants to use the native values instead. M perlio.c commit 79eead4d9b5bbcc20673938b26a1fb3a2b16f667 Author: Karl Williamson Date: Sat Apr 13 09:51:34 2013 -0600 unicode_constants.h: Add #defines for CR, LF M regen/unicode_constants.pl M unicode_constants.h commit 38ae23e772528ca2869e8f7b93b5ae0ab40e4d92 Author: Karl Williamson Date: Sun Apr 7 10:45:14 2013 -0600 t/op/goto.t: Generalize for EBCDIC M t/op/goto.t commit e146b0dbd1f5f932486cdc890710133a2e66cf51 Author: Karl Williamson Date: Wed Apr 3 20:15:17 2013 -0600 t/re/pat.t: Generalize for EBCDIC M t/re/pat.t commit 0b621743b4fcbd468f5e73627b785100a70a572b Author: Karl Williamson Date: Wed Apr 3 21:56:02 2013 -0600 XXX t/op/pack.t: Generalize for EBCDIC One unknown what to do: uuencode M t/op/pack.t commit 3775709fb496701f650a063940c9cade6e8139d6 Author: Karl Williamson Date: Sat Apr 6 12:56:52 2013 -0600 regcomp.c: In EBCDIC [i-j] exclude also ASCII i and j are not adjacent in EBCDIC. This excluded any alphabetic characters between them, but allowed other ascii ones. M regcomp.c M t/re/pat_advanced.t commit 495e427b992031dc2dd9a11718650e96fc37ed91 Author: Karl Williamson Date: Sat Apr 6 12:54:42 2013 -0600 utf8.c: Don't use slower general-purpose function There is a macro that accomplishes the same task for a two byte UTF-8 encoded character, and avoids the overhead of the general purpose function call. M utf8.c commit 4185b06c5b546f9c8e1797fc418c0aff0735d34e Author: Karl Williamson Date: Sat Apr 6 12:53:07 2013 -0600 utf8.c: Don't do ++ in macro parameter The formal parameter gets evaluated multiple times on an EBCDIC platform, thus incrementing more than the intended once. M utf8.c commit 81fa4afc35b316a131dbe3f8ccef9367ac36dbb0 Author: Karl Williamson Date: Sat Apr 6 12:50:48 2013 -0600 utf8.c: Use macro instead of duplicating code There is a macro that accomplishes this task, and is easier to read. M utf8.c commit bfde7995312c3e67800ffa98c16d657dcd416d13 Author: Karl Williamson Date: Sat Apr 6 10:15:05 2013 -0600 t/io/bom.t: Fix to run under EBCDIC M t/io/bom.t commit cbc28442dcf478f48e4e9ed88d2b27c465997c1d Author: Karl Williamson Date: Fri Apr 5 23:34:50 2013 -0600 t/uni/overload.t: EBCDIC fixes M t/uni/overload.t commit ad84b647dd681b26395d363e2e3fd07145606777 Author: Karl Williamson Date: Fri Apr 5 23:34:13 2013 -0600 t/uni/method.t: EBCDIC fixes M t/uni/method.t commit 7a939faff39c91bebe09b58c5c1d9b1a917b91b7 Author: Karl Williamson Date: Fri Apr 5 23:33:28 2013 -0600 t/op/utf8magic.t: EBCDIC fixes M t/op/utf8magic.t commit 757f0cbdd73d9e804476dadcf92e1be37714b718 Author: Karl Williamson Date: Fri Apr 5 23:32:57 2013 -0600 t/op/evalbytes.t: EBCDIC fixes M t/op/evalbytes.t commit a993fdd206819d4466d33c5cf597e872a18a03b2 Author: Karl Williamson Date: Fri Apr 5 13:27:42 2013 -0600 t/op/length.t: EBCDIC fixes M t/op/length.t commit 9e1198f03a7491f8e935fc00dc97d7427db0dd7f Author: Karl Williamson Date: Sat Apr 6 13:01:54 2013 -0600 t/op/utfhash.t: XXX Add debug M t/op/utfhash.t commit 9ca4e3deae9c1c2f0d30602376fb70e12325049e Author: Karl Williamson Date: Fri Apr 5 12:21:21 2013 -0600 Data-Dumper/Dumper.pm: Fix for EBCDIC M dist/Data-Dumper/Dumper.pm commit 2ccb29bebaec4ca9e7794ccb22c5771911c5f5cb Author: Karl Williamson Date: Fri Apr 5 12:15:58 2013 -0600 Dumper.xs: Don't translate character twice utf8_to_uvchr() already returns the native code point; no need to convert again. This code is only executed on Perls before 5.15 M dist/Data-Dumper/Dumper.xs commit 7db978d1bf537773b8d2e7486cc9d4568b83ab7c Author: Karl Williamson Date: Sat Apr 6 20:39:22 2013 -0600 dist/IO/t/io_utf8argv.t: Generalize and enable EBCDIC Infrastructure now exists to have this test run on EBCDIC platforms. M dist/IO/t/io_utf8argv.t commit 56f7ad754b3d81fa4ff93d86a9f0bd651c72776d Author: Karl Williamson Date: Wed Apr 3 21:59:16 2013 -0600 utf8.h: Clarify comments M utf8.h commit 8a1d8f33d96cafcbb30cd54943693a830febc13e Author: Karl Williamson Date: Wed Apr 3 19:06:52 2013 -0600 XXX CPAN cpan/Test/lib/Test.pm: Fixes for EBCDIC M cpan/Test/lib/Test.pm commit 9b41ed73a7a62d5d00151947f65f4ba485d5bc15 Author: Karl Williamson Date: Mon Apr 1 22:29:16 2013 -0600 t/re/pat_re_eval.t: Some EBCDIC fixes M t/re/pat_re_eval.t commit 36d11ade774a8c0fc533611d350d7146a01bbf9e Author: Karl Williamson Date: Tue Apr 2 07:11:19 2013 -0600 t/test.pl: Add fcn for UTF-EBCDIC conversion This adds the function byte_utf8a_to_utf8n(). This takes the bytes that form a UTF-8 string and convert them to the bytes that form that string on the native platform. M t/test.pl commit a799971fe1c6e7bff08c43f33e06895e94d907b7 Author: Karl Williamson Date: Mon Apr 1 22:28:43 2013 -0600 dist/Storable/t/utf8.t: Fix to run under EBCDIC M dist/Storable/t/utf8.t commit 26a42ea23cb343422db87e43b794ceedc79a2f61 Author: Karl Williamson Date: Mon Apr 1 22:28:08 2013 -0600 t/uni/variables.t: Fix to run under EBCDIC M t/uni/variables.t commit 0adf4cea5ffcb8c5ab198b0f4ba7c9f9d75f48aa Author: Karl Williamson Date: Mon Apr 1 21:08:20 2013 -0600 t/op/split.t: EBCDIC fixes M t/op/split.t commit 889fd63aca87c233c2fb86bc65190dac78811d2e Author: Karl Williamson Date: Mon Apr 1 20:43:03 2013 -0600 re/pat_advanced.t: EBCDIC fixes This includes not skipping some EBCDIC that formerly was, since we now have testing infrastructure that makes this easy. M t/re/pat_advanced.t commit fdc687c32a839c9d9e0af167c8f0a202f342ac62 Author: Karl Williamson Date: Mon Apr 1 20:01:04 2013 -0600 t/io/utf8.t: EBCDIC fixes M t/io/utf8.t commit c5982cbc7cbbbaf78b56676eb4d9e06b057c4777 Author: Karl Williamson Date: Sat Mar 30 21:13:38 2013 -0600 Unicode::UCD.pm: Nits M lib/Unicode/UCD.pm commit 52bdcb2ae24b0782fd67797389b1733ea75bf528 Author: Karl Williamson Date: Sat Mar 30 12:32:09 2013 -0600 t/uni/fold.t: Generalize for non-ASCII platforms M t/uni/fold.t commit 379d0ee3921e7734ab2811a627d222e91ca3938f Author: Karl Williamson Date: Fri Mar 29 15:22:28 2013 -0600 XXX t/op/tiehandle.t: skip for now; deep recursion M t/op/tiehandle.t commit 7b1392dec926c9b4a78468c2687ef8b4e102b410 Author: Karl Williamson Date: Fri Mar 29 14:56:16 2013 -0600 XXX better commit msg utf8.c: Avoid unnecessary UTF-8 conversions This changes the code so that converting to UTF-8 is avoided unless necessary. For such inputs, the conversion back from UTF-8 is also avoided. The cost of doing this is that the first swatches are combined into one that contains the values for all characters 0-255, instead of having multiple swatches. That means when first calculating the swatch it calculates all 256, instead of 128 (160 on EBCDIC). This also fixes an EBCDIC bug in which characters in this range were being translated twice. M utf8.c commit ca2a982ec84b76af5c17ec23d698447223ce05b9 Author: Karl Williamson Date: Fri Mar 29 13:34:59 2013 -0600 utf8.c: No need to check for UTF-8 malformations This function assumes that the input is well-formed UTF-8, even though until this commit, the preferatory comments didn't say so. The API does not pass the buffer length, so there is no way it could check for reading off the end of the buffer. One code path already calls valid_utf8_to_uvchr(); this changes the remaining code path to correspond. M utf8.c commit 2fe237b2ead3b7d5b182fd37360bea811b1d9032 Author: Karl Williamson Date: Thu Mar 28 17:19:16 2013 -0600 XXX enable _invlist_dump; M embed.fnc M embed.h M proto.h commit c6c6c4f2e9be23671adc0baa37efa3c4b3879137 Author: Karl Williamson Date: Fri Mar 8 11:01:32 2013 -0700 XXX EBCDIC header files M charclass_invlists.h M l1_char_class_tab.h M regcharclass.h M unicode_constants.h commit d65c5d77246f968588b4acc36613dfa7f308b4d2 Author: Karl Williamson Date: Fri Mar 15 12:26:15 2013 -0600 hints/os390.sh: Suppress bogus compiler message M hints/os390.sh commit 2d8ce2d001f8e1248a85a08097f36511cbf664aa Author: John Goodyear Date: Sat Mar 2 12:31:25 2013 -0700 XXX Temporary for z/OS long long support M Configure M hints/os390.sh commit 0fc51098579d6bac233ec614ec09995fb534916a Author: Karl Williamson Date: Wed Mar 27 18:17:28 2013 -0600 Add test that to/from native character set works For non-ASCII systems, there are character set translation tables. This makes sure the two accessible ones are inverses of each other. If not, nothing can be expected to work right. M MANIFEST A t/base/translate.t commit 7f2184e8086d3861c31310d4e4709120193fcab6 Author: Karl Williamson Date: Wed Mar 27 16:55:55 2013 -0600 lib/feature/bundle: Fix some things to pass under EBCDIC M t/lib/feature/bundle commit d17548980042f02f823a2ffd9369a77b82ae60c7 Author: Karl Williamson Date: Wed Mar 27 16:08:04 2013 -0600 XS-APItest/t/fetch_pad_names.t: Skip if EBCDIC This could be ported, but there's a lot of stuff to convert; would need a function to convert byte strings that form legal UTF-8 into legal UTF-EBCDIC M ext/XS-APItest/t/fetch_pad_names.t commit bef44e0fe1a91c3785672bc55224555f6252962d Author: Karl Williamson Date: Wed Mar 27 12:05:53 2013 -0600 XXX ext/XS-APItest/t/utf8.t: Fix so passes EBCDIC This involves skipping much of the tests. Reexamine later M ext/XS-APItest/t/utf8.t commit b3e5e5fdabbef83eae7f9a1152ce2ba585413328 Author: Karl Williamson Date: Wed Mar 27 11:27:06 2013 -0600 ext/re/t/re_funcs_u.t: Fix to work under EBCDIC M ext/re/t/re_funcs_u.t commit c9fd381b09cd9c8b8d9a2b2a86266f4482fa92a0 Author: Karl Williamson Date: Wed Mar 27 11:11:22 2013 -0600 XXX dist/IO/t/io_utf8argv.t: Temporarily skip if EBCDIC M dist/IO/t/io_utf8argv.t commit 8e11c594ca00985d49ed968778d40a02be9d4d06 Author: Karl Williamson Date: Wed Mar 27 10:33:44 2013 -0600 t/op/print.t: Skip an EBCDIC test This could be written (the values would probably change depending on the code page), but the code that would get exercised is unlikely to vary depending on character set. M t/op/print.t commit 1335f8cffc805cd1204c49f52acdeabbebdbb50c Author: Karl Williamson Date: Tue Mar 26 15:44:59 2013 -0600 XXX t/TEST: Avoid SIGPIPEs M t/TEST commit 5e656ef84e65173fb1fc92c6b9eaa82b2e96753d Author: Karl Williamson Date: Tue Mar 26 15:49:08 2013 -0600 XXX Temporarily test normalization M cpan/Unicode-Normalize/t/fcdc.t M cpan/Unicode-Normalize/t/form.t M cpan/Unicode-Normalize/t/func.t M cpan/Unicode-Normalize/t/illegal.t M cpan/Unicode-Normalize/t/norm.t M cpan/Unicode-Normalize/t/null.t M cpan/Unicode-Normalize/t/partial1.t M cpan/Unicode-Normalize/t/partial2.t M cpan/Unicode-Normalize/t/proto.t M cpan/Unicode-Normalize/t/split.t M cpan/Unicode-Normalize/t/test.t M cpan/Unicode-Normalize/t/tie.t commit f7a65aef1d3fea33341fa48a94aedb39b4aa433b Author: Karl Williamson Date: Tue Mar 26 14:06:50 2013 -0600 op/index.t: Fix tests for EBCDIC Commit 8a38a836 erroneously translates literals into the native encoding, causing a double translation, which is garbage. M t/op/index.t commit aa5a298fc0c028ca817ac17398af90f9cf26d576 Author: Karl Williamson Date: Mon Mar 25 20:43:38 2013 -0600 op/chop.t: Fix for EBCDIC One test is skipped because the code point is not representable on EBCDIC platforms. Another test is modified to work on EBCDIC. M t/op/chop.t commit 9f67212afd517b5aa1d73da530a436c5595c8a10 Author: Karl Williamson Date: Mon Mar 25 19:56:50 2013 -0600 t/op/lc.t: Fix to work under EBCDIC This had code that attempted this, but it was wrong. The conversion to EBCDIC must be done before the \U, or similar. M t/op/lc.t commit f2b728509d8a135907c2dd9caebcdb7d351fea8a Author: Karl Williamson Date: Mon Mar 25 15:33:55 2013 -0600 Skip some tests under EBCDIC EBCDIC won't work on these because of inherent differences from ASCII M t/porting/customized.t M t/porting/manifest.t commit 389a512f74d39be53b8aace59a683b86e952a7a9 Author: Karl Williamson Date: Mon Mar 25 15:04:14 2013 -0600 porting/bincompat.t: Skip under EBCDIC because the sorting order is different M t/porting/bincompat.t commit bf4b2050d3001fdb6517d7b01b186f99b753aad9 Author: Karl Williamson Date: Mon Mar 25 14:59:50 2013 -0600 t/re/regex_sets.t: So will pass under EBCDIC M t/re/regex_sets.t commit cbe575d208c24dc43daae745a57ce2e5a1e781d8 Author: Karl Williamson Date: Mon Mar 25 13:09:09 2013 -0600 XXX fix \x{too large} M dist/IO/IO.xs M doop.c M inline.h M pp.c M pp_pack.c M regcomp.c M sv.c M toke.c M utf8.c M utf8.h commit aa34f711a9e967761fba3b60b7da3067db7f5e53 Author: Karl Williamson Date: Sun Mar 24 13:16:08 2013 -0600 utf8.c: Fix so UTF-16 to UTF-8 conversion works under EBCDIC M utf8.c commit 9e8c1573cf235325af4671ed4f9d047ab1fed0cf Author: Karl Williamson Date: Sun Mar 24 13:14:34 2013 -0600 utf8.h, utfebcdic.h: Add #define M utf8.h M utfebcdic.h commit 3fb2a0221fcad500af19b4ee8d97bacd0203c7d8 Author: Karl Williamson Date: Wed Mar 20 22:15:58 2013 -0600 lib/Unicode/UCD.t: Allow to run under EBCDIC, M lib/Unicode/UCD.t commit b5f3c1728ff65d98c6816b4074eb48d99b50ff94 Author: Karl Williamson Date: Tue Mar 19 15:27:31 2013 -0600 t/op/quotemeta.t: EBCDIC fixes M t/op/quotemeta.t commit 5d172959135cac438bb816215bb3022cc30a8910 Author: Karl Williamson Date: Tue Mar 19 11:32:55 2013 -0600 t/re/fold_grind.t: Fixes for EBCDIC M t/re/fold_grind.t commit 9632f23ce8177ed543b40c62bf9cc1176dcfa66b Author: Karl Williamson Date: Tue Mar 19 11:21:09 2013 -0600 t/lib/charnames/alias: Fix some EBCDIC problems M t/lib/charnames/alias commit 629212b42c92dd6fce6f201c3e9b730194a658cc Author: Karl Williamson Date: Tue Mar 19 11:20:24 2013 -0600 t/uni/class.t: Make work on EBCDIC M t/uni/class.t commit 7a4fb27b9214b51d0f4990183ee2cf4c992eb8b9 Author: Karl Williamson Date: Tue Mar 19 11:01:57 2013 -0600 feature/unicode_strings.t: Fix to work on EBCDIC M lib/feature/unicode_strings.t commit 0090870daba8fec137c44459b98348d9dd7b844f Author: Karl Williamson Date: Tue Mar 19 10:10:46 2013 -0600 XXX rebase regen/regcharclass.pl: make more EBCDIC friendly XXX regen/regcharclass.pl: maybe temp comment out utf8_char One of the possible inputs to this process is a string. This clarifies that it must be specified in Unicode characters, and adds code to translate it to native, if necessary. M regen/regcharclass.pl M regen/regcharclass_multi_char_folds.pl commit 9595023f46e4c1f2bb1dc71807f6e7222737106f Author: Karl Williamson Date: Tue Mar 19 10:09:53 2013 -0600 XXX temporarily skip some folding tests M regen/regcharclass.pl M t/re/fold_grind.t M t/re/reg_fold.t commit 5d8179efd6dd1ada662f63aff81c348d9e7404b3 Author: Karl Williamson Date: Mon Mar 18 22:00:29 2013 -0600 XXX temp skip perl5db.t M lib/perl5db.t commit 35801f5e596e3cba174dba96ccfe696adb4a2d87 Author: Karl Williamson Date: Mon Mar 18 11:45:06 2013 -0600 pp.c: White-space only Make a ternary operation more clear M pp.c commit f98569b84642c0645365db18f2d454df149f5e7e Author: Karl Williamson Date: Mon Mar 18 11:43:42 2013 -0600 Fix valid_utf8_to_uvchr() for EBCDIC M utf8.c commit faaf8fbd9dea6941c340b198a832ff4ad9809127 Author: Karl Williamson Date: Sun Mar 17 21:42:20 2013 -0600 t/test.pl: Add comment about EBCDIC M t/test.pl commit 8034a4ee3a99a75e5d6e2b904cfe87347573f540 Author: Karl Williamson Date: Sun Mar 17 17:39:33 2013 -0600 XXX makedepend.SH: Why does 255 work and 250 not? M makedepend.SH commit 4b3174531d11ec8f151d48c54f2b3f16e46029e7 Author: Karl Williamson Date: Sat Mar 16 22:48:22 2013 -0600 XXX regen/mk_PL_charclass.pl: Make EBCDIC friendly need more of a commit message M regen/mk_PL_charclass.pl commit 97393aeb62a0fc1e71f9bc0e5b0ee4d8db751abb Author: Karl Williamson Date: Sat Mar 16 22:44:44 2013 -0600 XXX make various things more EBCDIC friendly Adds trailing white space errors Need to know what to do about ^A meaning 0x1, and M-foo meaning meta M lib/DB.pm M lib/dumpvar.pl M lib/perl5db.pl M lib/sigtrap.pm commit d24dc6ea3aa4066b6c891112c12b89ecd6d34be3 Author: Karl Williamson Date: Sat Mar 16 22:41:15 2013 -0600 XXX: Fixup commit message. Fix UTF8_ACUUMULATE, utf8.c M utf8.c M utf8.h commit 639ac22b318504dadb87825bee930535ca40d879 Author: Karl Williamson Date: Sat Mar 16 16:52:45 2013 -0600 regcomp.c: Fix bug in EBCDIC The POSIXA and NPOSIXA regnodes need to set the bits on only the ASCII code points, but under EBCDIC those code points are not 0-127. M regcomp.c commit 90d102f1dc5d5313967eb2edaea6cb6a7317a4a7 Author: Karl Williamson Date: Fri Mar 15 11:57:24 2013 -0600 re/charset.t: Allow to work on EBCDIC This just converts the hard-coded character numbers to native, so will work on any platform. M t/re/charset.t commit 0e6de6e038bf2d3c68f7dd807ee9d1bff3ad1f5c Author: Karl Williamson Date: Fri Mar 15 11:50:35 2013 -0600 XS-APItest/t/handy.t: Change output message On EBCDIC platforms, the output is not in terms of \N{U+}; change text to \x{ } M ext/XS-APItest/t/handy.t commit cc8d0fe963f61b877d8dede3ca7bd74ab7cca986 Author: Karl Williamson Date: Wed Mar 13 21:44:16 2013 -0600 XXX Dumper.xs: Don't know why this stopped compiling M dist/Data-Dumper/Dumper.xs commit 3cd919847e9441ed6f25c795c604b7a7e37611ae Author: Karl Williamson Date: Wed Mar 13 16:20:23 2013 -0600 toke.c: Simplify some code We don't have to test separately for lower vs uppercase here, as upper/lower case A-Z and a-z are not intermixed in the gaps in A-Z and a-z under EBCDIC. M toke.c commit e6ffc398630b03bb517436da4fb7869a0bff2e9a Author: Karl Williamson Date: Wed Mar 13 16:18:12 2013 -0600 genpacksizetables.pl: Correct comment typo M genpacksizetables.pl commit 87f6869b82bf13bc69b3e6a5ff5221d647b8a326 Author: Karl Williamson Date: Wed Mar 13 16:17:39 2013 -0600 APItest/t/handy.t: Make EBCDIC-friendly M ext/XS-APItest/t/handy.t commit d7d680934c3482d5a942214ef3e9dd9ea72ab24e Author: Karl Williamson Date: Wed Mar 13 16:16:14 2013 -0600 Data-Dumper: Make EBCDIC-friendly M dist/Data-Dumper/Dumper.xs commit dd047fff6d2ce6cef14fcaa9b1a72f3ce01b7910 Author: Karl Williamson Date: Wed Mar 13 16:14:31 2013 -0600 sv.c: Make less ASCII-centric M sv.c commit 3a77e4cc7afebb7d89d71ac11adab5276eea1fa4 Author: Karl Williamson Date: Wed Mar 13 16:07:52 2013 -0600 charnames.t: Generalize for non-ASCII platforms M lib/charnames.t commit 166335409f79605895a4320207ff6d201bc41915 Author: Karl Williamson Date: Wed Mar 13 16:05:46 2013 -0600 dump.c: Make less ASCII-centric: This has the added advantage of being clearer as to what is going on. M dump.c commit 2674e33eadf621297298f4871fe14cb8aa7e1147 Author: Karl Williamson Date: Wed Mar 13 16:02:52 2013 -0600 hv.c: Stop being ASCII-centric This uses macros which work cross-platform. This has the added advantge that it is much clearer what is going on. M hv.c commit e512104c7b4e199f6e4188341dade85fbc93d9be Author: Karl Williamson Date: Mon Mar 11 15:11:10 2013 -0600 Added Porting/reorder_charclass_invlists.pl This program is used too bootstrap perl onto a non-ASCII platform with no pre-existing perl. M MANIFEST A Porting/reorder_charclass_invlists.pl commit ddcc000109288b589aa01cc53e8f156c2f500cd5 Author: Karl Williamson Date: Sun Mar 10 22:17:31 2013 -0600 t/base/lex.t: Use char suitable for both ASCII and EBCDIC \xE2 is 'S' in EBCDIC, and so is going to be legal. \xDF is an alpha which has no ASCII equivalent in either character set M t/base/lex.t commit 1fb145e8b46f9968bf4056696dc90f1dc02acc28 Author: Karl Williamson Date: Sun Mar 10 13:11:07 2013 -0600 XXX Temporary comment out ParseXS check this is to get things to compile for now M dist/ExtUtils-ParseXS/lib/ExtUtils/ParseXS.pm commit 9c8e94cbdd0aad0bfa2aaccdb5c454a467e804a8 Author: Karl Williamson Date: Sun Mar 10 11:34:10 2013 -0600 XXX Collate, Normalize: Allow to compile under EBCDIC M cpan/Unicode-Collate/Collate.pm M cpan/Unicode-Collate/mkheader M cpan/Unicode-Normalize/Normalize.pm M cpan/Unicode-Normalize/mkheader commit 8ddf498a22c5505bb57c1a24559001c50f390fc4 Author: Karl Williamson Date: Sat Mar 9 21:57:38 2013 -0700 XXX dquote_static.c: Silence wrong warning on EBCDIC Unsure of whether to add the 2nd !isCNTRL_L1 to silence return trip, which should be a separate commit anyway. This silences an inappropriate warning that doesn't happen on ASCII platforms. CTRL-T maps to 0x14 on both ASCII and EBCDIC platforms. But 0x14 is a C1 control on EBCDIC, a C0 on ASCII. Therefore the test that it's a control should include both C0 and C1, which isCNTRL_L1() does. Also has a white-space change, outdenting a line so it doesn't wrap in an 80 column window. M dquote_static.c commit a050b48625691fe374873eefcc5091f61f5c7cbe Author: Karl Williamson Date: Thu Mar 7 12:08:41 2013 -0700 utfebcdic.h: Change 'unsigned char' to U8 This is for consistency with the rest of Perl M utfebcdic.h commit e0bb2ed995a6f527260fcf400d616012af6f317a Author: Karl Williamson Date: Fri Mar 8 08:11:38 2013 -0700 regen/regcharclass.pl: Make more EBCDIC-friendly This commit changes the code generated by the macros so that they work right out-of-the-box on non-ASCII platforms for non-UTF-8 inputs. THEY ARE WRONG for UTF-8, but this is good enough to get perl bootstrapped onto the target platform, and regcharclass.pl can be run there, generating macros correct UTF-8. M regcharclass.h M regen/regcharclass.pl commit e8a285a85629d8e88427745d6145c2725918d8f2 Author: Karl Williamson Date: Wed Mar 6 21:30:01 2013 -0700 utfebcdic.h: Add (UV) cast The operand of this macro is implicitly a UV. Make sure that it is. M utfebcdic.h commit 291715559e38ab395a06eba0a51d6c9b8db2caa1 Author: Karl Williamson Date: Wed Mar 6 17:04:58 2013 -0700 handy.h: Allow bootstrapping to non-ASCII platform This adds a bunch of macros and moves things around to support conditional compilation when Configure is called with -DBOOTSTRAP_CHARSET. Doing so causes the usual macros that are table-driven to not be used, since the table may not be valid when bringing Perl up for the first time on a non-ASCII platform. This allows it to compile using the platform's native C library ctype functions, which should work enough to compile miniperl, and allow the table to be changed to be valid. Then Configure can be re-run to not bootstrap, and normal compilation can proceed M handy.h M inline.h commit 93a99031285bb0e86036e022ff8cecc07095a107 Author: Karl Williamson Date: Mon Mar 4 13:00:47 2013 -0700 toke.c: Remove EBCDIC dependency M toke.c commit b2743d186c3861b22a3f39aeccd1f3b39ee65e19 Author: Karl Williamson Date: Mon Mar 4 09:14:25 2013 -0700 toke.c: Remove character set dependency Instead of hard-coding the bit patterns that comprise the Byte Order Mark in the UTF-8 or UTF-EBCDIC encodings, use the generated ones for the current platform. This removes some EBCDIC-only code. M toke.c commit d471c933c260c794aa78693503b8df9380014490 Author: Karl Williamson Date: Mon Mar 4 09:10:27 2013 -0700 unicode_constants.h: Add #defines for Byte Order Mark These will be used in future commits M regen/unicode_constants.pl M unicode_constants.h commit 11a2d037f2269bed17c51022580903d54cf081b1 Author: Karl Williamson Date: Sat Mar 2 15:04:18 2013 -0700 XXX: Find a cleaner way. Handle missing is_UTF8_CHAR_utf8_safe This macro may not be present, and is currently used exclusively in IS_UTF8_CHAR, which itself may be undefined, and code should cope with that. This is a work-around until a better solution is found. M utf8.c M utf8.h commit 32a6773f5f8fd714efe1f5ca1507f85199643d5d Author: Karl Williamson Date: Sat Mar 2 14:09:04 2013 -0700 Add Porting tool for help with non-ASCII platforms Porting/reorder_l1_char_class_tab.pl is used to bootstrap Perl onto a non-ASCII platform with no working Perl. M MANIFEST A Porting/reorder_l1_char_class_tab.pl M regen/mk_PL_charclass.pl commit 5e9252212361bb7d6cd42d36cb7f58c234d787ef Author: Karl Williamson Date: Sat Mar 2 13:06:58 2013 -0700 inline.h: Reorder functions The comment implied that the functions below it in the file were deprecated, but in fact only the next two functions were. This clarifies that and moves them so they are the final ones in the file M inline.h commit 188f5bf31aa2e84893702d6a60f2e9a91a3e26eb Author: Karl Williamson Date: Sat Mar 2 12:33:42 2013 -0700 utfebcdic.h: Add comment M utfebcdic.h commit 68c325fca7682b5445033bbdc362be4c01a46e3c Author: Karl Williamson Date: Sat Mar 2 12:12:11 2013 -0700 utf8.h: Clean up START_MARK definition and use The previous definition broke good encapsulation rules. UTF_START_MARK should return something that fits in a byte; it shouldn't be the caller that does this. So the mask is moved into the definition. This means it can apply only to the portion that creates something larger than a byte. Further, the EBCDIC version can be simplified, since 7 is the largest possible number of bytes in an EBCDIC UTF8 character. M utf8.h M utfebcdic.h commit 7cebc440be4825fe14c3719fc63e69e709c87507 Author: Karl Williamson Date: Sat Mar 2 12:05:26 2013 -0700 utf8.h: Move #includes These two files were only being #included for non-ebcdic compiles; they should be included always. M utf8.h commit 4a14c233522a14c98d15725c68be8cd4e76078c4 Author: John Goodyear Date: Sat Mar 2 11:49:14 2013 -0700 utfebcdic.h: Remove extra parameter expansions These two macros were improperly expanding the parameters as well as defining the operation, leading to compile errors. M utfebcdic.h commit c730f0f9f7712dab040d06c2440ab21df8caef09 Author: Karl Williamson Date: Fri Mar 1 08:28:52 2013 -0700 utf8.h: Simplify UTF8_EIGHT_BIT_foo on EBCDIC These macros were previously defined in terms of UTF8_TWO_BYTE_HI and UTF8_TWO_BYTE_LO. But the EIGHT_BIT versions can use the less general and simpler NATIVE_TO_LATN1 instead of NATIVE_TO_UNI because the input domain is restricted in the EIGHT_BIT. Note that on ASCII platforms, these both expand to the same thing, so the difference matters only on EBCDIC. M utf8.h commit 4b34d9ef26d69fb16ad6b431004a29dea0663a97 Author: Karl Williamson Date: Thu Feb 28 09:25:27 2013 -0700 XXX temp: show makedepend cerr M makedepend.SH commit 71e3c75dd008d2c96e16b178a46e4c6cd9caf461 Author: Karl Williamson Date: Wed Feb 27 21:59:11 2013 -0700 makedepend.SH: Split too long lines; properly join I had thought that a continuation introduced a space. But no, a continuation can happen in the middle of a token. And this splits lines that are getting very long to avoid preprocessor limitations. M makedepend.SH commit 2232c0495dfbefcfc270f6eadc9227b97f8f25de Author: Karl Williamson Date: Wed Feb 27 15:51:28 2013 -0700 makedepend.SH: White-space only Align continuation backslashes M makedepend.SH commit ce7db967db8f07ff8ffd4d6ad41d830e66c7dd8d Author: Karl Williamson Date: Wed Feb 27 14:39:28 2013 -0700 makedepend.SH: Remove some unnecessary white space Multi-line preprocessor directives are now joined into single lines. This can create lines too long for the preprocessor to handle. This commit removes blanks adjoining comments that get deleted. This makes things somewhat less likely to exceed the limit. This commit also fixes several [] which were meant to each match a tab or a blank, but editors converted the tabs to blanks M makedepend.SH commit 20ae44565cfddc53e64666e9d6083dcc14f10de2 Author: Karl Williamson Date: Wed Feb 27 14:30:51 2013 -0700 makedepend.SH: Retain '/**/' comments These comments may actually be necessary. M makedepend.SH commit da1e0a6e6a890939c0a283151915338501dbfa7f Author: Karl Williamson Date: Wed Feb 27 08:38:19 2013 -0700 handy.h: Remove extraneous parens M handy.h commit 79fe62c06aa2a250afe112ca549562fed76db68a Author: Andy Dougherty Date: Wed Feb 27 13:06:07 2013 -0500 Disable gcc-style function attributes on z/OS. John Goodyear reports that the z/OS C compiler supports the attribute keyword, but not exactly the same as gcc. Instead of a "warning", the compiler emits an "INFORMATIONAL" message that Configure fails to detect. Until Configure is fixed, just disable the attributes altogether. John Goodyear M hints/os390.sh commit 92eafeb88ec7b97340bcf80a0e71eef7221fb667 Author: Andy Dougherty Date: Wed Feb 27 09:12:13 2013 -0500 Change os390 custom cppstdin script to use fgrep. Grep appears to be limited to 2048 characters, and truncates the output for cppstin. Fgrep apparently doesn't have that limit. Thanks to John Goodyear for reporting this. M hints/os390.sh commit ddb74c9c41a3de6d05f340deb054b40781256c14 Author: Karl Williamson Date: Tue Feb 26 13:45:19 2013 -0700 utf8.c: Use more clearly named macro In the case of invariants these two macros should do the same thing, but it seems to me that the latter name more clearly indicates what is going on. M utf8.c commit c29e6931ed88b816c1ed584ceeca3e4d4d611a45 Author: Karl Williamson Date: Tue Feb 26 13:35:12 2013 -0700 Add macro OFFUNISKIP This means use official Unicode code point numbering, not native. Doing this converts the existing UNISKIP calls in the code to refer to native code points, which is what they meant anyway. The terminology is somewhat ambiguous, but I don't think will cause real confusion. NATIVE_SKIP is also introduced for situations where it is important to be precise. M toke.c M utf8.c M utf8.h M utfebcdic.h commit 23137845176046d76a4531c841c934d6b0904b86 Author: Karl Williamson Date: Tue Feb 26 13:22:19 2013 -0700 toke.c: white space only M toke.c commit 09452bcd9f3168f1a14c6e3eb0a4b12040428179 Author: Karl Williamson Date: Sun Feb 17 14:00:13 2013 -0700 toke.c: Don't remap \N{} for EBCDIC Everything is now in native, M toke.c commit d34c3b4e4c355753450589c06260cdad77444e9d Author: Karl Williamson Date: Tue Feb 26 12:08:50 2013 -0700 utf8.c: Deprecate two functions This is to force any code that has been using these functions to change. Since the Unicode tables are now stored in native order, these functions should only rarely be needed. However, the functionality of these is needed, and in actuality, on ASCII platforms, the native functions are #defined to these. So what this commit does is rename the functions to something else, and create wrappers with the old names, so that anyone using them will get the deprecation. M embed.fnc M embed.h M mathoms.c M proto.h M utf8.c M utf8.h commit 9601f7fae768cd6a7f778f02d5f3c03b12bc6410 Author: Karl Williamson Date: Tue Feb 26 11:26:09 2013 -0700 Deprecate uvuni_to_utf8() Code should almost never be dealing with non-native code points M embed.fnc M embed.h M proto.h M utf8.c M utf8.h commit f71011a407ce69d43e14ef407eea32b78f06c6c3 Author: Karl Williamson Date: Tue Feb 26 11:02:33 2013 -0700 Deprecate utf8_to_uni_buf() Now that the tables are stored in native order, there is almost no need for code to be dealing in Unicode order. M embed.fnc M proto.h M utf8.c commit c2aba261eec2d814326299fe129bf749385d9e77 Author: Karl Williamson Date: Tue Feb 26 09:00:18 2013 -0700 makedepend.SH: Comment out unnecessary code This causes problems currently for z/OS. But, since we don't know why it was there, I'm leaving it in as a placeholder. M makedepend.SH commit 24cd120f481f22163ea7bcba6c55277a3354ce34 Author: Karl Williamson Date: Mon Feb 25 20:26:44 2013 -0700 Deprecate valid_utf8_to_uvuni() Now that all the tables are stored in native format, there is very little reason to use this function; and those who do need this kind of functionality should be using the bottom level routine, so as to make it clear they are doing nonstandard stuff. M embed.fnc M proto.h M utf8.c commit 72fcfbd7276b6a257b19abfb54251de84caff9fa Author: Karl Williamson Date: Mon Feb 25 20:14:26 2013 -0700 utf8.c: Swap which fcn wraps the other This is in preparation for the current wrapee becoming deprecated M embed.fnc M embed.h M proto.h M utf8.c M utf8.h commit 40f2d4cdebbbfa0457287d529ca2d24930769501 Author: Karl Williamson Date: Mon Feb 25 19:29:34 2013 -0700 utf8.c: Skip a no-op Since the value is invariant under both UTF-8 and not, we already have it in 'uv'; no need to do anything else to get it M utf8.c commit 7d79ccbe4d2a031bea251f943aed70aaa9d0f9ff Author: Karl Williamson Date: Mon Feb 25 19:26:50 2013 -0700 utf8.c: Move comment to where makes more sense M utf8.c commit 60ad9ed5ad305d0456bd7c3cc05e7e7e86eaa7c8 Author: Karl Williamson Date: Mon Feb 25 17:30:10 2013 -0700 APItest: Test native code points, instead of Unicode M ext/XS-APItest/APItest.xs M ext/XS-APItest/t/utf8.t commit b8fd38230f209eff3ab3d6122ac7228cfb0b5946 Author: Karl Williamson Date: Mon Feb 25 17:12:53 2013 -0700 XXX CPAN Encode.xs Use core function if available. This will insulate this code from any future changes. M cpan/Encode/Encode.xs commit da724701c8aed26b50a05dd4cffd4d36cba89067 Author: Karl Williamson Date: Mon Feb 25 17:04:24 2013 -0700 XXX CPAN and unsure Encode M cpan/Encode/Encode.xs M cpan/Encode/Unicode/Unicode.xs commit 83e59b9664c9fa2e9a695a2dfa3b96f078be25be Author: Karl Williamson Date: Mon Feb 25 17:00:47 2013 -0700 XXX CPAN Encode.xs: fix indent M cpan/Encode/Encode.xs commit 0c53257d6df6c5e66887e7ceef264d39cb618cff Author: Karl Williamson Date: Sun Feb 24 17:23:15 2013 -0700 Don't refer to U+XXXX when mean native These messages say the output number is Unicode, but it is really native, so change to saying is 0xXXXX. M regen/regcharclass_multi_char_folds.pl M regexec.c commit 9920d20eac0a278b79bad315b3d3d83e0ba023d2 Author: Karl Williamson Date: Sun Feb 24 16:43:59 2013 -0700 Convert some uvuni() to uvchr() All the tables are now based on the native character set, so using uvuni() in almost all cases is wrong. M cygwin/cygwin.c M doop.c M op.c M pp_pack.c M regcomp.c M regexec.c M toke.c M utf8.c commit 403f02b0c88d069a668bd063d9d80545442010a0 Author: Karl Williamson Date: Sun Feb 24 16:25:47 2013 -0700 handy.h: White space only M handy.h commit ecfe98a97b7cace5aadabb40e9903badaf1eb415 Author: Karl Williamson Date: Sun Feb 24 16:19:49 2013 -0700 t/test.pl: Allow native/latin1 string conversions to work on utf8. These functions no longer have the hard-coded definitions in them, but now end up resolving to internal functions, so that new encodings could be added and these would automatically understand them. Instead of using tr///, these now go character by character and converting to/from ord, which is slower, but allows them to operate on utf8 strings. Peephole optimization should make these essentially no-ops on ascii platforms. M t/test.pl commit 159b69e84691d5f5240eb8efb1b4261f0617318f Author: Karl Williamson Date: Sun Feb 24 16:05:55 2013 -0700 t/test.pl: Simplify ord to/from native fcns This commit changes these functions from converting to/from a string to calling utf8:: functions which operate on ordinals instead. M t/test.pl commit 117ac6b127c015750e98cd9addaa4c4dcf85b294 Author: Karl Williamson Date: Sun Feb 24 15:35:38 2013 -0700 Make casing tables native These are final tables that haven't been converted to native character set casing. M perl.h M utfebcdic.h commit f01ec5e8b3d43058dbcde14eb901b3f977f7d0f0 Author: Karl Williamson Date: Sun Feb 24 15:32:30 2013 -0700 utfebcdic.h: Remove trailing spaces M utfebcdic.h commit 65b524a51ec4dc19af90afd51eb9cfa62989f606 Author: Karl Williamson Date: Fri Feb 22 18:55:26 2013 -0700 EBCDIC has the unicode bug too We have not had a working modern Perl on EBCDIC for some years. When I started out, comments and code led me to conclude erroneously that natively it supported semantics for all 256 characters 0-255. It turns out that I was wrong; it natively (at least on some platforms) has the same rules (essentially none) for the characters which don't correspond to ASCII onees, as the rules for these on ASCII platforms. A previous commit for 5.18 changed the docs about this issue. This current commit forces ASCII rules on EBCDIC platforms (even should there be one that natively uses all 256). To get all 256, the same things like 'use feature "unicode_strings"' must now be done. M handy.h commit c9e5384f5bd88cd9d67de2639b342c92c229a6be Author: Karl Williamson Date: Thu Feb 21 13:47:52 2013 -0700 handy.h: Solve a failure to compile problem under EBCDIC handy.h is included in files that don't include perl.h, and hence not utf8.h. We can't rely therefore on the ASCII/EBCDIC conversion macros being available to us. The best way to cope is to use the native ctype functions. Most, but not all, of the macros in this commit currently resolve to use those native ones, but a future commit will change that. M handy.h commit a0ebfdcdb7af7c10e9db737a587c25e5f5c3cd17 Author: Karl Williamson Date: Thu Feb 21 13:35:12 2013 -0700 handy.h: Simplify some macro definitions Now, only one of the macros relies on magic numbers (isPRINT), leading to clearer definitions. M handy.h commit 827a4d3c43097e2aa7766403efa63952c811dde9 Author: Karl Williamson Date: Thu Feb 21 13:26:49 2013 -0700 handy.h: Combine macros that are same in ASCII, EBCDIC These 4 macros can have the same RHS for their ASCII and EBCDIC versions, so no need to duplicate their definitions This also enables the EBCDIC versions to not have undefined expansions when compiling without perl.h M handy.h commit c7667e4a7566f014564ffb25ab04f4553bde0621 Author: Karl Williamson Date: Wed Feb 20 10:39:48 2013 -0700 Deprecate NATIVE_TO_NEED and ASCII_TO_NEED These macros are no longer called in the Perl core. This commit turns them into functions so that they can use gcc's deprecation facility. I believe these were defective right from the beginning, and I have struggled to understand what's going on. From the name, it appears NATIVE_TO_NEED taks a native byte and turns it into UTF-8 if the appropriate parameter indicates that. But that is impossible to do correctly from that API, as for variant characters, it needs to return two bytes. It could only work correctly if ch is an I8 byte, which isn't native, and hence the name would be wrong. Similar arguments for ASCII_TO_NEED. The function S_append_utf8_from_native_byte(const U8 byte, U8** dest) does what I think NATIVE_TO_NEED intended. M embed.fnc M mathoms.c M proto.h M toke.c M utf8.h M utfebcdic.h commit ae256637f5d9318c3ac4734682b4fa7f6b336b73 Author: Karl Williamson Date: Wed Feb 20 10:26:43 2013 -0700 Remove remaining calls of NATIVE_TO_NEED These calls are just copying the input to the output byte by byte. There is no need to worry about UTF-8 or not, as the output is just an exact copy of the input M toke.c commit 95eef86603b7f99aa46e17584f860910e21b81c2 Author: Karl Williamson Date: Wed Feb 20 08:12:15 2013 -0700 toke.c: Remove some NATIVE_TO_NEED calls I believe NATIVE_TO_NEED is defective, and will remove it in a future commit. But, just in case I'm wrong, I'm doing it in small steps so bisects will show the culprit. This removes the calls to it where the parameter is clearly invariant under UTF-8 and UTF-EBCDIC, and so the result can't be other than just the parameter. M toke.c commit 57daaa5e620c00830d2408f9c034a8faf0fe011d Author: Karl Williamson Date: Wed Feb 20 08:22:07 2013 -0700 toke.c: in [A-Za-z] use macros that exclude non-ASCII alphas This code is attempting to deal with the problem of holes in the ranges a-z and A-Z in EBCDIC. Prior to this patch, it accepeted things like A WITH GRAVE, etc, which shouldn't have the special processing to deal with the holes M toke.c commit 2449dee11c3c8809d0de13cf07f1a52bfaa20fd6 Author: Karl Williamson Date: Tue Feb 19 15:13:19 2013 -0700 Use real illegal UTF-8 byte The code here was wrong in assuming that \xFF is not legal in UTF-8 encoded strings. It currently doesn't work due to a bug, but that may eventually be fixed: [perl #116867]. The comments are also wrong that all bytes are legal in UTF-EBCDIC. It turns out that in well-formed UTF-8, the bytes C0 and C1 never appear (C2, C3, and C4 as well in UTF-EBCDIC), as they would be the start byte of an illegal overlong sequence. This creates a #define for an illegal byte using one of the real illegal ones, and changes the code to use that. No test is included due to #116867. M op.c M toke.c M utf8.h commit 49f04457f857b0a7dea565787d5f9fd6c98fceae Author: Karl Williamson Date: Sun Feb 17 13:50:45 2013 -0700 toke.c: Remove remapping for EBCDIC for octal The code prior to this commit converted something like \04 into its EBCDIC equivalent only in double-quoted strings. This was not done in patterns, and so gave inconsistent results. The correct thing to do should be to do the native thing, what someone who works on a platform would think \04 do. Platform independent characters are available through \N{}, either by name or by U+. The comment changed by this was wrong, as in some cases it was native, and in some cases Unicode. M toke.c commit 7324d3ae3f193aaf616ecfb70cb5a1e6abdc1ac3 Author: Karl Williamson Date: Sun Feb 17 13:47:13 2013 -0700 Remove EBCDIC remappings Now that the tables are stored in native format, we shouldn't be doing remapping. Note that this assumes that the Latin1 casing tables are stored in native order; not all of this has been done yet. M handy.h M perly.c M pp.c M regcomp.c M regexec.c M utf8.c commit 2c2605c19d666c5fcc440eda6e9ab98d128246ec Author: Karl Williamson Date: Sun Feb 17 12:46:05 2013 -0700 Add and use macro to return EBCDIC The conversion from UTF-8 to code point should generally be to the native code point. This adds a macro to do that, and converts the core calls to the existing macro to use the new one instead. The old macro is retained for possible backwards compatibility, though it probably should be deprecated. M handy.h M pp.c M regcomp.c M regexec.c M toke.c M utf8.c M utf8.h commit b126752ccd53766671f78385b34c24233d409556 Author: Karl Williamson Date: Sat Feb 16 11:05:44 2013 -0700 charnames: Make work in EBCDIC Now that mktables generates native tables, the we need to make U+ mean Unicode instead of native. M lib/_charnames.pm M lib/charnames.pm commit e878b052a7a05c2dbdcc284bbca77d5de2ab6f8b Author: Karl Williamson Date: Sat Feb 16 09:35:56 2013 -0700 Unicode::UCD: Work on non-ASCII platforms Now that mktables generates native tables, it is a fairly simple matter to get Unicode::UCD to work on those platforms. M lib/Unicode/UCD.pm commit 29a1d721c61031f49dfc23652a58474aa3f35b03 Author: Karl Williamson Date: Wed Mar 27 17:01:24 2013 -0600 Unicode::UCD: Typo in comment M lib/Unicode/UCD.pm commit bcb619e5ae6cd4c19e03073386cc4223d5b1a503 Author: Karl Williamson Date: Thu Feb 14 22:16:38 2013 -0700 mktables: Generate native code-point tables The output tables for mktables are now in the platform's native character set. This means there is no change for ASCII platforms, but is a change for EBCDIC ones. Since we currently don't have any EBCDIC test platforms, I tested this by faking it out to generate EBCDIC data, and then eye-balled the results. Code that didn't realize there was a potential difference between EBCDIC and non-EBCDIC platforms will now start to work; code that tried to do the right thing under these circumstances will no longer work. Fixing that comes in later commits. M lib/unicore/mktables commit 017a0259eea89468d5fc62ac98b0579d8e70ba04 Author: Karl Williamson Date: Tue Apr 2 21:36:28 2013 -0600 mktables: Move table creation code This code is moved later in the process. This is in preparation for mktables generating tables in the native character set. By moving it to later, the translation to native has already been done, and special coding need not be done. This also caught 7 code points that were omitted somehow in the previous logic M lib/unicore/mktables commit 487a62930f60cdaa7547343a0f9bd31a4ae86760 Author: Karl Williamson Date: Thu Feb 14 10:50:00 2013 -0700 Fix some EBCDIC problems These spots have native code points, so should be using the macros for native code points, instead of Unicode ones. M regcomp.c M sv.c M toke.c commit 843dd8d84cc5c52873823eb655e03eede7b56190 Author: Karl Williamson Date: Wed Feb 13 22:10:19 2013 -0700 Remove unnecessary temp variable in converting to UTF-8 These areas of code included a temporary that is unnecessary. M inline.h M regcomp.c M sv.c commit b20fa569455dea2165f2a6f4ac9dcaf39ee0b2c1 Author: Karl Williamson Date: Wed Feb 13 22:00:55 2013 -0700 utf8.h: Correct macros for EBCDIC These macros were incorrect for EBCDIC. The 3 step process given in utfebcdic.h wasn't being followed. M utf8.h commit 085ad129ba1ecbd24afcbfd889f21298734f5e3f Author: Karl Williamson Date: Sat Feb 9 21:23:30 2013 -0700 Extract common code to an inline function This fairly short paradigm is repeated in several places; a later commit will improve it. M embed.fnc M embed.h M inline.h M pp_pack.c M proto.h M sv.c M toke.c M utf8.c commit fd2cfe7a5ab723a07bc3ee2addeda8bb10300b13 Author: Karl Williamson Date: Thu Feb 7 21:35:57 2013 -0700 Don't use EBCDIC macro for a C language escape C recognizes '\a' (for BEL); just use that instead of a look-up. regen/unicode_constants.pl could be used to generate the character for the ESC (set in surrounding code), but I didn't do that because of potential bootstrapping problems when porting to an EBCDIC platform without a working perl. (The other characters generated in that .pl are less likely to cause problems when compiling perl.) M regcomp.c M toke.c commit 46dd7864d20ae39cffefd28d831e57b1b97abe48 Author: Karl Williamson Date: Thu Feb 7 19:53:38 2013 -0700 Use byte domain EBCDIC/LATIN1 macro where appropriate The macros like NATIVE_TO_UNI will work on EBCDIC, but operate on the whole Unicode range. In the locations affected by this commit, it is known that the domain is limited to a single byte, so the simpler ones whose names contain LATIN1 may be used. On ASCII platforms, all the macros are null, so there is no effective change. M handy.h M regcomp.c M utf8.c commit b64046a6a258ffd6b77e5b8b96c32d931543b2e4 Author: Karl Williamson Date: Thu Feb 7 14:31:09 2013 -0700 Use new clearer named #defines This converts several areas of code to use the more clearly named macros introduced in a recent commit M op.c M toke.c M utf8.c M utf8.h M utfebcdic.h commit 5a52ceee75ccfd0b5284d20d366b0bc1891ba0fd Author: Karl Williamson Date: Thu Feb 7 13:52:31 2013 -0700 utf8.h, utfebcdic.h: Create less confusing #defines This commit creates macros whose names mean something to me, and I don't find confusing. The older names are retained for backwards compatibility. Future commits will fix bugs I introduced from misunderstanding the meaning of the older names. The older names are now #defined in terms of the newer ones, and moved so that they are only defined once, valid for both ASCII and EBCDIC platforms. M utf8.h M utfebcdic.h commit bfeedb6c9f82b70b938f5d5f1e75ea85c750e048 Author: Karl Williamson Date: Mon Feb 4 14:22:02 2013 -0700 pp_ctl.c: Use isCNTRL instead of hard-coded mask This is clearer and portable to EBCDIC. M pp_ctl.c commit 47dffce247050957a8af0b64468bd67d44777786 Author: Karl Williamson Date: Tue Feb 26 13:51:05 2013 -0700 utf8.c: is_utf8_char_slow() should use native length What is passed is the actual length of the native utf8 character. What this was calculating was the length it would be if it were a Unicode character, and then compares, apples to oranges. M utf8.c ----------------------------------------------------------------------- -- Perl5 Master Repository [perl.git] branch smoke-me/nicholas/x2p-depend-later, updated. v5.19.0-108-g8ca91d3 by Nicholas Clark 2013-05-20T20:19:58Z From: Nicholas Clark In perl.git, the branch smoke-me/nicholas/x2p-depend-later has been updated - Log ----------------------------------------------------------------- commit 8ca91d39a4a96434bc909d710b033c8505e2cde3 Author: Nicholas Clark Date: Wed May 15 21:48:12 2013 +0200 Simplify x2p's Makefile. Eliminate plexe and plc, which relate to tests for the compiler and were added as part of commit 52cebf5efc9883c7 in May 1988. The compiler was removed by commit de1254415ffeb03b in Sept 2006. The files a2p.man and s2p.man were removed with Perl 5.000. The macros 'public', 'private' and 'util' mirror the structure of the top level Makefile.SH. However, x2p only ever used 'public', so simplify things by inlining the value of 'public' in the only place that uses it. 'addedbyconf' now only contains the value of 'plextract', so use the latter directly in the one place that had used 'addedbyconf'. M x2p/Makefile.SH commit e1da83eac92e1d2253d6b03218386b36cb4bb0af Author: Nicholas Clark Date: Wed May 15 21:34:21 2013 +0200 Remove rules to delete *.loadmap and *.prelmap files, generated on VM/ESA These rules to clean up properly on VM/ESA were added in Oct 1998 by perforce changes 1961, 1962 and 1963 in the confperl branch. VM/ESA support was removed by commit 043fec90e88a2e23 in Aug 2012. M Cross/Makefile-cross-SH M Makefile.SH M x2p/Makefile.SH commit 6105164f37e277ea0eb5dec4a432b86b49489be1 Author: Nicholas Clark Date: Wed May 15 21:26:59 2013 +0200 Add x2p/Makefile.SH to the *.SH files processed by the top level Makefile. This makes the top level Makefile responsible for deleting x2p/Makefile. With this, we can eliminate the 'sh' and 'shextract' from x2p's Makefile. M Makefile.SH M x2p/Makefile.SH commit 655e5c9e391de4eef96d3ad88908a53cddbf53e3 Author: Nicholas Clark Date: Wed May 15 21:17:31 2013 +0200 Remove the now unused 'shlist' Makefile targets. M Makefile.SH M x2p/Makefile.SH commit 9719c95bf27185afc59c14150d4f052f92217540 Author: Nicholas Clark Date: Wed May 15 20:57:37 2013 +0200 Programmatically generate dependencies for all *.SH files. Previously dependencies for Makefile, config.h and makedepend were explicitly coded into verbatim sections of Makefile.SH, with most of the others being generated by makedepend.SH M Makefile.SH M makedepend.SH commit fc40bd89f80e1d94b62f08a3ab57f39ea3333029 Author: Nicholas Clark Date: Wed May 15 20:25:47 2013 +0200 Add runtests.SH to the code that generates the targets sh and shextract. The generated Makefile is functionally identical. M Makefile.SH commit c0d271fb2cf4e67f892e8f1b575439d13b73a01f Author: Nicholas Clark Date: Wed May 15 17:14:50 2013 +0200 Eliminate x2p/cflags.SH. x2p's Makefile can use ../cflags x2p/Makefile is only used on *nix, so this should not present a problem. M MANIFEST M Porting/exec-bit.txt M x2p/Makefile.SH D x2p/cflags.SH commit 3ae7e562a2ff05c27052670c20b7ba3b7005bf7b Author: Nicholas Clark Date: Wed May 15 19:40:46 2013 +0200 Pass in to makedepend the name(s) of files that all objects depend on. M Makefile.SH M makedepend.SH M x2p/Makefile.SH commit 0ff64a71c493f41b23ddd052a465c3d40c3264f2 Author: Nicholas Clark Date: Wed May 15 16:54:40 2013 +0200 In the Makefile, generate the value of shextract programmatically. shextract lists the files generated by shell scripts named *.SH which need deleting. Instead of maintaining it by hand, generate it in Makefile.SH from the list of *.SH files. This indirectly eliminates the last reference to F. makedir.SH was eliminated by commit ebca06e95b02d1a8 and the reference to makedir.SH in Makefile.SH eliminated by commit 847471027aa42997 (both Sept 2009), but that commit missed that the reference to makedir a few lines later was related. M Makefile.SH commit 417de989d3cb17f83d8a6af00ff4b171a8682c8c Author: Nicholas Clark Date: Wed May 15 13:01:22 2013 +0200 Defer running makedepend in x2p/ until it's needed. We can run makedepend in x2p/ as part of the regular build process, which means that it can run as part of a parallel make. This gets a slight speedup (0.2s on this system), and increases flexibility on how x2p/Makefile is generated. M Makefile.SH commit 2fea0471858664126bb4c23b75f6bf15c262cd8b Author: Nicholas Clark Date: Wed May 15 12:49:44 2013 +0200 Add x2p/makefile to the dependencies for targets within x2p/ This will give us more flexibility in when and how we generate it. [x2p/makefile is x2p/GNUmakefile on OS X] M Makefile.SH commit d730c2986a70e4fa953cb2f352b7b5ebf41ea580 Author: Nicholas Clark Date: Wed May 15 12:33:05 2013 +0200 Remove vestigial *.SH code which can link the extracted file into ../ 4 of the *.SH files have code which links the extracted file into the parent directory if the current directory ends SH. This code dates from Perl 1. It seems that the intent was that the *.SH files could live in a SH/ directory, but that feature was neither used or mentioned in the documentation in Perl 1, and hasn't been used since. M Cross/Makefile-cross-SH M Makefile.SH M makedepend.SH M x2p/Makefile.SH ----------------------------------------------------------------------- Summary of changes: Cross/Makefile-cross-SH | 7 --- MANIFEST | 1 - Makefile.SH | 68 ++++++++++++++++--------------- Porting/exec-bit.txt | 1 - makedepend.SH | 39 +---------------- x2p/Makefile.SH | 35 ++------------- x2p/cflags.SH | 104 ----------------------------------------------- 7 files changed, 43 insertions(+), 212 deletions(-) delete mode 100755 x2p/cflags.SH diff --git a/Cross/Makefile-cross-SH b/Cross/Makefile-cross-SH index e5a14b4..e83cd4c 100644 --- a/Cross/Makefile-cross-SH +++ b/Cross/Makefile-cross-SH @@ -957,7 +957,6 @@ _mopup: -@test -f vms/README_vms.pod && rm -f vms/README_vms.pod -rm -f perl.exp ext.libs extra.pods uni.data opmini.o -rm -f perl.export perl.dll perl.libexp perl.map perl.def - -rm -f perl.loadmap miniperl.loadmap perl.prelmap miniperl.prelmap -rm -f perl.third lib*.so.perl.third perl.3log t/perl.third t/perl.3log -rm -f perl.pixie lib*.so.perl.pixie lib*.so.Addrs -rm -f perl.Addrs perl.Counts t/perl.Addrs t/perl.Counts *perl.xok @@ -1281,12 +1280,6 @@ ctags: !NO!SUBS! $eunicefix $Makefile -case `pwd` in -*SH) - $rm -f ../$Makefile - $ln $Makefile ../$Makefile - ;; -esac $rm -f $firstmakefile # Now do any special processing required before building. diff --git a/MANIFEST b/MANIFEST index 7115372..5916b98 100644 --- a/MANIFEST +++ b/MANIFEST @@ -5493,7 +5493,6 @@ x2p/a2p.h Global declarations x2p/a2p.pod Pod for awk to perl translator x2p/a2p.y A yacc grammar for awk x2p/a2py.c Awk compiler, sort of -x2p/cflags.SH A script that emits C compilation flags per file x2p/EXTERN.h Same as above x2p/find2perl.PL A find to perl translator x2p/hash.c Hashes again diff --git a/Makefile.SH b/Makefile.SH index ddb9be1..7502235 100755 --- a/Makefile.SH +++ b/Makefile.SH @@ -443,16 +443,27 @@ xconfig.h: config_h.SH Cross/config-\$(CROSS_NAME).sh ;; esac +SH_to_target() { + echo $@ | sed -e s/\\\.SH//g -e s/_/./g +} + +SH='Makefile.SH cflags.SH config_h.SH makedepend.SH myconfig.SH runtests.SH pod/Makefile.SH x2p/Makefile.SH' +shextract=`SH_to_target $SH` + ## In the following dollars and backticks do not need the extra backslash. -$spitshell >>$Makefile <<'!NO!SUBS!' +$spitshell >>$Makefile < -private = preplibrary $(CONFIGPM) $(CONFIGPOD) lib/ExtUtils/Miniperl.pm git_version.h lib/buildcustomize.pl +private = preplibrary \$(CONFIGPM) \$(CONFIGPOD) lib/ExtUtils/Miniperl.pm git_version.h lib/buildcustomize.pl # Files to be built with variable substitution before miniperl # is available. -sh = Makefile.SH cflags.SH config_h.SH makedepend.SH myconfig.SH pod/Makefile.SH +sh = $SH + +shextract = $shextract +!GROK!THIS! -shextract = Makefile cflags config.h makedepend makedir myconfig pod/Makefile +## In the following dollars and backticks do not need the extra backslash. +$spitshell >>$Makefile <<'!NO!SUBS!' addedbyconf = UU $(shextract) pstruct @@ -589,18 +600,21 @@ perl$(OBJ_EXT): git_version.h case "$usedl$static_cwd" in defineundef) util_deps='$(MINIPERL_EXE) $(CONFIGPM) lib/auto/Cwd/Cwd$(DLSUFFIX) FORCE' - x2p_deps='$(MINIPERL_EXE) $(CONFIGPM) $(dynamic_ext) FORCE' + x2p_deps="\$(MINIPERL_EXE) \$(CONFIGPM) \$(dynamic_ext) x2p/$firstmakefile FORCE" ;; definedefine) util_deps='$(PERL_EXE) $(CONFIGPM) FORCE' - x2p_deps='$(PERL_EXE) $(CONFIGPM) $(dynamic_ext) FORCE' + x2p_deps="\$(PERL_EXE) \$(CONFIGPM) \$(dynamic_ext) x2p/$firstmakefile FORCE" ;; *) util_deps='$(PERL_EXE) $(CONFIGPM) FORCE' - x2p_deps='$(PERL_EXE) $(CONFIGPM) FORCE' + x2p_deps="\$(PERL_EXE) \$(CONFIGPM) x2p/$firstmakefile FORCE" ;; esac $spitshell >>$Makefile <+x2p/$firstmakefile: makedepend + cd x2p; \$(MAKE) depend + translators: $x2p_deps @echo " "; echo " Making x2p stuff"; cd x2p; \$(LDLIBPTH) \$(MAKE) all @@ -1271,13 +1285,12 @@ _mopup: -@test -f vms/README_vms.pod && rm -f vms/README_vms.pod -rm -f perl.exp ext.libs $(generated_pods) uni.data opmini.o perlmini.o pod/roffitall -rm -f perl.export perl.dll perl.libexp perl.map perl.def - -rm -f perl.loadmap miniperl.loadmap perl.prelmap miniperl.prelmap -rm -f perl.third lib*.so.perl.third perl.3log t/perl.third t/perl.3log -rm -f perl.pixie lib*.so.perl.pixie lib*.so.Addrs -rm -f perl.Addrs perl.Counts t/perl.Addrs t/perl.Counts *perl.xok -rm -f cygwin.c libperl*.def libperl*.dll cygperl*.dll *.exe.stackdump -rm -f $(PERL_EXE) $(MINIPERL_EXE) $(LIBPERL) libperl.* microperl - -rm -f config.arch config.over $(DTRACE_H) runtests + -rm -f config.arch config.over $(DTRACE_H) # Do not 'make _tidy' directly. _tidy: @@ -1389,21 +1402,22 @@ MAKEDEPEND = Makefile makedepend $(FIRSTMAKEFILE): README $(MAKEDEPEND) $(MAKE) depend MAKEDEPEND= +!NO!SUBS! -config.h: config_h.SH config.sh - $(SHELL) config_h.SH +for f in $SH; do + file=`SH_to_target $f` + $spitshell >>$Makefile < -.PHONY: depend -depend: makedepend - sh ./makedepend MAKE=$(MAKE) - cd x2p; $(MAKE) depend +$file: $f config.sh + \$(SHELL) $f +!GROK!THIS! +done -# Cannot postpone this until $firstmakefile is ready ;-) -makedepend: makedepend.SH config.sh - sh ./makedepend.SH +$spitshell >>$Makefile <<'!NO!SUBS!' -runtests: runtests.SH config.sh - sh ./runtests.SH +.PHONY: depend +depend: makedepend + sh ./makedepend MAKE=$(MAKE) cflags .PHONY: test check test_prep test_prep_nodll test_prep_pre \ test_prep_reonly test_tty test-tty test_notty test-notty \ @@ -1576,7 +1590,7 @@ noknack: utilities nokfilenack: utilities $(RUN_PERL) -Ilib utils/perlbug -nok -s '(UNINSTALLED)' -F perl.nok -A -.PHONY: clist hlist shlist +.PHONY: clist hlist clist: $(c) echo $(c) | tr ' ' $(TRNL) >.clist @@ -1584,12 +1598,6 @@ clist: $(c) hlist: $(h) echo $(h) | tr ' ' $(TRNL) >.hlist -shlist: $(sh) - echo $(sh) | tr ' ' $(TRNL) >.shlist - -Makefile: Makefile.SH ./config.sh - $(SHELL) Makefile.SH - .PHONY: distcheck distcheck: FORCE perl '-MExtUtils::Manifest=&fullcheck' -e 'fullcheck()' @@ -1604,12 +1612,6 @@ ctags: !NO!SUBS! $eunicefix Makefile -case `pwd` in -*SH) - $rm -f ../Makefile - $ln Makefile ../Makefile - ;; -esac $rm -f $firstmakefile # Now do any special processing required before building. diff --git a/Porting/exec-bit.txt b/Porting/exec-bit.txt index e2f445b..0073e77 100644 --- a/Porting/exec-bit.txt +++ b/Porting/exec-bit.txt @@ -27,7 +27,6 @@ runtests.SH t/TEST vms/ext/filespec.t x2p/Makefile.SH -x2p/cflags.SH Porting/Maintainers.pl Porting/add-package.pl Porting/bisect.pl diff --git a/makedepend.SH b/makedepend.SH index 5669b8d..f992af3 100755 --- a/makedepend.SH +++ b/makedepend.SH @@ -39,7 +39,7 @@ mkdir .depending # This script should be called with # sh ./makedepend MAKE=$(MAKE) case "$1" in - MAKE=*) eval $1 ;; + MAKE=*) eval $1; shift ;; esac export PATH || (echo "OOPS, this isn't sh. Desperation time. I will feed myself to sh."; sh \$0; kill \$\$) @@ -206,35 +206,12 @@ for file in `$cat .clist`; do -e 's|\.c\.c|.c|' $uwinfix .cout .cerr| \ $uniq | $sort | $uniq >> .deptmp fi - echo "$filebase\$(OBJ_EXT): cflags" >> .deptmp + echo "$filebase\$(OBJ_EXT): $@" >> .deptmp done $sed <$mf >$mf.new -e '1,/^# AUTOMATICALLY/!d' -$MAKE shlist || ($echo "Searching for .SH files..."; \ - $echo *.SH | $tr ' ' $trnl | $egrep -v '\*' >.shlist) - -# Now extract the dependencies on makedepend.SH and Makefile.SH -# (they should reside in the main Makefile): -rm -f .shlist.old -mv .shlist .shlist.old -$egrep -v '^makedepend\.SH' <.shlist.old >.shlist -rm -f .shlist.old -mv .shlist .shlist.old -$egrep -v '^Makefile\.SH' <.shlist.old >.shlist -rm -f .shlist.old -mv .shlist .shlist.old -$egrep -v '^perl_exp\.SH' <.shlist.old >.shlist -rm -f .shlist.old -mv .shlist .shlist.old -$egrep -v '^config_h\.SH' <.shlist.old >.shlist -rm .shlist.old - if $test -s .deptmp; then - for file in `cat .shlist`; do - $echo `$expr X$file : 'X$.*$.SH'`: $file $TOP/config.sh \; \ - $sh $file >> .deptmp - done $echo "Updating $mf..." $echo "# If this runs make out of memory, delete /usr/include lines." \ >> $mf.new @@ -262,10 +239,6 @@ else $sed -f .hsed >> $mf.new <.deptmp $sed -n 's|h:#include "$.*$".*$|h: \1|p' | \ $sed -f .hsed >> $mf.new - for file in `$cat .shlist`; do - $echo `$expr X$file : 'X$.*$.SH'`: $file $TOP/config.sh \; \ - $sh $file >> $mf.new - done fi $rm -f $mf.old $cp $mf $mf.old @@ -273,15 +246,9 @@ $rm -f $mf $cp $mf.new $mf $rm $mf.new $echo "# WARNING: Put nothing here or make depend will gobble it up!" >> $mf -$rm -rf .deptmp UU .shlist .clist .hlist .hsed .cout .cerr +$rm -rf .deptmp UU .clist .hlist .hsed .cout .cerr rmdir .depending !NO!SUBS! $eunicefix makedepend chmod +x makedepend -case `pwd` in -*SH) - $rm -f ../makedepend - ln makedepend ../makedepend - ;; -esac diff --git a/x2p/Makefile.SH b/x2p/Makefile.SH index 3f16623..ae017b6 100755 --- a/x2p/Makefile.SH +++ b/x2p/Makefile.SH @@ -77,26 +77,10 @@ PERL = $perl cat >>Makefile <<'!NO!SUBS!' REALPERL = ../perl -CCCMD = `sh $(shellflags) cflags "optimize='$(OPTIMIZE)'" $@` - -public = a2p$(EXE_EXT) s2p find2perl - -private = - -manpages = a2p.man s2p.man - -util = - -sh = Makefile.SH cflags.SH -shextract = Makefile cflags +CCCMD = `sh $(shellflags) ../cflags "optimize='$(OPTIMIZE)'" $@` pl = find2perl.PL s2p.PL plextract = find2perl s2p -plexe = find2perl.exe s2p.exe -plc = find2perl.c s2p.c -plm = a2p.loadmap - -addedbyconf = $(shextract) $(plextract) h = EXTERN.h INTERN.h ../config.h ../handy.h hash.h a2p.h str.h util.h @@ -110,7 +94,7 @@ lintflags = -phbvxac .c$(OBJ_EXT): $(CCCMD) -DPERL_FOR_X2P $*.c -all: $(public) $(private) $(util) +all: a2p$(EXE_EXT) s2p find2perl @echo " " a2p$(EXE_EXT): $(obj) a2p$(OBJ_EXT) @@ -149,13 +133,13 @@ a2p$(OBJ_EXT): a2p.c a2py.c a2p.h EXTERN.h util.h INTERN.h \ $(CCCMD) a2p.c clean: - rm -f a2p$(EXE_EXT) psed *$(OBJ_EXT) $(plexe) $(plc) $(plm) + rm -f a2p$(EXE_EXT) psed *$(OBJ_EXT) $(plexe) $(plc) distclean: veryclean realclean: clean -rmdir .depending - rm -f core $(addedbyconf) all + rm -f core $(plextract) all rm -f $(FIRSTMAKEFILE) $(FIRSTMAKEFILE).old makefile.old veryclean: realclean @@ -170,7 +154,7 @@ lint: lint $(lintflags) $(defs) $(c) > a2p.fuzz depend: ../makedepend - sh ../makedepend MAKE=$(MAKE) + sh ../makedepend MAKE=$(MAKE) ../cflags clist: echo $(c) | tr ' ' $(TRNL) >.clist @@ -178,9 +162,6 @@ clist: hlist: echo $(h) | tr ' ' $(TRNL) >.hlist -shlist: - echo $(sh) | tr ' ' $(TRNL) >.shlist - $(plextract): $(RUN) $(PERL) -I../lib $@.PL @@ -194,10 +175,4 @@ $(obj): makedepend: depend !NO!SUBS! $eunicefix Makefile -case `pwd` in -*SH) - $rm -f ../Makefile - $ln Makefile ../Makefile - ;; -esac rm -f $firstmakefile diff --git a/x2p/cflags.SH b/x2p/cflags.SH deleted file mode 100755 index caeb336..0000000 --- a/x2p/cflags.SH +++ /dev/null @@ -1,104 +0,0 @@ -#!/bin/sh - -case $PERL_CONFIG_SH in -'') - if test -f config.sh; then TOP=.; - elif test -f ../config.sh; then TOP=..; - elif test -f ../../config.sh; then TOP=../..; - elif test -f ../../../config.sh; then TOP=../../..; - elif test -f ../../../../config.sh; then TOP=../../../..; - else - echo "Can't find config.sh."; exit 1 - fi - . $TOP/config.sh - ;; -esac -: This forces SH files to create target in same directory as SH file. -: This is so that make depend always knows where to find SH derivatives. -case "$0" in -*/cflags.SH) cd `expr X$0 : 'X$.*$/'` ;; -cflags.SH) ;; -*) case `pwd` in - */x2p) ;; - *) if test -d x2p; then cd x2p - else echo "Can't figure out where to write output."; exit 1 - fi;; - esac;; -esac -echo "Extracting x2p/cflags (with variable substitutions)" -: This section of the file will have variable substitutions done on it. -: Move anything that needs config subs from !NO!SUBS! section to !GROK!THIS!. -: Protect any dollar signs and backticks that you do not want interpreted -: by putting a backslash in front. You may delete these comments. -rm -f cflags -$spitshell >cflags <-!GROK!THIS! - -: In the following dollars and backticks do not need the extra backslash. -$spitshell >>cflags <<'!NO!SUBS!' -case $PERL_CONFIG_SH in -'') - if test -f config.sh; then TOP=.; - elif test -f ../config.sh; then TOP=..; - elif test -f ../../config.sh; then TOP=../..; - elif test -f ../../../config.sh; then TOP=../../..; - elif test -f ../../../../config.sh; then TOP=../../../..; - else - echo "Can't find config.sh."; exit 1 - fi - . $TOP/config.sh - ;; -esac - -case "X$1" in -Xoptimize=*|X"optimize=*") - eval "$1" - shift - ;; -esac - -also=': ' -case $# in -1) also='echo 1>&2 " CCCMD = "' -esac - -case $# in -0) set *.c; echo "The current C flags are:" ;; -esac - -set `echo "$* " | sed -e 's/\.[oc] / /g' -e 's/\.obj / /g' -e "s/\\$obj_ext / /g"` - -for file do - - case "$#" in - 1) ;; - *) echo $n " $file.c $c" ;; - esac - - : allow variables like str_cflags to be evaluated - - eval 'eval ${'"${file}_cflags"'-""}' - - : or customize here - - case "$file" in - a2p) ;; - a2py) ;; - hash) ;; - str) ;; - util) ;; - walk) ;; - *) ;; - esac - - ccflags="`echo $ccflags | sed -e 's/-DMULTIPLICITY//'`" - - echo "$cc -c $ccflags $optimize" - eval "$also "'"$cc -c $ccflags $optimize"' - - . $TOP/config.sh - -done -!NO!SUBS! -chmod 755 cflags -$eunicefix cflags -- Perl5 Master Repository [perl.git] branch smoke-me/nicholas/x2p-depend-later, created. v5.19.0-96-g9e7945e by Nicholas Clark 2013-05-20T20:19:55Z From: Nicholas Clark In perl.git, the branch smoke-me/nicholas/x2p-depend-later has been created <http://perl5.git.perl.org/perl.git/commitdiff/9e7945ee50c30a17a84968fb0c7e5c38223af6f9?hp=0000000000000000000000000000000000000000> at 9e7945ee50c30a17a84968fb0c7e5c38223af6f9 (commit) - Log ----------------------------------------------------------------- commit 9e7945ee50c30a17a84968fb0c7e5c38223af6f9 Merge: 39ecab1 7df6ffa Author: Nicholas Clark <nick@ccl4.org> Date: Mon May 20 21:42:33 2013 +0200 Merge the pack refactoring to blead. commit 7df6ffa3fb2efaa96f5e6001f1cc311ad9f8b185 Author: Nicholas Clark <nick@ccl4.org> Date: Mon May 20 21:40:27 2013 +0200 Notes in perldelta for the tiny XS-visible changes of the pack refactoring. M pod/perldelta.pod commit 228e69a781c23b2599bbd89f1de29c212f10ea31 Author: Nicholas Clark <nick@ccl4.org> Date: Wed May 8 10:40:07 2013 +0200 Annotate the "cold" paths in pp_pack.c with UNLIKELY(). The usual case is nice regular bytes in the host's nice regular order. M pp_pack.c commit 545872c5eddf1f00b9826a3d8d682387f1c5049d Author: Nicholas Clark <nick@ccl4.org> Date: Tue May 7 18:25:53 2013 +0200 Eliminate the implementations of [hv]to[vh][ls] for mixed-endian systems. As pp_pack.c has had mixed-endian support removed, there is little point in keeping code in perl.h and util.c only needed for architectures that cannot be built. M perl.h M util.c commit 1cb43330cc18f2eee5ab1eb58e65991407f4b243 Author: Nicholas Clark <nick@ccl4.org> Date: Tue May 7 18:17:39 2013 +0200 Eliminate Perl_my_swabn(), as it is now unused. It is not marked as part of the API, and no code on CPAN is using it. M embed.fnc M embed.h M proto.h M util.c commit 3a88beaa68dbb5bad93145daa0c829e0aeb40adb Author: Nicholas Clark <nick@ccl4.org> Date: Tue May 7 17:39:42 2013 +0200 When endian-swapping in pack, simply copy the bytes in reverse order. This should restore support for big endian Crays. It doesn't support mixed-endian systems. M embed.fnc M pp_pack.c M proto.h commit aaec8192358ffb8080ad85754ea9aeca93b06c8b Author: Nicholas Clark <nick@ccl4.org> Date: Tue May 7 16:22:08 2013 +0200 Pass 'needs_swap' as a parameter to SHIFT_BYTES. It feels wrong to have it as an implicit parameter sucked in via the textual expansion of the macro. Whilst it can be derived from the parameter 'datumtype', it seems that the C compiler generates a lot less efficient code that way. M pp_pack.c commit 20aa3a3813ca2068a61ea9582e99df57cc735e62 Author: Nicholas Clark <nick@ccl4.org> Date: Tue May 7 15:15:54 2013 +0200 When endian-swapping in unpack, simply copy the bytes in reverse order. It is considerably simpler to re-order the bytes before reading them into the variable of the desired type, than to read into the variable and then need a specialised "reverse this integer" function for each size of integer. This should restore support for big endian Crays. It doesn't support mixed-endian systems. Support for mixed-endian systems can be restored (if needed) by re-ordering the bytes correctly at the locations which currently only know how to reverse the bytes. M pp_pack.c commit cd7e75121be11e7979933a8f7a47cd842a685157 Author: Nicholas Clark <nick@ccl4.org> Date: Tue May 7 12:23:00 2013 +0200 Fold the implementation of DO_BO_UNPACK into SHIFT_BYTES(), its only user. M pp_pack.c commit 7285e3f43c94010f09aa1b7a3ccbdbe7de1aa3e8 Author: Nicholas Clark <nick@ccl4.org> Date: Tue May 7 11:53:17 2013 +0200 Re-implement SHIFT16 and SHIFT32 as wrappers to SHIFT_BYTES. M pp_pack.c commit e396d2357dcec18c01b1cc78d9edc80a3aa4066d Author: Nicholas Clark <nick@ccl4.org> Date: Tue May 7 10:08:53 2013 +0200 Refactor the two uses of SHIFT{16,32} not followed by DO_BO_UNPACK(). There is one place where SHIFT16() is not immediately followed by DO_BO_UNPACK() ('n!' and 'v!'), and one place where SHIFT32() is not immediately followed by DO_BO_UNPACK() ('N!' and 'V!'). Every other use of either macro is followed by DO_BO_UNPACK(). For the two locations that don't have DO_BO_UNPACK(), the modifiers '<' and '>' are illegal, hence DO_BO_UNPACK() will always be a no-op. Hence add DO_BO_UNPACK() to both (which permits refactoring to simplify things), and add assert()s that the assumptions are true. M pp_pack.c commit 2b4ad5699f46c7f63185ad4e7d1f61e16aaa73f6 Author: Nicholas Clark <nick@ccl4.org> Date: Tue May 7 09:25:39 2013 +0200 Remove the now unused 'type' argument from the macros DO_BO_(UN)?PACK. M pp_pack.c commit e8966daba327099373354b8935491872e853160e Author: Nicholas Clark <nick@ccl4.org> Date: Mon May 6 20:40:34 2013 +0200 As DO_BO_(UN)?PACK are now implemented with my_swabn(), simplify 'J' and 'j'. The conditionally compiled code to figure out the correct named integer size for UVs and IVs is no longer needed, as my_swabn() will swap UVs and IVs, whatever their length. M pp_pack.c M t/porting/diag.t commit 0c0fa9f456cec3d6fd9e033a5024063564ee1ca7 Author: Nicholas Clark <nick@ccl4.org> Date: Mon May 6 19:58:10 2013 +0200 Merge the identical definitions of DO_BO_UNPACK and DO_BO_PACK. Delete the now-unused mixed-endian definitions of DO_BO_UNPACK and DO_BO_PACK, and the macro BO_CANT_DOIT, as they were the last two things using it. M pp_pack.c commit a1219b5e0bb6c311848c834f67e70ff7a19c6bf4 Author: Nicholas Clark <nick@ccl4.org> Date: Mon May 6 19:41:10 2013 +0200 Swap byte order in DO_BO_(UN)?PACK based on a variable needs_swap. Add the macro NEEDS_SWAP to initialise needs_swap based on TYPE_ENDIANNESS(datumtype). This makes the two definitions of DO_BO_UNPACK identical, and the two definitions of DO_BO_PACK identical. This also makes building pp_pack.c on a mixed endian byteorder architecture a compile time error. The commit adds pointers on where to add code to re-instate support for such architectures. M pp_pack.c commit e8fda8c499f46bb801215ff1dbea51c1f3007a6c Author: Nicholas Clark <nick@ccl4.org> Date: Mon May 6 18:43:26 2013 +0200 DO_BO_{UN,}PACK can use my_swabn() directly. This removes the only users of my_{be,le,h}to{be,le,h}n, so remove the definitions for those macros. M perl.h M pp_pack.c commit 7e38f754fea50c46f3571cd67982e061b3cf8304 Author: Nicholas Clark <nick@ccl4.org> Date: Mon May 6 18:05:24 2013 +0200 Use DO_BO_{UN,}PACK in place of DO_BO_UNPACK_{N,PC} Also, eliminate the intermediate macros DO_BO_PACK_PTR and DO_BO_UNPACK_PTR. M pp_pack.c commit 79be8fb4ac8fa99573841ce07cea480399422f8f Author: Nicholas Clark <nick@ccl4.org> Date: Mon May 6 17:16:18 2013 +0200 Eliminate my_{hto[bl]e,[bl]etoh}{16,32,64,s,i,l} as nothing now uses them. M embed.fnc M embed.h M perl.h M proto.h M util.c commit 68a0946fb4222dd6bcf5c2881bf6cd64c2c2e5ef Author: Nicholas Clark <nick@ccl4.org> Date: Mon May 6 16:54:12 2013 +0200 In pp_pack.c, refactor DO_BO_(UN)?PACK to use my_letohn etc Previously they were using a my_letoh* function for the appropriate size. This commit probably breaks fixed 16 and 32 bit templates on big endian Crays where sizeof(short) and sizeof(int) are 8. A future commit will restore support. (Although it's not clear whether such Crays can still build blead cleanly, as we've not had any feedback since Boeing decommissioned their Cray.) M pp_pack.c commit 86c6fce076648c5abe5ab491fec9d3142d93bde1 Author: Nicholas Clark <nick@ccl4.org> Date: Mon May 6 16:36:09 2013 +0200 In pp_pack.c, refactor DO_BO_(UN)?PACK_PTR to use my_letohn etc Previously they were casting the pointer to an integer value, calling a my_letoh* function for the appropriate size, and casting the return value back to a pointer. M pp_pack.c commit df609c5f9dbf07fc7a18075911f5eaefdeea3ffd Author: Nicholas Clark <nick@ccl4.org> Date: Mon May 6 16:22:01 2013 +0200 In pp_pack.c, refactor DO_BO_(UN)?PACK_N to use sizeof() on the variable. Previously they were using sizeof() on the type argument passed to the macro. This change makes their implementation more like DO_BO_(UN)?PACK and DO_BO_(UN)?PACK_PTR. M pp_pack.c commit 3cc90e2e712615f2081743c38153954d60e3f0df Author: Nicholas Clark <nick@ccl4.org> Date: Mon May 6 15:46:02 2013 +0200 Refactor macros in pp_pack.c, removing support for mixed-endian byte orders. The byte-order handling can be simplified considerably if we don't have to support mixed-endian systems. It's not clear whether Perl 5.000 even compiled on PDP-11s, let alone more recent versions. Support probably can be added back at the end of the refactoring, if someone has time and a PDP-11 to test on, but for the intermediate stages it is a lot easier not to need to think about such platforms. M pp_pack.c commit 691a44df41bbec11b634fc7bbf78a6eeb1f768a3 Author: Nicholas Clark <nick@ccl4.org> Date: Mon May 6 14:40:04 2013 +0200 Eliminate the conditionally-compiled fallback functions for htonl etc. These are now only being used for mixed-endian platforms which do not provide their own htnol (etc) functions. Given that the fallbacks have been buggy since they were added in Perl 3.0, it's safe to conclude that no mixed-endian platforms were ever using these functions. It's also unclear why these functions were ever marked as 'A', part of the API. XS code can't call them directly, as it can't rely on them being compiled. Unsurprisingly, no code on CPAN references them. M embed.fnc M embed.h M perl.h M proto.h M util.c commit 66bb80e4324617840aa70c8af410bf76eedcbea1 Author: Nicholas Clark <nick@ccl4.org> Date: Mon May 6 13:28:20 2013 +0200 Remove buggy loop-based byte swapping code. The irony is that the union-based code special-cased for little endian systems actually works everywhere, even on mixed-endian systems. M pod/perldiag.pod M util.c commit 36bb5e1d607385b423cc4360bce20f3043900790 Author: Nicholas Clark <nick@ccl4.org> Date: Mon May 6 08:20:57 2013 +0200 Update the Win32 miniperl conditional compilation, for the htonl etc changes. As of commit 19253ae62cd13079 (Oct 2012), miniperl on Win32 avoids using Winsock. The win32_* wrappers for htonl etc had used the pre-processor macro MYSWAP to conditionally compile in the correct code. However, MYSWAP was defined as a side effect of using the htonl etc wrappers in util.c, which are no longer needed. Hence use the WIN32_NO_SOCKETS macro directly in win32sck.c for the correct conditional compilation. M win32/win32sck.c commit e4b03fedd1c92616345d5428c7b0d2981b0e00d8 Author: Nicholas Clark <nick@ccl4.org> Date: Sat May 4 22:50:12 2013 +0200 Fallbacks for ntohl, ntohs, htonl and htons for little endian systems. The host byteorder agnostic functions in util.c are now only used on mixed endian systems. M perl.h commit 8e2d7c446d5ab31e6c8e661a3c17f0f0b199d525 Author: Nicholas Clark <nick@ccl4.org> Date: Sat May 4 22:03:59 2013 +0200 Provide ntohl, ntohs, htonl and htons no-op macros on big endian systems. This means that there are always macros or functions for ntohl, ntohs, htonl and htons available, so eliminate use of HAS_NTOHL etc, and unconditionally compile the code that it was protecting. However, as code on CPAN is using these guard macros, define all of them in perl.h (Technically the 4 are not quite no-ops, as they truncate their values to 32 or 16 bits, to be consistent with the implementations for platforms which need re-ordering.) M perl.h M pp_pack.c M pp_sys.c commit 260c6fee6deba90b42b0734fbdea2fea1d69c9bc Author: Nicholas Clark <nick@ccl4.org> Date: Sat May 4 21:30:27 2013 +0200 Provide vtohl, vtohs, htovl and htovs no-op macros on little endian systems. This means that there are always macros or functions for vtohl, vtohs, htovl and htovs available, so eliminate HAS_VTOHL etc, and unconditionally compile the code that it was protecting. grep.cpan.me shows that no code on CPAN uses any of these macros. (Technically the 4 are not quite no-ops, as they truncate their values to 32 or 16 bits, to be consistent with the implementations for platforms which need re-ordering.) M perl.h M pp_pack.c M util.c commit 5ae724797a32a9a98d51759460d87f419fdd5a45 Author: Nicholas Clark <nick@ccl4.org> Date: Sat May 4 21:07:14 2013 +0200 vtohl, vtohs, htovl and htovs are no-ops on 64 bit little endian systems. Previously they were implemented as function calls on 64 bit little endian systems. Bit endian systems implemented them as byte-swapping macros. 32 little endian system didn't implement them at all. 32 and 64 bit little endian systems now behave identically. M perl.h commit fcdf154797de71462d8c8f5606482a86ed73943a Author: Nicholas Clark <nick@ccl4.org> Date: Fri May 3 17:06:01 2013 +0200 Eliminate the definitions for DO_BO_PACK_P and DO_BO_UNPACK_P. Commit 07409e015252427f in April 2005 replaced all the uses of these two macros with DO_BO_PACK_PC and DO_BO_UNPACK_PC. M pp_pack.c commit f8e5a5db38586c726249f161f1fbb581d1738ca2 Author: Nicholas Clark <nick@ccl4.org> Date: Fri May 3 15:52:38 2013 +0200 Eliminate pre-5.9.x conditional code for PERL_PACK_CAN_SHRIEKSIGN PERL_PACK_CAN_SHRIEKSIGN has been unconditionally defined for versions 5.9.x and greater, and undefined for 5.8.x. As we are never going to need to port changes back to maint-5.8 any more, eliminate all the 5.8.x related code and the macro that supports it. M packsizetables.c M pp_pack.c M regen/genpacksizetables.pl commit 5fa237154915dfcb787e0c5403e6260482a5fb3e Author: Nicholas Clark <nick@ccl4.org> Date: Fri May 3 15:26:34 2013 +0200 Eliminate pre-5.9.x conditional code for PERL_PACK_CAN_BYTEORDER PERL_PACK_CAN_BYTEORDER has been unconditionally defined for versions 5.9.x and greater, and undefined for 5.8.x. As we are never going to need to port changes back to maint-5.8 any more, eliminate all the 5.8.x related code, and the macro that supports it. M pp_pack.c commit 39ecab1c690aee1a1fdeefb6c706448af5d2981f Merge: 6d5abc6 843b375 Author: Nicholas Clark <nick@ccl4.org> Date: Mon May 20 21:12:01 2013 +0200 Merge the reworking of how genpacksizetables.pl is used. commit 843b37592a564e2e84c28a00619ac6c132c6dbc9 Author: Nicholas Clark <nick@ccl4.org> Date: Mon Apr 29 12:06:39 2013 +0200 Add regen/genpacksizetables.pl to t/porting/regen.t M t/porting/regen.t commit e3f061af802f7db7c760d810c495d83763fff668 Author: Nicholas Clark <nick@ccl4.org> Date: Mon Apr 29 11:54:19 2013 +0200 Move genpacksizetables.pl to regen/genpacksizetables.pl M MANIFEST D genpacksizetables.pl M packsizetables.c A regen/genpacksizetables.pl commit 6b05cb8115fc15fed0e6bf162a9cc797222572ee Author: Nicholas Clark <nick@ccl4.org> Date: Mon Apr 29 11:47:30 2013 +0200 Refactor genpacksizetables.pl to use regen/regen_lib.pl M genpacksizetables.pl M packsizetables.c commit 298bc19c0935302faa4957de141245bb64c6b2df Author: Nicholas Clark <nick@ccl4.org> Date: Mon Apr 29 11:23:11 2013 +0200 Extract the generated packprops array into packsizetables.c. The C source is the output from genpacksizetables.pl Previously it was pasted into pp_pack.c LocalWords: packprops M MANIFEST A packsizetables.c M pp_pack.c commit e2add64b70b6d5c55a6598257032bf0777c5976c Author: Nicholas Clark <nick@ccl4.org> Date: Mon Apr 29 11:08:05 2013 +0200 Propagate a change from commit 1651fc447620d361 into genpacksizetables.pl. pp_pack.c contains a table generated by genpacksizetables.pl, pasted into the C source. The C source was updated by commit 1651fc447620d361 in April 2007, but the table used to generate the code was not. M genpacksizetables.pl commit d8a3828919c2d8a9e1344ba4d05795d37ed9cdb6 Author: Karl Williamson <public@khwilliamson.com> Date: Wed Mar 13 16:18:12 2013 -0600 genpacksizetables.pl: Correct comment typo M genpacksizetables.pl commit 6d5abc62848da0e110869e2b13c68bf498ff3a90 Author: Nicholas Clark <nick@ccl4.org> Date: Wed Mar 6 16:27:08 2013 +0100 Change LIKELY() and UNLIKELY() to cast their first argument to a bool. This avoids problems such as pointers being compared with the literal value 1. Suggested by Zefram. Note that this is technically a change of behaviour, as the macro EXPECT(), which they are both both wrappers for, returns its value, so they will now be returning TRUE or FALSE, rather than the actual value passed to them. However, all uses in the core and on CPAN are only within if() statements, so this should not make any difference in practice. M perl.h commit b2bf251fd850e281217ec1d042c1839e3ed4a09c Author: Karl Williamson <public@khwilliamson.com> Date: Sat May 18 14:05:00 2013 -0600 handy.h: Change the error return of two macros These two undocumented macros returned the REPLACEMENT CHARACTER if the input was outside the Latin1 range. This was contrary to all other similar macros, which return their input if it is invalid. It caused warnings in some (dumber than average) compilers. These macros are undocumented; this changes the behavior only of illegal inputs to them. M handy.h commit cfe934daf72e37a0980ac3183d678e49a26726b3 Author: Karl Williamson <public@khwilliamson.com> Date: Sat May 18 13:55:50 2013 -0600 regcomp.c: Add comment M regcomp.c commit 4f48919406dde7316da0835587373a50763e73ae Author: Karl Williamson <public@khwilliamson.com> Date: Sat May 18 13:06:47 2013 -0600 utf8.c: White space only. Indent in newly formed block M utf8.c commit a3f3bf18711ca1acde9e5a41d4c7b1f3adc4afd1 Author: Karl Williamson <public@khwilliamson.com> Date: Sat May 18 13:09:05 2013 -0600 fold_grind.t: Add tests fold-grind stresses the /i regex handling by reading the Unicode folding rules and creating tests for problematic cases, as well as some non-problematic ones. Because of the large number of tests generated, it otherwise avoids tests that it doesn't think are problematic. Problematic cases include those whose folds cross the ordinal 127/128 or 255/256 boundaries, along with other considerations. Until this commit, fold_grind failed to realize that even if the pair of characters currently being tested don't cross those boundaries, if their eventual folds do, this could be problematic. Suppose X and Y are both on the same side of the boundaries, and both fold to Z which is on another side. Then X and Y should be equivalent under /i even if the fold to Z is prohibited because of /aa or /l. fold_grind was overzealous in pruning such tests. The previous patch fixed bugs in handling such cases; and this patch fixes the tests to look for similar things that might possibly arise in the future. Interestingly, this bug came to light during porting to an EBCDIC platform. The reason is that because of the different collation order, fold_grind chose one of the buggy cases to test as one its non-problematic tests. M t/re/fold_grind.t commit 1ca267a56acf698557ec1deec44af651acc88696 Author: Karl Williamson <public@khwilliamson.com> Date: Sat May 18 08:25:16 2013 -0600 Fix multi-char fold edge case use locale; fc("\N{LATIN CAPITAL LETTER SHARP S}") eq 2 x fc("\N{LATIN SMALL LETTER LONG S}") should return true, as the SHARP S folds to two 's's in a row, and the LONG S is an antique variant of 's', and folds to s. Until this commit, the expression was false. Similarly, the following should match, but didn't until this commit: "\N{LATIN SMALL LETTER SHARP S}" =~ /\N{LATIN SMALL LETTER LONG S}{2}/iaa The reason these didn't work properly is that in both cases the actual fold to 's' is disallowed. In the first case because of locale; and in the second because of /aa. And the code wasn't smart enough to realize that these were legal. The fix is to special case these so that the fold of sharp s (both capital and small) is two LONG S's under /aa; as is the fold of the capital sharp s under locale. The latter is user-visible, and the documentation of fc() now points that out. I believe this is such an edge case that no mention of it need be done in perldelta. M lib/locale.t M pod/perlfunc.pod M regcharclass.h M regcomp.c M regen/regcharclass_multi_char_folds.pl M t/uni/fold.t M utf8.c commit 519101418837cf0edacb710b2b38b42dad6e47c1 Author: Karl Williamson <public@khwilliamson.com> Date: Fri May 17 23:07:05 2013 -0600 Expand flags parameter from boolean in _to_fold_latin1 This will be used in future commits to pass more flags. M embed.fnc M proto.h M regcomp.c M regexec.c M utf8.c commit f2e063754de41c901afa81619aeace2fb9d03f9c Author: Karl Williamson <public@khwilliamson.com> Date: Thu May 16 21:37:43 2013 -0600 unicode_constants.h: Add some #defines These will be used in future commits M regen/unicode_constants.pl M unicode_constants.h commit a4f12ed78e91a49ab4fec1f1e637d980366cbc1a Author: Karl Williamson <public@khwilliamson.com> Date: Wed May 15 20:55:27 2013 -0600 utf8.c: Replace two macro calls with equiv single UTF8_IS_ABOVE_LATIN1() is equivalent to (! UTF8_IS_INVARIANT && !UTF8_IS_DOWNGRADEABLE_START) So we can use just it, for clearer code with fewer branches. M utf8.c commit 33bdd83dc2db4d07c1540394340770a6a07e793e Author: Karl Williamson <public@khwilliamson.com> Date: Sat May 11 16:00:02 2013 -0600 lib/charnames.t: Fix wrongly complemented test The 'if' test should be the opposite of what it is. I believe I had it this way for forcing the branch to be taken during testing, and forgot to restore it. It only matters if Perl is compiled with early Unicodes, or on a non-ASCII platform. M lib/charnames.t commit 91638220a6a521d89235c0a8e51d53f96e0e67c8 Author: Karl Williamson <public@khwilliamson.com> Date: Thu May 9 20:48:40 2013 -0600 t/re/fold_grind.t: White space only Commit 3345a47950127cf99a197eba4dce9c91f7bc9139 created a new block; but didn't indent it. This commit does. M t/re/fold_grind.t commit 58a428bba943d9d3e6057e7ea3202ea3562c557a Author: Karl Williamson <public@khwilliamson.com> Date: Tue May 7 10:04:40 2013 -0600 perlapi: Move 'experimental' warning to front of entries In a long multi-paragraph entry, the fact that the described function is considered experimental may be lost, as it comes at the end. This just moves it to the front. M autodoc.pl commit fdfb35c800b283d7f03f2b03dc5a1adb81a4a6e4 Author: Karl Williamson <public@khwilliamson.com> Date: Tue May 7 10:01:58 2013 -0600 regcomp.c: Remove always-true test In this code, j is guaranteed to be above 255, so no need to test for that. M regcomp.c commit ff5deb98f57138ca084471a3abb7b702b1347af8 Author: Karl Williamson <public@khwilliamson.com> Date: Sun May 5 18:26:39 2013 -0600 regcomp.c: White-space only The previous commit allows us to outdent a largish block, reflowing things to fit into the extra available width, and saving a few vertical pixels. M regcomp.c commit b35bbc3f802dc4b4abbbc28ed19bc5108f2b7ef1 Author: Karl Williamson <public@khwilliamson.com> Date: Sun May 5 10:40:56 2013 -0600 regcomp.c: Reorder two 'if's so shorter branches are first This makes it easier to understand what is going on M regcomp.c commit b9cc4f69dc87bbd21b54bb9a4466203cf97afacc Author: Karl Williamson <public@khwilliamson.com> Date: Sat May 4 22:22:53 2013 -0600 perllocale: Update to mention fc(), \F M pod/perllocale.pod commit d3cb55c1b8a3b80c74a4fde3bd7b67b7af31baea Author: Karl Williamson <public@khwilliamson.com> Date: Sat May 4 21:23:05 2013 -0600 regcomp.c: Clarify comment M regcomp.c commit a78bc3c6c5c9dd76fdfbdae88ef4e89136aff637 Author: Karl Williamson <public@khwilliamson.com> Date: Sat May 4 20:23:14 2013 -0600 pp.c: Eliminate custom macro and use Copy() instead I think it's clearer to use Copy. When I wrote this custom macro, we didn't have the infrastructure to generate a UTF-8 encoded string at compile time. M pp.c M regen/unicode_constants.pl M unicode_constants.h commit a96bc63507f57f3547f982d5e9a8b95f68e87e3b Author: Dagfinn Ilmari MannsÃ¥ker <ilmari@ilmari.org> Date: Sat May 4 20:18:35 2013 -0600 perl #117865] [PATCH] Eliminate useless variable and sizeof(char) bufsiz is always just set from bsiz (via a useless multiplication by sizeof(char), which is by definition 1), so instead of trying to keep them in sync, just get rid of bufsiz use bsiz directly# Please enter the commit message for your changes. Lines starting M util.c commit 7ccde1209c3d4319a2e0ab9a4528f6abc8eec0d2 Author: Karl Williamson <public@khwilliamson.com> Date: Sat May 4 16:09:39 2013 -0600 op.c: Don't fold constant fc() in locale We do compile time folding of calls to ops with constant parameters. This should be skipped if the op's result depends on locale, and it is being called from within the scope of 'use locale', as the result is not known until runtime. fc() was folding anyway. There is no test, as this only shows up when run in a locale that it makes a difference in, and there is no guarantee that such a locale would occur on any computer, and it is a real pain to go searching through the computer's available locales for such a one for just this error. M op.c commit 8c869419331313c43ddda5b1eecd01939b013125 Author: Karl Williamson <public@khwilliamson.com> Date: Tue Apr 30 08:42:08 2013 -0600 autodoc.pl: Don't list undocumented deprecated fcns in API autodoc creates a list of all the undocumented functions that are part of the API. It omits ones that are experimental and whose API may change; and now it omits ones that are deprecated (and whose API is planned to change to be non-existent) M autodoc.pl commit d4e99c76278c7e400307b1a7ceeb3b27676dfd3d Author: Karl Williamson <public@khwilliamson.com> Date: Tue Apr 30 08:39:44 2013 -0600 autodoc.pl: Add note for deprecated functions This causes each deprecated function to have a prominent note to that effect in its API documentation. M autodoc.pl M mg.c M utf8.c commit 6ef8d77a18f45460857e0866d103b834ec25d432 Author: Karl Williamson <public@khwilliamson.com> Date: Sat Apr 27 08:59:19 2013 -0600 embed.fnc: Slight clarification in comments M embed.fnc commit 2e1414cec3028c7f6c19d1229a4b9a460267fb15 Author: Karl Williamson <public@khwilliamson.com> Date: Mon May 20 10:39:56 2013 -0600 Add tests for case-change macros API M ext/XS-APItest/APItest.pm M ext/XS-APItest/APItest.xs M ext/XS-APItest/t/handy.t commit d22b930b662c8c65d2e856c42f4a59454444a111 Author: Karl Williamson <public@khwilliamson.com> Date: Thu Apr 25 10:07:47 2013 -0600 Use new case changing macros The previous commit added macros to do some case changing. This commit uses them in the core, where appropriate. M ext/File-Glob/bsd_glob.c M pp.c M utf8.c M utf8.h commit 25200305a51f99449a69c43f5d279893ab5a0188 Author: Karl Williamson <public@khwilliamson.com> Date: Thu Apr 25 10:05:22 2013 -0600 handy.h: Add some macro definitions These macros fill in all the missing case changing operations. They were omitted before because they are identical in their input domains to other operations. But by adding them here, that detail no longer need be known by the callers. toFOLD_LC is not documented, as is subject to change M handy.h commit 22e255cb04e4d885f89e7fa81ad967031dd8c193 Author: Karl Williamson <public@khwilliamson.com> Date: Thu Apr 25 08:41:13 2013 -0600 pp.c: Nit in comment M pp.c commit fa2b1084e3c2ca84300c9da8bdd7f808b78d35bf Author: Karl Williamson <public@khwilliamson.com> Date: Wed Apr 24 15:39:08 2013 -0600 perlclib.pod: Update character class macro descriptions Much has changed since this pod was last updated. M pod/perlclib.pod commit 5b2821400d3dea1b132a955e865a58e89133c569 Author: Karl Williamson <public@khwilliamson.com> Date: Wed Apr 24 15:36:07 2013 -0600 perlclib.pod: Fix too long verbatim lines M pod/perlclib.pod M t/porting/known_pod_issues.dat commit 1f607577c172af48670c82975935160a27e002b9 Author: Karl Williamson <public@khwilliamson.com> Date: Wed Apr 24 21:18:14 2013 -0600 perlapi: Add docs for some case-changing macros; clarify others The case changing macros are now almost all documented. The exception is toUPPER_LC, which may change in 5.19 In addition the functions in utf8.c that these macros call now refer to them instead of having their own documentation. People should really be using the macros instead of calling the functions directly. I'm not deprecating the functions because I can't foresee the need to change them, so code that uses them should continue to be ok. M handy.h M utf8.c ----------------------------------------------------------------------- -- Perl5 Master Repository [perl.git] branch smoke-me/nicholas/x2p-depend-later, deleted. v5.18.0-RC3-17-g12bc250 by Nicholas Clark 2013-05-20T20:19:50Z From: Nicholas Clark In perl.git, the branch smoke-me/nicholas/x2p-depend-later has been deleted was 12bc250a20b24833ce9d3178f1cdaa914d233bc7 ----------------------------------------------------------------------- 12bc250a20b24833ce9d3178f1cdaa914d233bc7 Simplify x2p's Makefile. ----------------------------------------------------------------------- -- Perl5 Master Repository [perl.git] branch smoke-me/nicholas/genpacksizetables, deleted. v5.17.11-146-gbe30ec3 by Nicholas Clark 2013-05-20T19:50:36Z From: Nicholas Clark In perl.git, the branch smoke-me/nicholas/genpacksizetables has been deleted was be30ec3cf98529429db8e80b068b3f3479644d7d ----------------------------------------------------------------------- be30ec3cf98529429db8e80b068b3f3479644d7d Annotate the "cold" paths in pp_pack.c with UNLIKELY(). ----------------------------------------------------------------------- -- Perl5 Master Repository [perl.git] branch blead, updated. v5.19.0-96-g9e7945e by Nicholas Clark 2013-05-20T19:50:36Z From: Nicholas Clark In perl.git, the branch blead has been updated - Log ----------------------------------------------------------------- commit 9e7945ee50c30a17a84968fb0c7e5c38223af6f9 Merge: 39ecab1 7df6ffa Author: Nicholas Clark Date: Mon May 20 21:42:33 2013 +0200 Merge the pack refactoring to blead. commit 7df6ffa3fb2efaa96f5e6001f1cc311ad9f8b185 Author: Nicholas Clark Date: Mon May 20 21:40:27 2013 +0200 Notes in perldelta for the tiny XS-visible changes of the pack refactoring. M pod/perldelta.pod commit 228e69a781c23b2599bbd89f1de29c212f10ea31 Author: Nicholas Clark Date: Wed May 8 10:40:07 2013 +0200 Annotate the "cold" paths in pp_pack.c with UNLIKELY(). The usual case is nice regular bytes in the host's nice regular order. M pp_pack.c commit 545872c5eddf1f00b9826a3d8d682387f1c5049d Author: Nicholas Clark Date: Tue May 7 18:25:53 2013 +0200 Eliminate the implementations of [hv]to[vh][ls] for mixed-endian systems. As pp_pack.c has had mixed-endian support removed, there is little point in keeping code in perl.h and util.c only needed for architectures that cannot be built. M perl.h M util.c commit 1cb43330cc18f2eee5ab1eb58e65991407f4b243 Author: Nicholas Clark Date: Tue May 7 18:17:39 2013 +0200 Eliminate Perl_my_swabn(), as it is now unused. It is not marked as part of the API, and no code on CPAN is using it. M embed.fnc M embed.h M proto.h M util.c commit 3a88beaa68dbb5bad93145daa0c829e0aeb40adb Author: Nicholas Clark Date: Tue May 7 17:39:42 2013 +0200 When endian-swapping in pack, simply copy the bytes in reverse order. This should restore support for big endian Crays. It doesn't support mixed-endian systems. M embed.fnc M pp_pack.c M proto.h commit aaec8192358ffb8080ad85754ea9aeca93b06c8b Author: Nicholas Clark Date: Tue May 7 16:22:08 2013 +0200 Pass 'needs_swap' as a parameter to SHIFT_BYTES. It feels wrong to have it as an implicit parameter sucked in via the textual expansion of the macro. Whilst it can be derived from the parameter 'datumtype', it seems that the C compiler generates a lot less efficient code that way. M pp_pack.c commit 20aa3a3813ca2068a61ea9582e99df57cc735e62 Author: Nicholas Clark Date: Tue May 7 15:15:54 2013 +0200 When endian-swapping in unpack, simply copy the bytes in reverse order. It is considerably simpler to re-order the bytes before reading them into the variable of the desired type, than to read into the variable and then need a specialised "reverse this integer" function for each size of integer. This should restore support for big endian Crays. It doesn't support mixed-endian systems. Support for mixed-endian systems can be restored (if needed) by re-ordering the bytes correctly at the locations which currently only know how to reverse the bytes. M pp_pack.c commit cd7e75121be11e7979933a8f7a47cd842a685157 Author: Nicholas Clark Date: Tue May 7 12:23:00 2013 +0200 Fold the implementation of DO_BO_UNPACK into SHIFT_BYTES(), its only user. M pp_pack.c commit 7285e3f43c94010f09aa1b7a3ccbdbe7de1aa3e8 Author: Nicholas Clark Date: Tue May 7 11:53:17 2013 +0200 Re-implement SHIFT16 and SHIFT32 as wrappers to SHIFT_BYTES. M pp_pack.c commit e396d2357dcec18c01b1cc78d9edc80a3aa4066d Author: Nicholas Clark Date: Tue May 7 10:08:53 2013 +0200 Refactor the two uses of SHIFT{16,32} not followed by DO_BO_UNPACK(). There is one place where SHIFT16() is not immediately followed by DO_BO_UNPACK() ('n!' and 'v!'), and one place where SHIFT32() is not immediately followed by DO_BO_UNPACK() ('N!' and 'V!'). Every other use of either macro is followed by DO_BO_UNPACK(). For the two locations that don't have DO_BO_UNPACK(), the modifiers '<' and '>' are illegal, hence DO_BO_UNPACK() will always be a no-op. Hence add DO_BO_UNPACK() to both (which permits refactoring to simplify things), and add assert()s that the assumptions are true. M pp_pack.c commit 2b4ad5699f46c7f63185ad4e7d1f61e16aaa73f6 Author: Nicholas Clark Date: Tue May 7 09:25:39 2013 +0200 Remove the now unused 'type' argument from the macros DO_BO_(UN)?PACK. M pp_pack.c commit e8966daba327099373354b8935491872e853160e Author: Nicholas Clark Date: Mon May 6 20:40:34 2013 +0200 As DO_BO_(UN)?PACK are now implemented with my_swabn(), simplify 'J' and 'j'. The conditionally compiled code to figure out the correct named integer size for UVs and IVs is no longer needed, as my_swabn() will swap UVs and IVs, whatever their length. M pp_pack.c M t/porting/diag.t commit 0c0fa9f456cec3d6fd9e033a5024063564ee1ca7 Author: Nicholas Clark Date: Mon May 6 19:58:10 2013 +0200 Merge the identical definitions of DO_BO_UNPACK and DO_BO_PACK. Delete the now-unused mixed-endian definitions of DO_BO_UNPACK and DO_BO_PACK, and the macro BO_CANT_DOIT, as they were the last two things using it. M pp_pack.c commit a1219b5e0bb6c311848c834f67e70ff7a19c6bf4 Author: Nicholas Clark Date: Mon May 6 19:41:10 2013 +0200 Swap byte order in DO_BO_(UN)?PACK based on a variable needs_swap. Add the macro NEEDS_SWAP to initialise needs_swap based on TYPE_ENDIANNESS(datumtype). This makes the two definitions of DO_BO_UNPACK identical, and the two definitions of DO_BO_PACK identical. This also makes building pp_pack.c on a mixed endian byteorder architecture a compile time error. The commit adds pointers on where to add code to re-instate support for such architectures. M pp_pack.c commit e8fda8c499f46bb801215ff1dbea51c1f3007a6c Author: Nicholas Clark Date: Mon May 6 18:43:26 2013 +0200 DO_BO_{UN,}PACK can use my_swabn() directly. This removes the only users of my_{be,le,h}to{be,le,h}n, so remove the definitions for those macros. M perl.h M pp_pack.c commit 7e38f754fea50c46f3571cd67982e061b3cf8304 Author: Nicholas Clark Date: Mon May 6 18:05:24 2013 +0200 Use DO_BO_{UN,}PACK in place of DO_BO_UNPACK_{N,PC} Also, eliminate the intermediate macros DO_BO_PACK_PTR and DO_BO_UNPACK_PTR. M pp_pack.c commit 79be8fb4ac8fa99573841ce07cea480399422f8f Author: Nicholas Clark Date: Mon May 6 17:16:18 2013 +0200 Eliminate my_{hto[bl]e,[bl]etoh}{16,32,64,s,i,l} as nothing now uses them. M embed.fnc M embed.h M perl.h M proto.h M util.c commit 68a0946fb4222dd6bcf5c2881bf6cd64c2c2e5ef Author: Nicholas Clark Date: Mon May 6 16:54:12 2013 +0200 In pp_pack.c, refactor DO_BO_(UN)?PACK to use my_letohn etc Previously they were using a my_letoh* function for the appropriate size. This commit probably breaks fixed 16 and 32 bit templates on big endian Crays where sizeof(short) and sizeof(int) are 8. A future commit will restore support. (Although it's not clear whether such Crays can still build blead cleanly, as we've not had any feedback since Boeing decommissioned their Cray.) M pp_pack.c commit 86c6fce076648c5abe5ab491fec9d3142d93bde1 Author: Nicholas Clark Date: Mon May 6 16:36:09 2013 +0200 In pp_pack.c, refactor DO_BO_(UN)?PACK_PTR to use my_letohn etc Previously they were casting the pointer to an integer value, calling a my_letoh* function for the appropriate size, and casting the return value back to a pointer. M pp_pack.c commit df609c5f9dbf07fc7a18075911f5eaefdeea3ffd Author: Nicholas Clark Date: Mon May 6 16:22:01 2013 +0200 In pp_pack.c, refactor DO_BO_(UN)?PACK_N to use sizeof() on the variable. Previously they were using sizeof() on the type argument passed to the macro. This change makes their implementation more like DO_BO_(UN)?PACK and DO_BO_(UN)?PACK_PTR. M pp_pack.c commit 3cc90e2e712615f2081743c38153954d60e3f0df Author: Nicholas Clark Date: Mon May 6 15:46:02 2013 +0200 Refactor macros in pp_pack.c, removing support for mixed-endian byte orders. The byte-order handling can be simplified considerably if we don't have to support mixed-endian systems. It's not clear whether Perl 5.000 even compiled on PDP-11s, let alone more recent versions. Support probably can be added back at the end of the refactoring, if someone has time and a PDP-11 to test on, but for the intermediate stages it is a lot easier not to need to think about such platforms. M pp_pack.c commit 691a44df41bbec11b634fc7bbf78a6eeb1f768a3 Author: Nicholas Clark Date: Mon May 6 14:40:04 2013 +0200 Eliminate the conditionally-compiled fallback functions for htonl etc. These are now only being used for mixed-endian platforms which do not provide their own htnol (etc) functions. Given that the fallbacks have been buggy since they were added in Perl 3.0, it's safe to conclude that no mixed-endian platforms were ever using these functions. It's also unclear why these functions were ever marked as 'A', part of the API. XS code can't call them directly, as it can't rely on them being compiled. Unsurprisingly, no code on CPAN references them. M embed.fnc M embed.h M perl.h M proto.h M util.c commit 66bb80e4324617840aa70c8af410bf76eedcbea1 Author: Nicholas Clark Date: Mon May 6 13:28:20 2013 +0200 Remove buggy loop-based byte swapping code. The irony is that the union-based code special-cased for little endian systems actually works everywhere, even on mixed-endian systems. M pod/perldiag.pod M util.c commit 36bb5e1d607385b423cc4360bce20f3043900790 Author: Nicholas Clark Date: Mon May 6 08:20:57 2013 +0200 Update the Win32 miniperl conditional compilation, for the htonl etc changes. As of commit 19253ae62cd13079 (Oct 2012), miniperl on Win32 avoids using Winsock. The win32_* wrappers for htonl etc had used the pre-processor macro MYSWAP to conditionally compile in the correct code. However, MYSWAP was defined as a side effect of using the htonl etc wrappers in util.c, which are no longer needed. Hence use the WIN32_NO_SOCKETS macro directly in win32sck.c for the correct conditional compilation. M win32/win32sck.c commit e4b03fedd1c92616345d5428c7b0d2981b0e00d8 Author: Nicholas Clark Date: Sat May 4 22:50:12 2013 +0200 Fallbacks for ntohl, ntohs, htonl and htons for little endian systems. The host byteorder agnostic functions in util.c are now only used on mixed endian systems. M perl.h commit 8e2d7c446d5ab31e6c8e661a3c17f0f0b199d525 Author: Nicholas Clark Date: Sat May 4 22:03:59 2013 +0200 Provide ntohl, ntohs, htonl and htons no-op macros on big endian systems. This means that there are always macros or functions for ntohl, ntohs, htonl and htons available, so eliminate use of HAS_NTOHL etc, and unconditionally compile the code that it was protecting. However, as code on CPAN is using these guard macros, define all of them in perl.h (Technically the 4 are not quite no-ops, as they truncate their values to 32 or 16 bits, to be consistent with the implementations for platforms which need re-ordering.) M perl.h M pp_pack.c M pp_sys.c commit 260c6fee6deba90b42b0734fbdea2fea1d69c9bc Author: Nicholas Clark Date: Sat May 4 21:30:27 2013 +0200 Provide vtohl, vtohs, htovl and htovs no-op macros on little endian systems. This means that there are always macros or functions for vtohl, vtohs, htovl and htovs available, so eliminate HAS_VTOHL etc, and unconditionally compile the code that it was protecting. grep.cpan.me shows that no code on CPAN uses any of these macros. (Technically the 4 are not quite no-ops, as they truncate their values to 32 or 16 bits, to be consistent with the implementations for platforms which need re-ordering.) M perl.h M pp_pack.c M util.c commit 5ae724797a32a9a98d51759460d87f419fdd5a45 Author: Nicholas Clark Date: Sat May 4 21:07:14 2013 +0200 vtohl, vtohs, htovl and htovs are no-ops on 64 bit little endian systems. Previously they were implemented as function calls on 64 bit little endian systems. Bit endian systems implemented them as byte-swapping macros. 32 little endian system didn't implement them at all. 32 and 64 bit little endian systems now behave identically. M perl.h commit fcdf154797de71462d8c8f5606482a86ed73943a Author: Nicholas Clark Date: Fri May 3 17:06:01 2013 +0200 Eliminate the definitions for DO_BO_PACK_P and DO_BO_UNPACK_P. Commit 07409e015252427f in April 2005 replaced all the uses of these two macros with DO_BO_PACK_PC and DO_BO_UNPACK_PC. M pp_pack.c commit f8e5a5db38586c726249f161f1fbb581d1738ca2 Author: Nicholas Clark Date: Fri May 3 15:52:38 2013 +0200 Eliminate pre-5.9.x conditional code for PERL_PACK_CAN_SHRIEKSIGN PERL_PACK_CAN_SHRIEKSIGN has been unconditionally defined for versions 5.9.x and greater, and undefined for 5.8.x. As we are never going to need to port changes back to maint-5.8 any more, eliminate all the 5.8.x related code and the macro that supports it. M packsizetables.c M pp_pack.c M regen/genpacksizetables.pl commit 5fa237154915dfcb787e0c5403e6260482a5fb3e Author: Nicholas Clark Date: Fri May 3 15:26:34 2013 +0200 Eliminate pre-5.9.x conditional code for PERL_PACK_CAN_BYTEORDER PERL_PACK_CAN_BYTEORDER has been unconditionally defined for versions 5.9.x and greater, and undefined for 5.8.x. As we are never going to need to port changes back to maint-5.8 any more, eliminate all the 5.8.x related code, and the macro that supports it. M pp_pack.c ----------------------------------------------------------------------- Summary of changes: embed.fnc | 86 +------- embed.h | 78 ------- packsizetables.c | 32 --- perl.h | 279 +++++------------------- pod/perldelta.pod | 12 +- pod/perldiag.pod | 5 - pp_pack.c | 522 ++++++++++++------------------------------- pp_sys.c | 10 - proto.h | 93 +-------- regen/genpacksizetables.pl | 8 +- t/porting/diag.t | 2 - util.c | 329 ---------------------------- win32/win32sck.c | 20 +-- 13 files changed, 227 insertions(+), 1249 deletions(-) diff --git a/embed.fnc b/embed.fnc index 8b73d6d..0e2d854 100644 --- a/embed.fnc +++ b/embed.fnc @@ -888,11 +888,6 @@ Ap |void |my_setenv |NULLOK const char* nam|NULLOK const char* val Apmb |I32 |my_stat pX |I32 |my_stat_flags |NULLOK const U32 flags Ap |char * |my_strftime |NN const char *fmt|int sec|int min|int hour|int mday|int mon|int year|int wday|int yday|int isdst -#if defined(MYSWAP) -ApPR |short |my_swap |short s -ApPR |long |my_htonl |long l -ApPR |long |my_ntohl |long l -#endif : Used in pp_ctl.c p |void |my_unexec Apa |OP* |newANONLIST |NULLOK OP* o @@ -1902,7 +1897,8 @@ sR |const char *|get_num |NN const char *patptr|NN I32 *lenptr ns |bool |need_utf8 |NN const char *pat|NN const char *patend ns |char |first_symbol |NN const char *pat|NN const char *patend sR |char * |sv_exp_grow |NN SV *sv|STRLEN needed -snR |char * |bytes_to_uni |NN const U8 *start|STRLEN len|NN char *dest +snR |char * |bytes_to_uni |NN const U8 *start|STRLEN len|NN char *dest \ + |const bool needs_swap #endif #if defined(PERL_IN_PP_CTL_C) @@ -2463,84 +2459,6 @@ sMd |SV* |find_uninit_var|NULLOK const OP *const obase \ |NULLOK const SV *const uninit_sv|bool top #endif -#ifdef PERL_NEED_MY_HTOLE16 -np |U16 |my_htole16 |U16 n -#endif -#ifdef PERL_NEED_MY_LETOH16 -np |U16 |my_letoh16 |U16 n -#endif -#ifdef PERL_NEED_MY_HTOBE16 -np |U16 |my_htobe16 |U16 n -#endif -#ifdef PERL_NEED_MY_BETOH16 -np |U16 |my_betoh16 |U16 n -#endif -#ifdef PERL_NEED_MY_HTOLE32 -np |U32 |my_htole32 |U32 n -#endif -#ifdef PERL_NEED_MY_LETOH32 -np |U32 |my_letoh32 |U32 n -#endif -#ifdef PERL_NEED_MY_HTOBE32 -np |U32 |my_htobe32 |U32 n -#endif -#ifdef PERL_NEED_MY_BETOH32 -np |U32 |my_betoh32 |U32 n -#endif -#ifdef PERL_NEED_MY_HTOLE64 -np |U64 |my_htole64 |U64 n -#endif -#ifdef PERL_NEED_MY_LETOH64 -np |U64 |my_letoh64 |U64 n -#endif -#ifdef PERL_NEED_MY_HTOBE64 -np |U64 |my_htobe64 |U64 n -#endif -#ifdef PERL_NEED_MY_BETOH64 -np |U64 |my_betoh64 |U64 n -#endif - -#ifdef PERL_NEED_MY_HTOLES -np |short |my_htoles |short n -#endif -#ifdef PERL_NEED_MY_LETOHS -np |short |my_letohs |short n -#endif -#ifdef PERL_NEED_MY_HTOBES -np |short |my_htobes |short n -#endif -#ifdef PERL_NEED_MY_BETOHS -np |short |my_betohs |short n -#endif -#ifdef PERL_NEED_MY_HTOLEI -np |int |my_htolei |int n -#endif -#ifdef PERL_NEED_MY_LETOHI -np |int |my_letohi |int n -#endif -#ifdef PERL_NEED_MY_HTOBEI -np |int |my_htobei |int n -#endif -#ifdef PERL_NEED_MY_BETOHI -np |int |my_betohi |int n -#endif -#ifdef PERL_NEED_MY_HTOLEL -np |long |my_htolel |long n -#endif -#ifdef PERL_NEED_MY_LETOHL -np |long |my_letohl |long n -#endif -#ifdef PERL_NEED_MY_HTOBEL -np |long |my_htobel |long n -#endif -#ifdef PERL_NEED_MY_BETOHL -np |long |my_betohl |long n -#endif - -: I think that these are only used by the above, which are macros, and in turn -: currently they are only used in pp_pack.c, but this is in util.c -np |void |my_swabn |NN void* ptr|int n - Ap |GV* |gv_fetchpvn_flags|NN const char* name|STRLEN len|I32 flags|const svtype sv_type Ap |GV* |gv_fetchsv|NN SV *name|I32 flags|const svtype sv_type diff --git a/embed.h b/embed.h index 9054358..1f398d6 100644 --- a/embed.h +++ b/embed.h @@ -774,11 +774,6 @@ #define dump_mstats(a) Perl_dump_mstats(aTHX_ a) #define get_mstats(a,b,c) Perl_get_mstats(aTHX_ a,b,c) #endif -#if defined(MYSWAP) -#define my_htonl(a) Perl_my_htonl(aTHX_ a) -#define my_ntohl(a) Perl_my_ntohl(aTHX_ a) -#define my_swap(a) Perl_my_swap(aTHX_ a) -#endif #if defined(PERL_GLOBAL_STRUCT) #define GetVars() Perl_GetVars(aTHX) #define free_global_struct(a) Perl_free_global_struct(aTHX_ a) @@ -1174,7 +1169,6 @@ #define my_clearenv() Perl_my_clearenv(aTHX) #define my_lstat_flags(a) Perl_my_lstat_flags(aTHX_ a) #define my_stat_flags(a) Perl_my_stat_flags(aTHX_ a) -#define my_swabn Perl_my_swabn #define my_unexec() Perl_my_unexec(aTHX) #define newATTRSUB_flags(a,b,c,d,e,f) Perl_newATTRSUB_flags(aTHX_ a,b,c,d,e,f) #define newSTUB(a,b) Perl_newSTUB(aTHX_ a,b) @@ -1701,78 +1695,6 @@ #define xmldump_sub_perl(a,b) Perl_xmldump_sub_perl(aTHX_ a,b) #define xmldump_vindent(a,b,c,d) Perl_xmldump_vindent(aTHX_ a,b,c,d) # endif -# if defined(PERL_NEED_MY_BETOH16) -#define my_betoh16 Perl_my_betoh16 -# endif -# if defined(PERL_NEED_MY_BETOH32) -#define my_betoh32 Perl_my_betoh32 -# endif -# if defined(PERL_NEED_MY_BETOH64) -#define my_betoh64 Perl_my_betoh64 -# endif -# if defined(PERL_NEED_MY_BETOHI) -#define my_betohi Perl_my_betohi -# endif -# if defined(PERL_NEED_MY_BETOHL) -#define my_betohl Perl_my_betohl -# endif -# if defined(PERL_NEED_MY_BETOHS) -#define my_betohs Perl_my_betohs -# endif -# if defined(PERL_NEED_MY_HTOBE16) -#define my_htobe16 Perl_my_htobe16 -# endif -# if defined(PERL_NEED_MY_HTOBE32) -#define my_htobe32 Perl_my_htobe32 -# endif -# if defined(PERL_NEED_MY_HTOBE64) -#define my_htobe64 Perl_my_htobe64 -# endif -# if defined(PERL_NEED_MY_HTOBEI) -#define my_htobei Perl_my_htobei -# endif -# if defined(PERL_NEED_MY_HTOBEL) -#define my_htobel Perl_my_htobel -# endif -# if defined(PERL_NEED_MY_HTOBES) -#define my_htobes Perl_my_htobes -# endif -# if defined(PERL_NEED_MY_HTOLE16) -#define my_htole16 Perl_my_htole16 -# endif -# if defined(PERL_NEED_MY_HTOLE32) -#define my_htole32 Perl_my_htole32 -# endif -# if defined(PERL_NEED_MY_HTOLE64) -#define my_htole64 Perl_my_htole64 -# endif -# if defined(PERL_NEED_MY_HTOLEI) -#define my_htolei Perl_my_htolei -# endif -# if defined(PERL_NEED_MY_HTOLEL) -#define my_htolel Perl_my_htolel -# endif -# if defined(PERL_NEED_MY_HTOLES) -#define my_htoles Perl_my_htoles -# endif -# if defined(PERL_NEED_MY_LETOH16) -#define my_letoh16 Perl_my_letoh16 -# endif -# if defined(PERL_NEED_MY_LETOH32) -#define my_letoh32 Perl_my_letoh32 -# endif -# if defined(PERL_NEED_MY_LETOH64) -#define my_letoh64 Perl_my_letoh64 -# endif -# if defined(PERL_NEED_MY_LETOHI) -#define my_letohi Perl_my_letohi -# endif -# if defined(PERL_NEED_MY_LETOHL) -#define my_letohl Perl_my_letohl -# endif -# if defined(PERL_NEED_MY_LETOHS) -#define my_letohs Perl_my_letohs -# endif # if defined(PERL_USES_PL_PIDSTATUS) && defined(PERL_IN_UTIL_C) #define pidgone(a,b) S_pidgone(aTHX_ a,b) # endif diff --git a/packsizetables.c b/packsizetables.c index afeb210..8c57d41 100644 --- a/packsizetables.c +++ b/packsizetables.c @@ -88,38 +88,22 @@ STATIC const packprops_t packprops[512] = { 0, 0, /* L */ sizeof(unsigned long), 0, -#if defined(PERL_PACK_CAN_SHRIEKSIGN) /* N */ SIZE32, -#else - 0, -#endif 0, 0, 0, 0, /* S */ sizeof(unsigned short), 0, 0, -#if defined(PERL_PACK_CAN_SHRIEKSIGN) /* V */ SIZE32, -#else - 0, -#endif 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* i */ sizeof(int), 0, 0, /* l */ sizeof(long), 0, -#if defined(PERL_PACK_CAN_SHRIEKSIGN) /* n */ SIZE16, -#else - 0, -#endif 0, 0, 0, 0, /* s */ sizeof(short), 0, 0, -#if defined(PERL_PACK_CAN_SHRIEKSIGN) /* v */ SIZE16, -#else - 0, -#endif 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, @@ -213,19 +197,11 @@ STATIC const packprops_t packprops[512] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, /* l */ sizeof(long), 0, -#if defined(PERL_PACK_CAN_SHRIEKSIGN) /* n */ SIZE16, -#else - 0, -#endif 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* s */ sizeof(short), 0, 0, -#if defined(PERL_PACK_CAN_SHRIEKSIGN) /* v */ SIZE16, -#else - 0, -#endif 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, @@ -233,19 +209,11 @@ STATIC const packprops_t packprops[512] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, /* L */ sizeof(unsigned long), 0, -#if defined(PERL_PACK_CAN_SHRIEKSIGN) /* N */ SIZE32, -#else - 0, -#endif 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* S */ sizeof(unsigned short), 0, 0, -#if defined(PERL_PACK_CAN_SHRIEKSIGN) /* V */ SIZE32, -#else - 0, -#endif 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 }; diff --git a/perl.h b/perl.h index dc5e411..87e7f0f 100644 --- a/perl.h +++ b/perl.h @@ -743,15 +743,6 @@ EXTERN_C int syscall(int, ...); EXTERN_C int usleep(unsigned int); #endif -/* Funky places that do not have socket stuff. */ -#if defined(__LIBCATAMOUNT__) -# define MYSWAP -#endif - -#ifdef PERL_MICRO /* Last chance to export Perl_my_swap */ -# define MYSWAP -#endif - #ifdef PERL_CORE /* macros for correct constant construction */ @@ -804,189 +795,6 @@ EXTERN_C int usleep(unsigned int); (((U64)(x) & U64_CONST(0xff00000000000000)) >> 56) )) # endif -/*----------------------------------------------------------------------------*/ -# if BYTEORDER == 0x1234 || BYTEORDER == 0x12345678 /* little-endian */ -/*----------------------------------------------------------------------------*/ -# define my_htole16(x) (x) -# define my_letoh16(x) (x) -# define my_htole32(x) (x) -# define my_letoh32(x) (x) -# define my_htobe16(x) _swab_16_(x) -# define my_betoh16(x) _swab_16_(x) -# define my_htobe32(x) _swab_32_(x) -# define my_betoh32(x) _swab_32_(x) -# ifdef HAS_QUAD -# define my_htole64(x) (x) -# define my_letoh64(x) (x) -# define my_htobe64(x) _swab_64_(x) -# define my_betoh64(x) _swab_64_(x) -# endif -# define my_htoles(x) (x) -# define my_letohs(x) (x) -# define my_htolei(x) (x) -# define my_letohi(x) (x) -# define my_htolel(x) (x) -# define my_letohl(x) (x) -# if SHORTSIZE == 1 -# define my_htobes(x) (x) -# define my_betohs(x) (x) -# elif SHORTSIZE == 2 -# define my_htobes(x) _swab_16_(x) -# define my_betohs(x) _swab_16_(x) -# elif SHORTSIZE == 4 -# define my_htobes(x) _swab_32_(x) -# define my_betohs(x) _swab_32_(x) -# elif SHORTSIZE == 8 -# define my_htobes(x) _swab_64_(x) -# define my_betohs(x) _swab_64_(x) -# else -# define PERL_NEED_MY_HTOBES -# define PERL_NEED_MY_BETOHS -# endif -# if INTSIZE == 1 -# define my_htobei(x) (x) -# define my_betohi(x) (x) -# elif INTSIZE == 2 -# define my_htobei(x) _swab_16_(x) -# define my_betohi(x) _swab_16_(x) -# elif INTSIZE == 4 -# define my_htobei(x) _swab_32_(x) -# define my_betohi(x) _swab_32_(x) -# elif INTSIZE == 8 -# define my_htobei(x) _swab_64_(x) -# define my_betohi(x) _swab_64_(x) -# else -# define PERL_NEED_MY_HTOBEI -# define PERL_NEED_MY_BETOHI -# endif -# if LONGSIZE == 1 -# define my_htobel(x) (x) -# define my_betohl(x) (x) -# elif LONGSIZE == 2 -# define my_htobel(x) _swab_16_(x) -# define my_betohl(x) _swab_16_(x) -# elif LONGSIZE == 4 -# define my_htobel(x) _swab_32_(x) -# define my_betohl(x) _swab_32_(x) -# elif LONGSIZE == 8 -# define my_htobel(x) _swab_64_(x) -# define my_betohl(x) _swab_64_(x) -# else -# define PERL_NEED_MY_HTOBEL -# define PERL_NEED_MY_BETOHL -# endif -# define my_htolen(p,n) NOOP -# define my_letohn(p,n) NOOP -# define my_htoben(p,n) my_swabn(p,n) -# define my_betohn(p,n) my_swabn(p,n) -/*----------------------------------------------------------------------------*/ -# elif BYTEORDER == 0x4321 || BYTEORDER == 0x87654321 /* big-endian */ -/*----------------------------------------------------------------------------*/ -# define my_htobe16(x) (x) -# define my_betoh16(x) (x) -# define my_htobe32(x) (x) -# define my_betoh32(x) (x) -# define my_htole16(x) _swab_16_(x) -# define my_letoh16(x) _swab_16_(x) -# define my_htole32(x) _swab_32_(x) -# define my_letoh32(x) _swab_32_(x) -# ifdef HAS_QUAD -# define my_htobe64(x) (x) -# define my_betoh64(x) (x) -# define my_htole64(x) _swab_64_(x) -# define my_letoh64(x) _swab_64_(x) -# endif -# define my_htobes(x) (x) -# define my_betohs(x) (x) -# define my_htobei(x) (x) -# define my_betohi(x) (x) -# define my_htobel(x) (x) -# define my_betohl(x) (x) -# if SHORTSIZE == 1 -# define my_htoles(x) (x) -# define my_letohs(x) (x) -# elif SHORTSIZE == 2 -# define my_htoles(x) _swab_16_(x) -# define my_letohs(x) _swab_16_(x) -# elif SHORTSIZE == 4 -# define my_htoles(x) _swab_32_(x) -# define my_letohs(x) _swab_32_(x) -# elif SHORTSIZE == 8 -# define my_htoles(x) _swab_64_(x) -# define my_letohs(x) _swab_64_(x) -# else -# define PERL_NEED_MY_HTOLES -# define PERL_NEED_MY_LETOHS -# endif -# if INTSIZE == 1 -# define my_htolei(x) (x) -# define my_letohi(x) (x) -# elif INTSIZE == 2 -# define my_htolei(x) _swab_16_(x) -# define my_letohi(x) _swab_16_(x) -# elif INTSIZE == 4 -# define my_htolei(x) _swab_32_(x) -# define my_letohi(x) _swab_32_(x) -# elif INTSIZE == 8 -# define my_htolei(x) _swab_64_(x) -# define my_letohi(x) _swab_64_(x) -# else -# define PERL_NEED_MY_HTOLEI -# define PERL_NEED_MY_LETOHI -# endif -# if LONGSIZE == 1 -# define my_htolel(x) (x) -# define my_letohl(x) (x) -# elif LONGSIZE == 2 -# define my_htolel(x) _swab_16_(x) -# define my_letohl(x) _swab_16_(x) -# elif LONGSIZE == 4 -# define my_htolel(x) _swab_32_(x) -# define my_letohl(x) _swab_32_(x) -# elif LONGSIZE == 8 -# define my_htolel(x) _swab_64_(x) -# define my_letohl(x) _swab_64_(x) -# else -# define PERL_NEED_MY_HTOLEL -# define PERL_NEED_MY_LETOHL -# endif -# define my_htolen(p,n) my_swabn(p,n) -# define my_letohn(p,n) my_swabn(p,n) -# define my_htoben(p,n) NOOP -# define my_betohn(p,n) NOOP -/*----------------------------------------------------------------------------*/ -# else /* all other byte-orders */ -/*----------------------------------------------------------------------------*/ -# define PERL_NEED_MY_HTOLE16 -# define PERL_NEED_MY_LETOH16 -# define PERL_NEED_MY_HTOBE16 -# define PERL_NEED_MY_BETOH16 -# define PERL_NEED_MY_HTOLE32 -# define PERL_NEED_MY_LETOH32 -# define PERL_NEED_MY_HTOBE32 -# define PERL_NEED_MY_BETOH32 -# ifdef HAS_QUAD -# define PERL_NEED_MY_HTOLE64 -# define PERL_NEED_MY_LETOH64 -# define PERL_NEED_MY_HTOBE64 -# define PERL_NEED_MY_BETOH64 -# endif -# define PERL_NEED_MY_HTOLES -# define PERL_NEED_MY_LETOHS -# define PERL_NEED_MY_HTOBES -# define PERL_NEED_MY_BETOHS -# define PERL_NEED_MY_HTOLEI -# define PERL_NEED_MY_LETOHI -# define PERL_NEED_MY_HTOBEI -# define PERL_NEED_MY_BETOHI -# define PERL_NEED_MY_HTOLEL -# define PERL_NEED_MY_LETOHL -# define PERL_NEED_MY_HTOBEL -# define PERL_NEED_MY_BETOHL -/*----------------------------------------------------------------------------*/ -# endif /* end of byte-order macros */ -/*----------------------------------------------------------------------------*/ - /* The old value was hard coded at 1008. (4096-16) seems to be a bit faster, at least on FreeBSD. YMMV, so experiment. */ #ifndef PERL_ARENA_SIZE @@ -3537,36 +3345,70 @@ struct ptr_tbl { #define HAS_NTOHS #endif #ifndef HAS_HTONL -#if (BYTEORDER & 0xffff) != 0x4321 #define HAS_HTONS #define HAS_HTONL #define HAS_NTOHS #define HAS_NTOHL -#define MYSWAP -#define htons my_swap -#define htonl my_htonl -#define ntohs my_swap -#define ntohl my_ntohl -#endif -#else -#if (BYTEORDER & 0xffff) == 0x4321 -#undef HAS_HTONS -#undef HAS_HTONL -#undef HAS_NTOHS -#undef HAS_NTOHL -#endif +# if (BYTEORDER & 0xffff) == 0x4321 +/* Big endian system, so ntohl, ntohs, htonl and htons do not need to + re-order their values. However, to behave identically to the alternative + implementations, they should truncate to the correct size. */ +# define ntohl(x) ((x)&0xFFFFFFFF) +# define htonl(x) ntohl(x) +# define ntohs(x) ((x)&0xFFFF) +# define htons(x) ntohs(x) +# elif BYTEORDER == 0x1234 || BYTEORDER == 0x12345678 + +/* Note that we can't straight out declare our own htonl and htons because + the Win32 build process forcibly undefines HAS_HTONL etc for its miniperl, + to avoid the overhead of initialising the socket subsystem, but the headers + that *declare* the various functions are still seen. If we declare our own + htonl etc they will clash with the declarations in the Win32 headers. */ + +PERL_STATIC_INLINE U32 +my_swap32(const U32 x) { + return ((x & 0xFF) << 24) | ((x >> 24) & 0xFF) + | ((x & 0x0000FF00) << 8) | ((x & 0x00FF0000) >> 8); +} + +PERL_STATIC_INLINE U16 +my_swap16(const U16 x) { + return ((x & 0xFF) << 8) | ((x >> 8) & 0xFF); +} + +# define htonl(x) my_swap32(x) +# define ntohl(x) my_swap32(x) +# define ntohs(x) my_swap16(x) +# define htons(x) my_swap16(x) +# else +# error "Unsupported byteorder" +/* The C pre-processor doesn't let us return the value of BYTEORDER as part of + the error message. Please check the value of the macro BYTEORDER, as defined + in config.h. The values of BYTEORDER we expect are + + big endian little endian + 32 bit 0x4321 0x1234 + 64 bit 0x87654321 0x12345678 + + If you have a system with a different byte order, please see + pod/perlhack.pod for how to submit a patch to add supporting code. +*/ +# endif #endif /* * Little-endian byte order functions - 'v' for 'VAX', or 'reVerse'. * -DWS */ -#if BYTEORDER != 0x1234 -# define HAS_VTOHL -# define HAS_VTOHS -# define HAS_HTOVL -# define HAS_HTOVS -# if BYTEORDER == 0x4321 || BYTEORDER == 0x87654321 +#if BYTEORDER == 0x1234 || BYTEORDER == 0x12345678 +/* Little endian system, so vtohl, vtohs, htovl and htovs do not need to + re-order their values. However, to behave identically to the alternative + implementations, they should truncate to the correct size. */ +# define vtohl(x) ((x)&0xFFFFFFFF) +# define vtohs(x) ((x)&0xFFFF) +# define htovl(x) vtohl(x) +# define htovs(x) vtohs(x) +#elif BYTEORDER == 0x4321 || BYTEORDER == 0x87654321 # define vtohl(x) ((((x)&0xFF)<<24) \ +(((x)>>24)&0xFF) \ +(((x)&0x0000FF00)<<8) \ @@ -3574,14 +3416,11 @@ struct ptr_tbl { # define vtohs(x) ((((x)&0xFF)<<8) + (((x)>>8)&0xFF)) # define htovl(x) vtohl(x) # define htovs(x) vtohs(x) -# endif - /* otherwise default to functions in util.c */ -#ifndef htovs -short htovs(short n); -short vtohs(short n); -long htovl(long n); -long vtohl(long n); -#endif +#else +# error "Unsupported byteorder" +/* If you have need for current perl on PDP-11 or similar, and can help test + that blead keeps working on a mixed-endian system, then see + pod/perlhack.pod for how to submit patches to things working again. */ #endif /* *MAX Plus 1. A floating point value. diff --git a/pod/perldelta.pod b/pod/perldelta.pod index cb99f64..cb7ab03 100644 --- a/pod/perldelta.pod +++ b/pod/perldelta.pod @@ -310,9 +310,12 @@ L section. =over 4 -=item XXX-some-platform +=item Mixed-endian platforms -XXX +The code supporting C and C operations on mixed endian +platforms has been removed. We believe that Perl has long been unable to +build on mixed endian architectures (such as PDP-11s), so we don't think +that this change will affect any platforms which are able to build v5.18.0. =back @@ -328,7 +331,10 @@ well. =item * -XXX +The functions C, C and C have been removed. +It is unclear why these functions were ever marked as I, part of the +API. XS code can't call them directly, as it can't rely on them being +compiled. Unsurprisingly, no code on CPAN references them. =back diff --git a/pod/perldiag.pod b/pod/perldiag.pod index 2dc5514..ec4812c 100644 --- a/pod/perldiag.pod +++ b/pod/perldiag.pod @@ -5411,11 +5411,6 @@ problems when being input or output, which is likely where this message came from. If you really really know what you are doing you can turn off this warning by C. -=item Unknown BYTEORDER - -(F) There are no byte-swapping functions for a machine with this byte -order. - =item Unknown charname '%s' (F) The name you used inside C<\N{}> is unknown to Perl. Check the diff --git a/pp_pack.c b/pp_pack.c index b41ec78..3cfc03c 100644 --- a/pp_pack.c +++ b/pp_pack.c @@ -82,11 +82,6 @@ typedef union { } ld_bytes; #endif -#if PERL_VERSION >= 9 -# define PERL_PACK_CAN_BYTEORDER -# define PERL_PACK_CAN_SHRIEKSIGN -#endif - #ifndef CHAR_BIT # define CHAR_BIT 8 #endif @@ -134,46 +129,50 @@ typedef union { # define OFF32(p) ((char *) (p)) #endif -/* Only to be used inside a loop (see the break) */ -#define SHIFT16(utf8, s, strend, p, datumtype) STMT_START { \ - if (utf8) { \ - if (!uni_to_bytes(aTHX_ &(s), strend, OFF16(p), SIZE16, datumtype)) break; \ - } else { \ - Copy(s, OFF16(p), SIZE16, char); \ - (s) += SIZE16; \ - } \ -} STMT_END - -/* Only to be used inside a loop (see the break) */ -#define SHIFT32(utf8, s, strend, p, datumtype) STMT_START { \ - if (utf8) { \ - if (!uni_to_bytes(aTHX_ &(s), strend, OFF32(p), SIZE32, datumtype)) break; \ - } else { \ - Copy(s, OFF32(p), SIZE32, char); \ - (s) += SIZE32; \ - } \ -} STMT_END +#define PUSH16(utf8, cur, p, needs_swap) \ + PUSH_BYTES(utf8, cur, OFF16(p), SIZE16, needs_swap) +#define PUSH32(utf8, cur, p, needs_swap) \ + PUSH_BYTES(utf8, cur, OFF32(p), SIZE32, needs_swap) -#define PUSH16(utf8, cur, p) PUSH_BYTES(utf8, cur, OFF16(p), SIZE16) -#define PUSH32(utf8, cur, p) PUSH_BYTES(utf8, cur, OFF32(p), SIZE32) +#if BYTEORDER == 0x4321 || BYTEORDER == 0x87654321 /* big-endian */ +# define NEEDS_SWAP(d) (TYPE_ENDIANNESS(d) == TYPE_IS_LITTLE_ENDIAN) +#elif BYTEORDER == 0x1234 || BYTEORDER == 0x12345678 /* little-endian */ +# define NEEDS_SWAP(d) (TYPE_ENDIANNESS(d) == TYPE_IS_BIG_ENDIAN) +#else +# error "Unsupported byteorder" + /* Need to add code here to re-instate mixed endian support. + NEEDS_SWAP would need to hold a flag indicating which action to + take, and S_reverse_copy and the code in uni_to_bytes would need + logic adding to deal with any mixed-endian transformations needed. + */ +#endif /* Only to be used inside a loop (see the break) */ -#define SHIFT_BYTES(utf8, s, strend, buf, len, datumtype) \ +#define SHIFT_BYTES(utf8, s, strend, buf, len, datumtype, needs_swap) \ STMT_START { \ - if (utf8) { \ + if (UNLIKELY(utf8)) { \ if (!uni_to_bytes(aTHX_ &s, strend, \ (char *) (buf), len, datumtype)) break; \ } else { \ - Copy(s, (char *) (buf), len, char); \ + if (UNLIKELY(needs_swap)) \ + S_reverse_copy(s, (char *) (buf), len); \ + else \ + Copy(s, (char *) (buf), len, char); \ s += len; \ } \ } STMT_END -#define SHIFT_VAR(utf8, s, strend, var, datumtype) \ - SHIFT_BYTES(utf8, s, strend, &(var), sizeof(var), datumtype) +#define SHIFT16(utf8, s, strend, p, datumtype, needs_swap) \ + SHIFT_BYTES(utf8, s, strend, OFF16(p), SIZE16, datumtype, needs_swap) + +#define SHIFT32(utf8, s, strend, p, datumtype, needs_swap) \ + SHIFT_BYTES(utf8, s, strend, OFF32(p), SIZE32, datumtype, needs_swap) -#define PUSH_VAR(utf8, aptr, var) \ - PUSH_BYTES(utf8, aptr, &(var), sizeof(var)) +#define SHIFT_VAR(utf8, s, strend, var, datumtype, needs_swap) \ + SHIFT_BYTES(utf8, s, strend, &(var), sizeof(var), datumtype, needs_swap) + +#define PUSH_VAR(utf8, aptr, var, needs_swap) \ + PUSH_BYTES(utf8, aptr, &(var), sizeof(var), needs_swap) /* Avoid stack overflow due to pathological templates. 100 should be plenty. */ #define MAX_SUB_TEMPLATE_LEVEL 100 @@ -237,160 +236,25 @@ S_mul128(pTHX_ SV *sv, U8 m) #define TYPE_MODIFIERS(t) ((t) & ~0xFF) #define TYPE_NO_MODIFIERS(t) ((t) & 0xFF) -#ifdef PERL_PACK_CAN_SHRIEKSIGN -# define SHRIEKING_ALLOWED_TYPES "sSiIlLxXnNvV@." -#else -# define SHRIEKING_ALLOWED_TYPES "sSiIlLxX" -#endif - -#ifndef PERL_PACK_CAN_BYTEORDER -/* Put "can't" first because it is shorter */ -# define TYPE_ENDIANNESS(t) 0 -# define TYPE_NO_ENDIANNESS(t) (t) - -# define ENDIANNESS_ALLOWED_TYPES "" - -# define DO_BO_UNPACK(var, type) -# define DO_BO_PACK(var, type) -# define DO_BO_UNPACK_PTR(var, type, pre_cast, post_cast) -# define DO_BO_PACK_PTR(var, type, pre_cast, post_cast) -# define DO_BO_UNPACK_N(var, type) -# define DO_BO_PACK_N(var, type) -# define DO_BO_UNPACK_P(var) -# define DO_BO_PACK_P(var) -# define DO_BO_UNPACK_PC(var) -# define DO_BO_PACK_PC(var) - -#else /* PERL_PACK_CAN_BYTEORDER */ - # define TYPE_ENDIANNESS(t) ((t) & TYPE_ENDIANNESS_MASK) # define TYPE_NO_ENDIANNESS(t) ((t) & ~TYPE_ENDIANNESS_MASK) # define ENDIANNESS_ALLOWED_TYPES "sSiIlLqQjJfFdDpP(" -# define DO_BO_UNPACK(var, type) \ - STMT_START { \ - switch (TYPE_ENDIANNESS(datumtype)) { \ - case TYPE_IS_BIG_ENDIAN: var = my_betoh ## type (var); break; \ - case TYPE_IS_LITTLE_ENDIAN: var = my_letoh ## type (var); break; \ - default: break; \ - } \ - } STMT_END - -# define DO_BO_PACK(var, type) \ - STMT_START { \ - switch (TYPE_ENDIANNESS(datumtype)) { \ - case TYPE_IS_BIG_ENDIAN: var = my_htobe ## type (var); break; \ - case TYPE_IS_LITTLE_ENDIAN: var = my_htole ## type (var); break; \ - default: break; \ - } \ - } STMT_END - -# define DO_BO_UNPACK_PTR(var, type, pre_cast, post_cast) \ - STMT_START { \ - switch (TYPE_ENDIANNESS(datumtype)) { \ - case TYPE_IS_BIG_ENDIAN: \ - var = (post_cast*) my_betoh ## type ((pre_cast) var); \ - break; \ - case TYPE_IS_LITTLE_ENDIAN: \ - var = (post_cast *) my_letoh ## type ((pre_cast) var); \ - break; \ - default: \ - break; \ - } \ - } STMT_END - -# define DO_BO_PACK_PTR(var, type, pre_cast, post_cast) \ - STMT_START { \ - switch (TYPE_ENDIANNESS(datumtype)) { \ - case TYPE_IS_BIG_ENDIAN: \ - var = (post_cast *) my_htobe ## type ((pre_cast) var); \ - break; \ - case TYPE_IS_LITTLE_ENDIAN: \ - var = (post_cast *) my_htole ## type ((pre_cast) var); \ - break; \ - default: \ - break; \ - } \ - } STMT_END - -# define BO_CANT_DOIT(action, type) \ - STMT_START { \ - switch (TYPE_ENDIANNESS(datumtype)) { \ - case TYPE_IS_BIG_ENDIAN: \ - Perl_croak(aTHX_ "Can't %s big-endian %ss on this " \ - "platform", #action, #type); \ - break; \ - case TYPE_IS_LITTLE_ENDIAN: \ - Perl_croak(aTHX_ "Can't %s little-endian %ss on this " \ - "platform", #action, #type); \ - break; \ - default: \ - break; \ - } \ - } STMT_END - -# if PTRSIZE == INTSIZE -# define DO_BO_UNPACK_P(var) DO_BO_UNPACK_PTR(var, i, int, void) -# define DO_BO_PACK_P(var) DO_BO_PACK_PTR(var, i, int, void) -# define DO_BO_UNPACK_PC(var) DO_BO_UNPACK_PTR(var, i, int, char) -# define DO_BO_PACK_PC(var) DO_BO_PACK_PTR(var, i, int, char) -# elif PTRSIZE == LONGSIZE -# if LONGSIZE < IVSIZE && IVSIZE == 8 -# define DO_BO_UNPACK_P(var) DO_BO_UNPACK_PTR(var, 64, IV, void) -# define DO_BO_PACK_P(var) DO_BO_PACK_PTR(var, 64, IV, void) -# define DO_BO_UNPACK_PC(var) DO_BO_UNPACK_PTR(var, 64, IV, char) -# define DO_BO_PACK_PC(var) DO_BO_PACK_PTR(var, 64, IV, char) -# else -# define DO_BO_UNPACK_P(var) DO_BO_UNPACK_PTR(var, l, IV, void) -# define DO_BO_PACK_P(var) DO_BO_PACK_PTR(var, l, IV, void) -# define DO_BO_UNPACK_PC(var) DO_BO_UNPACK_PTR(var, l, IV, char) -# define DO_BO_PACK_PC(var) DO_BO_PACK_PTR(var, l, IV, char) -# endif -# elif PTRSIZE == IVSIZE -# define DO_BO_UNPACK_P(var) DO_BO_UNPACK_PTR(var, l, IV, void) -# define DO_BO_PACK_P(var) DO_BO_PACK_PTR(var, l, IV, void) -# define DO_BO_UNPACK_PC(var) DO_BO_UNPACK_PTR(var, l, IV, char) -# define DO_BO_PACK_PC(var) DO_BO_PACK_PTR(var, l, IV, char) -# else -# define DO_BO_UNPACK_P(var) BO_CANT_DOIT(unpack, pointer) -# define DO_BO_PACK_P(var) BO_CANT_DOIT(pack, pointer) -# define DO_BO_UNPACK_PC(var) BO_CANT_DOIT(unpack, pointer) -# define DO_BO_PACK_PC(var) BO_CANT_DOIT(pack, pointer) -# endif - -# if defined(my_htolen) && defined(my_letohn) && \ - defined(my_htoben) && defined(my_betohn) -# define DO_BO_UNPACK_N(var, type) \ - STMT_START { \ - switch (TYPE_ENDIANNESS(datumtype)) { \ - case TYPE_IS_BIG_ENDIAN: my_betohn(&var, sizeof(type)); break;\ - case TYPE_IS_LITTLE_ENDIAN: my_letohn(&var, sizeof(type)); break;\ - default: break; \ - } \ - } STMT_END - -# define DO_BO_PACK_N(var, type) \ - STMT_START { \ - switch (TYPE_ENDIANNESS(datumtype)) { \ - case TYPE_IS_BIG_ENDIAN: my_htoben(&var, sizeof(type)); break;\ - case TYPE_IS_LITTLE_ENDIAN: my_htolen(&var, sizeof(type)); break;\ - default: break; \ - } \ - } STMT_END -# else -# define DO_BO_UNPACK_N(var, type) BO_CANT_DOIT(unpack, type) -# define DO_BO_PACK_N(var, type) BO_CANT_DOIT(pack, type) -# endif - -#endif /* PERL_PACK_CAN_BYTEORDER */ - #define PACK_SIZE_CANNOT_CSUM 0x80 #define PACK_SIZE_UNPREDICTABLE 0x40 /* Not a fixed size element */ #define PACK_SIZE_MASK 0x3F #include "packsizetables.c" +static void +S_reverse_copy(const char *src, char *dest, STRLEN len) +{ + dest += len; + while (len--) + *--dest = *src++; +} + STATIC U8 uni_to_byte(pTHX_ const char **s, const char *end, I32 datumtype) { @@ -426,6 +290,11 @@ uni_to_bytes(pTHX_ const char **s, const char *end, const char *buf, int buf_len int bad = 0; const U32 flags = ckWARN(WARN_UTF8) ? UTF8_CHECK_ONLY : (UTF8_CHECK_ONLY | UTF8_ALLOW_ANY); + const bool needs_swap = NEEDS_SWAP(datumtype); + + if (UNLIKELY(needs_swap)) + buf += buf_len; + for (;buf_len > 0; buf_len--) { if (from >= end) return FALSE; val = utf8n_to_uvchr((U8 *) from, end-from, &retlen, flags); @@ -437,7 +306,10 @@ uni_to_bytes(pTHX_ const char **s, const char *end, const char *buf, int buf_len bad |= 2; val &= 0xff; } - *(U8 *)buf++ = (U8)val; + if (UNLIKELY(needs_swap)) + *(U8 *)--buf = (U8)val; + else + *(U8 *)buf++ = (U8)val; } /* We have enough characters for the buffer. Did we have problems ? */ if (bad) { @@ -479,30 +351,45 @@ next_uni_uu(pTHX_ const char **s, const char *end, I32 *out) } STATIC char * -S_bytes_to_uni(const U8 *start, STRLEN len, char *dest) { - const U8 * const end = start + len; - +S_bytes_to_uni(const U8 *start, STRLEN len, char *dest, const bool needs_swap) { PERL_ARGS_ASSERT_BYTES_TO_UNI; - while (start < end) { - const UV uv = NATIVE_TO_ASCII(*start); - if (UNI_IS_INVARIANT(uv)) - *dest++ = (char)(U8)UTF_TO_NATIVE(uv); - else { - *dest++ = (char)(U8)UTF8_EIGHT_BIT_HI(uv); - *dest++ = (char)(U8)UTF8_EIGHT_BIT_LO(uv); - } - start++; + if (UNLIKELY(needs_swap)) { + const U8 *p = start + len; + while (p-- > start) { + const UV uv = NATIVE_TO_ASCII(*p); + if (UNI_IS_INVARIANT(uv)) + *dest++ = (char)(U8)UTF_TO_NATIVE(uv); + else { + *dest++ = (char)(U8)UTF8_EIGHT_BIT_HI(uv); + *dest++ = (char)(U8)UTF8_EIGHT_BIT_LO(uv); + } + } + } else { + const U8 * const end = start + len; + while (start < end) { + const UV uv = NATIVE_TO_ASCII(*start); + if (UNI_IS_INVARIANT(uv)) + *dest++ = (char)(U8)UTF_TO_NATIVE(uv); + else { + *dest++ = (char)(U8)UTF8_EIGHT_BIT_HI(uv); + *dest++ = (char)(U8)UTF8_EIGHT_BIT_LO(uv); + } + start++; + } } return dest; } -#define PUSH_BYTES(utf8, cur, buf, len) \ +#define PUSH_BYTES(utf8, cur, buf, len, needs_swap) \ STMT_START { \ - if (utf8) \ - (cur) = bytes_to_uni((U8 *) buf, len, (cur)); \ + if (UNLIKELY(utf8)) \ + (cur) = S_bytes_to_uni((U8 *) buf, len, (cur), needs_swap); \ else { \ - Copy(buf, cur, len, char); \ + if (UNLIKELY(needs_swap)) \ + S_reverse_copy((char *)(buf), cur, len); \ + else \ + Copy(buf, cur, len, char); \ (cur) += (len); \ } \ } STMT_END @@ -528,14 +415,14 @@ STMT_START { \ (start) = sv_exp_grow(cat, gl); \ (cur) = (start) + SvCUR(cat); \ } \ - PUSH_BYTES(utf8, cur, buf, glen); \ + PUSH_BYTES(utf8, cur, buf, glen, 0); \ } STMT_END #define PUSH_BYTE(utf8, s, byte) \ STMT_START { \ if (utf8) { \ const U8 au8 = (byte); \ - (s) = bytes_to_uni(&au8, 1, (s)); \ + (s) = S_bytes_to_uni(&au8, 1, (s), 0); \ } else *(U8 *)(s)++ = (byte); \ } STMT_END @@ -589,10 +476,8 @@ S_measure_struct(pTHX_ tempsym_t* symptr) Perl_croak(aTHX_ "Invalid type '%c' in %s", (int)TYPE_NO_MODIFIERS(symptr->code), _action( symptr ) ); -#ifdef PERL_PACK_CAN_SHRIEKSIGN case '.' | TYPE_IS_SHRIEKING: case '@' | TYPE_IS_SHRIEKING: -#endif case '@': case '.': case '/': @@ -779,9 +664,8 @@ S_next_symbol(pTHX_ tempsym_t* symptr ) switch (*patptr) { case '!': modifier = TYPE_IS_SHRIEKING; - allowed = SHRIEKING_ALLOWED_TYPES; + allowed = "sSiIlLxXnNvV@."; break; -#ifdef PERL_PACK_CAN_BYTEORDER case '>': modifier = TYPE_IS_BIG_ENDIAN; allowed = ENDIANNESS_ALLOWED_TYPES; @@ -790,7 +674,6 @@ S_next_symbol(pTHX_ tempsym_t* symptr ) modifier = TYPE_IS_LITTLE_ENDIAN; allowed = ENDIANNESS_ALLOWED_TYPES; break; -#endif /* PERL_PACK_CAN_BYTEORDER */ default: allowed = ""; modifier = 0; @@ -1009,6 +892,7 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c packprops_t props; I32 len; I32 datumtype = symptr->code; + bool needs_swap; /* do first one only unless in list context / is implemented by unpacking the count, then popping it from the stack, so must check that we're not in the middle of a / */ @@ -1046,6 +930,8 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c } } + needs_swap = NEEDS_SWAP(datumtype); + switch(TYPE_NO_ENDIANNESS(datumtype)) { default: Perl_croak(aTHX_ "Invalid type '%c' in unpack", (int)TYPE_NO_MODIFIERS(datumtype) ); @@ -1081,17 +967,11 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c *symptr = savsym; break; } -#ifdef PERL_PACK_CAN_SHRIEKSIGN case '.' | TYPE_IS_SHRIEKING: -#endif case '.': { const char *from; SV *sv; -#ifdef PERL_PACK_CAN_SHRIEKSIGN const bool u8 = utf8 && !(datumtype & TYPE_IS_SHRIEKING); -#else /* PERL_PACK_CAN_SHRIEKSIGN */ - const bool u8 = utf8; -#endif if (howlen == e_star) from = strbeg; else if (len <= 0) from = s; else { @@ -1106,16 +986,10 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c mXPUSHs(sv); break; } -#ifdef PERL_PACK_CAN_SHRIEKSIGN case '@' | TYPE_IS_SHRIEKING: -#endif case '@': s = strbeg + symptr->strbeg; -#ifdef PERL_PACK_CAN_SHRIEKSIGN if (utf8 && !(datumtype & TYPE_IS_SHRIEKING)) -#else /* PERL_PACK_CAN_SHRIEKSIGN */ - if (utf8) -#endif { while (len > 0) { if (s >= strend) @@ -1474,8 +1348,7 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c #if SHORTSIZE != SIZE16 while (len-- > 0) { short ashort; - SHIFT_VAR(utf8, s, strend, ashort, datumtype); - DO_BO_UNPACK(ashort, s); + SHIFT_VAR(utf8, s, strend, ashort, datumtype, needs_swap); if (!checksum) mPUSHi(ashort); else if (checksum > bits_in_uv) @@ -1494,8 +1367,7 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c #if U16SIZE > SIZE16 ai16 = 0; #endif - SHIFT16(utf8, s, strend, &ai16, datumtype); - DO_BO_UNPACK(ai16, 16); + SHIFT16(utf8, s, strend, &ai16, datumtype, needs_swap); #if U16SIZE > SIZE16 if (ai16 > 32767) ai16 -= 65536; @@ -1512,8 +1384,8 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c #if SHORTSIZE != SIZE16 while (len-- > 0) { unsigned short aushort; - SHIFT_VAR(utf8, s, strend, aushort, datumtype); - DO_BO_UNPACK(aushort, s); + SHIFT_VAR(utf8, s, strend, aushort, datumtype, needs_swap, + needs_swap); if (!checksum) mPUSHu(aushort); else if (checksum > bits_in_uv) @@ -1533,16 +1405,11 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c #if U16SIZE > SIZE16 au16 = 0; #endif - SHIFT16(utf8, s, strend, &au16, datumtype); - DO_BO_UNPACK(au16, 16); -#ifdef HAS_NTOHS + SHIFT16(utf8, s, strend, &au16, datumtype, needs_swap); if (datumtype == 'n') au16 = PerlSock_ntohs(au16); -#endif -#ifdef HAS_VTOHS if (datumtype == 'v') au16 = vtohs(au16); -#endif if (!checksum) mPUSHu(au16); else if (checksum > bits_in_uv) @@ -1551,7 +1418,6 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c cuv += au16; } break; -#ifdef PERL_PACK_CAN_SHRIEKSIGN case 'v' | TYPE_IS_SHRIEKING: case 'n' | TYPE_IS_SHRIEKING: while (len-- > 0) { @@ -1559,15 +1425,13 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c # if U16SIZE > SIZE16 ai16 = 0; # endif - SHIFT16(utf8, s, strend, &ai16, datumtype); -# ifdef HAS_NTOHS + SHIFT16(utf8, s, strend, &ai16, datumtype, needs_swap); + /* There should never be any byte-swapping here. */ + assert(!TYPE_ENDIANNESS(datumtype)); if (datumtype == ('n' | TYPE_IS_SHRIEKING)) ai16 = (I16) PerlSock_ntohs((U16) ai16); -# endif /* HAS_NTOHS */ -# ifdef HAS_VTOHS if (datumtype == ('v' | TYPE_IS_SHRIEKING)) ai16 = (I16) vtohs((U16) ai16); -# endif /* HAS_VTOHS */ if (!checksum) mPUSHi(ai16); else if (checksum > bits_in_uv) @@ -1576,13 +1440,11 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c cuv += ai16; } break; -#endif /* PERL_PACK_CAN_SHRIEKSIGN */ case 'i': case 'i' | TYPE_IS_SHRIEKING: while (len-- > 0) { int aint; - SHIFT_VAR(utf8, s, strend, aint, datumtype); - DO_BO_UNPACK(aint, i); + SHIFT_VAR(utf8, s, strend, aint, datumtype, needs_swap); if (!checksum) mPUSHi(aint); else if (checksum > bits_in_uv) @@ -1595,8 +1457,7 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c case 'I' | TYPE_IS_SHRIEKING: while (len-- > 0) { unsigned int auint; - SHIFT_VAR(utf8, s, strend, auint, datumtype); - DO_BO_UNPACK(auint, i); + SHIFT_VAR(utf8, s, strend, auint, datumtype, needs_swap); if (!checksum) mPUSHu(auint); else if (checksum > bits_in_uv) @@ -1608,16 +1469,7 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c case 'j': while (len-- > 0) { IV aiv; - SHIFT_VAR(utf8, s, strend, aiv, datumtype); -#if IVSIZE == INTSIZE - DO_BO_UNPACK(aiv, i); -#elif IVSIZE == LONGSIZE - DO_BO_UNPACK(aiv, l); -#elif defined(HAS_QUAD) && IVSIZE == U64SIZE - DO_BO_UNPACK(aiv, 64); -#else - Perl_croak(aTHX_ "'j' not supported on this platform"); -#endif + SHIFT_VAR(utf8, s, strend, aiv, datumtype, needs_swap); if (!checksum) mPUSHi(aiv); else if (checksum > bits_in_uv) @@ -1629,16 +1481,7 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c case 'J': while (len-- > 0) { UV auv; - SHIFT_VAR(utf8, s, strend, auv, datumtype); -#if IVSIZE == INTSIZE - DO_BO_UNPACK(auv, i); -#elif IVSIZE == LONGSIZE - DO_BO_UNPACK(auv, l); -#elif defined(HAS_QUAD) && IVSIZE == U64SIZE - DO_BO_UNPACK(auv, 64); -#else - Perl_croak(aTHX_ "'J' not supported on this platform"); -#endif + SHIFT_VAR(utf8, s, strend, auv, datumtype, needs_swap); if (!checksum) mPUSHu(auv); else if (checksum > bits_in_uv) @@ -1651,8 +1494,7 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c #if LONGSIZE != SIZE32 while (len-- > 0) { long along; - SHIFT_VAR(utf8, s, strend, along, datumtype); - DO_BO_UNPACK(along, l); + SHIFT_VAR(utf8, s, strend, along, datumtype, needs_swap); if (!checksum) mPUSHi(along); else if (checksum > bits_in_uv) @@ -1670,8 +1512,7 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c #if U32SIZE > SIZE32 ai32 = 0; #endif - SHIFT32(utf8, s, strend, &ai32, datumtype); - DO_BO_UNPACK(ai32, 32); + SHIFT32(utf8, s, strend, &ai32, datumtype, needs_swap); #if U32SIZE > SIZE32 if (ai32 > 2147483647) ai32 -= 4294967296; #endif @@ -1687,8 +1528,7 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c #if LONGSIZE != SIZE32 while (len-- > 0) { unsigned long aulong; - SHIFT_VAR(utf8, s, strend, aulong, datumtype); - DO_BO_UNPACK(aulong, l); + SHIFT_VAR(utf8, s, strend, aulong, datumtype, needs_swap); if (!checksum) mPUSHu(aulong); else if (checksum > bits_in_uv) @@ -1708,16 +1548,11 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c #if U32SIZE > SIZE32 au32 = 0; #endif - SHIFT32(utf8, s, strend, &au32, datumtype); - DO_BO_UNPACK(au32, 32); -#ifdef HAS_NTOHL + SHIFT32(utf8, s, strend, &au32, datumtype, needs_swap); if (datumtype == 'N') au32 = PerlSock_ntohl(au32); -#endif -#ifdef HAS_VTOHL if (datumtype == 'V') au32 = vtohl(au32); -#endif if (!checksum) mPUSHu(au32); else if (checksum > bits_in_uv) @@ -1726,23 +1561,20 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c cuv += au32; } break; -#ifdef PERL_PACK_CAN_SHRIEKSIGN case 'V' | TYPE_IS_SHRIEKING: case 'N' | TYPE_IS_SHRIEKING: while (len-- > 0) { I32 ai32; -# if U32SIZE > SIZE32 +#if U32SIZE > SIZE32 ai32 = 0; -# endif - SHIFT32(utf8, s, strend, &ai32, datumtype); -# ifdef HAS_NTOHL +#endif + SHIFT32(utf8, s, strend, &ai32, datumtype, needs_swap); + /* There should never be any byte swapping here. */ + assert(!TYPE_ENDIANNESS(datumtype)); if (datumtype == ('N' | TYPE_IS_SHRIEKING)) ai32 = (I32)PerlSock_ntohl((U32)ai32); -# endif -# ifdef HAS_VTOHL if (datumtype == ('V' | TYPE_IS_SHRIEKING)) ai32 = (I32)vtohl((U32)ai32); -# endif if (!checksum) mPUSHi(ai32); else if (checksum > bits_in_uv) @@ -1751,12 +1583,10 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c cuv += ai32; } break; -#endif /* PERL_PACK_CAN_SHRIEKSIGN */ case 'p': while (len-- > 0) { const char *aptr; - SHIFT_VAR(utf8, s, strend, aptr, datumtype); - DO_BO_UNPACK_PC(aptr); + SHIFT_VAR(utf8, s, strend, aptr, datumtype, needs_swap); /* newSVpv generates undef if aptr is NULL */ mPUSHs(newSVpv(aptr, 0)); } @@ -1809,8 +1639,7 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c EXTEND(SP, 1); if (s + sizeof(char*) <= strend) { char *aptr; - SHIFT_VAR(utf8, s, strend, aptr, datumtype); - DO_BO_UNPACK_PC(aptr); + SHIFT_VAR(utf8, s, strend, aptr, datumtype, needs_swap); /* newSVpvn generates undef if aptr is NULL */ PUSHs(newSVpvn_flags(aptr, len, SVs_TEMP)); } @@ -1819,8 +1648,7 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c case 'q': while (len-- > 0) { Quad_t aquad; - SHIFT_VAR(utf8, s, strend, aquad, datumtype); - DO_BO_UNPACK(aquad, 64); + SHIFT_VAR(utf8, s, strend, aquad, datumtype, needs_swap); if (!checksum) mPUSHs(aquad >= IV_MIN && aquad <= IV_MAX ? newSViv((IV)aquad) : newSVnv((NV)aquad)); @@ -1833,8 +1661,7 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c case 'Q': while (len-- > 0) { Uquad_t auquad; - SHIFT_VAR(utf8, s, strend, auquad, datumtype); - DO_BO_UNPACK(auquad, 64); + SHIFT_VAR(utf8, s, strend, auquad, datumtype, needs_swap); if (!checksum) mPUSHs(auquad <= UV_MAX ? newSVuv((UV)auquad) : newSVnv((NV)auquad)); @@ -1849,8 +1676,7 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c case 'f': while (len-- > 0) { float afloat; - SHIFT_VAR(utf8, s, strend, afloat, datumtype); - DO_BO_UNPACK_N(afloat, float); + SHIFT_VAR(utf8, s, strend, afloat, datumtype, needs_swap); if (!checksum) mPUSHn(afloat); else @@ -1860,8 +1686,7 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c case 'd': while (len-- > 0) { double adouble; - SHIFT_VAR(utf8, s, strend, adouble, datumtype); - DO_BO_UNPACK_N(adouble, double); + SHIFT_VAR(utf8, s, strend, adouble, datumtype, needs_swap); if (!checksum) mPUSHn(adouble); else @@ -1871,8 +1696,8 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c case 'F': while (len-- > 0) { NV_bytes anv; - SHIFT_BYTES(utf8, s, strend, anv.bytes, sizeof(anv.bytes), datumtype); - DO_BO_UNPACK_N(anv.nv, NV); + SHIFT_BYTES(utf8, s, strend, anv.bytes, sizeof(anv.bytes), + datumtype, needs_swap); if (!checksum) mPUSHn(anv.nv); else @@ -1883,8 +1708,8 @@ S_unpack_rec(pTHX_ tempsym_t* symptr, const char *s, const char *strbeg, const c case 'D': while (len-- > 0) { ld_bytes aldouble; - SHIFT_BYTES(utf8, s, strend, aldouble.bytes, sizeof(aldouble.bytes), datumtype); - DO_BO_UNPACK_N(aldouble.ld, long double); + SHIFT_BYTES(utf8, s, strend, aldouble.bytes, + sizeof(aldouble.bytes), datumtype, needs_swap); if (!checksum) mPUSHn(aldouble.ld); else @@ -2288,6 +2113,7 @@ S_pack_rec(pTHX_ SV *cat, tempsym_t* symptr, SV **beglist, SV **endlist ) howlen_t howlen = symptr->howlen; char *start = SvPVX(cat); char *cur = start + SvCUR(cat); + bool needs_swap; #define NEXTFROM (lengthcode ? lengthcode : items-- > 0 ? *beglist++ : &PL_sv_no) @@ -2337,6 +2163,8 @@ S_pack_rec(pTHX_ SV *cat, tempsym_t* symptr, SV **beglist, SV **endlist ) lengthcode = sv_2mortal(newSViv(count)); } + needs_swap = NEEDS_SWAP(datumtype); + /* Code inside the switch must take care to properly update cat (CUR length and '\0' termination) if it updated *cur and doesn't simply leave using break */ @@ -2348,9 +2176,7 @@ S_pack_rec(pTHX_ SV *cat, tempsym_t* symptr, SV **beglist, SV **endlist ) Perl_croak(aTHX_ "'%%' may not be used in pack"); { char *from; -#ifdef PERL_PACK_CAN_SHRIEKSIGN case '.' | TYPE_IS_SHRIEKING: -#endif case '.': if (howlen == e_star) from = start; else if (len == 0) from = cur; @@ -2363,17 +2189,11 @@ S_pack_rec(pTHX_ SV *cat, tempsym_t* symptr, SV **beglist, SV **endlist ) fromstr = NEXTFROM; len = SvIV(fromstr); goto resize; -#ifdef PERL_PACK_CAN_SHRIEKSIGN case '@' | TYPE_IS_SHRIEKING: -#endif case '@': from = start + symptr->strbeg; resize: -#ifdef PERL_PACK_CAN_SHRIEKSIGN if (utf8 && !(datumtype & TYPE_IS_SHRIEKING)) -#else /* PERL_PACK_CAN_SHRIEKSIGN */ - if (utf8) -#endif if (len >= 0) { while (len && from < cur) { from += UTF8SKIP(from); @@ -2841,7 +2661,7 @@ S_pack_rec(pTHX_ SV *cat, tempsym_t* symptr, SV **beglist, SV **endlist ) len+(endb-buffer)*UTF8_EXPAND); end = start+SvLEN(cat); } - cur = bytes_to_uni(buffer, endb-buffer, cur); + cur = S_bytes_to_uni(buffer, endb-buffer, cur, 0); } else { if (cur >= end) { *cur = '\0'; @@ -2875,8 +2695,7 @@ S_pack_rec(pTHX_ SV *cat, tempsym_t* symptr, SV **beglist, SV **endlist ) # else afloat = (float)anv; # endif - DO_BO_PACK_N(afloat, float); - PUSH_VAR(utf8, cur, afloat); + PUSH_VAR(utf8, cur, afloat, needs_swap); } break; case 'd': @@ -2897,8 +2716,7 @@ S_pack_rec(pTHX_ SV *cat, tempsym_t* symptr, SV **beglist, SV **endlist ) # else adouble = (double)anv; # endif - DO_BO_PACK_N(adouble, double); - PUSH_VAR(utf8, cur, adouble); + PUSH_VAR(utf8, cur, adouble, needs_swap); } break; case 'F': { @@ -2912,8 +2730,7 @@ S_pack_rec(pTHX_ SV *cat, tempsym_t* symptr, SV **beglist, SV **endlist ) #else anv.nv = SvNV(fromstr); #endif - DO_BO_PACK_N(anv, NV); - PUSH_BYTES(utf8, cur, anv.bytes, sizeof(anv.bytes)); + PUSH_BYTES(utf8, cur, anv.bytes, sizeof(anv.bytes), needs_swap); } break; } @@ -2930,38 +2747,30 @@ S_pack_rec(pTHX_ SV *cat, tempsym_t* symptr, SV **beglist, SV **endlist ) # else aldouble.ld = (long double)SvNV(fromstr); # endif - DO_BO_PACK_N(aldouble, long double); - PUSH_BYTES(utf8, cur, aldouble.bytes, sizeof(aldouble.bytes)); + PUSH_BYTES(utf8, cur, aldouble.bytes, sizeof(aldouble.bytes), + needs_swap); } break; } #endif -#ifdef PERL_PACK_CAN_SHRIEKSIGN case 'n' | TYPE_IS_SHRIEKING: -#endif case 'n': while (len-- > 0) { I16 ai16; fromstr = NEXTFROM; ai16 = (I16)SvIV(fromstr); -#ifdef HAS_HTONS ai16 = PerlSock_htons(ai16); -#endif - PUSH16(utf8, cur, &ai16); + PUSH16(utf8, cur, &ai16, FALSE); } break; -#ifdef PERL_PACK_CAN_SHRIEKSIGN case 'v' | TYPE_IS_SHRIEKING: -#endif case 'v': while (len-- > 0) { I16 ai16; fromstr = NEXTFROM; ai16 = (I16)SvIV(fromstr); -#ifdef HAS_HTOVS ai16 = htovs(ai16); -#endif - PUSH16(utf8, cur, &ai16); + PUSH16(utf8, cur, &ai16, FALSE); } break; case 'S' | TYPE_IS_SHRIEKING: @@ -2970,8 +2779,7 @@ S_pack_rec(pTHX_ SV *cat, tempsym_t* symptr, SV **beglist, SV **endlist ) unsigned short aushort; fromstr = NEXTFROM; aushort = SvUV(fromstr); - DO_BO_PACK(aushort, s); - PUSH_VAR(utf8, cur, aushort); + PUSH_VAR(utf8, cur, aushort, needs_swap); } break; #else @@ -2982,8 +2790,7 @@ S_pack_rec(pTHX_ SV *cat, tempsym_t* symptr, SV **beglist, SV **endlist ) U16 au16; fromstr = NEXTFROM; au16 = (U16)SvUV(fromstr); - DO_BO_PACK(au16, 16); - PUSH16(utf8, cur, &au16); + PUSH16(utf8, cur, &au16, needs_swap); } break; case 's' | TYPE_IS_SHRIEKING: @@ -2992,8 +2799,7 @@ S_pack_rec(pTHX_ SV *cat, tempsym_t* symptr, SV **beglist, SV **endlist ) short ashort; fromstr = NEXTFROM; ashort = SvIV(fromstr); - DO_BO_PACK(ashort, s); - PUSH_VAR(utf8, cur, ashort); + PUSH_VAR(utf8, cur, ashort, needs_swap); } break; #else @@ -3004,8 +2810,7 @@ S_pack_rec(pTHX_ SV *cat, tempsym_t* symptr, SV **beglist, SV **endlist ) I16 ai16; fromstr = NEXTFROM; ai16 = (I16)SvIV(fromstr); - DO_BO_PACK(ai16, 16); - PUSH16(utf8, cur, &ai16); + PUSH16(utf8, cur, &ai16, needs_swap); } break; case 'I': @@ -3014,8 +2819,7 @@ S_pack_rec(pTHX_ SV *cat, tempsym_t* symptr, SV **beglist, SV **endlist ) unsigned int auint; fromstr = NEXTFROM; auint = SvUV(fromstr); - DO_BO_PACK(auint, i); - PUSH_VAR(utf8, cur, auint); + PUSH_VAR(utf8, cur, auint, needs_swap); } break; case 'j': @@ -3023,16 +2827,7 @@ S_pack_rec(pTHX_ SV *cat, tempsym_t* symptr, SV **beglist, SV **endlist ) IV aiv; fromstr = NEXTFROM; aiv = SvIV(fromstr); -#if IVSIZE == INTSIZE - DO_BO_PACK(aiv, i); -#elif IVSIZE == LONGSIZE - DO_BO_PACK(aiv, l); -#elif defined(HAS_QUAD) && IVSIZE == U64SIZE - DO_BO_PACK(aiv, 64); -#else - Perl_croak(aTHX_ "'j' not supported on this platform"); -#endif - PUSH_VAR(utf8, cur, aiv); + PUSH_VAR(utf8, cur, aiv, needs_swap); } break; case 'J': @@ -3040,16 +2835,7 @@ S_pack_rec(pTHX_ SV *cat, tempsym_t* symptr, SV **beglist, SV **endlist ) UV auv; fromstr = NEXTFROM; auv = SvUV(fromstr); -#if UVSIZE == INTSIZE - DO_BO_PACK(auv, i); -#elif UVSIZE == LONGSIZE - DO_BO_PACK(auv, l); -#elif defined(HAS_QUAD) && UVSIZE == U64SIZE - DO_BO_PACK(auv, 64); -#else - Perl_croak(aTHX_ "'J' not supported on this platform"); -#endif - PUSH_VAR(utf8, cur, auv); + PUSH_VAR(utf8, cur, auv, needs_swap); } break; case 'w': @@ -3145,36 +2931,27 @@ S_pack_rec(pTHX_ SV *cat, tempsym_t* symptr, SV **beglist, SV **endlist ) int aint; fromstr = NEXTFROM; aint = SvIV(fromstr); - DO_BO_PACK(aint, i); - PUSH_VAR(utf8, cur, aint); + PUSH_VAR(utf8, cur, aint, needs_swap); } break; -#ifdef PERL_PACK_CAN_SHRIEKSIGN case 'N' | TYPE_IS_SHRIEKING: -#endif case 'N': while (len-- > 0) { U32 au32; fromstr = NEXTFROM; au32 = SvUV(fromstr); -#ifdef HAS_HTONL au32 = PerlSock_htonl(au32); -#endif - PUSH32(utf8, cur, &au32); + PUSH32(utf8, cur, &au32, FALSE); } break; -#ifdef PERL_PACK_CAN_SHRIEKSIGN case 'V' | TYPE_IS_SHRIEKING: -#endif case 'V': while (len-- > 0) { U32 au32; fromstr = NEXTFROM; au32 = SvUV(fromstr); -#ifdef HAS_HTOVL au32 = htovl(au32); -#endif - PUSH32(utf8, cur, &au32); + PUSH32(utf8, cur, &au32, FALSE); } break; case 'L' | TYPE_IS_SHRIEKING: @@ -3183,8 +2960,7 @@ S_pack_rec(pTHX_ SV *cat, tempsym_t* symptr, SV **beglist, SV **endlist ) unsigned long aulong; fromstr = NEXTFROM; aulong = SvUV(fromstr); - DO_BO_PACK(aulong, l); - PUSH_VAR(utf8, cur, aulong); + PUSH_VAR(utf8, cur, aulong, needs_swap); } break; #else @@ -3195,8 +2971,7 @@ S_pack_rec(pTHX_ SV *cat, tempsym_t* symptr, SV **beglist, SV **endlist ) U32 au32; fromstr = NEXTFROM; au32 = SvUV(fromstr); - DO_BO_PACK(au32, 32); - PUSH32(utf8, cur, &au32); + PUSH32(utf8, cur, &au32, needs_swap); } break; case 'l' | TYPE_IS_SHRIEKING: @@ -3205,8 +2980,7 @@ S_pack_rec(pTHX_ SV *cat, tempsym_t* symptr, SV **beglist, SV **endlist ) long along; fromstr = NEXTFROM; along = SvIV(fromstr); - DO_BO_PACK(along, l); - PUSH_VAR(utf8, cur, along); + PUSH_VAR(utf8, cur, along, needs_swap); } break; #else @@ -3217,8 +2991,7 @@ S_pack_rec(pTHX_ SV *cat, tempsym_t* symptr, SV **beglist, SV **endlist ) I32 ai32; fromstr = NEXTFROM; ai32 = SvIV(fromstr); - DO_BO_PACK(ai32, 32); - PUSH32(utf8, cur, &ai32); + PUSH32(utf8, cur, &ai32, needs_swap); } break; #ifdef HAS_QUAD @@ -3227,8 +3000,7 @@ S_pack_rec(pTHX_ SV *cat, tempsym_t* symptr, SV **beglist, SV **endlist ) Uquad_t auquad; fromstr = NEXTFROM; auquad = (Uquad_t) SvUV(fromstr); - DO_BO_PACK(auquad, 64); - PUSH_VAR(utf8, cur, auquad); + PUSH_VAR(utf8, cur, auquad, needs_swap); } break; case 'q': @@ -3236,8 +3008,7 @@ S_pack_rec(pTHX_ SV *cat, tempsym_t* symptr, SV **beglist, SV **endlist ) Quad_t aquad; fromstr = NEXTFROM; aquad = (Quad_t)SvIV(fromstr); - DO_BO_PACK(aquad, 64); - PUSH_VAR(utf8, cur, aquad); + PUSH_VAR(utf8, cur, aquad, needs_swap); } break; #endif /* HAS_QUAD */ @@ -3268,8 +3039,7 @@ S_pack_rec(pTHX_ SV *cat, tempsym_t* symptr, SV **beglist, SV **endlist ) else aptr = SvPV_force_flags_nolen(fromstr, 0); } - DO_BO_PACK_PC(aptr); - PUSH_VAR(utf8, cur, aptr); + PUSH_VAR(utf8, cur, aptr, needs_swap); } break; case 'u': { @@ -3315,7 +3085,7 @@ S_pack_rec(pTHX_ SV *cat, tempsym_t* symptr, SV **beglist, SV **endlist ) end = doencodes(hunk, aptr, todo); aptr += todo; } - PUSH_BYTES(utf8, cur, hunk, end-hunk); + PUSH_BYTES(utf8, cur, hunk, end-hunk, 0); fromlen -= todo; } break; diff --git a/pp_sys.c b/pp_sys.c index 04ccddb..9458d2e 100644 --- a/pp_sys.c +++ b/pp_sys.c @@ -4942,9 +4942,7 @@ PP(pp_gservent) #ifdef HAS_GETSERVBYPORT const char * const proto = POPpbytex; unsigned short port = (unsigned short)POPu; -#ifdef HAS_HTONS port = PerlSock_htons(port); -#endif sent = PerlSock_getservbyport(port, (proto && !*proto) ? NULL : proto); #else DIE(aTHX_ PL_no_sock_func, "getservbyport"); @@ -4962,11 +4960,7 @@ PP(pp_gservent) PUSHs(sv = sv_newmortal()); if (sent) { if (which == OP_GSBYNAME) { -#ifdef HAS_NTOHS sv_setiv(sv, (IV)PerlSock_ntohs(sent->s_port)); -#else - sv_setiv(sv, (IV)(sent->s_port)); -#endif } else sv_setpv(sv, sent->s_name); @@ -4977,11 +4971,7 @@ PP(pp_gservent) if (sent) { mPUSHs(newSVpv(sent->s_name, 0)); PUSHs(space_join_names_mortal(sent->s_aliases)); -#ifdef HAS_NTOHS mPUSHi(PerlSock_ntohs(sent->s_port)); -#else - mPUSHi(sent->s_port); -#endif mPUSHs(newSVpv(sent->s_proto, 0)); } diff --git a/proto.h b/proto.h index a1b3e97..f1d303f 100644 --- a/proto.h +++ b/proto.h @@ -2628,11 +2628,6 @@ PERL_CALLCONV char * Perl_my_strftime(pTHX_ const char *fmt, int sec, int min, i #define PERL_ARGS_ASSERT_MY_STRFTIME \ assert(fmt) -PERL_CALLCONV void Perl_my_swabn(void* ptr, int n) - __attribute__nonnull__(1); -#define PERL_ARGS_ASSERT_MY_SWABN \ - assert(ptr) - PERL_CALLCONV void Perl_my_unexec(pTHX); PERL_CALLCONV int Perl_my_vsnprintf(char *buffer, const Size_t len, const char *format, va_list ap) __attribute__nonnull__(1) @@ -5345,20 +5340,6 @@ PERL_CALLCONV MEM_SIZE Perl_malloced_size(void *p) assert(p) #endif -#if defined(MYSWAP) -PERL_CALLCONV long Perl_my_htonl(pTHX_ long l) - __attribute__warn_unused_result__ - __attribute__pure__; - -PERL_CALLCONV long Perl_my_ntohl(pTHX_ long l) - __attribute__warn_unused_result__ - __attribute__pure__; - -PERL_CALLCONV short Perl_my_swap(pTHX_ short s) - __attribute__warn_unused_result__ - __attribute__pure__; - -#endif #if defined(NO_MATHOMS) /* PERL_CALLCONV void Perl_sv_nounlocking(pTHX_ SV *sv); */ #endif @@ -6263,7 +6244,7 @@ STATIC SV* S_method_common(pTHX_ SV* meth, U32* hashp) #endif #if defined(PERL_IN_PP_PACK_C) -STATIC char * S_bytes_to_uni(const U8 *start, STRLEN len, char *dest) +STATIC char * S_bytes_to_uni(const U8 *start, STRLEN len, char *dest, const bool needs_swap) __attribute__warn_unused_result__ __attribute__nonnull__(1) __attribute__nonnull__(3); @@ -7611,78 +7592,6 @@ PERL_CALLCONV void Perl_xmldump_vindent(pTHX_ I32 level, PerlIO *file, const cha assert(file); assert(pat) #endif -#if defined(PERL_NEED_MY_BETOH16) -PERL_CALLCONV U16 Perl_my_betoh16(U16 n); -#endif -#if defined(PERL_NEED_MY_BETOH32) -PERL_CALLCONV U32 Perl_my_betoh32(U32 n); -#endif -#if defined(PERL_NEED_MY_BETOH64) -PERL_CALLCONV U64 Perl_my_betoh64(U64 n); -#endif -#if defined(PERL_NEED_MY_BETOHI) -PERL_CALLCONV int Perl_my_betohi(int n); -#endif -#if defined(PERL_NEED_MY_BETOHL) -PERL_CALLCONV long Perl_my_betohl(long n); -#endif -#if defined(PERL_NEED_MY_BETOHS) -PERL_CALLCONV short Perl_my_betohs(short n); -#endif -#if defined(PERL_NEED_MY_HTOBE16) -PERL_CALLCONV U16 Perl_my_htobe16(U16 n); -#endif -#if defined(PERL_NEED_MY_HTOBE32) -PERL_CALLCONV U32 Perl_my_htobe32(U32 n); -#endif -#if defined(PERL_NEED_MY_HTOBE64) -PERL_CALLCONV U64 Perl_my_htobe64(U64 n); -#endif -#if defined(PERL_NEED_MY_HTOBEI) -PERL_CALLCONV int Perl_my_htobei(int n); -#endif -#if defined(PERL_NEED_MY_HTOBEL) -PERL_CALLCONV long Perl_my_htobel(long n); -#endif -#if defined(PERL_NEED_MY_HTOBES) -PERL_CALLCONV short Perl_my_htobes(short n); -#endif -#if defined(PERL_NEED_MY_HTOLE16) -PERL_CALLCONV U16 Perl_my_htole16(U16 n); -#endif -#if defined(PERL_NEED_MY_HTOLE32) -PERL_CALLCONV U32 Perl_my_htole32(U32 n); -#endif -#if defined(PERL_NEED_MY_HTOLE64) -PERL_CALLCONV U64 Perl_my_htole64(U64 n); -#endif -#if defined(PERL_NEED_MY_HTOLEI) -PERL_CALLCONV int Perl_my_htolei(int n); -#endif -#if defined(PERL_NEED_MY_HTOLEL) -PERL_CALLCONV long Perl_my_htolel(long n); -#endif -#if defined(PERL_NEED_MY_HTOLES) -PERL_CALLCONV short Perl_my_htoles(short n); -#endif -#if defined(PERL_NEED_MY_LETOH16) -PERL_CALLCONV U16 Perl_my_letoh16(U16 n); -#endif -#if defined(PERL_NEED_MY_LETOH32) -PERL_CALLCONV U32 Perl_my_letoh32(U32 n); -#endif -#if defined(PERL_NEED_MY_LETOH64) -PERL_CALLCONV U64 Perl_my_letoh64(U64 n); -#endif -#if defined(PERL_NEED_MY_LETOHI) -PERL_CALLCONV int Perl_my_letohi(int n); -#endif -#if defined(PERL_NEED_MY_LETOHL) -PERL_CALLCONV long Perl_my_letohl(long n); -#endif -#if defined(PERL_NEED_MY_LETOHS) -PERL_CALLCONV short Perl_my_letohs(short n); -#endif #if defined(PERL_USES_PL_PIDSTATUS) && defined(PERL_IN_UTIL_C) STATIC void S_pidgone(pTHX_ Pid_t pid, int status); #endif diff --git a/regen/genpacksizetables.pl b/regen/genpacksizetables.pl index 6d70cad..da500fb 100644 --- a/regen/genpacksizetables.pl +++ b/regen/genpacksizetables.pl @@ -99,8 +99,8 @@ S! unsigned short v =SIZE16 n =SIZE16 S =SIZE16 -v! =SIZE16 PERL_PACK_CAN_SHRIEKSIGN -n! =SIZE16 PERL_PACK_CAN_SHRIEKSIGN +v! =SIZE16 +n! =SIZE16 i int i! int I unsigned int @@ -112,8 +112,8 @@ l =SIZE32 L! unsigned long V =SIZE32 N =SIZE32 -V! =SIZE32 PERL_PACK_CAN_SHRIEKSIGN -N! =SIZE32 PERL_PACK_CAN_SHRIEKSIGN +V! =SIZE32 +N! =SIZE32 L =SIZE32 p * char * w * * char diff --git a/t/porting/diag.t b/t/porting/diag.t index bcf853e..c5dcbee 100644 --- a/t/porting/diag.t +++ b/t/porting/diag.t @@ -533,8 +533,6 @@ Invalid type '%c' in %s Invalid type '%c' in unpack Invalid type ',' in %s ioctlsocket not implemented! -'j' not supported on this platform -'J' not supported on this platform killpg not implemented! length() used on %s (did you mean "scalar(%s)"?) length() used on %hash (did you mean "scalar(keys %hash)"?) diff --git a/util.c b/util.c index 14d9f2c..66793a3 100644 --- a/util.c +++ b/util.c @@ -2160,335 +2160,6 @@ vsprintf(char *dest, const char *pat, void *args) #endif /* HAS_VPRINTF */ -#ifdef MYSWAP -#if BYTEORDER != 0x4321 -short -Perl_my_swap(pTHX_ short s) -{ -#if (BYTEORDER & 1) == 0 - short result; - - result = ((s & 255) << 8) + ((s >> 8) & 255); - return result; -#else - return s; -#endif -} - -long -Perl_my_htonl(pTHX_ long l) -{ - union { - long result; - char c[sizeof(long)]; - } u; - -#if BYTEORDER == 0x1234 || BYTEORDER == 0x12345678 -#if BYTEORDER == 0x12345678 - u.result = 0; -#endif - u.c[0] = (l >> 24) & 255; - u.c[1] = (l >> 16) & 255; - u.c[2] = (l >> 8) & 255; - u.c[3] = l & 255; - return u.result; -#else -#if ((BYTEORDER - 0x1111) & 0x444) || !(BYTEORDER & 0xf) - Perl_croak(aTHX_ "Unknown BYTEORDER\n"); -#else - I32 o; - I32 s; - - for (o = BYTEORDER - 0x1111, s = 0; s < (sizeof(long)*8); o >>= 4, s += 8) { - u.c[o & 0xf] = (l >> s) & 255; - } - return u.result; -#endif **** PATCH TRUNCATED AT 2000 LINES -- 344 NOT SHOWN **** -- Perl5 Master Repository [perl.git] branch blead, updated. v5.19.0-64-g39ecab1 by Nicholas Clark 2013-05-20T19:19:15Z From: Nicholas Clark In perl.git, the branch blead has been updated - Log ----------------------------------------------------------------- commit 39ecab1c690aee1a1fdeefb6c706448af5d2981f Merge: 6d5abc6 843b375 Author: Nicholas Clark Date: Mon May 20 21:12:01 2013 +0200 Merge the reworking of how genpacksizetables.pl is used. commit 843b37592a564e2e84c28a00619ac6c132c6dbc9 Author: Nicholas Clark Date: Mon Apr 29 12:06:39 2013 +0200 Add regen/genpacksizetables.pl to t/porting/regen.t M t/porting/regen.t commit e3f061af802f7db7c760d810c495d83763fff668 Author: Nicholas Clark Date: Mon Apr 29 11:54:19 2013 +0200 Move genpacksizetables.pl to regen/genpacksizetables.pl M MANIFEST D genpacksizetables.pl M packsizetables.c A regen/genpacksizetables.pl commit 6b05cb8115fc15fed0e6bf162a9cc797222572ee Author: Nicholas Clark Date: Mon Apr 29 11:47:30 2013 +0200 Refactor genpacksizetables.pl to use regen/regen_lib.pl M genpacksizetables.pl M packsizetables.c commit 298bc19c0935302faa4957de141245bb64c6b2df Author: Nicholas Clark Date: Mon Apr 29 11:23:11 2013 +0200 Extract the generated packprops array into packsizetables.c. The C source is the output from genpacksizetables.pl Previously it was pasted into pp_pack.c LocalWords: packprops M MANIFEST A packsizetables.c M pp_pack.c commit e2add64b70b6d5c55a6598257032bf0777c5976c Author: Nicholas Clark Date: Mon Apr 29 11:08:05 2013 +0200 Propagate a change from commit 1651fc447620d361 into genpacksizetables.pl. pp_pack.c contains a table generated by genpacksizetables.pl, pasted into the C source. The C source was updated by commit 1651fc447620d361 in April 2007, but the table used to generate the code was not. M genpacksizetables.pl commit d8a3828919c2d8a9e1344ba4d05795d37ed9cdb6 Author: Karl Williamson Date: Wed Mar 13 16:18:12 2013 -0600 genpacksizetables.pl: Correct comment typo M genpacksizetables.pl ----------------------------------------------------------------------- Summary of changes: MANIFEST | 3 +- packsizetables.c | 254 ++++++++++++++++++++ pp_pack.c | 250 +------------------- genpacksizetables.pl => regen/genpacksizetables.pl | 12 +- t/porting/regen.t | 2 +- 5 files changed, 265 insertions(+), 256 deletions(-) create mode 100644 packsizetables.c rename genpacksizetables.pl => regen/genpacksizetables.pl (92%) diff --git a/MANIFEST b/MANIFEST index 90b5042..7115372 100644 --- a/MANIFEST +++ b/MANIFEST @@ -3886,7 +3886,6 @@ fakethr.h Fake threads header feature.h Feature header form.h Public declarations for formats generate_uudmap.c Generate uudmap.h, the uuencode decoding map -genpacksizetables.pl Generate the size tables for pack/unpack globals.c File to declare global symbols (for shared library) globvar.sym Global variables that need hiding when embedded gv.c Glob value code @@ -4424,6 +4423,7 @@ os2/perlrexx.c Support perl interpreter embedded in REXX os2/perlrexx.cmd Test perl interpreter embedded in REXX overload.c generated overload enum (public) overload.h generated overload name table (implementation) +packsizetables.c The generated packprops array used in pp_pack.c pad.c Scratchpad functions pad.h Scratchpad headers parser.h parser object header @@ -4712,6 +4712,7 @@ regcomp.sym Data for regnodes.h regen/embed_lib.pl Reads embed.fnc and regen/opcodes regen/embed.pl Produces {embed,embedvar,proto}.h regen/feature.pl Generates feature.pm +regen/genpacksizetables.pl Generate the size tables for pack/unpack regen/keywords.pl Program to write keywords.h regen/mg_vtable.pl generate mg_vtable.h regen/mk_invlists.pl Generates charclass_invlists.h diff --git a/packsizetables.c b/packsizetables.c new file mode 100644 index 0000000..afeb210 --- /dev/null +++ b/packsizetables.c @@ -0,0 +1,254 @@ +/* -*- buffer-read-only: t -*- + !!!!!!! DO NOT EDIT THIS FILE !!!!!!! + This file is built by regen/genpacksizetables.pl from its data. + Any changes made here will be lost! + */ + +#if TYPE_IS_SHRIEKING != 0x100 + ++++shriek offset should be 256 +#endif + +typedef U8 packprops_t; +#if 'J'-'I' == 1 +/* ASCII */ +STATIC const packprops_t packprops[512] = { + /* normal */ + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, + /* C */ sizeof(unsigned char), +#if defined(HAS_LONG_DOUBLE) && defined(USE_LONG_DOUBLE) + /* D */ LONG_DOUBLESIZE, +#else + 0, +#endif + 0, + /* F */ NVSIZE, + 0, 0, + /* I */ sizeof(unsigned int), + /* J */ UVSIZE, + 0, + /* L */ SIZE32, + 0, + /* N */ SIZE32, + 0, 0, +#if defined(HAS_QUAD) + /* Q */ sizeof(Uquad_t), +#else + 0, +#endif + 0, + /* S */ SIZE16, + 0, + /* U */ sizeof(char) | PACK_SIZE_UNPREDICTABLE, + /* V */ SIZE32, + /* W */ sizeof(unsigned char) | PACK_SIZE_UNPREDICTABLE, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + /* c */ sizeof(char), + /* d */ sizeof(double), + 0, + /* f */ sizeof(float), + 0, 0, + /* i */ sizeof(int), + /* j */ IVSIZE, + 0, + /* l */ SIZE32, + 0, + /* n */ SIZE16, + 0, + /* p */ sizeof(char *) | PACK_SIZE_CANNOT_CSUM, +#if defined(HAS_QUAD) + /* q */ sizeof(Quad_t), +#else + 0, +#endif + 0, + /* s */ SIZE16, + 0, 0, + /* v */ SIZE16, + /* w */ sizeof(char) | PACK_SIZE_UNPREDICTABLE | PACK_SIZE_CANNOT_CSUM, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + /* shrieking */ + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, + /* I */ sizeof(unsigned int), + 0, 0, + /* L */ sizeof(unsigned long), + 0, +#if defined(PERL_PACK_CAN_SHRIEKSIGN) + /* N */ SIZE32, +#else + 0, +#endif + 0, 0, 0, 0, + /* S */ sizeof(unsigned short), + 0, 0, +#if defined(PERL_PACK_CAN_SHRIEKSIGN) + /* V */ SIZE32, +#else + 0, +#endif + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, + /* i */ sizeof(int), + 0, 0, + /* l */ sizeof(long), + 0, +#if defined(PERL_PACK_CAN_SHRIEKSIGN) + /* n */ SIZE16, +#else + 0, +#endif + 0, 0, 0, 0, + /* s */ sizeof(short), + 0, 0, +#if defined(PERL_PACK_CAN_SHRIEKSIGN) + /* v */ SIZE16, +#else + 0, +#endif + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0 +}; +#else +/* EBCDIC (or bust) */ +STATIC const packprops_t packprops[512] = { + /* normal */ + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, + /* c */ sizeof(char), + /* d */ sizeof(double), + 0, + /* f */ sizeof(float), + 0, 0, + /* i */ sizeof(int), + 0, 0, 0, 0, 0, 0, 0, + /* j */ IVSIZE, + 0, + /* l */ SIZE32, + 0, + /* n */ SIZE16, + 0, + /* p */ sizeof(char *) | PACK_SIZE_CANNOT_CSUM, +#if defined(HAS_QUAD) + /* q */ sizeof(Quad_t), +#else + 0, +#endif + 0, 0, 0, 0, 0, 0, 0, 0, 0, + /* s */ SIZE16, + 0, 0, + /* v */ SIZE16, + /* w */ sizeof(char) | PACK_SIZE_UNPREDICTABLE | PACK_SIZE_CANNOT_CSUM, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + /* C */ sizeof(unsigned char), +#if defined(HAS_LONG_DOUBLE) && defined(USE_LONG_DOUBLE) + /* D */ LONG_DOUBLESIZE, +#else + 0, +#endif + 0, + /* F */ NVSIZE, + 0, 0, + /* I */ sizeof(unsigned int), + 0, 0, 0, 0, 0, 0, 0, + /* J */ UVSIZE, + 0, + /* L */ SIZE32, + 0, + /* N */ SIZE32, + 0, 0, +#if defined(HAS_QUAD) + /* Q */ sizeof(Uquad_t), +#else + 0, +#endif + 0, 0, 0, 0, 0, 0, 0, 0, 0, + /* S */ SIZE16, + 0, + /* U */ sizeof(char) | PACK_SIZE_UNPREDICTABLE, + /* V */ SIZE32, + /* W */ sizeof(unsigned char) | PACK_SIZE_UNPREDICTABLE, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, + /* shrieking */ + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, + /* i */ sizeof(int), + 0, 0, 0, 0, 0, 0, 0, 0, 0, + /* l */ sizeof(long), + 0, +#if defined(PERL_PACK_CAN_SHRIEKSIGN) + /* n */ SIZE16, +#else + 0, +#endif + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + /* s */ sizeof(short), + 0, 0, +#if defined(PERL_PACK_CAN_SHRIEKSIGN) + /* v */ SIZE16, +#else + 0, +#endif + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, + /* I */ sizeof(unsigned int), + 0, 0, 0, 0, 0, 0, 0, 0, 0, + /* L */ sizeof(unsigned long), + 0, +#if defined(PERL_PACK_CAN_SHRIEKSIGN) + /* N */ SIZE32, +#else + 0, +#endif + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + /* S */ sizeof(unsigned short), + 0, 0, +#if defined(PERL_PACK_CAN_SHRIEKSIGN) + /* V */ SIZE32, +#else + 0, +#endif + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 +}; +#endif + +/* ex: set ro: */ diff --git a/pp_pack.c b/pp_pack.c index 6c3dc5f..b41ec78 100644 --- a/pp_pack.c +++ b/pp_pack.c @@ -389,255 +389,7 @@ S_mul128(pTHX_ SV *sv, U8 m) #define PACK_SIZE_UNPREDICTABLE 0x40 /* Not a fixed size element */ #define PACK_SIZE_MASK 0x3F -/* These tables are regenerated by genpacksizetables.pl (and then hand pasted - in). You're unlikely ever to need to regenerate them. */ - -#if TYPE_IS_SHRIEKING != 0x100 - ++++shriek offset should be 256 -#endif - -typedef U8 packprops_t; -#if 'J'-'I' == 1 -/* ASCII */ -STATIC const packprops_t packprops[512] = { - /* normal */ - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, - /* C */ sizeof(unsigned char), -#if defined(HAS_LONG_DOUBLE) && defined(USE_LONG_DOUBLE) - /* D */ LONG_DOUBLESIZE, -#else - 0, -#endif - 0, - /* F */ NVSIZE, - 0, 0, - /* I */ sizeof(unsigned int), - /* J */ UVSIZE, - 0, - /* L */ SIZE32, - 0, - /* N */ SIZE32, - 0, 0, -#if defined(HAS_QUAD) - /* Q */ sizeof(Uquad_t), -#else - 0, -#endif - 0, - /* S */ SIZE16, - 0, - /* U */ sizeof(char) | PACK_SIZE_UNPREDICTABLE, - /* V */ SIZE32, - /* W */ sizeof(unsigned char) | PACK_SIZE_UNPREDICTABLE, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - /* c */ sizeof(char), - /* d */ sizeof(double), - 0, - /* f */ sizeof(float), - 0, 0, - /* i */ sizeof(int), - /* j */ IVSIZE, - 0, - /* l */ SIZE32, - 0, - /* n */ SIZE16, - 0, - /* p */ sizeof(char *) | PACK_SIZE_CANNOT_CSUM, -#if defined(HAS_QUAD) - /* q */ sizeof(Quad_t), -#else - 0, -#endif - 0, - /* s */ SIZE16, - 0, 0, - /* v */ SIZE16, - /* w */ sizeof(char) | PACK_SIZE_UNPREDICTABLE | PACK_SIZE_CANNOT_CSUM, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, - /* shrieking */ - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, - /* I */ sizeof(unsigned int), - 0, 0, - /* L */ sizeof(unsigned long), - 0, -#if defined(PERL_PACK_CAN_SHRIEKSIGN) - /* N */ SIZE32, -#else - 0, -#endif - 0, 0, 0, 0, - /* S */ sizeof(unsigned short), - 0, 0, -#if defined(PERL_PACK_CAN_SHRIEKSIGN) - /* V */ SIZE32, -#else - 0, -#endif - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, - /* i */ sizeof(int), - 0, 0, - /* l */ sizeof(long), - 0, -#if defined(PERL_PACK_CAN_SHRIEKSIGN) - /* n */ SIZE16, -#else - 0, -#endif - 0, 0, 0, 0, - /* s */ sizeof(short), - 0, 0, -#if defined(PERL_PACK_CAN_SHRIEKSIGN) - /* v */ SIZE16, -#else - 0, -#endif - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0 -}; -#else -/* EBCDIC (or bust) */ -STATIC const packprops_t packprops[512] = { - /* normal */ - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, - /* c */ sizeof(char), - /* d */ sizeof(double), - 0, - /* f */ sizeof(float), - 0, 0, - /* i */ sizeof(int), - 0, 0, 0, 0, 0, 0, 0, - /* j */ IVSIZE, - 0, - /* l */ SIZE32, - 0, - /* n */ SIZE16, - 0, - /* p */ sizeof(char *) | PACK_SIZE_CANNOT_CSUM, -#if defined(HAS_QUAD) - /* q */ sizeof(Quad_t), -#else - 0, -#endif - 0, 0, 0, 0, 0, 0, 0, 0, 0, - /* s */ SIZE16, - 0, 0, - /* v */ SIZE16, - /* w */ sizeof(char) | PACK_SIZE_UNPREDICTABLE | PACK_SIZE_CANNOT_CSUM, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - /* C */ sizeof(unsigned char), -#if defined(HAS_LONG_DOUBLE) && defined(USE_LONG_DOUBLE) - /* D */ LONG_DOUBLESIZE, -#else - 0, -#endif - 0, - /* F */ NVSIZE, - 0, 0, - /* I */ sizeof(unsigned int), - 0, 0, 0, 0, 0, 0, 0, - /* J */ UVSIZE, - 0, - /* L */ SIZE32, - 0, - /* N */ SIZE32, - 0, 0, -#if defined(HAS_QUAD) - /* Q */ sizeof(Uquad_t), -#else - 0, -#endif - 0, 0, 0, 0, 0, 0, 0, 0, 0, - /* S */ SIZE16, - 0, - /* U */ sizeof(char) | PACK_SIZE_UNPREDICTABLE, - /* V */ SIZE32, - /* W */ sizeof(unsigned char) | PACK_SIZE_UNPREDICTABLE, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, - /* shrieking */ - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, - /* i */ sizeof(int), - 0, 0, 0, 0, 0, 0, 0, 0, 0, - /* l */ sizeof(long), - 0, -#if defined(PERL_PACK_CAN_SHRIEKSIGN) - /* n */ SIZE16, -#else - 0, -#endif - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - /* s */ sizeof(short), - 0, 0, -#if defined(PERL_PACK_CAN_SHRIEKSIGN) - /* v */ SIZE16, -#else - 0, -#endif - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, - /* I */ sizeof(unsigned int), - 0, 0, 0, 0, 0, 0, 0, 0, 0, - /* L */ sizeof(unsigned long), - 0, -#if defined(PERL_PACK_CAN_SHRIEKSIGN) - /* N */ SIZE32, -#else - 0, -#endif - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - /* S */ sizeof(unsigned short), - 0, 0, -#if defined(PERL_PACK_CAN_SHRIEKSIGN) - /* V */ SIZE32, -#else - 0, -#endif - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -}; -#endif +#include "packsizetables.c" STATIC U8 uni_to_byte(pTHX_ const char **s, const char *end, I32 datumtype) diff --git a/genpacksizetables.pl b/regen/genpacksizetables.pl similarity index 92% rename from genpacksizetables.pl rename to regen/genpacksizetables.pl index 9dffc2c..6d70cad 100644 --- a/genpacksizetables.pl +++ b/regen/genpacksizetables.pl @@ -1,8 +1,9 @@ #!/usr/bin/perl -w # I'm assuming that you're running this on some kind of ASCII system, but -# it will generate EDCDIC too. (TODO) +# it will generate EBCDIC too. (TODO) use strict; use Encode; +require 'regen/regen_lib.pl'; sub make_text { my ($chrmap, $letter, $unpredictable, $nocsum, $size, $condition) = @_; @@ -66,10 +67,9 @@ my @lines = grep { my %asciimap = map {chr $_, chr $_} 0..255; my %ebcdicmap = map {chr $_, Encode::encode("posix-bc", chr $_)} 0..255; -print <<"EOC"; -/* These tables are regenerated by genpacksizetables.pl (and then hand pasted - in). You're unlikely ever to need to regenerate them. */ +my $fh = open_new('packsizetables.c', '>', { by => $0, from => 'its data'}); +print $fh <<"EOC"; #if TYPE_IS_SHRIEKING != 0x100 ++++shriek offset should be 256 #endif @@ -84,11 +84,13 @@ typedef U8 packprops_t; #endif EOC +read_only_bottom_close_and_rename($fh); + __DATA__ #Symbol unpredictable # nocsum size c char -C * unsigned char +C unsigned char W * unsigned char U * char s! short diff --git a/t/porting/regen.t b/t/porting/regen.t index e38ea81..ec8e62d 100644 --- a/t/porting/regen.t +++ b/t/porting/regen.t @@ -19,7 +19,7 @@ if ( $^O eq "VMS" ) { my $in_regen_pl = 23; # I can't see a clean way to calculate this automatically. my @files = qw(perly.act perly.h perly.tab keywords.c keywords.h uconfig.h); my @progs = qw(regen/regcharclass.pl regen/mk_PL_charclass.pl - regen/unicode_constants.pl); + regen/unicode_constants.pl regen/genpacksizetables.pl); plan (tests => $in_regen_pl + @files + @progs + 2); -- Perl5 Master Repository [perl.git] branch smoke-me/nicholas/likely-cBOOL, deleted. v5.17.9-104-gc0b8e78 by Nicholas Clark 2013-05-20T17:39:21Z From: Nicholas Clark In perl.git, the branch smoke-me/nicholas/likely-cBOOL has been deleted was c0b8e78149fde4b5e8c0319f943946a178b3109e ----------------------------------------------------------------------- c0b8e78149fde4b5e8c0319f943946a178b3109e Change LIKELY() and UNLIKELY() to cast their first argument to a bool. ----------------------------------------------------------------------- -- Perl5 Master Repository [perl.git] branch blead, updated. v5.19.0-57-g6d5abc6 by Nicholas Clark 2013-05-20T17:39:02Z From: Nicholas Clark In perl.git, the branch blead has been updated - Log ----------------------------------------------------------------- commit 6d5abc62848da0e110869e2b13c68bf498ff3a90 Author: Nicholas Clark Date: Wed Mar 6 16:27:08 2013 +0100 Change LIKELY() and UNLIKELY() to cast their first argument to a bool. This avoids problems such as pointers being compared with the literal value 1. Suggested by Zefram. Note that this is technically a change of behaviour, as the macro EXPECT(), which they are both both wrappers for, returns its value, so they will now be returning TRUE or FALSE, rather than the actual value passed to them. However, all uses in the core and on CPAN are only within if() statements, so this should not make any difference in practice. ----------------------------------------------------------------------- Summary of changes: perl.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/perl.h b/perl.h index 89f4c98..dc5e411 100644 --- a/perl.h +++ b/perl.h @@ -3329,8 +3329,8 @@ typedef pthread_key_t perl_key; #else # define EXPECT(expr,val) (expr) #endif -#define LIKELY(cond) EXPECT(cond,1) -#define UNLIKELY(cond) EXPECT(cond,0) +#define LIKELY(cond) EXPECT(cBOOL(cond),TRUE) +#define UNLIKELY(cond) EXPECT(cBOOL(cond),FALSE) #ifdef HAS_BUILTIN_CHOOSE_EXPR /* placeholder */ #endif -- Perl5 Master Repository [perl.git] branch blead, updated. v5.19.0-56-gb2bf251 by Karl Williamson 2013-05-20T16:57:54Z From: Karl Williamson In perl.git, the branch blead has been updated <http://perl5.git.perl.org/perl.git/commitdiff/b2bf251fd850e281217ec1d042c1839e3ed4a09c?hp=4572fa3eeef0e6fc24d4f16823ed87d5a8e4778b> - Log ----------------------------------------------------------------- commit b2bf251fd850e281217ec1d042c1839e3ed4a09c Author: Karl Williamson <public@khwilliamson.com> Date: Sat May 18 14:05:00 2013 -0600 handy.h: Change the error return of two macros These two undocumented macros returned the REPLACEMENT CHARACTER if the input was outside the Latin1 range. This was contrary to all other similar macros, which return their input if it is invalid. It caused warnings in some (dumber than average) compilers. These macros are undocumented; this changes the behavior only of illegal inputs to them. M handy.h commit cfe934daf72e37a0980ac3183d678e49a26726b3 Author: Karl Williamson <public@khwilliamson.com> Date: Sat May 18 13:55:50 2013 -0600 regcomp.c: Add comment M regcomp.c commit 4f48919406dde7316da0835587373a50763e73ae Author: Karl Williamson <public@khwilliamson.com> Date: Sat May 18 13:06:47 2013 -0600 utf8.c: White space only. Indent in newly formed block M utf8.c commit a3f3bf18711ca1acde9e5a41d4c7b1f3adc4afd1 Author: Karl Williamson <public@khwilliamson.com> Date: Sat May 18 13:09:05 2013 -0600 fold_grind.t: Add tests fold-grind stresses the /i regex handling by reading the Unicode folding rules and creating tests for problematic cases, as well as some non-problematic ones. Because of the large number of tests generated, it otherwise avoids tests that it doesn't think are problematic. Problematic cases include those whose folds cross the ordinal 127/128 or 255/256 boundaries, along with other considerations. Until this commit, fold_grind failed to realize that even if the pair of characters currently being tested don't cross those boundaries, if their eventual folds do, this could be problematic. Suppose X and Y are both on the same side of the boundaries, and both fold to Z which is on another side. Then X and Y should be equivalent under /i even if the fold to Z is prohibited because of /aa or /l. fold_grind was overzealous in pruning such tests. The previous patch fixed bugs in handling such cases; and this patch fixes the tests to look for similar things that might possibly arise in the future. Interestingly, this bug came to light during porting to an EBCDIC platform. The reason is that because of the different collation order, fold_grind chose one of the buggy cases to test as one its non-problematic tests. M t/re/fold_grind.t commit 1ca267a56acf698557ec1deec44af651acc88696 Author: Karl Williamson <public@khwilliamson.com> Date: Sat May 18 08:25:16 2013 -0600 Fix multi-char fold edge case use locale; fc("\N{LATIN CAPITAL LETTER SHARP S}") eq 2 x fc("\N{LATIN SMALL LETTER LONG S}") should return true, as the SHARP S folds to two 's's in a row, and the LONG S is an antique variant of 's', and folds to s. Until this commit, the expression was false. Similarly, the following should match, but didn't until this commit: "\N{LATIN SMALL LETTER SHARP S}" =~ /\N{LATIN SMALL LETTER LONG S}{2}/iaa The reason these didn't work properly is that in both cases the actual fold to 's' is disallowed. In the first case because of locale; and in the second because of /aa. And the code wasn't smart enough to realize that these were legal. The fix is to special case these so that the fold of sharp s (both capital and small) is two LONG S's under /aa; as is the fold of the capital sharp s under locale. The latter is user-visible, and the documentation of fc() now points that out. I believe this is such an edge case that no mention of it need be done in perldelta. M lib/locale.t M pod/perlfunc.pod M regcharclass.h M regcomp.c M regen/regcharclass_multi_char_folds.pl M t/uni/fold.t M utf8.c commit 519101418837cf0edacb710b2b38b42dad6e47c1 Author: Karl Williamson <public@khwilliamson.com> Date: Fri May 17 23:07:05 2013 -0600 Expand flags parameter from boolean in _to_fold_latin1 This will be used in future commits to pass more flags. M embed.fnc M proto.h M regcomp.c M regexec.c M utf8.c commit f2e063754de41c901afa81619aeace2fb9d03f9c Author: Karl Williamson <public@khwilliamson.com> Date: Thu May 16 21:37:43 2013 -0600 unicode_constants.h: Add some #defines These will be used in future commits M regen/unicode_constants.pl M unicode_constants.h commit a4f12ed78e91a49ab4fec1f1e637d980366cbc1a Author: Karl Williamson <public@khwilliamson.com> Date: Wed May 15 20:55:27 2013 -0600 utf8.c: Replace two macro calls with equiv single UTF8_IS_ABOVE_LATIN1() is equivalent to (! UTF8_IS_INVARIANT && !UTF8_IS_DOWNGRADEABLE_START) So we can use just it, for clearer code with fewer branches. M utf8.c commit 33bdd83dc2db4d07c1540394340770a6a07e793e Author: Karl Williamson <public@khwilliamson.com> Date: Sat May 11 16:00:02 2013 -0600 lib/charnames.t: Fix wrongly complemented test The 'if' test should be the opposite of what it is. I believe I had it this way for forcing the branch to be taken during testing, and forgot to restore it. It only matters if Perl is compiled with early Unicodes, or on a non-ASCII platform. M lib/charnames.t commit 91638220a6a521d89235c0a8e51d53f96e0e67c8 Author: Karl Williamson <public@khwilliamson.com> Date: Thu May 9 20:48:40 2013 -0600 t/re/fold_grind.t: White space only Commit 3345a47950127cf99a197eba4dce9c91f7bc9139 created a new block; but didn't indent it. This commit does. M t/re/fold_grind.t commit 58a428bba943d9d3e6057e7ea3202ea3562c557a Author: Karl Williamson <public@khwilliamson.com> Date: Tue May 7 10:04:40 2013 -0600 perlapi: Move 'experimental' warning to front of entries In a long multi-paragraph entry, the fact that the described function is considered experimental may be lost, as it comes at the end. This just moves it to the front. M autodoc.pl commit fdfb35c800b283d7f03f2b03dc5a1adb81a4a6e4 Author: Karl Williamson <public@khwilliamson.com> Date: Tue May 7 10:01:58 2013 -0600 regcomp.c: Remove always-true test In this code, j is guaranteed to be above 255, so no need to test for that. M regcomp.c commit ff5deb98f57138ca084471a3abb7b702b1347af8 Author: Karl Williamson <public@khwilliamson.com> Date: Sun May 5 18:26:39 2013 -0600 regcomp.c: White-space only The previous commit allows us to outdent a largish block, reflowing things to fit into the extra available width, and saving a few vertical pixels. M regcomp.c commit b35bbc3f802dc4b4abbbc28ed19bc5108f2b7ef1 Author: Karl Williamson <public@khwilliamson.com> Date: Sun May 5 10:40:56 2013 -0600 regcomp.c: Reorder two 'if's so shorter branches are first This makes it easier to understand what is going on M regcomp.c commit b9cc4f69dc87bbd21b54bb9a4466203cf97afacc Author: Karl Williamson <public@khwilliamson.com> Date: Sat May 4 22:22:53 2013 -0600 perllocale: Update to mention fc(), \F M pod/perllocale.pod commit d3cb55c1b8a3b80c74a4fde3bd7b67b7af31baea Author: Karl Williamson <public@khwilliamson.com> Date: Sat May 4 21:23:05 2013 -0600 regcomp.c: Clarify comment M regcomp.c commit a78bc3c6c5c9dd76fdfbdae88ef4e89136aff637 Author: Karl Williamson <public@khwilliamson.com> Date: Sat May 4 20:23:14 2013 -0600 pp.c: Eliminate custom macro and use Copy() instead I think it's clearer to use Copy. When I wrote this custom macro, we didn't have the infrastructure to generate a UTF-8 encoded string at compile time. M pp.c M regen/unicode_constants.pl M unicode_constants.h commit a96bc63507f57f3547f982d5e9a8b95f68e87e3b Author: Dagfinn Ilmari MannsÃ¥ker <ilmari@ilmari.org> Date: Sat May 4 20:18:35 2013 -0600 perl #117865] [PATCH] Eliminate useless variable and sizeof(char) bufsiz is always just set from bsiz (via a useless multiplication by sizeof(char), which is by definition 1), so instead of trying to keep them in sync, just get rid of bufsiz use bsiz directly# Please enter the commit message for your changes. Lines starting M util.c commit 7ccde1209c3d4319a2e0ab9a4528f6abc8eec0d2 Author: Karl Williamson <public@khwilliamson.com> Date: Sat May 4 16:09:39 2013 -0600 op.c: Don't fold constant fc() in locale We do compile time folding of calls to ops with constant parameters. This should be skipped if the op's result depends on locale, and it is being called from within the scope of 'use locale', as the result is not known until runtime. fc() was folding anyway. There is no test, as this only shows up when run in a locale that it makes a difference in, and there is no guarantee that such a locale would occur on any computer, and it is a real pain to go searching through the computer's available locales for such a one for just this error. M op.c commit 8c869419331313c43ddda5b1eecd01939b013125 Author: Karl Williamson <public@khwilliamson.com> Date: Tue Apr 30 08:42:08 2013 -0600 autodoc.pl: Don't list undocumented deprecated fcns in API autodoc creates a list of all the undocumented functions that are part of the API. It omits ones that are experimental and whose API may change; and now it omits ones that are deprecated (and whose API is planned to change to be non-existent) M autodoc.pl commit d4e99c76278c7e400307b1a7ceeb3b27676dfd3d Author: Karl Williamson <public@khwilliamson.com> Date: Tue Apr 30 08:39:44 2013 -0600 autodoc.pl: Add note for deprecated functions This causes each deprecated function to have a prominent note to that effect in its API documentation. M autodoc.pl M mg.c M utf8.c commit 6ef8d77a18f45460857e0866d103b834ec25d432 Author: Karl Williamson <public@khwilliamson.com> Date: Sat Apr 27 08:59:19 2013 -0600 embed.fnc: Slight clarification in comments M embed.fnc commit 2e1414cec3028c7f6c19d1229a4b9a460267fb15 Author: Karl Williamson <public@khwilliamson.com> Date: Mon May 20 10:39:56 2013 -0600 Add tests for case-change macros API M ext/XS-APItest/APItest.pm M ext/XS-APItest/APItest.xs M ext/XS-APItest/t/handy.t commit d22b930b662c8c65d2e856c42f4a59454444a111 Author: Karl Williamson <public@khwilliamson.com> Date: Thu Apr 25 10:07:47 2013 -0600 Use new case changing macros The previous commit added macros to do some case changing. This commit uses them in the core, where appropriate. M ext/File-Glob/bsd_glob.c M pp.c M utf8.c M utf8.h commit 25200305a51f99449a69c43f5d279893ab5a0188 Author: Karl Williamson <public@khwilliamson.com> Date: Thu Apr 25 10:05:22 2013 -0600 handy.h: Add some macro definitions These macros fill in all the missing case changing operations. They were omitted before because they are identical in their input domains to other operations. But by adding them here, that detail no longer need be known by the callers. toFOLD_LC is not documented, as is subject to change M handy.h commit 22e255cb04e4d885f89e7fa81ad967031dd8c193 Author: Karl Williamson <public@khwilliamson.com> Date: Thu Apr 25 08:41:13 2013 -0600 pp.c: Nit in comment M pp.c commit fa2b1084e3c2ca84300c9da8bdd7f808b78d35bf Author: Karl Williamson <public@khwilliamson.com> Date: Wed Apr 24 15:39:08 2013 -0600 perlclib.pod: Update character class macro descriptions Much has changed since this pod was last updated. M pod/perlclib.pod commit 5b2821400d3dea1b132a955e865a58e89133c569 Author: Karl Williamson <public@khwilliamson.com> Date: Wed Apr 24 15:36:07 2013 -0600 perlclib.pod: Fix too long verbatim lines M pod/perlclib.pod M t/porting/known_pod_issues.dat commit 1f607577c172af48670c82975935160a27e002b9 Author: Karl Williamson <public@khwilliamson.com> Date: Wed Apr 24 21:18:14 2013 -0600 perlapi: Add docs for some case-changing macros; clarify others The case changing macros are now almost all documented. The exception is toUPPER_LC, which may change in 5.19 In addition the functions in utf8.c that these macros call now refer to them instead of having their own documentation. People should really be using the macros instead of calling the functions directly. I'm not deprecating the functions because I can't foresee the need to change them, so code that uses them should continue to be ok. M handy.h M utf8.c ----------------------------------------------------------------------- Summary of changes: autodoc.pl | 17 ++- embed.fnc | 4 +- ext/File-Glob/bsd_glob.c | 4 +- ext/XS-APItest/APItest.pm | 2 +- ext/XS-APItest/APItest.xs | 224 ++++++++++++++++++++ ext/XS-APItest/t/handy.t | 159 ++++++++++++++- handy.h | 145 ++++++++++++-- lib/charnames.t | 4 +- lib/locale.t | 2 +- mg.c | 4 +- op.c | 1 + pod/perlclib.pod | 195 ++++++++++-------- pod/perlfunc.pod | 12 +- pod/perllocale.pod | 6 +- pp.c | 35 ++-- proto.h | 2 +- regcharclass.h | 6 + regcomp.c | 282 ++++++++++++++----------- regen/regcharclass_multi_char_folds.pl | 23 ++ regen/unicode_constants.pl | 6 + regexec.c | 2 +- t/porting/known_pod_issues.dat | 1 - t/re/fold_grind.t | 361 +++++++++++++++++++------------- t/uni/fold.t | 6 + unicode_constants.h | 6 + utf8.c | 156 +++++++-------- utf8.h | 4 +- util.c | 6 +- 28 files changed, 1167 insertions(+), 508 deletions(-) diff --git a/autodoc.pl b/autodoc.pl index 925f2f5..59dc6f0 100644 --- a/autodoc.pl +++ b/autodoc.pl @@ -113,6 +113,7 @@ DOC: if ($docref and %$docref) { $embed_where = $docref->{flags} =~ /A/ ? 'api' : 'guts'; $embed_may_change = $docref->{flags} =~ /M/; + $flags .= 'D' if $docref->{flags} =~ /D/; } else { $missing{$name} = $file; } @@ -171,8 +172,15 @@ sub docout ($$$) { # output the docs for one function my($flags, $docs, $ret, $file, @args) = @$docref; $name =~ s/\s*$//; - $docs .= "NOTE: this function is experimental and may change or be -removed without notice.\n\n" if $flags =~ /x/; + if ($flags =~ /D/) { + $docs = "\n\nDEPRECATED! It is planned to remove this function from a +future release of Perl. Do not use it for new code; remove it from +existing code.\n\n$docs"; + } + else { + $docs = "\n\nNOTE: this function is experimental and may change or be +removed without notice.\n\n$docs" if $flags =~ /x/; + } $docs .= "NOTE: the perl_ form of this function is deprecated.\n\n" if $flags =~ /p/; $docs .= "NOTE: this function must be explicitly called as Perl_$name with an aTHX_ parameter.\n\n" @@ -374,8 +382,9 @@ foreach (sort keys %missing) { # walk table providing an array of components in each line to # subroutine, printing the result -# List of funcs in the public API that aren't also marked as experimental. -my @missing_api = grep $funcflags{$_}{flags} =~ /A/ && $funcflags{$_}{flags} !~ /M/ && !$docs{api}{$_}, keys %funcflags; +# List of funcs in the public API that aren't also marked as experimental nor +# deprecated. +my @missing_api = grep $funcflags{$_}{flags} =~ /A/ && $funcflags{$_}{flags} !~ /[MD]/ && !$docs{api}{$_}, keys %funcflags; output('perlapi', <<'_EOB_', $docs{api}, \@missing_api, <<'_EOE_'); =head1 NAME diff --git a/embed.fnc b/embed.fnc index 480de45..8b73d6d 100644 --- a/embed.fnc +++ b/embed.fnc @@ -39,7 +39,7 @@ : : proto.h: add __attribute__deprecated__ : -: d Function has documentation in the source: +: d Function has documentation (somewhere) in the source: : : enables 'no docs for foo" warning in autodoc.pl : @@ -633,7 +633,7 @@ iDMPR |bool |isALNUM_lazy |NN const char* p sR |U8 |to_lower_latin1|const U8 c|NULLOK U8 *p|NULLOK STRLEN *lenp #endif #if defined(PERL_IN_UTF8_C) || defined(PERL_IN_REGCOMP_C) || defined(PERL_IN_REGEXEC_C) -EXp |UV |_to_fold_latin1|const U8 c|NN U8 *p|NN STRLEN *lenp|const bool flags +EXp |UV |_to_fold_latin1|const U8 c|NN U8 *p|NN STRLEN *lenp|const unsigned int flags #endif #if defined(PERL_IN_UTF8_C) || defined(PERL_IN_PP_C) p |UV |_to_upper_title_latin1|const U8 c|NN U8 *p|NN STRLEN *lenp|const char S_or_s diff --git a/ext/File-Glob/bsd_glob.c b/ext/File-Glob/bsd_glob.c index 72d8fc7..7ec24df 100644 --- a/ext/File-Glob/bsd_glob.c +++ b/ext/File-Glob/bsd_glob.c @@ -617,12 +617,12 @@ ci_compare(const void *p, const void *q) const char *qq = *(const char **)q; int ci; while (*pp && *qq) { - if (toLOWER(*pp) != toLOWER(*qq)) + if (toFOLD(*pp) != toFOLD(*qq)) break; ++pp; ++qq; } - ci = toLOWER(*pp) - toLOWER(*qq); + ci = toFOLD(*pp) - toFOLD(*qq); if (ci == 0) return compare(p, q); return ci; diff --git a/ext/XS-APItest/APItest.pm b/ext/XS-APItest/APItest.pm index 410477a..b87c4e1 100644 --- a/ext/XS-APItest/APItest.pm +++ b/ext/XS-APItest/APItest.pm @@ -5,7 +5,7 @@ use strict; use warnings; use Carp; -our $VERSION = '0.51'; +our $VERSION = '0.52'; require XSLoader; diff --git a/ext/XS-APItest/APItest.xs b/ext/XS-APItest/APItest.xs index dbb4f50..3f76dd7 100644 --- a/ext/XS-APItest/APItest.xs +++ b/ext/XS-APItest/APItest.xs @@ -4443,3 +4443,227 @@ test_isQUOTEMETA(UV ord) RETVAL = _isQUOTEMETA(ord); OUTPUT: RETVAL + +UV +test_toLOWER(UV ord) + CODE: + RETVAL = toLOWER(ord); + OUTPUT: + RETVAL + +UV +test_toLOWER_L1(UV ord) + CODE: + RETVAL = toLOWER_L1(ord); + OUTPUT: + RETVAL + +UV +test_toLOWER_LC(UV ord) + CODE: + RETVAL = toLOWER_LC(ord); + OUTPUT: + RETVAL + +AV * +test_toLOWER_uni(UV ord) + PREINIT: + U8 s[UTF8_MAXBYTES_CASE + 1]; + STRLEN len; + AV *av; + SV *utf8; + CODE: + av = newAV(); + av_push(av, newSVuv(toLOWER_uni(ord, s, &len))); + + utf8 = newSVpvn((char *) s, len); + SvUTF8_on(utf8); + av_push(av, utf8); + + av_push(av, newSVuv(len)); + RETVAL = av; + OUTPUT: + RETVAL + +AV * +test_toLOWER_utf8(SV * p) + PREINIT: + U8 *input; + U8 s[UTF8_MAXBYTES_CASE + 1]; + STRLEN len; + AV *av; + SV *utf8; + CODE: + input = (U8 *) SvPV(p, len); + av = newAV(); + av_push(av, newSVuv(toLOWER_utf8(input, s, &len))); + + utf8 = newSVpvn((char *) s, len); + SvUTF8_on(utf8); + av_push(av, utf8); + + av_push(av, newSVuv(len)); + RETVAL = av; + OUTPUT: + RETVAL + +UV +test_toFOLD(UV ord) + CODE: + RETVAL = toFOLD(ord); + OUTPUT: + RETVAL + +UV +test_toFOLD_LC(UV ord) + CODE: + RETVAL = toFOLD_LC(ord); + OUTPUT: + RETVAL + +AV * +test_toFOLD_uni(UV ord) + PREINIT: + U8 s[UTF8_MAXBYTES_CASE + 1]; + STRLEN len; + AV *av; + SV *utf8; + CODE: + av = newAV(); + av_push(av, newSVuv(toFOLD_uni(ord, s, &len))); + + utf8 = newSVpvn((char *) s, len); + SvUTF8_on(utf8); + av_push(av, utf8); + + av_push(av, newSVuv(len)); + RETVAL = av; + OUTPUT: + RETVAL + +AV * +test_toFOLD_utf8(SV * p) + PREINIT: + U8 *input; + U8 s[UTF8_MAXBYTES_CASE + 1]; + STRLEN len; + AV *av; + SV *utf8; + CODE: + input = (U8 *) SvPV(p, len); + av = newAV(); + av_push(av, newSVuv(toFOLD_utf8(input, s, &len))); + + utf8 = newSVpvn((char *) s, len); + SvUTF8_on(utf8); + av_push(av, utf8); + + av_push(av, newSVuv(len)); + RETVAL = av; + OUTPUT: + RETVAL + +UV +test_toUPPER(UV ord) + CODE: + RETVAL = toUPPER(ord); + OUTPUT: + RETVAL + +UV +test_toUPPER_LC(UV ord) + CODE: + RETVAL = toUPPER_LC(ord); + OUTPUT: + RETVAL + +AV * +test_toUPPER_uni(UV ord) + PREINIT: + U8 s[UTF8_MAXBYTES_CASE + 1]; + STRLEN len; + AV *av; + SV *utf8; + CODE: + av = newAV(); + av_push(av, newSVuv(toUPPER_uni(ord, s, &len))); + + utf8 = newSVpvn((char *) s, len); + SvUTF8_on(utf8); + av_push(av, utf8); + + av_push(av, newSVuv(len)); + RETVAL = av; + OUTPUT: + RETVAL + +AV * +test_toUPPER_utf8(SV * p) + PREINIT: + U8 *input; + U8 s[UTF8_MAXBYTES_CASE + 1]; + STRLEN len; + AV *av; + SV *utf8; + CODE: + input = (U8 *) SvPV(p, len); + av = newAV(); + av_push(av, newSVuv(toUPPER_utf8(input, s, &len))); + + utf8 = newSVpvn((char *) s, len); + SvUTF8_on(utf8); + av_push(av, utf8); + + av_push(av, newSVuv(len)); + RETVAL = av; + OUTPUT: + RETVAL + +UV +test_toTITLE(UV ord) + CODE: + RETVAL = toTITLE(ord); + OUTPUT: + RETVAL + +AV * +test_toTITLE_uni(UV ord) + PREINIT: + U8 s[UTF8_MAXBYTES_CASE + 1]; + STRLEN len; + AV *av; + SV *utf8; + CODE: + av = newAV(); + av_push(av, newSVuv(toTITLE_uni(ord, s, &len))); + + utf8 = newSVpvn((char *) s, len); + SvUTF8_on(utf8); + av_push(av, utf8); + + av_push(av, newSVuv(len)); + RETVAL = av; + OUTPUT: + RETVAL + +AV * +test_toTITLE_utf8(SV * p) + PREINIT: + U8 *input; + U8 s[UTF8_MAXBYTES_CASE + 1]; + STRLEN len; + AV *av; + SV *utf8; + CODE: + input = (U8 *) SvPV(p, len); + av = newAV(); + av_push(av, newSVuv(toTITLE_utf8(input, s, &len))); + + utf8 = newSVpvn((char *) s, len); + SvUTF8_on(utf8); + av_push(av, utf8); + + av_push(av, newSVuv(len)); + RETVAL = av; + OUTPUT: + RETVAL diff --git a/ext/XS-APItest/t/handy.t b/ext/XS-APItest/t/handy.t index eb620ec..84ffb02 100644 --- a/ext/XS-APItest/t/handy.t +++ b/ext/XS-APItest/t/handy.t @@ -6,7 +6,7 @@ use Config; use XS::APItest; -use Unicode::UCD qw(prop_invlist); +use Unicode::UCD qw(prop_invlist prop_invmap); sub truth($) { # Converts values so is() works return (shift) ? 1 : 0; @@ -94,7 +94,8 @@ foreach my $name (sort keys %properties) { push @code_points, 0x110000; # Above Unicode, no prop should match no warnings 'non_unicode'; - for my $i (@code_points) { + for my $j (@code_points) { + my $i = utf8::native_to_unicode($j); my $function = uc($name); my $matches = Unicode::UCD::_search_invlist(\@invlist, $i); @@ -206,6 +207,160 @@ foreach my $name (sort keys %properties) { } } +my %to_properties = ( + FOLD => 'Case_Folding', + LOWER => 'Lowercase_Mapping', + TITLE => 'Titlecase_Mapping', + UPPER => 'Uppercase_Mapping', + ); + + +foreach my $name (sort keys %to_properties) { + my $property = $to_properties{$name}; + my ($list_ref, $map_ref, $format, $missing) + = prop_invmap($property, ); + if (! $list_ref || ! $map_ref) { + fail("No inversion map found for $property"); + next; + } + if ($format ne "al") { + fail("Unexpected inversion map format ('$format') found for $property"); + next; + } + + # Include all the Latin1 code points, plus 0x100. + my @code_points = (0 .. 256); + + # Then include the next few multi-char folds above those from this + # property, and include the next few single folds as well + my $above_latins = 0; + my $multi_char = 0; + for my $i (0 .. @{$list_ref} - 1) { + my $range_start = $list_ref->[$i]; + next if $range_start < 257; + if (ref $map_ref->[$i] && $multi_char < 5) { + push @code_points, $range_start - 1 if $code_points[-1] != $range_start - 1; + push @code_points, $range_start; + $multi_char++; + } + elsif ($above_latins < 5) { + push @code_points, $range_start - 1 if $code_points[-1] != $range_start - 1; + push @code_points, $range_start; + $above_latins++; + } + last if $above_latins >= 5 && $multi_char >= 5; + } + + # And finally one non-Unicode code point. + push @code_points, 0x110000; # Above Unicode, no prop should match + no warnings 'non_unicode'; + + # $j is native; $i unicode. + for my $j (@code_points) { + my $i = utf8::native_to_unicode($j); + my $function = $name; + + my $index = Unicode::UCD::_search_invlist(\@{$list_ref}, $j); + + my $ret; + my $char_name = charnames::viacode($j) // "No name"; + my $display_name = sprintf "\\N{U+%02X, %s}", $i, $char_name; + + # Test the base function + $ret = eval "test_to${function}($j)"; + if ($@) { + fail($@); + } + else { + my $should_be = ($i < 128 && $map_ref->[$index] != $missing) + ? $map_ref->[$index] + $j - $list_ref->[$index] + : $j; + is ($ret, $should_be, sprintf("to${function}( $display_name ) == 0x%02X", $should_be)); + } + + # Test _L1 + if ($name eq 'LOWER') { + $ret = eval "test_to${function}_L1($j)"; + if ($@) { + fail($@); + } + else { + my $should_be = ($i < 256 && $map_ref->[$index] != $missing) + ? $map_ref->[$index] + $j - $list_ref->[$index] + : $j; + is ($ret, $should_be, sprintf("to${function}_L1( $display_name ) == 0x%02X", $should_be)); + } + } + + if ($name ne 'TITLE' && defined $locale) { + require locale; import locale; + + # Test _LC; titlecase is not defined in locales. + $ret = eval "test_to${function}_LC($j)"; + if ($@) { + fail($@); + } + else { + my $should_be = ($i < 128 && $map_ref->[$index] != $missing) + ? $map_ref->[$index] + $j - $list_ref->[$index] + : $j; + is ($ret, $should_be, sprintf("to${function}_LC( $display_name ) == 0x%02X", $should_be)); + } + } + + # The _uni and _utf8 functions return both the ordinal of the first + # code point of the result, and the result in utf8. The .xs tests + # return these in an array, in [0] and [1] respectively, with [2] the + # length of the utf8 in bytes. + my $utf8_should_be = ""; + my $first_ord_should_be; + if (ref $map_ref->[$index]) { # A multi-char result + for my $j (0 .. @{$map_ref->[$index]} - 1) { + $utf8_should_be .= chr $map_ref->[$index][$j]; + } + + $first_ord_should_be = $map_ref->[$index][0]; + } + else { # A single-char result + $first_ord_should_be = ($map_ref->[$index] != $missing) + ? $map_ref->[$index] + $j - $list_ref->[$index] + : $j; + $utf8_should_be = chr $first_ord_should_be; + } + utf8::upgrade($utf8_should_be); + + # Test _uni + my $s; + my $len; + $ret = eval "test_to${function}_uni($j)"; + if ($@) { + fail($@); + } + else { + is ($ret->[0], $first_ord_should_be, sprintf("to${function}_uni( $display_name ) == 0x%02X", $first_ord_should_be)); + is ($ret->[1], $utf8_should_be, sprintf("utf8 of to${function}_uni( $display_name )")); + use bytes; + is ($ret->[2], length $utf8_should_be, sprintf("number of bytes in utf8 of to${function}_uni( $display_name )")); + } + + # Test _utf8 + my $char = chr($j); + utf8::upgrade($char); + $char = quotemeta $char if $char eq '\\' || $char eq "'"; + $ret = eval "test_to${function}_utf8('$char')"; + if ($@) { + fail($@); + } + else { + is ($ret->[0], $first_ord_should_be, sprintf("to${function}_utf8( $display_name ) == 0x%02X", $first_ord_should_be)); + is ($ret->[1], $utf8_should_be, sprintf("utf8 of to${function}_utf8( $display_name )")); + use bytes; + is ($ret->[2], length $utf8_should_be, sprintf("number of bytes in utf8 of to${function}_uni( $display_name )")); + } + + } +} + # This is primarily to make sure that no non-Unicode warnings get generated is(scalar @warnings, 0, "No warnings were generated " . join ", ", @warnings); diff --git a/handy.h b/handy.h index 26d7366..1015479 100644 --- a/handy.h +++ b/handy.h @@ -700,19 +700,125 @@ Behaviour is only well defined when isXDIGIT(*str) is true. =head1 Character case changing -=for apidoc Am|char|toUPPER|char ch -Converts the specified character to uppercase, if possible; otherwise returns -the input character itself. Only ASCII characters are changed. Variant +=for apidoc Am|U8|toUPPER|U8 ch +Converts the specified character to uppercase. If the input is anything but an +ASCII lowercase character, that input character itself is returned. Variant C<toUPPER_A> is equivalent. -=for apidoc Am|char|toLOWER|char ch -Converts the specified character to lowercase, if possible; otherwise returns -the input character itself. Only ASCII characters are changed. Variant +=for apidoc Am|UV|toUPPER_uni|UV cp|U8* s|STRLEN* lenp +Converts the Unicode code point C<cp> to its uppercase version, and +stores that in UTF-8 in C<s>, and its length in bytes in C<lenp>. Note +that the buffer pointed to by C<s> needs to be at least C<UTF8_MAXBYTES_CASE+1> +bytes since the uppercase version may be longer than the original character. + +The first code point of the uppercased version is returned +(but note, as explained just above, that there may be more.) + +=for apidoc Am|UV|toUPPER_utf8|U8* p|U8* s|STRLEN* lenp +Converts the UTF-8 encoded character at C to its uppercase version, and +stores that in UTF-8 in C<s>, and its length in bytes in C<lenp>. Note +that the buffer pointed to by C<s> needs to be at least C<UTF8_MAXBYTES_CASE+1> +bytes since the uppercase version may be longer than the original character. + +The first code point of the uppercased version is returned +(but note, as explained just above, that there may be more.) + +The input character at C is assumed to be well-formed. + +=for apidoc Am|U8|toFOLD|U8 ch +Converts the specified character to foldcase. If the input is anything but an +ASCII uppercase character, that input character itself is returned. Variant +C<toFOLD_A> is equivalent. (There is no equivalent C<to_FOLD_L1> for the full +Latin1 range, as the full generality of L</toFOLD_uni> is needed there.) + +=for apidoc Am|UV|toFOLD_uni|UV cp|U8* s|STRLEN* lenp +Converts the Unicode code point C<cp> to its foldcase version, and +stores that in UTF-8 in C<s>, and its length in bytes in C<lenp>. Note +that the buffer pointed to by C<s> needs to be at least C<UTF8_MAXBYTES_CASE+1> +bytes since the foldcase version may be longer than the original character. + +The first code point of the foldcased version is returned +(but note, as explained just above, that there may be more.) + +=for apidoc Am|UV|toFOLD_utf8|U8* p|U8* s|STRLEN* lenp +Converts the UTF-8 encoded character at C to its foldcase version, and +stores that in UTF-8 in C<s>, and its length in bytes in C<lenp>. Note +that the buffer pointed to by C<s> needs to be at least C<UTF8_MAXBYTES_CASE+1> +bytes since the foldcase version may be longer than the original character. + +The first code point of the foldcased version is returned +(but note, as explained just above, that there may be more.) + +The input character at C is assumed to be well-formed. + +=for apidoc Am|U8|toLOWER|U8 ch +Converts the specified character to lowercase. If the input is anything but an +ASCII uppercase character, that input character itself is returned. Variant C<toLOWER_A> is equivalent. +=for apidoc Am|U8|toLOWER_L1|U8 ch +Converts the specified Latin1 character to lowercase. The results are undefined if +the input doesn't fit in a byte. + +=for apidoc Am|U8|toLOWER_LC|U8 ch +Converts the specified character to lowercase using the current locale's rules, +if possible; otherwise returns the input character itself. + +=for apidoc Am|UV|toLOWER_uni|UV cp|U8* s|STRLEN* lenp +Converts the Unicode code point C<cp> to its lowercase version, and +stores that in UTF-8 in C<s>, and its length in bytes in C<lenp>. Note +that the buffer pointed to by C<s> needs to be at least C<UTF8_MAXBYTES_CASE+1> +bytes since the lowercase version may be longer than the original character. + +The first code point of the lowercased version is returned +(but note, as explained just above, that there may be more.) + +=for apidoc Am|UV|toLOWER_utf8|U8* p|U8* s|STRLEN* lenp +Converts the UTF-8 encoded character at C to its lowercase version, and +stores that in UTF-8 in C<s>, and its length in bytes in C<lenp>. Note +that the buffer pointed to by C<s> needs to be at least C<UTF8_MAXBYTES_CASE+1> +bytes since the lowercase version may be longer than the original character. + +The first code point of the lowercased version is returned +(but note, as explained just above, that there may be more.) + +The input character at C is assumed to be well-formed. + +=for apidoc Am|U8|toLOWER_LC|U8 ch +Converts the specified character to lowercase using the current locale's rules, +if possible; otherwise returns the input character itself. + +=for apidoc Am|U8|toTITLE|U8 ch +Converts the specified character to titlecase. If the input is anything but an +ASCII lowercase character, that input character itself is returned. Variant +C<toTITLE_A> is equivalent. (There is no C<toTITLE_L1> for the full Latin1 range, +as the full generality of L</toTITLE_uni> is needed there. Titlecase is not a +concept used in locale handling, so there is no functionality for that.) + +=for apidoc Am|UV|toTITLE_uni|UV cp|U8* s|STRLEN* lenp +Converts the Unicode code point C<cp> to its titlecase version, and +stores that in UTF-8 in C<s>, and its length in bytes in C<lenp>. Note +that the buffer pointed to by C<s> needs to be at least C<UTF8_MAXBYTES_CASE+1> +bytes since the titlecase version may be longer than the original character. + +The first code point of the titlecased version is returned +(but note, as explained just above, that there may be more.) + +=for apidoc Am|UV|toTITLE_utf8|U8* p|U8* s|STRLEN* lenp +Converts the UTF-8 encoded character at C to its titlecase version, and +stores that in UTF-8 in C<s>, and its length in bytes in C<lenp>. Note +that the buffer pointed to by C<s> needs to be at least C<UTF8_MAXBYTES_CASE+1> +bytes since the titlecase version may be longer than the original character. + +The first code point of the titlecased version is returned +(but note, as explained just above, that there may be more.) + +The input character at C is assumed to be well-formed. + =cut -XXX Still undocumented isVERTWS_uni and _utf8, and the other toUPPER etc functions +XXX Still undocumented isVERTWS_uni and _utf8; it's unclear what their names +really should be. Also toUPPER_LC and toFOLD_LC, which are subject to change. Note that these macros are repeated in Devel::PPPort, so should also be patched there. The file as of this writing is cpan/Devel-PPPort/parts/inc/misc @@ -1048,24 +1154,33 @@ EXTCONST U32 PL_charclass[]; # define toUPPER(c) (isLOWER(c) ? (c) - ('a' - 'A') : (c)) #endif +/* In the ASCII range, these are equivalent to what they're here defined to be. + * But by creating these definitions, other code doesn't have to be aware of + * this detail */ +#define toFOLD(c) toLOWER(c) +#define toFOLD_LC(c) toLOWER_LC(c) +#define toTITLE(c) toUPPER(c) + #define toLOWER_A(c) toLOWER(c) #define toUPPER_A(c) toUPPER(c) +#define toFOLD_A(c) toFOLD(c) +#define toTITLE_A(c) toTITLE(c) /* Use table lookup for speed; return error character for input * out-of-range */ -#define toLOWER_LATIN1(c) (FITS_IN_8_BITS(c) \ - ? UNI_TO_NATIVE(PL_latin1_lc[ \ - NATIVE_TO_UNI( (U8) (c)) ]) \ - : UNICODE_REPLACEMENT) +#define toLOWER_LATIN1(c) ((! FITS_IN_8_BITS(c)) \ + ? (c) \ + : UNI_TO_NATIVE(PL_latin1_lc[ \ + NATIVE_TO_UNI( (U8) (c)) ])) #define toLOWER_L1(c) toLOWER_LATIN1(c) /* Synonym for consistency */ /* Modified uc. Is correct uc except for three non-ascii chars which are * all mapped to one of them, and these need special handling; error * character for input out-of-range */ -#define toUPPER_LATIN1_MOD(c) (FITS_IN_8_BITS(c) \ - ? UNI_TO_NATIVE(PL_mod_latin1_uc[ \ - NATIVE_TO_UNI( (U8) (c)) ]) \ - : UNICODE_REPLACEMENT) +#define toUPPER_LATIN1_MOD(c) ((! FITS_IN_8_BITS(c)) \ + ? (c) \ + : UNI_TO_NATIVE(PL_mod_latin1_uc[ \ + NATIVE_TO_UNI( (U8) (c)) ])) #ifdef USE_NEXT_CTYPE diff --git a/lib/charnames.t b/lib/charnames.t index c58ccbe..bf413a6 100644 --- a/lib/charnames.t +++ b/lib/charnames.t @@ -1086,8 +1086,8 @@ is("\N{U+1D0C5}", "\N{BYZANTINE MUSICAL SYMBOL FTHORA SKLIRON CHROMA VASIS}", 'V my @name_aliases; use Unicode::UCD; - if (ord('A') != 65 - || pack( "C*", split /\./, Unicode::UCD::UnicodeVersion()) ge v6.1.0) + if (ord('A') == 65 + && pack( "C*", split /\./, Unicode::UCD::UnicodeVersion()) ge v6.1.0) { open my $fh, "<", "../../lib/unicore/NameAliases.txt" or die "Can't open ../../lib/unicore/NameAliases.txt: $!"; diff --git a/lib/locale.t b/lib/locale.t index a9a5a26..5806138 100644 --- a/lib/locale.t +++ b/lib/locale.t @@ -1517,7 +1517,7 @@ setlocale(LC_ALL, "C"); $above_latin1_case_change_delta = -1; } else { - @list = ("", "A", "\xC0", "\x{1E9E}", "\x{100}"); + @list = ("", "A", "\xC0", "\x{17F}", "\x{100}"); $ascii_case_change_delta = +32; $above_latin1_case_change_delta = +1; } diff --git a/mg.c b/mg.c index 6b8984e..5f0f758 100644 --- a/mg.c +++ b/mg.c @@ -286,9 +286,7 @@ Perl_mg_set(pTHX_ SV *sv) /* =for apidoc mg_length -This function is deprecated. - -It reports on the SV's length in bytes, calling length magic if available, +Reports on the SV's length in bytes, calling length magic if available, but does not set the UTF8 flag on the sv. It will fall back to 'get' magic if there is no 'length' magic, but with no indication as to whether it called 'get' magic. It assumes the sv is a PVMG or diff --git a/op.c b/op.c index 8457869..3bdbdf6 100644 --- a/op.c +++ b/op.c @@ -3229,6 +3229,7 @@ S_fold_constants(pTHX_ OP *o) case OP_LCFIRST: case OP_UC: case OP_LC: + case OP_FC: case OP_SLT: case OP_SGT: case OP_SLE: diff --git a/pod/perlclib.pod b/pod/perlclib.pod index ef0b6b0..0cdee24 100644 --- a/pod/perlclib.pod +++ b/pod/perlclib.pod @@ -48,73 +48,76 @@ types. Don't forget that with the new PerlIO layered I/O abstraction C<FILE*> types may not even be available. See also the C<perlapio> documentation for more information about the following functions: - Instead Of: Use: + Instead Of: Use: - stdin PerlIO_stdin() - stdout PerlIO_stdout() - stderr PerlIO_stderr() + stdin PerlIO_stdin() + stdout PerlIO_stdout() + stderr PerlIO_stderr() - fopen(fn, mode) PerlIO_open(fn, mode) - freopen(fn, mode, stream) PerlIO_reopen(fn, mode, perlio) (Deprecated) - fflush(stream) PerlIO_flush(perlio) - fclose(stream) PerlIO_close(perlio) + fopen(fn, mode) PerlIO_open(fn, mode) + freopen(fn, mode, stream) PerlIO_reopen(fn, mode, perlio) (Dep- + recated) + fflush(stream) PerlIO_flush(perlio) + fclose(stream) PerlIO_close(perlio) =head2 File Input and Output - Instead Of: Use: + Instead Of: Use: - fprintf(stream, fmt, ...) PerlIO_printf(perlio, fmt, ...) + fprintf(stream, fmt, ...) PerlIO_printf(perlio, fmt, ...) - [f]getc(stream) PerlIO_getc(perlio) - [f]putc(stream, n) PerlIO_putc(perlio, n) - ungetc(n, stream) PerlIO_ungetc(perlio, n) + [f]getc(stream) PerlIO_getc(perlio) + [f]putc(stream, n) PerlIO_putc(perlio, n) + ungetc(n, stream) PerlIO_ungetc(perlio, n) Note that the PerlIO equivalents of C<fread> and C<fwrite> are slightly different from their C library counterparts: - fread(p, size, n, stream) PerlIO_read(perlio, buf, numbytes) - fwrite(p, size, n, stream) PerlIO_write(perlio, buf, numbytes) + fread(p, size, n, stream) PerlIO_read(perlio, buf, numbytes) + fwrite(p, size, n, stream) PerlIO_write(perlio, buf, numbytes) - fputs(s, stream) PerlIO_puts(perlio, s) + fputs(s, stream) PerlIO_puts(perlio, s) There is no equivalent to C<fgets>; one should use C<sv_gets> instead: - fgets(s, n, stream) sv_gets(sv, perlio, append) + fgets(s, n, stream) sv_gets(sv, perlio, append) =head2 File Positioning - Instead Of: Use: + Instead Of: Use: - feof(stream) PerlIO_eof(perlio) - fseek(stream, n, whence) PerlIO_seek(perlio, n, whence) - rewind(stream) PerlIO_rewind(perlio) + feof(stream) PerlIO_eof(perlio) + fseek(stream, n, whence) PerlIO_seek(perlio, n, whence) + rewind(stream) PerlIO_rewind(perlio) - fgetpos(stream, p) PerlIO_getpos(perlio, sv) - fsetpos(stream, p) PerlIO_setpos(perlio, sv) + fgetpos(stream, p) PerlIO_getpos(perlio, sv) + fsetpos(stream, p) PerlIO_setpos(perlio, sv) - ferror(stream) PerlIO_error(perlio) - clearerr(stream) PerlIO_clearerr(perlio) + ferror(stream) PerlIO_error(perlio) + clearerr(stream) PerlIO_clearerr(perlio) =head2 Memory Management and String Handling - Instead Of: Use: + Instead Of: Use: - t* p = malloc(n) Newx(p, n, t) - t* p = calloc(n, s) Newxz(p, n, t) - p = realloc(p, n) Renew(p, n, t) - memcpy(dst, src, n) Copy(src, dst, n, t) - memmove(dst, src, n) Move(src, dst, n, t) - memcpy(dst, src, sizeof(t)) StructCopy(src, dst, t) - memset(dst, 0, n * sizeof(t)) Zero(dst, n, t) - memzero(dst, 0) Zero(dst, n, char) - free(p) Safefree(p) + t* p = malloc(n) Newx(p, n, t) + t* p = calloc(n, s) Newxz(p, n, t) + p = realloc(p, n) Renew(p, n, t) + memcpy(dst, src, n) Copy(src, dst, n, t) + memmove(dst, src, n) Move(src, dst, n, t) + memcpy(dst, src, sizeof(t)) StructCopy(src, dst, t) + memset(dst, 0, n * sizeof(t)) Zero(dst, n, t) + memzero(dst, 0) Zero(dst, n, char) + free(p) Safefree(p) - strdup(p) savepv(p) - strndup(p, n) savepvn(p, n) (Hey, strndup doesn't exist!) + strdup(p) savepv(p) + strndup(p, n) savepvn(p, n) (Hey, strndup doesn't + exist!) - strstr(big, little) instr(big, little) - strcmp(s1, s2) strLE(s1, s2) / strEQ(s1, s2) / strGT(s1,s2) - strncmp(s1, s2, n) strnNE(s1, s2, n) / strnEQ(s1, s2, n) + strstr(big, little) instr(big, little) + strcmp(s1, s2) strLE(s1, s2) / strEQ(s1, s2) + / strGT(s1,s2) + strncmp(s1, s2, n) strnNE(s1, s2, n) / strnEQ(s1, s2, n) Notice the different order of arguments to C<Copy> and C<Move> than used in C<memcpy> and C<memmove>. @@ -122,12 +125,12 @@ in C<memcpy> and C<memmove>. Most of the time, though, you'll want to be dealing with SVs internally instead of raw C<char *> strings: - strlen(s) sv_len(sv) - strcpy(dt, src) sv_setpv(sv, s) - strncpy(dt, src, n) sv_setpvn(sv, s, n) - strcat(dt, src) sv_catpv(sv, s) - strncat(dt, src) sv_catpvn(sv, s) - sprintf(s, fmt, ...) sv_setpvf(sv, fmt, ...) + strlen(s) sv_len(sv) + strcpy(dt, src) sv_setpv(sv, s) + strncpy(dt, src, n) sv_setpvn(sv, s, n) + strcat(dt, src) sv_catpv(sv, s) + strncat(dt, src) sv_catpvn(sv, s) + sprintf(s, fmt, ...) sv_setpvf(sv, fmt, ...) Note also the existence of C<sv_catpvf> and C<sv_vcatpvfn>, combining concatenation with formatting. @@ -140,45 +143,67 @@ any code attempting to use the data without forethought will break sooner rather than later. Poisoning can be done using the Poison() macros, which have similar arguments to Zero(): - PoisonWith(dst, n, t, b) scribble memory with byte b - PoisonNew(dst, n, t) equal to PoisonWith(dst, n, t, 0xAB) - PoisonFree(dst, n, t) equal to PoisonWith(dst, n, t, 0xEF) - Poison(dst, n, t) equal to PoisonFree(dst, n, t) + PoisonWith(dst, n, t, b) scribble memory with byte b + PoisonNew(dst, n, t) equal to PoisonWith(dst, n, t, 0xAB) + PoisonFree(dst, n, t) equal to PoisonWith(dst, n, t, 0xEF) + Poison(dst, n, t) equal to PoisonFree(dst, n, t) =head2 Character Class Tests -There are two types of character class tests that Perl implements: one -type deals in C<char>s and are thus B<not> Unicode aware (and hence -deprecated unless you B<know> you should use them) and the other type -deal in C<UV>s and know about Unicode properties. In the following -table, C<c> is a C<char>, and C is a Unicode codepoint. - - Instead Of: Use: But better use: - - isalnum(c) isALNUM(c) isALNUM_uni(u) - isalpha(c) isALPHA(c) isALPHA_uni(u) - iscntrl(c) isCNTRL(c) isCNTRL_uni(u) - isdigit(c) isDIGIT(c) isDIGIT_uni(u) - isgraph(c) isGRAPH(c) isGRAPH_uni(u) - islower(c) isLOWER(c) isLOWER_uni(u) - isprint(c) isPRINT(c) isPRINT_uni(u) - ispunct(c) isPUNCT(c) isPUNCT_uni(u) - isspace(c) isSPACE(c) isSPACE_uni(u) - isupper(c) isUPPER(c) isUPPER_uni(u) - isxdigit(c) isXDIGIT(c) isXDIGIT_uni(u) - - tolower(c) toLOWER(c) toLOWER_uni(u) - toupper(c) toUPPER(c) toUPPER_uni(u) +There are several types of character class tests that Perl implements. +The only ones described here are those that directly correspond to C +library functions that operate on 8-bit characters, but there are +equivalents that operate on wide characters, and UTF-8 encoded strings. +All are more fully described in L<perlapi/Character classes> and +L<perlapi/Character case changing>. + +The C library routines listed in the table below return values based on +the current locale. Use the entries in the final column for that +functionality. The other two columns always assume a POSIX (or C) +locale. The entries in the ASCII column are only meaningful for ASCII +inputs, returning FALSE for anything else. Use these only when you +B<know> that is what you want. The entries in the Latin1 column assume +that the non-ASCII 8-bit characters are as Unicode defines, them, the +same as ISO-8859-1, often called Latin 1. + + Instead Of: Use for ASCII: Use for Latin1: Use for locale: + + isalnum(c) isALPHANUMERIC(c) isALPHANUMERIC_L1(c) isALPHANUMERIC_LC(c) + isalpha(c) isALPHA(c) isALPHA_L1(c) isALPHA_LC(u ) + isascii(c) isASCII(c) isASCII_LC(c) + isblank(c) isBLANK(c) isBLANK_L1(c) isBLANK_LC(c) + iscntrl(c) isCNTRL(c) isCNTRL_L1(c) isCNTRL_LC(c) + isdigit(c) isDIGIT(c) isDIGIT_L1(c) isDIGIT_LC(c) + isgraph(c) isGRAPH(c) isGRAPH_L1(c) isGRAPH_LC(c) + islower(c) isLOWER(c) isLOWER_L1(c) isLOWER_LC(c) + isprint(c) isPRINT(c) isPRINT_L1(c) isPRINT_LC(c) + ispunct(c) isPUNCT(c) isPUNCT_L1(c) isPUNCT_LC(c) + isspace(c) isSPACE(c) isSPACE_L1(c) isSPACE_LC(c) + isupper(c) isUPPER(c) isUPPER_L1(c) isUPPER_LC(c) + isxdigit(c) isXDIGIT(c) isXDIGIT_L1(c) isXDIGIT_LC(c) + + tolower(c) toLOWER(c) toLOWER_L1(c) toLOWER_LC(c) + toupper(c) toUPPER(c) toUPPER_LC(c) + +To emphasize that you are operating only on ASCII characters, you can +append C<_A> to each of the macros in the ASCII column: C<isALPHA_A>, +C<isDIGIT_A>, and so on. + +(There is no entry in the Latin1 column for C<isascii> even though there +is an C<isASCII_L1>, which is identical to C<isASCII>; the +latter name is clearer. There is no entry in the Latin1 column for +C<toupper> because the result can be non-Latin1. You have to use +C<toUPPER_uni>, as described in L<perlapi/Character case changing>.) =head2 F<stdlib.h> functions - Instead Of: Use: + Instead Of: Use: - atof(s) Atof(s) - atol(s) Atol(s) - strtod(s, &p) Nothing. Just don't use it. - strtol(s, &p, n) Strtol(s, &p, n) - strtoul(s, &p, n) Strtoul(s, &p, n) + atof(s) Atof(s) + atol(s) Atol(s) + strtod(s, &p) Nothing. Just don't use it. + strtol(s, &p, n) Strtol(s, &p, n) + strtoul(s, &p, n) Strtoul(s, &p, n) Notice also the C<grok_bin>, C<grok_hex>, and C<grok_oct> functions in F<numeric.c> for converting strings representing numbers in the respective @@ -189,15 +214,15 @@ built on doesn't actually have strtol and strtoul. But as those 2 functions are part of the 1989 ANSI C spec we suspect you'll find them everywhere by now. - int rand() double Drand01() - srand(n) { seedDrand01((Rand_seed_t)n); - PL_srand_called = TRUE; } + int rand() double Drand01() + srand(n) { seedDrand01((Rand_seed_t)n); + PL_srand_called = TRUE; } - exit(n) my_exit(n) - system(s) Don't. Look at pp_system or use my_popen + exit(n) my_exit(n) + system(s) Don't. Look at pp_system or use my_popen - getenv(s) PerlEnv_getenv(s) - setenv(s, val) my_putenv(s, val) + getenv(s) PerlEnv_getenv(s) + setenv(s, val) my_putenv(s, val) =head2 Miscellaneous functions diff --git a/pod/perlfunc.pod b/pod/perlfunc.pod index 676644f..08b9df9 100644 --- a/pod/perlfunc.pod +++ b/pod/perlfunc.pod @@ -2174,8 +2174,16 @@ Case Charts available at L<http://www.unicode.org/charts/case/>. If EXPR is omitted, uses C<$_>. -This function behaves the same way under various pragma, such as in a locale, -as L</lc> does. +This function behaves the same way under various pragma, such as within +S<C<"use feature 'unicode_strings">>, as L</lc> does, with the single +exception of C<fc> of LATIN CAPITAL LETTER SHARP S (U+1E9E) within the +scope of S<C<use locale>>. The foldcase of this character would +normally be C<"ss">, but as explained in the L</lc> section, case +changes that cross the 255/256 boundary are problematic under locales, +and are hence prohibited. Therefore, this function under locale returns +instead the string C<"\x{17F}\x{17F}">, which is the LATIN SMALL LETTER +LONG S. Since that character itself folds to C<"s">, the string of two +of them together should be equivalent to a single U+1E9E when foldcased. While the Unicode Standard defines two additional forms of casefolding, one for Turkic languages and one that never maps one character into multiple diff --git a/pod/perllocale.pod b/pod/perllocale.pod index ad8e64b..8d5ba72 100644 --- a/pod/perllocale.pod +++ b/pod/perllocale.pod @@ -666,9 +666,9 @@ setting, characters like "E<aelig>", "E<eth>", "E<szlig>", and The C<LC_CTYPE> locale also provides the map used in transliterating characters between lower and uppercase. This affects the case-mapping -functions--lc(), lcfirst, uc(), and ucfirst(); case-mapping -interpolation with C<\l>, C<\L>, C<\u>, or C<\U> in double-quoted strings -and C<s///> substitutions; and case-independent regular expression +functions--fc(), lc(), lcfirst(), uc(), and ucfirst(); case-mapping +interpolation with C<\F>, C<\l>, C<\L>, C<\u>, or C<\U> in double-quoted +strings and C<s///> substitutions; and case-independent regular expression pattern matching using the C modifier. Finally, C<LC_CTYPE> affects the POSIX character-class test diff --git a/pp.c b/pp.c index ed6fd5f..48710cc 100644 --- a/pp.c +++ b/pp.c @@ -47,6 +47,9 @@ extern Pid_t getpid (void); _LIB_VERSION_TYPE _LIB_VERSION = _IEEE_; #endif +static const STRLEN small_mu_len = sizeof(GREEK_SMALL_LETTER_MU_UTF8) - 1; +static const STRLEN capital_iota_len = sizeof(GREEK_CAPITAL_LETTER_IOTA_UTF8) - 1; + /* variations on pp_null */ PP(pp_stub) @@ -3430,15 +3433,6 @@ PP(pp_crypt) /* Generally UTF-8 and UTF-EBCDIC are indistinguishable at this level. So * most comments below say UTF-8, when in fact they mean UTF-EBCDIC as well */ -/* Generates code to store a unicode codepoint c that is known to occupy - * exactly two UTF-8 and UTF-EBCDIC bytes; it is stored into p and p+1, - * and p is advanced to point to the next available byte after the two bytes */ -#define CAT_UNI_TO_UTF8_TWO_BYTE(p, c) \ - STMT_START { \ - *(p)++ = UTF8_TWO_BYTE_HI(c); \ - *((p)++) = UTF8_TWO_BYTE_LO(c); \ - } STMT_END - PP(pp_ucfirst) { /* Actually is both lcfirst() and ucfirst(). Only the first character @@ -3762,10 +3756,8 @@ PP(pp_uc) if (in_iota_subscript && ! _is_utf8_mark(s)) { /* A non-mark. Time to output the iota subscript */ -#define GREEK_CAPITAL_LETTER_IOTA 0x0399 -#define COMBINING_GREEK_YPOGEGRAMMENI 0x0345 - - CAT_UNI_TO_UTF8_TWO_BYTE(d, GREEK_CAPITAL_LETTER_IOTA); + Copy(GREEK_CAPITAL_LETTER_IOTA_UTF8, d, capital_iota_len, U8); + d += capital_iota_len; in_iota_subscript = FALSE; } @@ -3775,6 +3767,8 @@ PP(pp_uc) u = UTF8SKIP(s); uv = _to_utf8_upper_flags(s, tmpbuf, &ulen, cBOOL(IN_LOCALE_RUNTIME), &tainted); +#define GREEK_CAPITAL_LETTER_IOTA 0x0399 +#define COMBINING_GREEK_YPOGEGRAMMENI 0x0345 if (uv == GREEK_CAPITAL_LETTER_IOTA && utf8_to_uvchr_buf(s, send, 0) == COMBINING_GREEK_YPOGEGRAMMENI) { @@ -3800,7 +3794,8 @@ PP(pp_uc) s += u; } if (in_iota_subscript) { - CAT_UNI_TO_UTF8_TWO_BYTE(d, GREEK_CAPITAL_LETTER_IOTA); + Copy(GREEK_CAPITAL_LETTER_IOTA_UTF8, d, capital_iota_len, U8); + d += capital_iota_len; } SvUTF8_on(dest); *d = '\0'; @@ -4198,23 +4193,20 @@ PP(pp_fc) } } /* Unflagged string */ else if (len) { - /* For locale, bytes, and nothing, the behavior is supposed to be the - * same as lc(). - */ if ( IN_LOCALE_RUNTIME ) { /* Under locale */ TAINT; SvTAINTED_on(dest); for (; s < send; d++, s++) - *d = toLOWER_LC(*s); + *d = toFOLD_LC(*s); } else if ( !IN_UNI_8_BIT ) { /* Under nothing, or bytes */ for (; s < send; d++, s++) - *d = toLOWER(*s); + *d = toFOLD(*s); } else { /* For ASCII and the Latin-1 range, there's only two troublesome * folds, \x{DF} (\N{LATIN SMALL LETTER SHARP S}), which under full - * casefolding becomes 'ss', and \x{B5} (\N{MICRO SIGN}), which + * casefolding becomes 'ss'; and \x{B5} (\N{MICRO SIGN}), which * under any fold becomes \x{3BC} (\N{GREEK SMALL LETTER MU}) -- * For the rest, the casefold is their lowercase. */ for (; s < send; d++, s++) { @@ -4234,7 +4226,8 @@ PP(pp_fc) (send -s) * 2 + 1); d = (U8*)SvPVX(dest) + len; - CAT_UNI_TO_UTF8_TWO_BYTE(d, GREEK_SMALL_LETTER_MU); + Copy(GREEK_SMALL_LETTER_MU_UTF8, d, small_mu_len, U8); + d += small_mu_len; s++; for (; s < send; s++) { STRLEN ulen; diff --git a/proto.h b/proto.h index c2fe6f3..a1b3e97 100644 --- a/proto.h +++ b/proto.h @@ -7457,7 +7457,7 @@ PERL_CALLCONV UV Perl__to_upper_title_latin1(pTHX_ const U8 c, U8 *p, STRLEN *le #endif #if defined(PERL_IN_UTF8_C) || defined(PERL_IN_REGCOMP_C) || defined(PERL_IN_REGEXEC_C) -PERL_CALLCONV UV Perl__to_fold_latin1(pTHX_ const U8 c, U8 *p, STRLEN *lenp, const bool flags) +PERL_CALLCONV UV Perl__to_fold_latin1(pTHX_ const U8 c, U8 *p, STRLEN *lenp, const unsigned int flags) __attribute__nonnull__(pTHX_2) __attribute__nonnull__(pTHX_3); #define PERL_ARGS_ASSERT__TO_FOLD_LATIN1 \ diff --git a/regcharclass.h b/regcharclass.h index e51fe64..a0bd93d 100644 --- a/regcharclass.h +++ b/regcharclass.h @@ -643,6 +643,8 @@ ( ( ( 0xCC == ((U8*)s)[1] ) && ( 0x88 == ((U8*)s)[2] ) ) ? 3 : 0 ) \ : ( 0x77 == ((U8*)s)[0] || 0x79 == ((U8*)s)[0] ) ? \ ( ( ( 0xCC == ((U8*)s)[1] ) && ( 0x8A == ((U8*)s)[2] ) ) ? 3 : 0 ) \ + : ( 0xC5 == ((U8*)s)[0] ) ? \ + ( ( ( ( 0xBF == ((U8*)s)[1] ) && ( 0xC5 == ((U8*)s)[2] ) ) && ( 0xBF == ((U8*)s)[3] ) ) ? 4 : 0 )\ : ( 0xCA == ((U8*)s)[0] ) ? \ ( ( ( 0xBC == ((U8*)s)[1] ) && ( 0x6E == ((U8*)s)[2] ) ) ? 3 : 0 ) \ : ( 0xCE == ((U8*)s)[0] ) ? \ @@ -699,6 +701,8 @@ ( ( ( 0xCC == ((U8*)s)[1] ) && ( 0x88 == ((U8*)s)[2] ) ) ? 3 : 0 ) \ : ( 0x77 == ((U8*)s)[0] || 0x79 == ((U8*)s)[0] ) ? \ ( ( ( 0xCC == ((U8*)s)[1] ) && ( 0x8A == ((U8*)s)[2] ) ) ? 3 : 0 ) \ + : ( 0xC5 == ((U8*)s)[0] ) ? \ + ( ( ( ( 0xBF == ((U8*)s)[1] ) && ( 0xC5 == ((U8*)s)[2] ) ) && ( 0xBF == ((U8*)s)[3] ) ) ? 4 : 0 )\ : ( 0xCA == ((U8*)s)[0] ) ? \ ( ( ( 0xBC == ((U8*)s)[1] ) && ( 0x6E == ((U8*)s)[2] ) ) ? 3 : 0 ) \ : ( 0xCE == ((U8*)s)[0] ) ? \ @@ -776,6 +780,8 @@ ( ( ( 0xCC == ((U8*)s)[1] ) && ( 0x88 == ((U8*)s)[2] ) ) ? 3 : 0 ) \ : ( 0x77 == ((U8*)s)[0] || 0x79 == ((U8*)s)[0] ) ? \ ( ( ( 0xCC == ((U8*)s)[1] ) && ( 0x8A == ((U8*)s)[2] ) ) ? 3 : 0 ) \ + : ( 0xC5 == ((U8*)s)[0] ) ? \ + ( ( ( ( 0xBF == ((U8*)s)[1] ) && ( 0xC5 == ((U8*)s)[2] ) ) && ( 0xBF == ((U8*)s)[3] ) ) ? 4 : 0 )\ : ( 0xCA == ((U8*)s)[0] ) ? \ ( ( ( 0xBC == ((U8*)s)[1] ) && ( 0x6E == ((U8*)s)[2] ) ) ? 3 : 0 ) \ : ( 0xCE == ((U8*)s)[0] ) ? \ diff --git a/regcomp.c b/regcomp.c index 3e20db1..05d15a4 100644 --- a/regcomp.c +++ b/regcomp.c @@ -1456,7 +1456,7 @@ is the recommended Unicode-aware way of saying len = 0; \ } else { \ len = 1; \ - uvc = _to_fold_latin1( (U8) *uc, foldbuf, &foldlen, 1); \ + uvc = _to_fold_latin1( (U8) *uc, foldbuf, &foldlen, FOLD_FLAGS_FULL); \ skiplen = UNISKIP(uvc); \ foldlen -= skiplen; \ scan = foldbuf + skiplen; \ @@ -2686,29 +2686,37 @@ S_make_trie_failtable(pTHX_ RExC_state_t *pRExC_state, regnode *source, regnode * this file makes sure that in EXACTFU nodes, the sharp s gets folded to * 'ss', even if the pattern isn't UTF-8. This avoids the issues * described in the next item. - * 4) A problem remains for the sharp s in EXACTF nodes. Whether it matches - * 'ss' or not is not knowable at compile time. It will match iff the - * target string is in UTF-8, unlike the EXACTFU nodes, where it always - * matches; and the EXACTFL and EXACTFA nodes where it never does. Thus - * it can't be folded to "ss" at compile time, unlike EXACTFU does (as - * described in item 3). An assumption that the optimizer part of - * regexec.c (probably unwittingly) makes is that a character in the - * pattern corresponds to at most a single character in the target string. - * (And I do mean character, and not byte here, unlike other parts of the - * documentation that have never been updated to account for multibyte - * Unicode.) This assumption is wrong only in this case, as all other - * cases are either 1-1 folds when no UTF-8 is involved; or is true by - * virtue of having this file pre-fold UTF-8 patterns. I'm - * reluctant to try to change this assumption, so instead the code punts. - * This routine examines EXACTF nodes for the sharp s, and returns a - * boolean indicating whether or not the node is an EXACTF node that - * contains a sharp s. When it is true, the caller sets a flag that later - * causes the optimizer in this file to not set values for the floating - * and fixed string lengths, and thus avoids the optimizer code in - * regexec.c that makes the invalid assumption. Thus, there is no - * optimization based on string lengths for EXACTF nodes that contain the - * sharp s. This only happens for /id rules (which means the pattern - * isn't in UTF-8). + * 4) A problem remains for the sharp s in EXACTF and EXACTFA nodes when the + * pattern isn't in UTF-8. (BTW, there cannot be an EXACTF node with a + * UTF-8 pattern.) An assumption that the optimizer part of regexec.c + * (probably unwittingly, in Perl_regexec_flags()) makes is that a + * character in the pattern corresponds to at most a single character in + * the target string. (And I do mean character, and not byte here, unlike + * other parts of the documentation that have never been updated to + * account for multibyte Unicode.) sharp s in EXACTF nodes can match the + * two character string 'ss'; in EXACTFA nodes it can match + * "\x{17F}\x{17F}". These violate the assumption, and they are the only + * instances where it is violated. I'm reluctant to try to change the + * assumption, as the code involved is impenetrable to me (khw), so + * instead the code here punts. This routine examines (when the pattern + * isn't UTF-8) EXACTF and EXACTFA nodes for the sharp s, and returns a + * boolean indicating whether or not the node contains a sharp s. When it + * is true, the caller sets a flag that later causes the optimizer in this + * file to not set values for the floating and fixed string lengths, and + * thus avoids the optimizer code in regexec.c that makes the invalid + * assumption. Thus, there is no optimization based on string lengths for + * non-UTF8-pattern EXACTF and EXACTFA nodes that contain the sharp s. + * (The reason the assumption is wrong only in these two cases is that all + * other non-UTF-8 folds are 1-1; and, for UTF-8 patterns, we pre-fold all + * other folds to their expanded versions. We can't prefold sharp s to + * 'ss' in EXACTF nodes because we don't know at compile time if it + * actually matches 'ss' or not. It will match iff the target string is + * in UTF-8, unlike the EXACTFU nodes, where it always matches; and + * EXACTFA and EXACTFL where it never does. In an EXACTFA node in a UTF-8 + * pattern, sharp s is folded to "\x{17F}\x{17F}, avoiding the problem; + * but in a non-UTF8 pattern, folding it to that above-Latin1 string would + * require the pattern to be forced into UTF-8, the overhead of which we + * want to avoid.) */ #define JOIN_EXACT(scan,min_subtract,has_exactf_sharp_s, flags) \ @@ -2899,13 +2907,30 @@ S_join_exact(pTHX_ RExC_state_t *pRExC_state, regnode *scan, UV *min_subtract, b next_iteration: ; } } - else if (OP(scan) != EXACTFL && OP(scan) != EXACTFA) { + else if (OP(scan) == EXACTFA) { - /* Here, the pattern is not UTF-8. Look for the multi-char folds - * that are all ASCII. As in the above case, EXACTFL and EXACTFA - * nodes can't have multi-char folds to this range (and there are - * no existing ones in the upper latin1 range). In the EXACTF - * case we look also for the sharp s, which can be in the final + /* Non-UTF-8 pattern, EXACTFA node. There can't be a multi-char + * fold to the ASCII range (and there are no existing ones in the + * upper latin1 range). But, as outlined in the comments preceding + * this function, we need to flag any occurrences of the sharp s */ + while (s < s_end) { + if (*s == LATIN_SMALL_LETTER_SHARP_S) { + *has_exactf_sharp_s = TRUE; + break; + } + s++; + continue; + } + } + else if (OP(scan) != EXACTFL) { + + /* Non-UTF-8 pattern, not EXACTFA nor EXACTFL node. Look for the + * multi-char folds that are all Latin1. (This code knows that + * there are no current multi-char folds possible with EXACTFL, + * relying on fold_grind.t to catch any errors if the very unlikely + * event happens that some get added in future Unicode versions.) + * As explained in the comments preceding this function, we look + * also for the sharp s in EXACTF nodes; it can be in the final * position. Otherwise we can stop looking 1 byte earlier because * have to find at least two characters for a multi-fold */ const U8* upper = (OP(scan) == EXACTF) ? s_end : s_end -1; @@ -11029,98 +11054,100 @@ tryagain: goto loopdone; } - if (FOLD) { - if (UTF - /* See comments for join_exact() as to why we fold - * this non-UTF at compile time */ - || (node_type == EXACTFU - && ender == LATIN_SMALL_LETTER_SHARP_S)) - { - - - /* Prime the casefolded buffer. Locale rules, which - * apply only to code points < 256, aren't known until - * execution, so for them, just output the original - * character using utf8. If we start to fold non-UTF - * patterns, be sure to update join_exact() */ - if (LOC && ender < 256) { - if (UNI_IS_INVARIANT(ender)) { - *s = (U8) ender; - foldlen = 1; - } else { - *s = UTF8_TWO_BYTE_HI(ender); - *(s + 1) = UTF8_TWO_BYTE_LO(ender); - foldlen = 2; - } + if (! FOLD) { + if (UTF) { + const STRLEN unilen = reguni(pRExC_state, ender, s); + if (unilen > 0) { + s += unilen; + len += unilen; } - else { - UV folded = _to_uni_fold_flags( - ender, - (U8 *) s, - &foldlen, - FOLD_FLAGS_FULL - | ((LOC) ? FOLD_FLAGS_LOCALE - : (ASCII_FOLD_RESTRICTED) - ? FOLD_FLAGS_NOMIX_ASCII - : 0) - ); - /* If this node only contains non-folding code - * points so far, see if this new one is also - * non-folding */ - if (maybe_exact) { - if (folded != ender) { - maybe_exact = FALSE; + /* The loop increments <len> each time, as all but this + * path (and one other) through it add a single byte to + * the EXACTish node. But this one has changed len to + * be the correct final value, so subtract one to + * cancel out the increment that follows */ + len--; + } + else { + REGC((char)ender, s++); + } + } + else /* FOLD */ + if (! ( UTF + /* See comments for join_exact() as to why we fold this + * non-UTF at compile time */ + || (node_type == EXACTFU + && ender == LATIN_SMALL_LETTER_SHARP_S))) + { + *(s++) = (char) ender; + maybe_exact &= ! IS_IN_SOME_FOLD_L1(ender); + } + else { /* UTF */ + + /* Prime the casefolded buffer. Locale rules, which apply + * only to code points < 256, aren't known until execution, + * so for them, just output the original character using + * utf8. If we start to fold non-UTF patterns, be sure to + * update join_exact() */ + if (LOC && ender < 256) { + if (UNI_IS_INVARIANT(ender)) { + *s = (U8) ender; + foldlen = 1; + } else { + *s = UTF8_TWO_BYTE_HI(ender); + *(s + 1) = UTF8_TWO_BYTE_LO(ender); + foldlen = 2; + } + } + else { + UV folded = _to_uni_fold_flags( + ender, + (U8 *) s, + &foldlen, + FOLD_FLAGS_FULL + | ((LOC) ? FOLD_FLAGS_LOCALE + : (ASCII_FOLD_RESTRICTED) + ? FOLD_FLAGS_NOMIX_ASCII + : 0) + ); + + /* If this node only contains non-folding code points + * so far, see if this new one is also non-folding */ + if (maybe_exact) { + if (folded != ender) { + maybe_exact = FALSE; + } + else { + /* Here the fold is the original; we have + * to check further to see if anything + * folds to it */ + if (! PL_utf8_foldable) { + SV* swash = swash_init("utf8", + "_Perl_Any_Folds", + &PL_sv_undef, 1, 0); + PL_utf8_foldable = + _get_swash_invlist(swash); + SvREFCNT_dec_NN(swash); } - else { - /* Here the fold is the original; we have - * to check further to see if anything - * folds to it */ - if (! PL_utf8_foldable) { - SV* swash = swash_init("utf8", - "_Perl_Any_Folds", - &PL_sv_undef, 1, 0); - PL_utf8_foldable = - _get_swash_invlist(swash); - SvREFCNT_dec_NN(swash); - } - if (_invlist_contains_cp(PL_utf8_foldable, - ender)) - { - maybe_exact = FALSE; - } + if (_invlist_contains_cp(PL_utf8_foldable, + ender)) + { + maybe_exact = FALSE; } } - ender = folded; } - s += foldlen; - - /* The loop increments <len> each time, as all but this - * path (and the one just below for UTF) through it add - * a single byte to the EXACTish node. But this one - * has changed len to be the correct final value, so - * subtract one to cancel out the increment that - * follows */ - len += foldlen - 1; - } - else { - *(s++) = (char) ender; - maybe_exact &= ! IS_IN_SOME_FOLD_L1(ender); - } - } - else if (UTF) { - const STRLEN unilen = reguni(pRExC_state, ender, s); - if (unilen > 0) { - s += unilen; - len += unilen; + ender = folded; } - - /* See comment just above for - 1 */ - len--; + s += foldlen; + + /* The loop increments <len> each time, as all but this + * path (and one other) through it add a single byte to the + * EXACTish node. But this one has changed len to be the + * correct final value, so subtract one to cancel out the + * increment that follows */ + len += foldlen - 1; } - else { - REGC((char)ender, s++); - } if (next_is_quantifier) { @@ -13027,17 +13054,17 @@ parseit: /* <multi_char_matches> is actually an array of arrays. * There will be one or two top-level elements: [2], * and/or [3]. The [2] element is an array, each - * element thereof is a character which folds to two - * characters; likewise for [3]. (Unicode guarantees a - * maximum of 3 characters in any fold.) When we - * rewrite the character class below, we will do so - * such that the longest folds are written first, so - * that it prefers the longest matching strings first. - * This is done even if it turns out that any - * quantifier is non-greedy, out of programmer - * laziness. Tom Christiansen has agreed that this is - * ok. This makes the test for the ligature 'ffi' come - * before the test for 'ff' */ + * element thereof is a character which folds to TWO + * characters; [3] is for folds to THREE characters. + * (Unicode guarantees a maximum of 3 characters in any + * fold.) When we rewrite the character class below, + * we will do so such that the longest folds are + * written first, so that it prefers the longest + * matching strings first. This is done even if it + * turns out that any quantifier is non-greedy, out of + * programmer laziness. Tom Christiansen has agreed + * that this is ok. This makes the test for the + * ligature 'ffi' come before the test for 'ff' */ if (av_exists(multi_char_matches, cp_count)) { this_array_ptr = (AV**) av_fetch(multi_char_matches, cp_count, FALSE); @@ -13541,8 +13568,7 @@ parseit: * doesn't allow them between above and below 256 */ if ((ASCII_FOLD_RESTRICTED && (isASCII(c) != isASCII(j))) - || (LOC && ((c < 256) != (j < 256)))) - { + || (LOC && c < 256)) { continue; } diff --git a/regen/regcharclass_multi_char_folds.pl b/regen/regcharclass_multi_char_folds.pl index f0fd6b3..f04be85 100644 --- a/regen/regcharclass_multi_char_folds.pl +++ b/regen/regcharclass_multi_char_folds.pl @@ -104,6 +104,29 @@ sub multi_char_folds ($) { } } + # \x17F is the small LONG S, which folds to 's'. Both Capital and small + # LATIN SHARP S fold to 'ss'. Therefore, they should also match two 17F's + # in a row under regex /i matching. But under /iaa regex matching, all + # three folds to 's' are prohibited, but the sharp S's should still match + # two 17F's. This prohibition causes our regular regex algorithm that + # would ordinarily allow this match to fail. This is the only instance in + # all Unicode of this kind of issue. By adding a special case here, we + # can use the regular algorithm (with some other changes elsewhere as + # well). + # + # It would be possible to re-write the above code to automatically detect + # and handle this case, and any others that might eventually get added to + # the Unicode standard, but I (khw) don't think it's worth it. I believe + # that it's extremely unlikely that more folds to ASCII characters are + # going to be added, and if I'm wrong, fold_grind.t has the intelligence + # to detect them, and test that they work, at which point another special + # case could be added here if necessary. + # + # No combinations of this with 's' need be added, as any of these + # containing 's' are prohibted under /iaa. + push @folds, "\"\x{17F}\x{17F}\""; + + return @folds; } diff --git a/regen/unicode_constants.pl b/regen/unicode_constants.pl index 1977fbd..ba4cf54 100644 --- a/regen/unicode_constants.pl +++ b/regen/unicode_constants.pl @@ -140,15 +140,21 @@ print $out_fh "\n#endif /* H_UNICODE_CONSTANTS */\n"; read_only_bottom_close_and_rename($out_fh); __DATA__ +U+017F string U+0300 string U+0301 string U+0308 string +U+0399 string +U+03BC string + U+03B9 string U+03C5 string +U+1E9E string + U+2010 string U+D800 first FIRST_SURROGATE diff --git a/regexec.c b/regexec.c index bc38839..4590a81 100644 --- a/regexec.c +++ b/regexec.c @@ -1252,7 +1252,7 @@ STMT_START { \ len=0; \ } else { \ len = 1; \ - uvc = _to_fold_latin1( (U8) *uc, foldbuf, &foldlen, 1); \ + uvc = _to_fold_latin1( (U8) *uc, foldbuf, &foldlen, FOLD_FLAGS_FULL); \ skiplen = UNISKIP( uvc ); \ foldlen -= skiplen; \ uscan = foldbuf + skiplen; \ diff --git a/t/porting/known_pod_issues.dat b/t/porting/known_pod_issues.dat index 735cd6d..b175a81 100644 --- a/t/porting/known_pod_issues.dat +++ b/t/porting/known_pod_issues.dat @@ -238,7 +238,6 @@ pod/perlapio.pod Verbatim line length including indents exceeds 79 by 5 pod/perlbook.pod Verbatim line length including indents exceeds 79 by 1 pod/perlcall.pod Verbatim line length including indents exceeds 79 by 2 pod/perlce.pod Verbatim line length including indents exceeds 79 by 2 -pod/perlclib.pod Verbatim line length including indents exceeds 79 by 3 pod/perlcygwin.pod Verbatim line length including indents exceeds 79 by 24 pod/perldbmfilter.pod Verbatim line length including indents exceeds 79 by 1 pod/perldebguts.pod Verbatim line length including indents exceeds 79 by 34 diff --git a/t/re/fold_grind.t b/t/re/fold_grind.t index bb45a69..9d9f426 100644 --- a/t/re/fold_grind.t +++ b/t/re/fold_grind.t @@ -131,6 +131,8 @@ my %tests; # The set of tests. # 7838 # LATIN_CAPITAL_LETTER_SHARP_S # ], +my %folds; # keys are code points that fold; + # values are each a list of code points the key folds to my %inverse_folds; # keys are strings of the folded-to; # values are lists of characters that fold to them @@ -267,8 +269,11 @@ if (ord('A') == 65 next if $fold_type eq 'S'; # If Unicode's tables are correct, the F # should be a superset of S - my $folded_str = pack ("U0U*", map { hex $_ } @hex_folded); - push @{$inverse_folds{$folded_str}}, chr hex $hex_from; + my $from = hex $hex_from; + my @to = map { hex $_ } @hex_folded; + @{$folds{$from}} = @to; + my $folded_str = pack ("U0U*", @to); + push @{$inverse_folds{$folded_str}}, chr $from; } } else { # Here, can't use the .txt file: read the Unicode rules file and @@ -287,8 +292,9 @@ else { # Here, can't use the .txt file: read the Unicode rules file and my $adjust = -1; for my $j ($invlist_ref->[$i] .. $invlist_ref->[$i+1] -1) { $adjust++; - my $folded_str - = pack "U0U*", map { $_ + $adjust } @{$invmap_ref->[$i]}; + my @to = map { $_ + $adjust } @{$invmap_ref->[$i]}; + push @{$folds{$j}}, @to; + my $folded_str = pack "U0U*", @to; #note (sprintf "%d: %04X: %s", __LINE__, $j, join " ", # map { sprintf "%04X", $_ + $adjust } @{$invmap_ref->[$i]}); push @{$inverse_folds{$folded_str}}, chr $j; @@ -502,6 +508,34 @@ foreach my $test (sort { numerically } keys %tests) { # it on one set in the other ranges just to make sure it doesn't break # them. if ($charset eq 'aa') { + + # It may be that this $pair of code points to test are both + # non-ascii, but if either of them actually fold to ascii, that is + # suspect and should be tested. So for /aa, use whether their folds + # are ascii or not + my $target_has_ascii = $target_has_ascii; + my $pattern_has_ascii = $pattern_has_ascii; + if (! $target_has_ascii) { + foreach my $cp (@target) { + if (exists $folds{$cp} + && grep { ord_native_to_latin1($_) < 128 } @{$folds{$cp}} ) + { + $target_has_ascii = 1; + last; + } + } + } + if (! $pattern_has_ascii) { + foreach my $cp (@pattern) { + if (exists $folds{$cp} + && grep { ord_native_to_latin1($_) < 128 } @{$folds{$cp}} ) + { + $pattern_has_ascii = 1; + last; + } + } + } + if (! $target_has_ascii && ! $pattern_has_ascii) { if ($target_above_latin1 || $pattern_above_latin1) { next if defined $has_tested_aa_above_latin1 @@ -528,7 +562,31 @@ foreach my $test (sort { numerically } keys %tests) { # For l, don't need to test beyond one set those things that are # all above latin1, because unlikely to have different successes - # than /u + # than /u. But, for the same reason as described in the /aa above, + # it is suspect and should be tested, if either of the folds are to + # latin1. + my $target_has_latin1 = $target_has_latin1; + my $pattern_has_latin1 = $pattern_has_latin1; + if (! $target_has_latin1) { + foreach my $cp (@target) { + if (exists $folds{$cp} + && grep { $_ < 256 } @{$folds{$cp}} ) + { + $target_has_latin1 = 1; + last; + } + } + } + if (! $pattern_has_latin1) { + foreach my $cp (@pattern) { + if (exists $folds{$cp} + && grep { $_ < 256 } @{$folds{$cp}} ) + { + $pattern_has_latin1 = 1; + last; + } + } + } if (! $target_has_latin1 && ! $pattern_has_latin1) { next if defined $has_tested_above_latin1_l && $has_tested_above_latin1_l != $test; @@ -667,146 +725,162 @@ foreach my $test (sort { numerically } keys %tests) { next if $bracketed && @pattern != 1; # bracketed makes these # or's instead of a sequence foreach my $optimize_bracketed (0, 1) { - next if $optimize_bracketed && ! $bracketed; - foreach my $inverted (0,1) { - next if $inverted && ! $bracketed; # inversion only valid in [^...] - next if $inverted && @target != 1; # [perl #89750] multi-char - # not valid in [^...] - - # In some cases, add an extra character that doesn't fold, and - # looks ok in the output. - my $extra_char = "_"; - foreach my $prepend ("", $extra_char) { - foreach my $append ("", $extra_char) { - - # Assemble the rhs. Put each character in a separate - # bracketed if using charclasses. This creates a stress on - # the code to span a match across multiple elements - my $rhs = ""; - foreach my $rhs_char (@rhs) { - $rhs .= '[' if $bracketed; - $rhs .= '^' if $inverted; - $rhs .= $rhs_char; - - # Add a character to the class, so class doesn't get - # optimized out, unless we are testing that optimization - $rhs .= '_' if $optimize_bracketed; - $rhs .= ']' if $bracketed; - } - - # Add one of: no capturing parens - # a single set - # a nested set - # Use quantifiers and extra variable width matches inside - # them to keep some optimizations from happening - foreach my $parend (0, 1, 2) { - my $interior = (! $parend) - ? $rhs - : ($parend == 1) - ? "(${rhs},?)" - : "((${rhs})+,?)"; - foreach my $quantifier ("", '?', '*', '+', '{1,3}') { - - # Perhaps should be TODOs, as are unimplemented, but - # maybe will never be implemented - next if @pattern != 1 && $quantifier; - - # A ? or * quantifier normally causes the thing to be - # able to match a null string - my $quantifier_can_match_null = $quantifier eq '?' || $quantifier eq '*'; - - # But since we only quantify the last character in a - # multiple fold, the other characters will have width, - # except if we are quantifying the whole rhs - my $can_match_null = $quantifier_can_match_null && (@rhs == 1 || $parend); - - foreach my $l_anchor ("", '^') { # '\A' didn't change result) - foreach my $r_anchor ("", '$') { # '\Z', '\z' didn't change result) - - # The folded part can match the null string if it - # isn't required to have width, and there's not - # something on one or both sides that force it to. - my $both_sides = ($l_anchor && $r_anchor) || ($l_anchor && $append) || ($r_anchor && $prepend) || ($prepend && $append); - my $must_match = ! $can_match_null || $both_sides; - # for performance, but doing this missed many failures - #next unless $must_match; - my $quantified = "(?$charset:$l_anchor$prepend$interior${quantifier}$append$r_anchor)"; - my $op; - if ($must_match && $should_fail) { - $op = 0; - } else { - $op = 1; - } - $op = ! $op if $must_match && $inverted; - - if ($inverted && @target > 1) { - # When doing an inverted match against a - # multi-char target, and there is not something on - # the left to anchor the match, if it shouldn't - # succeed, skip, as what will happen (when working - # correctly) is that it will match the first - # position correctly, and then be inverted to not - # match; then it will go to the second position - # where it won't match, but get inverted to match, - # and hence succeeding. - next if ! ($l_anchor || $prepend) && ! $op; - - # Can't ever match for latin1 code points non-uni - # semantics that have a inverted multi-char fold - # when there is something on both sides and the - # quantifier isn't such as to span the required - # width, which is 2 or 3. - $op = 0 if $ord < 255 - && ! $uni_semantics - && $both_sides - && ( ! $quantifier || $quantifier eq '?') - && $parend < 2; - - # Similarly can't ever match when inverting a multi-char - # fold for /aa and the quantifier isn't sufficient - # to allow it to span to both sides. - $op = 0 if $target_has_ascii && $charset eq 'aa' && $both_sides && ( ! $quantifier || $quantifier eq '?') && $parend < 2; - - # Or for /l - $op = 0 if $target_has_latin1 && $charset eq 'l' && $both_sides && ( ! $quantifier || $quantifier eq '?') && $parend < 2; - } - - - my $desc = "my \$c = \"$prepend$lhs$append\"; " - . "my \$p = qr/$quantified/i;" - . "$upgrade_target$upgrade_pattern " - . "\$c " . ($op ? "=~" : "!~") . " \$p; "; - if ($DEBUG) { - $desc .= ( - "; uni_semantics=$uni_semantics, " - . "should_fail=$should_fail, " - . "bracketed=$bracketed, " - . "prepend=$prepend, " - . "append=$append, " - . "parend=$parend, " - . "quantifier=$quantifier, " - . "l_anchor=$l_anchor, " - . "r_anchor=$r_anchor; " - . "pattern_above_latin1=$pattern_above_latin1; " - . "utf8_pattern=$utf8_pattern" - ); - } + next if $optimize_bracketed && ! $bracketed; + foreach my $inverted (0,1) { + next if $inverted && ! $bracketed; # inversion only valid + # in [^...] + next if $inverted && @target != 1; # [perl #89750] multi-char + # not valid in [^...] + + # In some cases, add an extra character that doesn't fold, and + # looks ok in the output. + my $extra_char = "_"; + foreach my $prepend ("", $extra_char) { + foreach my $append ("", $extra_char) { + + # Assemble the rhs. Put each character in a separate + # bracketed if using charclasses. This creates a stress on + # the code to span a match across multiple elements + my $rhs = ""; + foreach my $rhs_char (@rhs) { + $rhs .= '[' if $bracketed; + $rhs .= '^' if $inverted; + $rhs .= $rhs_char; + + # Add a character to the class, so class doesn't get + # optimized out, unless we are testing that optimization + $rhs .= '_' if $optimize_bracketed; + $rhs .= ']' if $bracketed; + } - my $c = "$prepend$lhs_str$append"; - my $p = qr/$quantified/i; - utf8::upgrade($c) if length($upgrade_target); - utf8::upgrade($p) if length($upgrade_pattern); - my $res = $op ? ($c =~ $p): ($c !~ $p); - - if (!$res || $list_all_tests) { - # Failed or debug; output the result - $count++; - ok($res, "test $count - $desc"); - } else { - # Just count the test as passed - $okays++; + # Add one of: no capturing parens + # a single set + # a nested set + # Use quantifiers and extra variable width matches inside + # them to keep some optimizations from happening + foreach my $parend (0, 1, 2) { + my $interior = (! $parend) + ? $rhs + : ($parend == 1) + ? "(${rhs},?)" + : "((${rhs})+,?)"; + foreach my $quantifier ("", '?', '*', '+', '{1,3}') { + + # Perhaps should be TODOs, as are unimplemented, but + # maybe will never be implemented + next if @pattern != 1 && $quantifier; + + # A ? or * quantifier normally causes the thing to be + # able to match a null string + my $quantifier_can_match_null = $quantifier eq '?' + || $quantifier eq '*'; + + # But since we only quantify the last character in a + # multiple fold, the other characters will have width, + # except if we are quantifying the whole rhs + my $can_match_null = $quantifier_can_match_null + && (@rhs == 1 || $parend); + + foreach my $l_anchor ("", '^') { # '\A' didn't change + # result) + foreach my $r_anchor ("", '$') { # '\Z', '\z' didn't + # change result) + # The folded part can match the null string if it + # isn't required to have width, and there's not + # something on one or both sides that force it to. + my $both_sides = ($l_anchor && $r_anchor) + || ($l_anchor && $append) + || ($r_anchor && $prepend) + || ($prepend && $append); + my $must_match = ! $can_match_null || $both_sides; + # for performance, but doing this missed many failures + #next unless $must_match; + my $quantified = "(?$charset:$l_anchor$prepend$interior${quantifier}$append$r_anchor)"; + my $op; + if ($must_match && $should_fail) { + $op = 0; + } else { + $op = 1; + } + $op = ! $op if $must_match && $inverted; + + if ($inverted && @target > 1) { + # When doing an inverted match against a + # multi-char target, and there is not something on + # the left to anchor the match, if it shouldn't + # succeed, skip, as what will happen (when working + # correctly) is that it will match the first + # position correctly, and then be inverted to not + # match; then it will go to the second position + # where it won't match, but get inverted to match, + # and hence succeeding. + next if ! ($l_anchor || $prepend) && ! $op; + + # Can't ever match for latin1 code points non-uni + # semantics that have a inverted multi-char fold + # when there is something on both sides and the + # quantifier isn't such as to span the required + # width, which is 2 or 3. + $op = 0 if $ord < 255 + && ! $uni_semantics + && $both_sides + && ( ! $quantifier || $quantifier eq '?') + && $parend < 2; + + # Similarly can't ever match when inverting a + # multi-char fold for /aa and the quantifier + # isn't sufficient to allow it to span to both + # sides. + $op = 0 if $target_has_ascii + && $charset eq 'aa' + && $both_sides + && ( ! $quantifier || $quantifier eq '?') + && $parend < 2; + + # Or for /l + $op = 0 if $target_has_latin1 && $charset eq 'l' + && $both_sides + && ( ! $quantifier || $quantifier eq '?') + && $parend < 2; + } + + + my $desc = "my \$c = \"$prepend$lhs$append\"; " + . "my \$p = qr/$quantified/i;" + . "$upgrade_target$upgrade_pattern " + . "\$c " . ($op ? "=~" : "!~") . " \$p; "; + if ($DEBUG) { + $desc .= ( + "; uni_semantics=$uni_semantics, " + . "should_fail=$should_fail, " + . "bracketed=$bracketed, " + . "prepend=$prepend, " + . "append=$append, " + . "parend=$parend, " + . "quantifier=$quantifier, " + . "l_anchor=$l_anchor, " + . "r_anchor=$r_anchor; " + . "pattern_above_latin1=$pattern_above_latin1; " + . "utf8_pattern=$utf8_pattern" + ); + } + **** PATCH TRUNCATED AT 2000 LINES -- 424 NOT SHOWN **** -- Perl5 Master Repository