Front page | perl.perl5.porters |
Postings from March 2000
Re: What's left to do? [LONG]
From:
Tom Christiansen
Date:
March 22, 2000 05:36
Subject:
Re: What's left to do? [LONG]
Message ID:
15248.953732192@chthon
>*sigh*. I think the most important thing in here is the
>line disciplines. This is something that's been promised for a
>while.
>!add support for I/O disciplines
>! - a way to specify disciplines when opening things:
>! open(F, "<:crlf :utf16", $file)
>! - a way to specify disciplines for an already opened handle:
>! binmode(STDIN, ":slurp :raw")
>!- a way to set default disciplines for all handle constructors:
>! use open IN => ":any", OUT => ":utf8", SYS => ":utf16"
>Extending binmode to set these flags as well as the current
>disciplines doesn't look like it would be too hard to do.
>That said, it still needs doing. Is anyone actively working on
>this?
Larry and Sarathy have signed off on the user-visible interface
here. The code still needs elaboration there. If you look at
open.pm, and if you go into doio.c and look at mode_from_discipline
and at the top of do_open9, you'll see this has begun. Also, binmode
is now prototyped in opcode.pl to take "F S?", although this isn't
in doio.c yet.
Another place where there's a stub for a bit of agreed-upon future
expansion is pipe open with a list. Notice opcode.pl has open
taking "F S? L". That's because these need to become (nearly)
equivalent:
# these are now the same, but might call the shell
open FH, "|tr a-z A-Z" or die;
open FH, "|-", tr a-z A-Z" or die;
# these don't call the shell; the second needs implementation
open FH, "|-" or exec 'tr', 'a-z', 'A-Z' or die;
open FH, "|-", 'tr', 'a-z', 'A-Z' or die;
# these are now the same, but might call the shell
open FH, "cat -n file|" or die;
open FH, "-|", "cat -n file" or die;
# these don't call the shell; the second needs implementation
open FH, "-|" or exec 'cat', '-n', 'file' or die;
open FH, "-|", 'cat', '-n', 'file' or die;
But the work for that hasn't been done in utils.c's my_popen yet.
It's the third one that needs implementation.
Whoever tackles this look into making this all work, too:
# these three might all call the shell
open FH, "|bc -l|" or die;
open FH, "|-|" or exec "bc -l" or die;
open FH, "|-|", "bc -l" or die;
# but these two won't
open FH, "|-|" or exec 'bc', '-l' or die;
open FH, "|-|", 'bc', '-l' or die;
You could just turn this into an autorequired IPC::Open2::open2,
but there's the issue that you need to make one handle go both ways.
This is easy. You'll do something like this, plus error checking:
int fd[2];
PerlProc_pipe(fd);
io = GvIOn(gv);
IoIFP(io) = PerlIO_fdopen(fd[0], "r");
IoOFP(io) = PerlIO_fdopen(fd[1], "w");
I guess you'll have to set the IoTYPE(io) to "|". Looks like that'll
make things work right. One could argue for "||" or "|+" or
something, but the existing code that checks IoTYPEs would have to
be changed to know about that. You'll probably want to look into
the sneaky "tell me if the fork's exec failed" stuff so it patches
one-ended pipe.
This should be enough for an enterprising person to go on.
>(Threading)
># Which of the standard modules are thread-safe? Which CPAN modules?
># How easy is it to fix those non-safe modules?
># Threading is still experimental. Every reproducible bug identifies
># something else for us to fix. Find and submit more of these problems.
>Basically lots of destruction testing. Anyone can do this.
But the ithread/pthread issues have yet to be worked out. Probably
with ithreads, you'll need something like
our ($x :shared, $y :shared :rdonly);
It's possible you could implement this now even for fork() using
the attributes.pm tie-ins to :attrs and IPC::Shareable.
I do not how something like
my $z :shared;
will work, since a lot more dynamic bookkeeping will seem to need
to happen to make it all work.
Even in an ithread land, there will be people who prefer unsafe
operation where everything my default is shared. Who knows how
that will be supported. One can imagine a user interface of
use vars ':shared';
use unsafe ':vars';
use shared ':unsafe';
use shared ':ALL';
or something. But how this works at all, let alone for lexicals,
is unclear.
># Auto-produce executable.
>Don't understand. Isn't this what perlcc does, or do we want to
>produce true executables from bytecode? I still can't get
> perlcc -b -o hello -e 'print qq/Hello world\n/'
>to do anything interesting.
Really? It works for me.
% perlcc -b -o hello -e 'print qq/Hello world\n/'
--------------------------------------------------------------------------------
Compiling comp3145.p:
--------------------------------------------------------------------------------
Making Bytecode(hello) for comp3145.p!
/usr/local/bin/perl -I/usr/local/lib/perl5/5.6.0/OpenBSD.i386-openbsd -I/usr/local/lib/perl5/5.6.0 -I/usr/local/lib/perl5/site_perl/5.6.0/OpenBSD.i386-openbsd -I/usr/local/lib/perl5/site_perl/5.6.0 -I/usr/local/lib/perl5/site_perl -I. -MB::Stash -c comp3145.p
/usr/local/bin/perl -I/usr/local/lib/perl5/5.6.0/OpenBSD.i386-openbsd -I/usr/local/lib/perl5/5.6.0 -I/usr/local/lib/perl5/site_perl/5.6.0/OpenBSD.i386-openbsd -I/usr/local/lib/perl5/site_perl/5.6.0 -I/usr/local/lib/perl5/site_perl -I. -MO=Bytecode,-umain,-uattributes,-uDB comp3145.p
comp3145.p syntax OK
% ./hello
./hello: Permission denied.
Exit 1
jhereg(tchrist)% chmod +x ./hello
jhereg(tchrist)% ./hello
Hello world
But that long message is insane. Utilities shouldn't be spitting out
all those horizontal rules. In fact, it shouldn't say anything at all
if it works. And it shouldn't forget to turn on the execute bit.
These are bugs anyone can fix, though.
It's still huge, though.
% ls -l hello
-rwxr-xr-x 1 tchrist wheel 161127 Mar 22 05:56 hello
And someone peculiar:
% file hello
hello: perl commands text
Because of this:
% cat -v hello | head
#!/usr/local/bin/perl
use ByteLoader 0.03;
^F
^C^@^@^@^@^E^A^C^@^@^@^A^F^@^C^@^@^@^B^N^@^@^@^A^P^@^@^A^@^E^@^C^@^@^@^C^A^@^@^@^@^N^@^@^@^A^P^@^@^@
B^@^@^@^C9^A8M-^?M-^?M-^?M-^?7M-^?M-^?M-^?M-^?6^@^@^@^A6^@^@^@^B^G^D^D^@^@^@^D^G^@^D^@^@^@^E^G^D^D^@^@^@^FR^@^@^@^CS^@^@^@^CV^@M-2W@bU^@^@^@^@X^MY@Z^@^@^@^E[^@^@^@^D]^@^@^@^C^G
^D^@^@^@^G^B^@^@^@^ER^@^@^@^GS^@^@^@^GV^@M-1W@]U^@^@^@^@X^@Y^@^G^@^D^@^@^@^H^B^@^@^@^GR^@^@^@^HS^@^@^@^DV^@M-.W@^U^@^@^@^@X^AY^@ ^@^@^@^@l ^@^@^@^Emain^@mo^@^@^DM-^[ ^@^@^@^Kcomp3145.p^@np^@^@^@^@q^@^Ar^@^@^@^C^G^F^D^@^@^@ ^B^@^@^@^DR^@^@^@^FS^@^@^@^CV^@M-QW@aU^@^@^@^@X^EY^@Z^@^@^@^H[^@^@^@ ]^@^@^@^A^B^@^@^@^HR^@^@^@ S^@^@^@ V^@^CW@_U^@^@^@^@X^BY^@^F^D^C^@^@^@
^B^@^@^@ R^@^@^@^DS^@^@^@^CV^@^EW@`U^@^@^@^@X^BY^@e^@^@^@
^N^@^@^@^A^P^DM-^D^@^D ^@^@^@^MHello world
^@^RCwarnings::import^@^C^@^@^@^K^P^@^@`^MQ^B^O^@^@^@^AO^@M-w ^@^@^@'/usr/local/lib/perl5/5.6.0/warnings.pm^@KG^@^@^@^B^F^@^C^@^@^@^L^F^L^C^@^@^@^M^A^@^@^@^KE^@^@^@^LH^@^@^@^CI^@^@^@^CJ^@^@^@^MM^@^@^@^CL^@^@^@^C^A^@^@^@^L^N^@^@^@^A^P^@^@^@^@^A^@^@^@^M^N^@^@^@^A^P^@^@^@^L^S^@^@^@^@^U0.000000^@B^@^@^@^C^G^A^D^@^@^@^N^G
^D^@^@^@^ODwarnings^@^C^@^@^@^P^F
>Namespace issues are being thrashed out in another thread and I'll
>see what the outcome is there. From Todo-5.6:
I know this isn't what you're talking about, but the namespace
issues on threads themselves need working out. If one ithread loads
module Foo, how will another ithread know about this? If not, you
get huge code bloat. requires need to be shared amongst all
interpreters, or it will be too big. This also becomes important
as Safe turns into an ithread model.
># POSIX on non-POSIX
>} use posix calls internally where possible
The way POSIX currently works makes it hard to know whether you even
got something or not. If you say "use POSIX 'EINTR'" (just as a simple
example), you don't get a compilation error if that's not there.
It's delayed until run-time. Same with use Fcntl. I believe the
Exporter has some magic that might make fixing this feasible. People
expect that if use cannot provide something usable, it will fail. This
is not hte case here.
One still might want to get at the old functionality though, especially
if not explicitly imported.
use Fcntl;
my $openflags = O_CREAT | O_EXCL | O_RDWR;
for my $oflag (qw/FOLLOW BINARY LARGEFILE EXLOCK NOINHERIT TEMPORARY/) {
my ($bit, $func) = (0, "Fcntl::O_" . $oflag);
no strict 'refs';
$openflags |= $bit if eval { $bit = &$func(); 1 };
}
>(Security)
>! use fchown, fchmod (and futimes?) internally when possible
>! use fchdir(how portable?)
>How are we doing with these? Do we have configure tests in
>place, and is someone prepared to go on a search-and-destroy
>mission?
Yes, Configure knows fchown and fchmod. Perl, however, does not.
They should all know to take a FH and call the f-version.
>! create secure reliable portable temporary file modules
>Well, we now appear to have one of these, File::Temp.
Yes, so it would appear.
>Is this going in core?
I'd certainly like it to find its way into some future release,
like perhaps 5.6.1 or so.
>! audit the standard utilities for security problems and fix them
>Tom, you were looking at this - how much is done, how much is
>still screamably dangerous?
perldoc and perl5db are "done" in the sense of no longer making you
want to fix-by-clri(8) (or is that fix-by-clri-2? :-), but there are
still races. I have comments in perl5db explaining that issue. It all
comes down to if (-filetest) { open ... } begin the wrong thing. You
must open first, then filetest that handle. So for perl5db, you'd need
to read the config file into an eval string, then eval it. The current
safety check is not provably correct for tight races.
Other utilities are still bad. perlbug is very nasty as far as
coding goes, and my recollection is that s2p and perlcc at least
still have problems. Many of the Perl utilities are incredibly
substandard in their code quality. They somehow got snuckered into
the release without being subjected to the sort of scrutiny that
the core's C code is used to getting; eg, perlbug will make you
weep. What happens is Perl makes it quick to whip up a prototype,
one that's just hacked till it works. But then we never go back
and rework them from a software design and maintainability perspective.
Russ has done this for pod2man. Thank goodness. But perldoc and
perlbug are completely nasty. And I have my doubts about perlcc.
>} optimize away @_ where possible
>There is work going on with this, isn't there?
Don't think so. Should be. Gets talked about a lot. Want some
subjects? :-)
>} tail recursion removal
>Yikes. That's a biggy. Anyone thinking about it?
Not very important.
>} cache eval tree (unless lexical outer scope used (mark in &compiling?))
We used to do this for eval 'blah' in perl4.
>} switch structures
>Everyone wants a switch statement but nobody can agree on the right
>way to do it.
There will be something at the Perl Conference regarding this. I
don't feel ethical pre-announcing details of unpublished work not
my own, but perhaps the author shall, upon reading this response,
choose to offer the group a synopsis, sample, or snapshot of his
experiences with this.
># C<magic_setisa> should be made to update %FIELDS
>I have no idea what this means. Is it a scary pseudohash thing?
Don't be redundant. :(
># there was talk of a mark-and-sweep garbage collector at TPC2
>Is this idea still alive or dead?
All I reclal was that Raphael had some ideas.
># Make embedded Perl easier to use
>Similarly, we now have storming documentation on embedding
>Perl.
Don't think so. I think it's still hard to use.
>} use less
>Optimisation tradeoffs. Do we *have* any tradeoff areas
>at the moment we could use for this?
Not that I know of. Remember you can always say
use less 'this is a cool comment because import failures are ignored';
I think UNIVERSAL::import needs appearance, per earlier discussion.
># Procedural options
>Turning IO::* and friends into procedural interfaces. Is
>this really wanted, and is someone working on it?
Virtually all of IO::File and IO::Handle are grautuitous/superfluous.
We already have a procedural interface to them: it's called Perl.
The only ones that are not very easy to do with real Perl are these:
HANDLE->clearerr
HANDLE->blocking(EXPR)
HANDLE->error
HANDLE->formline(PICTURE, LIST)
HANDLE->getpos
FileHandle->new_tmpfile
HANDLE->setbuf(BUFFER)
HANDLE->setpos(EXPR)
HANDLE->setvbuf(BUFFER, TYPE, SIZE)
HANDLE->sync
HANDLE->untaint
But you can call those as
FileHandle::untaint($myfh);
I believe though, that something for IO::Socket that's procedural would
be what's talked about. For example
package IO::Socket;
use IO::Socket;
sub tcp_connect { return IO::Socket::INET::->new(@_) }
or some such. People use these as filehandles anyway, because that's
what they are. There's no hiddenness.
># Finish a proper Ioctl module.
>What's improper about the current one?
% perl -MIoctl -e 1
Can't locate Ioctl.pm in @INC (@INC contains: /usr/local/lib/perl5/5.6.0/OpenBSD.i386-openbsd /usr/local/lib/perl5/5.6.0 /usr/local/lib/perl5/site_perl/5.6.0/OpenBSD.i386-openbsd /usr/local/lib/perl5/site_perl/5.6.0 /usr/local/lib/perl5/site_perl .).
BEGIN failed--compilation aborted.
Exit 2
I surely get tired of scrounging about for SIOCGIFBRDADDR and TIOCSTI
and FIONREAD and all the rest of them.
># perfect a Perl version of expect
>Work with IO::Tty and put these in core replacing Comm.pl
>Is this really what we want/need?
There's an Expect.pm you know.
># Separate function manpages by default
>splitman to create man 3p pages for functions/operators and
>install by default. Is this a good idea? Will you do it?
Who is "you"? :-) I've done it. I use it.
But this is very makeshift, and the functions don't necessarily all
lend themselves to being splat. Look at getpw* and their ilk. They
don't have separate entries, so automated splitting doesn't do the
right thing.
--tom