[perl #119395] readdir() return value should be documented as always downgraded

# New Ticket Created by  Victor Efimov 
# Please include the string:  [perl #119395]
# in the subject line of all future correspondence about this issue. 
# <URL: https://rt.perl.org:443/rt3/Ticket/Display.html?id=119395 >


readdir() return value should be documented as always downgraded. otherwise
there is an inconsistency in file functions workflow.

assumptions:
1. readdir returns binary strings
2. binary data can be (randomly) upgraded or downgraded by 3rd party code
(it's hard for programmer to control this)
3. Functions working with filenames ( here is list
http://perldoc.perl.org/perlunicode.html#When-Unicode-Does-Not-Happen )
simply ignore UTF-8 flag
4. Programmer might want to work with filenames as with binary strings (not
character strings) because filesystem encoding unknown/hard to detect.

example:

opendir(my $dh, '.');;
for (readdir($dh)) {
$ARGV[0] ? utf8::upgrade($_) : utf8::downgrade($_);
die $_ unless -e $_;
}

above code fails if binary strings were upgraded and there are
non-ASCII-7bit filenames in current directory.

line "$ARGV[0] ? utf8::upgrade($_) : utf8::downgrade($_);" represent the
fact that binary string was randomly downgraded or upgraded (perhaps
after concatenation with ASCII string with UTF-8 bit, or maybe after
serialization/deserialization)

solution: programmer should explicitly call utf8::downgrade($_) before "-e"
test:

opendir(my $dh, '.');;
for (readdir($dh)) {
$ARGV[0] ? utf8::upgrade($_) : utf8::downgrade($_);
utf8::downgrade($_);
die $_ unless -e $_;
}

but that is correct solution only if we are sure that readdir returns
downgraded byte strings.

same probably true for readlink and @ARGV

• [perl #119395] readdir() return value should be documented as always downgraded by Victor Efimov
• Re: [perl #119395] readdir() return value should be documented asalways downgraded by David Golden
• Re: [perl #119395] readdir() return value should be documented asalways downgraded by Zefram
• Re: [perl #119395] readdir() return value should be documented asalways downgraded by David Golden
• Re: [perl #119395] readdir() return value should be documented asalways downgraded by Mark Overmeer
• Re: [perl #119395] readdir() return value should be documented asalways downgraded by Victor Efimov
• Re: [perl #119395] readdir() return value should be documented asalways downgraded by Leon Timmermans
• [perl #119395] readdir() return value should be documented as always downgraded by Victor Efimov via RT
• Re: [perl #119395] readdir() return value should be documented asalways downgraded by Zefram
• Re: [perl #119395] readdir() return value should be documented asalways downgraded by Leon Timmermans
• Re: [perl #119395] readdir() return value should be documented asalways downgraded by Eric Brine
• Re: [perl #119395] readdir() return value should be documented asalways downgraded by Aristotle Pagaltzis
• Re: [perl #119395] readdir() return value should be documented asalways downgraded by Leon Timmermans
• Re: [perl #119395] readdir() return value should be documented asalways downgraded by Victor Efimov
• Re: [perl #119395] readdir() return value should be documented asalways downgraded by Leon Timmermans
• [perl #119395] readdir() return value should be documented as always downgraded by Victor Efimov via RT
• [perl #119395] readdir() return value should be documented as always downgraded by Victor Efimov via RT
• Re: [perl #119395] readdir() return value should be documented asalways downgraded by Leon Timmermans
• [perl #119395] readdir() return value should be documented as always downgraded by Victor Efimov via RT
• Re: [perl #119395] readdir() return value should be documented asalways downgraded by Brad Gilbert
• [perl #119395] readdir() return value should be documented as always downgraded by Victor Efimov via RT