develooper Front page | perl.perl5.porters | Postings from September 2000

Re: unicode support and perl

Thread Previous | Thread Next
Marc Lehmann
September 15, 2000 09:29
Re: unicode support and perl
Message ID:
On Fri, Sep 15, 2000 at 05:03:44PM +0100, Nicholas Clark <> wrote:
> I was contemplating obsoleting "PV" scalars and having "BV" and "SV" scalars
> (except that it can't be SV for string and CV is code and aargh what letter?)

BNut I think the current situation is very much like that. PV's can store
a lot of things, actually.

> as long as all the macros work. But I don't see how PV maros can be kept
> working, as they'd need to be doing implicit conversions with the same
> insufficient data that is currently exposed as SvUTF8_on and friends

Well, there is no other choice. Or what semantics would the BV's have, if
not the same as PV's (on the perl level)? Checking for UTF8-ness on the xs
level is just the same as checking for type currently, and conversions are

Or do you mean, *just* on the XS level, that SvBV would implicitly convert
utf8 to latin1?

No, my real point in this discussion is that the internal representation
(UTF8 or plain bytes) must be visible (but implicit= on the perl
_source_ level. Trying to hide the fact that there are two different
representations of pv data in perl through any mechanism (Encode API or
SvBV's) is wrong.

For example, XML::Parser automatically "taints" string values as utf8, on
5.6 and above. This means that programs working with 5.5 will not work
with 5.6 and vice versa. Similarly, many of my programs/modules will start
to break as soon as DBI tags my data as UTF-8 (database-dependent, as

<offtopic>Does Storable retain utf8-ness? oh no, it doesn't...</>

      -----==-                                             |
      ----==-- _                                           |
      ---==---(_)__  __ ____  __       Marc Lehmann      +--
      --==---/ / _ \/ // /\ \/ / |e|
      -=====/_/_//_/\_,_/ /_/\_\       XX11-RIPE         --+
    The choice of a GNU generation                       |

Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About