develooper Front page | perl.perl5.porters | Postings from May 2008

Re: on the almost impossibility to write correct XS modules

Thread Previous | Thread Next
Marc Lehmann
May 15, 2008 16:02
Re: on the almost impossibility to write correct XS modules
Message ID:
On Wed, May 14, 2008 at 11:55:40PM +0200, demerphq <> wrote:
> >  I'm still digesting the rest of the ideas in the thread.
> This problem crops in the core as well as in XS. For instance our

Didn't know that, at least in the core it would be "easily" fixable,

The bigger problem seems to be that there is no universal agreement on
how unicode works and what the utf-8 flag means between perl5-porters,
and this issue would have to be tackled first. Violations of the official
model should be treated as bugs. I remember well how hard it was to get
the unpack fix in, and don't really want to repeat the experience.

> internal file apis use char* parameters. (This is relevant for Win32
> in particular)

From what I saw, however, filename handling on win32 is broken beyond
repair, as much of the win32 code overloads the utf-8 flag with the
additional meaning "unicode encoded, not ansi codepage" (see for example
almost all functions dealing with paths in Win32.xs).

In the BDB module, I had to manually convert from ansi to utf-8 *iff* the
utf-8 flag isn't set, as that is the only way to find out the encoding of
the filename. Pray there are no upgrades/downgrades.

This is the code I use, and of course, it doesn't work reliable, but at
least with manual enforcement on the perl level you can access files if
you outwit perl:

So in practise, this is not a real problem, as non-ascii filenames don't
really work under perl on windows anyways :) *oink*.

                The choice of a       Deliantra, the free code+content MORPG
      -----==-     _GNU_    
      ----==-- _       generation
      ---==---(_)__  __ ____  __      Marc Lehmann
      --==---/ / _ \/ // /\ \/ /
      -=====/_/_//_/\_,_/ /_/\_\

Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About