develooper Front page | perl.perl5.porters | Postings from May 2008

Re: on the almost impossibility to write correct XS modules

Juerd Waalboer
May 21, 2008 14:33
Re: on the almost impossibility to write correct XS modules
Message ID:
Glenn Linderman skribis 2008-05-21 10:44 (-0700):
> >(You'd want to do this based on column
> >introspection but that's very slow and very hard to write portably.)
> Does your parenthetical remark mean that it should be done by detecting 
> that the column is SQL BINARY or VARBINARY vs SQL CHAR or VARCHAR?


> If so, I agree with it.  It is not clear why it is slow, nor hard to
> write portably.  SQL BINARY has existed for years, and shouldn't have
> any portability problems.

You would need to ask the database about the column's type, and wait for
a response. Of course this is infinitely slower than not doing anything
at all. And to make the performance hit even worse: before you know
which column to introspect, you have to parse the SQL. If you're
unlucky, the column name isn't even there and you have to introspect
tables first.

> No, it currently requires knowledge of how to use utf8::is_utf8() to 
> help isolate where the upgrade happens.

Please advocate the use of Devel::Peek for peeking at internals, and
discourage the use of utf8::is_utf8.

> Unless utf8::is_utf8() is removed (which would break some amount of
> compatibility), that is all the knowledge required.

Yes please. Almost every single use of is_utf8 is either for debugging
(and can thus be removed without affecting semantics) or JUST PLAIN
WRONG. Removing (or renaming) it will be immensely helpful.

Note that the SvUTF8 flag is different from the new flag that we're
discussing now. The new flag would forcefully prevent SvUTF8 from ever
getting set.

> If it is cheap enough to add the "stricture", I'm not against having it 
> added; I'm just trying to figure out if there are enough benefits to pay 
> the cost, whatever the cost is.

The cost of the stricture will in many cases be more than compensated by
alerting programmers when relatively expensive upgrades happen.

It's adding one bit, and checking one bit. I assume that this can't
possibly hurt performance.

> Certainly it is more convenient to be told the exact line where the
> stricture would be violated

Line and operations. I'm thinking of a warning like "Binary-incompatible
string added to binary string as UTF-8 data in assignment at line

> but a few debugging calls to utf8::is_utf8() would tell the tale.

I can tell you from experience that "a few" debugging calls can easily
take up to half a day in a complex program. Even more for inexperienced
programmers who don't know about the Perl internals, and don't even know
where to begin debugging the "strange" issues.
Met vriendelijke groet,  Kind regards,  Korajn salutojn,

  Juerd Waalboer:  Perl hacker  <>  <>
  Convolution:     ICT solutions and consultancy <>
1; Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About