develooper Front page | perl.perl5.porters | Postings from February 2014

Re: EBCDIC support is on the chopping block

Thread Previous | Thread Next
Nicholas Clark
February 21, 2014 09:48
Re: EBCDIC support is on the chopping block
Message ID:
On Fri, Feb 21, 2014 at 01:10:23AM +0100, Leon Timmermans wrote:
> On Thu, Feb 20, 2014 at 6:43 AM, Konovalov, Vadim
> <>wrote:
> > Indeed.
> >
> > from technical POV - this support UTF-EBCDIC - while it obviously costs
> > some
> > maintenance efforts - does it cost any CPU speed for non-EBCDICs?
> IMO it requires significant maintenance efforts. For example, we have been
> working on a new UTF-8 IO layer. Adding UTF-EBCDIC support to that is
> enough work that I'd qualify it as a rewrite (validation and translation
> are fundamentally different problems). Not that the current :utf8 DWIMs on
> I don't think it costs now if we don't touch any unicode related code, but
> we probably will.

There are other pain points which hurt the non-Unicode code too.

Specifically one that I hit was that there isn't even *one* EBCDIC.
There are several mappings of characters to code points, and they don't
agree on what byte value '~' has, which you can't even pre-generate lookup
tables for "ASCII" and "EBCDIC" systems, because the latter doesn't exist.

That's certainly not the only one.

Deep and tricky parts of the codebase, and therefore anyone looking at them,
have to be aware that they aren't allowed to assume ASCII (which is several
subtle things many folks take for granted), and, independently, that they
aren't allowed to assume one encoding system for multibyte characters
(ie UTF-8). Because both EBCDIC (for fixed width characters) and UTF-EBCDIC
(for variable width) are very different from ASCII and UTF-8.

So a bunch of this stuff ends up being conditional compilation, or extra
build steps, and probably other stuff, which just makes the code more complex
than it would be.

And we don't even know if it works, because no-one working on the core has
access to these systems, and no-one with these systems is providing reports.

We don't have infinite developer resources. (It's rather closer to 0 than
infinity.) So everything is a trade off, and effort being consumed by this
means that something else isn't being done. So effectively EBCDIC support is
stealing progress from the rest of the Perl userbase.

Nicholas Clark

Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About