develooper Front page | perl.perl5.porters | Postings from June 2013

Re: [perl #118537] Add non-XS extensions to known_extensions

Thread Previous | Thread Next
Andy Dougherty
June 21, 2013 21:11
Re: [perl #118537] Add non-XS extensions to known_extensions
Message ID:
On Wed, Jun 19, 2013 at 02:25:32AM -0700, Nicholas Clark wrote:
> # New Ticket Created by  Nicholas Clark 
> # Please include the string:  [perl #118537]
> # in the subject line of all future correspondence about this issue. 
> # <URL: >
> I think that we should change Configure so that known_extensions includes
> non-XS extensions. Currently, not having the non-XS modules in
> known_extensions means that unlike XS modules, you can't skip building any
> non-XS module, as t/TEST and t/harness will always run their tests (which
> will fail), because the extension-test-skip logic uses known_extensions.
> This means that
> 1) you can't elect not to build to-be-evicted modules
> 2) we can't put any platform-specific modules in ext (eg VMS::Filespec)
> If we change things so that known_extensions includes the nonxs_ext, then
> both of the above are solved.
> I don't think that it will break anything - suggests that there
> are only 10 non-core mentions of known_extensions, and none of them rely on
> this existing behaviour.

Broadly speaking, I think it should be safe to change it to whatever works.  

I think t/TEST and t/harness are wrong.  Their extension-test-skip logic
should just use $extensions.  Nothing running after Configure should use
$known_extensions.  It was not intended to be useful for anything after
Configure time, though the documentation isn't explicit about that.
$known_extensions lists all the extensions Configure 'knows' about.
Only the subset in $extensions was actually selected by the user to be
included with perl.  The user is not required to install every extension
shipped with the perl tarball.

> Right now we have the wonderfully inconsistent situation (for
> Configure-generated of
> dynamic_ext:        built dynamically linked XS modules
> static_ext:         built statically linked XS modules
> nonxs_ext:          built pure-Perl modules (from ext/, dist/ and cpan/)
> extensions:         "$dynamic_ext $static_ext $nonxs_ext"
> known_extensions:   *just* the XS modules shipped in ext/, dist/ and cpan/
> ie "extensions" is typically much larger than "known_extensions"

Yes, but the intent was you wouldn't care, since known_extensions wasn't 
intended to be used.

> This situation has come about through "organic growth", rather than design.

Yes, that's undoubtedly true.

> I guess it's summarised as
> 0) Perl 5 predates CPAN
> 1) Originally Perl 5 only shipped with XS code in ext/

Yes.  libraries were in lib/.  In perl4, you had to relink against uperl.o
and build a new executable (e.g. curseperl) to link in other C libraries.  
In perl5, you could use dynamic loading to load in the extensions.

> 2) Originally there was no concept of dual-life - if you wanted the
>    extensions in ext/, you had to build them with perl

There were outside modules; they just didn't have a ready toolchain.
Tk was one of the first big ones that I recall.  If you dropped them in
the ext/ directory, Configure was supposed to handle them automatically,
but if you wanted to build them later, that was ok too.

> 3) 15 years ago (today), Configure was patched to add nonxs_ext
>    (commit 4318d5a0158916ac) ready to support Errno
>    (Errno was added about two weeks later in commit eab60bb1f2e96e20)
>    [curiously that commit adds Errno to known_extensions but not to
>     extensions]
> 4) A few days later commit bfb7748a896459cc updates Configure so that
>    nonxs_ext *are* in extensions, but are *not* in known_extensions.
>    The description of the change is:   
>     Explicitly split list of extensions into 3 kinds:  dynamic, static,
>     and non-xs.  The Configure variable $extensions now holds all three.
>     (The only current non-xs extension is Errno).
>     It also updates Porting/Glossary, explicitly changing the description
>     of known_extensions from "list of all extensions included" to
>     "list of all XS extensions included", and extensions from
>     "all extension files linked into the package" to
>     "all extension files (both XS and non-xs linked into the package."
>     [oooh, that lack of closing ) is still present today]

Yeah, I tend to do that.

Anyway, feel free to fix and document it however makes sense in the new
era.  However, I'd recommend documenting that nobody should actually be
*using* known_extensions for anything.  They should be using $extensions.

    Andy Dougherty

Thread Previous | Thread Next Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About