perl.makemaker

perl.makemaker http://www.nntp.perl.org/group/perl.makemaker/ ... Copyright 1998-2013 perl.org Thu, 20 Jun 2013 02:51:47 +0000 ask@perl.org ExtUtils::MakeMaker 6.68 released by Chris 'BinGOs' Williams We're leaving together But still it's farewell And maybe we'll come back, To earth, who can tell? I guess there is no one to blame We're leaving ground (leaving ground) Will things ever be the same again? It's the final countdown. -- "The Final Countdown" - Europe V6.68 of ExtUtils-MakeMaker has been liberated to CPAN The following changes were made since the last stable release V6.66 6.68 Fri Jun 14 23:26:11 BST 2013 No changes from 6.67_05 6.67_05 Thu Jun 13 21:52:46 BST 2013 Doc fixes: * RT#86007 - Restore meaning for divorced sentence 6.67_04 Mon Jun 10 20:18:25 BST 2013 Bug fixes * Address RT#85406, where specifying 'meta-spec' in META_[ADD|MERGE] would remove all prereqs (bingos) 6.67_03 Wed Jun 5 22:03:28 BST 2013 Doc Fixes * Document how to specify meta-spec in META_MERGE (bingos) 6.67_02 Sun Jun 2 18:27:45 BST 2013 Bug Fixes * Allow v-prefixed version strings once more (bingos) * Typos fixed (David Steinbrunner) * Resolve test failure with latest CPAN::Meta 6.67_01 Thu Apr 25 21:03:58 BST 2013 Doc Fixes * Change references to makemaker.org in the docs (Reported as RT#83246 by dolmen) VOS Fixes * 'core' files are keep files (*.kp) on vos, adjust the 'clean' target to account for this (Paul Green) Win32 Fixes * Increase dmake MAXLINELENGTH to 800000 (RT#77215) as per kmx's recommendations (kmx) Cygwin Fixes * Allow linking of Cygwin libraries (Reini Urban) The full change log is available online at: https://metacpan.org/source/BINGOS/ExtUtils-MakeMaker-6.68/Changes Further detail is available in the source code repository at https://github.com/Perl-Toolchain-Gang/ExtUtils-MakeMaker -- Chris Williams aka BinGOs PGP ID 0x4658671F http://www.gumbynet.org.uk ========================== http://www.nntp.perl.org/group/perl.makemaker/2013/06/msg3359.html Sat, 15 Jun 2013 13:59:43 +0000 ExtUtils::MakeMaker 6.66 released by Chris 'BinGOs' Williams Night was black, was no use holding back 'Cause I just had to see, was someone watching me? In the mist, dark figures move and twist Was all this for real or just some kind of Hell? 6 6 6, the number of the beast Hell and fire was spawned to be released -- "The Number Of The Beast" - Iron Maiden V6.66 of ExtUtils-MakeMaker has been unleashed upon CPAN The following changes were made since the last stable release V6.64 6.66 Fri Apr 19 17:53:13 BST 2013 No changes from 6.65_03 6.65_03 Mon Apr 15 13:44:24 BST 2013 Test Fixes * Use File::Temp in parse_* tests to resolve race conditions on 64bit Windows (bingos) 6.65_02 Sun Apr 14 10:56:41 BST 2013 Test Fixes * t/xs.t is now running tests against the XS build. (Michael G Schwern) (Leon Timmermans) 6.65_01 Tue Mar 19 00:06:17 CET 2013 New Features * Improvements perlcritic support. (M. Schwern) * Improvements to dynamic linking for gcc (Tobias Leich) [github #43] * Change $(PERL_HDRS) from a hard coded list of headers to reading install directory for available header files. Allows us to work with any version of Perl properly. (Yves Orton, Craig A. Berry) [github #47] Doc Fixes * Numerous typo fixes. (Ben Bullock) [github #33] [github #34] [github #35] * Various FAQ and doc improvements (M. Schwern, Ivan Bessarabov) [github #44] Bug Fixes * fixes relating to hash ordering (Yves Orton) [github #46] [rt.cpan.org #83441] [rt.perl.org #116857] * fixes to Mksymlists (Ben Bullock, Yves Orton) [github #48] [github #49] [github #51] The full change log is available online at: https://metacpan.org/source/BINGOS/ExtUtils-MakeMaker-6.66/Changes -- Chris Williams aka BinGOs PGP ID 0x4658671F http://www.gumbynet.org.uk ========================== http://www.nntp.perl.org/group/perl.makemaker/2013/04/msg3358.html Fri, 19 Apr 2013 19:12:21 +0000 Re: ExtUtils::MakeMaker needs a release BY YOU by demerphq If noone else steps up, I would like to make an alpha release. My CPAN id is YVES, and github is 'demerphq'. If you give me the relevant permissions I will get one rolled this week. Yves On 16 March 2013 01:42, Michael G. Schwern <schwern@pobox.com> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Hi folks, > > 5.18.0 needs a MakeMaker release in order to fix some hash order bugs. > Fortunately, its all already been patched in, its just a matter of us > putting out an alpha, testing it, and then putting out a stable. > > And I'm not doing it. Been doing it too long, I'm just getting in the > way. > > I'd like somebody to step up and offer to do the next stable. > > To that end, I've written down how to do it as best I can. > > Here's how you make a release, in general. > https://github.com/Perl-Toolchain-Gang/ExtUtils-MakeMaker/wiki/How-To-Make-A-Release > > And here's the special considerations for a stable release. > https://github.com/Perl-Toolchain-Gang/ExtUtils-MakeMaker/wiki/How-To-Make-A-Stable-Release > > An alpha release isn't much different from releasing any other CPAN > module. A stable release involves a bit more rigor, but that's mostly > reading CPAN Testers. And I'll be happy to be available as a resource > and answer questions and hold hands. > > So, who wants to make a MakeMaker release? > > > - -------- Original Message -------- > Subject: ExtUtils::MakeMaker > Date: Wed, 6 Mar 2013 10:38:29 -0500 > From: Ricardo Signes <rjbs@cpan.org> > To: Michael Schwern <schwern@pobox.com> > > > Perl 5.18.0 is very likely to include improvements to hash traversal > randomness. Right now, I believe the only part of the core left that > has not > been patched in blead is ExtUtils::MakeMaker. I see at > > https://github.com/Perl-Toolchain-Gang/ExtUtils-MakeMaker/commits/master > > that you've merged the work from Yves Oton to fix the bugs made > visible by the > changes to perl. Can we expect a release of EUMM more or less > immediately? > > If not, we'll apply these patches in blead and create a _0x version > number, > which I am really not eager to do. > > - -- > rjbs > > > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG/MacGPG2 v2.0.18 (Darwin) > Comment: GPGTools - http://gpgtools.org > Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ > > iQEcBAEBAgAGBQJRQ7/qAAoJEJxzdbWzfct37rUH/RwHuCJcDdFydWoltV5tHVq4 > Rfk17hCN7ZZP3Bsd2GQT/0xRqqcycck4JGeHmtypZTtzCv9Eg3wlK67uWDasCogV > iSs/wHSyffx/25C0JKZ7Wjag1U//G+HLQSopTXnAf9PSmTM3o5eE46E/9DjcC29W > cf0K+NqgWk0vjW1Ez+XzOSBwki0EKz68MJxOf5lgwvs1yIt/Qjm62WCPmMdZRuNd > /WF2ojp+pCWD5Rat5rnpFricTQeUSEoJ7FF3y7fzpzVARRE1ujAgDOcACU0OgcC4 > ibHcYjGZQFU1S3d6F6mYKigxiMpEkCm/yj+TTs/sL9Nn0E/SsP630C2PJfTALQ8= > =umdv > -----END PGP SIGNATURE----- -- perl -Mre=debug -e "/just|another|perl|hacker/" http://www.nntp.perl.org/group/perl.makemaker/2013/03/msg3357.html Sun, 17 Mar 2013 13:02:58 +0000 ExtUtils::MakeMaker needs a release BY YOU by Michael G. Schwern -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi folks, 5.18.0 needs a MakeMaker release in order to fix some hash order bugs. Fortunately, its all already been patched in, its just a matter of us putting out an alpha, testing it, and then putting out a stable. And I'm not doing it. Been doing it too long, I'm just getting in the way. I'd like somebody to step up and offer to do the next stable. To that end, I've written down how to do it as best I can. Here's how you make a release, in general. https://github.com/Perl-Toolchain-Gang/ExtUtils-MakeMaker/wiki/How-To-Make-A-Release And here's the special considerations for a stable release. https://github.com/Perl-Toolchain-Gang/ExtUtils-MakeMaker/wiki/How-To-Make-A-Stable-Release An alpha release isn't much different from releasing any other CPAN module. A stable release involves a bit more rigor, but that's mostly reading CPAN Testers. And I'll be happy to be available as a resource and answer questions and hold hands. So, who wants to make a MakeMaker release? - -------- Original Message -------- Subject: ExtUtils::MakeMaker Date: Wed, 6 Mar 2013 10:38:29 -0500 From: Ricardo Signes <rjbs@cpan.org> To: Michael Schwern <schwern@pobox.com> Perl 5.18.0 is very likely to include improvements to hash traversal randomness. Right now, I believe the only part of the core left that has not been patched in blead is ExtUtils::MakeMaker. I see at https://github.com/Perl-Toolchain-Gang/ExtUtils-MakeMaker/commits/master that you've merged the work from Yves Oton to fix the bugs made visible by the changes to perl. Can we expect a release of EUMM more or less immediately? If not, we'll apply these patches in blead and create a _0x version number, which I am really not eager to do. - -- rjbs -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.18 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJRQ7/qAAoJEJxzdbWzfct37rUH/RwHuCJcDdFydWoltV5tHVq4 Rfk17hCN7ZZP3Bsd2GQT/0xRqqcycck4JGeHmtypZTtzCv9Eg3wlK67uWDasCogV iSs/wHSyffx/25C0JKZ7Wjag1U//G+HLQSopTXnAf9PSmTM3o5eE46E/9DjcC29W cf0K+NqgWk0vjW1Ez+XzOSBwki0EKz68MJxOf5lgwvs1yIt/Qjm62WCPmMdZRuNd /WF2ojp+pCWD5Rat5rnpFricTQeUSEoJ7FF3y7fzpzVARRE1ujAgDOcACU0OgcC4 ibHcYjGZQFU1S3d6F6mYKigxiMpEkCm/yj+TTs/sL9Nn0E/SsP630C2PJfTALQ8= =umdv -----END PGP SIGNATURE----- http://www.nntp.perl.org/group/perl.makemaker/2013/03/msg3356.html Sat, 16 Mar 2013 00:42:32 +0000 Re: C++ support by Michael G. Schwern Hey folks, This patch needs a review of somebody who understands XS and C++. Vladimir's done what looks to me like good work and I'd like to incorporate it so he does more. It's a small amount of change and shouldn't take long to look over. https://github.com/Perl-Toolchain-Gang/ExtUtils-MakeMaker/pull/45 Schwern On 2/17/13 11:51 AM, Vladimir Timofeev wrote: > Hi, please review pull request > https://github.com/Perl-Toolchain-Gang/ExtUtils-MakeMaker/pull/45 > > This patch make suffix of target for .xs file processing configurable > (defaults .c as before). But now we really can make process .xs files > to .cpp (for example) to support C++. > > Before this change I found two methods to work with C++ in .xs (both > are defective in some ways): > 1. Use CC=>'g++' parameter. > Bad, because C++ compiler not always gcc (it may be clang, microsoft > compiler or any other). > 2. Use CCFLAGS with "-x c++" it at least works for gcc and clang (may > be in others...) > Also not portable... > > Both solutions will not work if module has mix of .c and .cpp files... > because .c files also will be processed as c++ > > Solution with custom subclassing of EUMM works, but it is very ugly > (see my post about > https://plus.google.com/110578993733685478222/posts/fvisV7ktDtd ) > > So I propose to remove hardcode of .c suffix of xs targets and make it > configurable. It is small change, but will make life of C++ module > writers math easy. http://www.nntp.perl.org/group/perl.makemaker/2013/03/msg3355.html Fri, 15 Mar 2013 22:03:17 +0000 Re: Allow Perl to specify the list of headers that are XS dependencies by Craig A. Berry On Mon, Feb 25, 2013 at 1:08 AM, demerphq <demerphq@gmail.com> wrote: > On 19 February 2013 17:57, demerphq <demerphq@gmail.com> wrote: >> Currently EU::MM maintains a list of headers that will cause XS >> modules to be rebuilt. > > Hey guys, getting a release made of the current patches as well as > > https://github.com/Perl-Toolchain-Gang/ExtUtils-MakeMaker/pull/47 After verifying it causes no new test failures on VMS and OS X, I have merged this into master. > is holding up applying important changes to Perl for 5.18, and time is > wasting away... > > Please can you arrange a release ASAP. I don't think I can do that as it would require PAUSE access as well as a commit bit in the github repository. And it should really be Schwern who decides when to make a release. There haven't been that many changes since the last stable release so it should be reasonably safe to make another one. http://www.nntp.perl.org/group/perl.makemaker/2013/02/msg3354.html Wed, 27 Feb 2013 04:01:53 +0000 Re: Allow Perl to specify the list of headers that are XS dependencies by demerphq On 19 February 2013 17:57, demerphq <demerphq@gmail.com> wrote: > Currently EU::MM maintains a list of headers that will cause XS > modules to be rebuilt. Hey guys, getting a release made of the current patches as well as https://github.com/Perl-Toolchain-Gang/ExtUtils-MakeMaker/pull/47 is holding up applying important changes to Perl for 5.18, and time is wasting away... Please can you arrange a release ASAP. As a temporary work around I can make a core only release myself, but I would prefer not to. Thanks, Yves -- perl -Mre=debug -e "/just|another|perl|hacker/" http://www.nntp.perl.org/group/perl.makemaker/2013/02/msg3353.html Mon, 25 Feb 2013 07:08:10 +0000 Re: Allow Perl to specify the list of headers that are XS dependencies by Craig A. Berry On Wed, Feb 20, 2013 at 12:41 AM, demerphq <demerphq@gmail.com> wrote: >> I have pushed a patch in the yves/hv_h_split branch which should fix >> this. I plan to merge this branch to blead once 5.17.9 is out. >> >> Assuming it passes tests in core on VMS The core and its extensions build fine. The generated include file dependencies look like: # --- MakeMaker perldepend section: $(OBJECT) : $(PERL_INC)EXTERN.h, $(PERL_INC)INTERN.h, $(PERL_INC)XSUB.h $(OBJECT) : $(PERL_INC)av.h, $(PERL_INC)charclass_invlists.h, $(PERL_INC)cop.h $(OBJECT) : $(PERL_INC)cv.h, $(PERL_INC)dosish.h, $(PERL_INC)embed.h $(OBJECT) : $(PERL_INC)embedvar.h, $(PERL_INC)fakesdio.h, $(PERL_INC)fakethr.h $(OBJECT) : $(PERL_INC)feature.h, $(PERL_INC)form.h, $(PERL_INC)git_version.h $(OBJECT) : $(PERL_INC)gv.h, $(PERL_INC)handy.h, $(PERL_INC)hv.h $(OBJECT) : $(PERL_INC)hv_func.h, $(PERL_INC)inline.h, $(PERL_INC)intrpvar.h $(OBJECT) : $(PERL_INC)iperlsys.h, $(PERL_INC)keywords.h $(OBJECT) : $(PERL_INC)l1_char_class_tab.h, $(PERL_INC)malloc_ctl.h $(OBJECT) : $(PERL_INC)metaconfig.h, $(PERL_INC)mg.h, $(PERL_INC)mg_raw.h $(OBJECT) : $(PERL_INC)mg_vtable.h, $(PERL_INC)mydtrace.h, $(PERL_INC)nostdio.h $(OBJECT) : $(PERL_INC)op.h, $(PERL_INC)op_reg_common.h, $(PERL_INC)opcode.h $(OBJECT) : $(PERL_INC)opnames.h, $(PERL_INC)overload.h, $(PERL_INC)pad.h $(OBJECT) : $(PERL_INC)parser.h, $(PERL_INC)patchlevel.h, $(PERL_INC)perl.h $(OBJECT) : $(PERL_INC)perlapi.h, $(PERL_INC)perlio.h, $(PERL_INC)perliol.h $(OBJECT) : $(PERL_INC)perlsdio.h, $(PERL_INC)perlsfio.h, $(PERL_INC)perlvars.h $(OBJECT) : $(PERL_INC)perly.h, $(PERL_INC)pp.h, $(PERL_INC)pp_proto.h $(OBJECT) : $(PERL_INC)proto.h, $(PERL_INC)reentr.h, $(PERL_INC)regcharclass.h $(OBJECT) : $(PERL_INC)regcomp.h, $(PERL_INC)regexp.h, $(PERL_INC)regnodes.h $(OBJECT) : $(PERL_INC)scope.h, $(PERL_INC)sv.h, $(PERL_INC)thread.h $(OBJECT) : $(PERL_INC)time64.h, $(PERL_INC)time64_config.h $(OBJECT) : $(PERL_INC)uconfig.h, $(PERL_INC)unicode_constants.h $(OBJECT) : $(PERL_INC)unixish.h, $(PERL_INC)utf8.h, $(PERL_INC)utfebcdic.h $(OBJECT) : $(PERL_INC)util.h, $(PERL_INC)vmsish.h, $(PERL_INC)warnings.h which is really the same as it was before but with a bunch of includes that were missing from the list. I'm not sure the VMS-specific perldepend needs to be as different as it is. I will try to look into consolidating anything in it that can be with the one in MM_Unix. >> and the various core smoke >> environments I will push a patch to my EUMM fork on github and then >> file a Pull Request with the master repo (also on github). At which >> point we can update EUMM to the latest. >> >> I consider this patch and getting it or something equivalent merged >> into EUMM and perl core a 5.18 release blocker. > > Patch ID is: 0b340fe025421621146aac814778d80ad1df28c2 > > Cheers, > Yves > > > -- > perl -Mre=debug -e "/just|another|perl|hacker/" http://www.nntp.perl.org/group/perl.makemaker/2013/02/msg3352.html Thu, 21 Feb 2013 04:16:53 +0000 Re: Allow Perl to specify the list of headers that are XS dependencies by demerphq On 20 February 2013 07:40, demerphq <demerphq@gmail.com> wrote: > On 20 February 2013 05:47, Craig A. Berry <craig.a.berry@gmail.com> wrote: >> On Tue, Feb 19, 2013 at 1:00 PM, Nicholas Clark <nick@ccl4.org> wrote: >>> On Tue, Feb 19, 2013 at 05:57:12PM +0100, demerphq wrote: >>> >>>> I want to fix this. >>>> >>>> My current plan is to create a new module ExtUtils::PerlHeaders and >>>> then move the list there. >>>> >>>> EU::MakeMaker can then require this module, and if it exists use it >>>> for the header list. >>>> >>>> I am curious if anyone has any better suggestions or comments? >>>> >>>> The code involved is in lib/ExtUtils/MM_Unix.pm as the sub perldepend(): >>> >>> It's also in ExtUtils::MM_VMS as perldepend(). The intent is the same. The >>> list is different. >> >> Omissions are probably maintenance failures. The one addition I can >> think of at the moment is vmsish.h. >> >>> I noticed this problem a while back. I got some way towards addressing it - >>> I added a routine Config::header_files() to give the list of header files. >>> (So that's your ExtUtils::PerlHeaders already done) >>> >>> However, at the time I didn't have access to VMS, so couldn't fix up the >>> forked code to do it in one place. >>> >>> However, since then, it's been suggested by someone (forget who, sorry) - >>> why do we need a list? Given that the directory of installed header files >>> is under our control, why not just glob it, and depend on everything that >>> is <*.h> ? >>> >>> This seems a simpler suggestion than having to maintain an explicit list. >> >> You might be thinking of: >> >> <http://www.nntp.perl.org/group/perl.perl5.porters/2012/12/msg196426.htmll> > > I have pushed a patch in the yves/hv_h_split branch which should fix > this. I plan to merge this branch to blead once 5.17.9 is out. > > Assuming it passes tests in core on VMS and the various core smoke > environments I will push a patch to my EUMM fork on github and then > file a Pull Request with the master repo (also on github). At which > point we can update EUMM to the latest. > > I consider this patch and getting it or something equivalent merged > into EUMM and perl core a 5.18 release blocker. Patch ID is: 0b340fe025421621146aac814778d80ad1df28c2 Cheers, Yves -- perl -Mre=debug -e "/just|another|perl|hacker/" http://www.nntp.perl.org/group/perl.makemaker/2013/02/msg3351.html Wed, 20 Feb 2013 06:41:39 +0000 Re: Allow Perl to specify the list of headers that are XS dependencies by demerphq On 20 February 2013 05:47, Craig A. Berry <craig.a.berry@gmail.com> wrote: > On Tue, Feb 19, 2013 at 1:00 PM, Nicholas Clark <nick@ccl4.org> wrote: >> On Tue, Feb 19, 2013 at 05:57:12PM +0100, demerphq wrote: >> >>> I want to fix this. >>> >>> My current plan is to create a new module ExtUtils::PerlHeaders and >>> then move the list there. >>> >>> EU::MakeMaker can then require this module, and if it exists use it >>> for the header list. >>> >>> I am curious if anyone has any better suggestions or comments? >>> >>> The code involved is in lib/ExtUtils/MM_Unix.pm as the sub perldepend(): >> >> It's also in ExtUtils::MM_VMS as perldepend(). The intent is the same. The >> list is different. > > Omissions are probably maintenance failures. The one addition I can > think of at the moment is vmsish.h. > >> I noticed this problem a while back. I got some way towards addressing it - >> I added a routine Config::header_files() to give the list of header files. >> (So that's your ExtUtils::PerlHeaders already done) >> >> However, at the time I didn't have access to VMS, so couldn't fix up the >> forked code to do it in one place. >> >> However, since then, it's been suggested by someone (forget who, sorry) - >> why do we need a list? Given that the directory of installed header files >> is under our control, why not just glob it, and depend on everything that >> is <*.h> ? >> >> This seems a simpler suggestion than having to maintain an explicit list. > > You might be thinking of: > > <http://www.nntp.perl.org/group/perl.perl5.porters/2012/12/msg196426.htmll> I have pushed a patch in the yves/hv_h_split branch which should fix this. I plan to merge this branch to blead once 5.17.9 is out. Assuming it passes tests in core on VMS and the various core smoke environments I will push a patch to my EUMM fork on github and then file a Pull Request with the master repo (also on github). At which point we can update EUMM to the latest. I consider this patch and getting it or something equivalent merged into EUMM and perl core a 5.18 release blocker. Yves -- perl -Mre=debug -e "/just|another|perl|hacker/" http://www.nntp.perl.org/group/perl.makemaker/2013/02/msg3350.html Wed, 20 Feb 2013 06:40:52 +0000 Re: Allow Perl to specify the list of headers that are XS dependencies by Craig A. Berry On Tue, Feb 19, 2013 at 1:00 PM, Nicholas Clark <nick@ccl4.org> wrote: > On Tue, Feb 19, 2013 at 05:57:12PM +0100, demerphq wrote: > >> I want to fix this. >> >> My current plan is to create a new module ExtUtils::PerlHeaders and >> then move the list there. >> >> EU::MakeMaker can then require this module, and if it exists use it >> for the header list. >> >> I am curious if anyone has any better suggestions or comments? >> >> The code involved is in lib/ExtUtils/MM_Unix.pm as the sub perldepend(): > > It's also in ExtUtils::MM_VMS as perldepend(). The intent is the same. The > list is different. Omissions are probably maintenance failures. The one addition I can think of at the moment is vmsish.h. > I noticed this problem a while back. I got some way towards addressing it - > I added a routine Config::header_files() to give the list of header files. > (So that's your ExtUtils::PerlHeaders already done) > > However, at the time I didn't have access to VMS, so couldn't fix up the > forked code to do it in one place. > > However, since then, it's been suggested by someone (forget who, sorry) - > why do we need a list? Given that the directory of installed header files > is under our control, why not just glob it, and depend on everything that > is <*.h> ? > > This seems a simpler suggestion than having to maintain an explicit list. You might be thinking of: <http://www.nntp.perl.org/group/perl.perl5.porters/2012/12/msg196426.htmll> http://www.nntp.perl.org/group/perl.makemaker/2013/02/msg3349.html Wed, 20 Feb 2013 04:47:27 +0000 Re: Allow Perl to specify the list of headers that are XS dependencies by demerphq On 20 February 2013 03:39, Leon Timmermans <fawaka@gmail.com> wrote: > On Wed, Feb 20, 2013 at 3:29 AM, demerphq <demerphq@gmail.com> wrote: >> Cool thanks. Ill make EUMM try to use that. > > Actually, it seems MakeMaker already defines a $self->{PERL_SRC}, so > you could skip that part and just do something like > «$self->{PERL_SRC} || $self->catdir($Config{archlibexp}, 'CORE')». > It's tricky enough to get MakeMaker bootstrapped without adding even > more dependencies. Heh. If you want to take this task off me id be obliged... :-) Otherwise ill do my best to follow your recommendation here. cheers, Yves -- perl -Mre=debug -e "/just|another|perl|hacker/" http://www.nntp.perl.org/group/perl.makemaker/2013/02/msg3348.html Wed, 20 Feb 2013 02:41:32 +0000 Re: Allow Perl to specify the list of headers that are XS dependencies by Leon Timmermans On Wed, Feb 20, 2013 at 3:29 AM, demerphq <demerphq@gmail.com> wrote: > Cool thanks. Ill make EUMM try to use that. Actually, it seems MakeMaker already defines a $self->{PERL_SRC}, so you could skip that part and just do something like «$self->{PERL_SRC} || $self->catdir($Config{archlibexp}, 'CORE')». It's tricky enough to get MakeMaker bootstrapped without adding even more dependencies. Leon http://www.nntp.perl.org/group/perl.makemaker/2013/02/msg3347.html Wed, 20 Feb 2013 02:40:05 +0000 Re: Allow Perl to specify the list of headers that are XS dependencies by demerphq On 20 February 2013 03:25, Leon Timmermans <fawaka@gmail.com> wrote: > On Wed, Feb 20, 2013 at 3:21 AM, demerphq <demerphq@gmail.com> wrote: >> Is that going to work while building perl itself when that directory wont exist? > > No, ExtUtils::CBuilder->perl_inc knows exactly where to find those > headers in either condition. Cool thanks. Ill make EUMM try to use that. Yves -- perl -Mre=debug -e "/just|another|perl|hacker/" http://www.nntp.perl.org/group/perl.makemaker/2013/02/msg3346.html Wed, 20 Feb 2013 02:29:26 +0000 Re: Allow Perl to specify the list of headers that are XS dependencies by Leon Timmermans On Wed, Feb 20, 2013 at 3:21 AM, demerphq <demerphq@gmail.com> wrote: > Is that going to work while building perl itself when that directory wont exist? No, ExtUtils::CBuilder->perl_inc knows exactly where to find those headers in either condition. Leon http://www.nntp.perl.org/group/perl.makemaker/2013/02/msg3345.html Wed, 20 Feb 2013 02:26:20 +0000 Re: Allow Perl to specify the list of headers that are XS dependencies by demerphq On 20 February 2013 02:46, Leon Timmermans <fawaka@gmail.com> wrote: > On Wed, Feb 20, 2013 at 2:43 AM, demerphq <demerphq@gmail.com> wrote: >> Sounds great. I did some rudimentary poking but couldnt find the right >> dir... Do you happen to know the config var name? > > catdir($Config{archlibexp}, 'CORE') Is that going to work while building perl itself when that directory wont exist? Yves -- perl -Mre=debug -e "/just|another|perl|hacker/" http://www.nntp.perl.org/group/perl.makemaker/2013/02/msg3344.html Wed, 20 Feb 2013 02:21:24 +0000 Re: Allow Perl to specify the list of headers that are XS dependencies by Leon Timmermans On Wed, Feb 20, 2013 at 2:43 AM, demerphq <demerphq@gmail.com> wrote: > Sounds great. I did some rudimentary poking but couldnt find the right > dir... Do you happen to know the config var name? catdir($Config{archlibexp}, 'CORE') Leon http://www.nntp.perl.org/group/perl.makemaker/2013/02/msg3343.html Wed, 20 Feb 2013 01:46:36 +0000 Re: Allow Perl to specify the list of headers that are XS dependencies by demerphq On 19 February 2013 20:00, Nicholas Clark <nick@ccl4.org> wrote: > On Tue, Feb 19, 2013 at 05:57:12PM +0100, demerphq wrote: > >> I want to fix this. >> >> My current plan is to create a new module ExtUtils::PerlHeaders and >> then move the list there. >> >> EU::MakeMaker can then require this module, and if it exists use it >> for the header list. >> >> I am curious if anyone has any better suggestions or comments? >> >> The code involved is in lib/ExtUtils/MM_Unix.pm as the sub perldepend(): > > It's also in ExtUtils::MM_VMS as perldepend(). The intent is the same. The > list is different. > > > I noticed this problem a while back. I got some way towards addressing it - > I added a routine Config::header_files() to give the list of header files. > (So that's your ExtUtils::PerlHeaders already done) > > However, at the time I didn't have access to VMS, so couldn't fix up the > forked code to do it in one place. > > However, since then, it's been suggested by someone (forget who, sorry) - > why do we need a list? Given that the directory of installed header files > is under our control, why not just glob it, and depend on everything that > is <*.h> ? > > This seems a simpler suggestion than having to maintain an explicit list. Sounds great. I did some rudimentary poking but couldnt find the right dir... Do you happen to know the config var name? Yves -- perl -Mre=debug -e "/just|another|perl|hacker/" http://www.nntp.perl.org/group/perl.makemaker/2013/02/msg3342.html Wed, 20 Feb 2013 01:44:07 +0000 Re: Allow Perl to specify the list of headers that are XSdependencies by Nicholas Clark On Tue, Feb 19, 2013 at 05:57:12PM +0100, demerphq wrote: > I want to fix this. > > My current plan is to create a new module ExtUtils::PerlHeaders and > then move the list there. > > EU::MakeMaker can then require this module, and if it exists use it > for the header list. > > I am curious if anyone has any better suggestions or comments? > > The code involved is in lib/ExtUtils/MM_Unix.pm as the sub perldepend(): It's also in ExtUtils::MM_VMS as perldepend(). The intent is the same. The list is different. I noticed this problem a while back. I got some way towards addressing it - I added a routine Config::header_files() to give the list of header files. (So that's your ExtUtils::PerlHeaders already done) However, at the time I didn't have access to VMS, so couldn't fix up the forked code to do it in one place. However, since then, it's been suggested by someone (forget who, sorry) - why do we need a list? Given that the directory of installed header files is under our control, why not just glob it, and depend on everything that is <*.h> ? This seems a simpler suggestion than having to maintain an explicit list. Nicholas Clark http://www.nntp.perl.org/group/perl.makemaker/2013/02/msg3341.html Tue, 19 Feb 2013 19:00:34 +0000 Allow Perl to specify the list of headers that are XS dependencies by demerphq Currently EU::MM maintains a list of headers that will cause XS modules to be rebuilt. If this list is not up to date then modifications to core headers during core development do not cause extensions to be rebuilt. This means that we (perl core devs) cannot add new headers to Perl without build problems when they change and are used in XS code. I personally have encountered this in a few places, the regex engine and the hash code. I plan to merge a change which splits hv.h in two. Once this happens XS modules that depend on the perl hash function will break if the file is modified. I want to fix this. My current plan is to create a new module ExtUtils::PerlHeaders and then move the list there. EU::MakeMaker can then require this module, and if it exists use it for the header list. I am curious if anyone has any better suggestions or comments? The code involved is in lib/ExtUtils/MM_Unix.pm as the sub perldepend(): sub perldepend { my($self) = shift; my(@m); my $make_config = $self->cd('$(PERL_SRC)', '$(MAKE) lib/Config.pm'); push @m, sprintf <<'MAKE_FRAG', $make_config if $self->{PERL_SRC}; # Check for unpropogated config.sh changes. Should never happen. # We do NOT just update config.h because that is not sufficient. # An out of date config.h is not fatal but complains loudly! $(PERL_INC)/config.h: $(PERL_SRC)/config.sh -$(NOECHO) $(ECHO) "Warning: $(PERL_INC)/config.h out of date with $(PERL_SRC)/config.sh"; $(FALSE) $(PERL_ARCHLIB)/Config.pm: $(PERL_SRC)/config.sh $(NOECHO) $(ECHO) "Warning: $(PERL_ARCHLIB)/Config.pm may be out of date with $(PERL_SRC)/config.sh" %s MAKE_FRAG return join "", @m unless $self->needs_linking; push @m, q{ PERL_HDRS = \ $(PERL_INC)/EXTERN.h \ $(PERL_INC)/INTERN.h \ $(PERL_INC)/XSUB.h \ $(PERL_INC)/av.h \ $(PERL_INC)/config.h \ $(PERL_INC)/cop.h \ $(PERL_INC)/cv.h \ $(PERL_INC)/dosish.h \ $(PERL_INC)/embed.h \ $(PERL_INC)/embedvar.h \ $(PERL_INC)/fakethr.h \ $(PERL_INC)/form.h \ $(PERL_INC)/gv.h \ $(PERL_INC)/handy.h \ $(PERL_INC)/hv.h \ $(PERL_INC)/intrpvar.h \ $(PERL_INC)/iperlsys.h \ $(PERL_INC)/keywords.h \ $(PERL_INC)/mg.h \ $(PERL_INC)/nostdio.h \ $(PERL_INC)/op.h \ $(PERL_INC)/opcode.h \ $(PERL_INC)/patchlevel.h \ $(PERL_INC)/perl.h \ $(PERL_INC)/perlio.h \ $(PERL_INC)/perlsdio.h \ $(PERL_INC)/perlsfio.h \ $(PERL_INC)/perlvars.h \ $(PERL_INC)/perly.h \ $(PERL_INC)/pp.h \ $(PERL_INC)/pp_proto.h \ $(PERL_INC)/proto.h \ $(PERL_INC)/regcomp.h \ $(PERL_INC)/regexp.h \ $(PERL_INC)/regnodes.h \ $(PERL_INC)/scope.h \ $(PERL_INC)/sv.h \ $(PERL_INC)/thread.h \ $(PERL_INC)/unixish.h \ $(PERL_INC)/util.h $(OBJECT) : $(PERL_HDRS) } if $self->{OBJECT}; push @m, join(" ", values %{$self->{XS}})." : \$(XSUBPPDEPS)\n" if %{$self->{XS}}; join "\n", @m; } IMO this list has no business being hard coded into EU::MM now that EU::MM is not a core maintained module! Cheers, Yves -- perl -Mre=debug -e "/just|another|perl|hacker/" http://www.nntp.perl.org/group/perl.makemaker/2013/02/msg3340.html Tue, 19 Feb 2013 16:57:24 +0000 C++ support by Vladimir Timofeev Hi, please review pull request https://github.com/Perl-Toolchain-Gang/ExtUtils-MakeMaker/pull/45 This patch make suffix of target for .xs file processing configurable (defaults .c as before). But now we really can make process .xs files to .cpp (for example) to support C++. Before this change I found two methods to work with C++ in .xs (both are defective in some ways): 1. Use CC=>'g++' parameter. Bad, because C++ compiler not always gcc (it may be clang, microsoft compiler or any other). 2. Use CCFLAGS with "-x c++" it at least works for gcc and clang (may be in others...) Also not portable... Both solutions will not work if module has mix of .c and .cpp files... because .c files also will be processed as c++ Solution with custom subclassing of EUMM works, but it is very ugly (see my post about https://plus.google.com/110578993733685478222/posts/fvisV7ktDtd ) So I propose to remove hardcode of .c suffix of xs targets and make it configurable. It is small change, but will make life of C++ module writers math easy. -- Vladimir Timofeev <vovkasm@gmail.com> http://www.nntp.perl.org/group/perl.makemaker/2013/02/msg3339.html Mon, 18 Feb 2013 17:58:49 +0000 Re: How To Build A Perl Package Database by jvromans [Quoting Philippe Bruhat (BooK), on January 2 2013, 01:23, in "Re: How To Build A P"] > One issue I had when trying to store distributions with > Git::CPAN::Hook is that if a file does not change between two > versions of a distribution, then git won't "detect" it. This doesn't look like a problem if the approach is to checkout a particular tag corresponding to a specific install. -- Johan http://johan.vromans.org/seasons_greetings.html http://www.nntp.perl.org/group/perl.makemaker/2013/01/msg3338.html Wed, 02 Jan 2013 21:30:04 +0000 Re: How To Build A Perl Package Database by demerphq On 31 December 2012 19:38, Leon Timmermans <fawaka@gmail.com> wrote: > On Mon, Dec 31, 2012 at 6:50 PM, Jan Dubois <jand@activestate.com> wrote: >> Mostly I would prohibit sharing of directories between Perl installations, >> and even within a single installation, the sharing of directories between >> install locations. >> >> E.g. the default configuration right now has $Config{installbin} and >> $Config{installsitebin} pointing to the the same directory. This means that >> if you install ExtUtils::ParseXS from CPAN, you end up with the new version >> of the module in $Config{installsitelib}, but the xsubpp script installed >> into $Config{installsitebin} will overwrite the core version already in >> $Config{installbin} because they are the same directory. >> >> This means it is now impossible to remove the ExtUtils::ParseXS module from >> the "site" install location and reverting to the core version. >> >> Even if you don't care about "delete" functionality in your package manager, >> you may still want to preserve the integrity of core install. Otherwise it >> is possible that the package manager updates a package it relies upon itself >> that breaks the package manager. Then it is impossible to fix this situation >> for a regular user without doing a complete reinstall of Perl itself. >> >> For this reason the ActivePerl package manager explicitly removes the "site" >> directories from @INC and only uses the modules originally included in the >> distribution. > > I think that would clash with most vendor distributed perls (or at > least it does with both Debian and Red Hat). It would be nice if this > system was instead able to integrate with them instead of them nuking > it to prevent users from doing something stupid. FWIW I think that Perl should use one install format and the distros should not fight that. IMO a lot of MakeMakers problems come from trying to please too many people and ending up pleasing no one. Yves -- perl -Mre=debug -e "/just|another|perl|hacker/" http://www.nntp.perl.org/group/perl.makemaker/2013/01/msg3337.html Wed, 02 Jan 2013 14:18:21 +0000 Re: How To Build A Perl Package Database by Philippe Bruhat On Thu, Dec 20, 2012 at 05:48:10PM +0100, Johan Vromans wrote: > Tim Bunce <Tim.Bunce@pobox.com> writes: > > > A separate install database for each install location seems like the > > only workable approach. > > Store the complete distribution in a git repository? > One issue I had when trying to store distributions with Git::CPAN::Hook is that if a file does not change between two versions of a distribution, then git won't "detect" it. That was an issue for me, as I tried to create special tree objects for each distribution, so that "install this version of the distribution" would basically mean "apply the patch that creates the whole tree". This does not work if the tree does not contain files that haven't changed between versions. Depending on what you meant, that could be an issue for you too. -- Philippe Bruhat (BooK) When you double-cross a friend, you triple-cross yourself. (Moral from Groo The Wanderer #8 (Epic)) http://www.nntp.perl.org/group/perl.makemaker/2013/01/msg3336.html Wed, 02 Jan 2013 00:41:48 +0000 Re: How To Build A Perl Package Database by Leon Timmermans On Mon, Dec 31, 2012 at 6:50 PM, Jan Dubois <jand@activestate.com> wrote: > Mostly I would prohibit sharing of directories between Perl installations, > and even within a single installation, the sharing of directories between > install locations. > > E.g. the default configuration right now has $Config{installbin} and > $Config{installsitebin} pointing to the the same directory. This means that > if you install ExtUtils::ParseXS from CPAN, you end up with the new version > of the module in $Config{installsitelib}, but the xsubpp script installed > into $Config{installsitebin} will overwrite the core version already in > $Config{installbin} because they are the same directory. > > This means it is now impossible to remove the ExtUtils::ParseXS module from > the "site" install location and reverting to the core version. > > Even if you don't care about "delete" functionality in your package manager, > you may still want to preserve the integrity of core install. Otherwise it > is possible that the package manager updates a package it relies upon itself > that breaks the package manager. Then it is impossible to fix this situation > for a regular user without doing a complete reinstall of Perl itself. > > For this reason the ActivePerl package manager explicitly removes the "site" > directories from @INC and only uses the modules originally included in the > distribution. I think that would clash with most vendor distributed perls (or at least it does with both Debian and Red Hat). It would be nice if this system was instead able to integrate with them instead of them nuking it to prevent users from doing something stupid. Leon http://www.nntp.perl.org/group/perl.makemaker/2012/12/msg3335.html Mon, 31 Dec 2012 18:39:05 +0000 Re: How To Build A Perl Package Database by Jan Dubois On Fri, Dec 28, 2012 at 4:18 PM, Leon Timmermans <fawaka@gmail.com> wrote: > > Anyways, I just wanted to say that without putting some restrictions on > how > > modules (and corresponding scripts) can be installed, creating a package > > manager would seem to get even more complex than ExtUtils::Makemaker... > :) > > What kind of restrictions are you thinking about exactly? > Mostly I would prohibit sharing of directories between Perl installations, and even within a single installation, the sharing of directories between install locations. E.g. the default configuration right now has $Config{installbin} and $Config{installsitebin} pointing to the the same directory. This means that if you install ExtUtils::ParseXS from CPAN, you end up with the new version of the module in $Config{installsitelib}, but the xsubpp script installed into $Config{installsitebin} will overwrite the core version already in $Config{installbin} because they are the same directory. This means it is now impossible to remove the ExtUtils::ParseXS module from the "site" install location and reverting to the core version. Even if you don't care about "delete" functionality in your package manager, you may still want to preserve the integrity of core install. Otherwise it is possible that the package manager updates a package it relies upon itself that breaks the package manager. Then it is impossible to fix this situation for a regular user without doing a complete reinstall of Perl itself. For this reason the ActivePerl package manager explicitly removes the "site" directories from @INC and only uses the modules originally included in the distribution. Cheers, -Jan http://www.nntp.perl.org/group/perl.makemaker/2012/12/msg3334.html Mon, 31 Dec 2012 17:50:16 +0000 Re: How To Build A Perl Package Database by Leon Timmermans On Fri, Dec 28, 2012 at 2:44 AM, Jan Dubois <jand@activestate.com> wrote: > I wonder if it isn't time to deprecate all the complex install combinations > that may have made sense when hard disks was rather limited. > > In ActivePerl we enforce a pretty simplified install layout to be able to > create an intuitive package manager: > > no sharing of directories between different versions > every install area has its own bin, etc, lib, and html subdirectories > the etc subdirectory records packages installed in that location I think that ship already sailed, if only because different distributions have very different layouts. > Anyways, I just wanted to say that without putting some restrictions on how > modules (and corresponding scripts) can be installed, creating a package > manager would seem to get even more complex than ExtUtils::Makemaker... :) What kind of restrictions are you thinking about exactly? Leon http://www.nntp.perl.org/group/perl.makemaker/2012/12/msg3333.html Sat, 29 Dec 2012 00:19:18 +0000 Fwd: How To Build A Perl Package Database by Jan Dubois ---------- Forwarded message ---------- From: Jan Dubois <jand@activestate.com> Date: Thu, Dec 27, 2012 at 5:40 PM Subject: Re: How To Build A Perl Package Database To: Leon Timmermans <fawaka@gmail.com> On Thu, Dec 20, 2012 at 2:42 AM, Leon Timmermans <fawaka@gmail.com> wrote: > On Mon, Dec 17, 2012 at 9:36 AM, Tim Bunce <Tim.Bunce@pobox.com> wrote: > > A separate install database for each install location seems like the only > > workable approach. > > One minor complication of that is the strictest sense an "install > location" isn't all that well defined. Or perhaps I should say every > dist has 8 install locations with unspecified relationship on the > filesystem (lib, arch, bin, script, libdoc, bindoc, libhtml, binhtml). > In practice you may want to use lib *and* arch (they are never used > simultaneously, but stuff in lib may be shared over different perl > installations, contrary to arch which should be for one specific > install). > I wonder if it isn't time to deprecate all the complex install combinations that may have made sense when hard disks was rather limited. In ActivePerl we enforce a pretty simplified install layout to be able to create an intuitive package manager: - no sharing of directories between different versions - every install area has its own bin, etc, lib, and html subdirectories - the etc subdirectory records packages installed in that location Here is how PPM looks (on my Mac, but it is rather similar on Windows or Unix systems too): $ ppm area ┌────────┬──────┬────────────────────────────────────────┐ │ name │ pkgs │ lib │ ├────────┼──────┼────────────────────────────────────────┤ │ user* │ 38 │ /Users/jan/Library/ActivePerl-5.16/lib │ │ (site) │ n/a │ /usr/local/ActivePerl-5.16/site/lib │ │ (perl) │ 245 │ /usr/local/ActivePerl-5.16/lib │ └────────┴──────┴────────────────────────────────────────┘ $ ls /Users/jan/Library/ActivePerl-5.16 bin etc html lib $ ppm query XML ┌──────────────────────┬─────────┬─────────────────────────────────────────────────────────┬──────┐ │ name │ version │ abstract │ area │ ├──────────────────────┼─────────┼─────────────────────────────────────────────────────────┼──────┤ │ XML-NamespaceSupport │ 1.11 │ a simple generic namespace support class │ user │ │ XML-Parser │ 2.41-r1 │ A perl module for parsing XML documents │ perl │ │ XML-SAX │ 0.99 │ Simple API for XML │ user │ │ XML-SAX-Base │ 1.08 │ Base class for SAX Drivers and Filters │ user │ │ XML-Simple │ 2.20 │ Easily read/write XML (esp config files) │ perl │ │ XML-Stream │ 1.23 │ Creates an XML Stream connection and parses return data │ user │ └──────────────────────┴─────────┴─────────────────────────────────────────────────────────┴──────┘ (6 packages installed matching 'XML') $ ppm files XML-Stream /Users/jan/Library/ActivePerl-5.16/lib/XML/Stream.pm /Users/jan/Library/ActivePerl-5.16/lib/XML/Stream/IO/Select/Win32.pm /Users/jan/Library/ActivePerl-5.16/lib/XML/Stream/Namespace.pm /Users/jan/Library/ActivePerl-5.16/lib/XML/Stream/Node.pm /Users/jan/Library/ActivePerl-5.16/lib/XML/Stream/Parser.pm /Users/jan/Library/ActivePerl-5.16/lib/XML/Stream/Parser/DTD.pm /Users/jan/Library/ActivePerl-5.16/lib/XML/Stream/Tree.pm /Users/jan/Library/ActivePerl-5.16/lib/XML/Stream/XPath.pm /Users/jan/Library/ActivePerl-5.16/lib/XML/Stream/XPath/Op.pm /Users/jan/Library/ActivePerl-5.16/lib/XML/Stream/XPath/Query.pm /Users/jan/Library/ActivePerl-5.16/lib/XML/Stream/XPath/Value.pm /Users/jan/Library/ActivePerl-5.16/lib/auto/XML/Stream/.packlist $ ppm remove XML-NamespaceSupport XML-NamespaceSupport: required by XML-SAX ppm remove failed: No packages uninstalled Having a separate perl/bin and perl/site/bin and perl/vendor/bin is somewhat inconvenient for adding things to the $PATH, but it makes it possible to install an updated core package into the site directory, and later uninstall it without breaking the original core version. We don't use perl/vendor but instead merge all pre-installed packages into the core directories to keep $PATH a little shorter. Anyways, I just wanted to say that without putting some restrictions on how modules (and corresponding scripts) can be installed, creating a package manager would seem to get even more complex than ExtUtils::Makemaker... :) Cheers, -Jan http://www.nntp.perl.org/group/perl.makemaker/2012/12/msg3332.html Fri, 28 Dec 2012 01:45:07 +0000 Re: How To Build A Perl Package Database by Tim Bunce On Thu, Dec 20, 2012 at 11:42:06AM +0100, Leon Timmermans wrote: > On Mon, Dec 17, 2012 at 9:36 AM, Tim Bunce <Tim.Bunce@pobox.com> wrote: > > A separate install database for each install location seems like the only > > workable approach. > > One minor complication of that is the strictest sense an "install > location" isn't all that well defined. Or perhaps I should say every > dist has 8 install locations with unspecified relationship on the > filesystem (lib, arch, bin, script, libdoc, bindoc, libhtml, binhtml). > In practice you may want to use lib *and* arch (they are never used > simultaneously, but stuff in lib may be shared over different perl > installations, contrary to arch which should be for one specific > install). Good point. Strengthening the definition of an "install location" would be good. Supporting only the common subset of possible layouts seems reasonable. Tim. http://www.nntp.perl.org/group/perl.makemaker/2012/12/msg3331.html Thu, 20 Dec 2012 17:56:17 +0000 Re: How To Build A Perl Package Database by Johan Vromans Tim Bunce <Tim.Bunce@pobox.com> writes: > A separate install database for each install location seems like the > only workable approach. Store the complete distribution in a git repository? -- Johan http://www.nntp.perl.org/group/perl.makemaker/2012/12/msg3330.html Thu, 20 Dec 2012 16:48:54 +0000 Re: How To Build A Perl Package Database by Leon Timmermans On Mon, Dec 17, 2012 at 9:36 AM, Tim Bunce <Tim.Bunce@pobox.com> wrote: > A separate install database for each install location seems like the only > workable approach. One minor complication of that is the strictest sense an "install location" isn't all that well defined. Or perhaps I should say every dist has 8 install locations with unspecified relationship on the filesystem (lib, arch, bin, script, libdoc, bindoc, libhtml, binhtml). In practice you may want to use lib *and* arch (they are never used simultaneously, but stuff in lib may be shared over different perl installations, contrary to arch which should be for one specific install). Leon http://www.nntp.perl.org/group/perl.makemaker/2012/12/msg3329.html Thu, 20 Dec 2012 10:42:38 +0000 Re: How To Build A Perl Package Database by Tim Bunce On Mon, Dec 17, 2012 at 08:23:51PM +0100, Ask Bjørn Hansen wrote: > On Dec 17, 2012, at 9:36, Tim Bunce <Tim.Bunce@pobox.com> wrote: > > > A separate install database for each install location seems like the only > > workable approach. > > It seems to me that the database indeed will have to be[1] "per library root" and the tools using the database will need to know to do lookups everywhere and merge the results. > > Ask > > [1] Where "will have to" means "to fit *my* universe". Yes, that's what I had in mind. Tim. http://www.nntp.perl.org/group/perl.makemaker/2012/12/msg3328.html Thu, 20 Dec 2012 10:38:37 +0000 Re: How To Build A Perl Package Database by Adam Kennedy Packlist 2.0 MYMETA + installed file details ? On Dec 17, 2012 12:36 AM, "Tim Bunce" <Tim.Bunce@pobox.com> wrote: > On Sun, Dec 16, 2012 at 04:53:49PM -0800, Michael G Schwern wrote: > > On 2012.12.16 11:57 AM, Leon Timmermans wrote: > > > > >> * Where to put the database? What about non-standard install > locations? > > > >> Another is to have a separate install database for non-standard > install > > >> locations. > > A separate install database for each install location seems like the only > workable approach. > > > >> This makes sense to me, but it brings in the sticky problem > > >> of having to merge install databases. Sticky, but still a SMOP. > Once you > > >> have to implement merging anyway, it now makes sense to have an > install > > >> database for each install location. One for core. One for vendor. > One for > > >> perl. And one for each custom location. This has a lot of > advantages to > > >> better fit how Perl layers module installs. > > >> > > >> * allows separation of permissions > > >> * allows queries of what's installed based on what's in @INC > > Perhaps that could be taken one step further: one per installed > distribution. > > Then, what's kept at each install location is a cached summary of what's > installed below it. One that can be cross-checked against the individual > distribution 'databases' and rebuilt from it. > > That seems more robust against various kinds of 'damage'. > > > >> That second one is important. When a normal user queries the > database, they > > >> want to get what's installed in the standard library location. When a > > >> local::lib user queries the database, they want to get what's > installed in the > > >> standard library locations AND their own local lib. > > I.e., the default view is "what's installed in my @INC". > > > > The combination of these is problematic. You might upgrade EU::Install > > > in your local module path, but not have write permissions on the > > > system paths. In practice, we might have to support all our older > > > versions :-| > > > > Erg, good point. That very likely scenario is definitely going to > require > > some thought. > > *nods* > > Here's where "one install database per distribution with a cache > database at the install location" offers another benefit. > The "per distribution install database" can be kept in a very simple > plain text format that targets readability and future-proofing, > while the "cache database at the install location" can target > performance. > > If an install location has an incompatible version of the db, > the per distribution dbs could be read instead. That's slow but workable > and seems reasonable for that presumably uncommon case. > I can think of a few further options as well. > > Tim. > http://www.nntp.perl.org/group/perl.makemaker/2012/12/msg3327.html Mon, 17 Dec 2012 19:34:53 +0000 Re: How To Build A Perl Package Database by Ask Bjørn Hansen On Dec 17, 2012, at 9:36, Tim Bunce <Tim.Bunce@pobox.com> wrote: > A separate install database for each install location seems like the only > workable approach. It seems to me that the database indeed will have to be[1] "per library root" and the tools using the database will need to know to do lookups everywhere and merge the results. Ask [1] Where "will have to" means "to fit *my* universe". http://www.nntp.perl.org/group/perl.makemaker/2012/12/msg3326.html Mon, 17 Dec 2012 19:34:50 +0000 Re: How To Build A Perl Package Database by Leon Timmermans On Mon, Dec 17, 2012 at 1:53 AM, Michael G Schwern <schwern@pobox.com> wrote: > I was thinking about what you said about packlists, and I wonder how much > information one could scrape out of them. Would it be enough to reconstruct > at least that a group of files belongs to a release? That would be enough to > be able to fully uninstall a requested module. For example, if the user asks > to uninstall ExtUtils::MakeMaker the database could have seen that > ExtUtils/MakeMaker.pm was in a packlist together with ExtUtils/MM_Unix.pm and > so on and uninstall them. Probably given their original purpose was to > provide an uninstaller. You can use them to uninstall (I assume that's the reason why Debian disables them for vendor packages). It can get a little messy when modules are split or some such, but that's relatively rare anyway. > Also what's with this .meta directory I see popping up? I missed a memo. AFAIK that's cpanminus specific. AFAIK it stores meta information so that carton can use it. Ask Miyagawa for the details. > That's a good point about Storable. JSON requires a dependency. > ExtUtils::Install could bundle JSON::PP, but it would be simpler to use > Data::Dumper. It makes de/serialization faster and simpler. The main > disadvantage is its only readable by Perl, but that's ok since this pile of > DBM files will be opaque to everything but the Perl API. Too much of a mess > to contemplate otherwise. JSON::PP is already in modern perl releases, so it only requires a dependency on older perls. Leon http://www.nntp.perl.org/group/perl.makemaker/2012/12/msg3325.html Mon, 17 Dec 2012 19:28:58 +0000 Re: How To Build A Perl Package Database by Leon Timmermans On Sat, Dec 15, 2012 at 11:59 PM, Michael G Schwern <schwern@pobox.com> wrote: > We have a lot of serious problems because we lack a database of installed > distributions, releases and files. There are serious problems with > implementing one given A) the limitations of the standard Perl install and B) > wedging it into existing systems. But I think I have a solution. Its similar > to how meta data was slipped into the ecosystem without requiring authors to > rewrite their releases or install a bunch of extra modules. It just happens > as part of the normal CPAN module upgrade process. > > I've been thinking that a minimal package database could be created by putting > some hooks into ExtUtils::Install::install(), which every Perl build system > ultimately uses, to record what gets installed. That way when > ExtUtils::Install is upgraded, the user gets a build database without > upgrading everything else. > > This would be a fairly straight forward process at install time... > > 1) Copy everything to a temp directory > 2) Record everything in that temp directory > 3) Copy everything from temp into the real location > > You could probably optimize this by skipping the copy to temp and just have > install() record stuff as it goes by, but this is the dumb, simple, robust way > to do it. > > Storage is a problem. The only reliable "database" Perl ships with is DBM, an > on disk hash, so we can't get too fancy. It might take several DBM files, but > this is enough to record information and do simple queries. What are those > queries? > > * What version of the database is this? > * What distributions are installed? > * What release of a distribution is installed? > * What files are in that release? > * What version is that release? > * What location was a release installed into? (core, vendor, site, custom) > * What are the checksums of those files? > > And the basic operations we need to support. > > * Add a release (ie. install). > * Delete a release (and its files). > * Delete an older version of a release (as part of install). > * Delete an older version of a release, only if its in the same release > location. This is so CPAN installs don't delete vendor installed modules. > * Verify the files of a release. > * List distributions/releases installed. > > It would also store the MYMETA data which gives us a lot of information (such > as dependencies) for free. I can agree with all of that. Actually, starting a discussion about this was on my todo-list for the last QA hackathon but I didn't get around to it. Ideally, it should replace not only packlists but also perllocal > This is all totally doable, and efficient enough, with a small pile of DBM > files and Storable. Where to put the database is a bit more complicated, see > the list of open problems below. Given that Storable's format isn't forward-compatible, something more stable such as JSON would be more appropriate. > There's lots and lots and lots of additional information which could be stored > and queries and operations to allow, but if we can get the basics working > it'll allow a heap of new solutions. And I think this is a SMOP. > > > Future possibilities include... > > * Auto-upgrade to SQLite if ExtUtils::Install::DB::SQLite is installed. > > If a special module is installed we can offer SQLite support (or whatever) for > a more advanced database. At install time it would copy the existing DBM > system into its own database. > > In general, more functionality can be added as more optional (or bundled) > dependencies are available to the system. Through it all the basic DBM > database would continue to be redundantly maintained to provide a fallback > should those optional modules break or go away. Having a proper database would be really nice, but I'm not sure if it's going to be worth the hassle if we have a robust system already. > * Upgrading the database. > > I'd like to put some thought into how things are laid out initially to avoid a > lot of major revisions, and thought into what information should be recorded > so its available later, but eventually we're going to want to change the > "schema", such as it is with DBM files. > > I figure this can happen as part of upgrading ExtUtils::Install. It checks > what version of the database you have and performs the necessary transforms to > bring it up to the current version. We know how to do this, just have to keep > it in mind and remember to implement it. > > * Where to put the database? What about non-standard install locations? > > $Config{archlib} would seem the obvious location, but it presents a > permissions problem. If a non-root user installs into their home > directory, you don't want them needing root to write to the installation > database. There's several ways to deal with this. > > One is to simply not record non-standard install locations, but this loses > data and punishes all those local::lib users out there. > > Another is to have a separate install database for non-standard install > locations. This makes sense to me, but it brings in the sticky problem > of having to merge install databases. Sticky, but still a SMOP. Once you > have to implement merging anyway, it now makes sense to have an install > database for each install location. One for core. One for vendor. One for > perl. And one for each custom location. This has a lot of advantages to > better fit how Perl layers module installs. > > * allows separation of permissions > * allows queries of what's installed based on what's in @INC > > That second one is important. When a normal user queries the database, they > want to get what's installed in the standard library location. When a > local::lib user queries the database, they want to get what's installed in the > standard library locations AND their own local lib. The combination of these is problematic. You might upgrade EU::Install in your local module path, but not have write permissions on the system paths. In practice, we might have to support all our older versions :-| > Not perfect, but gets us off the ground. Its not a great database, but it > does the important job of recording the critical install-time data for later > use. Its implementable within the current system. It doesn't require a bunch > of dependencies, just one upgrade. It works with most existing module > releases. It solves a major design problem with the Perl module system. > > I think it's a Simple(?!) Matter Of Programming in ExtUtils::Install to get it > off the ground. IMO the most important bit of coordination is putting some > thought into what the basic database should look like so we don't have to > worry about complicated upgrades later. I'm not sure it's as simple as you make it sound, but it is a good idea nonetheless. Leon http://www.nntp.perl.org/group/perl.makemaker/2012/12/msg3324.html Mon, 17 Dec 2012 19:28:33 +0000 Re: How To Build A Perl Package Database by Leon Timmermans On Sun, Dec 16, 2012 at 6:34 PM, Johan Vromans <jvromans@squirrel.nl> wrote: > Debian, and most other systems have decent package- and install > managers. *They* maintain the database with installed distributions, > releases and files. The only good approach for us is to play with them. > > So, an enhanced META.yaml or whatsoever may be a good idea, but *only* > to generate a deb control file, or rpm spec file, or innosetup file and > so on. > > It is no longer neccessary to handle everything ourselves. We're not > alone anymore. There are many ways to deploy stuff, not everyone uses rpm/deb, there are good reasons not to do so: for starters it assumes you have root privileges. Leon http://www.nntp.perl.org/group/perl.makemaker/2012/12/msg3323.html Mon, 17 Dec 2012 19:28:31 +0000 Re: How To Build A Perl Package Database by Johan Vromans Michael G Schwern <schwern@pobox.com> writes: > We have a lot of serious problems because we lack a database of > installed distributions, releases and files. No, that is not our problem. Our problem is that we want to handle it ourselves. This may have been a good approach in the dark ages, but nowadays there are better solutions. For example, on Sat, 15 Dec 2012 14:15:24 -0800 you wrote: > When it comes to packaging, my first thought is always WDDD? What Does > Debian Do? They usually do it right. Debian, and most other systems have decent package- and install managers. *They* maintain the database with installed distributions, releases and files. The only good approach for us is to play with them. So, an enhanced META.yaml or whatsoever may be a good idea, but *only* to generate a deb control file, or rpm spec file, or innosetup file and so on. It is no longer neccessary to handle everything ourselves. We're not alone anymore. -- Johan http://www.nntp.perl.org/group/perl.makemaker/2012/12/msg3322.html Mon, 17 Dec 2012 19:28:29 +0000 Re: How To Build A Perl Package Database by Alberto Simoes > > Thoughts? > Seems a good idea. Anybody wiling to submit a TPF grant to work on this? O:-) http://www.nntp.perl.org/group/perl.makemaker/2012/12/msg3321.html Mon, 17 Dec 2012 19:28:22 +0000 Re: How To Build A Perl Package Database by demerphq On 17 December 2012 01:53, Michael G Schwern <schwern@pobox.com> wrote: > On 2012.12.16 11:57 AM, Leon Timmermans wrote: >> I can agree with all of that. Actually, starting a discussion about >> this was on my todo-list for the last QA hackathon but I didn't get >> around to it. Ideally, it should replace not only packlists but also >> perllocal > > I was thinking about what you said about packlists, and I wonder how much > information one could scrape out of them. Would it be enough to reconstruct > at least that a group of files belongs to a release? That would be enough to > be able to fully uninstall a requested module. For example, if the user asks > to uninstall ExtUtils::MakeMaker the database could have seen that > ExtUtils/MakeMaker.pm was in a packlist together with ExtUtils/MM_Unix.pm and > so on and uninstall them. Probably given their original purpose was to > provide an uninstaller. > > Also what's with this .meta directory I see popping up? I missed a memo. > > >>> This is all totally doable, and efficient enough, with a small pile of DBM >>> files and Storable. Where to put the database is a bit more complicated, see >>> the list of open problems below. >> >> Given that Storable's format isn't forward-compatible, something more >> stable such as JSON would be more appropriate. > > That's a good point about Storable. JSON requires a dependency. > ExtUtils::Install could bundle JSON::PP, but it would be simpler to use > Data::Dumper. It makes de/serialization faster and simpler. The main > disadvantage is its only readable by Perl, but that's ok since this pile of > DBM files will be opaque to everything but the Perl API. Too much of a mess > to contemplate otherwise. IMO the question is whether you want the data human readable or not. If you dont care then use Sereal as a replacement for Storable. Same feature set pretty much except it is faster and produces smaller output. Yves -- perl -Mre=debug -e "/just|another|perl|hacker/" http://www.nntp.perl.org/group/perl.makemaker/2012/12/msg3320.html Mon, 17 Dec 2012 13:21:37 +0000