develooper Front page | perl.perl5.porters | Postings from November 2009

PATCH make mktables run less often

Thread Previous | Thread Next
From:
karl williamson
Date:
November 23, 2009 15:54
Subject:
PATCH make mktables run less often
Message ID:
4B0B2091.4020201@khwilliamson.com
Attached is a patch to cut down the frequency of mktables running.  It 
turns out the problem was a combination of both Makefile and mktables. 
I didn't realize that the files I removed were listed as dependencies in 
Makefile, so it always thought it was out-of-date, and called mktables. 
Part of the old code I hadn't changed caused mktables to remove all the 
files when called with the -makelist option.  This code is no longer 
necessary, so I removed it.

I also changed Makefile to always call mktables, to let it decide if it 
actually should recompile.  I just commented it out for easy reversal. 
The problem is that mktables depends on almost 800 files, located in the 
To and lib subdirectories, and Makefile doesn't know about them all; and 
they change from Unicode release to release.  I'm assuming mktables.lst 
was created because of the issues of trying to get them all known by 
Makefile.

I did a search for the removed files and removed the only other 
reference to them (in .gitignore); except I see them in Makefiles for 
other platforms.  I assume these get automatically generated from the 
changed Makefile.SH

I also changed some typos in comments, rephrased a little of the 
documentation, removed a no-longer-used subroutine, and fixed another 
bug, in which it would fail to run if there was no mktables.lst at all, 
added -p option to its call.

I'll work soon on a wrapper to the .t.  I'm wondering though about the 
generated pod.  It should be put into the toc, etc.  Would a zero-length 
place-holder that mktables overwrites be the right solution?

Rafael Garcia-Suarez wrote:
> 2009/11/23 karl williamson <public@khwilliamson.com>:
>> There needs to be some discussion about how mktables gets called.  It
>> generates all three of t/re/uniprops.t, lib/unicore/mktables.lst, and
>> pod/perluniprops.pod (besides Heavy.pl and all the tables).  I meant to
>> patch Makefile to cause all three of these to be generated, but didn't
>> realize that I should really patch Makefile.SH instead, so my changes were
>> lost.  But let's figure out what they should be before I submit a revision.
> 
> As far as I can tell it will only generate perluniprops.pod when passed
> "-P pod", and t/re/uniprops.t when passed "-T t/re/uniprops.t".
> 
>> These three files are essentially static from Unicode release to Unicode
>> release (as are all the tables, except if someone changes mktables itself to
>> generate different things.)  The reason you saw mktables.lst change is
>> because it has a time stamp in it (that should have been the only
>> difference, and it was because I did a final make test, just to be sure,
>> after I had committed, but before I delivered; sorry). mktables.lst retains
>> the same format as it traditionally has had.
> 
> No, I saw other changes, notably in the list of output files, since by
> default it doesn't generate the two files I just mentioned.
> 
>> I was attempting to add the generation of all three of them to Makefile
>> because of what Nicolas said several months ago about wanting to generate
>> the pod file every time, as it was one less thing for the pumpking to worry
>> about when getting things out the door.  I figured that we should be
>> consistent, so made that the behavior for all three. But this is a change
>> for mktables.lst.  I don't know when that got generated in the past. Perhaps
>> it is something on the pumpking's list?
> 
> I would favor generating everything by default, and making uniprops.t
> be, as Nicholas said, a wrapper around a generated file (so uniprops.t
> is listed in the MANIFEST and is picked up by the test harness.)
> 
> I would also make -p the default (show progress of mktables during run),
> except maybe when stdout isn't a tty.
> 
>> I see a couple of possibilities.  One is to generate all three every time.
>>  I guess that means they should come off the MANIFEST.  But mktables.lst has
>> always been generated, and has been in MANIFEST.
>>
>> The other option I see is to generate them by hand when a new release of
>> Unicode is installed, and to put them in the MANIFEST.  But if someone
>> changes mktables to generate different files, these would have to be
>> regenerated at that time as well, and someone might forget.
>>
>> What are people's opinions?
>>
>> Somewhat related, this new mktables takes longer than the old one to
>> execute.  In part it is because it's doing a lot more things; in part it's
>> because it uses more object-oriented, inside-out-hash, functions and is
>> getting them in pure Perl.  (Nicholas' suggestion from last week helped
>> quite a bit here, though).  Even on my this-year's-model Linux box, it seems
>> like it takes forever.  (I compile my development Perls without
>> optimization; it seems like it runs twice as fast on a 5.10 Perl that has
>> optimization, but I haven't done any benchmarking.)  Adding the -p option to
>> the call to show progress helps me see how things are progressing, and
>> realize how much work it's actually doing.  Should I do that in the
>> Makefile?
>>
>> I think this longer compilation time would be perfectly acceptable if it
>> didn't actually happen very often, like with the Encode tables.  And there
>> is no reason to compile more frequently than the Encode tables. As I
>> mentioned, the underlying tables are static.  New Unicode releases come out
>> about once a year or so.  But Makefile is removing outputs before testing
>> whether it should call mktables, and of course, decides it should.  I
>> haven't sat down to figure out how this is happening yet; nor have I
>> investigated when the Encode tables get cleaned; is it a normal clean, I
>> don't think so; maybe its a realclean.  But I think mktables' outputs
>> shouldn't be cleaned at a lower level than Encode's. Comments?
>>
>>
> 


Thread Previous | Thread Next


nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About