develooper Front page | perl.perl5.porters | Postings from March 2006

Module::Compile and .pmc

From:
Audrey Tang
Date:
March 13, 2006 15:31
Subject:
Module::Compile and .pmc
Message ID:
441600AA.4070201@autrijus.org
Here is some clarification on my .pmc proposals, based on a #p5p
discussion with Robert and Nicholas.

= Preamble

To explain this, I'd need to explain Module::Compile first.
Its intended users are modules like Inline.pm and Switch.pm.

The module documentation is at:
    http://search.cpan.org/dist/Module-Compile/lib/Module/Compile.pm

It works like Filter::Simple, except it allows multiple source filters
to be composed, and the result cached.  Moreover, the generated .pmc
file can be deployed along with the .pm file.  Because the source
filter is never triggered at runtime, it removes the dependency of
the filtering module, and avoids the penalty of running source
filtering at the user site.

Note that the code _using_ the filtering modules do not need
to be changed; if Switch.pm switches to use Module::Compile
instead of Filter::Simple, existing Switch.pm-using code needs
not be modified.

By default, Module::Compile writes a self-validating BEGIN block
in .pmc, based on a running 32-bit checksum.  The idea is to embed
validation information in the .pmc file, instead of relying on the
filesystem's mtime, which often gets destroyed by tools such as
File::Copy, Subversion, etc.

Currently, to get past 5.6~5.8's mtime check, we use Module::Install
to generate a MakeMaker postamble that touches the .pmc files before
they get copied to blib, ensuring that they will have a newer mtime
than the .pm counterparts.

= 1: Disabling the mtime<mtime check on bleadperl.

This is suggested on the ground that there is currently nothing
in the core or CPAN toolchain that depends on the mtime<mtime
behaviour that we are aware of.

More over, the mtime<mtime documentation was added on 5.8.1 by rgs;
at that time, nowhere else mentions the .pmc mechanism.  Early
versions of B::Bytecode contains .pmc in one of its examples,
but did not mention anything about the mtime guarantee.

Given that Module::Compile already generates a stronger check
in the .pmc file, then the mtime<mtime check consumes stat()
calls, may cause irregular problems due to broken filesystem/tools,
and adds no extra safety.

= 2: Changing the mtime<mtime check to mtime<=mtime on maintperl.

This is purely to counter FAT's 2-second granularity, as on FAT
the .pm and .pmc are often of the same mtime, even though .pmc was
generated or checked-out a fraction of second afterwards.

The problem has happened to any system that mounts FAT.  I don't consider
it critical enough to warrant maintperl really, but if it's deemed a
bugfix, then maybe it's okay.

= Postscript

Robert points out that if we invent a new cache extension (e.g. ".pmx"),
which collects a unified set of metadata (mtime, checksum, etc) and
store them inside the .pmx file, much like Python's .pyc does, then
the perl5 runtime can perform a static check, instead of yielding
the control to the generator of .pmc (currently only Module::Compile).

If implemented, then we can obsolete .pmc altogether, and switch to
this new format.  I am fine with this, if it turns out that there is
existing tools that relies on the mtime<mtime guarantee, and cannot
recover from the change in 5.10.

Finally, there may be concerns about using the .pmc extension, as Parrot
also uses them internalls for Parrot Magic Cookie source files.

The .pmc feature is added to Perl 5 on 1999; Parrot did not appear
until April 2001.  Moreover, Parrot's .pmc files is never installed;
they are preprocessed to generate .c and .h files.




nntp.perl.org: Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at ask@perl.org | Group listing | About