Front page | perl.module-authors |
Postings from June 2008
Re: CPANTS has_test_pod* metrics
From: Ricardo SIGNES
June 10, 2008 11:33
Re: CPANTS has_test_pod* metrics
Message ID: 20080610183313.GA10669@minion169.office.icgroup.com
* Dave Rolsky <firstname.lastname@example.org> [2008-06-10T13:23:09]
> The point is that you should ship a dist that is complete enough for an
> end-user to untar it, hack on the distro, run all the tests, and send you a
See, I think this is a lousy goal. More and more, I am not building my
distribution tarballs like this:
$ perl Makefile.PL
$ make manifest && make disttest && make dist && cpan-upload Dist.tgz
When I do that, I have to do all kinds of obnoxious boring crap like include
boilerplate POD, have a build tool that can run my release tests (but not let
them get run by installing users), make sure that my changelog is up to date,
update $VERSION in every module, and so on. It sucks.
Sure, things like ShipIt and Module::Release and Perl::Version's perl-reversion
help with a lot of this. They just don't do as much as I want.
I want my build process to turn the code in my repository into a distribution
for the CPAN. In other words, I supply:
* the code and documentation specifically about that code
* tests that all installers should run
* tests that should be run before releasing
* tests that I should run all the time when developing, but others shouldn't
* a few pieces of metadata like license info and prereqs
Then I get a tarball that has added a $VERSION to all my code, updated the POD
=head1 VERSION section, made a Makefile.PL that includes the right prereqs, and
made a MANIFEST that I know will be accurate.
This saves me a huge amount of time. It's enormous. Further, the more work
that I turn over to build automation, the more time I save as all of my dists
can drop code they don't need.
With this in mind, the idea that "my dist should look just like a snapshot of
my repo" becomes insane. So why do I have to ship the darn pod-coverage.t? I
never, ever want anybody to run it. Okay, I can put it in ./xt and not tell
anything about it -- but then what's the point? The point is to lie to CPANTS.
I am happy to accept patches against my dists, even if I have to do a bit of
work to make them work against my repo. I'd much rather get patches against my
repo, but I *know* that I get few enough patches that the time I save myself
in doing my own daily work is much greater than the time I might spend fixing
The Test::NoWarnings kwalitee check is just an exaggerated form of "includes
a POD coverage test." It really seems to mean, "The user claims to be using a
tool that solves a problem -- but we don't know if he is, or whether he has
solved the problem a different way."
I really, really like CPANTS. It has helped me notice many problems. As it
gets more metrics, though, I like it less and less. It's a lot of noise, and
not enough signal.
I think that a lot of the tests could do with a shift in mindset. Instead of
has_test_pod_coverage, maybe known_pod_coverage_tool, or even
asserts_pod_covered. Make it clear that the test is for, "the author has
attempted to let us know that he has documented everything," not "the author
uses a specific tool to do this job."
I think, too, that if it were easier for authors (or anyone) to change how he
sees the data -- removing all the points he doesn't care about and viewing only
the ones he does -- people would complain less. They'd check which tests they
cared about and they'd have a very useful report.
People might keep complaining about whatever the official report for the
scoreboard is, but I'm long past caring about staying at the top of the board.
I'd rather just stay on top of my workload.