Front page | perl.perl5.porters |
Postings from July 2011
on changing the API without breaking the build
Thread Next
From:
Craig A. Berry
Date:
July 17, 2011 17:03
Subject:
on changing the API without breaking the build
Message ID:
CA+vYcVyJKRZH-QXsfj64yEXSX86QtAqqHzQUt+uq_+KVV3V4pQ@mail.gmail.com
It's harder than it should be, apparently. Seasoned wizards as well
as the most promising new core hackers all seem to be breaking things
on a regular basis lately. The most common symptom is build failures
in the smoke reports on stricter platforms or configurations,[1] but
the problem is almost never just a "platform issue"; it's almost
always something genuinely wrong that needs fixing.
I would like us to think of ways to make it easier for everyone to see
what's wrong before submitting or committing changes to the API, and
to a lesser extent core changes more generally. I'm not talking about
the hard part: actually understanding the internals enough to modify
them and designing an appropriate API to expose those changes. I am
talking about basic sanity checking to make sure we're exporting what
we think we're exporting and that new code has some modicum of
correctness. I suspect some combination of documentation, policy,
and/or tool development could make this easier to ensure than it is
now. The following are some semi-random thoughts of mine in this
direction; folks who know things please correct and amplify. Folks
who feel they don't know things, please ask questions and describe
your difficulties.
The only reason I'm aware of the frequent build breakages is that I
routinely build with rather strict native tools on VMS. If anyone
wants to start developing on VMS, I'll help you, or on Win32, I won't
stop you, but I'm pretty sure all the modern tools have ways of
catching most of the errors we've been seeing. It's now doctrine to
C<use strict> when writing Perl; isn't it time to do the same thing
when *building* Perl?
For example, a significant subset of the recent breakages could have
been spotted early simply by building with -Dcc=g++. You don't have
to know anything about C++ (I don't); just think of it for our
purposes as a meaner C compiler that will slam the door in your face
if you did something you shouldn't. clang also appears to be suitably
pugnacious about things it doesn't like, and there are various ways
for various C compilers to turn up the warning level and take more
aggressive actions on what they warn about. Use whatever tools you
like and are comfortable with, but learn the equivalent of C<use
strict> for those tools.
You of course have to read what warnings you're getting and act on
them, and it's hard to see them when you just build at your terminal
and let it all fly by, looking only for the "All tests successful"
message at the end and ignoring the rest. Consider building in a way
that gives you a log (such as with C<at>) and then parse the log for
compiler messages using Jarkko's grepccerr, available from
<http://osdir.com/ml/lang.perl.perl5.porters/2005-03/msg00594.html> or
as part of Test::Smoke::Reporter.
Then read those complier messages, particularly scanning for those
bits of the code you've just modified, and don't consider warnings
"just warnings" unless you really understand why the compiler writer
thought you might get into trouble and why the particular code in
question isn't going to get you into that type of trouble. Don't
assume that you don't know enough C to do something about the
warnings. If the compiler tells you, "conversion from 'I32' to 'U8',
possible loss of data," ask any second-grader in the family whether
it's possible to pour (without spilling) a quart of water into a
one-cup measure. For signedness mismatches, you'll probably have to
find and consult a teenager.
Linker laxity is a bit tougher nut to crack than compiler laxity, or
at least I didn't find a way to do it with configuration capabilities
we already have. I think it would be advantageous to have a
configuration option for building perllib on unixy platforms so that
it exports only those symbols that are explicitly intended to be part
of the API, essentially making it match the default behavior on VMS,
Windows, and AIX. The advantage for development purposes seems
obvious: if people are fiddling with what we export, they'd know right
away if the fiddling had the intended effect. Side note: if we
actually shipped releases this way, it would be harder for people to
shoot themselves in the foot by using private functions
unintentionally, but that's really a different discussion.
This sort of export strictness might be a bit tricky to do portably.
On Darwin's ld man page I see a -exported_symbols_list switch, and on
the Linux ld man page I see --dynamic-list. Both of them need a list
of symbols fed to them, but in different formats. makedef.pl would
need modifications and probable refactoring; I suspect many of its
platform-specific inclusions and exclusions really need to be tied to
specific features. Has anyone ever tried to do use makedef.pl on
Linux or Unix before?
Of course we already have the smoke-me branches, and perhaps the
easiest, safest thing to do would be simply to put all API changes
into a smoke-me branch first and only merge them when they've made it
through one or more of the pickier build environments. I don't know
that this needs to be a policy (we have rather more policies than I
can keep track of already). But it would require paying attention to
the smoke reports and taking action on them, even if action means
asking for help.
I hope this is helpful to someone. It would be helpful to me if folks
could break things just a little bit less often.
[1] When a smoke report has "FAIL(M)" in the subject line, that means
the make failed. As in the compiler and/or linker encountered errors,
or possibly we've mangled Makefile.SH or its platform-specific
equivalents, or something in the process driven by make_ext.pl. This
may seem obvious, but I was recently congratulated on a build failure
because it didn't have as many test failures as a previous smoke. I
suppose not crashing because the car wouldn't start has its
advantages, but you still don't go anywhere.
Thread Next
-
on changing the API without breaking the build
by Craig A. Berry