develooper Front page | perl.cpan.workers | Postings from June 2014

Re: RFC Consensus on Network testing ENV?

Thread Previous
Kent Fredric
June 12, 2014 07:29
Re: RFC Consensus on Network testing ENV?
Message ID:
On 12 June 2014 06:37, Jens Rehsack <> wrote:

> > 1. begin test
> > 2. load test networking module module
> > 4. can access specified resources?  yes -> run tests
> >                                                       no   -> SKIP!
> In that case I suggest to FAIL instead of skip. Maybe is that part of the
> error.

Really whether or not that fails would be an installer choice.

For instance, me, I would basically say "If you cant access the internet,
don't bother doing tests that require the internet". After all, the running
of the test suite is not to give me a "pass/fail: Your internet works", but
to test the code in question functions as intended.

The only time I'd want the tests to fail is if I was running the tests
_AND_ want assurredness that the network tests in fact work as intended,
and that the non-network tests are not sufficient for my personal installer

Then those tests would fail every time they couldn't use the internet, and
I'd know I have to fix my internet and run them again.

But that case is very much not the usecase of a bulk smoke tester.

If a tester box doesn't have internet visible during 'make test', I'd
expect tests that require internet access to be skipped ( or exit N/A
somehow ), not give the developer of the module spurious FAIL results that
revolve around something the developer can do nothing about, FAIL reports
that don't indicate the code is broken, merely that the person who ran the
tests has broken internet.

Based on some of the comments on #perl-qa , it seems there may also be a
use for declaring 2 or 3 grades of internet access.

1. Loopback only access: Basic network functions work, you can spawn a HTTP
test server for something like PSGI, and connect to it from another Perl
process and IPC between the two.

2. LAN only access: There are other machines on a LAN, some semblance of
DNS will probably work and resolve HOSTNAME to a network routable IP, but
internet traffic is still expected to not work. ( Or there's a (broken?)
firewall standing between LAN and WAN than can resolve global DNS addresses
but not relay packets )

3. Full internet

Now we probably don't need to represent all of these, just they're scopes
to take into consideration. You'd think case #1 would never need blocking
off, however, the person who raised that case mentioned in the context of
"Localhost is firewalled".  And that would mess things up. ( And I doubt
anyone is interested in going through a firewall just to allow open ports
for some random Perl test )

Most of those levels we could indeed check for with code, but having a
"Don't even check, its not worth the cycles, you'll probably get it wrong,
and trying may even cause an explosion because somebody attached C4 to a
network device somewhere in the house"  control would be still useful.

( And I guess the "weird networks are weird all over the place" problem is
one reason why the default is to have network tests opt-in instead of
opt-out.  )


Thread Previous Perl Programming lists via nntp and http.
Comments to Ask Bjørn Hansen at | Group listing | About