Tests fail in cpan, succeed manually

[J Krugman]

I'm trying to understand what's going on, and decide what to do.
When I attemptto install the module XML::Validator::Schema via
CPAN.pm, all attempted tests fail:

Failed Test Status Wstat Total Fail Failed List of Failed
--------------------------------------------------------------------------------t/01basic.t 2 2 100.00% 1-2
t/02yaml.t 152 152 100.00% 1-152
t/03types.t 163 163 100.00% 1-163
t/04model.t 38 38 100.00% 1-38
2 tests skipped.
Failed 4/6 test scripts, 33.33% okay. 355/355 subtests failed, 0.00% okay.
make: *** [test_dynamic] Error 29
/usr/bin/make test -- NOT OK
Running make install
make test had returned bad status, won't install without force

But when I run these tests "by hand" (e.g. in the build directory,
perl -Mlib='./blib/lib' t/01basic.t), they *all* succeed.

This is a very unusual situation. My installations via CPAN.pm
typically succeed, and if they fail because of some failed test,
this test fails also when invoked manually.

Can someone tell me why these tests would fail when invoked through
CPAN.pm, but not when they are invoked manually? And should I
disregard CPAN.pm's warnings in this case and force the installation?

Thanks!

jill

 

[James Willmore]

I'm trying to understand what's going on, and decide what to do. When I
attemptto install the module XML::Validator::Schema via CPAN.pm, all
attempted tests fail:

Failed Test Status Wstat Total Fail Failed List of Failed

--------------------------------------------------------------------------------
t/01basic.t> 2 2 100.00% 1-2
t/02yaml.t 152 152 100.00% 1-152
t/03types.t 163 163 100.00% 1-163
t/04model.t 38 38 100.00% 1-38 2 tests skipped.
Failed 4/6 test scripts, 33.33% okay. 355/355 subtests failed, 0.00%
okay. make: *** [test_dynamic] Error 29
/usr/bin/make test -- NOT OK
Running make install
make test had returned bad status, won't install without force

But when I run these tests "by hand" (e.g. in the build directory, perl
-Mlib='./blib/lib' t/01basic.t), they *all* succeed.

This is a very unusual situation. My installations via CPAN.pm
typically succeed, and if they fail because of some failed test, this
test fails also when invoked manually.

Can someone tell me why these tests would fail when invoked through
CPAN.pm, but not when they are invoked manually? And should I disregard
CPAN.pm's warnings in this case and force the installation?

Some modules respond well to (when making by hand):
make test TEST_VERBOSE=1

This, if the module is willing, will produce verbose output that *may*
help you make this decision.

You could look at the test(s) being performed. In this case, you can type
(in the CPAN shell) "look XML::Validator::Schema", then examine the files
in the 't' directory.

You could also look over the README and INSTALL files for the module. I've
found that (for example, the various DBD modules) there need to be some
environment settings that need to be set *before* the install is done.

Just FYI - there *should* be no difference doing an install by hand or
using CPAN. CPAN just "automates" the exercise of installing a module.

Just my $0.02

HTH

Jim

 

[J Krugman]

I'm trying to understand what's going on, and decide what to do. When I
attemptto install the module XML::Validator::Schema via CPAN.pm, all
attempted tests fail:

Failed Test Status Wstat Total Fail Failed List of Failed

--------------------------------------------------------------------------------
t/01basic.t> 2 2 100.00% 1-2
t/02yaml.t 152 152 100.00% 1-152
t/03types.t 163 163 100.00% 1-163
t/04model.t 38 38 100.00% 1-38 2 tests skipped.
Failed 4/6 test scripts, 33.33% okay. 355/355 subtests failed, 0.00%
okay. make: *** [test_dynamic] Error 29
/usr/bin/make test -- NOT OK
Running make install
make test had returned bad status, won't install without force

But when I run these tests "by hand" (e.g. in the build directory, perl
-Mlib='./blib/lib' t/01basic.t), they *all* succeed.

This is a very unusual situation. My installations via CPAN.pm
typically succeed, and if they fail because of some failed test, this
test fails also when invoked manually.

Can someone tell me why these tests would fail when invoked through
CPAN.pm, but not when they are invoked manually? And should I disregard
CPAN.pm's warnings in this case and force the installation?

Some modules respond well to (when making by hand):
make test TEST_VERBOSE=1

This, if the module is willing, will produce verbose output that *may*
help you make this decision.

This helped, but I'm not quite there yet. I discovered that the
problem is not with CPAN.pm. If I run

make test

outside of the cpan shell, I get the same failure messages, but if
I run the tests individually (as described above) they all succeed.

When I ran

make test TEST_VERBOSE=1

as you suggested, I discovered that all the subtests succeed, but
somehow make (or something else) declares these tests failed. For
example:

t/01basic...........ok 1 - use XML::Validator::Schema;
ok 2
1..2
FAILED tests 1-2
Failed 2/2 tests, 0.00% okay

....and so on for every test in t. Does anyone have any guesses as to
why these tests are being incorrectly classified as failures?

I have a hunch that the problem has something to do with the fact
that these tests use Test::More instead of Test, but I don't know
enough about Test::More to know for sure whether this is the case,
or whether the problem is in Test::More or in how it is being used
by the author of the failing tests.

BTW, I tried reporting a bug about the XML::Validator::Schema
installation at http://rt.cpan.org (per instructions in the
XML::Validator::Schema man page), but I get a very strange error
message to the effect that the site is down (the error message is
strange because it gives no indication that the requested page has
anything to do with Perl or CPAN). *sigh*

It looks like I'm standing on a mountain of bugs...

It is ironic that someone who bothered to set up a test suite for
his/her module, did not bother to make sure that the installation
itself was bug-free...

At times like these I wonder about the future of open software.
It's not like I'm not doing my part. I devote most of my paid and
spare time to coding free, open software products... But I spend
an inordinate amount of time, far too much, chasing and reporting
bugs... Sorry, but it really pisses me off.

Maybe open the software paradigm is not such a good idea after all.
True, paid software can be very buggy, and those who sell software
(notably Microsoft) seem to be able to get away with murder in this
respect, but with the paid software model at least there is a
*possibility* that in the future those selling buggy software will
be held accountable. With free software even that slim hope is
out of the question.

jill

 

[Jim Keenan]

The module's author recommends this mailing list for support:

http://listserv.activestate.com/mailman/listinfo/perl-xml

Have you tried that yet?

Jim Keenan

 

[Jim Keenan]

For the heck of it, I decided to try to install this module via the
CPAN shell. I was not anticipating good results, because the box on
which I was installing it is Windows98, Perl 5.6.1 only -- and in the
past, whenever I have tried to install a module with many
prerequisites such as this one, my system has gagged. And since I had
few of the prereqs installed, the installation took nearly 10 minutes.
At least 5 different prereqs had to be installed before the shell got
to this module.

However, it worked. All the tests passed and XML::Validator::Schema
was installed. So I haven't been able to reproduce your problem.
Sorry.

Jim Keenan

 

[J Krugman]

The module's author recommends this mailing list for support:

Have you tried that yet?

Yes, long ago. I'm still waiting to get my request to join the
list approved.

jill

 

[James Willmore]

I'm trying to understand what's going on, and decide what to do. When I
attemptto install the module XML::Validator::Schema via CPAN.pm, all
attempted tests fail:

Failed Test Status Wstat Total Fail Failed List of Failed
--------------------------------------------------------------------------------t/01basic.t
2 2 100.00% 1-2 t/02yaml.t 152
152 100.00% 1-152 t/03types.t 163 163 100.00% 1-163
t/04model.t 38 38 100.00% 1-38 2 tests skipped.
Failed 4/6 test scripts, 33.33% okay. 355/355 subtests failed, 0.00%
okay. make: *** [test_dynamic] Error 29
/usr/bin/make test -- NOT OK
Running make install
make test had returned bad status, won't install without force

But when I run these tests "by hand" (e.g. in the build directory, perl
-Mlib='./blib/lib' t/01basic.t), they *all* succeed.

This is a very unusual situation. My installations via CPAN.pm
typically succeed, and if they fail because of some failed test, this
test fails also when invoked manually.

Can someone tell me why these tests would fail when invoked through
CPAN.pm, but not when they are invoked manually? And should I disregard
CPAN.pm's warnings in this case and force the installation?

Some modules respond well to (when making by hand): make test
TEST_VERBOSE=1

This, if the module is willing, will produce verbose output that *may*
help you make this decision.

You could look at the test(s) being performed. In this case, you can type
(in the CPAN shell) "look XML::Validator::Schema", then examine the files
in the 't' directory.

You could also look over the README and INSTALL files for the module. I've
found that (for example, the various DBD modules) there need to be some
environment settings that need to be set *before* the install is done.

Just FYI - there *should* be no difference doing an install by hand or
using CPAN. CPAN just "automates" the exercise of installing a module.

Just my $0.02

HTH

Jim

 

[James Willmore]

I'm trying to understand what's going on, and decide what to do. When I
attemptto install the module XML::Validator::Schema via CPAN.pm, all
attempted tests fail:

Failed Test Status Wstat Total Fail Failed List of Failed

--------------------------------------------------------------------------------
t/01basic.t> 2 2 100.00% 1-2
t/02yaml.t 152 152 100.00% 1-152
t/03types.t 163 163 100.00% 1-163
t/04model.t 38 38 100.00% 1-38 2 tests skipped.
Failed 4/6 test scripts, 33.33% okay. 355/355 subtests failed, 0.00%
okay. make: *** [test_dynamic] Error 29
/usr/bin/make test -- NOT OK
Running make install
make test had returned bad status, won't install without force

But when I run these tests "by hand" (e.g. in the build directory, perl
-Mlib='./blib/lib' t/01basic.t), they *all* succeed.

This is a very unusual situation. My installations via CPAN.pm
typically succeed, and if they fail because of some failed test, this
test fails also when invoked manually.

Can someone tell me why these tests would fail when invoked through
CPAN.pm, but not when they are invoked manually? And should I disregard
CPAN.pm's warnings in this case and force the installation?

Some modules respond well to (when making by hand):
make test TEST_VERBOSE=1

This, if the module is willing, will produce verbose output that *may*
help you make this decision.

You could look at the test(s) being performed. In this case, you can type
(in the CPAN shell) "look XML::Validator::Schema", then examine the files
in the 't' directory.

You could also look over the README and INSTALL files for the module. I've
found that (for example, the various DBD modules) there need to be some
environment settings that need to be set *before* the install is done.

Just FYI - there *should* be no difference doing an install by hand or
using CPAN. CPAN just "automates" the exercise of installing a module.

Just my $0.02

HTH

Jim

 

[James Willmore]

I'm trying to understand what's going on, and decide what to do. When I
attemptto install the module XML::Validator::Schema via CPAN.pm, all
attempted tests fail:

Failed Test Status Wstat Total Fail Failed List of Failed
--------------------------------------------------------------------------------t/01basic.t
2 2 100.00% 1-2 t/02yaml.t 152
152 100.00% 1-152 t/03types.t 163 163 100.00% 1-163
t/04model.t 38 38 100.00% 1-38 2 tests skipped.
Failed 4/6 test scripts, 33.33% okay. 355/355 subtests failed, 0.00%
okay. make: *** [test_dynamic] Error 29
/usr/bin/make test -- NOT OK
Running make install
make test had returned bad status, won't install without force

But when I run these tests "by hand" (e.g. in the build directory, perl
-Mlib='./blib/lib' t/01basic.t), they *all* succeed.

This is a very unusual situation. My installations via CPAN.pm
typically succeed, and if they fail because of some failed test, this
test fails also when invoked manually.

Can someone tell me why these tests would fail when invoked through
CPAN.pm, but not when they are invoked manually? And should I disregard
CPAN.pm's warnings in this case and force the installation?

Some modules respond well to (when making by hand): make test
TEST_VERBOSE=1

This, if the module is willing, will produce verbose output that *may*
help you make this decision.

You could look at the test(s) being performed. In this case, you can type
(in the CPAN shell) "look XML::Validator::Schema", then examine the files
in the 't' directory.

You could also look over the README and INSTALL files for the module. I've
found that (for example, the various DBD modules) there need to be some
environment settings that need to be set *before* the install is done.

Just FYI - there *should* be no difference doing an install by hand or
using CPAN. CPAN just "automates" the exercise of installing a module.

Just my $0.02

HTH

Jim