automake-1.16: Serial Test Harness
15.2.2 Older (and discouraged) Serial Test Harness
--------------------------------------------------
First, note that today the use of this harness is strongly discouraged
in favour of the parallel test harness (⇒Parallel Test Harness).
Still, there are a _few_ situations when the advantages offered by the
parallel harness are irrelevant, and when test concurrency can even
cause tricky problems. In those cases, it might make sense to still use
the serial harness, for simplicity and reliability (we still suggest
trying to give the parallel harness a shot though).
The serial test harness is enabled by the Automake option
‘serial-tests’. It operates by simply running the tests serially, one
at the time, without any I/O redirection. It’s up to the user to
implement logging of tests’ output, if that’s required or desired.
For historical and implementation reasons, the ‘AM_TESTS_ENVIRONMENT’
variable is _not_ supported by this harness (it will be silently ignored
if defined); only ‘TESTS_ENVIRONMENT’ is, and it is to be considered a
developer-reserved variable. This is done so that, when using the
serial harness, ‘TESTS_ENVIRONMENT’ can be defined to an invocation of
an interpreter through which the tests are to be run. For instance, the
following setup may be used to run tests with Perl:
TESTS_ENVIRONMENT = $(PERL) -Mstrict -w
TESTS = foo.pl bar.pl baz.pl
It’s important to note that the use of ‘TESTS_ENVIRONMENT’ endorsed here
would be _invalid_ with the parallel harness. That harness provides a
more elegant way to achieve the same effect, with the further benefit of
freeing the ‘TESTS_ENVIRONMENT’ variable for the user (⇒Parallel
Test Harness).
Another, less serious limitation of the serial harness is that it
doesn’t distinguish between simple failures and hard errors; this is for
historical reasons, and might be fixed in future Automake versions.
Info Catalog
automake-1.16: Scripts-based Testsuites
automake-1.16: Simple Tests
automake-1.16: Parallel Test Harness