cons-test - Run Cons tests
cons-test [-qw] [-d dir] [-I dir] [-x cons] [test_script ...]
-d dir search for tests in specified dir
-I dir add dir to Perl search path
-q quiet, supress warnings about undefined variables
-w execute the cons script with perl -w flag
-x cons test specified cons script
By default, the cons-test script will test the script cons or
cons.pl in the current directory.
An alternate cons script name may be specified via the -x flag:
$ cons-test -x cons.NEW
$ cons-test -x /usr/foo/cons.experiment/cons
The cons-test script will arrange for each test to use the specified
cons script. The cons script under test need not have execute
permission set.
The cons script under test may be executed with the Perl -w flag,
which warns about conditions such as uninitialized variables:
$ cons-test -w
By default, the cons-test script executes all the tests it finds in
the following directories:
. t /usr/lib/cons-2.2.0/test/t /usr/lib/cons/test/t
Any file with a .t extension is assumed to be a test. The cons-test
script can also execute tests with .pl or .sh extensions, although
they must be explicitly listed on the command line.
By default, the executed tests use the supporting Test::Cmd and
Test::Cmd::Cons perl modules found under the current directory or
/usr/lib/cons-2.2.0. Additional directories to be
searched for these modules may be specified with -I flag.
Each test is executed with the specified cons script
passed in via the CONS environment variable. The cons script
will be executed with any Perl flags specified via the PERLFLAGS
environment variable.
After all the tests have been executed, the cons-test script reports
a summary of the pass/fail score:
cons-test: 'cons' passed all 130 tests.
cons-test: 'cons' passed 128 tests, failed 2:
/usr/lib/cons-2.2.0/test/t/t0003.t
/usr/lib/cons-2.2.0/test/t/t0026.t
The cons-test script can be given one or more tests as arguments,
in which case it will only execute the specified tests:
$ perl cons-test t0007.t t0023.t cons-test: perl -w /usr/lib/cons-2.2.0/test/t/t0007.t PASSED cons-test: perl -w /usr/lib/cons-2.2.0/test/t/t0023.t PASSED
cons-test: 'cons' passed all 2 tests. $
The cons-test script may also be given environment variable assignments
as arguments. These will be evaluated in order with the test arguments.
This allows, for example, executing the same test(s) with different
compilers in the same cons-test invocation:
$ perl cons-test CC=gcc t/t0001.t CC=no_compiler t/t0001.t cons-test: perl -w /usr/lib/cons-2.2.0/test/t/t0001.t PASSED cons-test: perl -w /usr/lib/cons-2.2.0/test/t/t0001.t FAILED test #1 of cons [single-module Program]: no_compiler -c foo.c -o foo.o cons: failed to execute "no_compiler" (No such file or directory). Is this an executable on path "/bin:/usr/bin"? cons: *** [foo.o] Error 2 cons: errors constructing foo.o FAILED test #1 of cons [single-module Program]
cons-test: 'cons' passed 1 tests, failed 1:
/usr/lib/cons-2.2.0/test/t/t0001.t
$
By default, the cons-test script expects that the environment variables
CC, AR, and RANLIB are set, and will generate warnings if they
are not. A -q option may be used to suppress these warnings.
Tests conform to requirements of the Aegis project change supervisor, which integrates creation and execution of regression tests into the software development process. Information about Aegis can be found at:
http://www.tip.net.au/~millerp/aegis.html
Each test is a completely self-contained Perl script, and may be directly
executed by explicitly passing it to perl (-w flag preferred):
$ perl -w t/t0001.t
The cryptic names are by Aegis convention only, and could have been named to reflect the functionality being tested. The numbering has been preserved to try to give order to the tests: Simpler, underlying functionality is tested before more complicated features that rely on functionality tested by prior tests.
Each test creates a temporary work directory under $TMPDIR (under
/tmp by default) and populates it with files from in-line here
documents.
Each test checks the return value from every subroutine it calls and ever
command it executes, expecting a successful return value or exit code
from each. When it's important, each test compares actual command output
against expected output. Note that, in the usual case, this is actually
output of the executables generated by Cons, not the ``cc'' command lines
reported as the actions Cons executes to build the test executables.
There are some exceptions, tests which do examine the build actions;
these are noted in the Tests.txt file and in the commentary at the
top of each test.
By default, the tests rely on having normal C compilation, linking and
archiving tools available through the environment's PATH, and expect
that printf() is available through whatever library is linked into
a program via the default Cons build environment. These tests use the
following environment variables for specification of alternate paths to
various utilities:
AR (default: 'ar') library archiver CC (default: 'cc') C compiler RANLIB (default: 'ranlib') ranlib
As described previously, the cons-test script accepts environment
variable assignments in its command-line arguments. This may be
used, for example, to specify different C compilers for different
test invocations:
$ perl cons-test CC=gcc t/t0001.t CC=cc t/t0001.t
Each test reports one of three results:
Each test removes its temporary work directory upon completion.
For post facto debugging, the work directory can be preserved (and its name reported) by setting environment variables:
Steven Knight <knight@baldmt.com>
Report bugs to <bug-cons@gnu.org>.
Copyright 2000 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
cons