Tewiba

Tewiba: TEst WIth BAsh is a simple integration and functional test suite, in the unix spirit.

This page is at https://colas.nahaboo.net/Code/Tewiba

Principles

• Each test or series of tests is a shell script, or any executable file: This system is more geared to test unix programs the unix way - handling them via a shell and looking at the exit code and sterr, rather than testing only bash scripts. (hence the WIth BAsh)
• No API to learn to write scripts. Just use the normal unix conventions:
  • Success is indicated by exiting with the 0 exit code
  • Description of problems in stderr
  • Mundane info on stdout
• Does not try to reinvent the wheel. If you want to write sophisticated rules to test the results, there is already all the power of Bash, no need to invent another language, like most test suite do.
• Some convenience functions are provided however: TEST, TERR, TEND, ...
• Tewiba is geared towards integration and functional testing: complex, large grain, where the difficulty is to setup the testing environnement (a database, a web server, a subversion server, ...), and allows grouping tests in directories, with specialized code to set up and clean up before and after the tests. But since each test consists in running an unix program, it is not adapted for fine-grained unit testing.
• Tewiba is a self-contained script, with no dependencies, easily embeddable in your source code with a non-viral license (MIT)
• Geared for various custom tests in mixed environnements, in silent mode. Useful for integration tests where each test can be quite complex and do not additional complexity from the test framework.

For another shell test framework, more specialised in unit tests for shell scripts, please see SHunit2 at https://github.com/kward/shunit2

Quickstart

• copy the "tewiba" script somewhere in your distrib, or put it somewhere in your path. It is self-contained and has no dependencies
• decide of a dir TTT where you will put your tests, and write them there For instance, a subdir dir named "tests" of current dir, where you can put also tewiba, and run all tests by: cd tests && ./tewiba
• run the tests by: tewiba TTT. For instance, the tewiba test suite is run by ./tewiba tests

For more info, run: tewiba -?

License

Copyright (c) 2013-2020 Colas Nahaboo (https://colas.nahaboo.net) MIT License: http://opensource.org/licenses/MIT

Download

From its source repository at http://hg.colas.nahaboo.net/tewiba/. Click the "gz" link to download a .tar.gz of the latest version.

History

• 2020-04-10 v1.4.3 Less ambiguous prints of the test name and test filename in verbose mode
• 2020-04-10 v1.4.2 new convenience function provided: DOTEST
• 2020-04-08 v1.4.0 all tewiba functions print on stderr. Also TEST now prints in a TERR occurs, even in non-verbose mode. We now use Semantic Versioning version numbers.
• 2020-03-28 v1.2 tewiba removes temporary data on exit even if the tests override the trap on signal 0
• 2020-03-27 v1.1a no code change, only added documentation of the TCLEANUP trap used on signal 0
• 2017-09-06 v1.1 verbose now prints the error messages inside the output, not on top anymore.
• 2015-10-25 v1.0 First published version
• 2013-08-07 v0.1 First version, used internally

Example

Here is a simple test file to be run by Tewiba to test that the date command is the GNU version, at least v8. The first line means that it is actually also an executable a bash script, the second line allows this file to be run as argument to tewiba, or as a standalone script, in which case it calles tewiba on itself. Note how we can use all the power of bash to match strings in the results of the date command

#!/bin/bash
[ -n "$TEWIBA" ] || exec tewiba -v "$0"
TEST date installed
hash date || TERR date command not found
TEST check that date is GNU date, at least version 8
version=$(date --version)
[[ $version =~ "date (GNU coreutils)" ]] || TERR date is not the GNU version of date
[[ $version =~ "date (GNU coreutils) *"([0-9]+) ]] TERR could not determine date version
let "${BASH_REMATCH[1]} >= 8" || TERR date version is not at least v8
TEST just check date works as intended on a simple case
[ $(date -d @1234567890 +%Y-%m-%d) = 2009-02-13 ] || TERR date error converting time 1234567890
[ $(date -d 2009-02-13T23:31:30Z +%s) = 1234567890 ] || TERR date error converting date 2009-02-13T23:31:30Z
TEND

Manual

"tewiba -?" prints:

tewiba [options] [tests...]
Tewiba: TEst WIth BAsh: a simple test suite, in the spirit of shell scripting
Runs the test suite, on all tests in the current dir (default), 
or on specific tests or directories of tests given as arguments.
   * Tests are given as arguments, and executed in order, they can be either
      * an executable file
      * a directory, all executable files in it will be run as tests
        Filenames beginning with a . # _ or ending with "~" are ignored
        no recursion is done, subdirs will not be explicitely scanned
        except the ones ending in .subtests wthat will be also run
     If an executable file __INIT__ is found, it will be sourced into the shell
     (read) so that its definitions will be available for all the tests in it
     Same, if an __END__ file is found, it will be sourced after the tests
   * Nothing is printed if all tests OK. On failure, the stderr is printed
   * Tests exit code must be the number of failed tests (0 means OK)
   * A warning is shown if the test was OK but printed on stderr
   * Tests are run in their directory
   * Tewiba exit code is the total number of failed tests. 

How to write tests:
   * Tests are executable files (in any language), that exit with code 0 if OK,
     and the number of failures if not
   * Test file names can be meaningful, e.g: "Test for empty input"
   * Filenames beginning with a "." "#" "_" or ending with "~" are ignored
     as well as files named "tewiba"
   * They must print on stderr an explanation of the error, or a warning if 
     exit code is 0. Prints on stdout are only shonw in verbose mode
   * Tests can create temporary files and dirs prefixed by "$tmp."
     they will be automatically cleaned afterwards by rm -rf $tmp $tmp.*
   * Test data can be put in subdirectories, as subdirectories are not tested
   * Convenience functions & variables that you can use in bash script tests:
     TLEVEL chars  only runs if a level in chars is in the ones given by -l,
                   or -l not specified. Each level is an alphanumeric char.
     TLEVEL_ONLY chars    Same as above but does not run if -l not given 
     TEST message  prints sub-test (the file name is the test name) name if 
                   verbose, or before the first error of this subtest
     TERR message  declares an error, but continues the tests
     FERR message  fatal error: also aborts the current test file
     TEND          MANDATORY: terminates your test script.
                   (an alias for: exit $TFAILS)
                   This is the only function really mandatory in a test file.
     TINIT id...   For each id (composed of letters, numbers, and _ - .)
                   ensures that files __INIT__id and __END__id are sourced,
                   if they exist, before and after all tests of the dir,
                   after and before __INIT__ and __END__ respectively
     DOTEST [-o|-e|-s] label expected-output-string parameters
                   runs the command in $TOTEST - that you must define - with
                   the parameters as its command line options, compare its
                   output to the expected-output-string (or its stderr with
                   -e, or its exit code status with -s. The default is -o for
                   stdout), and in case of difference triggers a TERR 
                   displaying the label, the result and the expected string.
     $TV           is true in verbose mode, false otherwise
     $tmp          temporary data prefix (see above)
     $nl           is the newline character
     $TEWIBA       the version number of tewiba. 
                   Trick: put after #!/bin/bash the line:
                   [ -n "$TEWIBA" ] || exec tewiba -v "$0"
                   so that the test file can be executed standalone
     $TEWIBADIR    the original directory where tewiba was run
   * All tewiba functions above print on stderr.
   * IMPORTANT: the shell function TCLEANUP must be called at the end of the
     tests, to clean the temporary data. tewiba traps the signal 0 for this.
     if you redefine trap 0 in your tests, or in a script you source, you 
     should either make your trap function also call TCLEANUP, or redo a
     "trap TCLEANUP 0" at the end of your tests.

Options:
   -v      verbose: prints each test name as it is run, as well as its stdout
   -V      just prints tewiba version number
   -x      debug: execute each test in set -x mode copying also the output to
           /tmp/tewiba.$LOGNAME.out Note: does not imply -v
   -l levels  runs only tests of the levels (characters) in the string levels
           "TLEVEL x" in a test runs when given "-l y" if [y] matches x:
           thus ranges are possible: -l 0-3 matches a TLEVEL 17a
   -s      standalone: do not read __INIT__ nor __END__ files
   -e file outputs the total number of errors in the file. 
   -f      forces running tests files even without a TEND directive in them.
           Or add this line in the test file: #TNOEND

Exemple simple test file:
  #!/bin/bash
  [ -n "$TEWIBA" ] || exec tewiba -v "$0"
  TEST screen installed
  hash screen || TERR screen command not found
  TEND

Version 1.4.2 - Web site: https://%HTTP_HOST%/Code/Tewiba
Copyright (c) 2013-2017 Colas Nahaboo (https://%HTTP_HOST%)
MIT License: http://opensource.org/licenses/MIT