You are here: Colas>Software Web>Tewiba (10 Sep 2017, ColasNahaboo)Edit Attach

Tewiba

Tewiba: TEst WIth BAsh is a simple integration and functional test suite, in the unix spirit.

This page is at http://colas.nahaboo.net/Software/Tewiba

Principles

  • Each test or series of tests is a shell script, or any executable file: This system is more geared to test unix programs the unix way - handling them via a shell and looking at the exit code and sterr, rather than testing only bash scripts. (hence the WIth BAsh)
  • No API to learn to write scripts. Just use the normal unix conventions:
    • Success is indicated by exiting with the 0 exit code
    • Description of problems in stderr
    • Mundane info on stdout
  • Does not try to reinvent the wheel. If you want to write sophisticated rules to test the results, there is already all the power of Bash, no need to invent another language, like most test suite do.
  • Some convenience functions are provided however: TEST, TERR, TEND, ...
  • Tewiba is geared towards integration and functional testing: complex, large grain, where the difficulty is to setup the testing environnement (a database, a web server, a subversion server, ...), and allows fro grouping tests in directories, with specialized code to set up and clean up before and after the tests. But since each test consists in running an unix program, it is not adapted for fine-grained unit testing.
  • Tewiba is a self-contained script, with no dependencies, easily embeddable in your source code with a non-viral license (MIT)
  • Geared for various custom tests in mixed environnements, in silent mode. Useful for integration tests where each test can be quite complex and do not additional complexity from the test framework.

For another shell test framework, more specialised in unit tests for shell scripts, please see SHunit2 at https://github.com/kward/shunit2

Quickstart

  • copy the "tewiba" script somewhere in your distrib, or put it somewhere in your path. It is self-contained and has no dependencies
  • decide of a dir TTT where you will put your tests, and write them there For instance, a subdir dir named "tests" of current dir, where you can put also tewiba, and run all tests by: cd tests && ./tewiba
  • run the tests by: tewiba TTT. For instance, the tewiba test suite is run by ./tewiba tests

For more info, run: tewiba -?

License

Copyright (c) 2013-2017 Colas Nahaboo (http://colas.nahaboo.net) MIT License: http://opensource.org/licenses/MIT

Download

From its source repository at http://hg.colas.nahaboo.net/tewiba/. Click the "gz" link to download a .tar.gz of the latest version.

History

  • 2017-09-06 v1.1 verbose more now prints the error messages inside the output, not on top anymore.
  • 2015-10-25 v1.0 First published version
  • 2013-08-07 v0.1 First version, used internally

Example

Here is a simple test file to be run by Tewiba to test that the date command is the GNU version, at least v8. The first line means that it is actually also an executable a bash script, the second line allows this file to be run as argument to tewiba, or as a standalone script, in which case it calles tewiba on itself. Note how we can use all the power of bash to match strings in the results of the date command
#!/bin/bash
[ -n "$TEWIBA" ] || exec tewiba -v "$0"
TEST date installed
hash date || TERR date command not found
TEST check that date is GNU date, at least version 8
version=$(date --version)
[[ $version =~ "date (GNU coreutils)" ]] || TERR date is not the GNU version of date
[[ $version =~ "date (GNU coreutils) *"([0-9]+) ]] TERR could not determine date version
let "${BASH_REMATCH[1]} >= 8" || TERR date version is not at least v8
TEST just check date works as intended on a simple case
[ $(date -d @1234567890 +%Y-%m-%d) = 2009-02-13 ] || TERR date error converting time 1234567890
[ $(date -d 2009-02-13T23:31:30Z +%s) = 1234567890 ] || TERR date error converting date 2009-02-13T23:31:30Z
TEND

Manual

"tewiba -?" prints:

tewiba [options] [tests...]
Tewiba: TEst WIth BAsh: a simple test suite, in the spirit of shell scripting
Runs the test suite, on all tests in the current dir (default), 
or on specific tests or directories of tests given as arguments.
   * Tests are given as arguments, and executed in order, they can be either
      * an executable file
      * a directory, all executable files in it will be run as tests
        Filenames beginning with a . # _ or ending with "~" are ignored
        no recursion is done, subdirs will not be explicitely scanned
        except the ones ending in .subtests wthat will be also run
     If an executable file __INIT__ is found, it will be sourced into the shell
     (read) so that its definitions will be available for all the tests in it
     Same, if an __END__ file is found, it will be sourced after the tests
   * Nothing is printed if all tests OK. On failure, the stderr is printed
   * Tests exit code must be the number of failed tests (0 means OK)
   * A warning is shown if the test was OK but printed on stderr
   * Tests are run in their directory
   * Tewiba exit code is the total number of failed tests. 

How to write tests:
   * Tests are executable files (in any language), that exit with code 0 if OK,
     and the number of failures if not
   * Test file names can be meaningful, e.g: "Test for empty input"
   * Filenames beginning with a "." "#" "_" or ending with "~" are ignored
     as well as files named "tewiba"
   * They must print on stderr an explanation of the error, or a warning if 
     exit code is 0. Prints on stdout are only shonw in verbose mode
   * Tests can create temporary files and dirs prefixed by "$tmp."
     they will be automatically cleaned afterwards by rm -rf $tmp $tmp.*
   * Test data can be put in subdirectories, as subdirectories are not tested
   * Convenience functions & variables that you can use in bash script tests:
     TLEVEL chars  only runs if a level in chars is in the ones given by -l,
                   or -l not specified. Each level is an alphanumeric char.
     TLEVEL_ONLY chars    Same as above but does not run if -l not given 
     TEST message  prints (sub)test name if verbose
     TERR message  declares an error, but continues the tests
     FERR message  fatal error: also aborts the current test file
     TEND          terminates your test script (alias for: exit $TFAILS)
                   the only function really mandatory.
     TINIT id...   For each id (composed of letters, numbers, and _ - .)
                   ensures that files __INIT__id and __END__id are sourced,
                   if they exist, before and after all tests of the dir,
                   after and before __INIT__ and __END__ respectively
     $TV           is true in verbose mode, false otherwise
     $tmp          temporary data prefix (see above)
     $nl           is the newline character
     $TEWIBA       the version number of tewiba. 
                   Trick: put after #!/bin/bash the line:
                   [ -n "$TEWIBA" ] || exec tewiba -v "$0"
                   so that the test file can be executed standalone
     $TEWIBADIR    the original directory where tewiba was run

Options:
   -v      verbose: prints each test name as it is run, as well as its stdout
   -V      just prints tewiba version number
   -x      debug: execute each test in set -x mode copying also the output to
           /tmp/tewiba.$LOGNAME.out Note: does not imply -v
   -l levels  runs only tests of the levels (characters) in the string levels
           "TLEVEL x" in a test runs when given "-l y" if [y] matches x:
           thus ranges are possible: -l 0-3 matches a TLEVEL 17a
   -s      standalone: do not read __INIT__ nor __END__ files
   -e file outputs the total number of errors in the file. 

Exemple simple test file:
  #!/bin/bash
  [ -n "$TEWIBA" ] || exec tewiba -v "$0"
  TEST screen installed
  hash screen || TERR screen command not found
  TEND

Version 1.1 - Web site: http://colas.nahaboo.net/Software/Tewiba
Copyright (c) 2013-2017 Colas Nahaboo (http://colas.nahaboo.net)
MIT License: http://opensource.org/licenses/MIT
Topic revision: r2 - 10 Sep 2017, ColasNahaboo
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Colas? Send feedback