Contents:
Tewiba
Tewiba: TEst WIth BAsh is a simple integration and functional test suite, in the unix spirit. This page is at https://colas.nahaboo.net/Code/TewibaPrinciples
- Each test or series of tests is a shell script, or any executable file: This system is more geared to test unix programs the unix way - handling them via a shell and looking at the exit code and sterr, rather than testing only bash scripts. (hence the WIth BAsh)
- No API to learn to write scripts. Just use the normal unix conventions:
- Success is indicated by exiting with the 0 exit code
- Description of problems in stderr
- Mundane info on stdout
- Does not try to reinvent the wheel. If you want to write sophisticated rules to test the results, there is already all the power of Bash, no need to invent another language, like most test suite do.
- Some convenience functions are provided however: TEST, TERR, TEND, ...
- Tewiba is geared towards integration and functional testing: complex, large grain, where the difficulty is to setup the testing environnement (a database, a web server, a subversion server, ...), and allows grouping tests in directories, with specialized code to set up and clean up before and after the tests. But since each test consists in running an unix program, it is not adapted for fine-grained unit testing.
- Tewiba is a self-contained script, with no dependencies, easily embeddable in your source code with a non-viral license (MIT)
- Geared for various custom tests in mixed environnements, in silent mode. Useful for integration tests where each test can be quite complex and do not additional complexity from the test framework.
Quickstart
- copy the "tewiba" script somewhere in your distrib, or put it somewhere in your path. It is self-contained and has no dependencies
- decide of a dir TTT where you will put your tests, and write them there For instance, a subdir dir named "tests" of current dir, where you can put also tewiba, and run all tests by: cd tests && ./tewiba
- run the tests by: tewiba TTT. For instance, the tewiba test suite is run by
./tewiba tests
License
Copyright (c) 2013-2020 Colas Nahaboo (https://colas.nahaboo.net) MIT License: http://opensource.org/licenses/MITDownload
From its source repository at http://hg.colas.nahaboo.net/tewiba/. Click the "gz" link to download a .tar.gz of the latest version.History
- 2020-04-10 v1.4.3 Less ambiguous prints of the test name and test filename in verbose mode
- 2020-04-10 v1.4.2 new convenience function provided: DOTEST
- 2020-04-08 v1.4.0 all tewiba functions print on stderr. Also TEST now prints in a TERR occurs, even in non-verbose mode. We now use Semantic Versioning version numbers.
- 2020-03-28 v1.2 tewiba removes temporary data on exit even if the tests override the trap on signal 0
- 2020-03-27 v1.1a no code change, only added documentation of the TCLEANUP trap used on signal 0
- 2017-09-06 v1.1 verbose now prints the error messages inside the output, not on top anymore.
- 2015-10-25 v1.0 First published version
- 2013-08-07 v0.1 First version, used internally
Example
Here is a simple test file to be run by Tewiba to test that the date command is the GNU version, at least v8. The first line means that it is actually also an executable a bash script, the second line allows this file to be run as argument to tewiba, or as a standalone script, in which case it calles tewiba on itself. Note how we can use all the power of bash to match strings in the results of the date command#!/bin/bash [ -n "$TEWIBA" ] || exec tewiba -v "$0" TEST date installed hash date || TERR date command not found TEST check that date is GNU date, at least version 8 version=$(date --version) [[ $version =~ "date (GNU coreutils)" ]] || TERR date is not the GNU version of date [[ $version =~ "date (GNU coreutils) *"([0-9]+) ]] TERR could not determine date version let "${BASH_REMATCH[1]} >= 8" || TERR date version is not at least v8 TEST just check date works as intended on a simple case [ $(date -d @1234567890 +%Y-%m-%d) = 2009-02-13 ] || TERR date error converting time 1234567890 [ $(date -d 2009-02-13T23:31:30Z +%s) = 1234567890 ] || TERR date error converting date 2009-02-13T23:31:30Z TEND
Manual
"tewiba -?" prints:tewiba [options] [tests...] Tewiba: TEst WIth BAsh: a simple test suite, in the spirit of shell scripting Runs the test suite, on all tests in the current dir (default), or on specific tests or directories of tests given as arguments. * Tests are given as arguments, and executed in order, they can be either * an executable file * a directory, all executable files in it will be run as tests Filenames beginning with a . # _ or ending with "~" are ignored no recursion is done, subdirs will not be explicitely scanned except the ones ending in .subtests wthat will be also run If an executable file __INIT__ is found, it will be sourced into the shell (read) so that its definitions will be available for all the tests in it Same, if an __END__ file is found, it will be sourced after the tests * Nothing is printed if all tests OK. On failure, the stderr is printed * Tests exit code must be the number of failed tests (0 means OK) * A warning is shown if the test was OK but printed on stderr * Tests are run in their directory * Tewiba exit code is the total number of failed tests. How to write tests: * Tests are executable files (in any language), that exit with code 0 if OK, and the number of failures if not * Test file names can be meaningful, e.g: "Test for empty input" * Filenames beginning with a "." "#" "_" or ending with "~" are ignored as well as files named "tewiba" * They must print on stderr an explanation of the error, or a warning if exit code is 0. Prints on stdout are only shonw in verbose mode * Tests can create temporary files and dirs prefixed by "$tmp." they will be automatically cleaned afterwards by rm -rf $tmp $tmp.* * Test data can be put in subdirectories, as subdirectories are not tested * Convenience functions & variables that you can use in bash script tests: TLEVEL chars only runs if a level in chars is in the ones given by -l, or -l not specified. Each level is an alphanumeric char. TLEVEL_ONLY chars Same as above but does not run if -l not given TEST message prints sub-test (the file name is the test name) name if verbose, or before the first error of this subtest TERR message declares an error, but continues the tests FERR message fatal error: also aborts the current test file TEND MANDATORY: terminates your test script. (an alias for: exit $TFAILS) This is the only function really mandatory in a test file. TINIT id... For each id (composed of letters, numbers, and _ - .) ensures that files __INIT__id and __END__id are sourced, if they exist, before and after all tests of the dir, after and before __INIT__ and __END__ respectively DOTEST [-o|-e|-s] label expected-output-string parameters runs the command in $TOTEST - that you must define - with the parameters as its command line options, compare its output to the expected-output-string (or its stderr with -e, or its exit code status with -s. The default is -o for stdout), and in case of difference triggers a TERR displaying the label, the result and the expected string. $TV is true in verbose mode, false otherwise $tmp temporary data prefix (see above) $nl is the newline character $TEWIBA the version number of tewiba. Trick: put after #!/bin/bash the line: [ -n "$TEWIBA" ] || exec tewiba -v "$0" so that the test file can be executed standalone $TEWIBADIR the original directory where tewiba was run * All tewiba functions above print on stderr. * IMPORTANT: the shell function TCLEANUP must be called at the end of the tests, to clean the temporary data. tewiba traps the signal 0 for this. if you redefine trap 0 in your tests, or in a script you source, you should either make your trap function also call TCLEANUP, or redo a "trap TCLEANUP 0" at the end of your tests. Options: -v verbose: prints each test name as it is run, as well as its stdout -V just prints tewiba version number -x debug: execute each test in set -x mode copying also the output to /tmp/tewiba.$LOGNAME.out Note: does not imply -v -l levels runs only tests of the levels (characters) in the string levels "TLEVEL x" in a test runs when given "-l y" if [y] matches x: thus ranges are possible: -l 0-3 matches a TLEVEL 17a -s standalone: do not read __INIT__ nor __END__ files -e file outputs the total number of errors in the file. -f forces running tests files even without a TEND directive in them. Or add this line in the test file: #TNOEND Exemple simple test file: #!/bin/bash [ -n "$TEWIBA" ] || exec tewiba -v "$0" TEST screen installed hash screen || TERR screen command not found TEND Version 1.4.2 - Web site: https://%HTTP_HOST%/Code/Tewiba Copyright (c) 2013-2017 Colas Nahaboo (https://%HTTP_HOST%) MIT License: http://opensource.org/licenses/MIT