A script for functional testing of ab initio calculations
The Beetest program was forked out of a test suite for VASP installations that I started develop while in Uppsala. It was originally designed as a tool designed to run VASP calculations and analyze the output. But the system is very flexible, so it has found use in other programs as well (RSPt). In general, beetest can be used to construct test suites for any kind of binary that reads input files and produces output files, i.e. typical batch jobs.
- Python >= 2.6
- A Unix-compatible system (the program has never been tested on Windows)
Get the latest version: 0.6 (September 2011), by cloning the git repository, i.e.:
git clone git://www.nsc.liu.se/~pla/beetest.git
Inside the folder, you find the actual program, which is called “beetest”. Copy this file to a directory in your $PATH, e.g. /usr/local/bin.
The folder structure of a typical test suite using beetest looks like this:
TestSuite/ |---Tests/ | |---Cu-fcc/ | |---Fe-bcc/ | |---Fe-bcc/ | |---Si-cd/ | |---TiO2-rutile/ |---Scripts/ | |---Energy.sh | |---Pressure.sh | |---Volume.sh | |---Forces.sh | |---beetest.xml
To construct a test suite, one would follow these steps:
Put some folders with input files into a folder called “Tests”. These are your “tests” / “jobs” / “calculations”. Make sure that they are set to start from scratch every time (e.g ISTART=0 for VASP).
Write some scripts to extract interesting results from your output files (energies, forces etc). Place these in a folder called “Scripts”.
Make an XML file defining your test suite. See “beetest.xml” for a detailed example. The idea is to first connect each script to a property to be examined. Then, you make test cases using these defined properties.
Run the tests by changing directory to TestSuite/ in this case, and issue
beetest [path of executable]in the shell.
The beetest.xml file
<?xml version="1.0"?> <beetest> <properties> <property name="energy"> <description>Total energy (eV)</description> <script>Scripts/Energy.sh</script> <type>Float</type> <relativeprecision>1.0E-5</relativeprecision> </property> <property name="magmom"> <description>Magnetic moment (uB)</description> <script>Scripts/Magmom.sh</script> <type>Float</type> <absoluteprecision>1.0E-2</absoluteprecision> </property>
<testsuite name="quick"> <!-- These are quick calculations of simple periodic systems with high symmetry. To ensure that these are sane calculations, the results have also been cross-checked against previous published results in the literature --> <description>VASP 4/5 (small calculations)</description> <date>2011-02-03</date> <author>Peter Larsson</author> <testcase name="Fe bcc spin-polarized"> <path>Tests/Fe-bcc</path> <energy>-8.431093</energy> <magmom>2.2105</magmom> <iterations>16</iterations> <firstncg>884</firstncg> <potcar>PAW_PBE/Fe_pv</potcar> </testcase>
There are two kind of objects: test suites and test cases. You group together several
testcase, into a
testsuite, which can then be run as one set of calculations/tests.
Beetest has some flags to invoke more advanced features like regression versus earlier runs. Pass the “–help” flag to inspect available options.
[pla@matter1 vasptest]$ beetest --help Usage: beetest [options] Options: -h, --help show this help message and exit -f TESTINPUT, --file=TESTINPUT File with test suite definitions -p, --protocol Write test protocol to disk -r REGRESSION, --regression=REGRESSION Test for regression from earlier protocol -k, --keep-going Don't stop if there are errors -v, --verbose Print extra info -s SUITE, --suite=SUITE Run named test suite(s)
The “-s” flag is especially useful if the beetest.xml file contains many test suites, and only want to run one of them.