Leif Madsen April 29th, 2010
The Asterisk Test Suite is a way of developing external tests to Asterisk in order to test for functionality differences between different versions of Asterisk. By creating automated tests you are able to verify functionality you depend on remains undisturbed by changes in the code base. Tests you write can even be submitted back to the Asterisk project in order for them to be included directly in the test suite where the automated testing framework can execute the tests so developers will be alerted if a change they make causes a test to fail.
In order to write these tests we need to be able to run the test suite on our development machine (typically a virtual machine or some other box which you won’t be using for production). We’ll be installing the test suite on an Ubuntu Server 9.10 along with the dependencies, then running the automated tests. Writing automated tests though is beyond the scope of this document. See the README.txt file in the testsuite/ directory for more information (http://svn.asterisk.org/svn/testsuite/asterisk/trunk/README.txt).
We’ll be starting with a fresh, minimal install of Ubuntu Server 9.10, so once you’ve gotten that installed, feel free to move on.
Installing Dependencies
Before we get started building the test suite, we need to install the dependencies it requires. The following applications are necessary to run all the currently developed tests, but additional dependencies surely will be required in the future. The current list of dependencies is available in the README.txt file within the testsuite/ repository (referenced earlier). Don’t worry about installing all these dependencies now, because we’re going to step through all the applications you need to install throughout the post.
- SIPp
- python >= 2.4
- python-yaml
- bash
- asttest
- StarPy
- python-twisted
Building Asterisk and the Test Suite
The Asterisk Test Suite must be installed and run as root, so if you’re not root, switch now (sudo su -) and we’ll get started!
First thing we want to do is create our directory structure and then checkout the version of Asterisk we want to test, and then grab the test suite. If you haven’t already installed Subversion, then do that now (apt-get install subversion).
# cd /usr/src/
# mkdir asterisk-complete
# cd asterisk-complete
# mkdir asterisk
# cd asterisk
# svn co http://svn.asterisk.org/svn/asterisk/branches/1.6.2 1.6.2-vanilla
# cd 1.6.2-vanilla
# svn co http://svn.asterisk.org/svn/testsuite/asterisk/trunk testsuite
Notice that we checked out the latest 1.6.2 branch and named it 1.6.2-vanilla. We then changed into the 1.6.2-vanilla directory and checked out the test suite. This is because the scripts being run from the test suite expect to be located in a subdirectory of Asterisk.
The test suite expects Asterisk to already be installed the first time it runs, so we should make sure we can compile Asterisk. We can find the bare dependencies we need with the ./configure script and install those one at a time as it fails to find the necessary components for compiling.
Note: Of course you don’t have to run ./configure multiple times — you could just install all the necessary dependencies by running: apt-get install gcc g++ libxml2-dev
# pwd
/usr/src/asterisk-complete/asterisk/1.6.2-vanilla
# ./configure
configure: error: no acceptable C compiler found in $PATH
So we have our first dependency to resolve. In this case we’re missing the GCC compiler, so lets install that now with: apt-get install gcc. Several package dependencies will be listed. Press Y to continue with the installation of GCC and its dependencies.
Now that we have our GCC compiler installed lets see what our next dependency is.
# ./configure
configure: error: C++ preprocessor "/lib/cpp" fails sanity check
We’re missing the GCC C++ preprocessor, which can be installed by running: apt-get install g++. You will see several dependencies listed and they can all be installed by pressing Y.
If we run ./configure again we appear to get much closer, but not quite all the way.
# ./configure
checking for xml2-config... no
configure: *** XML documentation will not be available because the 'libxml2' development package is missing.
configure: *** Please run the 'configure' script with the '--disable-xmldoc' parameter option
configure: *** or install the 'libxml2' development package.
Appears we’re missing the development libraries for the XML documentation. Lets install that now with: apt-get install libxml2-dev. A couple of dependencies will be installed along with libxml2-dev after pressing Y.
# ./configure
configure: error: *** termcap support not found (on modern systems, this typically means the ncurses development package is missing)
Lets install the missing ncurses-dev package and dependencies with: apt-get install ncurses-dev
# ./configure
configure: Package configured for:
configure: OS type : linux-gnu
configure: Host CPU : i686
configure: build-cpu:vendor:os: i686 : pc : linux-gnu :
configure: host-cpu:vendor:os: i686 : pc : linux-gnu :
That’s it! If you see the big Asterisk symbol in ASCII text then you’ve made it to the end. Of course this is the bare minimum to get Asterisk compiled and installed, but we’ll use this for now. We’ll likely need to install some additional packages later to meet the dependencies of the test suite.
Lets try running the runtests.py script found in the testsuite/ subdirectory located in our Asterisk source. Remember that we checked this out earlier and is not part of the standard Asterisk checkout.
# cd testsuite/
# ./runtests.py
Traceback (most recent call last):
File "./runtests.py", line 15, in <module>
import yaml
ImportError: No module named yaml
OK, so we’re missing the python yaml package. We can install this with: apt-get install python-yaml
If we run the runtests.py script again we’ll get the following error:
# ./runtests.py
I/O Error getting Asterisk version from ../include/asterisk/version.h
This means we didn’t compile and install Asterisk prior to running our tests! So before continuing with our test suite install, we need to install Asterisk.
# cd ..
# pwd
/usr/src/asterisk-complete/asterisk/1.6.2-vanilla
# make install
Now that we have installed Asterisk, lets see where that gets us with our runtests.py script. We’re going to pass it the -l flag in order to list the tests we could potentially run, and the status of the dependencies they require.
# cd testsuite/
# ./runtests.py -l
Configured tests:
001) example
--> Summary: Check to see if the Asterisk binary has been installed
--> Minimum Version: 1.4 (True)
--> Dependency: bash -- Met: True
002) ami-login
--> Summary: Test loggin in to the Asterisk Manager Interface
--> Minimum Version: 1.4 (True)
--> Dependency: twisted -- Met: True
--> Dependency: starpy -- Met: False
003) blind-transfer-accountcode
--> Summary: Test account code propagation for SIP blind transfers.
--> Minimum Version: 1.4 (True)
--> Dependency: bash -- Met: True
--> Dependency: sipp -- Met: False
--> Dependency: asttest -- Met: False
004) rfc2833_dtmf_detect
--> Summary: Test RFC2833 DTMF detection
--> Minimum Version: 1.4 (True)
--> Dependency: bash -- Met: True
--> Dependency: sipp -- Met: False
--> Dependency: asttest -- Met: False
005) sip_channel_params
--> Summary: Test Retrieval of SIP CHANNEL parameters
--> Minimum Version: 1.8 (False)
--> Dependency: bash -- Met: True
--> Dependency: sipp -- Met: False
--> Dependency: asttest -- Met: False
006) iax-call-basic
--> Summary: Test a basic IAX2 call
--> Minimum Version: 1.4 (True)
--> Dependency: twisted -- Met: True
--> Dependency: starpy -- Met: False
007) manager-action-events-response
--> Summary: Test the presence and absence of a response from the Events manager action
--> Minimum Version: 1.4 (True)
--> Dependency: bash -- Met: True
--> Dependency: asttest -- Met: False
008) originate-cdr-disposition
--> Summary: Test for proper CDR dispositions when originating calls.
--> Minimum Version: 1.4 (True)
--> Dependency: bash -- Met: True
--> Dependency: sipp -- Met: False
--> Dependency: asttest -- Met: False
009) func_srv
--> Summary: Test func_srv for correctness
--> Minimum Version: 1.8 (False)
--> Dependency: asttest -- Met: False
010) sip_outbound_address
--> Summary: Test explicit outbound host for SIP calls
--> Minimum Version: 1.4 (True)
--> Dependency: bash -- Met: True
--> Dependency: sipp -- Met: False
--> Dependency: asttest -- Met: False
There will likely be more by the time you try this out, but I think that’s a pretty good sampling. Looking through the tests we can see several dependencies being met with True and several others being met with False. Our goal now is to make sure we satisfy all the dependencies for the tests prior to running them. Starting at the top, the first dependency we haven’t satisfied is starpy.
From the StarPy website at http://www.vrplumber.com/programming/starpy/, “StarPy is a Python + Twisted protocol that provides access to the Asterisk PBX’s Manager Interface (AMI) and Fast Asterisk Gateway Interface (FastAGI). Together these allow you write both command-and-control interfaces (used, for example to generate new calls) and to customise user interactions from the dial-plan. You can readily write applications that use the AMI and FastAGI protocol together with any of the already-available Twisted protocols“. Basically some tests are using it to generate calls in Asterisk to perform various tests.
We can install StarPy with the following commands:
# cd /usr/src/asterisk-complete/
# mkdir thirdparty
# cd thirdparty
# svn co https://starpy.svn.sourceforge.net/svnroot/starpy
# cd starpy/trunk/
# python setup.py install
You can see that we created a thirdparty/ directory within our asterisk-complete/ directory and checked out the latest version of StarPy. We then installed it via the setup.py script, and that’s it for satisfying the StarPy dependency.
After we change back to our testsuite/starpy dependency is satisfied:
# cd /usr/src/asterisk-complete/asterisk/1.6.2-vanilla/testsuite/
# ./runtests.py -l
...
--> Dependency: starpy -- Met: True
...
Let’s look and see what our next dependency is. Scrolling down a bit we can see the following:
--> Dependency: sipp -- Met: False
We need to install SIPp which is used for generating calls into the system. SIPp is a very useful utility for load testing Asterisk and executing detailed call flows. More information about SIPp is available at http://sipp.sourceforge.net/. We must install SIPp from source because some of the tests are already using advanced functionality for testing things like DTMF, so we can’t use the precompiled sip-tester package.
Note: Be sure to grab the latest SIPp snapshot. Right now that is version 2009-07-29.
# cd /usr/src/asterisk-complete/thirdparty
# mkdir sipp
# cd sipp
# wget http://sipp.sourceforge.net/snapshots/sipp.2009-07-29.tar.gz
# tar zxvf sipp.2009-07-29.tar.gz
# cd sipp.svn
# make pcapplay
send_packets.c:44:18: error: pcap.h: No such file or directory
make[1]: *** [send_packets.o] Error 1
make[1]: Leaving directory `/usr/src/asterisk-complete/thirdparty/sipp/sipp.svn'
make: *** [pcapplay] Error 2
Uh oh! We’re missing pcap support! Lets install that now.
# apt-get install libpcap-dev
Now we can compile SIPp with pcap support. There is no ‘make install‘ target to run, so we can just copy the resulting sipp binary to /usr/bin (or some other appropriate location that exists in your path).
# make pcapplay
# cp sipp /usr/bin
# cd /usr/src/asterisk-complete/asterisk/1.6.2-vanilla/testsuite/
After verifying that the SIPp dependency is available, we notice the next one in our list is asttest.
--> Dependency: asttest -- Met: False
The asttest dependency is an application distributed as part of the Asterisk test suite. It is dependent on Lua and the Lua development libraries. Switching in to the asttest/ directory and running ‘make‘ will provide the following error:
# cd asttest/
# pwd
/usr/src/asterisk-complete/asterisk/1.6.2-vanilla/testsuite/asttest
# make
make[1]: *** [src/lfs.o] Error 1
make[1]: Leaving directory `/usr/src/asterisk-complete/asterisk/1.6.2-vanilla/testsuite/asttest/lib/lua/luafilesystem-1.4.2'
make: *** [lib/lua/luafilesystem-1.4.2/src/lfs.o] Error 2
We can install the Lua development libraries and dependencies by running: apt-get install liblua5.1-0-dev lua5.1
# apt-get install liblua5.1-0-dev lua5.1
After installing Lua, we can run ‘make install‘ to install asttest.
At this point if you run ./runtests -l you will notice that all the dependencies have been satisfied (unless of course new tests have been added which require additional dependencies not yet required). So go ahead and run the tests and see if your checkout of Asterisk will pass all the tests!
# cd ..
# pwd
/usr/src/asterisk-complete/asterisk/1.6.2-vanilla/testsuite
# ./runtests.py
Once all the tests have completed, you’ll be provided a summary of the test that were run and their pass/fail status. The last part of the output will also be some XML output that you could use in scripts.
=== TEST RESULTS ===
--> example --- PASSED
--> ami-login --- PASSED
--> blind-transfer-accountcode --- PASSED
--> rfc2833_dtmf_detect --- PASSED
--> iax-call-basic --- PASSED
--> manager-action-events-response --- PASSED
--> originate-cdr-disposition --- PASSED
<?xml version="1.0" encoding="UTF-8"?>
<testsuite errors="0" time="158.71" tests="10" name="AsteriskTestSuite">
<testcase time="0.01" name="example"/>
<testcase time="25.51" name="ami-login"/>
<testcase time="12.24" name="blind-transfer-accountcode"/>
<testcase time="12.05" name="rfc2833_dtmf_detect"/>
<testcase time="20.35" name="iax-call-basic"/>
<testcase time="67.09" name="manager-action-events-response"/>
<testcase time="21.48" name="originate-cdr-disposition"/>
</testsuite>
As you can see, all 10 of our tests passed and took 158 seconds to execute. Now that you have the test suite infrastructure setup and executing all the tests, you can start building your own tests. More information about buildings tests is available in the README.txt file within the top level of your testsuite/ directory. Go forth and create tests!
