ssokol July 30th, 2010

• Uncategorized
• Comments(1)

Leif Madsen May 13th, 2010

The Asterisk 1.6.0 and 1.6.1 branches are now nearing their full maintenance mode period and will be shortly switching to security fixes only. Administrators are asked to migrate to the Asterisk 1.6.2 branch for continued bug fixes if they are required. The current set of release candidates for Asterisk 1.6.0 and 1.6.1 are 1.6.0.28-rc1 and 1.6.1.20-rc1 but a second set of release candidates will be created later today as some additional fixes have been merged into the branches.

Developers are asked to stop merging bug fix changes into these branches as the only changes should be security related.

More information about the maintenance periods for branches, please see http://www.asterisk.org/asterisk-versions which contains a table outlining when branches will move between the various maintenance cycles. Currently Asterisk 1.4 and 1.6.2 branches are scheduled to move to security fix only modes in December of 2010, although that date is likely to be extended until Asterisk 1.8 is fully available.

• News
• Comments(2)

Leif Madsen April 29th, 2010

The Asterisk Test Suite is a way of developing external tests to Asterisk in order to test for functionality differences between different versions of Asterisk. By creating automated tests you are able to verify functionality you depend on remains undisturbed by changes in the code base. Tests you write can even be submitted back to the Asterisk project in order for them to be included directly in the test suite where the automated testing framework can execute the tests so developers will be alerted if a change they make causes a test to fail.

In order to write these tests we need to be able to run the test suite on our development machine (typically a virtual machine or some other box which you won’t be using for production). We’ll be installing the test suite on an Ubuntu Server 9.10 along with the dependencies, then running the automated tests. Writing automated tests though is beyond the scope of this document. See the README.txt file in the testsuite/ directory for more information (http://svn.asterisk.org/svn/testsuite/asterisk/trunk/README.txt).

We’ll be starting with a fresh, minimal install of Ubuntu Server 9.10, so once you’ve gotten that installed, feel free to move on.

Installing Dependencies

Before we get started building the test suite, we need to install the dependencies it requires. The following applications are necessary to run all the currently developed tests, but additional dependencies surely will be required in the future. The current list of dependencies is available in the README.txt file within the testsuite/ repository (referenced earlier). Don’t worry about installing all these dependencies now, because we’re going to step through all the applications you need to install throughout the post.

• SIPp
• python >= 2.4
• python-yaml
• bash
• asttest
• StarPy
• python-twisted

Building Asterisk and the Test Suite

The Asterisk Test Suite must be installed and run as root, so if you’re not root, switch now (sudo su -) and we’ll get started!

First thing we want to do is create our directory structure and then checkout the version of Asterisk we want to test, and then grab the test suite. If you haven’t already installed Subversion, then do that now (apt-get install subversion).

# cd /usr/src/
# mkdir asterisk-complete
# cd asterisk-complete
# mkdir asterisk
# cd asterisk
# svn co http://svn.asterisk.org/svn/asterisk/branches/1.6.2 1.6.2-vanilla
# cd 1.6.2-vanilla
# svn co http://svn.asterisk.org/svn/testsuite/asterisk/trunk testsuite

Notice that we checked out the latest 1.6.2 branch and named it 1.6.2-vanilla. We then changed into the 1.6.2-vanilla directory and checked out the test suite. This is because the scripts being run from the test suite expect to be located in a subdirectory of Asterisk.

The test suite expects Asterisk to already be installed the first time it runs, so we should make sure we can compile Asterisk. We can find the bare dependencies we need with the ./configure script and install those one at a time as it fails to find the necessary components for compiling.

Note: Of course you don’t have to run ./configure multiple times — you could just install all the necessary dependencies by running: apt-get install gcc g++ libxml2-dev

# pwd
/usr/src/asterisk-complete/asterisk/1.6.2-vanilla
# ./configure
configure: error: no acceptable C compiler found in $PATH

So we have our first dependency to resolve. In this case we’re missing the GCC compiler, so lets install that now with: apt-get install gcc. Several package dependencies will be listed. Press Y to continue with the installation of GCC and its dependencies.

Now that we have our GCC compiler installed lets see what our next dependency is.

# ./configure
configure: error: C++ preprocessor "/lib/cpp" fails sanity check

We’re missing the GCC C++ preprocessor, which can be installed by running: apt-get install g++. You will see several dependencies listed and they can all be installed by pressing Y.

If we run ./configure again we appear to get much closer, but not quite all the way.

# ./configure
checking for xml2-config... no
configure: *** XML documentation will not be available because the 'libxml2' development package is missing.
configure: *** Please run the 'configure' script with the '--disable-xmldoc' parameter option
configure: *** or install the 'libxml2' development package.

Appears we’re missing the development libraries for the XML documentation. Lets install that now with: apt-get install libxml2-dev. A couple of dependencies will be installed along with libxml2-dev after pressing Y.

# ./configure
configure: error: *** termcap support not found (on modern systems, this typically means the ncurses development package is missing)

Lets install the missing ncurses-dev package and dependencies with: apt-get install ncurses-dev

# ./configure
configure: Package configured for:
configure: OS type  : linux-gnu
configure: Host CPU : i686
configure: build-cpu:vendor:os: i686 : pc : linux-gnu :
configure: host-cpu:vendor:os: i686 : pc : linux-gnu :

That’s it! If you see the big Asterisk symbol in ASCII text then you’ve made it to the end. Of course this is the bare minimum to get Asterisk compiled and installed, but we’ll use this for now. We’ll likely need to install some additional packages later to meet the dependencies of the test suite.

Lets try running the runtests.py script found in the testsuite/ subdirectory located in our Asterisk source. Remember that we checked this out earlier and is not part of the standard Asterisk checkout.

# cd testsuite/
# ./runtests.py
Traceback (most recent call last):
 File "./runtests.py", line 15, in <module>
 import yaml
ImportError: No module named yaml

OK, so we’re missing the python yaml package. We can install this with: apt-get install python-yaml

If we run the runtests.py script again we’ll get the following error:

# ./runtests.py
I/O Error getting Asterisk version from ../include/asterisk/version.h

This means we didn’t compile and install Asterisk prior to running our tests! So before continuing with our test suite install, we need to install Asterisk.

# cd ..
# pwd
/usr/src/asterisk-complete/asterisk/1.6.2-vanilla
# make install

Now that we have installed Asterisk, lets see where that gets us with our runtests.py script. We’re going to pass it the -l flag in order to list the tests we could potentially run, and the status of the dependencies they require.

# cd testsuite/
# ./runtests.py -l
Configured tests:
001) example
 --> Summary: Check to see if the Asterisk binary has been installed
 --> Minimum Version: 1.4 (True)
 --> Dependency: bash -- Met: True
002) ami-login
 --> Summary: Test loggin in to the Asterisk Manager Interface
 --> Minimum Version: 1.4 (True)
 --> Dependency: twisted -- Met: True
 --> Dependency: starpy -- Met: False
003) blind-transfer-accountcode
 --> Summary: Test account code propagation for SIP blind transfers.
 --> Minimum Version: 1.4 (True)
 --> Dependency: bash -- Met: True
 --> Dependency: sipp -- Met: False
 --> Dependency: asttest -- Met: False
004) rfc2833_dtmf_detect
 --> Summary: Test RFC2833 DTMF detection
 --> Minimum Version: 1.4 (True)
 --> Dependency: bash -- Met: True
 --> Dependency: sipp -- Met: False
 --> Dependency: asttest -- Met: False
005) sip_channel_params
 --> Summary: Test Retrieval of SIP CHANNEL parameters
 --> Minimum Version: 1.8 (False)
 --> Dependency: bash -- Met: True
 --> Dependency: sipp -- Met: False
 --> Dependency: asttest -- Met: False
006) iax-call-basic
 --> Summary: Test a basic IAX2 call
 --> Minimum Version: 1.4 (True)
 --> Dependency: twisted -- Met: True
 --> Dependency: starpy -- Met: False
007) manager-action-events-response
 --> Summary: Test the presence and absence of a response from the Events manager action
 --> Minimum Version: 1.4 (True)
 --> Dependency: bash -- Met: True
 --> Dependency: asttest -- Met: False
008) originate-cdr-disposition
 --> Summary: Test for proper CDR dispositions when originating calls.
 --> Minimum Version: 1.4 (True)
 --> Dependency: bash -- Met: True
 --> Dependency: sipp -- Met: False
 --> Dependency: asttest -- Met: False
009) func_srv
 --> Summary: Test func_srv for correctness
 --> Minimum Version: 1.8 (False)
 --> Dependency: asttest -- Met: False
010) sip_outbound_address
 --> Summary: Test explicit outbound host for SIP calls
 --> Minimum Version: 1.4 (True)
 --> Dependency: bash -- Met: True
 --> Dependency: sipp -- Met: False
 --> Dependency: asttest -- Met: False

There will likely be more by the time you try this out, but I think that’s a pretty good sampling. Looking through the tests we can see several dependencies being met with True and several others being met with False. Our goal now is to make sure we satisfy all the dependencies for the tests prior to running them. Starting at the top, the first dependency we haven’t satisfied is starpy.

From the StarPy website at http://www.vrplumber.com/programming/starpy/, “StarPy is a Python + Twisted protocol that provides access to the Asterisk PBX’s Manager Interface (AMI) and Fast Asterisk Gateway Interface (FastAGI). Together these allow you write both command-and-control interfaces (used, for example to generate new calls) and to customise user interactions from the dial-plan.  You can readily write applications that use the AMI and FastAGI protocol together with any of the already-available Twisted protocols“. Basically some tests are using it to generate calls in Asterisk to perform various tests.

We can install StarPy with the following commands:

# cd /usr/src/asterisk-complete/
# mkdir thirdparty
# cd thirdparty
# svn co https://starpy.svn.sourceforge.net/svnroot/starpy
# cd starpy/trunk/
# python setup.py install

You can see that we created a thirdparty/ directory within our asterisk-complete/ directory and checked out the latest version of StarPy. We then installed it via the setup.py script, and that’s it for satisfying the StarPy dependency.

After we change back to our testsuite/starpy dependency is satisfied:

# cd /usr/src/asterisk-complete/asterisk/1.6.2-vanilla/testsuite/
# ./runtests.py -l
...
--> Dependency: starpy -- Met: True
...

Let’s look and see what our next dependency is. Scrolling down a bit we can see the following:

--> Dependency: sipp -- Met: False

We need to install SIPp which is used for generating calls into the system. SIPp is a very useful utility for load testing Asterisk and executing detailed call flows. More information about SIPp is available at http://sipp.sourceforge.net/. We must install SIPp from source because some of the tests are already using advanced functionality for testing things like DTMF, so we can’t use the precompiled sip-tester package.

Note: Be sure to grab the latest SIPp snapshot. Right now that is version 2009-07-29.

# cd /usr/src/asterisk-complete/thirdparty
# mkdir sipp
# cd sipp
# wget http://sipp.sourceforge.net/snapshots/sipp.2009-07-29.tar.gz
# tar zxvf sipp.2009-07-29.tar.gz
# cd sipp.svn
# make pcapplay
send_packets.c:44:18: error: pcap.h: No such file or directory
make[1]: *** [send_packets.o] Error 1
make[1]: Leaving directory `/usr/src/asterisk-complete/thirdparty/sipp/sipp.svn'
make: *** [pcapplay] Error 2

Uh oh! We’re missing pcap support! Lets install that now.

# apt-get install libpcap-dev

Now we can compile SIPp with pcap support. There is no ‘make install‘ target to run, so we can just copy the resulting sipp binary to /usr/bin (or some other appropriate location that exists in your path).

# make pcapplay
# cp sipp /usr/bin
# cd /usr/src/asterisk-complete/asterisk/1.6.2-vanilla/testsuite/

After verifying that the SIPp dependency is available, we notice the next one in our list is asttest.

--> Dependency: asttest -- Met: False

The asttest dependency is an application distributed as part of the Asterisk test suite. It is dependent on Lua and the Lua development libraries. Switching in to the asttest/ directory and running ‘make‘ will provide the following error:

# cd asttest/
# pwd
/usr/src/asterisk-complete/asterisk/1.6.2-vanilla/testsuite/asttest
# make
make[1]: *** [src/lfs.o] Error 1
make[1]: Leaving directory `/usr/src/asterisk-complete/asterisk/1.6.2-vanilla/testsuite/asttest/lib/lua/luafilesystem-1.4.2'
make: *** [lib/lua/luafilesystem-1.4.2/src/lfs.o] Error 2

We can install the Lua development libraries and dependencies by running: apt-get install liblua5.1-0-dev lua5.1

# apt-get install liblua5.1-0-dev lua5.1

After installing Lua, we can run ‘make install‘ to install asttest.

At this point if you run ./runtests -l you will notice that all the dependencies have been satisfied (unless of course new tests have been added which require additional dependencies not yet required). So go ahead and run the tests and see if your checkout of Asterisk will pass all the tests!

# cd ..
# pwd
/usr/src/asterisk-complete/asterisk/1.6.2-vanilla/testsuite
# ./runtests.py

Once all the tests have completed, you’ll be provided a summary of the test that were run and their pass/fail status. The last part of the output will also be some XML output that you could use in scripts.

=== TEST RESULTS ===
--> example --- PASSED
--> ami-login --- PASSED
--> blind-transfer-accountcode --- PASSED
--> rfc2833_dtmf_detect --- PASSED
--> iax-call-basic --- PASSED
--> manager-action-events-response --- PASSED
--> originate-cdr-disposition --- PASSED

<?xml version="1.0" encoding="UTF-8"?>
<testsuite errors="0" time="158.71" tests="10" name="AsteriskTestSuite">
 <testcase time="0.01" name="example"/>
 <testcase time="25.51" name="ami-login"/>
 <testcase time="12.24" name="blind-transfer-accountcode"/>
 <testcase time="12.05" name="rfc2833_dtmf_detect"/>
 <testcase time="20.35" name="iax-call-basic"/>
 <testcase time="67.09" name="manager-action-events-response"/>
 <testcase time="21.48" name="originate-cdr-disposition"/>
</testsuite>

As you can see, all 10 of our tests passed and took 158 seconds to execute. Now that you have the test suite infrastructure setup and executing all the tests, you can start building your own tests. More information about buildings tests is available in the README.txt file within the top level of your testsuite/ directory. Go forth and create tests!

• Documentation
• Comments(7)

Kevin Fleming April 19th, 2010

As most people who have been following industry news are probably aware, the move to IPv6 addressing is getting underway in a big way right now, as worldwide the IP address registries are scrambling to recover ‘lost’ IPv4 address space from companies/institutions that were granted large chunks in the past. In an effort to help with that transition for the Asterisk community, we’ve IPv6-enabled many of the community services you know and love (through the use of an IPv6 tunnel from tunnelbroker.net). Users should be able to access these sites using IPv6:

http://downloads.asterisk.org

http://downloads.digium.com

https://issues.asterisk.org

http://packages.asterisk.org

https://reviewboard.asterisk.org

http://svn.digium.com

http://svnview.digium.com

Enjoy!

• News
• Comments Off

Leif Madsen January 29th, 2010

About Asterisk 1.6.2

Now that Asterisk 1.6.2.0 (and 1.6.2.1) has recently been released, we
thought it wise to create an announcement about some of the new items and
changes available in the new feature release, along with where Asterisk is going
in the next few months.

As with all new releases, if you plan on upgrading from any previous release
be sure you test thoroughly in a test environment, and carefully read both the
CHANGES and the UPGRADE.txt files, which contain useful information about
functionality changes between versions.

Some of the new features included in the Asterisk 1.6.2.0 release are:

SIP Changes

• if the channel variable ATTENDED_TRANSFER_COMPLETE_SOUND is set, the sound will be played to the target of an attended transfer
• Added support for subscribing to MWI on a remote server and making the status available as a mailbox. Please see the sip.conf.sample file for more information.
• Added support for ITU G.722.1 and G.722.1C (Siren7 and Siren14) media streams.

DAHDI Changes

• chan_dahdi now supports MFC/R2 signalling when Asterisk is compiled with support for LibOpenR2. http://www.libopenr2.org/
• As of version 1.6.1, Asterisk no longer depends on DAHDI as the sole timing source. In 1.6.2, an additional timing provider was introduced, res_timing_timerfd, which takes advantage of the timerfd API provided by newer versions of the Linux kernel. This timing provider provides much higher performance than the other non-DAHDI timing module, res_timing_pthread.

Dialplan Functions

• Added a new dialplan function, CURLOPT, which permits setting various options that may be useful with the CURL dialplan function, such as cookies, proxies, connection timeouts, passwords, etc.
• Added debugging CLI functions to func_odbc, ‘odbc read’ and ‘odbc write’.
• func_odbc now may specify an insert query to execute, when the write query affects 0 rows (usually indicating that no such row exists).
• func_odbc now supports database transactions across multiple queries.
• Added REALTIME_FIELD and REALTIME_HASH, which should aid users in better obtaining realtime data from the dialplan.

Dialplan Applications

• Scheduled meetme conferences may now have their end times extended by using MeetMeAdmin.
• A new application, Originate, has been introduced, that allows asynchronous call origination from the dialplan.
• Added ConfBridge dialplan application which does conference bridges without DAHDI.

Additional Features

• extensions.conf now allows you to use keyword “same” to define an extension without actually specifying an extension. It uses exactly the same pattern as previously used on the last “exten” line. For example:
  

       exten => 123,1,NoOp(something)
       same  =>     n,SomethingElse()
  

• All deprecated CLI commands are removed from the source code. They are now handled by the new clialiases module. See cli_aliases.conf.sample file.
• The contrib/scripts/ directory now has a script called sip_nat_settings that will give you the correct output for an Asterisk box behind nat. It will give you the externhost and localnet settings.
• The Asterisk core now supports ITU G.722.1 and G.722.1C media streams, and can connect calls in passthrough mode, as well as record and play back files.

Version Numbering Changes

Asterisk 1.6.2 will be the last release in the 1.6 series of feature releases.
The original purpose of the Asterisk 1.6 releases was to change how often a
release was created, in order to allow shorter time periods between versions so
the community wouldn’t have to wait 2-3 years for a new feature. The intention
was to release a new 1.6 version, which included feature additions and
performance increases, every 3-4 months.

Unfortunately, the time frame ended up being closer to every 6 months, so the
original intention wasn’t fully developed. In addition, the numbering scheme
was confusing to some users. (Additional information about the how and the why of
the 1.6 numbering scheme can be read at:
http://blogs.asterisk.org/2009/06/24/about-the-new-asterisk-versioning-method/
)

So after about a year and a half of releases (1.6.0.x, 1.6.1.x, and 1.6.2.x),
community feedback was considered, and all that was learned during the 1.6
release cycle was put together to create a better release scheme, which takes
the advantages of both the long stable release cycles, and shorter feature
release cycles of Asterisk, while adding a layer of predictability allowing the
community to plan deployments accordingly.

Whenever a release is created, it will now be tagged either as Standard or LTS
(Long Term Support). Asterisk 1.4 would be considered an LTS release, meaning it
receives bug fixes for a longer period of time, of at least 4 years. A standard
release would have a shorter bug fix release period of at least 1 year.

After the support period expires, that release would then receive security
updates for at least 1 year (Asterisk 1.2 is an example of this), and eventually
would become end-of-life (Asterisk 1.0), thereby no longer receiving updates.

All Asterisk 1.6.x releases are considered Standard releases.

The next LTS (Long Term Support) release will be Asterisk 1.8, slated to be
branched from trunk around Q2 of 2010, at which point time will be spent testing
and getting it ready for full release as Asterisk 1.8.0.

For more information about the current state of releases for versions of
Asterisk currently available see: http://www.asterisk.org/asterisk-versions.

Upcoming Features in Asterisk 1.8

Asterisk 1.8 is shaping up to be an exciting LTS release which will contain
several performance improvements and many new features. Some of the new features
to look forward to in Asterisk 1.8 include:

• The much awaited SRTP support in chan_sip will be added.
• The chan_mgcp module has added PacketCable NCS 1.0 support for Docsis/Eurodocsis Networks. See configs/res_pktccops.conf for more information.
• The chan_spy module has several new options added, including:
  • Added c() option allowing custom DTMF to be set to cycle through the next available channel. By default this is still ‘*’.
  • Added x() option allowing DTMF to be set to exit the application.
  • Added the ‘S’ option, which makes the application automatically exit once it hits a point where no more channels are available to spy on.
  • Added the ‘E’ option, which spies on a single channel and exits when that channel hangs up.
• The MeetMe application now turns on the DENOISE() function by default, for each participant. In our tests, this has significantly decreased background noise (especially noisy data centers).
• A new interface, Channel Event Logging (CEL), is introduced here. CEL logs single events, much like the AMI, but it differs from the AMI in that it logs to database backends much like CDR does.
• A new set of modules were added supporting calendar integration with Asterisk. Dialplan functions for reading from and writing to calendars are included, as well as the ability to execute dialplan logic upon calendar event notifications.
• A new RTP engine and channel driver have been added which supports Multicast RTP. The channel driver can be used with the Page application to perform multicast RTP paging.
• In app_queue, you can now specify a position at which the caller will enter the queue.
• Information regarding the party that a person is currently connected to may be updated dynamically throughout the call. This has the advantage of allowing for the display of a phone to be updated during situations such as a call forward or a transfer.
• A new feature, call completion, will be added. This feature allows for a caller who reaches an unresponsive or busy party to be automatically contacted when the called party becomes available again.
• Updates
• Comments(1)

Kevin Fleming December 3rd, 2009

Over the past year, some of us at Digium have spent many (MANY) hours working on FAX support in Asterisk (even though we’d all prefer to see FAX go away as the obsolete technology that it should be <G>). Some of this work was part of our release of our commercial FAX for Asterisk product, some of it was driven by our desires to have solid FAX support in our commercial PBX products, but most of it was driven by the need to get open source Asterisk (and its community of users) able to use FAX reliably with as many non-Asterisk endpoints as possible.

In Asterisk 1.4 and prior releases, there was limited support for FAX of any kind; Asterisk 1.4 can pass through T.38 sessions on SIP channels, but that’s all. With an open source addon (based on Steve Underwood’s excellent spandsp library), it can have dialplan applications to send and receive FAX over audio (G.711 TDM) links.

During the development of Asterisk 1.6.0, Steve’s FAX dialplan applications were merged into asterisk-addons, and then directly into the Asterisk source tree, where they became the ‘app_fax’ module. In addition, the T.38 negotiation process was redesigned to allow Asterisk applications to actually be T.38 endpoints; this resulted in the ability to send and receive FAX over audio *and* T.38 links.

Once this got into the community’s hands, we started seeing large numbers of bug reports because users could not successfully FAX over T.38 with various ATAs and service providers.  I won’t go into the gory details of why this was the case, except to repeat a recent quote from Steve himself on IRC: “The T.38 spec was written after a night of heavy drinking as a joke.”. While that’s not technically true, it is true that compliance with the T.38 recommendation, primarily in the SIP/SDP negotiations area, is very poor across the industry. Producing a T.38 endpoint that interoperates widely is a complex and difficult process, that can only be achieved through hours and hours (and hours) of testing.

As these problem reports got more severe, we took a significant step: we rewrote the T.38 negotiation mechanisms in Asterisk *again*. These changes first appeared in the 1.6.0.14 and 1.6.1.5 releases. The in-tree app_fax application was updated to support these changes along with them, so open source FAX users got the benefits of these changes immediately… and the result has been wonderful. We get very few T.38 negotiation-related bug reports now, and in nearly every case we can point to a misconfiguration or severely broken far endpoint that is the cause of the problem. For many, many people, FAX over T.38 in Asterisk 1.6.0 and 1.6.1 ‘just works’ now.

While all of this was going on, Digium was also working on our commercial Fax for Asterisk product, which provides comparable functionality to app_fax, but uses a commercial FAX stack. When we began the development of this product, we knew that we wanted as many portions of it as possible to end up open source, so rather than build it as a monolithic module, we built two modules: res_fax and res_fax_digium. res_fax is similar to app_fax, in that it provides dialplan applications, dialplan functions, and other associated components to send and receive FAX. However, it does not actually include any FAX stack; for that, it provides a plugin interface that allows one (or more) additional modules to be loaded to actually provide the FAX technology implementation. It *does* handle all T.38 negotiations, however, using the 3rd generation T.38 implementation in Asterisk.

Of course, res_fax_digium is Digium’s commercial plugin module for res_fax, and we have been delivering the combination of those two modules to our Fax For Asterisk customers for quite a few months now. (Unfortunately it has taken longer to get these modules updated to match Asterisk’s T.38 API than we would have liked, but that’s not important for open source users). Recently, with a final set of changes to the UDPTL stack in Asterisk, we’ve reached the point where we think we’ve implemented all the parts of the T.38 recommendation that we can implement, in as compatible and interoperable a way as we can, with a few configuration options that provide the ability to override broken behavior by far endpoints when necessary. In fact, the combination of the very most recent Asterisk open source releases and either app_fax or res_fax+res_fax_digium now successfully interoperates with quite a few
T.38 ATAs we have in our lab and a couple of service providers, and we’re testing more of both as I write this. In nearly every case, this works *without* requiring any configuration changes in Asterisk or the FAX applications, which is good news for end users indeed. We’ve even extended (and fixed) the ‘faxdetect’ functionality in chan_sip so that it works as users expect it to, in a similar way to the faxdetect feature in chan_dahdi.

Now that this point has been reached, it’s time for us to act on our plan to open source res_fax; to that end, I’ve created a new branch in Subversion (/svn/asterisk/team/group/res_fax) based on Asterisk trunk, and merged the most recent version of res_fax into it, updating it to compile against trunk’s recent API changes. By the time Asterisk 1.8 is released, res_fax will *replace* app_fax, as it provides the identical dialplan applications (same names, options and operations), but has additional features and compatibility functionality. To achieve this, though, we’re going to need a res_fax_spandsp plugin module, so that the combination of res_fax + res_fax_spandsp provides the same functionality as app_fax did, and it will be a seamless transition for Asterisk 1.6.x FAX users to move to Asterisk 1.8 when they are ready. We’ve tasked one of our developers to start working on this over the next few weeks, and I’m sure you’ll see some initial steps toward that end shortly. For those of you who have contributed improvements to app_fax and use it heavily (or even those who just use it and can provide testing), I’d encourage you to post a comment in the thread on the asterisk-dev mailing list if you are interested and willing to assist in this effort.

Now, on to even more interesting stuff: as I’ve worked on this over the past 6-8 months, I’ve also spent time finally trying to understand how best to fit TDM-T.38 FAX relay into Asterisk. I know that there are number of community developers who have been working on this, and there are multiple patches in the issue tracker that provide this functionality in various forms. There was also an attempt by Daniel Ferenci to start a discussion on the asterisk-dev list about the best long-term approach for relay support, but he didn’t really get any responses.

Once res_fax and res_fax_spandsp are ready for users to use, it’ll be time for them to be extended to support FAX relay as well. I would like to propose that this be implemented by res_fax providing a new API, a ‘FAX relay session’, that a channel driver (one which services TDM channels) can use to offer T.38 support to the rest of Asterisk, as if it supported T.38 natively. We can start a separate discussion on the list to talk about particulars, but I believe this is the cleanest approach with the least impact on existing code in Asterisk, and I’d like to get opinions from other interested parties and discussion going.

In summary, it appears that Asterisk 1.6.x has achieved pretty solid FAX support (especially over T.38), and that with Asterisk 1.8 we’ll be able to improve it further and begin moving towards FAX relay support as well. For all of you who still insist on using this obsolete technology <G>, I hope this has provided what you need to be able to keep using Asterisk
in every place that it makes sense in your networks!

• News
• Comments(4)

Russell Bryant November 10th, 2009

For the last three years, I have had the opportunity to give an Asterisk Project Update presentation at AstriCon. It really feels good to look back over what we have accomplished during the past year and share some of the highlights. This year, the Asterisk Project Update presentation was a joint presentation between myself and Kevin P. Fleming.

For those who were not able to attend the conference, I would like to share what Kevin and I had to say. There were three main topics that we covered:

1. Growth of the Asterisk developer community
2. Asterisk release process updates
3. New features and architectural improvements

Developer Community Growth

Asterisk trunk is the main development branch for Asterisk. This is where we are preparing the newest changes for the next major release. For example, new features that go into Asterisk trunk today will be first available in Asterisk 1.8 (wait, what?! 1.8?! Yeah, yeah. I’ll get back to that in a bit!) Asterisk trunk stays very busy. Here are some measurements regarding activity in trunk over the last year:

• 2320 commits
• 825 files changed
• 322148 Lines of code added
• 53251 Lines of code removed

While lines of code by itself is not a terribly meaningful measurement, it can be used along with other measurements to provide some insight. For example, if we look at the growth of lines of code over time in Asterisk trunk, we can see how the rate of growth in the size of the code base has continued to increase over the project’s lifetime.

This growth has not come from one single person. However, if someone can teach me how to exponentially increase my productivity over time, I would love to talk to you! A lot of people contribute to Asterisk. Among those writing code for Asterisk, there is a select group that has direct access to make changes to the source (committers). At the beginning of the project, there was effectively one committer, Mark Spencer. As the project has grown, we have worked very hard at scaling our development community such that we can process more code. The number of committers today is over 50 (with the number of contributors much higher than that). The following graph demonstrates lines of code over time by committer and helps demonstrate how this has changed over the life of the project.

There are a number of ways we can take a look at the size of the group of people contributing code to Asterisk. One such way is to look at how many people have signed the contributor license agreement over time. Previously, this was a written form handled via fax. As of a couple of years ago, we now track these license agreements electronically. From this data, we can see that over 800 people have signed up to contribute to Asterisk over the past couple of years.

Asterisk Release Policy Updates

Release policy is effectively the agreement between the developer and user communities on how and when updates will be delivered. There are needs to meet on both sides. The development community wants to continually improve the project while the user community needs stable software that continues to operate in a defined manner.

The history of Asterisk releases begins 10 years ago. Here are some dates on releases during the first half of the project’s lifetime:

• 0.1 – December 1999
• 0.2 – September 2002
• 0.3 – February 2003
• 0.4 – April 2003
• 0.5 – September 2003
• 0.7 – January 2004
• 0.9 – April 2004

If you’re reading this and have been using Asterisk long enough to remember these releases, cool! That means you were involved in the project longer than me. I got involved in the Asterisk project in the middle of 2004. By the first Astricon in the Fall of 2004, Mark Spencer decided to release Asterisk 1.0 and asked me to maintain it.

• Asterisk 1.0
  • Fall of 2004
  • Regular 1.0.X updates with bug fixes only
  • Eventually went into security only maintenance, and is no longer maintained today

Asterisk development continued and over the next couple of years and we released Asterisk 1.2 and Asterisk 1.4. We made some changes to development processes regarding how and when to port bug fixes and how they were merged between releases. However, the release policies regarding what went into updates and roughly how often they were released didn’t change.

• Asterisk 1.2
  • Released November of 2005
  • Still updated with security fixes only
• Asterisk 1.4
  • Released December of 2006
  • Still fully maintained

At this point in the project, we had identified some problems, and decided we’d like to try some changes to improve our releases. We decided to modify our approach with releases for the Asterisk 1.6.X series. Our goals were:

1. Release a feature update as 1.6.X every 3 months or so.
2. Deliver small feature increments
3. Maintain backwards compatibility.
4. Maintain each increment for at least one year with regular bug fix updates labeled as 1.6.X.Y.

The problems we had identified that we wanted to address were:

1. Asterisk 1.4 took off to a rough start. It took longer to stabilize this code than it really should have.
2. The way we were managing our releases meant that the time to market on new features was longer than we liked.
3. Some of our policies regarding how changes were implemented meant that upgrades between major versions were sometimes painful.

Some of these issues we worked on addressing with the 1.6.X release plans. Others we have worked in by improving development processes.

1. Release quality
   • We have improved our release candidate processes. We now do a much better job at ensuring that releases, including bug fix updates, get more testing before they are published.
   • We decided that we wanted to shorten our major release cycle so that when it was time to test a new feature set, the target was more manageable.
   • We have deployed tools and implemented processes for more strict code reviews on changes before they are merged.
   • We have worked to educate the development team on best practices and how to avoid common problems encountered in Asterisk.
2. Long time to market on features
   • By shortening the release cycle, we made it such that features would be available sooner than they would have been before.
3. Painful upgrades
   • We have come to embrace backwards compatibility much more aggressively than in the past. Even if there is a newer better way to do something, we keep the old way around if we can.
   • We document every single change that users upgrading between versions should be aware of in UPGRADE.txt.

In reality, Asterisk 1.6.X has not worked out exactly as we had envisioned it. While we had planned on doing updates every 3 months, it has been closer to every 6 months.

• Asterisk 1.6.0 – October 2008
• Asterisk 1.6.1 – April 2009
• Asterisk 1.6.2 – Q4 2009

Another interesting point is that we envisioned the changes between 1.6.X and 1.6.X+1 to be much smaller than it has been in reality. The longer release cycle and the growth in the development community has made it such that the changes in Asterisk between updates are very significant.

With all of that history out of the way, we are now up to present time. After reviewing how our Asterisk 1.6.X plans have worked out, we have decided to propose some new updates to our release policies. The first change is to introduce the concept of a release type for feature releases. The purpose of this is to give users a clear understanding up front how long a major release of Asterisk will be maintained. There are two release types: standard and LTS.

1. Standard
   • Fully maintained for 1 year
   • Security fixes only for 1 additional year
2. LTS
   • Fully maintained for 4 years
   • Security fixes only for 1 additional year

The other change to the release policies that we would like to make is to switch back to our original numbering scheme. We started giving feature updates 1.6.X labels since the changes between each were supposed to be much smaller than a jump between 1.0 and 1.2, or 1.2 to 1.4, etc. So, instead of the next release being called 1.6.3, we’ll just call it 1.8 to better reflect what it really is.

Here is the anatomy of an Asterisk version number:

<Concept>.<Feature>.<Minor>[.Patch]

• Concept – Something close to a complete rewrite would be required to change this
• Feature – An update to this number indicates a change to the feature set
• Minor – This number reflects an update with bug fixes only
• Patch – Trivial Changes (usually for a security release)

Here is a table of releases, their type, and associated dates:

Asterisk 1.8 is currently targeted for Q2 of 2010, so we still have time to make changes. Feel free to post your comments here or on the Asterisk-dev mailing list if you have feedback!

Asterisk Features

A whole lot of code has been written in the last year. Some of it is very visible to users, while other changes are architectural in nature and provide a boost under the hood. Here is a list of some cool things that have been worked on. For a full list of new features, see the CHANGES file in Asterisk trunk.

1. Fax Support Improvements (All versions)
   • T.38 negotiation support in Asterisk has been completely rewritten to work much better.
   • Full support for T.38 send/receive (gateway support in progress)
   • Configuration options added to improve interoperability with non-compliant T.38 implementations.
   • Many changes have been made to chan_dahdi and DAHDI to improve the stability of FAX over PSTN.
   • Hundreds of hours have been put in to rigorous testing of FAX support in Asterisk.
2. XMPP/Jabber Integration Improvements (1.8+)
   • The JABBER_RECEIVE() function has been added to allow you to receive XMPP messages in the Asterisk dialplan.
   • There is code in testing to use XMPP as a transport for distributed events. This will allow Asterisk servers peered via XMPP to share device state and MWI information.
3. Connected / Redirecting Party ID Support (1.8+)
   • Full control over connected party ID updates. Phone displays will now be correct after transfers!
   • Redirecting party ID support means you can see when your call has been forwarded, or when you receive a call that has been forwarded.
   • See Mark Michelson’s AstriCon presentation video for more information on this functionality.
4. Call Completion Supplementary Services (Hopefully 1.8+)
   • “Camp on extensions”
   • Support for both CCNR and CCBS
   • Generic support in Asterisk, as well as support for doing CCSS over SIP and ISDN
5. Calendar Integration (1.8+)
   • Support for iCal, CalDAV, Exchange 2003
   • Use calendar information as a device state provider
   • Access calendar state in the dialplan
   • Originate calls based on calendar entries.
   • See Terry Wilson’s AstriCon presentation video for more information on this feature.
6. Security Events Framework (1.8+)
   • This infrastructure allows Asterisk components to report events that are potentially related to attacks on the system.
   • This also includes a module that will write security events out to a log file in a defined format that can be used by outside applications.
7. SIP TCP/TLS Improvements (1.6.X+)
   • There has been a lot of additional testing on this functionality.
   • The related configuration options have been improved.
   • There have been successful reports of integration with Microsoft OCS.
   • There is continued work going on to make this functionality more robust under adverse network conditions.
8. Updated PSTN Support
   • There have been many improvements made to BRI support via mISDN in all supported versions of Asterisk.
   • Native BRI support in LibPRI and chan_dahdi has been added to Asterisk 1.6. Features are being actively worked on in this code.
   • MFC/R2 signaling support has been added to chan_dahdi by using libopenr2 in Asterisk version 1.6.2 and later.
   • SS7 support was added in Asterisk 1.6.0 and it continues to mature.
9. Core Bridging API (1.6.2+)
   • It is now much easier to write Asterisk modules that bridge channels.
   • This new bridging infrastructure can conference channels together without DAHDI installed on the system.
   • A new conferencing application (ConfBridge) has been provided that allows conferencing using this new infrastructure.
10. Core Timing API (1.6.1+)
    • Support for timing in Asterisk has been abstracted instead of relying on DAHDI timers directly. So, DAHDI is no longer required as a timing source for Asterisk.
11. Core Channel API Update (1.8+)
    • The management of the most widely used data structure in Asterisk, ast_channel, has been rewritten. It now uses the astobj2 object model. The result is that less locking is needed on this data, and code that does channel iterations or lookups is much more efficient.
12. Core Scheduler API Update (1.6.2+)
    • The scheduler API is used in Asterisk when components need to schedule some task to happen in the future. For example, this is used for message retransmissions and timer expirations. It is used very heavily in Asterisk channel drivers. The scheduler API went through two rounds of performance improvements in Asterisk 1.6 (1.6.1 and again in 1.6.2). While doing profiling, the results of this work show the scheduler going from near the top of the CPU consumers down to completely off of the radar.

Summary

A lot has happened in the last year. Our development community continues to grow at a healthy rate. Our release processes are changing to better suit the needs of both users and developers. A lot of really good code is being written.

I am really looking forward to the next year. I can promise you now that it is going to be even more exciting than this one.

Thanks for reading.

• News
• Comments(11)

ssokol October 29th, 2009

A couple of weeks ago at AstriCon we announced the opening of the AsteriskForge, a collaborative development site for all kinds of Asterisk-related open source projects.  The idea behind AsteriskForge is to create a public center for the development and distribution of open source code (both binary and source) that connect with Asterisk.

As a challenge to the community, I’m offering a $25 gift certificate from ThinkGeek to the first 10 working software projects (i.e. projects with useful, downloadable, runnable, Asterisk-related software) to move in and set up shop in the Forge.  (Sorry – no hardware projects, artwork or poetry  qualify for this promotion.)

The winners will be picked by the official Digium Prize Selection Committee (me and whomever I can round up in the Digium cafeteria) and all decisions are final.  If we have to debate the usefulness of an app, it’s probably not useful, so no stuffing the ballot box with Perl AGI scripts that read back the current time.  May the forge be with you…

Forge Q&A

I’ve received a few questions about the Forge since the announcement and would like to share the answers with everyone.  Please let me know if you have any additional questions.

Q) What is the AsteriskForge?
A) AsteriskForge is a web site that provides free development and hosting tools for Asterisk-related open source projects.

Q) Where is AsteriskForge?
A) http://forge.asterisk.org

Q) What tools does AsteriskForge include?
A) AsteriskForge is built on a platform called GForge Advanced Server.  It includes source code control (SVN), file download hosting and tracking (just counts, nothing invasive), mailing lists, forums, documentation, development team management, project management and road-mapping.

Q) Who can use AsteriskForge?
A) Any open source project with a focus on extending or integrating with Asterisk.  The project must be released under an OSI-approved license.

Q) What are examples of the kinds of things that will be hosted on AsteriskForge?
A) Dialplan and AEL snippets (we have a special section for snippets).  AGI scripts and programs in various languages.  Desktop tools including screen pop utilities, operator consoles, CRM integration components, and monitoring utilities.  Server-side integration tools including unified messaging and collaboration components, scalability and redundancy solutions, web services mash-ups, etc.  Administration GUI tools and projects.  Prompt packages.  Industry-specific (vertical) applications.  Power and predictive dialer systems.  Pretty much anything that connects with Asterisk.

Q) Can I mirror my existing Asterisk-related open source project on AsteriskForge?
A) Yes.  We are happy to host mirrors of existing Asterisk-related projects.  We do require that source downloads be included for each mirrored project, not just binary installers.

Q) Is AsteriskForge open to non-software projects?
A) Yes, though the tools tend to be fairly software oriented.  We are open to hosting documentation, voice prompts, hardware CAD/CAM files, even Asterisk-related poetry if it tickles your fancy.

Q) Does Digium get to use AsteriskForge code in commercial products outside of the terms of the selected open source license?
A) No.  The copyright and commercial rights to the code remain with the author(s).  Use of AsteriskForge is not predicated on accepting the Digium Open Source Software Project Submission Agreement.

Q) What other rules and regulations should I know about?
A) You can’t post stuff you don’t own.  We’ll honor take-down notices if they are deemed to be legitimate by our legal staff.  No development of proprietary commercial products – you must release your code as open source.  We won’t host .ISO install images or other huge binaries.  The full list of terms and conditions are available here: http://www.asterisk.org/forge/terms

Q) Is the core of Asterisk moving into AsteriskForge?
A) No.  There’s already a full set of development tools and processes in place for Asterisk, and moving to the AsteriskForge site would be disruptive to the process.  We will continue to use the existing issue tracker, mailing lists, review board, forums and subversion repositories for Asterisk.

• News
• Comments Off

Leif Madsen June 24th, 2009

(Originally posted at Leif Madsen’s blog.)

Recently I have been seeing more and more questions about how Asterisk is being released now, and what the 1.6.x versions really mean. I’ll try and clear up some of your questions and make it more clear as to how Asterisk versions work. First, lets talk about the Asterisk 1.2 and 1.4 branches.

Previous release methodology

When we just had Asterisk 1.2 and 1.4, all new development was carried out in trunk (it still is) and only bug fixes went into the 1.2 and 1.4 branches. Currently, the 1.2 branch has been marked as EOL (End of Life), and is currently only receiving security updates. Bug fixes are reserved for the 1.4 branch. This means that until 1.6 came along, all new development was done in trunk, without the ability for people to get access to new features and functionality until the 1.6 branch was created. It’s not to say the new functionality wasn’t available, but with all the changes that can happen in trunk, running a production server based on it would require a very Asterisk savvy (and C code savvy) administrator.

Before we talk about 1.6, lets make sure we understand what trunk, branches, and tags mean.

Branches, and Tags, and Trunk; oh my!

Asterisk development is basically a linear process. All changes happen one after the other, and new features are always put on the end. This is referred to as trunk. All the latest changes to core Asterisk code, new features, and major changes take place here. This is why running trunk in production is not recommended, because the developers essentially have free reign as to what goes in here.

Note: with the advent of review board, the larger commits that go into trunk are of much higher quality, as large changes are not put into trunk without any sort of peer review. The review board application allows these larger changes (such as new features, major core restructuring changes, etc.) to be reviewed by other developers to find common coding issues that would otherwise have had to be found in the wild.

Like a tree, the branches grow from the trunk. These branches are referred to as 1.2, 1.4, etc. A branch is a snapshot of the trunk at the time of creation, and from there, has its own timeline. When a bug is found, fixed, and resolved, the resulting commit is then placed into trunk, and any branches that it effects. When a new feature is created, it is only put into trunk, thereby leaving the branches in a relatively known state that is safe for production use.

From a branch, we create snapshots in time called tags. These tags are the version numbers and tar ball files you can download from the Asterisk.org website. These tags are numbered releases that never change, such as 1.4.18, 1.4.22, etc. In image 1.1, you can see a visual representation of the Asterisk branching and tagging process where the x-axis represents time.

Image 1.1 – Visual representation of Asterisk release branching and tagging.

Understanding the Asterisk 1.6 release process

With the advent of Asterisk 1.6, the release process has differed slightly, in that there are multiple 1.6 branches supported at any given time. The reason for this is because the time frame between the creation of the 1.0 & 1.2 branches, and the 1.2 & 1.4 branches was so great (approximately 1 year in the former, and 2 years in the latter!) that new features would sit in trunk, and many changes would take place over that period of time. Due to the rapid development of Asterisk, these changes were grandiose and thus the user experience between these releases were such that major changes had taken place. Backwards compatibility was difficult to maintain, and simply, who wants to wait 3 years to get the next great feature?

So with Asterisk 1.6, a new release process that allows branches to be created in a more regular fashion (approximately 6-8 months) was developed. Whenever a branch is created, we now update the 3rd most significant digit, which represents a new branch with new features, and the 4th most significant digit is now the tag number in that release. So for example, we have the branch Asterisk 1.6.0, which is a branch created from trunk at a period in time, and within that branch, bug fixes are performed, but no new features are added. The tags from this branch will be version numbers such as 1.6.0.0, 1.6.0.1, 1.6.0.2, etc. Then some period in time, a new branch will be created from trunk such as 1.6.1 (note that the 3rd significant number has increased by one). Within this branch will be new features and changes — that different of the 1.6.0 branch. Within the 1.6.1 branch, we’ll have tags just like all our other branches, such as: 1.6.1.0, 1.6.1.1, 1.6.1.2, etc.

This process continues throughout the lifetime of the 1.6 series of Asterisk. The trunk continues to have development, new features merged and potential core changes to make Asterisk more efficient (which would otherwise be deemed too invasive for a release branch) are committed. The regular release nature of this process allows less drastic changes between major versions (and note that the difference between a 1.6.0 and a 1.6.1 *is* considered a major version upgrade, similar in nature to an upgrade between 1.2 and 1.4, but not quite so drastic), and access to new features on a more regular basis. Because these are considered a major change between versions, it is in your interest to read the UPGRADE-1.6.txt files in your Asterisk source, and to test thoroughly before deploying in a production environment.

It should be noted that at any given time, there will be at least 3 branches of the 1.6 nature maintained. That means bug fixes and maintenance will be performed on your preferred branch for an extended period of time. For example, because there can be code refactoring, and core changes between 1.6.x versions (where x is the major version number), you may wish to maintain your production system on the 1.6.0 branch while your development system is running the 1.6.1 series. This way you don’t have to worry about moving your 1.6.0 system immediately to a 1.6.1 based branch as soon as it is created. Image 1.2 shows a visual representation of the 1.6 release process, which should look similar to the release process between 1.2 and 1.4. The x-axis still represents time, but in this case, time is not over quite so long a period.

Image 1.2 – Visual representation of the Asterisk 1.6 release process

So which version do I use?

It always comes down to the question of, “so which version do I use?” and this can be a tricky one to answer. It probably boils down to what features you need to utilize. By checking the CHANGES file in your Asterisk source tree, you can see what changes have happened between each of the various branches (i.e. the differences between 1.6.0 and 1.6.1). While writing this article, I was asked by someone on IRC the question, “What version do you recommend?”, and the answer I gave was, “I recommend the version that has the features you require, and that passes the testing you have done prior to rolling it out”. He followed up with, “But they all have the feature I require!”, to which I responded, “Do some forward planning, and try to determine that the version you pick has the features you may possibly require in the future so that you don’t have to re-architect your system sooner than you want”. This can save you a lot of time and effort. Do some testing, and play with all the various branches on a development server. See what dialplan applications and functions exist, and how they work. Try to determine what kinds of things you need your phone system to do, and make sure the version you pick can do it. You can also make use of the various mailing lists and IRC chat rooms (asterisk-users mailing list, and #asterisk IRC channel) to ask specific questions if you’re not able to find an answer.

Hopefully this has made the release process a bit more clear!

• News
• Comments(1)

Russell Bryant April 23rd, 2009

The accepted projects for Google Summer of Code 2009 have been announced. I have posted information about the students participating with the Asterisk project to the asterisk-dev list and russellbryant.net.

Congratulations to all accepted students!

• News
• Comments(1)