The Open Sourcerer

Book Review: Asterisk AGI Programming with Packt

[Please Note: We now have an affiliate scheme setup with Packt, so if you follow the links from this site to theirs and decide to buy *any* book from their site, we will get a small commision that we'll use towards the upkeep of our servers etc.]

As you may have read previously, I was approached by Packt Publishing to see if I would like to review their new book on AGI Programming by Nir Simionovich. Time has conspired against me to actually use it for a real project so instead I resorted to choosing it as my bedtime reading for a few days.

I’ve now read the book and first off I’d like to thank Packt for asking me. I have enjoyed it far more than I thought I would. As for how to review it, I thought I’d give my overall impression and then just go through it chapter by chapter.

It isn’t a long book – about 190 pages – which to my mind is no bad thing. It managed to concentrate on the subject and had little in the way of superfluous text and language (Apart from chapter 1 that is). The author knows his subject well and writes in a fairly informal and easy-to-read style which I personally quite liked. The meat of the book concentrates on – as you would probably guess – programming with AGI and focusses almost exclusively on using PHP. As much as I am happy around PHP, I would have liked to see some examples with alternative languages such as Python perhaps. The subtitle of the book is “Design and Develop Asterisk-based VoIP telephony platforms and services using PHP and PHPAGI” so I guess that’s why other languages don’t get much of a look in. They are mentioned here and there and there are some good links to various other libraries and open source projects. I did like the fact that he mentioned throughout the book – where necessary – the functional difference between the main versions of Asterisk (1.2, 1.4 and 1.6) and ways to deal with those differences when you come up against them.

OK – so here is my chapter-by-chapter review:

Sorry, but I didn’t really get the first chapter at all. This is a book aimed at programmers and developers and yet the first chapter was a repetitive cut-and-paste of how to build the various asterisk components from source, including lots of screen shots showing the output of things like ./configure which personally I found a bit trivial and uninteresting. I was a bit concerned that the rest of the book was going to follow suit but thankfully I was mistaken.

Chapter 2 is really good. It explains the workings of Asterisk’s dialplan and applications – the infamous extension.conf – in a very clear and understandable way. I recall when I first started to look at Asterisk and was delving into as much information on-line as I could get, the Asterisk TFOT book [pdf download] and whatever else I could find, that it was several days before the penny finally dropped. It isn’t difficult really but it isn’t quite the same as “normal” programming or scripting concepts and the language itself is far from obvious (e.g. an extension is not the phone on your desk in Asterisk’s configuration files). But then this is telephony we are talking about. Using the example of a basic IVR or AA the author examines the diaplan syntax and construction.

So I got quite a lot out of chapter 2. I thought it was well written and clear and useful. Chapter 3 develops the IVR theme further, introduces other features of the Asterisk application pool and covers the scripting language in more detail examining branching, expressions, operators and flow control. It’s a fairly short chapter but covers a good deal of ground if you are unfamiliar with Asterisk programming.

Your level of knowledge and familiarity with Asterisk will dictate what you get from this first section (Chapters 1-3). Although I have used Asterisk for a couple of years now and have felt quite comfortable with the platform’s configuration and use, I got quite a lot of new information and ideas from this early part of the book. For me, this initial part has been very useful and will be a good reference for the future. I think, though, if you are very familiar with Asterisk then you might find it a bit slow going. The book hasn’t examined AGI whatsoever up to this point and we are about a third of the way through already! The author does suggest that a coffee is a good before starting on Chapter 4 as “the journey becomes more and more complicated”…

Chapter 4 introduces the reader to AGI in a fairly gentle way and also offers 10 “rules” to help make your AGI programming more successful – they all make sense to me and will I’m sure prove to be a very useful monitoring/checking tool. Packt sent me an extract from this chapter which you are free to read here if you want to get a flavour of it.

The following chapter introduces us to some real code (PHP) and we build our first, simple AGI application. Nothing to hard, but a useful introduction into how to actually get the conversation happening between Asterisk and your script. There are couple of nice flow charts which are helpful for visualising the traffic flow back and forth between Asterisk and your script too. Again not too complicated but helpful in getting the novice AGI programmer, i.e. me, thinking about things the right way.

In chapter 6 Nir examines and recommends the use of a set of PHP classes (library) called PHPAGI. Being completely new to AGI programming I am in no position to contradict the author’s recommendation, but looking on the Sourceforge site for this library, it is quite old and has not been updated for 3 years or more. Of course that may be because it is perfect and needs no further development, or there might be other reasons but I would have really liked to have had some more discussion regarding this choice of library before continuing – just for my own piece of mind more than anything else. Perhaps if Nir reads this he could leave a comment about this? My own assumptions after reading the rest of the chapter are that the AGI interface itself is fairly simple and so – perhaps – the need for a more dynamic or complex library is just not there and this one does the job just fine. Anyway, the rest of the chapter we look at a new AGI application using the class library above and also we discover the main – and what seems to me to be a first-class – concept for building AGI applications: Atomic AGI or Particle Programming. Sounds great doesn’t it? It really does make a lot of sense. Basically it’s a bit like the traditional ideals behind Unix/Linux command line applications; write small applications that do one thing and do it well. To summarise then, in chapter 6 we are shown the author’s recommended path to AGI Nirvana through adopting some rules, and practices. It is hard for me to draw any solid conclusions from his approach as I am a novice with AGI and so have nothing by way of comparison, but it certainly seems to make a great deal of sense and is clear and well explained. Good stuff.

The remainder of the book goes a bit wider than just pure AGI. Chapters 7 and 8, examine some of the closely related applications and facilities of Asterisk. We get an overview of FastAGI (AGI over TCP) and Nir shows us some further PHP libraries that are available to assist with producing FastAGI applications. Chapter 8 offers an overview of the AMI (Asterisk Manager Interface) and some example code to get you started.

In the penultimate chapter, the reader is given a challenge: to create an application that is used in the real world – an Asterisk Call Recording Gateway. There is no code in this chapter – that’s for you to do. But Nir provides some useful guidance about the way to think about the development and plan the project itself.

And finally, chapter 10 discusses how to make sure your Asterisk applications can scale and offers several ideas and techniques to improve performance such as database query caching and using web services.

As I said at the beginning – I enjoyed reading this book much more than I thought I would. It is not overly technical and Nir has an engaging style of writing. The book is a great introduction into Asterisk programming. It is not “The Bible of Asterisk Programming” and does not set out to be. It is clearly aimed at developers who have not had much to do with Asterisk before but are familiar with traditional programming methods. I really liked the fact that it is quite short. You can read the whole book in a couple of evenings and being laid out the way it is it will become a very useful reference document for me in the future.

Nir Simionovich has a blog.

Asterisk: AGI Programming

Recently I received a flattering email from Packt Publishing telling me about a new book entitled: “Asterisk Gateway Interface 1.4 and 1.6 Programming” – by Nir Simionovich. The author of the email refered to this blog and asked if I would like to review their book for my blog. Very flattering indeed.

[Update: Book review is done now. You can read it here]

I was, of course, already familiar with Packt. They do produce lots of books on Open Source technologies. What I wasn’t aware of, until I looked, was this (from their “about us” page):

Packt Open Source Project Royalty Scheme

Packt believes in Open Source. When we sell a book written on an Open Source project, we pay a royalty directly to that project. As a result of purchasing one of our Open Source books, Packt will have given some of the money received to the Open Source project.

In the long term, we see ourselves and yourselves, as customers and readers of our books, as part of the Open Source ecosystem, providing sustainable revenue for the projects we publish on. Our aim at Packt is to establish publishing royalties as an essential part of the service and support business model that sustains Open Source.

Now that seems like a fine scheme to me and is one that would certainly weight my purchasing decision toward a Packt book over another similar publication. I am aware that they will obviously get some free publicity from our review but having read the above I am quite happy about that.

Anyway, back to AGI.

Asterisk Gateway Interface (AGI)

The Asterisk Gateway Interface is an interface for adding functionality to Asterisk with many different programming languages. Perl, PHP, C, Pascal, Bourne Shell – it’s your choice, really.

• AGI may control the dial plan, called in extensions.conf.
• Async AGI Introduced in Asterisk 1.6, allows asynchronous AGI scripting.
• EAGI gives the application the possibility to access and control the sound channel in addition to interaction with the dial plan.
• FastAGI can be used to do the processing on a remote machine via a network connection.
• DeadAGI gives access to a dead channel, after hangup. Deprecated since Asterisk 1.6

Being a fan and user of Asterisk in our business, I think what we will do is actually use the book and create or modify an application that will be of use to us. This should allow us to give a decent overview of the content and how easy/hard it was to transfer the knowledge into a real world result.

We use vtiger as our CRM, OpenEMM for email marketing and Asterisk for our telephony – all running on Ubuntu Servers. Tying these together would be a fine goal.

But I am open to other suggestions too. Anyone got any good ideas for a fairly simple AGI application?

Asterisk, Zaptel, Oslec and Ubuntu Server [Updated]

I have recently migrated my server at home from a custom Linux build to Ubuntu Server (8.10 Intrepid). The main migration went very smoothly and I learned a few new tricks on the way too.

One function the server performs is as my telephone system for work and home. It runs Asterisk. I have a couple of IAX2 trunks from our VOIP provider for mine and my wife’s businesses and I also have a cheap x100p clone analogue card for PSTN backup purposes. On my old system software, I had compiled the device drivers (zaptel) and kernel modules for the card manually and used a, frankly fantastic, echo canceller called Oslec (the Open Source Line Echo Canceller). You can read the couple of posts I made about when I first tried it out here.

On my new server OS, I installed the Asterisk server via Ubuntu’s package management system sudo apt-get install asterisk. After some digging around on the ‘net (and it wasn’t obvious) I discovered that the zaptel drivers (for the PSTN hardware) need to be installed slightly differently:

sudo m-a -t build zaptel Which retrieves the zaptel package and builds it for your running kernel. You might have to run m-a prepare in advance of this to retrieve your Linux kernel headers.

The m-a (Module Assistant) command will compile and create a .deb package in the /usr/src directory. On my system the package was called zaptel-modules-2.6.27-11-server_1.4.11~dfsg-2+2.6.27-11.27_i386.deb.

It can then be installed using dpkg: sudo dpkg -i zaptel-modules-2.6.27-11-server_1.4.11~dfsg-2+2.6.27-11.27_i386.deb.

This went fine and I had read on launchpad that as of an earlier version of the zaptel package the Oslec echo canceller was now the default. Unfortunately this didn’t quite work as I expected. The zaptel module was in fact using the standard MG2 EC which is not very good with my x100p card at all.

After a bit more digging around in the source code, there is a file in the zaptel package called zconfig.h which is where the chosen EC is defined. It is specified as MG2 in the package. What I did to fix it was as follows.

• Unpack the zaptel.tar.bz2 package that was in /usr/src.
• Edit the kernel/zconfig.h file so the line #define ECHO_CAN_MG2 is commented out
• and added in a line that reads #define ECHO_CAN_OSLEC instead.
• Re-assemble the zaptel package: sudo tar jcvf zaptel.tar.bz2 modules (”modules” is the directory name where the zaptel package extracts to).
• Delete the existing zaptel-blah-blah.deb file and the modules directory too.
• Re-run the m-a -t build zaptel command.

Thanks to Tzafir Cohen on the asterisk mailing list for this. There is a far simpler method to use for the time being although this is a known bug and is now fixed in the development tree so I guess it will be unnecessary once the package has been updated. Do please check first if you are following this in the months to come. Anyway, instead of the commands above, these commands work for me and are far simpler:

sudo m-a -f get zaptel-source This simply gets the source package and saves it in /usr/src.
sudo ECHO_CAN_NAME=OSLEC m-a -t a-i zaptel And this builds and installs the modules and tells the build scripts to choose the Oslec EC by default. The -t switch puts the command into text mode so you actually see what is going on. I find the process rather opaque and uninformative without this switch.

After rebuilding, the zaptel module now requires, and loads the Oslec EC by default. The command modinfo zaptel is a good test. The output of it should be something like this:

filename:       /lib/modules/2.6.27-11-server/misc/zaptel.ko
version:        1.4.11
license:        GPL
description:    Zapata Telephony Interface
author:         Mark Spencer <markster@digium.com>
srcversion:     4433ADDE0493C798A455677
depends:        oslec,crc-ccitt
vermagic:       2.6.27-11-server SMP mod_unload modversions 686
parm:           debug:int
parm:           deftaps:int

Note the “depends” line.

You could also type lsmod | grep 'zaptel' once you have reloaded your server:

zaptel                199844  5 wcfxo
oslec                  16668  1 zaptel
crc_ccitt              10112  1 zaptel

This command shows the oslec ec module installed along with the the zaptel and wcfxo drivers.

One final point to note. If you just want to load a particular telephony hardware driver and not all of them, I think you need a file /etc/default/zaptel like this with the relevant driver(s) uncommented:

TELEPHONY=yes
DEBUG=yes

# Un-comment as per your requirements; modules to load/unload
#Module Name                      Hardware

#MODULES="$MODULES tor2"                # T400P - Quad Span T1 Card
                                # E400P - Quad Span E1 Card

#MODULES="$MODULES wct4xxp"     # TE405P - Quad Span T1/E1 Card (5v version)
                                # TE410P - Quad Span T1/E1 Card (3.3v version)
#wct4xxp_ARGS="t1e1override=15" # Additional parameters for TE4xxP driver

#MODULES="$MODULES wct1xxp"     # T100P - Single Span T1 Card
                                # E100P - Single Span E1 Card

#MODULES="$MODULES wcte11xp"    # TE110P - Single Span T1/E1 Card

#MODULES="$MODULES wctdm24xxp"  # TDM2400P - Modular FXS/FXO interface (1-24 ports)

MODULES="$MODULES wcfxo"        # X100P - Single port FXO interface
                                # X101P - Single port FXO interface 

#MODULES="$MODULES wctdm"       # TDM400P - Modular FXS/FXO interface (1-4 ports)

I can’t recall the exact origins of this file and whether or not it is really necessary, but I had it on my old system and the Ubuntu provided zaptel init script checks for it’s presence; although it doesn’t look like it does much with its contents though…

Hopefully this will help others and also act as a bit of an aide memoir for me when I next build an Ubuntu server with Asterisk.

S685IP and the VCard format

Last night my wife harangued me about not having her phone directory installed on our new handsets so she can see who is calling.

She sent me a spreadsheet with all the contacts and their respective numbers. The manual for the S685IP is a little unintuitive regarding the formatting and options for the vcard format so I started doing some digging about.

Firstly, I created a dummy entry in my handset using all the fields I could find. Then I turned on Bluetooth and transferred it to my laptop so I could see what it looked like. The Bluetooth link worked great. I could have got the vcard from the handset via the web interface, but I just wanted to test the Bluetooth functionality. Here it is:

BEGIN:VCARD
VERSION:2.1
N:Lord;Alan
TEL;HOME:XXXX79XXXX
TEL;WORK:XXXX27XXXX
TEL;CELL:XXXX457XXXX
EMAIL:test@testingcentre.com
BDAY:1900-11-01T00:00
END:VCARD

I didn’t find a perfect csv to vcard converter for this structure, although this online one worked pretty well and only needed a bit of local massaging to import correctly. I also tried a rather old but still useful ruby project from sourceforge here, that also worked but left out the vital VERSION:X.X line altogether.

Anyway, as you should be able to see from above, the name field N: takes two parameters separated by a semicolon and does not use the discreet FN: and LN: format. Note that it is last name first.

The rest should be obvious. If you don’t have data for a field, leave the entire field out. I didn’t try sending a blank field to the handset, but leaving the fields out entirely worked just fine.

The only other “gotcha” is that the file containing your vcard data needs to be DOS formatted and not Unix formatted: CR+LF vs LF. If you are on Windows then you won’t have a problem but Linux users will need to use the tofrodos package (Ubuntu users just do sudo apt-get install tofrodos) and run the file through the unix2dos command before sending it to the handset. If you don’t, the transfer fails.

Once you have the format right, using the Web interface on the base station to upload the directory seemed to work absolutely fine. I was able to send a directory containing 70 entries to each handset. It isn’t blisteringly fast (I guess it took about 2 minutes/handset), but it’s a whole lot quicker than typing the entries in by hand!

Siemens Gigaset 685IP Phones and Asterisk Review

[Update and Introduction: This article was originally posted on the 27th April 2008. I am still using the phones and am very happy with them indeed. I have had no operational problems to speak of. I make this updated comment because this is still a very popular article and new visitors read this daily but the date is only shown at the bottom.]

Last week I purchased a triple set of the brand new Siemens S685IP telephones. This is a DECT home telephone system with support for both PSTN and VOIP services. I’d spent quite some time looking for a decent replacement for our aging and now unreliable existing DECT handsets.

I bought these from a UK based on-line telephone vendor DSTelecom and their service and price was very good. I’d been waiting for this model to become available for a couple of months and they were offering the best price and the were first to get them in stock too!

There were a few reasons I selected these Siemens phones, but the main one is their ability to act as a basic telephone switch between handsets and incoming services. The Base unit can support up to 8 incoming services: 1 x PSTN, 1 x Gigaset VOIP Network, and 6 further VOIP (SIP) services.

This is the latest release and the handsets support very high quality voice between each other and on compatible networks. They also support Bluetooth so you can use a wireless headset and upload/download your mobile phone’s address book etc.

Here’s my personal review of these new phones for use in our home network. (Just click on the thumbnails for bigger images)

My initial impressions: Nice packaging and a good looking handset.

There are very easy to follow 1st time instructions that get the system installed and running. Once I’d plugged the phones in and got them charging, and base station in to the LAN, the phone started showing me the weather forecast in Lisbon!

Once the physical install is done, you can do almost everything else from the web based interface of the base station. I’ll cover that in detail in a minute.

But first the handsets. When you get them you need to do a first time charge and discharge cycle. The batteries are supplied (a pair of AAA rechargeables). The first full charge took between 3 and 5 hours depending on the handset. To discharge them all, I made internal calls between the phones and put them all on handsfree. It took a good 8-10 hours for them to get fully discharged. So talk-time is excellent.

The first thing my wife commented on when we were talking internally was the voice quality. She said it was brilliant! And having now just had a conversation with her somewhat hard-of-hearing father, he also attested to the much better sound than our previous telephones. So that’s good!

They have a nice big colour screen where you can – apparently although I haven’t done this yet – add pictures to your directory so the phone shows the caller’s face or avatar when ringing.

Anyway, all-in-all my initial impressions of the system was very positive indeed. Now let’s look at how the whole DECT/VOIP thing is configured and what really makes these stand out for a home phone…

Accessing the Web based user interface was easy. Here’s the login screen you first see:

Due to the way I have our home network set-up, I used a static IP address for the base unit. It also supports DHCP however.

Next is to configure some VOIP service providers. For me it’s my Asterisk server… Only one change seemed to be needed to get the registration to work: Add subscribemwi=yes to your sip.conf. I’m not actually sure if this made any difference, as I think I didn’t click the “Active” box first time round. But apparently this setting is needed to get the message waiting light working properly when you are using Asterisk’s voice-mail anyway.

You configure the details for each service by clicking the “Edit” button. Here’s a default screen showing most of the options.

The next section allows you to select the codecs you’d like each service to support and their priority. As you can see again, it’s a simple, clean and easy-to-understand interface.

After that, you are ready to choose which handsets work with what services. I configured my handsets first to give them meaningful names: “Alan’s, Helen’s and Kitchen”. Note also you can upload/download your handset’s directory from here too. The directory needs to be in vcard format. I haven’t done this yet but I can’t see any major obstacles apart from the time it will take to get a csv spreadsheet into vcard.

This is, for me at least, the coolest feature. From this screen, you can choose which handset rings depending on the service it is coming in on. And you can decide which calls use the built-in answer machine and which do not. It’s basic switch functionality and when you stick Asterisk behind this you have a really flexible solution to handling multiple incoming lines and different types of users.

My wife and I both run our respective businesses from our home offices. Now we have individual incoming VOIP lines into Asterisk plus the home PSTN service. Calls for my business ring, my phone and the kitchen phone. Calls for Helen’s ring her phone and the kitchen phone. Neither uses the Siemens Answer machine but the features of Asterisk’s voice-mail system. Calls for the family ring all three phones and use the built-in answering machine. You can configure this any way you wish basically.

Now we have a basic phone system set-up there are various add-on features to play with

Here is the call forwarding screen where for each service you can decide what to do under certain conditions: “When Busy, No Reply or Always”. Simple but this is a home telephone system.

Next, you have a section for creating dialling plans. You have here, the ability to choose which service gets used for particular number sequences. So, for example, you could put in the international prefix for Australia and only allow numbers with that prefix to go via a particular VOIP service. You can also block certain number sequences completely too.

The Network Mailbox screen allows you to configure the voice-mail service for each provider. So for Asterisk that will be the numbers configured in voicemail.conf. This enables the service to work with the handsets so you get message waiting indications and access to the mailbox without needing to know the mailbox number.

Also in the Telephony section of the configuration tree is an “Advanced Settings” screen. This lets you configure the way DTMF tones are handled, SIP and RTP port numbering and a few other odds and sods.

That’s it for the telephony section. Comprehensive, easy-to-use and nicely laid out.

The next area is “Messaging”. The handsets support SMS type texting and there is an option to configure a jabber server (IM). The Siemens Gigaset VOIP network, which you get automatically subscribed too when you buy your phone is the default configuration, but you can change it to your favourite IM network should you wish. Not being a big user of either SMS or IM I haven’t used this. Maybe one of my kids will show me later!

Also under the Messaging section you can configure a POP3 email server. It is for the network, not per handset so I am not sure of it’s value. I suppose for a family who have a single email account it might be useful. But this feature doesn’t really do it for me. If it was per handset or per network service it would make far more sense.

After Messaging come the last few configuration pages.

The first is called “Info Services” and you can, via the Gigaset Network, configure a few somewhat limited network based information feeds. You can enter an RSS feed, or choose a weather forecasting service. The weather seemed more useful for me so I set it to show me the weather for London for the next three days.

The final screen is the ubiquitous “Miscellaneous” settings. Here you can update the firmware directly from Siemens or use a locally stored file. You also get to choose the NTP server for the clock and whether to automatically deal with daylight saving time changes.

That’s the Siemens S685IP phone system. Having had them running for a few days now, I’m very pleased. Everything has worked, call quality is excellent, ease-of-use is superb.

I have come up with a couple of tricks I’d like Siemens to do that would really enhance the overall functionality however. Two are to do with time. And as there is an accurate and network-synchronised clock in the base station, I can’t see this being terribly hard to do to be honest.

1. Use the in-built clock to allow you to configure different network connection settings. So, for example, after 6pm, If a call comes in my office number I might not want it to ring the other (our kitchen or family) phone. Perhaps during a weekend also.
2. Ditto with call forwarding. After 6pm or during weekends forward calls to my mobile for example…
3. Ditto with the dial plans. Being able to route certain type of call via different networks is great, but being able to choose to do it during certain hours would be even better…
4. The email and IM features are nice but in my opinion they would be far more useful if it was configurable based on per handset or service basis rather than a single account for all devices.
5. Also to do with the email feature; I’d like the option of IMAP as well as POP3.

I’m sure there will be some other ideas that will crop up have as we get used to them, but all-in-all my first impressions are that this is an absolutely cracking phone system for home and small business. When you use this with Asterisk of course, they get even better. Some of the time based features could possibly be got around by some fiddling with Asterisk. If I get chance to work something out I’ll write it up here.

Open Source Cars and More

I love this article on zdnet from David Greenfield. It’s a round-up of what’s happening in the up and coming area of Open Source Hardware. According to David,

A burgeoning trend in open source hardware is putting up some devices on the Web — from machines that make anything (including themselves) to cars — with the specs to make them yourself (See our list below). While still in its infancy, the trend could redefine hardware cost models much as its done for software.

And there are some neat really ideas like this one which I have been following myself for a while:

Now that you’ve got Asterisk, what hardware platform will you run the software on? Usually folk settle on a Intel or AMD based-server of one kind or another. You can build your own PBX hardware with the Astfin Project or buy one for just $450 from the Free Telephony Project store.

This Asterisk appliance project has the chap who wrote the brilliant Open Source Echo Canceller I mentioned before in it.

But how about your own, Open Source Car…

Open Source isn’t just for your office. The OScar aims to be the first open source automobile. The goal is to create a utilitarian car that aims to move people from place-to-place sans a lot of the high-tech gadgetry that runs in today cards. Initial concepts call for a four-door, four meter length vehicle weighing about 1000 Kilo capable of reaching 145 KM/hour.

Cool – just the thing to keep a man happy and content in his shed for months.

Untangle, Asterisk PBX and File Server; All-in-One. Part 8

If you’ve been following the story so far you’ll now where I am. If you haven’t, please go back to Part 1 and read from there. Alternatively if you click on the Untangle tag in the tag cloud then you should get all of the posts so far.

Hi all,

I’ve not yet got any further with the Untangle portion, but pretty much everything else is now in place and working

Last night I built and installed the few remaining applications that are necessary to support my objectives:

• MySQL (I need this for Joomla! and vtiger)
• Postgresql (I need this for untangle)
• Apache
• PHP (and some associated libraries for added functionality, i.e. HTML-Tidy, mm, libmcrypt, mhash…)

I have also been thinking about what it is actually I am trying to achieve. I find a picture really helps so here’s a block diagram of the applications I want and how they should interface to the outside world…

This was a good exercise that helped me to understand the flow of traffic and what needs to be prevented from passing through the server. The dotted line from Apache to the Internet is because I’m not sure yet whether I’ll actually provide any sort of public web presence from this box or not. I doubt it somehow but you never know…

If anyone has any comments or suggestions for improvements I’d be happy to hear them. I made the original diagram in OOo draw. Here’s the original file if you want to use it or alter it. As with all other stuff on here, its CC licensed.

Playing with Asterisk and VOIP

I haven’t written much recently – the day job keeps getting in the way

Anyway – some of my spare time has been used getting Asterisk working just the way we want. The more I play with it, the more I really like it.

We now have multiple voice mail accounts set-up for our various business interests, linked to an Automated Attendant or IVR (which is currently using my voice, but we are hoping to get one of our wives to record the greetings as girls are better than boys!). Once the caller has pressed the appropriate digit on their phone, Asterisk simultaneously calls multiple extensions (some on another Asterisk server connected across the ‘net), and if no-one answers within a pre-defined time, then the call is routed to the relevant voice mail account. The audio message left by the caller is then attached and emailed to one or more recipients depending on how we configure Asterisk and Exim. When the call is initially presented to our SIP phones, not only do we see the caller ID (which we could use to trigger application events in a CRM like vtiger for example), but we also see which business (or choice) the caller selected, thus we are able to appropriately answer the call.

This is a pretty advanced feature set for any PBX. But when you realise it’s running on a box that cost under £200, which also provides file server, content filtering/firewall, and local web based, application services too, and the software costs have been zero, it’s really quite amazing. Obviously for more “serious” telephony you’d want to run Asterisk on hardware that is designed to be ultra-reliable. But still the software is free. However for a home or home-office set-up I would suggest that this is an excellent platform.

The other thing I have just set-up is a free (for me) local rate DID (Direct Inward Dialling) number that connects, via the net, to my asterisk server using the IAX2 protocol. The provider of the service is the not-for-profit organisation http://www.voipuser.org. This is quite an amazing service:

VoIP User is a non-profit community, formed with the intention of creating a base for early adopters, consumers and professionals alike to exchange ideas, discuss new developments and generally experiment with VoIP technology.We have a natural bias toward open standards. In the VoIP space, this is primarily SIP and IAX (Asterisk).

…

When we started building the foundations of what became VoIP User at the end of 2002 there was nothing available to the developer and early adopter that enabled experimentation with PSTN gateways. Of course with the majority of telephone users on the PSTN, only being able to call another VoIP User in a test environment was too restrictive – some form of free access gateway was required.

In order to fill this need we setup a service system provisioning a PSTN/VoIP gateway for public access at no charge. In order to make this possible we created a unique financial model whereby calls outbound from the gateway are financed by calls inbound to the gateway. Every member signing up for access obtains an inbound PSTN number, on which revenue is generated and VoIP User receives a share. It’s that share of revenue that funds the outbound calls.

All inbound funds are credited to a central “pot” which is made available to those members who are making reasonable use of the facility. Excessive use is dealt with by member and number restrictions. We aim to run this in as fair and reasonable a manner as we possibly can but the over-riding principle is it is offered for experimentational use. We are not a VoIP provider. If you require long term telephony services, you will need to look elsewhere. If you’re looking to test a new piece of SIP or Asterisk equipment, you’re in the right place.

As they state, this service is for experimentation and testing but it works, is very easy to setup and has enabled me to test many of the features of Asterisk. If you are going to start experimenting, pay them a visit. And remember incoming calls are charged at the UK local call rate, and as long as you don’t abuse the service, outbound PSTN calls are free.

Back to the grindstone then…

Untangle, Asterisk PBX and File Server; All-in-One. Part 7.1 (OSLEC)

O.K. I said I’d write a bit about an excellent new echo canceller which happens to work with Asterisk. Here it is it’s called OSLEC the Open Source Line Echo Canceller and it’s written by a chap called David Rowe.

As readers may recall, I’ve built a small home server (VIA CN700) on which I plan to run Asterisk, Samba and Untangle. Samba is up and running and Asterisk is too. I have a single port, very cheap (about £15 inc postage from the USA) x100p card providing an interface to a normal analogue PSTN telephone line.

When we got everything working, we noticed a great deal of echo on voice calls over the x100p. Lots of playing with gains and various settings in the zaptel configuration failed to make any noticeable difference.

I came across this site whilst looking for something completely different and started to read… It sounded like just the thing. A bit of jiggery and a quick patch to the zaptel-1.4.5.1 sources – thanks to the asterisk mailing list – and I got the OSLEC canceller working.

Basically here’s what to do:

1. Build the OSLEC module (it will need to find your kernel sources – just like zaptel) according to the instructions on the website. Once built and you’ve checked that you can install it by inserting the module into your running kernel, copy it (oslec.ko) to your kernel’s loadable module directory: on my system the zaptel modules reside in /lib/modules/2.6.23/misc/ so that’s where I put the oslec module too.
2. Patch your zaptel source tree (if you have version 1.4.5.1 you will need to patch Makefile.kernel26 or OSLEC will never get loaded) and rebuild and re-install as described. (Caution: Backup your /etc/zaptel.conf, /etc/asterisk/zapata.conf and your modified SysV init scripts so you can simply overwrite the default files installed when you rerun make install on the zaptel sources.)
3. Edit your zapata.conf so the following are as below:
   echocancel=yes
echocancelwhenbridged=no
;echotraining=400
4. Reload everything (if in doubt, stop asterisk and zaptel using your SysV init scripts, e.g # /etc/rc.d/init.d/{asterisk,zaptel} stop. Then start them again. When zaptel starts you should see a message saying Echo canceller OSLEC or something like that; if it says MG2 then it isn’t working so you need to go back and recheck your build and patching and module loading.

That’s it.

Now make or receive a call through your cheap x100p card and marvel at the clear echo free sound! It worked brilliantly for me. Of course YMMV but it is definitely worth a try. Most of the reports on the ‘net are incredibly positive about this.

Untangle, Asterisk PBX and File Server; All-in-One. Part 7

There’s no Untangle in this installment – I’m awaiting a new kernel from the developers before I can get any further; it should be here shortly however.

In the previous article of this series I mentioned that I’d explain how to get Asterisk built and running as a non-root user. It wasn’t too hard to be honest but I’ll document it anyway.

The problem: Asterisk by default, when compiled from source, expects to be run as root. For userspace applications, this is NEVER a good idea in my opinion. After all we’re running on a multi user system that can support non-root processes – unlike Windows – so we really should…

The solution for Asterisk-1.4.13 on my LFS based system, is as follows:

Build any hardware add-ons and codecs you need before building Asterisk. I built the Zaptel module for my x100p card and the Speex Open Source VBR codec. The zaptel module needs to find your kernel source tree (usually in /usr/src/linux-2.6.x.x). To build the zaptel source:

./configure --prefix=/usr
make menuselect (to select/deselect the modules you wish to build)
make

then as root:

make install

and optionally:

make config This will install the SysV init scripts and some default configuration files. You may need to modify the init scripts depending on your system.

Simply follow a similar process for the Speex codec…

For Asterisk, start off by creating a group and user that will run and own the asterisk process and files (select {G,U}IDs and names that are appropriate for your system).

groupadd -g 75 asterisk
useradd -c "Asterisk PBX" -d /var/lib/asterisk -g asterisk -s /bin/false -u 75 asterisk

Edit the Makefile in the top of the asterisk source tree so that the line:

ASTVARRUNDIR=${localstatedir}/run becomes ASTVARRUNDIR=${localstatedir}/run/asterisk

Then build as normal

./configure --prefix=/usr
make menuselect (Turn on/off various modules and options. Select sound files/language/format and extra sounds. Type “s” to save and exit)
make

Then as root:

make install

Asterisk is now installed. But because we will run the process as non-root it needs write permissions for these directories and their contents:

/var/lib/asterisk, /var/log/asterisk, /var/run/asterisk, /var/spool/asterisk, /dev/zap/*.

If you installed the zaptel modules and used the ‘make config’ command, a udev rules file (zaptel.rules) will be written to /etc/udev/rules.d. This enables, by default, udev to create the zaptel device files as user:group asterisk. If you chose another name above you will need to edit this file accordingly.

O.K., lets sort out the ownership and access to the files Asterisk needs. First change the owner:

chown -R asterisk:asterisk /var/{lib/asterisk,log/asterisk,run/asterisk,spool/asterisk}

Now set read/write only by owner, read only by group and no access by other:

chmod 750 /var/{lib/asterisk,log/asterisk,run/asterisk,spool/asterisk}
chmod -R o= /var/{lib/asterisk,log/asterisk,run/asterisk,spool/asterisk}

This switch (chmod -R o=) is pretty cool by the way. It removes all access to all files and directories for the “other” classification, effectively setting them to “0″ but does not change or overwrite any of the permissions for owner and/or group access.

The asterisk process itself only needs read permission for the configuration directory (/etc/asterisk) and its contents (almost¹):

chown -R root:asterisk /etc/asterisk
chmod 750 /etc/asterisk
chmod 640 /etc/asterisk/*

Some of Asterisk’s ‘.conf’ files contain cleartext passwords and other potentially sensitive information. Setting the files as above permits read/write only by the user root and read only by members of the group asterisk.

That’s it basically. When you start asterisk from the SysV init scripts, pass the following arguments to have it run, safely, as your new user:

asterisk ${DEBUG} ${ZAP_TIMING} -U ${USER} -G ${GROUP}

In my startup script, I’ve set those constants above to be:
# If you want debug messages to the console and the logs switch the
# comments below
DEBUG=""
#DEBUG="-d"
# The user and group we created earlier
GROUP="asterisk"
USER="asterisk"
# Use this if you want to limit the maximum number of simultaneous calls
# to prevent system failure for example
MAXCALLS=""
#MAXCALLS="-m 20"
# If you have a Zaptel card/timing source, enable it here
#ZAP_TIMING=""
ZAP_TIMING="-I"

That’s it.

When asterisk starts, it will run as the user and group defined above. In the next installment, I will write a bit about a fantastic new echo canceler which sorts out cheap x100p cards and makes them work properly… It’s really excellent.

1. Except for voicemail.conf. When a user changes their pin asterisk writes these changes to the voicemail.conf file.