http://www.ranetcommons.

net

Methods for Machine Based SMS

Weaver Technical and Project Notes from RANET

Author: K. Sponberg, IEPAS Program Manager, UCAR JOSS, sponberg ‘at’ joss.ucar.edu
Date: June 6, 2009

In this contribution to Weaver, I provide a basic overview of the different methods and
tools available to integrate mobile text messaging (SMS) with automated systems. Or more
specifically, I will discuss the options available to allow SMSs to be sent from and received by
existing server systems or other ‘machine’ processes. By ‘machine’ I mean a server (computer)
that automatically sends messages to a human recipient, a server that automatically processes
SMSs received from another mobile device, or even applications where two machines exchange
information via SMS.

In this article I have not provided example scripts or a detailed ‘how to’, but rather the intent
here is to provide you with a basic project manager’s understanding of the tools available for
integrating SMS with existing ‘machine’ applications. And more importantly the intent is to
provide some general guidance on cost implications, operational integrity, etc. Just as RANET
has provided a tutorial series on FrontlineSMS, a potential tool for integrating some operations
with the mobile network, in time we will provide tutorials on the other methods as well. In the
meantime this article will help you understand the basics of ‘what it takes’ to integrate SMS
with your existing systems.

What are some potential applications?

Before outlining the basics of machine based SMS, let’s look at a few potential applications.

One of the more common uses is for automated dissemination of messages to pre-defined subscribers with
mobile devices. The specific application would likely be for alert and notification, such as supported through
the RANET Alert Watcher (RAW). In example the RAW passes messages from the Pacific Tsunami Warning
Center via SMS to key government and regional points of contact. These contacts are officially involved in
warning or emergency response. (The benefits and risks associated with using SMS for public or even closed
user group alert is discussed in a prior Weaver entry, “Mobile Text Messaging in Information Dissemination and
Public Alert”.) A similar alert application might be to send key technicians a message, if infrastructure is down
or otherwise malfunctioning. However, less urgent information services, such as the delivery of a daily forecast
or local market prices, could be as easily implemented.

To achieve this functionality, you need to setup three basic processes. First, information products need to
be logged to a database or other digital format such as an RSS feed, text file, etc. Doing so may require
altering internal operations to change how forecasts, observations, warning messages, and similar are created.
Secondly, another script or process then needs to monitor the database or repository of information for new
information (updates), and finally a third process needs to be able to communicate with the mobile network via
a SMSC (SMS Center), third-party SMS gateway, or through a local modem in order to send out the messages
based upon a subscription list. Clearly there are some intermediary steps related to formatting of messages,
determining which if all subscribers receive a message, logging of sent SMSs, etc.

As another use scenario, machine based SMS could support on-demand requests for information. For instance,
rather than setting up a list of subscribers (as above) where every new update is sent out to each person

This document was prepared by the IEPAS and RANET program lead under award number – NA06OAR4310119 from the National Oceanic and Atmospheric Administration, U.S. Department of Commerce.
The statements, findings, conclusions, and recommendations are those of the author(s) and do not necessarily reflect the views of the National Oceanic and Atmospheric Administration or the Department
of Commerce. Base support for RANET is provided by the USAID Office of US Foreign Disaster Assistance (OFDA) and the NOAA NWS. It is administered by the UCAR Joint Office for Scientific Support.
Significant funding and in-kind support is provided by a number of national weather services, donor agencies, and the communities with which RANET works.
automatically, you might want to allow a user, such as a farmer, to retrieve this information when it is needed
most. In this scenario a farmer sends an SMS to a mobile number associated with your server. The message
sent by the farmer is ‘coded’ to request a particular product by name, perhaps catalog number, or other similar
reference that is easily remembered and equally easy to type into a mobile device. Based upon this request,
your server infrastructure then retrieves and then sends this information in an SMS directly to this user.

For setup the basic components described above are required, but in addition you will need a way to route SMS
messages to your server, a script or process to read the received message, and then some sort of queue or script
that retrieves the information and sends it back to the requester via SMS. The benefit of this sort of application
is that you can better assess demand, your information becomes more widely available beyond print, e-mail, and
web pages. There are also some potential cost savings by not sending information (as through a subscription
model) when it is likely not needed. Of course there is less control and likely awareness of who specifically is
retrieving the information, unless you setup a registration process, which in the end adds complexity and likely
turns away potential consumers of your information.

Finally, one of the most obvious applications within the hydro-meteorological community is the collection
of field observations. Whether the source of information is a human observer or an automated station, SMS
can be a viable means of retrieving data in a timely manner. The basic setup for the above applications would
support data collection, but you will likely need to factor in some sort of filter for quality control, and perhaps
you might even require a registration process so that data is received and logged only from trusted sources.
Often SMS is used for data collection as it is the most affordable and available means of communication in
remote locations. This is generally the case, but you should be wary of cost implications. On a per bit or byte
basis, SMS is one of the most expensive means of network communication. It may be that for your application,
mobile data service (internet) is more effective and affordable, so some cost comparison should be undertaken
first. (See Weaver article, Uganda Diversifies Use of RIPI Account, for an example where GRPS data service is
used instead of SMS.)

Regardless of the application, at the end of the day, each will require some technical capability to maintain a
database and program scripts that can process information based upon a set of rules. More importantly your
server infrastructure will need to interface with one or more of the methods described below, if it is to send and/
or receive SMSs.

Types of SMS Integration

There are two basic ways to go about interfacing your server infrastructure with the mobile network. The first
and easiest is using a web based service; I’ll call this method ‘indirect network integration.’ It is indirect in
that your server does not directly connect to the mobile network via a locally attached mobile device (modem
or phone). Rather it connects via the Internet to a third-party or perhaps directly to the mobile carrier’s SMSC
(SMS Center). As a crude description, your ‘machine’ communicates with another ‘machine’ via e-mail, an
HTTP API (URL call with a POST or GET of data), a specific protocol called SMPP, or even FTP to provide
the outgoing message and list of recipients as mobile numbers. The recipient machine then handles routing of
messages over the mobile network. As a general rule these sorts of services are easiest to integrate with existing
server based systems, tend to be better for time sensitive and high volume messaging, work more effectively
for international applications, and when all costs are considered (not just message cost, but deployment and
maintenance of infrastructure) are typically the least expensive, although per message fees may be higher. The
down side is that unless you pay additional fees for two way services, you can generally only send messages. A
major weakness of indirect network integration is reliance on Internet connectivity. If Internet connectivity is
slow or vulnerable to frequent outages, then your ability to send or receive messages may be limited. Another
concern with indirect methods relates to the involvement of third-party services when critical communications
are involved. It is often difficult to secure a Service Level Agreement (SLA) with specific guarantees of
reliability necessary for intensive operations such as warnings and alerts.

The more complex method is to develop your own mobile hub or gateway; I’ll call this ‘direct mobile
integration.’ You generally do this by purchasing multiple GSM/GPRS modems or simply mobile phones,
attach these to one or more computers, and then acquire or write your own software application that serves as
an interface between the modems and computers. Direct mobile integration can be challenging and will require
technicians with a broad range of skill sets. In short the task is not just one of data processing and networking,
but now it also involves integration of hardware. You can offset the technical capacity required to setup and
maintain such a system by using either commercial or free software that handles the hardware interface, but
these tend to be less flexible, and for high volume messaging are expensive. Moreover, the more powerful
software applications in turn require their own learning curve to fully integrate with existing server systems.
Another downside of managing your own infrastructure is that it tends to become ineffective or costly for time
sensitive and high volume messaging.

The benefit of a direct to mobile network architecture, however, is largely control. As long as your local
infrastructure remains operable, as well as the mobile network, the system is typically unaffected by Internet
outages. (This may not be the case if you distribute your mobile hub over a network of computers connected
only by the Internet. And similarly if the information you want to distribute or log is not locally stored.)
Another benefit of direct mobile integration is that you will most often have the ability to receive incoming
messages without added software or infrastructure costs; beyond those required to send messages. An added
strength of direct mobile integration is that you can reach mobiles on a carrier not accessible via a larger
gateway service. At times, some mobile gateway providers and SMSCs cannot route messages to other
carrier networks. This is particularly true of telcos in smaller, less populated countries. By simply placing
your infrastructure directly on the mobile network of the target carrier, you can bypass such routing obstacles.
Finally, if your mobile hub is designed for messaging within the same country, typically the per message cost
will be cheaper. If you expect to operate a long term messaging application with relatively low volume through
put, such as for collection of weather data, then direct mobile integration is cheaper in the long run, although it
may require some significant up front capital investment.
Indirect Network Integration: Direct SMSC Connection

I’ll jump right in with one of the easier

methods to get up and running, although the
details vary greatly with each SMSC /
carrier. Most mobile wireless service
providers will make a direct connection
available to their SMSC. This is in effect
the carrier’s own gateway for processing and
routing SMS messages. Access to the
SMSC is not typically provided to individual
users, but rather it is targeted at high volume
users, such as government and commercial
entities. In short, you will need to approach
the carrier to determine how access is
granted, to whom, and what fees (setup,
recurring, and per message) apply. In our
experience most carriers are willing to open
up access to the SMSC for government and
similarly civil society oriented NGOs, even though the expected messaging use is low. The difficulty we have
had with such services is negotiating charge limits or the ability to turn off services. Most seem to prefer a
post-pay scenario, which leaves you vulnerable to cost overruns if your number is ‘spammed’ by unrelated
messages, or a similar scenario arises where unwanted messages cause you to incur a cost nonetheless.

If you secure access with the SMSC, then typically you will interface to that SMSC via an Internet connection
/ leased line. SMSCs may offer one or more methods for sending and receiving SMS. A common interface
for ‘non-enterprise’ users is through formatted e-mail. Upload of formatted text files via FTP is also
common. Many of also provide some form of HTTP API, which is usually the preferred method. In this
case you send messages by calling a special URL and appending the data in the URL string through either a
POST or GET. (For those familiar with the term, this is referred to as a REST-API or REST service). This
might look something like the following: smsc.carrier.net/userid=2383&userpass=j234ks89321&msg=Hello
There&mobile=555555555. Similarly you would call an API to access messages sent to your assigned number.
In some cases the HTTP API will be revered. You will be required to provide the URL that processes either a
POST or GET from the SMSC.

While the above methods are relatively straight forward for most programmers, some SMSCs may only
provide SMPP access. SMPP is a protocol to allow peering relationships between SMSCs. For those without
experience, the protocol will have a steep learning curve, and therefore either a different method will need to
be found, or some form of middleware script / software acquired. If you search for ‘SMPP library’ and then the
programming language of choice, you will likely find a number of free and commercial programs/resources to
help you use SMPP.

As a final note some carriers will require that you provide your own servers and internet connection. By this I
mean they will require that you place a server at their SMSC. This is generally only advisable for high volume
or otherwise critical applications. And, of course doing so increases your overall capital investment and
operational costs.

In general a direct SMSC connection enables high volume messaging quite well. Specifics on how to connect
and costs are likely to vary considerably, and of course carriers may choose to limit access or have a criteria
for which you may not qualify. Like most indirect methods, the greatest weakness is reliance on a network
connectivity. If you Internet fails or slows significantly, then messaging capabilities are effectively lost.

Indirect Network Integration: SMS Gateway

An equally easy method of connecting your

server infrastructure to the mobile network
is by working through an SMS Gateway.
There are many commercial providers who,
like the carrier SMSCs, provide a series of
APIs (Application Programming Interfaces).
Rather than connecting to an SMSC via one
of the above mentioned Internet methods,
you will connect to a commercial facility;
again via an HTTP API, FTP, e-mail, or
other. The commercial entity in turn handles
the connection to the SMSC(s). In short a
commercial SMS Gateway serves as an
intermediary -- often with the intent of
making access easier and potentially helping
you avoid complex contracts. In some cases
it is cheaper. Popular SMS Gateways
(sometimes referred to as SMS Resellers)
are Clickatell and mBlox. Many more exist; particularly if your interest is regional or even country specific. A
simple web search for SMS Gateway will bring up many of the providers.

You will likely work through an SMS Gateway for a number of reasons. First, if you are working
internationally or regionally, then often you may have a better (simplified) service by using a single commercial
SMS gateway provider. More significantly, many SMS Gateways operate via pre-paid accounts, such that
you are not locked into recurring service fees. As a result all of your funding will go towards messaging. The
exception to this is if you need two-way communication, are renting a short code, or perhaps require some sort
of premium service. Of course by avoiding monthly service fees, you will pay a higher per message fee.

A major difference between working through a commercial gateway rather than a carrier is that your service
(without paying addition fees) will be one-way. Quite often if you setup a service with a carrier to send
messages through their SMSC, you will automatically be provided with the capacity to receive messages as
well. That is not the case with a commercial SMS gateway; unless you are willing to pay for added service.
But, if your application is limited to outgoing messages, then the SMS gateway is often easier and more cost
effective to implement than a direct carrier connection.

The major limitation of many SMS Gateways is that they do not provide any guarantee of throughput. As
a result if SMS traffic through the commercial gateway is significant, you may end up with added latency.
Additionally, many commercial gateways can only send and receive messages from certain carriers or countries.
So, before signing up for a SMS Gateway service, it is worth researching the provider to determine service
reliability (speed and uptime), as well as coverage.
Direct Mobile Integration : Overview

In the above methods, your server or other

computing infrastructure connects to the
mobile network through an intermediary be
it an SMSC or commercial SMS Gateway
service. The connection is typically via an
Internet service. You can alternatively set
up machine based messaging by directly
connecting your infrastructure to the mobile
network through a mobile device. In this
case a GSM/GPRS modem, or simply a
mobile phone, is attached to a computer. As
described earlier the reasons for doing so
may be for desired control, unreliable
internet connections, low-cost two-way
communication, or simply your messaging
application is modest and does not require a
larger service. RANET, for both its RAW
and Community Reporter Program (CRP),
utilizes a combination of commercial SMS Gateways and directly attached modems. In fact the directly
attached modems are our primary means of communication, while the gateways are utilized for redundancy or
in other very special applications. We do so as generally we have found the direct connection to modems more
reliable and faster, but a significant capital investment (numerous modems and servers) was required to achieve
this performance.

Regardless of the scale of your application, making a direct mobile connection requires two things. First you
will need one or more mobile devices, and second you will need some sort of software that will allow your
scripts to talk with the modems or mobile phones.

Before we look at hardware and software, there is one thing to consider. Some carriers may place limits on
automated messaging, and/or there may be national regulations. For instance a carrier may allow you to send
messages from a modem, but will restrict the number of messages that can be sent on a daily or other basis.
Similarly, a carrier may limit the number of modems present at any one location that can send and receive
messages automatically. The reason in both cases is to prevent an unnecessary load on a local tower. Generally,
the cost of acquiring enough modems or sending messages will prevent you from reaching their limits, but
it is better to be safe and contact the carriers you plan to utilize. Finally, some countries may require you to
purchase a short code (in essence working through an SMSC) if your daily or monthly messaging is expected to
reach a certain level. Again, it is unlikely you will reach such a level, but it is better to discuss these potential
restrictions before investing time and resources.

Direct Mobile Integration : Hardware

Let’s take a look at the hardware you will need. Again, you will need a modem or mobile phone. A GSM/
GPRS modem is in essence specifically designed for machine integration, and where possible should be your
first choice. (As a note I refer to GSM/GPRS, but there are modems available for CDMA networks as well.
GSM tends to be the most common network standard, however.) While a mobile phone can be connected to a
computer or server, these devices tend to cause some complications. Many mobile phone manufacturers utilize
non-standard / proprietary protocols, and therefore the device may not be compatible with any of the software
you might use to interface the device to your computer. Additionally, mobile phones have added functionality
and processes that may interrupt or disconnect the device from your computer unless these are turned off. For
instance in our RANET tests years ago, one mobile phone disconnected each time it received an incoming
call. Modems generally follow standard AT commands / protocols, are semi-ruggedized, and lack a screen or
other human interface, which makes it unappealing to any passerby. Regardless of your choice, you should
determine if either the modem or mobile phone is supported by the software you will use to interface the device
to your server. RANET utilizes the MultiTech GSM/GRPS Multimodem in many of its applications, but other
common modems manufacturers are Falcom, Huawei, and Wavecom. You may be able to by a modem through
your carrier.

Most modems will cost somewhere between $150-300. Both modems and mobile phones have roughly the
same messaging rate, so performance in terms of speed is not a determining factor. That said, before building
a direct connection system, you should try to determine your expected messaging load. How fast do you need
to get messages out, and how many incoming messages will you receive? If you do a little searching, most will
estimate that a single modem / mobile phone can process a message in about 5 to 15 seconds. From RANET
experience we budget and estimate that a message can be processed every 12 seconds by a single modem. So,
the math becomes pretty simple here. If you envision sending out a message to 100 recipients, and you need
to get the message sent (not necessarily received -- you can’t control network latency) within one minute, then
you will need at least 20 modems. (Each modem can send 5 messages a minute; 60 seconds / 12 seconds. So,
100/5 = 20.). As you can see, for high volume or time critical applications, the capital investment to support a
direct connection can be expensive. (As another note your speed will be determined in part by the software you
select. Generally I find graphical interfaces slower at messaging than scripts that simply send AT commands to
the modem.)

Before investing in one or more modems / phones, you will also need to consider the GSM band of your local
carriers. There are four bands typically paired as 850/1900 MHz and 900/1800 MHz. Although, it is not
uncommon for a carrier to use only one band, such as 900 MHz. You can deal with band issues in a couple
different ways. First, many modems available are quad band, so while you may need to configure them, if you
get a quad band modem you can be assured that it will be compatible with your local carrier. Alternatively you
can contact your carrier to find out what band or bands are utilized, or you can look at online resources, such
as the GSM World coverage maps and country listings. The important thing here is to determine the bands
available before purchasing modems, or alternatively buy your modems directly through the carrier.

There is one more factor you should consider before acquiring your modem(s) or mobile phone(s), and that
is the connection type. Generally you have the option of USB, serial (RS-232), and ethernet. Unless you are
writing your own programs and scripts, you can most likely discount the ability to connect to a modem via
ethernet. No mobile phones that I’m aware of use this connection. An ethernet connection in modems is often
used for remote control applications utilizing GRPS mobile data service, and most software to interface with a
modem (at least described in this article) is incapable of doing so over an ethernet connection.

Most mobile phones and newer modems use a USB connection. A USB connection is not a true port, as a
result an ‘emulator’/driver to create a port will come with the device. So with USB devices it is critical that
you ensure it can connect and be recognized by your operating system. Namely, if your server infrastructure
is Linux / Unix, then you should ensure that the modem comes with software necessary to make the modem
visible in the *nix file paths. Similarly, if operating a Windows OS, you should ensure that the drivers are
compatible with the version you are operating so that the USB modem is available as a COM port.

Another consideration with USB devices is power. The advantage of USB modems / phones is that they almost
always draw their power from the computer (5 volts). This makes installation cleaner and easier as you do not
need to find power outlets for the device. The downside is that if you are attaching many modems to a single
computer / server, it may not actually be able to provide adequate power. This is particularly true if you have
added an internal USB card to increase the number of connections, while not upgrading an older power supply.
Simply, your modems may not all work or get enough electricity to operate correctly. Of course this highlights
another issue, which is that you may require more USB or RS-232 connections than any single server or
computer can accommodate.

The other connection type is RS-232. This is a common and old standard. It is very stable, and the only
downside is perhaps that it will require its own power source, which can make installation of many modems
difficult. The other downside is that if you are attaching more than one or two modems via an RS-232
connection, you will almost certainly need to add more connections to your existing server / computer.

A final consideration for your hardware is interference. Namely, before investing in modems and such, go
to the building and specific room where you plan to house equipment. Then, try to see if you can get an
adequate connection from your and other mobile phones. Many server rooms may either be constructed or
have equipment that interferes with the mobile signal. When RANET first tried building its system, we later
determined that satellite dishes / systems placed on the roof of building interfered with the mobile carriers’
signal.

Direct Mobile Integration : Software

Once you have identified the modems and servers you will use, you now need to select a software application
that will allow your other server processes to send and receive messages from the attached modems. Rather
than cover these in great detail, I will just list some common applications -- both commercial and freeware
options.

I should note first, however, that you can attempt to write your own software to interface with the modems
-- particularly for simple outbound messaging. You can communicate with all modems via a series of AT
Commands. AT Commands are simple instructions sent to the modem that request information or provide it
with information. Most modem manufacturers provide a list of AT Commands that can be used with a particular
modem model, and a simple online search will produce a list of standard commands you can utilize. The tricky
part will be figuring out how to connect with the modem in the first place. Languages like Perl and PHP have
modules / classes readily available for serial / COM port communication. You can also purchase software /
scripts, such as ActiveXperts COMPort Toolkit. In this later case a command line tool is provided to interface
with a COM port. This allows you to select most any programming language to interface, and of course you
are then able to focus on writing the AT Commands and processes necessary to send and receive SMS. Clearly
there is more flexibility and potential power in this sort of setup, but it is complicated and requires some
technicians with broader experience.

For many applications where you just want to send a message and perhaps process incoming SMS, you will
want to look at many of the already available applications. We can divide these into three types; 1) desktop
applications, 2) services, and 3) scripts and libraries.

One of the most popular and free to use desktop applications is FrontlineSMS. This was covered as a three part
Weaver tutorial. FrontlineSMS has the look and feel of an desktop e-mail client or similar desktop application.
It requires no scripting / programming for basic applications, but at the same time it is powerful enough to
integrate with your servers. While FrontlineSMS has some serious power that rivals other enterprise level
packages, it is first and foremost designed as a personal messenger. It will, at least not from RANET testing,
immediately install as a service. As a result it is not particularly designed to run on a server environment where
there will not be an actively logged in user.

The general benefit of desktop applications is that they require little if any technical expertise to utilize. The
downside is that many cannot easily integrate with server processes, and moreover will only allow or utilize
a single modem. In the later case, this means you cannot use a number of mobile devices to increase your
throughput and speed. The exception to this FrontlineSMS, which will allow and utilize a number of modems
to increase messaging rate. Other applications worth examining are XGnokii, Gnocky, and even the Microsoft
SMS Sender. Although the Microsoft SMS Sender appears to be discontinued to a large degree. All of these
applications are free to download and use.

There are some full SMS servers / services that will be more appropriate for most applications. In a word these
can be complicated. The most well known and powerful is Kannel. It, however, is largely limited to Linux/
Unix environments and for many will require a lot of time to understand and fully implement. NowSMS is
another very powerful SMS server. It is considerably easier to use, however, it requires a commercial license to
use. The cost of the license relates to the expected message throughput. Yet another SMS server is ActiveSMS.
It lacks some of the flexibility and power of NowSMS or Kannel, but it is for the most part extremely easy
to use and integrate. It can be integrated via e-mail or your own scripts. It too, however, is a commercial
application with set licenses. These licenses correspond to the features needed, as well as the number of
modems you plan to attach. At the time of this writing ActiveSMS, with its highest license, can control and
command eight modems.

Another commercial application, which is more of a command line utility than service, is ActiveXperts SMS
and MMS Toolkit. Like the previously mentioned ActiveComport Toolkit, this application provides a simple
command line utility that can be called by most any programming language, if you are planning to work in only
Windows environments.

Finally, there are open source projects such as Gnokii and Gammu. These are popular and powerful
applications, although like many open source efforts, are not fully documented. As a result use of these tools by
inexperienced programmers may be a bit elusive.

---------------------------------

I hope this provides a little insight into what resources are needed to begin to plan and develop your machine
based SMS application. There are a lot of options available, which while providing considerable flexibility,
make development of such systems appear complicated. A lot of this complexity, however, can be reduced once
you have a general sense of the pieces necessary to deploy a messaging system. Hopefully this article provided
you with a first step. Future Weaver articles will provide a step-by-step walk through of some of the above
mentioned applications.