Professional Documents
Culture Documents
Applications of Machine Learning To CR
Applications of Machine Learning To CR
Licensed signal channel access ABSTRACT can sense and adapt to its environment. The
term cognitive implies awareness, perception,
Cognitive radio offers the promise of intelli- reasoning, and judgment. However, nowhere is
gent radios that can learn from and adapt to learning required.
their environment. To date, most cognitive radio So far this has fueled much research into pol-
research has focused on policy-based radios that icy-based cognitive radios. These are radios
are hard-coded with a list of rules on how the whose operation is governed by a reasoning
radio should behave in certain scenarios. Some engine that examines the current state of the
work has been done on radios with learning environment and makes decisions on how the
engines tailored for very specific applications. radio should operate. An example of this might
This article describes a concrete model for a be an IEEE 802.11 modulation controller that
generic cognitive radio to utilize a learning switches from 16-quadrature amplitude modula-
Secondary user channel access engine. The goal is to incorporate the results of tion (QAM) to quaternary phase shift keying
the learning engine into a predicate calculus- (QPSK) to binary PSK (BPSK) as the signal-to-
based reasoning engine so that radios can noise ratio (SNR) decreases [1].
The authors describe remember lessons learned in the past and act
quickly in the future. We also investigate the dif-
Generic learning-based cognitive radio is a
relatively unchartered research area. Various
a concrete model for ferences between reasoning and learning, and the projects have used techniques such as genetic
fundamentals of when a particular application algorithms to evolve radio parameters with the
a generic cognitive requires learning, and when simple reasoning is goal of optimizing performance [2]. In contrast,
sufficient. The basic architecture is consistent this article examines the fundamentals of learn-
radio to utilize a with cognitive engines seen in AI research. The ing and reasoning, and proposes an architecture
focus of this article is not to propose new to use them together. We then apply the frame-
learning engine. machine learning algorithms, but rather to for- work to two common problems in cognitive
The goal is to malize their application to cognitive radio and
develop a framework from within which they can
radio: capacity maximization and dynamic spec-
trum access.
incorporate the be useful. We describe how our generic cognitive In this article we describe the cognitive radio
engine can tackle problems such as capacity max- architecture, and discuss reasoning and learning
results of the imization and dynamic spectrum access. engines. We then describe applications and how
they work with the described model, and outline
learning engine into our cognitive radio implementation. We then
INTRODUCTION conclude the article.
a predicate In todays literature, cognitive radio is often
calculus-based treated as a buzz word rather than a scientific COGNITIVE RADIO ARCHITECTURE
term since it has been used by so many different
reasoning engine. people to mean so many different things. The A software radio (SR) can be defined as a radio
most generally accepted definition is a radio that implemented with generic hardware that can be
programmed to transmit and receive a variety of
waveforms. Cognitive radio is often thought of
This work was completed while J. Hecker, E. Stuntebeck, as an extension to software radio, and here we
and T. O'Shea were with the Laboratory for Telecommu- treat it as such. A cognitive radio extends a soft-
nication Sciences, U.S. Department of Defense, which ware radio by adding an independent cognitive
funded this research. The opinions expressed in this docu- engine, composed of a knowledge base, reason-
ment represent those of the authors, and should not be ing engine, and learning engine, to drive soft-
considered an official opinion or endorsement by the ware modifications. A well defined application
Department of Defense or U.S. federal government. programming interface (API) dictates communi-
The SR exports
Radio parameters and
variables that are statistics are exported to Stores true predicates and
knowledge base as predicates set of available actions
either read-only or
read-write. The Executes actions from
the knowledge base
read-only parameters
Reasoning
represent statistics engine
Evolves predicates
Software directly to maximize
maintained by the radio
Knowledge
base objective function
Learning
SR, such as signal to engine
Evolutionary
Cognitive
Single cognitive engine
radio RF link exchanges application techniques such as
engine data, radio configuration
controls both the
transmitter and receiver
information, and radio statistics genetic algorithms
could evolve the
Master Slave
Source
application
software RF link software Destination
application
radios state in order
radio radio
to maximize the
Figure 2. Configuration of a basic simplex cognitive radio communications link. objective function.
After each state
Many different learning algorithms are available equates to our learning engine generating the
to a cognitive radio, including hidden Markov following actions for the policy engine: change, the resulting
models [3], neural networks [4], and genetic
algorithms [5].
action: learnedActionX state will be
precond: n N 1 in on (5)
Nearly every learning technique involves the
use of an objective function to determine the
postcond: in iN. evaluated. This
value of the learned data. In a cognitive radio, If the radio environment changes, however, process continues
these objective functions reflect the overall goal oN may not result from iN, and iN may no longer
of the application, such as maximizing channel be optimal. If this occurs, the learning engine until a globally
capacity. The goal of the learning engine is to should be aware of the changes and remove the
determine which input state will optimize the suboptimal learned action from the knowledge- optimal state is
objective function. However, unlike the objective base. This can be expressed as
function used by the reasoning engine, there is found.
action: unlearnActionX
no simple mathematical relationship between the precond: iN oN (6)
system inputs and the objective function. postcond: learnedActionX.
In order for learning to be necessary, the
precise effects of the changing inputs on the The cognitive process, both reasoning and
outputs must not be known. This is often the learning, is then reinstatiated in search of a new
case in non-ideal radio channels. For example, optimal state. The next section describes con-
we typically know that decreasing coding rate crete instantiations of the policy and learning
will also decrease bit error rate, but we do not engines for particular applications.
necessarily know by how much for an arbitrary
communications channel. Using a learning APPLICATIONS
engine, we can estimate our channel statistics;
then, using that data, we can make decisions In this section we describe several applications
that are optimal for our current radio frequency of learning to real radio problems. For each
(RF) environment. application, the inputs and outputs of the SR are
More formally, we define the radios state- defined, along with an objective function. We
space S. A particular state s S is made up of then describe strategies for how a cognitive radio
both the radios input predicates i and output can solve the problem.
predicates o. That is, s = i o. All the applications assume the basic commu-
We have an objective function fL: S R that nications architecture shown in Fig. 2. This
can be evaluated on a particular state s S and architecture defines a simplex link between a
returns a real number. State s 1 is preferable to source and a destination SR. The source radio
state s2 if fL(s1) > fL(s2). has a cognitive engine that controls the source
The learning algorithm will try to precisely radios parameters, and uses the existing link to
characterize fL() in an effort to find an optimal exchange configuration information (e.g., modu-
state. For example, a neural network using unsu- lation scheme, frequency, or bandwidth) and
pervised learning could try to find statistical cor- statistics (e.g., noise power or bit error rate).
relations between inputs and outputs to come up In order for this system to work, we must
with a mathematical representation of fL(). assume that both the master and slave radio
Similarly, evolutionary techniques such as start off with the same initial configuration c 0.
genetic algorithms could evolve the radios state When the cognitive radio decides a change to
in order to maximize the objective function. configuration c i+1 is necessary, the master SR
After each state change, the resulting state is sends a message containing c i+1 to the slave
evaluated. This process continues until a globally using configuration ci. A simple example of this
optimal state is found. would be a frequency hopping radio system
Let us assume that the learning algorithm where a hop to frequency f i+1 is signaled by
evaluated N different inputs i 1 , i 2 , , i N with sending a control message at frequency fi.
resulting outputs o1, o2, , oN before finding the To extend this basic architecture to a duplex
optimal one. This implies that link between two SRs, a cognitive radio engine
can be added to the second radio. However,
fL(iN oN) fL(in on) 1 n N 1. (4)
here we assume that each cognitive engine acts
Thus, we now know that if our radio is ever as an independent agent controlling only out-
in state i 1 o 1 , , i N1 o N1 , the optimal bound data and not communicating with other
behavior is to set the radio inputs to i N . This cognitive radio engines.
Frequency
Dynamic spectrum access (DSA) involves locat-
ing frequency bands and times when a cognitive
radio can transmit without causing harmful
interference to other transceivers [7]. For exam-
ple, consider a cognitive radio network operat-
ing in the UHF television bands, where
transmission is permissible provided devices can
guarantee they will not interfere with licensed
broadcasts.
More concretely, the goal is to locate center
frequencies, bandwidths, and times when a cog- Time
nitive radio can transmit, while maximizing Secondary user channel access
capacity and minimizing interference. This type
of problem is very different from capacity maxi-
mization where we wanted to learn how our Figure 4. Example showing cognitive radio coexisting with bursty signals in
radio inputs affected our radio outputs. DSA both frequency and time.
involves learning where and when other radios
will be transmitting.
If we assume all signals with which we are
trying to coexist are continuous in time, the Another interesting extension to this idea is
problem simplifies to one solvable by a policy- to consider non-continuous signals, such as ones
based radio. Imagine our SR exports predicates using time-division multiple access. For these sit-
to the knowledge base regarding detected signals uations, we can coexist not only in frequency but
s1, s2, , sN that are of the form also in time. Using various cyclostationary statis-
tical learning algorithms [8], we can compute the
signalFreq(si, fi) signalBW(si, Wi). (9)
expected length of channel vacancy and incorpo-
Our goal is to find some f c and W that do not rate that into our decision making.
overlap any detected signal, while maximizing W Imagine a neural network has computed for
and consequently the radios capacity. us the probability Pi(t) that signal si is transmit-
First, let us assume we have a helper function ting at time t, where the probability is periodic
notOverlap(fc, W, si) that evaluates to true if the over time i. Our learning engine can then export
band occupied by a signal centered at f c with to the knowledge base predicates describing this
bandwidth W does not overlap signal si. Then we signal.
can define our predicate,
signalFreq(si, fi) signalBW(si, Wi)
action: moveBand(fold, Wold, fnew,Wnew) (t < , Pi(t) > signalON(si, t) (13)
precond: i N notOverlap(fnew, Wnew, si) (t < , Pi(t) < signalOFF(si, t).
(centerFreq(f{old) bandwidth(Wold))
A threshold is used to indicate that the sig-
postcond: (centerFreq(fold) bandwidth(Wold))
nal is present with very low probability. We can
(centerFreq(fnew) bandwidth(Wnew)).
now extend our notOverlap function to take into
(10)
consideration whether si is transmitting at time t,
Then we define our objective function, and our radio can compute its optimal course of
action.
fR(centerFreq(fc) bandwidth(W)) = W. (11)
action: moveBand(fold, Wold, fnew, Wnew)
We now have a policy-based cognitive radio that
precond: i N : notOverlap(fnew, Wnew, si, t)
will search out the largest continuous piece of
postcond: (centerFreq(fold) bandwidth(Wold))
bandwidth for communication.
(centerFreq(fnew) bandwidth(Wnew))
In more complex noise and interference envi-
(14)
ronments, this approach is too simplistic, as dif-
ferent frequency bands may have different noise We can use the same objective function as
floors. Extension to this environment will require before, or incorperate SNR if necessary.
the use of a learning engine. Imagine our SR
exports predicate snr(S) for the currently tuned IMPLEMENTATION
center frequency and bandwidth. This gives us
more information, and we can define a better To build on the ideas developed in this article,
objective function based on the Shannon-Hartley we have implemented a cognitive radio based on
law: a generic cognitive engine [9]. It combines
OSSIE, which is an open source software com-
fL(centerFreq(fc) bandwidth(W) snr(S)) (12)
munications architecture (SCA) core framework
= W log2(1 + S).
implementation from Virginia Tech, with the
This learning objective function can only be eval- Soar cognitive engine from the University of
uated when S is known, and this can only be Michigan.
computed after the radio has been tuned to cen- Within it we have implemented both the poli-
ter frequency fc. cy-based capacity maximization and the learning-