THE USE OF MOBIUS TRANSFORMATIONS

IN N E U R A L N E T W O R K S A N D SIGNAL
PROCESS1N G

Danilo P. Mandic
School of Information Systems
University of East Anglia, Norwich, NR4 7TJ, UK
Phone: +44 1603 592 569
Fax: +44 1603 593 345
E-mail: d.mandic@uea.ac.uk
Web: sys.uea.ac.uk

Abstract. A framework for the use of Mobius transformations in

general neural networks (NNs) and signal processing is provided.
It is first shown that both a nonlinear activation function of a neu-
ron and a first order all-pass filter section can be considered as
Mobius transformations. Further, the global input-output rela-
tionship in layered NNs is shown to belong to a modular group
of compositions of Mobius transformations, whereas cascaded all-
pass digital filters are shown to represent the Blaschke product of
Mobius transformations. Finally, Routh stability in nonlinear field
filters is briefly addressed in this context. For rigour, existence and
uniqueness of such an approach is considered.

INTRODUCTION

Mobius and related mappings have been used in signal processing [6], linear
circuits [18], microwave circuits [7], and neural filters [20, 5, 121. Hence, there
is a need for a unified approach that would connect some inherent properties
of neural networks, such as fixed points, invertibility and stability, digital
filters, and modular groups of composition of Mobius transformations. Let
us start with the definition of a Mobius transformation.
Definition 1 (Mobius mapping) Let U , b,c, and d denote complex con-
stants and ad # bc. The function
uz + b
w =f(2)= -
cz + d
is called a Mobius transformation, bilinear transformation, or linear frac-
tional transformation.

0-7803-6278-0/OO$10.00 (C) IEEE 185

The condition ad # bc is necessary t o provide a one-to-one mapping, since
for complex numbers z1 and z2, f ( z L )- f ( z 2 ) = ~ ~ = const,
~ for 1 $ ~ ~

ad - bc = 0. A Mobius transformation is a one-to-one and onto mapping,

transformation w = S ( z ) = e
and its inverse is also a Mobius transformation. For example, the Mobius
maps the unit disk IzI < 1 one-to-one and
onto the upper half plane S{w} > 0, where S{.}is the imaginary part of a
complex number.
In this paper, we first identify the activation function of a neuron and a
single-pole all-pass digital filter section as Mobius transformations. Then,
in order to consider a general neural network and a general cascaded filter in
the same framework, the theory of modular groups and Blaschke products is
introduced. Some inherent properties of neural networks, such as fixed points
and invertibility, and group delay properties of cascaded all-pass filters, are
shown t o be the consequence of their Mobius representations. Finally, sta-
bility issues in some N N architectures are addressed via Routh stability and
Mobius mappings.

SIGMOID ACTIVATION FUNCTIONS A N D HOLOMORPHIC

MAPPINGS

Two typical choices for activation functions employed in neural networks are
the logistic function [9] given by

and the hyperbolic tangent

1 - e-Onet
f(n4= + e-Onet 1 P E IR+ (3)

Notice that both functions are holomorphic and conformal.

Activation Functions and Mobius Transformations

Equating the coefficients of a tanh activation function (3), t o those of the

Mobius transformation ( 1 ) , we have
1 - e-Bnet
-
az + b
1 + e-Ocnet cz + d
-
(4)
The tanh function satisfies the conditions of a Mobius transformation for
z = e-Onet and

az-1, b=l, c=l, d=l (5)

The logistic function represents a Mobius transformation for a = 0, b = 1, c =
1 , d = 1. In both cases the condition ad - bc # 0 is satisfied. Hence the
following proposition.

(C)IEEE
0-7803-6278-O/OO$lO.OO 186
Proposition 1 The sigmoidal transformation f ( z ) performed by a neuron
i n a n NN o n a complex input signal z = a + j , B is a Mobius transformation.

ALL-PASS FILTERS AND MOBIUS TRANSFORMATIONS

All-pass systems arise in multimedia signal processing [19], processing of

seismic waves [16], and in the case of reflexion in array signal processing [8].
A detailed review of all-pass filters is given in 1171. A first order all-pass
section has a transfer function

where z is the complex variable, p is a (possibly complex) pole of H ( z ) , and

(.)* denotes the complex conjugation. For a pole p = rej', the phase function
for (6) is given by

@)(U)= -w - 2arctan
T sin (w- 0)

I - cos - e) 1 (7)

The group delay .(U) of (6) is the negative derivative of the phase a, and is
given by

1- r 2
=
1+ r2 - 2 r c o s ( w - 0)
.(W)

For a stable H ( z ) , T < 1, and the group delay is positive.

Matching the coefficients associated with the powers of z in the transfer func-
tion of an all-pass filter (6), and in a general form of the Mobius transform
( l ) ,we have

2-l -p* -p*z+l - U Z + ~

- -
1-pz-l Z-p C Z + ~
(9)

The first-order all-pass filter satisfies the conditions of a Mobius transforma-

tion for

Notice that ad - bc # 0 if Ipl # 1.

Fixed Points of All-Pass Sections

Definition 2 A point x* which is mapped onto itself under a map G so that

x* = G ( x * )is called a fixed point.

They are also called invariant points, or fixed elements. The Brower fixed
point theorem shows that a continuous function on an interval has a fixed

(C) I E E
0-7803-6278-0/00$10.00 187
point [23]. This means that, for a single-dimensional case, for a continuous
function g(z) E ( a ,b ) , Vx E [a,b], g has a fixed point in [a,b]. As a first
order all-pass transfer function is a Mobius transformation, its fixed points
are given by

z =f(z) =
-p*z +1
z-P

+
and can be found at z1,2 = S { p } f J ( ~ { P } ) ~1, wherg S { p } denotes the
imaginary part of p .
For first-order sections, p is always real, and z 1 , 2 = f l ,which correspond t o
w = 0, and w = n in frequency. Equation (8) shows that the fixed points of
(6) correspond to the maximum and minimum of the group delay at w = 0
and w = n.

MODULAR FUNCTIONS AND LAYERED NEURAL NETWORKS

In order t o consider a general neural network, we introduce a modular group

of composition of Mobius transformations.
Remark 1 The Mobius transformation does not determine the coeficients
a, b,c,d uniquely, i.e. if 'p E C \ 0, then coeficients 'pa,'pb, 'pc, 'pd correspond
to the same transformation.
This means that a Mobius transformation remains unchanged if all the coeffi-
cients ( a ,b, c, d ) are multiplied by the same nonzero constant'. In the follow-
ing analysis, without loss in generality, we will therefore assume ad - be = 1.
To analyse an n-dimensional cascaded all-pass filter that consists of first-
order modules, we introduce the notion of a modular group [2].

Modular Groups

Definition 3 ([2]) The set of all Mobius transformations of the f o r m

where a , b, c, d are integers with ad - bc = 1 is called a modular group and is

denoted by r'.
To analyse properties of a modular group, it suffices t o look a t matrices

A = [ : pl], A=] (13)

'This property is often used in the bilinear transformation for synthesis of IIR digital
filters. Namely, although the mapping from the s into z domain is defined as s = $6,
the constant is often skipped [21].

0-7803-6278-0/00$10.00 (C) IEEE 188

that represent the group. To build the group under composition, we introduce
the identity transformation f ( z ) = z = H,
described by the identity matrix
I , and the inverse element A-l which is associated with the inverse o f f given

this context as
s.
by f - ' ( z ) = The Mobius transformation can be now represented in

az + b
AZ = -
cz + d
Modular group is generated by two transformations, T z = z + 1 and S z =
-5 [2], as shown in the following theorem.
Theorem 1 The modular group r is generated by the two matrices

(15)

and every A E can be expressed in the f o r m

A = T"'STn2S. . . ST"k (16)
where ni are integers.

So, for instance, modular representation for the matrix A =

A = ST-3ST-4ST2. This representation is not unique.

MODULAR FUNCTIONS AND NESTED NONLINEARITIES

If A and B are matrices associated with Mobius transformations f and g ,

then a composition f o g is described by the matrix product A B .

Proposition 2 The input-output relationship between two neurons in con-

secutive layers within a general neural network belongs to a modular group
r.
Proof: Without loss in generality let us consider only two neurons from any
two consecutive layers in a NN. The functions performed by these neurons
are respectively H l ( z ) = ~:~~fi:
and H 2 ( z ) = *. Their composition
(transfer function from a layer t o a layer) is

and belongs t o r.
The Mobius mappings performed by H I and H Z are re-
spectively described by matrices A41 = bi
dl [: ]
and A 4 2 = [ : 2 1,

0-7803-6278-0/00$10.00 (C)IEEE 189

which gives

which complies with the theory of modular groups. 0

The global input-output relationship in a neural network can be therefore
considered in the framework of compositions of Mobius transformations.

FIXED POINTS, INVERTIBILITY, AND STABILITY IN NEU-

RAL NETWORKS VIA MOBIUS TRANSFORMATIONS

Recall that a point x* which is mapped onto itself under a map G so that
x* = G(z*) is called a fixed point. Fixed points are important in the analysis
of stability criteria, neural networks [4, lo], and learning algorithms [13, 111.
Therefore to obtain fixed points for a general NN (which is a composition of
Mobius transformations) we look at fixed points of the nonlinear activation
function of a neuron, which is a Mobius transformation itself.
Observation 1 Fixed points of a neural network are determined by fixed
points of the employed activation function.
Obviously, the existence for fixed points of an activation function are guar-
anteed by the underlying Mobius transformation (one or two fixed points).
As the global input-output relationship in an NN can be considered in the
framework of Mobius transformations, it has one or two fixed points.

Invertibility in NNs

The problem of inverting trained NNs is t o find the inputs which yield a given
output. This problem is an ill-posed problem because the mapping from the
output space t o the input space is a one-to-many mapping. The existence
and non-uniqueness conditions of an inverse of a neural network are provided
by the theory of modular groups of underlying Mobius transformations.
Existence: The input-output relationship between two neurons within an N N
is a Mobius transformation, whose inverse f - l ( z ) =
transformat ion.
s, is also a Mobius

Uniqueness: By Remark 1, Mobius transformations are determined up t o

multiplication by a constant. Hence, the inverse of an N N is not unique.

Routh Stability Criterion via Mobius Transformations

Stability is an important issue in nonlinear neural field filters. A nonlinear

field filter has a vector field input and output [l,201. On the other hand the

(C) IEEE
0-7803-6278-O/OO$lO.OO 190
Routh stability criterion is given by

where C ( T ( z ) )is the system response transform, R ( T ( z ) )is the input trans-
form, W ( T ( z ) ) / Q ( T ( zis) )a transformation of the transfer function. Fol-
lowing the approach from [20], we introduce the Mobius transformation as a
DG - a(e~~*)+b
transformation T ( z ) in (19), T(e- ) - c ( e - D e ) + d l which can then be ap-
plied to (19) to give the Routh stability criterion based on the Hamiltonian
system of equations for a nonlinear neural field filter.
Thus, for input-to-output and output-to-input transfer functions of a neural
field filter, denoted respectively by H I and H2, we have the stability criterion
given by

which gives the stability defined by the extended nonlinear Routh stability
criterion which uses Mobius transformation operators instead of Laplace and
Fourier operators. These filters can be used as basic building blocks for
speech and vision recognition and image processing [l].

CASCADED ALL-PASS SYSTEMS AS BLASCHKE PRODUCTS

OF MOBIUS TRANSFORMATIONS

To consider cascaded all-pass systems, the so-called Blaschke product is first

introduced. The Blaschke factor <(a,z ) is defined by

a* a--z
[ ( a , z )= -~
la1 1 - a*z
where [ ( a ,z ) = z if a = 0. Let {o,},~N be a sequence and let

Then the product

is called the n-th finite Blaschke product with respect t o the sequence { a n } n E ~ ,
where B o ( z )= 1. (We assume that a0 = 0).
A normalised Blaschke term is given by

0-7803-6278-0/00$10.00 ( C ) IEEE 191

 Notice that terms (21) in product (24) are Mobius transformations. In fact,
both the top and bottom term in the rational function (24) are Klein terms
+
f l ( z ) = z i- a : M1 = {{l,a),{O,l}> and fZ(z) = 1/(az 1) : M2 =
{{0,1>,{a,111.
Further, every proper holomorphic map f : ID -+ ID, where D is the open unit
disk, is a finite Blaschke product of the form [15]

j=1

where la1 < 1 and 1x1 = 1. Such a map is uniquely determined up t o a

rotation by its zeros { a l ,. . . , ad+l).
In [22] it is shown that two proper holomorphic maps, f,g : D + D have the
same critical points, counted with multiplicity, if and only if f = T 0 g for
some conformal automorphism T of the unit disc.
Hence, the Blaschke product provides the existence conditions for fixed points
of cascaded first order all-pass filters. For instance, for a second order all-
pass function

H(z)=
-p1z +
1 -pzz+ 1
(26)
z-Pl z-p2
with poles p 1 and pz, under the above conditions, such that the modules
represent Mobius transformations, H ( z ) represents a Blaschke product. The
matrices associated with H I and Hz are respectively M I =
[ -? ]
and M2 =
[ -”1 -P2 1-
The product HlH2 has the same form of critical points as H1( z ) and H 2 ( z ) .
This complies with theory of iterated Mobius transformations and their at-
tractors [3, 141.
Proposition 3 An n-th order cascaded all-pass filter consisting of first or-
der real-pole sections has the maxima of the group delay at w = 0 and w = T .

Experimental Results. A simulation of a cascaded all-pass transfer func-

tion is undertaken. In the experiment, 1600 positive and 400 negative poles
were generated from a uniform distribution, with lpil < 1, i = 1,.. . ,2000.
The group delay was calculated and normalised by the number of cascades.
The fixed points of the cascaded system were at 0 and T , and coincide with
the maxima. of the group delay, as shown in Figure 1. This justifies the
theoretically derived result.

CONCLUSIONS

A framework that encompasses the theory of modular groups of compositions

of Mobius txansformations I?, global input-output relationships in neural

0-7803-6278-0/00$10,00( C ) IEEE 192

 >
m
-

; 6 ~ ’
s
4-

2-

0-
05 1 15 2 25 3

Figure 1: Group delay for a cascaded all-pass structure with 1600 positive and 400
negative uniformly distributed real poles
networks, and cascaded all-pass filters has been provided. It has been shown
that using this framework, some inherent properties of cascaded systems and
layered (nested) neural networks can be derived as a consequence of belonging
t o group r. The Blaschke product of Mobius transformations has been further
introduced and shown t o represent cascaded digital filters. These results
apply for both simple elements and complex functions and help to unify the
theory of signal processing and neural networks.

REFERENCES

[l] S. Amari, “Field Theory of Self-organizing Neural Nets,” IEEE Transac-

tions on Systems, Man and Cybernetics, vol. 13, pp. 741-748, 1983.
[2] T. M. Apostol, Modular Functions and Dirichlet Series in N u m b e r
Theory, Springer-Verlag, 2nd edn., 1997.
[3] A. Barrlund, H. Wallin and J. Karlsson, “Iteration of Mobius Transformations
and Attractors on the Real Line,” C o m p u t e r s and Mathematics with
Applications, vol. 33, no. 1-2, pp. 1-12, 1997.
[4] A. Cichocki and R. Unbehauen, Neural networks for optimization and
signal processing, Wiley, 1993.
[5] A. Dumitras and V. Lazarescu, “On Viewing the Transform Performed by
a Hidden Layer in a Feedforward ANN as a Complex Mobius Mapping,” in
Proceedings of International Conference on Neural Networks, 1997,
vol. 2, pp. 1148-1151.
[6] R. Harvey, D. P. Mandic and D. H. Kolonic, “Some Potential Pittfalls with s
to z-plane Mappings,” in Proceedings of the International Conference
on Acoustics, Speech and Signal Processing (ICASSP-2000), 2000,

0-7803-6278-0/00$10.00 ( C ) IEEE 193

 vol. VI, pp. 3530-3533.
[7] A. Jahnsen, B. Schiek and V. Hansen, “Fullwave Analysis of Planar Microwave
Circuits by Integral Equation Methods and Bilinear Transformations,” IEEE
Transactions on Microwave Theory and Techniques, vol. 40, no. 7,
pp. 1581-1584, 1992.
[8] D. H. Johnson and D. E. Dudgeon, Array signal processing: Concepts
and techniques, Prentice-Hall, 1993.
[9] J. Kilian and H. T. Siegelmann, “The Dynamic Universality of Sigmoidal
Neural Networks,” Information and Computation, vol. 128, no. 1, pp. 48-
56, 1996.
[lo] D. P. Mandic and J. A. Chambers, “Global Asymptotic Stability of Nonlinear
Relaxation Equations Realised Through a Recurrent Perceptron,” in Pro-
ceedings of the International Conference on Acoustics, Speech and
Signal Processing (ICASSP-OO), 1999, vol. 2, pp. 1037-1040.
[ll] D. P. Ma.ndic and J. A. Chambers, “A Posteriori Error Learning in Nonlinear
Adaptive Filters,” IEE Proceedings-Vision, Image and Signal Process-
ing, vol. 146, no. 6, pp. 293-296, 1999.
[12] D. P. Mandic and J. A. Chambers, “On Robust Stability of Time-Variant
Discrete-Time Nonlinear Systems with Bounded Parameter Perturbations,”
IEEE Transactions on Circuits and Systems-I: Fundamental Theory
and Applications, vol. 47, no. 2, pp. 185-188, 2000.
[13] D. P. Mandic and J. A. Chambers, “Relations between the A Priori and A Pos-
teriori Errors in Nonlinear Adaptive Neural Filters,” Neural Computation,
vol. 12, no. 6, pp. 1285-1292, 2000.
[14] G. Mantica, “On computing Jacobi matrices associated with recurrent and
Mobius iterated function systems,” Journal of Computational and Ap-
plied Mathematics, vol. 115, pp. 419-431, 2000.
[15] J. H. Mathews and R. W. Howell, Complex Analysis: for Mathematics
and Engineering, Jones and Bartlett Pub. Inc., 3rd edn., 1997.
[16] A. V. Oppenheim, J . R. Buck and R. W. Schafer, Discretetime signal
processing, Prentice Hall, 1999.
[17] P. A. Regalia, S. K. Mitra and P. P. Vaidyanathan, “The Digital All-Pass
Filter: A Versatile Signal Processing Building Block,” Proceedings of the
IEEE, vol. 76, no. 1, pp. 19-37, 1988.
[18] T. E. Rozzi, J . H. C. van Heuven and A. Meyer, “Linear Networks as
Mobius Transformations and Their Invariance Problems,” Proceedings of
the IEEE, vol. 59, no. 5, pp. 802-803, 1971.
[19] K. Shenoi, Digital Signal Processing in Telecommunications, Prentice
Hall, 1995.
[ZO] H. T. Sherief and H. A. Fatmi, “Nonlinear Neural Field Filters,” in Pro-
ceedings of International Joint Conference on Neural Networks,
IJCNNS1, 1991, vol. 3, pp. 1885-1890.
[21] C. S. Williams, Designing digital filters, Prentice-Hall, 1986.
[22] S. Zakeri, “On Critical Points of Proper Holomorphic Maps on the Unit Disc,”
Bulletin of the London Mathematical Society, vol. 30, pp. 62-66, 1996.
[23] E. Zeidler, Nonlinear functional analysis and its applications, vol. 1.
Fixed-point theorems, Springer-Verlag, 1986.

0-7803-6278-0/00$10.00 (C)IEEE 194