Professional Documents
Culture Documents
p250 Fredman
p250 Fredman
on-hne complexity
1. Introduction
Let A ( j ) , 1 _< j _< N, be a n a r r a y o f fixed length N. C o n s i d e r the two p r o g r a m s ,
Update(j, x)
A ( j ) := A ( j ) + x
Retrieve(k)
O u t p u t ~ = 1 A (i)
(1)
For i := j until N do
S(i) :-- S(i) + x
Retrieve(k)
Output S(k)
(2)
This work was supported m part by the National Soence Foundation under Grant MCS 76-08543
Author's address' Department of Electrical Engineering and Computer Sciences, University of California
at San Diego, La Jolla, CA 92093
Permission to copy without fee all or part of this material is granted provided that the copies are not made
or distributed for direct commercial advantage, the ACM copyright notice and the title of the publication
and its date appear, and notice is given that copying is by permission of the Association for Computing
Machinery To copy otherwise, or to republish, requires a fee and/or specific permission
1982 ACM 0004-5411/82/0100-0250 $00.75
Journal of the Association for Computing Machinery, Vol 29, No I, January 1982, pp 250-260
251
2. Algebraic Complexity
In this section we assume that the array A (j), 1 _<j_< N, stores values in an arbitrarily
specified commutative group G. We consider a class of straight-line programs which
are constrained to function correctly independently of the choice of G.
The three approaches outlined in the introduction have the following features in
common. A set of variables is maintained (organized as an array in (1) and (2),
identified with tree nodes in the third approach). Retrieve(k) is performed by
evaluating a linear function of these variables, and Update(j, x) is performed by
incrementing these variables by amounts which depend linearly on x. Our algebraic
model comprises all programs having these features.
Thus our algebraic model consists of triples, (V, R, U), where V = {vl, . . . , vm) is
a set of variables, R is an N x m matrix, R = (ak3, and U is an m N matrix,
U = 680) (m can vary). Associated with a triple (V, R, U) are the programs,
Update(j, x)
for l := 1 until m do
v~ := vt + Box
Retrieve(k)
output ~l% C~klVz
(3)
252
MICHAEL
L. F R E D M A N
C~kzVl
and
lsl
l~1
{0
z2-zl=
[]
I
1
I
2
I
3
I
...
I
N
(b) Horizontal, labeled line segments above and parallel to the line in (a), whose
endpoints appear directly above interval boundary points of the line in (a), for
example,
t
I
1
I
2
I
3
I
4
I
5
253
N(i, j)
if j = l ,
J(i N
1, j) + U(i, j +N(i-l,j-l)+l
if
1)
i = 1,
if i > 1, j >
1.
Our diagrams are interpreted as follows: The line segments uniquely correspond
to variables. The values of these variables will be maintained so that at a given
instant, the variable associated with the line segment passing over the intervals
numbered g i + 1. . . . . j will have value A (i) + . . . + A(j). The label of a line
segment will also be the name of its associated variable. Now consider the following
dmgram:
r
s
I
t
I
v
I
w
I
.v
1
a
I
b
I
d
c
I
8
I
9
e
I
10
(4)
I
11
I
12
z := z + x.
Y,A(i),
I<_j<_N,
can be directly "translated" into a data structure which computes the pamal sums,
re+J--1
A(i),
1 <_ j_< N,
(5)
t=m
as well as "translated and reflected" into a data structure which computes the partial
sums,
m
A (i),
1 <_j <_ N.
(6)
t=m-J+l
-....-.4
The notation ~
refers to the complex of hne segments comprising a (possibly)
translated (u, r)-system such as described in (5); it would extend precisely over the
intervals m, m + 1. . . . . m + N 1, where N = N(u, r). The notation
analogously refers to the line segments of a translated and reflected (u, r)-system as
in (6).
254
MICHAEL
L. FREDMAN
Su
$3
S2
al
a2
a3
ar
Sl
I
1
(a)
(b)
II
(u-- l,r)
...
m~
i
. .
(u,r-
1)
I
?/12
..
N = N(u,
r)
ml= N ( u - l,r)
m2= N ( u - l,r)+ N ( u - l , r - 1)+ 1
FIG. 2
1 variables in [(u -
1, r -
sum
zf r = l ,
N(u-
tf u = 1,
f u,r > 1.
N(u, r) =
l,O + N(u,r-
1) + N ( u -
1, r -
1)+ 1
255
. . . . .
PROOF. First we define some notation. Let [1, j] denote (1, 2 . . . . . j}. G i v e n a
permutation r over [1, N ] and a set S C [1, N ] , we let or(S) denote { J l j ffi ~r(0 for
some i E S}. I f S C__[1, N ] a n d j ~ S, we let r a n k s ( j ) denote the n u m b e r o f elements
in S which are _<j.
N o w let s = A1 tO . . . tO AN to B~ tO . . . to BN = {al . . . . . am}, say. G i v e n
a ~ ~ , we let Ta = {ila E A, and a ~ B,}, X~ = {ila ~ A, and a ~ B,}, and
Y~ = {ila f~ A, and a ~ B,}. T h e sets T~, X~, and Y~ are disjoint. W e let Sa =
(7)
a~_zag
O u r strategy is to utilize the sets T~, X~, and Y~, ranging over a ~ .~, to uniquely
encode each of the N! permutations r over [1, N]. G i v e n r and a ~ ~, we define
T~,~ = {tJl = rank~(s~)(j)
Xa,. = {i I i = rank.ts,~(j)
Ya., = {i] i = rank~cso)(j)
for some
for some
for some
j E ~r(T~)),
j E ~r(X~)},
j E ~r(Y~)).
256
MICHAEL
L. FREDMAN
Because Ta, X,, and Y, are disjoint, so are r(T& n(X,), and n( Ya). Moreover, so are
T,,,, Xa,,,, and Y,,,. Further!:iore, Ta,n U X,,,, U Ya,, = [ 1, ] S, 11.We let P,,, denote the
ordered triple (T,,,, X,,,,, Y,,,,). Thus Pa,,, defines a partition of [ 1, 1S, I] into three
blocks, one of size ] T, 1, one of size I X, I, and one of size I Y, I. Now let L, denote the
vector (Pa,,,, Paz,=,. . . , P,,,,,). (RecalI that & = {al, . . . , a,) .) The jth component of
L, can assume at most
(8)
N! I 8.
Lemma 3 (below) combined with (8) and (9) gives
yoo 2 N!
(1 + JZ)Cd72+w+l
(9
(10)
PROOF.
Choose z so that K(y) = n;i(y) for ally < z and n:(z) # ~Gl(z). Let
i = r;*(z) and j = ril(z). Since A, tl B, Z A, n B,, there exists a E &such that, say,
a E A, n B, and a $! A, n B,. (The symmetrical case is treated the same way.) We
show that P,,,, # P,,,. Because a E A, and a E B,, both i, j E S,. Since z = ri(i) =
qz(j), z E n@,) and z E ~@,4. Moreover, rank,,(saj(z) = rank,,csJz) because for
each y < z, y E ?ri(&) if and only if y E 7rz(S,), in accordance with our choice of z.
Because a E A, tl B, and a $ AI tl B,, i and j do not appear in the same set, Ta, X,,
or Y,. Let r = rank,,(s,,(z) = rank+,,(z).
It follows that r does not appear in
corresponding blocks of Pa,n, and Pa,flz.In other words, Pa,=, # Pa,,,, as asserted. 0
LEMMA
( 1
;,+; + k 5 (1 + &y+2J+k.
9
PROOF.
(By induction of i + j + k.) If i, j, or k = 0, then (:,-,i)
coefficient, and so,
5
2r+J+k
(1 +
,/$+J+k
(1 +
4r+2J+k.
If i, j, k > 0, then
(1 +
,J$r+2J+k--l
+ (1 +
+ (1 + A)L+2J+k--1
&)r+2J+k-2
(by hypothests)
is a binomial
257
(u-l,r)
(u,r-1)
I
]
I
I
N
(IA, I +
(11)
This follows because the sets Xa (see proof of Theorem 3) are always empty, and
therefore the partiuons Pa,, are in actuality partitions into two blocks instead of three
blocks. In place of (10), the stronger equality,
2E,~o,(I7",1+1Y,I)> N!
(12)
results. Inequahty (11) follows from (12).
258
M I C H A E L L. F R E D M A N
Now define A, = {j]a,j = 1), 1 <_ i <_ N - 1, where R = (ast), and define B~ =
{llfl,j+a -- 1}, 1 _<j_< N - 1, where U = (fist). I f R U = ,IN, then
IA, n B~I --
{0
if i<_j,
if i > j .
259
among these N vectors (no two of these vectors, when provided as input, define the same
path through T) and has hetght h. Then
(Observe that when k = oo, this gives the classical bound, 2 h > S . )
PROOF. We show how to construct a one-to-one mapping from the vectors v~into
vectors v,* having Hamming weight at most k and dimension h. Since the possible
number of such vectors v,* is (~) + . . . + (h), Lemma 4 is an immediate consequence.
Let nl, n2. . . . , nt, I <_ h, be the nodes on the path through T defined by input v,, in
descending order beginning from the root. Let cj be the vector component number
examined by node nk, 1 _<j _< /. We define v,* to be the binary vector of dimension
h, having a one in itsjth component if and only if v, has a one in its c~th component.
To see that v~* # v,*, when t # i', let nj be the node at which the paths defined by v,
and v,, diverge. Then v, and v,, differ in the cjth component, and so v~* and v,*, differ
in the j t h component. []
PROOF or THEOREM 5. To simplify the exposition, we assume initially that the
memory configuration Co with all bits set to zero represents the situation where the
maintained array A(i) has a zero stored in each location. We also identify each
memory configuration C with the binary vector whosejth component is given by the
contents, in configuration C, of the bit with addressj. Let Cj, 1 _<j _< N, denote the
memory configuration resulting from an execution of the program Update(j), starting
from Co. We construct, in the manner described below, a binary search tree which
distinguishes among C1, C2. . . . . CN.
Let TN be a mimmal height ([log2N]) binary tree with N leaves numbered (left-toright) from 1 to N. Also label the internal nodes of TN from 1 to N - 1 in symmetric
(left-to-right) order. Let wj, 1 _<j _< N, denote the binary vector of dimension N - 1
with zeros in the firstj - 1 components and ones in the remaining N - j components.
The tree TN can be regarded as a search tree which identifies the vectors wj as
follows. An internal node with label i tests the ith component. Control passes to the
left son if the value of the ith component is 1, otherwise control passes to the right
son. Now let T' be the following algorithm which identifies the vectors C1, C ~ , . . . ,
CN. The algorithm T' is obtained by modifying the algorithm T2vso that when control
is centered at an internal node with label i, the branching choice is determined by the
output produced by the program Retrieve(t) when it is executed on input Cj. (Observe
that the ith component of wj has the same value as the output returned by Retrieve(i)
executed from configuration Cj. Therefore the algorithm T' identifies the Cj vectors,
since TN identifies the wj vectors.) The algorithm T' can be expanded into a binary
search tree which directly examines the components of the C~ vectors, namely, by
utilizing the internal node portions of the binary trees corresponding to the Retrieve(t)
programs. Let T denote this expanded search tree. If the heights of the Retrieve(t)
trees do not exceed r, then the height h of T does not exceed r[log2N]. More strictly,
we can choose r to be the maximum length of those paths defined by the input
vectors Cj, 1 <_ j <_ N, through each of the Retrieve(i) program trees. Now let k
denote the maximum Hamming weight of the C s vectors, 1 _<j _< N. (Observe that
the Hamming weight of Cj is a lower bound on the cost of executing Update(j) from
configuration Co.) By Lemma 4 we have that
N_(h)+
... +(h)_<(k+
l ) h k < - 2 k r k ( l + l o g 2 N ) k,
260
MICHAEL L. FREDMAN
log2N
2 log21o~N '
r _>
log2N
2 log21og2N"
(13)
or
(14)
If (13) holds, then for somej the program Update(j) takes time _>log2N/(2 log21og2N)
when executed starting from Co. If (14) holds, then for some pair (j,/) the program
sequence Update(j) Retrieve(/) takes time >_log2N/(21og21og2N) when executed
starting from Co. The reader should be able to see that the above argument readily
generalizes and allows us to conclude the following. Given an arbitrary memory
configuration C corresponding to a storage state of the maintained array A(I), there
exists a pair (j, 0, depending on C, such that the program sequence Update(j)
Retrieve(/) takes time >_(log2N)/(2 log21og2N) to execute starting from configuration
C. Theorem 5 now follows by induction on m. []
We conjecture that the lower bound in Theorem 5 can be strengthened to read
m log~.83N for sufficiently large m. Some evidence supporting this conjecture is the
following, which we state without proof. If the N programs Update(j), 1 _<j _ N, are
implementations within the information-theoretic model of programs having the
form in (3), then from the standpoint of choosing Retrieve programs having minimal
complexity, one cannot do better than choosing Retrieve programs having the form
in (3). Moreover, the dual statement, obtained by interchanging the occurrences of
Update and Retrieve in the above statement, is likewise true.
4. A n Open Question