Special Lecture on Vector Norms

By Professor Wu
A special query function

We showed that a function/query is

just a way to extract information from
a vector or a matrix.

This function is called the

Now we will learn a particularly Norm

special one of these query functions.

In a sentence, we would
say that the norm of a
vector x is …..

This would be equivalent

to Norm(x)
 The norm function is special…

We can write the norm

function just like any This is how norms are written
other function. But the norm function is so special, it
has its own notation.

Notice the double lines

Most of us have seen a

similar looking function
|x| vs ||x||

The fact that a norm ||x|| and absolute

value |x| look similar is not an accident.
The absolute value function
was also given its own notation
Absolute value |x| is actually just norm
in 1 dimension.
How is |x| just ||x|| in 1 dimension???

Let’s interpret them as length

Notice that the absolute value function ignores the

direction and gives us the length ?

So we can conclude that the abs(x) or |x| is a function that

extracts the length information from a 1 dimensional vector.

But how then do we extract the length information from a

higher dimensional vector?

The answer is the Norm !!!

If you have a vector and want to know its size, length, or

magnitude, Norm is the function that extracts this information.
So many types of norms

There are many types of norms

The norm symbol is always the same.
used by data scientists. The most
common ones are

We use the subscript to tell which

norm it is.

You might be wondering

Isn’t distance just a length from point

A to B? Why do we need so many
norms???
 Distance from A to B is not always a straight line

Image two possible cases, In situation 3

start where you are trying to go
from start to destination.
1. You are a bird
1 2. You are a pedestrian.
You want to go on a nice
long walk the the numbers
represents the distance of
the 3 paths.

End
1

If you are a bird, the distance If you are a pedestrian, the In this case, L-infinity norm
you travel would be the L1 norm is more accurate makes more sense. You just
Euclidean norm/L2 norm. pick the largest value.

The concept of a norm completely depends on the meaning of the vector.

 The most commonly used norm

Example:
The Lp norm is the most used family of norms.
Depending on the value of p, you get a different norm.

Equation for Lp norm

If p = 1, we get L1

If p = 2, we get L2

If p = infinity
 The norm that is not a norm ||x|| 0 / cardinality

You may see in the literature norm 0 or

This is not a norm b/c it doesn’t satisfy the definition.

It is still a useful query/function for communication purpose.

Essentially norm 0 is the cardinality of the vector !!

( the number of elements that are not 0 )

The norm 0 of this vector is 3

because out of 5 elements, 3 of
them are not 0.
 Alternative names for these norms

L1 norm, Manhattan norm, taxicab norm,

L2 norm, Euclidean norm

L-infinity norm, uniform norm, maximum norm, supremum norm

 Questions

We have only seen the most common norms, but how Example:
do we tell if a function is a norm?

We check for 3 conditions

1. Triangle inequality :

2. Scaling:

3. Positive Definite:

Note: Triangle inequality automatically implies a reverse

triangle inequality.
 Double checking the definition of a norm

Double Check:

Triangle inequality Scaling Positive Definite

True
True

True
Triangle
 Quick Review

Today we learned the norm To be a norm, the function must

functions that extract various satisfy these 3 conditions.
size information from a vector. Each norm is useful depending
on the meaning of the vector.
 Finding Norms with Python

>>> import numpy as np

>>> v = np.array([[4],[3]])

>>> np.linalg.norm(v, ord=1)

7.0

>>> np.linalg.norm(v, ord=2)

5.0

>>> np.linalg.norm(v, ord=np.inf)

4.0
 Large data sets are commonly stored in csv files. For example,
Loading data given a file test.csv, when you open the file, you’ll see

1,2,3
0,0,1
3,8,4
1,1,1

This is basically a 4 x 3 matrix.

You can read this file into a variable X with the following code

>>> import numpy as np

>>> from numpy import genfromtxt
>>> X = genfromtxt('test.csv', delimiter=',')
>>> X Can be
array([ [1., 2., 3.], %d or %f or %.3f
[0., 0., 1.],
[3., 8., 4.],
[1., 1., 1.]] )

np.savetxt('test2.csv', Y, delimiter=',', fmt='%d')