Image Compression
* Note: Sub-titles are not captured in Xplore and should not be used
Muhammad Ammar Abbas
Electrical Department
Air University
Islamabad Sector E-9
210203@students.au.edu.pk
Abstract—This project investigates image compression using
low-rank approximation via alternating least squares (ALS)
optimization. It involves developing a QR factorization routine,
implementing the ALS algorithm, and evaluating its performance
on compressing grayscale images. Through experimentation and
analysis, it aims to understand the efficiency and effectiveness of
ALS-based compression compared to SVD methods.
I. I NTRODUCTION
In response to the escalating demand for efficient visual data
management, particularly in the face of storage and bandwidth
constraints, this project delves into the exploration of low-
rank approximation techniques for image compression, with
a particular focus on alternating least squares (ALS) opti-
mization. ALS optimization, by decomposing the Frobenius
norm minimization problem into manageable sub-problems,
seeks to iteratively optimize matrices representing images
to find an optimal low-rank approximation that minimizes
reconstruction error. This approach offers a compelling so-
lution to representing images with fewer parameters, making
it suitable for resource-constrained environments. The project
aims to comprehensively investigate ALS-based image com-
pression, from the development of QR factorization routines
for efficient computation to the evaluation of compression
quality against traditional singular value decomposition (SVD)
methods. Through meticulous experimentation and analysis,
the project seeks to uncover the computational trade-offs and
performance characteristics of ALS optimization compared to
established compression techniques, ultimately paving the way
for the development of more effective and resource-efficient
image compression solutions to meet the evolving needs of
diverse real-world applications.
II. P ROCEDURE
1.Implementation of QR Factorization Routine:
The QR factorization routine takes an n×rn×r matrix BB
with rnrn as input and returns an n×rn×r matrix QQ with or-
thonormal columns and an r×rr×r upper triangular matrix RR.
We implemented the Householder QR factorization algorithm,
which is numerically stable and efficient.
2. Validation of Algorithm:
Identify applicable funding agency here. If none, delete this.
Aal e Fatima Shahzaib Anwer
Electrical Department Electrical Department
Air University Air University
Islamabad Sector E-9 Islamabad Sector E-9
210752@students.au.edu.pk 210720@students.au.edu.pk
To ensure our algorithm behaves as expected, we tested it
with various input matrices and verified that it produces valid
QR factorizations. We tested the algorithm’s computational
scaling by analyzing its runtime for matrices of increasing
size.
3. Choice of QR Factorization Variant:
We opted for the Householder QR factorization method
due to its numerical stability and efficiency. The Householder
algorithm exhibits O(n2r)O(n2r) computational complexity,
making it suitable for large-scale applications.
4.Testing and Validation:
We tested our implementation by comparing its outputs
against known QR factorizations. To demonstrate the expected
scaling, we measured the algorithm’s runtime for increasing
matrix sizes and analyzed its computational complexity.
.
III. A PPROACH
1.QR Factorization Implementation:
Implement a function that computes the QR factorization
of a given matrix. Test your implementation to ensure it
provides correct outputs for various matrices and verify its
computational scaling.
2. Algorithm 1 Implementation:
Implement Algorithm 1, the alternating least squares
method, using the QR factorization routine. Break down
the Frobenius norm into solvable sub-problems and explain
this process in your report. Optimize the implementation by
considering efficient block operations.
3. Image Compression:
Load the provided image file, ”einstein.jpg”. Compute the
best rank 75 approximation of the image using your ALS
implementation. Compare the result with the best rank 75
image from the SVD using built-in routines like those available
in Matlab or Julia. Analyze and compare the qualitative and
quantitative differences between the ALS and SVD compres-
sion methods.
4. Compression Rate and Runtimes:
Report the compression rate and run times of your ALS
function for fitting least squares models on the image data at
ranks r=45,55,65,75,85r=45,55,65,75,85.
5. Different Starting Points:
 Choose an rr value from the provided set and start your
algorithm from different initial points WW and ZZ. Compare
the objective value J(W,Z)J(W,Z) and the resulting matrices
WW and ZZ. Explain any differences or similarities observed
in the objective value and resulting matrices
6. Method of QR Factorization:
The QR factorization done by using Householder reflec-
tions. The function takes an input matrix B of size n x r and
returns the orthogonal matrix Q and the upper triangular matrix
R. The Householder reflections are computed iteratively for
each column of B.
IV. I MPLEMENTATION M ETHOD
Householder reflections are a powerful method for comput-
ing QR factorization of a matrix. This method systematically
transforms a given matrix into an upper triangular form using
a series of orthogonal transformations called Householder re-
flections. Here’s an explanation of the Householder reflections
method for QR factorization:
1. Householder Transformation:
A Householder transformation, also known as a House-
holder reflection or a Householder matrix, is a unitary matrix
that reflects vectors through hyperplanes. Given a vector v in
R n , the Householder transformation H is defined as:
= 2
2 H=I2 v 2
vv T
where I is the identity matrix, vv T is the outer product
of v, and v is the norm of v.
2. Householder QR Factorization:
The Householder QR factorization algorithm starts with the
original matrix A and iteratively transforms it into an upper
triangular matrix R by applying a series of Householder
transformations. These transformations are chosen to zero out
specific entries below the diagonal of A, thereby gradually
transforming it into R.
3. Algorithm Steps:
Initialization: Start with = A=B, the original matrix.
Iterative Transformation: For each column j of A, compute
the Householder reflection H j that zeros out the sub-
diagonal entries of the jth column below the diagonal. Update
Matrix A: Apply the Householder transformation to A from
the left: : = A:=H j A. This step effectively eliminates
sub-diagonal entries below the jth column. Extract R: The
upper triangular matrix R is obtained from the upper-left
corner of the transformed matrix A. Build Q: The orthogonal
matrix Q is constructed as the product of all Householder
transformations: = 1 1 Q=H n H n1 H 1 .
4. Result:
At the end of the process, matrix A is transformed into an
upper triangular matrix R, and the product of all Householder
reflections yields an orthogonal matrix Q such that = A=QR,
representing the QR factorization of the original matrix A.
5.Implementation Considerations:
The choice of vectors v for the Householder transforma-
tions is crucial for numerical stability and efficiency. Care
should be taken to optimize the computation of Householder
transformations to minimize numerical errors and maximize
performance.
V. T EST T HE R ESULT
Testing the implementation of Householder reflections for
QR factorization involves verifying that the algorithm pro-
duces correct results for a variety of input matrices while also
assessing its performance in terms of numerical stability and
efficiency. Here’s how you can test the design:
1. Correctness Testing:
Test with Identity Matrix: Start by testing the implemen-
tation with a simple identity matrix to verify that Q is
indeed orthogonal and R is an upper triangular matrix. Test
with Random Matrices: Generate random input matrices of
various sizes and structures. Compute the QR factorization
using your implementation and compare the product QR
to the original matrix to ensure it equals the original matrix
within a certain tolerance. Test with Known Solutions: Use
matrices with known QR factorizations to validate the results
produced by your implementation. Compare the computed Q
and R matrices to the expected values. Test for Numerical
Stability: Introduce matrices with varying condition numbers
to assess the numerical stability of the algorithm. Check if
small changes in the input matrix result in small changes in
the computed QR factorization.
2. Performance Testing:
Execution Time Measurement: Measure the execution time
of the algorithm for matrices of different sizes. Compare the
performance with alternative QR factorization methods.
Memory Usage: Monitor the memory usage of the algo-
rithm, especially for large matrices, to ensure efficient memory
management. 3. Edge Cases Testing:
Empty Matrix Handling: Test the implementation with
empty matrices to ensure proper handling of edge cases.
Square vs. Rectangular Matrices: Test the implementation with
both square and rectangular matrices to ensure it handles
different input shapes correctly.
4. Integration Testing: Integration with Other Algorithms:
Integrate the QR factorization algorithm into larger numerical
algorithms, such as least squares regression or eigenvalue
computation. Verify that the integrated algorithms produce
correct results
VI. D ISCUSSION ON RESULTS
An important result of implementing the Householder re-
flections method for QR factorization is its ability to efficiently
decompose a given matrix into its orthogonal and upper trian-
gular components. This decomposition has several significant
implications and applications in various areas of mathematics,
engineering, and computational science. Let’s discuss some of
the key aspects and implications of this result:
1. Numerical Stability:
The Householder reflections method offers superior numer-
ical stability compared to other QR factorization methods,
such as Gram-Schmidt orthogonalization. By systematically
eliminating sub-diagonal entries using orthogonal transfor- an alternative approach, particularly useful in scenarios in-
mations, Householder reflections minimize numerical errors volving sparse matrices or selective modification of elements.
and floating-point round-off effects, making it suitable for Although it may require more operations compared to House-
solving ill-conditioned systems of equations and numerical holder reflections, especially for dense matrices, Givens rota-
simulations. tions offer advantages in specific contexts. Overall, while each
2. Orthogonality and Orthonormality: method has its strengths and weaknesses, the Householder
The orthogonal matrix Q produced by Householder re- reflections method emerges as the preferred choice for QR
flections has orthonormal columns, meaning that = Q T factorization in most practical scenarios due to its numerical
Q=I. This property preserves the lengths and angles between stability, efficiency, and versatility.
vectors, making it useful for preserving geometrical properties
and performing orthogonal transformations in applications
such as rotation matrices, coordinate transformations, and
signal processing.
3. Applications in Least Squares and Regression:
QR factorization is a key component in solving least squares
problems, where it is used to find the least squares solution
to overdetermined systems of linear equations. By decompos-
ing the design matrix into orthogonal and upper triangular
components, Householder QR factorization enables efficient
and numerically stable computation of least squares solutions,
which has applications in regression analysis, curve fitting, and
optimization.
4. Eigenvalue Computation:
QR factorization plays a crucial role in computing eigenval-
ues and eigenvectors of a matrix. By transforming the original
matrix into Hessenberg form using Householder reflections,
it becomes easier to compute eigenvalues and eigenvectors
accurately and efficiently. This has applications in spectral
analysis, control theory, and structural mechanics.
5. Signal Processing and Compression:
In signal processing and data compression, Householder
QR factorization can be used for orthogonalizing signals and
reducing redundancy. Orthogonal transformations performed
by Householder reflections can decorrelate signals, extract
principal components, and compress data efficiently while
preserving important information.

VII. C ONCLUSION
In conclusion, the classical Gram-Schmidt algorithm,
Householder reflections method, and Givens rotations method
each offer unique approaches to QR factorization, with distinct
implications for numerical stability, efficiency, and versatility
in various applications. The classical Gram-Schmidt algorithm,
while straightforward, suffers from numerical instability due
to accumulation of rounding errors, limiting its applicability
in scenarios where numerical precision is crucial. Conversely,
the Householder reflections method provides a robust and
efficient solution by systematically transforming matrices into
upper triangular form using orthogonal transformations. This
method minimizes numerical errors and floating-point round-
off effects, making it well-suited for solving ill-conditioned
systems of equations and numerical simulations. Furthermore,
the resulting orthogonal matrix with orthonormal columns fa-
cilitates applications in least squares, eigenvalue computation,
and signal processing. The Givens rotations method presents