Professional Documents
Culture Documents
A Course On Integral Equations With Numerical Analysis: Tofigh Allahviranloo Armin Esfandiari
A Course On Integral Equations With Numerical Analysis: Tofigh Allahviranloo Armin Esfandiari
A Course On Integral Equations With Numerical Analysis: Tofigh Allahviranloo Armin Esfandiari
Tofigh Allahviranloo
Armin Esfandiari
A Course
on Integral
Equations
with Numerical
Analysis
Advanced Numerical Analysis
Mathematical Engineering
Series Editors
Jörg Schröder, Institute of Mechanics, University of Duisburg-Essen, Essen,
Germany
Bernhard Weigand, Institute of Aerospace Thermodynamics, University of
Stuttgart, Stuttgart, Germany
Today, the development of high-tech systems is unthinkable without mathematical
modeling and analysis of system behavior. As such, many fields in the modern
engineering sciences (e.g. control engineering, communications engineering,
mechanical engineering, and robotics) call for sophisticated mathematical methods
in order to solve the tasks at hand.
The series Mathematical Engineering presents new or heretofore little-known
methods to support engineers in finding suitable answers to their questions,
presenting those methods in such manner as to make them ideally comprehensible
and applicable in practice.
Therefore, the primary focus is—without neglecting mathematical accuracy—on
comprehensibility and real-world applicability.
To submit a proposal or request further information, please use the PDF Proposal
Form or contact directly: Dr. Thomas Ditzinger (thomas.ditzinger@springer.com)
Indexed by SCOPUS, zbMATH, SCImago.
A Course on Integral
Equations with Numerical
Analysis
Advanced Numerical Analysis
Tofigh Allahviranloo Armin Esfandiari
Faculty of Engineering and Natural Faculty of Engineering and Natural
Sciences Sciences
Bahcesehir University Bahcesehir University
Istanbul, Turkey Istanbul, Turkey
© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature
Switzerland AG 2022
This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether
the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse
of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and
transmission or information storage and retrieval, electronic adaptation, computer software, or by similar
or dissimilar methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this book
are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or
the editors give a warranty, expressed or implied, with respect to the material contained herein or for any
errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional
claims in published maps and institutional affiliations.
This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Preface
As the title of the book suggests, the numerical analysis subjects’ matter are the impor-
tant tools of the book topic. Because numerical errors and methods have important
roles in solving integral equations. Therefore, all needed topics including a brief
description of interpolation are explained in the book.
The integral equations have many applications in the engineering, medical and
economic sciences, so the present book contains new and useful materials about
interval computations including interval interpolations that are going to be used in
interval integral equations.
The concepts of integral equations are going to be discussed in two directions,
analytical concepts and numerical solutions which both are necessary for these kinds
of dynamic systems. The differences between this book with the others are a full
discussion of error topics and also using interval interpolations concepts to obtain
interval integral equations.
All researchers and students in the field of mathematical, computer and also
engineering sciences can benefit the subjects of the book.
v
Contents
vii
viii Contents
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
Chapter 1
Introduction to Numerical Analysis
1.1 Introduction
Bases on the discussed numerical methods and considering other chapters of the
book, we need to explain the error analysis first. In addition to explore the numerical
integration methods, the interpolation of integrand function is needed, and this is one
of the reasons to discuss about interpolation methods.
other. Currently, due to the advances in technology and the construction of advanced
satellites and long-range air-to-air missiles and missiles with nuclear warheads, an
approximate estimate of the target with the lowest error rate and the calculations of
missile or satellite launch with the least amount of error is important. This is because
the missile is trying to hit a specified target over a distance of, for example, thou-
sands of kilometers, which may also be approximate. However, how to launch the
missile, initial speed, initial acceleration, traveled distance, obstacles in the path of
the missile such as air resistance, winds blowing from lateral directions, etc., and how
to hit the target, all are factors that required to be considered, and obviously none of
these factors can be accessed accurately and without an error. Therefore, taking into
account these factors and problems, hitting the target with a missile should be done
with an error of, for example, a maximum of 0.01. Obviously, all models related to
this process are in the form of mathematical models, for example, the differential
equations with the initial conditions, the integral equations, the differential equations
with partial derivatives, the calculations of integral series and so on. So we need to
examine the errors of such models and estimate the upper and lower bounds of such
errors. In this regard, some problems about error analysis are presented.
Suppose that Y = φ(X ), where φ is a combination of all the steps of the algorithm.
For this purpose, we define:
φ : D → Rm
i.e.,
⎡ ⎤ ⎡ ⎤
y1 ϕ1 (x1 , ..., xn )
⎢ ⎥ ⎢ ⎥
Y = ⎣ ... ⎦ = ⎣ ..
. ⎦
ym ϕm (x1 , ..., xn )
To calculate φ in each algorithm, we have the ordered series of ϕ (i) operators with
the sum equal to φ, so that the output of one operator will be the input of the next
operator, and finally the output of the last operator will be Y .
If in ith step of the algorithm, the vector X (i) has n i inputs for the operator ϕ (i) ,
then we have:
So that
ϕ (i) X (i) = X (i+1)
φ : D → Rm , D ⊆ Rn
⎡ ⎤
ϕ1 (x1 , ..., xn )
⎢ .. ⎥
φ(X ) = ⎣ . ⎦
ϕm (x1 , ..., xn )
We assume that elements of φ(X ), i.e., ϕi s, have a continuous first derivative and
X̃ is an approximation of x. We know that absolute error and relative error of X̃ are
defined as follows:
X̃ = X̃ − X (1.1)
X̃ − X
ε X̃ = (1.2)
X
n
∂ϕi (X )
Ỹ ≤
∂ x
.x̃ j (1.3)
j=1 j
n
∂ϕi (X )
x j
ε ȳi ≤
∂ x
. ϕ (X ) .εx̄ j
j=1 j i
∂ϕi (X )
In relation (1.4), Dφ(X ) is the Jacobian matrix. The factors ∂x j
in (1.3) are
∂ϕi (X ) x
called sensitivity numbers and also the factors . ϕi (Xj )
in (1.5) are called condition
∂x j
numbers. If in a problem, all condition numbers are small, we say that the problem
is well-established.
The following problem describes the analysis of the series calculation error with
a computational tool. Due to the widespread application of series, it is important to
explain such problems.
1.2.1.1 Problem
n
Calculating i=1 ai results in an arbitrary large relative error. If all terms of ai have
the same sign, then the relative error is bounded, find a bound for this error, ignoring
the higher-order terms.
n
Solution: Suppose that y = i=1 ai and ỹ is an approximation of it. According to
the relative error formula:
n
x j
∂φ(x)
ε ỹ ≤ . .εx
j=1
φ(x)
∂ x j
j
we have:
n
aj
ε ỹ ≤ .|1|.εa j
j=1
a1 + · · · + an
a1 a2 an
= .εa + .εa + · · · + .εa
a1 + · · · + an 1 a1 + · · · + an 2 a1 + · · · + an n
So:
a1
an
ε ỹ ≤
.
εa
+ · · · +
.
εa
a1 + · · · + an 1
a1 + · · · + an
n
1.2.1.2 Problem
Consider a quadratic equation. If b > 0 and b2 ab, how should the root of the
equation be calculated to minimize the calculation error?
Solution: The roots of the equation are:
√ √
−b + b2 − ac −b − b2 − ac
x1 = , x2 =
a a
√
Since b > 0 and b2 ab, so the value of b2 − ac is very close to b, and
therefore, in the nominator of x1 , we will have a subtraction of two close numbers,
so there is a big error for calculating x1 , but in calculating x 2 , there is not. So, first
we obtain x2 and then, we also calculate x1 as follows:
√ √
−b + b2 − ac −b − b2 − ac
x1 = . √
a −b − b2 − ac
b2 − b2 − ac
=
a.x2 .a
c
=
ax2
1.2.1.3 Definition
If x̃ is an approximation of x, then
e(x̃) = |x − x̃|
1.2.1.4 Example
It can be observed that the larger the n, the smaller the n1 , and as a result, the
an gets closer to one. If we want the error of an to be smaller than, for example,
10−3 , it is enough to write:
1
< 10−3
n
Therefore, n > 103 . The first n that holds in the recent inequality is 1001,
for which:
1002
an = = 1.000999
1001
√
2. We know that 1.141 is an approximation of 2. What is the absolute error of
1.41? √
Obviously, if we write the decimal expansion of 2 using a calculator, i.e.,
√
2 = 1.414213562(9D)
then
e(1.41) = 0.0042135562000
2.66 ≤ l ≤ 2.74
From the above examples, it follows that what determines the accuracy of an
approximation is the error per unit of quantity; the smaller the error, the accurate the
approximation. Therefore, we have the following definition.
|x − x̃|
δ(x̃) =
|x|
In most numerical analysis problems, there is not a real number x. For this purpose,
an upper bound can be obtained for δ(x̃) without the existence of x.
1.2.1.6 Problem
1.2.1.7 Theorem
|x − x̃| ≤ ex̃
1.2.1.8 Remark
Practically, the right side of the above inequality is non-computable, and therefore,
assuming that ex̃ is negligible in comparison with |x̃|, the relative error is defined as
follows.
ex̃
δ(x̃)
(1.6)
|x̃|
1.2.1.9 Example
√
Suppose x̃ = 1.41 and x = 2; The relative error of x̃ is as follows
√
2 − 1.41
δ(x̃) = √ = 0.002979438000
2
According to what is stated, a list of the error sources is presented in the following.
(A) The error of the model. This error includes omissions, ignorance and
simplifications to determine the mathematical model of the problem.
(B) The error of the data. This error occurs when measuring and estimating
problem assumptions.
(C) The error of the number representation. Decimal or binary representation
of most numbers with finite figures is not possible. Therefore, selecting a finite
number of expansion digits of a number causes this error.
(D) The error of the arithmetic operation. The result of some operations on two
factors has an infinite number of digits, and selecting a finite number of these
digits causes this error.
(E) The error of the method. Numerical methods are generally iterative and
give an approximate of the exact answer. The accuracy of this approximation
depends on the type of method and the stopping step.
Among the five error sources mentioned, the error of the model and the error of
the data depend on the type of problem, and the people who determine the model of
problem in different disciplines are responsible for them. But the next three errors
are related to the numerical analysis.
The problem (Sect. 1.2.1.7) is in some way, the same as the theorem (Sect. 1.2.1.1)
but is examined from another perspective. Therefore, it is necessary to introduce the
definitions and concepts related to floating point. First, we define the floating point.
1.2.2.1 Note
For each computational device, a set of real numbers with finite digits is defined. This
set of numbers is called the set of machine numbers and is denoted by the symbol A.
1.2.2.2 Definition
Suppose that x ∈
/ A, r d(x) ∈ A is x approximation if it satisfies the following
inequality:
If the computational device has t digits, i.e., the number of Mantis significant
digits of machine number is maximum of t digits, then r d(x) will be defined as
follows:
1.2.2.3 Definition
Therefore, it can be said that floating point arithmetic operations do not satisfy
the ordinary arithmetic rules such as having a neutral element, associativity and
distributivity.
1.2.2.4 Note
Suppose E is an algebraic term derived from a floating point calculus. In this case,
f l(E) is called the value of the term E in the computational device.
The following theorem shows that the error in the truncating method is almost
twice the error in the rounding method.
1.2.2.5 Remark
where
1 r ounding
l=
2 runcating
According to the theorem (Sect. 1.2.2.5), computational devices use the rounding
method to store numbers, in which case:
|x − f l(x)|
≤ 5 × 10−t = eps
|x|
as a result:
It can be easily shown that performing floating point operations on numbers will
cause the error to propagate and grow, which we will examine in the next section.
Now, we are going to solve some important and applied problems related to the
concept of floating point.
1.2.2.6 Problem
n
xi = x1 + x2 + · · · + xn
i=1
s1 = x 1
sr = f l(sr −1 + xr )
= (sr −1 + xr )(1 + εr ), r = 2, ..., n
sn = l(sn−1 + xn )(1 + εn )
= (sr −2 + xn−1 )(1 + εn−1 ) + xn (1 + εn )
..
.
12 1 Introduction to Numerical Analysis
= x1 (1 + η1 ) + · · · + xn (1 + ηn )
where
So:
(1 − eps)n−r +1 ≤ 1 + ηr ≤ (1 + eps)n−r +1
As a result:
n n
n xi ηi
fl xi = xi 1 + i=1
n
i=1 i=1 i=1 x i
1.2.2.7 Problem
Solution: We know:
δ can be defined such that it satisfies the property of ε. For this purpose, it is sufficient
to consider δ such that |δ| ≤ 5 × 10−t .
−ε
We put δ = 1+ε , then:
1.2.2.8 Problem
1.2.2.9 Problem
Suppose | f l(ab) − ab| ≤ |ab|β −t+1 . Calculate the upper bound of the following
equation:
f l( f l(ab).c) − abc
Solution:
1.2.2.10 Problem
n ∗
Suppose Sn = i=1 xi where every xi is a machine number and suppose that Sn is
∗ ∗
what the computer calculates. In this case, Sn = f l Sn−1 + xn . Prove:
Also show:
Solution: We put:
Also
1.2.2.11 Problem
Therefore, |ρk | is relative errors for approximating the kth partial sum of Sk by
the calculated partial sum of Sk∗ and |δk | is relative errors in the approximation of
Sk∗ + xk+1 by the quantity f l Sk∗ + xk+1 . Using the relationships that define ρk and
δk , we have:
∗
Sk+1 − Sk+1
ρk+1 =
Sk+1
∗
Sk + xk+1 (1 + δk ) − (Sk + xk+1 )
=
Sk+1
Sk (1 + δk ) + xk+1 (1 + δk ) − (Sk + xk+1 )
=
Sk+1
Sk
= δk + ρk (1 + δk )
Sk+1
|ρ0 | = 0
|ρ1 | ≤ ε
|ρ2 | ≤ ε + θ ε
|ρ3 | ≤ ε + θ (ε + θ ε) = ε + θ ε + θ 2 ε
..
.
16 1 Introduction to Numerical Analysis
|ρn | ≤ ε + θ ε + θ 2 ε + · · · + θ n−1 ε
= ε 1 + θ + θ 2 + · · · + θ n−1
θn − 1
= ε.
θ −1
(1 + ε)n − 1
= ε.
ε
= (1 + ε)n − 1
1.2.2.12 Problem
n
Suppose that f (x) = x 2 and x = x0 is not machine number. So, on the computer, it
will have a modified value of x = x0 (1 + δ). Estimate how much f (x0 ) will change?
Solution: It is obvious that the value of function f (x) at point x0 is calculated using
n squares by performing a sequence of calculations as follows:
where for i = 1, ..., n, |δi | ≤ eps. The nested substitution of relationships can result
in:
n n−1
x n = x02 (1 + δn )2 ...(1 + δn−1 )2 (1 + δn )
n n
−1
x n ≈ x02 (1 + δn )2
(1 + δ)r ≈ (1 + η)r +1
So, we conclude:
n n
x n ≈ x02 (1 + η)2 = f (x0 (1 + η))
φ = ϕ (r ) ◦ ... ◦ ϕ (0)
Suppose that we have the map ψ (i) (also called the remaining map) as follows:
where ψ (0) ≡ φ, which means that none of algorithm steps have been performed.
So, for i = 0, ..., r , we have:
Dψ (i) X (i) = Dϕ (r ) X (r ) .Dϕ (r −1) X (r −1) ...Dϕ (i) X (i) (1.9)
where Dψ (i) is a Jacobian matrix of map ψ (i) . Therefore, the propagation of rounding
error in the steps of performing an algorithm can be defined as follows:
which is called the inherent error of the algorithm. It is clear that the inherent error
depends on the input vector of X and the output vector of Y will appear in any case.
If for every i, the rounding error propagations (E i ’s) on the total error are such
that:
Dψ (i) X (i) .E i .X (i)
(0) Ỹ
Then, we say the rounding errors are harmless. If the rounding errors are harmless
at all steps, the algorithm is well-established or numerically stable.
1.2.3.1 Problem
1
f (θ, kc ) :=
cos2 θ + kc2 sin2 θ
k 2 := 1 − kc2
1
f (θ, kc ) :=
1 − +k 2 sin2 θ
In this case, we do not need to calculate cos θ and the method is faster. Compare
both methods for numerical stability, as follows.
Solution: Algorithm 1:
1.2 Error Analysis 19
b
⎡ 2 ⎤
cos a
ϕ (1) x (1) = ⎣ sin2 a ⎦ = x (2)
b
cos2 a
ϕ (2) x (2) = = x (3)
bsin2 a
ϕ (3) x (3) = cos2 a + bsin2 a = x (4) = φ(a, b)
So, we have:
Then:
Dψ (1) (u, v, w) = 2u, 2wv, v2
and also:
So:
and also:
20 1 Introduction to Numerical Analysis
ψ (3) (m, n) = m + n
So:
and finally:
Dφ(x) = Dφ(a, b) = −2 sin a cos a + 2b sin a cos a, sin2 a
Since:
ϕ (3) x (3) = 1 − (1 − b)sin2 a = x (4) = φ(a, b)
So:
Then:
Dψ (1) (m, n) = −2mn, −m 2
and also:
So:
and also:
ψ (3) (t) = 1 − t
So:
Dψ (3) (t) = −1
and finally:
Dφ(x) = Dφ(a, b) = −2(1 − b) sin a cos a, sin2 a
For the Algorithm 2 to be numerically better than the Algorithm 1, we must have:
3
6 − 11b ≤ 3 ⇒ b ≥
11
3
So, if 11 < b < 1, the Algorithm 2 is better than the Algorithm 1, otherwise, the
Algorithm 1 would be better, that is, if 0 < b < 11 3
, the Algorithm 1 is better. If
π
b = 11 and 0 ≤ a ≤ 2 , the Algorithm 2 still works better than the Algorithm 1.
3
x = p.10q , 1 ≤ | p| < 10
where y is an integer.
In this case, we can state that x is scientifically represented. In this display, p is
called the mantissa and q is called the exponent of the number x.
1.2.3.3 Definition
If p is a decimal number and 1 ≤ | p| < 10, then the significant figures of p are
the nonzero figures of p, the zeros between these figures and the zeros in front of the
number to indicate the accuracy. The significant figures of the nonzero number x are
same as the significant figures of the mantissa of x.
1.2.3.4 Example
(A) If x = 301.57, then x = 3.0157 × 102 and the number of significant figures
of x is 5.
(B) If x = 0.000497, then x = 4.97 × 10−4 ; and x has three significant figures.
(C) If x = 2000m, then x = 2.000 × 103 m; and x has four significant figures.
1.3 Interpolation
the runway to be a very smooth path with the least curvature. For this purpose, the
function that approximates the behavior of these coordinates must be a smooth, low-
curvature function. This can be done using a very powerful interpolation function
such as spline. Sometimes, as we consider the approximate location of points or
coordinates, the drawn curve may not pass through some points, in which case, we
will have another form of approximation called the curve fitting. In both cases, for
better approximation, we select an interpolation function that behaves similar to
the approximate behavior of the points and their distribution. Recently, the topic of
interpolation, specially, the splines, has been introduced in medical sciences. For
example, it is used in very fine devices that take films or photographs from the inside
of the intestine and stomach. Other applications of this type of interpolation function
are in various engineering sciences, of which everyone is aware.
A general but brief introduction to interpolation functions is given below.
Assume that the values of the function f are f 0 , ..., f n , respectively, for the mutual
distinct points of x0 , ..., xn . We call such a function a tabular function. Estimating
the value of f (x) when x ∈ [x0 , xn ] and x = xi , i = 0, ..., n is called interpolation
and estimating the value of f (x) when x ∈ [x0 , xn ] is called extrapolation.
Estimating the value of a function for a point like x that is not in the table but
between table points is also an interpolation.
Suppose that φ is a family of single-variable functions of x with n + 1 parameters
of a0 , ..., an . The problem of interpolation φ is to determine the parameters a0 , ..., an
in φ(x; a0 , ..., an ) so that for pairs (xi , f i ), i = 0, ..., n we have:
where the mutual distinct points of (xi , f i ), i = 0, ..., n can be real or complex.
Equation (1.12) is called the interpolation problem and points (xi , f i ) are called the
interpolation points.
Obviously, the goal is to determine the n + 1 unknown parameters of a0 , ..., an
using the set of equations (1.12). If φ is linearly dependent on parameters a0 , ..., an ,
then the interpolation problem (1.12) is called linear, i.e.,
Nonlinear:
a0 +a1 x+···+aμ x μ
(4) If φ x; a0 , ..., aμ , b0 , ..., bν = b0 +b1 x+···+bν x ν
, φ is a fractional interpolator.
1.3 Interpolation 25
(5) If φ x; a0 , ..., aμ , λ0 , ..., λμ = a0 eλ0 x +· · ·+aμ eλμ x , then φ is an exponential
interpolator.
It is worth noting that for each of interpolations that have been proposed, the intro-
duced interpolation of φ is a linear combination of a set of functions of {φ0 , ..., φn },
that is, they are generated by its members. These types of functions are defined so that
they form an independent linear set. Therefore, it can be claimed that all interpolator
functions form a vector space that can have different bases, and each of the interpo-
lators, according to the structure of the basic members, belongs to their own space.
Obviously, the representation of each interpolation function by the basic members in
the corresponding space is unique, so generally, in Eq. (1.13), the linear combination
coefficients of ai , i = 0, ..., n are obtained uniquely. Therefore, it can be concluded
that any kind of interpolator is unique in its own space. If we want to have a kind of
classification in the order of the appearance of the interpolation functions, we can
say:
n
p(x) = L j (x) f j (1.14)
j=0
n
x − xi
L j (x) = , j = 0, ..., n (1.15)
i=0, j= j
x j − xi
1.3.1.1 Problem
Prove that
n
L i (x) = 1
i=0
Solution: Suppose that f (x) = 1, and the interpolation polynomials p(x) for points
of x0 , ..., xn are such that p ∈ n and for i = 0, ..., n, p(xi ) = 1. On the other hand,
according to Eq. (1.14) we have:
n
n
p(x) = f i L i (x) = L i (x)
i=0 i=0
1.3.1.2 Problem
n
If w(x) = j=0 x − x j , show
w(x)
L i (x) =
(x − xi )w (xi )
Solution: According to the definition of the derivative of the function in the point xi ,
we have:
n
x − xj
L i (x) =
j=0, j=i
xi − x j
1.3.1.3 Problem
Supposeh(x) = L i (x)+ L i+1 (x) for 0 ≤ i < n. Prove that for every x of the interval
xi , xi+1 we have:
x0 < x1 < · · · < xi−1 < xi < x < xi+1 < · · · < xn−1 < xn
According to the Rolle’s theorem, for each of the subintervals of (xi , xi+1 ), i =
0, 1, ..., i − 1, i + 2, ..., n − 1, h (x) = 0 has one root, and since the number of these
subintervals is n − 3, then h (x) = 0 has n − 3 roots in the mentioned subintervals.
Now according to the average value theorem:
h(xi ) − h(xi−1 )
∃β1 β1 ∈ (xi−1 , xi )&h (β1 ) = >0
xi − xi−1
h(x) − h(xi )
∃β2 β2 ∈ (xi , x)&h (β2 ) = <0
x − xi
h(xi+1 ) − h(x)
∃β3 β3 ∈ (x, xi+1 )&h (β3 ) = >0
xi+1 − x
h(xi+2 ) − h(xi+1 )
∃β4 β4 ∈ (xi+1 , xi+2 )&h (β4 ) = <0
xi+2 − xi+1
Therefore, h (x) = 0 has three other roots. As a result, totally, it will have n roots.
But since h(x) is of at most degree n, then h (x) is of at most degree n − 1 and cannot
have n roots. Therefore, the assumption is invalid and h(x) ≥ 1.
1.3.1.4 Problem
We should prove a0 = a1 = · · · = an = 0.
By substituting x j in the above relation we have:
Since the Lagrange polynomials satisfy the Kronecker delta condition, then the
left hand of the above equation is equal to a j . As a result:
a j = 0, j = 0, ..., n
1.3.1.5 Problem
a0 + a1 (x − x0 ) + · · · + ai (x − x0 )...(x − xi−1 )
+ · · · + an (x − x0 )...(x − xn−1 ) = 0
Because the points are mutually distinctive, then ai=0 which is contrary to the
assumption. As a result, the assumed statement is invalid and a0 = a1 = · · · = an =
0.
This type of interpolation is type (A) interpolation based on the Neville recursive
method. The polynomial
1.3.2.1 Problem
Solution: We assume that f (x) = x n without losing generality. It is clear that f (x)
passes through interpolation points, so it can be considered as an interpolation poly-
nomial. On the other hand, if we want to write the divided differences interpolation
polynomial formula, due to the uniqueness of the interpolation polynomial, we have:
1.3.2.2 Problem
f (x1 ) − f (x0 )
f (x0 ) = f (x1 ) − f (x0 ) = (x1 − x0 ) = h f [x0 , x1 ]
x1 − x0
1.3.2.3 Problem
Prove that:
n
f (xi )
f [x0 , ..., xn ] = n
i=0 j=0, j=i x i − x j
n
p(x) = f i L i (x)
i=0
n
n
x − xj
= fi
i=0 j=0, j=i
xi − x j
n
n
fi
= x − x j n
i=0 j=0, j=i j=0, j=i xi − x j
n
f (xi )
f [x0 , ..., xn ] = n
i=0 j=0, j=i x i − x j
1.3.2.4 Problem
n
n f (x0 ) = (−1)n−i Cni f (xi )
i=0
n
n
xi − x j = (i − j)h
j=0, j=i j=0, j=i
i−1
n
= (i − j)h · (i − l)h
j=0 l=i+1
n
f (xi )
f [x0 , ..., xn ] = n
i=0 j=0, j=i x i − x j
n
f (xi )
=
i=0
(−1) n−i
i!(n − i)!h n
1
n
= (−1)n−i Cni f (xi )
n!h n i=0
n
= (−1)n−i Cni f (xi )
i=0
1.3.2.5 Problem
Assume that the nth derivatives of the function f (x) in I [x0 , ..., xn ] are continuously
exist. If x0 , ..., xn are distinct points, then:
1 t1 tn−1
f [x0 , ..., xn ] = dt1 dt2 · · · dtn
0 0 0
1
f [x0 , x1 ] = dt1 × f (t1 (x1 − x0 ) + x0 )
0
For this purpose, we introduce a new integration variable named ξ for which
dξ
ξ = t1 (x1 − x0 ) + x0 , dt1 =
x1 − x0
t1 = 0 ⇒ ξ = x 0
t1 = 1 ⇒ ξ = x 1
1 x1
1
dt1 × f (t1 (x1 − x0 ) + x0 ) = dξ × f (ξ )
x1 − x0
0 x0
f (x1 ) − f (x0 )
=
x1 − x0
= f [x0 , x1 ]
1 t1 tn−2
f x0 , ..., xn−1 = dt1 dt2 · · · dtn−1
0 0 0
dξ
ξ = tn (xn − xn−1 ) + · · · + t1 (x1 − x0 ) + x0 , dtn =
xn − xn−1
tn−1
dtn × f (n) (tn (xn − xn−1 ) + · · · + t1 (x1 − x0 ) + x0 )
0
ξ1
dξ f (n−1) (ξ1 ) − f (n−1) (ξ0 )
= f (n) (ξ ) =
xn − xn−1 xn − xn−1
ξ0
1 t1 tn−2
f (n−1) (ξ1 ) − f (n−1) (ξ0 )
dt1 dt2 · · · dtn−1 ×
xn − xn−1
0 0 0
f x0 , ..., xn−2 , xn − f x0 , ..., xn−2 , xn−1
=
xn − xn−1
= f [x0 , ..., xn ]
1.3.2.6 Note
(1) If f (n) (x) is continuous on the interval [a, b] and y0 , ..., yn are points in the
[a, b] and x ∈ [a, b] is distinct from yi , then
34 1 Introduction to Numerical Analysis
where
g(x) ≡ f x, y0 , ..., yq , h(y) ≡ f x0 , ..., x p , y
f (n) (ξ )
f [x0 , ..., xn ] = , ξ ∈ I [x0 , ..., xn ]
n!
1.3.2.7 Problem
where
γ ∈ I [z 0 , ..., zr ], η ∈ I y0 , ..., yq , ξ ∈ I x0 , ..., x p
1.3.2.8 Problem
Suppose that x0 , ..., xn are mutual distinct points. Prove that the coefficients a0 , ..., an
in interpolation polynomials p ∈ Pn are continuously dependent on the values of
y0 , ..., yn .
Solution: Suppose that
f (x0 ) = p(x0 ) = a0
f (x1 ) = p(x1 ) = a0 + a1 (x1 − x0 )
= f (x0 ) + a1 (x1 − x0 )
So:
f (x1 ) − f (x0 )
a1 =
x1 − x0
Then:
f (x2 ) − f (x0 ) f (x1 ) − f (x0 )
a2 = −
(x2 − x0 )(x2 − x1 ) (x1 − x0 )(x2 − x1 )
36 1 Introduction to Numerical Analysis
a0 = y0
y1 − y0
a1 =
x1 − x0
y2 − y0 y1 − y0
a2 = −
(x2 − x0 )(x2 − x1 ) (x1 − x0 )(x2 − x1 )
Similarly, by continuing the same process, the other ai is obtained in terms of the
points and yi .
1.3.2.9 Note
f i = f i+1 − f i
1.3.2.10 Problem
Prove that
Solution:
1.3.2.11 Problem
If f (x) = 1
x+c
, where c is a constant (real or complex) number, prove that:
(−1)n
f [x0 , ..., xn ] =
(x0 + c) · · · (xn + c)
1.3 Interpolation 37
f (x1 ) − f (x0 )
f [x0 , x1 ] =
x1 − x0
1
x1 +c
− x01+c
=
x1 − x0
x0 − x1 1
= ·
(x0 + c)(x1 + c) x1 − x0
−1
=
(x0 + c)(x1 + c)
(−1)k
f x1 , ..., xk+1 =
(x1 + c)...(xk+1 + c)
(−1)k
f [x0 , ..., xk ] =
(x0 + c)...(xk + c)
So
f x1 , ..., xk+1 − f [x0 , ..., xk ]
f x0 , ..., xk , xk+1 =
xk+1 , x0
(−1)k (−1)k
= =
(x1 + c)...(xk+1 + c) (x0 + c)...(xk + c)
1
×
xk+1 − x0
(−1)k (x0 − xk+1 ) 1
= ·
(x0 + c)...(xk+1 + c) xk+1 − x0
(−1)k+1
=
(x0 + c)...(xk+1 + c)
1.3.2.12 Problem
Suppose that x0 , ..., xn are distinct points, and f ∈ cn (I (x0 , ..., xn )), in this case:
38 1 Introduction to Numerical Analysis
f [x0 , ..., xn ] = ... f (n) (t0 x0 + · · · + tn xn )dt1 dt2 ...dtn
τn
n n !
where are t0 = 1 − j=1 t j and τn = (t0 , ..., tn )|t j ≥ 0, t
j=1 j ≤ 1 .
1
f (t0 x0 + t1 x1 )dt1 = f (x0 + t1 (x1 − x0 ))dt1
τ1 0
f (x1 ) − f (x0 )
=
x1 − x0
= f [x0 , x1 ]
t˜k
= ... f (k) (x0 + t1 (x1 − x0 ) + · · · + tk (xk − x0 ))dtk dt1 · · · dtk−1
τk−1 0
1
... [ f (k−1) x0 + t1 (x1 − x0 ) + · · · + t˜k (xk − x0 )
xk − x0
τk−1
− f (k−1) t˜k x0 + t1 x1 + · · · + tk−1 xk−1 ]dt1 · · · dtk−1
1 1−t1 1−t1
−···−tk−2
f (k−1) t1 x1 + · · · + t˜k xk
= ... dt1 · · · dtk−1
xk − x0
0 0 0
(k−1)
f t˜k x0 + t1 x1 + · · · + tk−1 xk−1
− ... dt1 ...dtk−1
xk − x0
f [x1 , ..., xk ] − f x0 , ..., xk−1
=
xk − x0
= f [x0 , ..., xk ]
1.3 Interpolation 39
1.3.2.13 Problem
f (n) (x0 )
lim f [x0 , ..., xn ] =
x j →x0 n!
Solution:
lim f [x0 , ..., xn ] = ... f (n) ((t0 + · · · + tn )x0 )dt1 · · · dtn
x j →x0
τn
= ... f (n) (x0 )dt1 · · · dtn
τn
= f (n) (x0 ) ... dt1 · · · dtn
τn
f (n) (x0 )
=
n!
1.3.2.14 Problem
Show that if α is constant and positive, and f (x) = α x , then in what conditions
fi = fi
always holds.
Solution: We have to find the condition for f i = f i+1 − f i = f i .
f i+1 = 2 f i
⇒ α xi +1 = 2α xi
⇒ ln α xi +1 = ln 2α xi
⇒ xi+1 ln α = ln 2 + xi ln α
⇒ (xi+1 − xi ) ln α = ln 2
ln 2
⇒ xi+1 = xi +
ln α
So, by choosing h = ln 2
ln α
, f i = f i will always hold.
40 1 Introduction to Numerical Analysis
Now we are going to review some of the basic concepts of vector norms and linear
system of equation which are essential for a deeper understanding of intervals.
In this section, first we introduce the norm of a real-valued vector and then point out
its application in finding the bounds of errors and its approximate solution.
Usually, the norm of a vector is used to define a distance in Rn .
1.4.1.1 Definition
· : Rn → R
(1)
∀X X ∈ Rn ⇒ X ≥ 0
(2)
∀X X ∈ Rn ⇒ (X = (0, ..., 0) = 0 ⇐⇒ X = 0)
(3)
∀X ∀λ X ∈ Rn &λ ∈ R ⇒ λX = |λ| · X
(4)
∀X ∀Y X ∈ Rn &Y ∈ R ⇒ X + Y ≤ X + Y
1.4.1.2 Definition
If a function does not hold property number (2) of those mentioned above, then, it is
called semi-norm.
1.4 A Short Review on Vector Norms and Linear System of Equations 41
1.4.1.3 Definition
n
X 1 = |xi |
i=1
(2) Norm L2
n 21
X 2 = |xi |2
i=1
i=1
(4) Norm L ∞
X ∞ = max|xi |
i
where B is a definite positive symmetric matrix and satisfies the norm properties.
R . In
n t H
All defined norms are on space the case, H , H are Hermitian and B isn
the definite positive Hermitian B = B matrix, norm (1.16) is defined on space C
t
and is easily proved to be an inner product. The following is the definition of inner
product.
1.4.1.4 Definition
(2) ∀X X ∈ Rn ⇒ (X = 0 ⇐⇒ (X, Y ) = 0)
42 1 Introduction to Numerical Analysis
(3) ∀X ∀Y ∀α X, Y ∈ Rn &α ∈ R ⇒ (α X, Y ) = α(X, Y )
(4) ∀X ∀Y X, Y ∈ Rn ⇒ (X, Y ) = (Y, X )
(5) ∀X ∀Y ∀Z X, Y, Z ∈ Rn ⇒ (X + Y, Z ) = (X, Z ) + (Y, Z )
(X, Y ) = (Y, X )
We should only prove the triangle inequality property. For this purpose, given the
Cauchy–Schwarz inequality we have already proved, we have:
Therefore
As a result
X + Y ≤ X + Y
Now, considering Eqs. (1.16) and (1.17), for a definite positive symmetric matrix
B, the inner product of two vectors X and Y is defined as follows:
(X, Y ) = X t BY
To prove that Eq. (1.16) is a norm, it is sufficient to prove that it satisfies the
properties of inner product, and since the inner product defines a norm, Eq. (1.16)
defines a vector norm.
If we want to have a two-dimensional image of the defined norms, we need to
consider a unit sphere to which the norms are constrained. Level {X |X = 1} is
1.4 A Short Review on Vector Norms and Linear System of Equations 43
called unit sphere and set {X |X ≤ 1} is called unit ball where each norm on this
ball is represented by the following shapes (Fig. 1.1).
These mentioned norms are theoretically useful to produce other type of norms.
After defining the norm and introducing their types, we will discuss it further by
providing some theorems.
The following theorem is known as the norm equivalence theorem. Given this
theorem, it can be claimed that if a vector converges under one norm, it also converges
under other norms. That is, if a sequence of vectors converges under norm L ∞ , it
converges under other norms such as L1 , L2 and Lp .
Suppose · and · are two arbitrary norms on Rn or Cn . In this case, there are
constants 0 < c1 ≤ c2 such that for each vector X,
c1 X ≤ X ≤ c2 X
44 1 Introduction to Numerical Analysis
Several numerical methods for solving the integral equations use the system of linear
equations. To solve these systems, there are two types of methods, direct and iterative
methods.
In this method, we use a procedure that converts the matrix above into an upper
triangular matrix with nonzero diagonal elements. These conversions are performed
according to the matrix rank discussed in the previous section.
The first step in the Gaussian elimination method is to set elements below a11
equal to zero, provided that a11 = 0. If a11 = 0, given that det(A) = 0, then in the
first column, there is a row, for example pth row, where a p1 = 0, because otherwise,
det(A) = 0, which is a contradiction. Now we interchange the first
and pth rows so
−ai1
that the (1, 1) element of the matrix is nonzero. Then, we add a11 -times of the
first row to the second to nth rows, denoting the new coefficients by ai(1)
j .
If we assume m i1 = −a i1
a11
for i = 1, ..., n, we have:
ai1
ai(1)
j = ai j − a1 j = ai j + m i1 · a1 j
a11
i = 2, ..., n, j = 2, ..., n + 1
The second step of the Gaussian elimination method is to set the elements below
(1) (1)
a22 equal to zero, which according to the discussion of the first step, a22 = 0.
Otherwise, there must be an index of rows like r, where 3 ≤ r ≤ n and ar(1) 2 = 0.
Because if it is not the case, the first and second columns are multiplications of each
1.4 A Short Review on Vector Norms and Linear System of Equations 45
We continue the same process until reaching the following matrix after n − 1 step:
⎡ ⎤
a11 a12 a13· · · a1n : a1,n+1
⎢ 0 (1) (1)
· · · a2n(1) (1)
: a2,n+1 ⎥
⎢ a22 a23 ⎥
⎢ (2) (2) (2) ⎥
⎢ 0 0 a33· · · a3n : a3,n+1 ⎥ (1.20)
⎢ . .. .. . .. ⎥
⎢ . .. ⎥
⎣ . . . . .. . ⎦
(n−1) (n−1)
0 0 0 · · · ann : an,n+1
All of the diagonal coefficients are nonzero. In general, the coefficients are
obtained as follows:
ai(k) (k−1)
j = ai j + m ik · ak(k−1)
j , k = 1, ..., n − 1
(k−1)
aik
m ik = − (k−1)
, i = k + 1, ..., n
akk
ai(0)
j = ai j , j = k + 1, ..., n + 1 (1.21)
The system of equations that is obtained from the last step is as follows:
..
.
(n−1) (n−1)
ann xn = an,n+1 (1.22)
1.4.2.2 Remark
A = LU
where U = u i j and L = li j are defined as follows:
"
ai(i)
j , i = 1, ..., j − 1
ui j =
0, i = j + 1, ..., n
⎧
⎨ 0, i = 1, ..., j − 1
li j = 1, i = j
⎩
m i j , i = j + 1, ..., n
Therefore,
n
n
det(A) = det(LU ) = det(L) · det(U ) = u ii × lii
i=1 i=1
So det(A) = 0.
The non-singularity of the matrix A is a necessary condition for the LU
decomposition, but it is not a sufficient condition.
In this section, it is tried to convert matrix A to a product of the lower triangular matrix
L and the upper triangular matrix U . This decomposition has many applications in
solving the system of linear equations. The decomposition is performed in two ways,
1.4 A Short Review on Vector Norms and Linear System of Equations 47
one using the Gaussian elimination method and the other using direct decomposition,
which is discussed in the following.
In this case,
⎡ ⎤
a11 a12 · · · a1i · · · a1n
⎢ . .. .. .. ⎥
⎢ .. . . . ⎥
⎢ ⎥
⎢ ⎥
⎢ a ai2 · · · aii · · · ain ⎥
L i A = ⎢ (1)i1 (1) (1) ⎥
⎢ ai+1,1 ai+1,2 · · · 0 · · · ai+1,n ⎥
⎢ ⎥
⎢ .. .. .. .. ⎥
⎣ . . . . ⎦
(1)
an1 an2 · · · 0 · · · ann
That is, by multiplying into L i , the elements below aii become zero. Suppose that
L i−1 = L i . The matrix L i is the same as the matrix L i , with only one exception that
the signs of m i j ’s are changed.
The obtained matrix from the application of the first step of the Gaussian
elimination method on the matrix of coefficients A can be written as follows:
48 1 Introduction to Numerical Analysis
A1 = L 1 P1r1 A, r1 ≥ 1
That is, the first and second rows are interchanged before removing. (If no
interchange is required, i.e., a11 = 0, then r1 = 1 and P1,r1 = I ).
In general, if we show the coefficient matrix in the ith step of the Gaussian
elimination method by Ai , then
Before the i’st elimination stage, rth and r i th rows are interchanged. Continuing
the same process in the (n − 1)th step, the upper triangular
coefficient matrix An−1
in the system of equations (3.17) will be as U = u i j .
If matrix Pi,ri was already known, we would have to use it for matrix A from the
beginning.
where P is the permutation matrix in the form of P = Pn−1,rn−1 ...P1,r1 and we will
be able to perform the Gaussian elimination method without interchanging the rows,
as a result, Eq. (1.25) will be written as follows
U = L n−1 · · · L 2 L 1 Pn−1,rn−1 · · · P1,r1 A = L̃ Ã (1.26)
where
L̃ := L n−1 L n−2 · · · L 2 L 1
as a result
L̃ −1 = L −1 −1 −1
1 L 2 · · · L n−1 := L
1.4 A Short Review on Vector Norms and Linear System of Equations 49
which is as follows
⎡ ⎤
1
⎢ −m 21 1 0 ⎥
⎢ ⎥
⎢ .. ⎥
L̃ = ⎢
⎢ −m 31 −m 32 .
⎥
⎥
⎢ . ⎥
⎣ .. 1 ⎦
−m n1 −m n2 −m n,n−1 1
So we have
à = P A = LU
LU = L 1 U1
In this case UU1−1 = L −1 L 1 because U1 and L are non-singular. Since the inverse
of an identity lower triangular matrix (on the diagonal, one) has the same form and
the product of the two lowers triangular (identity) matrices has also the same form,
an upper triangular matrix is equal to a lower triangular matrix, and this is possible
if both matrices are identity matrices. i.e.,
U1 = U, L 1 = L
min(i, j)
ai j = lir u r j
r =1
k
k−1
aik = lir u r k = lir u r k + lik
r =1 r =1
k−1
lik = aik = lir u r k , i = k, k + 1, ..., n (1.29)
r =1
According to the inductive hypothesis, rk−1 =1 lir u r k is uniquely specified. So i =
k, k +1, ..., n, lik , that are the elements of the kth column of L are uniquely identified.
Now, we uniquely specify u k,k+1 , ..., u kn .
Suppose k ≤ j, so min(i, j) = k. We have:
k
k−1
ak j = lkr u r j = lkr u r j + lkk u k j , j = k, k + 1, ..., n
r =1 r =1
k−1
r =1 l kr u r j is uniquely specified according to the induction hypothesis, and since
all the forward submatrices are non-singular, then lkk = 0 and it is known according
to the previous case.
1
k−1
uk j = ak j − lkr u r j , j = k + 1, ..., n (1.30)
lkk r =1
n
n3
(n − k+)(k − 1)
k=2
6
n
n3
(n − k)k
k=2
6
n3 n3 n3
+ + (n − 1)
6 6 3
multiplication operations can be counted
3 in the algorithm, so it can be said that the
complexity is of the order of about O n3 .
The application of the decomposition A = LU in the form of u ii = 1, i = 1, ..., n
to solve the system AX = b is as follows:
AX = b ⇒ (LU )X = b ⇒ L(U X ) = b
b1
y1 = , l11 = 0
l11
1
i−1
yi = bi − lir yr , i = 2, ..., n
lii r =1
n
n(n + 1)
1+ i=
i=2
2
So,
⎡√ ⎤
d1 0
⎢ .. ⎥
E =⎣ . ⎦
√
0 dn
A = L E 2 L t = L E · E L t = (L E)(L E)t
A = M Mt
where
Ak Ck+1 Mk 0
Ak+1 = t , M k+1 =
Ck+1 ak+1,k+1 m tk+1 m k+1,k+1
m k+1,k+1 , m tk+1
By substitution, we have:
Mk · m k+1 = Ck+1
1.4.2.6 Example
Why?
AX = b ⇒ M M t X = b ⇒ M M t X = b
And as a result
⎡ ⎤
2
Mt X = C ⇒ X = ⎣ 2 ⎦
1
1.4.2.7 Theorem
it can be written:
0 < X t AX = X t L DL t X = Y t DY = yi2 dii
A + D = A − λI
det(A − λI ) = 0 (1.31)
In this section, we discuss three types of simple iterative methods. These methods
are not generally convergent but in convergence cases, the SOR methods have some
advantages over the others.
1.4 A Short Review on Vector Norms and Linear System of Equations 55
Suppose that det(A) = 0, then A can be converted to a matrix with nonzero diagonal
elements using permutation matrices.
A = D − (L + U )
= D− L −U
where
⎡ ⎤
a11 0 ··· 0
⎢ 0 a22 0 0 ⎥
⎢ ⎥
D=⎢ . .. . . .. ⎥
⎣ .. . . . ⎦
0 0 · · · ann
⎡ ⎤
0 0
⎢ −a21 0 ⎥
⎢ ⎥
L=⎢ . .. .. ⎥
⎣ .. . . ⎦
−an1 −an2 · · · −an,n−1 0
⎡ ⎤
0 −a12 · · · −a1n
⎢ .. ⎥
⎢ 0 . ⎥
U =⎢
⎢
⎥
⎥
⎣0 ..
. −an−1,n ⎦
0
We have
[D − (L + U )]X = b
Then,
D X = (L + U )X + b
as a result
X = D −1 (L + U )X + D −1 b
Then,
⎛ ⎞
−1 ⎝
n
xi(k+1) = ai j x (k)
j
⎠, i = 1, ..., n
aii j=1, j=i
56 1 Introduction to Numerical Analysis
Step (3) If x (k) is accurate enough, we go to the next step, otherwise we add one
to k and go to step (2).
Step (4) The procedure is complete.
This algorithm requires that aii = 0 for i = 1, ..., n, otherwise the order of the
equations can be changed so that aii become nonzero, and it is also better for aii ’s
to have the maximum possible values to accelerate convergence.
The criterion for stopping the algorithm is when
X (k+1) − X (k)
or
X (k+1) − X (k)
X (k+1)
1.4.3.3 Example
10x1 − x2 + 2x3 = 6
− x1 + 11x2 − x3 + 3x4 = 25
1.4 A Short Review on Vector Norms and Linear System of Equations 57
which has an answer of X = (1, 2, −1, 1)t . To obtain the iterative matrix of the
Jacobi’s method and the answer of the system, we proceed as follows:
1 1 3
x1 = x2 − x3 +
10 5 5
1 1 3 25
x2 = x1 + x3 − x4 +
11 11 11 11
−1 1 1 11
x3 = x1 + x2 + x4 −
5 10 10 10
3 1 15
x4 = − x2 + x3 +
8 8 8
Then
⎡ 1 −1 ⎤ ⎡ 3 ⎤
0 10 5
0 5
⎢ 1 0 1 −3 ⎥ ⎢ 25 ⎥
H =⎢ 11
⎣ −1 1
11 11
1
⎥, c = ⎢
⎦ ⎣
11
−11
⎥
⎦
5 10
0 10 10
−3 1 15
8
For an initial approximation, we set X (0) = (0, 0, 0, 0)t and produce X (1) :
1 (0) 1 (0) 3
x1(1) = x − x3 +
10 2 5 5
(1) 1 (0) 1 (0) 3 25
x2 = x1 + x3 − x4(0) +
11 11 11 11
(1) −1 (0) 1 (0) 1 (0) 11
x3 = x + x2 + x4 −
5 1 10 10 10
3 (0) 1 (0) 15
x4 = − x2 + x3 +
8 8 8
t
So, X (1) = 35 , 25 , −11 , 15
11 10 8
and are generated in the same way as X (k) until the
stop condition is met. The following table shows the iteration for five steps (Table
1.1).
1.4.3.4 Theorem
If the matrix A is a strictly diagonally dominant row matrix, then the Jacobi’s iterative
method is convergent.
Proof The matrix A is the strictly diagonally dominant row matrix, so:
58 1 Introduction to Numerical Analysis
n
ai j
< |aii |, i = 1, ..., n
j=1, j=i
and the elements of the matrix H are the iteration matrix of this method that are
stated as follows:
"
−ai j
, i = j
hi j = aii
0i = j
Then,
n
n
−ai j
n
ai j
h i j
=
= <1
a
|ai |
j=1 j=1, j=i ii j=1, j=i
As a result,
n
max
h i j
< 1
1≤i≤n
j=1
Therefore H ∞ < 1, then according to the theorem (Sect. 1.4.3.6), the iteration
matrix is convergent.
We examine the convergence of the Jacobi’s method according to the following
theorem.
1.4.3.5 Theorem
If the matrix A is a strictly diagonally dominant row matrix, then the Jacobi’s iterative
method is convergent.
Proof The matrix A is the strictly diagonally dominant row matrix, so:
1.4 A Short Review on Vector Norms and Linear System of Equations 59
n
ai j
< |aii |, i = 1, ..., n
j=1, j=i
and the elements of the matrix H are the iteration matrix of this method that are
stated as follows:
"
−ai j
, i = j
hi j = aii
0i = j
Then,
n
n
−ai j
n
ai j
h i j
=
= <1
a
|ai |
j=1 j=1, j=i ii j=1, j=i
As a result,
n
max
h i j
< 1
1≤i≤n
j=1
Therefore H ∞ < 1, then according to the theorem (Sect. 1.4.3.4), the iteration
matrix is convergent.
A = D − L − U = (D − L) − U
where U, L , D = diag(a11 , ..., ann ), respectively, are the lower and upper triangular
matrices as follows:
−ai j i > j 0, i ≥ j
li j = , ui j =
0i ≤ j −ai j i < j
So, we have
[(D − L) − U ]X = b
Then,
(D − L)X = U X + b
60 1 Introduction to Numerical Analysis
and as a result
X = (D − L)−1 U X + (D − L)−1 b
where H = (D − L)−1 U is the iterative matrix of this method and to solve the
system AX = b we write:
Then,
as a result
X (k+1) = D −1 L X (k+1) + U X (k) + b
then
⎡ ⎤
1 ⎣
i−1 n
xi(k+1) = bi − (k+1)
ai j x j − (k) ⎦
ai j x j , i = 1, ..., n (1.32)
aii j=1 j=i+1
Step (3) If x (k+1) has become accurate enough, we go to the next step, otherwise,
we add one to k and go to step (2).
Step (4) The procedure is complete.
This algorithm also requires that aii = 0 for every i = 1, ..., n, otherwise, the
order of the equations can be changed so that aii becomes nonzero. Also, in this
method, x1(k+1) is obtained using x1(k) (Jacobi’s method).
The computer program of this method is presented in the appendix number (9)
based on the MATLAB (6.1) software.
The convergence of the Jacobi’s method usually does not result in the convergence
of the Gauss–Seidel method, and vice versa, but if both methods are convergent, the
(k+1)
Gauss–Seidel method converges faster because it uses the values xi−1 , ..., x1(k+1) to
(k+1) (k) (k)
calculate xi , which are far better than x1 , ..., xi−1 .
1.4.3.8 Example
10x1 − x2 + 2x3 = 6
− x1 + 11x2 − x3 + 3x4 = 25
2x1 − x2 + 10x3 − x4 = −11
3x2 − x3 + 4x4 = 15
1 (k−1) 1 (k−1) 3
x1(k) = x − x3 +
10 2 5 5
1 (k) 1 (k−1) 3 (k−1) 25
x2(k) = x + x3 − x4 +
11 1 11 11 11
−1 (k) 1 (k) 1 (k−1) 11
x3(k) = x + x2 + x4 −
5 1 10 10 10
3 (k) 1 (k) 15
x4(k) = − x2 + x3 +
8 8 8
In the same way, X (k) ’s are generated until the stop condition is met.
According to the X (k) ’s that are obtained, the required iterations in the Jacobi’s
method are almost twice as many iterations of the Gauss–Seidel method.
62 1 Introduction to Numerical Analysis
A = (D − L) − U (1.33)
[(D − L) − U ]X = b
So
(D − L)X = U X + b
(D − L)X (k+1) = U X (K ) + b
Therefore,
Then,
where
1.4.3.10 Example
4x1 + 3x2 = 24
3x1 + 4x2 − x3 = 30
− x2 4x3 = 24
The answer of the system is X = (3, 4, −5)t . To solve the system, we use the
Gauss–Seidel and SOR methods with w = 1.25. We suppose that X (0) = (1, 1, 1)t
1.4 A Short Review on Vector Norms and Linear System of Equations 63
is the initial vector. For every k = 1, ..., the equations of the Gauss–Seidel method
are as follows:
x1(k) = −0.75x2(k−1) + 6
x2(k) = −0.75x1(k) + 0.25x3(k−1) + 7.5
x3(k) = 0.25x2(k) − 6
and the equations of the SOR method for w = 1.25 are as follows:
The first seven iterations of the Gauss–Seidel and SOR methods are given in the
following tables, respectively (Tables 1.2 and 1.3).
Then:
b b b
f (x)d x = pn (x)d x + Rn (x)d x
a a a
i.e.,
1.5 Numerical Integration 65
b
I ( f ) = Q( f ) + E( f ), E( f ) = Rn (x)d x
a
n
n
x − xj
pn (x) = L i (x) f i , L i (x) =
i=0 j=0, j=i
xi − x j
n
t−j
L i (x) = = ϕi (t)
j=0, j=i
i−j
Therefore,
b
n
n
pn (x)d x = h αi f i = wi f i (1.37)
a i=0 i=0
where
n
αi = ϕi (t)dt (1.38)
0
The following theorem shows that the Newton–Cotes integration weights are
unique.
1.5.1.1 Theorem
n b
γi p(xi ) = p(x)d x
i=0 a
n
αi = n (1.39)
i=0
αi = αn−i (1.40)
1.5.1.2 Remark
Suppose
n
1
cin = ϕi (θ )dθ
n
0
in this case
n
cin = 1 (1.41)
i=0
cin = cn−i
n
(1.42)
In this section, we state and prove a theorem that can be used for easily acquiring the
error of a Newton–Cotes integration rule. As previously seen in the trapezoidal and
Simpson’s, mean value methods, etc., no specific equation was available to obtain
the error, and we could calculate the error by comparing the values and by using the
Taylor expansion. But the Peano’s error theorem introduces an equation for finding
the error of the most Newton–Cotes integration procedures.
m0
m1
mn
I˜( f ) := αk0 f (xk0 ) + αk1 f (xk1 ) + · · · + αkn f (xkn ) (1.43)
k=0 k=0 k=0m
b
R( f ) := I˜( f ) − f (x)d x (1.44)
a
V = C n [a, b]
i.e., V is a space that consists of all functions whose derivatives up to order n are
continuous on [a, b]. Or V = n is a space consisting of all polynomials with
degrees no more than n.
1.5.2.1 Theorem
Suppose that we have R( p) = 0 for every polynomial like p ∈ n . That is, the
integral rule is accurate for all polynomials of at most degree n.
In this case, for every function f ∈ C n+1 [a, b], we have:
b
R( f ) = f (n+1) (t)k(t)dt
a
where
1 (x − t)n , x ≥ t
k(t) := Rx (x − t)n+ , (x − t)n+ :=
n! 0, x < t
and Rx (x − t)n+ represents the error of (x − t)n+ in the case that a function of x is
considered. (The function k(t) is usually called the kernel of the operator R or the
Peano kernel).
Proof We write the Taylor expansion of the function f (x) about the point x = a as
follows:
n
f (i) (a)
f (x) = (x − a)i + rn (x) (1.45)
i=0
i!
x
f (n+1) (ξ ) 1
rn (x) = (x − a)n+1 = f (n+1) (t)(x − t)n dt
(n + 1)! n!
a
68 1 Introduction to Numerical Analysis
x
1
= f (n+1) (t)(x − t)n+ dt
n!
a
Now, we prove that the operator Rx and the integral can be interchanged. To do
this, first we should indicate that the differentiation and integral operators can be
commuted. We show by weak induction that for 1 ≤ k ≤ n,
⎡ b ⎤
b k
dk ⎣ (n+1) ⎦ (n+1) d
f (t)(x − t) n
+ dt = f (t) (x − t)n
+ dt (1.47)
dxk dxk
a a
i.e.,
⎡ ⎤
n b x n
d ⎣ d
f (n+1)
(t)(x − t)n+ dt ⎦ = f (n+1) (t) (x − t)n
+ dt
dxn dxn
a a
it can be said that the operator I˜ can also be commutated by the integral.
Now, to indicate that the operator Rx can be commutated by the integral, it is
sufficient that by considering Eq. (1.47) we show that:
⎡ ⎤ ⎡ b ⎤
b b x
⎣ f (n+1) (t)(x − t)n+ dt ⎦d x = f (n+1) (t)⎣ (x − t)n+ d x ⎦dt
a a a a
b
1
R( f ) = f (n+1) (t)Rx (x − t)n+ dt
n!
a
b
1
R( f ) = f (n+1) (t) Rx (x − t)n+ dt
n!
a
b
:= f (n+1) (t)k(t)dt
a
where
1
k(t) := Rx (x − t)n+
n!
Given the continuity of f (n+1) on [a, b] and supposing that k(t) does not change
the sign in the same interval, according to the integral mean value theorem we can
write:
b
(n+1)
R( f ) = f (ξ ) k(t)dt, ∃ξ ∈ (a, b) (1.48)
a
By considering Eq. (1.46) and comparing it with (1.48) and given that for every
P ∈ n , R(P) = 0
we have
f (n+1) (ξ )
R( f ) = R (x − a)n+1
(n + 1)!
f (n+1) (ξ ) n+1
= R x , ∃ξ ∈ (a, b) (1.49)
(n + 1)!
1.5 Numerical Integration 71
b n
f (x)d x = Hj f aj + E (1.50)
j=1
a
if we have
f (x) = x k &k = 0, 1, ..., 2n − 1 ⇒ E = 0
n
αk = H j a kj , k = 0, 1, ..., 2n − 1 (1.51)
j=0
b
1 k+1
αk = xkdx = b − a k+1 (1.52)
k+1
a
n
1 k+1
H j a kj = b − a k+1 , k = 0, 1, ..., 2n − 1 (1.53)
j=0
k+1
In fact, by solving the system (1.53), we can obtain the unknowns in such a way
that the integration rule (1.50) is exact for the polynomial functions of degree at most
2n − 1.
Given the nonlinearity of (1.53), it is not easy to solve it for large orders (large
n). But it is more accurate than Newton–Cotes methods. As we have already seen,
72 1 Introduction to Numerical Analysis
in the Newton–Cotes method, only unknowns were the integral weights, whereas in
these methods, in addition to the integration weights, a j ’s are unknown, too. So, we
have 2n unknowns and need 2n equations. It should be noted that a j ’s are the roots
of orthogonal polynomials. In this section, we deal with orthogonal polynomials that
are briefly discussed in chapter three. Considering that orthogonal polynomials are
defined in a particular interval and by their specific weight function, so there are
corresponding Gaussian numerical integration methods for each set of orthogonal
polynomial functions and their roots. First, without the consideration of the weight
function and given that the error term of such methods must be proportional to the
2nth derivative of the function f , we obtain the general form of Gaussian method
using simple Hermite interpolation for n points of a1 , a2 , ..., an . If we write the simple
Hermite interpolation formula for the mentioned n points with the new notation, we
have:
n
n
R 2 (x) (2n)
f (x) = h j (x) f a j + h̄ j (x) f a j + f (ξ ) (1.54)
j=1 j=1
(2n)!
where
n
R(x) = x − aj (1.55)
j=1
h j (x) = 1 − 2l j a j x − a j l 2j (x), h̄ j (x) = x − a j l 2j (x) (1.56)
It is obvious that the error term of Eq. (1.54) for polynomial functions of at most
degree 2n − 1 is equal to zero. By integrating both sides of Eq. (1.54) on the interval
[a, b], the following equation can be obtained.
b
n
n
f (x)d x = Hj f aj + H̄ j f a j + E (1.57)
a j=1 j=1
where
b b
Hj = h j (x)d x, H̄ j = h̄ j (x)d x (1.58)
a a
b
R 2 (x) (2n)
E= f (ξ )d x (1.59)
(2n)!
a
1.5 Numerical Integration 73
1.5.3.1 Lemma
If (1.50) is accurate for the polynomials of at most degree 2n − 1, then the necessary
and sufficient condition for Eq. (1.57) to be accurate for the polynomials of at most
degree 2n − 1 is that H̄ j = 0 for every j.
That is, for every polynomial of at most degree 2n − 1, the error term is equal to
zero. So, if we choose f (x) as h̄ k (x), then E = 0. Because h̄ k is a polynomial of at
most degree 2n − 1 (why?). So, we have:
b
n
n
h̄ k (x)d x = H j h̄ k a j + H̄ j h̄ k a j , k = 1, 2, ..., n (1.60)
a j=1 j=1
b
h̄ k (x)d x = H̄k
a
Because
h̄ k a j = 0, h̄ k a j = δ jk
f (x) = h̄ k (x)
so
b
n
h̄ k (x)d x = H j h̄ k a j
a j=1
74 1 Introduction to Numerical Analysis
That is, for every k, H̄k = 0. Therefore, the sentence is true. The proof of adequacy
is straightforward.
According to the lemma (Sect. 1.5.3.1), we can show that under the following
conditions
b
H̄ j = h̄ j (x)d x = 0
a
i.e.
b b
l j (x)
x − a j l 2j (x)d x = R(x) · dx = 0 (1.61)
R a j
a a
2n (n!)2
R(x) = Pn (x) (1.62)
(2n)!
It was observed that the Legendre polynomials, having a constant weight of one,
form an orthogonal set on the interval [−1, 1]. For this purpose, it should be noted
that any desired interval [a, b] can be converted to the interval [−1, 1] by a change
of variables as y = 2x−a−b
b−a
.
In the following table, the zeros of the Legendre polynomial up to degree 5 along
with their weights are listed.
Now, we calculate the Gauss–Legendre integration weight.
1 1
Hj = h j (x)d x = 1 − 2l j a j x − a j l 2j (x)d x
−1 −1
1 1
= l 2j (x)d x − 2l j x − a j l 2j (x)d x
−1 −1
1
= l 2j (x)d x − 2l j a j H̄ j
−1
1.5 Numerical Integration 75
1
= l 2j (x)d x (1.63)
−1
The above equation indicates that all of the Gauss–Legendre integration weights
are positive. If we want to have a simpler form for calculating the weights of H j ,
we should employ Eq. (1.50) such that in this equation, the term E for the function
f (x) = lk (x) become
zero. (Why?).
Considering lk a j = δk j , we have:
1
n
lk (x)d x = H j lk a j = Hk (1.64)
−1 j=1
1 1
l 2j (x)d x = l j (x)d x (1.65)
−1 −1
1
f (2n) (η)
E= R 2 (x)d x (1.66)
(2n)!
−1
where η ∈ (−1, 1). Of course, it should be noted that since R 2 (x) does not change
the sign on [−1, 1], the integral mean value theorem has been used.
Therefore, the Gauss–Legendre integration method was introduced where the
integration weights are obtained from (1.64) and the integration points are the roots
of the Legendre polynomials.
1.5.3.2 Example
*3 dx
We compute 1 x
using the three-point Gauss–Legendre method.
Solution: If we use the corresponding change of variables, the transformation y =
x − 2 converts the interval [1, 3] to the interval [−1, 1]. Using Table 1.4 for n = 3
we have:
1
dy 5 1 8 1 5 1
× + × + ·
1.098039
y+2 9 1.225403 9 2 9 2.774597
−1
76 1 Introduction to Numerical Analysis
1
f (6) (η)
E= R 2 (x)d x
6!
−1
where
Sometimes the points used for interpolation may not be exactly available, and we
may have some parts of a data. For this type of data, the word “approximately”
is commonly used. Now, if we want to make the word approximately and about
meaningful, one of the ways is to use the interval; for example, we show about
2 with [2 − ε, 2 + ε], where ε is introduced arbitrarily. So instead of about 2 or
approximately 2, you can use a range that contains 2.
In this section, in order to discuss interval data error, we will first have a brief overview
on the interval calculations.
a ∈ R, a ∈ Q
a
where
ã := a, a = x|x ∈ R, a ≤ x ≤ a
where
c = min a · b, a · b, a · b, a · b
c = max a · b, a · b, a · b, a · b
where
a a a a
c = min , , ,
b b b b
a a a a
c = max , , ,
b b b b
n
φ(x; a0 , ..., an ) = a j φ j (x) (2.1)
j=0
Such that the interpolation condition is satisfied, we have the following interval
data
According to the right side of the equation, it is obvious that the left side must
also be an interval. Therefore, it can be written
φ i , φ i = f i , f i , i = 0, ..., n
where
φ = min u|φ ≤ u ≤ φ
φ = max u|φ ≤ u ≤ φ
L : φ(xi ; a0 , ..., an ) = f i
U : φ(xi ; a0 , ..., an ) = f i , i = 0, ..., n
n
n
φ(x; a0 , ..., an ) = a j φ j (x) = a j φ j (x)
j=0 j=0
80 2 Interval Interpolation
= f i , i = 0, ..., n
n
n
φ(x; a0 , ..., an ) = a j φ j (x) = a j φ j (x)
j=0 j=0
and as before
a j φ j (x), φ j (x) ≥ 0
a j φ j (x) =
a j φ j (x), φ j (x) < 0
So we will have
φ(x; a0 , ..., an ) = a j φ j (x) + a j φ j (x) (2.3)
φ j (x)≥0 φ j (x)<0
= f i , i = 0, ..., n
as a result
2.2 Interval Interpolation 81
L: a j φ j (xi ) + a j φ j (xi ) = f i
φ j (xi )≥0 φ j (xi )<0
U: a j φ j (xi ) + a j φ j (xi ) = f i (2.4)
φ j (xi )≥0 φ j (xi )<0
i =0
According to Eqs. (2.2) and (2.3), it can be said that φ and φ form an interval.
So, in this case, we always have two types of interpolation problems as(L) and (U),
each of which must uniquely exist, and (L) and (U) interpolate points xi , f i and
xi , f i , respectively. So, in this case, instead of an interpolating function, we have
a band of an interpolating function (Fig. 2.1).
Therefore, it is necessary for each group of the points xi , f i and xi , f i ,
i = 0, ..., n to form a function, and an interpolation problem corresponded to each
of them is introduced.
Now if φ j (x) = x j , the interpolation interval is as follows
L : p(x) = ajx j + ajx j
x j ≥0 x j <0
(2.5)
U : p(x) = a j x j (xi ) + a j x j (xi )
x j ≥0 x j <0
It is clear that by considering the j th power of x, as even and odd, we can write:
L : p(x) = ajx j + ajx jx < 0
j=2k j=2k+1
n
p(x) = ajx jx ≥ 0
j=0
n
k = 0, ...,
2
U : p(x) = ajx j + ajx jx < 0
j=2k j=2k+1
n
p(x) = ajx jx ≥ 0
j=0
n
k = 0, ...,
2
The following cases can be considered for system (2.5):
(1) If for every x and every j, x j is non-negative, then
n
L : p(x) = ajx j
j=0
(2.6)
n
U : p(x) = ajx j
j=0
(2) If for every x and every, is negative (although this never happens due to a
constant multiplication), then we have
n
L : p(x) = ajx j
j=0
(2.7)
n
U : p(x) = ajx j
j=0
(3) If for some and, is non-negative and for some and, are negative, then the system
(2.5) is established in the same way.
In case 1, (L) and (U) systems always can be separately solved and also these two
systems can be written as a combination of themselves with double dimensions and
examines as follows
2.2 Interval Interpolation 83
B 0 a f
=
0 B a f
where
n
j
B = xi
i, j=0
t
a = a 0 , ..., a n , a = [a 0 , ..., a n ]
t t
f = f 0 , ..., f n , f = f 0 , ..., f n
In case (2), (L) and (U) systems always can still be separately solved. But with
double dimensions, the combination is as follows:
0 C a f
=
C 0 a f
where a, a, f , and f are the same as the above vectors, and matrix C is defined
similar to matrix B. In case (3), if we want to solve systems (L) and (U) separately, then
we either have no answer or have infinite answer, because, the number of unknowns
exceeds the number of constraints. So, we have to solve the combined system of
double order, which is obtained as follows:
BC a f
= (2.8)
C B a f
where B ≥ 0 and C ≤ 0.
In all three cases of (1), (2) and (3), it can be claimed that the matrix form is as
(2.6). So assuming
BC a f
S= ,X = ,Y =
C B a f
j
a j xi = f i , f i , i = 0, ..., n
j=0
In this case
84 2 Interval Interpolation
⎡ ⎤
1 x0 · · · x0n
⎢1 x1 · · · x1n ⎥
⎢ ⎥
A=⎢. .. .. ⎥ = B + C
⎣ .. . . ⎦
1 xn · · · xnn
And by assuming that for every i, points xi are distinct, then det(B + C) = 0.
2.2.1 Theorem
det(S) = det(B − C) · det(B + C)
Proof If we subtract the last n + 1 rows from the first n + 1 rows, respectively, we
have
B −C C − B
det(S) = det
C B
And now if we add the first n+1 columns to the second n+1 columns, respectively,
we have
B −C 0
det(S) = det
C B +C
2.2.2 Theorem
S1 ≥ 0 S2 ≤ 0
Let S be nonsingular. Then the unique solution X of S X = Y →
S2 ≤ 0 S1 ≥ 0
is always an interval vector for arbitrary vector Y, if S −1 is nonnegative.
From X = S −1 Y, we have
BY − CY = X
(2.9)
−CY − BY = X
Thus
X − X = (B + C) Y − Y ≥ 0 (2.10)
Because S −1 i j = ti j ≥ 0 and Y − Y ≥ 0. Since Y is monotonically decreasing
and Y is monotonically increasing, Eq. (2.9) is also necessary and sufficient for X
and X to be monotonically decreasing and increasing, respectively. The bounded left
continuity of X and X is obvious since they are linear combinations of Y and Y. The
proof in completed.
2.2.3 Theorem
also
2n
n
n
si j = si j + sn+i j , j = 1, 2, . . . , 2n
i=1,i= j i=1, j= j i=1,i= j
If si j > 0, i, j = 1, 2, . . . , n then
86 2 Interval Interpolation
⎧
⎪ 2n
⎪
⎪ si j
n
⎨
si j = i=1,i= j
⎪ 2n
i=1,i= j ⎪
⎪ si,n+ j
⎩
i=1,i= j
⎧
⎪
n
si j + sn+i, j
n
⎪
⎨
i=1,i= j i=1,i= j
=
n
n
⎪
⎪ si,n+ j + sn+i,n+ j , j = 1, 2, . . . , n
⎩
i=1,i= j i=1,i= j
From (2.11)
⎧
⎪
n
si j < a j j = s j j
⎪
⎨
2n
si j i=1,i= j
⎪
n
i=1,i= j ⎪
⎩ si j < a j j = sn+i,n+ j , j = 1, 2, . . . , n
i=1,i= j
Then
2n
si j < s j j , j = 1, 2, . . . , n
i=1,i= j
2n
2n
2n
si j = si j + sn+i, j
i=1,i= j i=1,i= j i=1,i= j
2n
2n
si j = ai j < s j j = a j j , j = 1, . . . , n
i=1,i= j i=1,i= j
Then
The results in the matrix form of the Jacobi iterative technique are X k+1 = P X k +
C where
−1
−D1−1 (L 1 + U1 ) −D1−1 S2 D1 Y X
= , C = , X =
−D1−1 S2 −D1−1 (L 1 + U1 ) D1−1 Y X
Then
k+1 t
The elements of X k+1 = (X k+1 , X ) are
⎡ ⎤
1
i−1 n n
x ik+1 = = ⎣ yi − si, j x ik+1 − si, j x ik − si,n+ j x kj ⎦
si, j j=1 j=i+1 j=1
⎡ ⎤
1
i−1 n n
x k+1
j = = ⎣ yi − si, j x k+1
j − si, j x kj − si,n+ j x ik ⎦
si, j j=1 j=i+1 j=1
k = 1, 2, . . . i = 1, 2, . . . , n
The results in the matrix form of the Gauss Sidel iterative techniques are.
X k+1 = P X k + C where
−(D1 + L 1 )−1 U1 −(D1 + L 1 )−1 S2
P= ,
−(D1 + L 1 )−1 S2 −(D1 + L 1 )−1 U1
−(D1 + L 1 )−1 Y X
C= , X =
−(D1 + L 1 )−1 Y X
From Theorems (2.2.3) (2.2.2) both the Jacobi iterates and Gauss Sidel iterates
are
converge
to the unique solution of X = A−1 Y , for any X 0 , where X ∈ R 2n and
X , X ∈ E . The stopping criterion with tolerance > 0 is
n
k+1 k
X −X X k+1 − X k
k+1
< , < , K = 0, 1, . . .
X X k+1
2.2.4 Example
⎡ ⎤
x1
⎢x ⎥
X =⎢ 2 ⎥ −1
⎣ x1 ⎦ = S Y
x2
⎡ ⎤
+0.1250 −0.1250 −0.3750 +0.3750
⎢ −0.3750 −0.3750 −0.3750 −0.1250 ⎥
=⎢
⎣ −0.3750
⎥
+0.3750 +0.3750 −0.1250 ⎦
+0.1250 −0.1250 −0.3750 +0.3750
The Hausdorff distance of solutions with = 10−2 , in the Jacobi method is 0.0027
and in the Gauss Sidel method is 2.31180e − 004.
2.2.5 Example
x1 = [−0.4125, 0.0351]
x2 = [0.9125, 1.1076]
x3 = [−0.6969, −0.7353]
90 2 Interval Interpolation
By the fact that x3 is not an interval number, The interval solution in this case is
a weak solution given by
u 1 = [−0.4125, 0.0351]
u 2 = [0.9125, 1.1076]
u 3 = [−0.7353, −0.6969]
The Hausdorff distance of solution with = 10−3 , in the Jacobi method is 0.0145
and in the Gauss Sidel method is 0.0139.
Chapter 3
Orthogonal Polynomials and Least
Square Approximation
Different types of polynomials play important roles in applied and numerical math-
ematics. The Legendre, Laguerre and Hermit polynomials are introduced in the
theory of integration just like the way that Chebyshev polynomials are used in the
theory of approximation. All of these polynomials are orthogonal families and also
satisfy certain second-order differential equations. Moreover, by using each three
terms in successive order, a general recursive relationship can be defined for them.
In the following, we will show how these properties are generally obtained when the
interval as a domain and weight function are known.
Suppose [a, b] is an arbitrary interval, and w(x) is a definite positive function in this
interval. The inner product of two definite functions p(x) and q(x) in the interval
[a, b] in continuous and discrete states, respectively, is defined by two following
terms:
b
p(x), q(x) = ∫ w(x) p(x) · q(x)d x (3.1)
a
N
p(x), q(x) = w(xn ) p(xn ) · q(xn ) (3.2)
n=1
The second formula is out of the scope of this book. However, we assume in the
first formula that for all the given functions of p(x) and q(x), this integral exists (at
least one improper integral).
3.1.3 Example
Assuming w(x) = 1, it can be easily verified that functions p(x) = 1 and q(x) = x
are orthogonal on the interval [−1, 1].
3.1.4 Example
1 x
p(x), q(x) = ∫ √ dx
−11 − x2
1 1
= − · 2 1 − x2 =0
2 −1
3.1.5 Example
The functions p(x) = sin nx and q(x) = cos mx, for the integers m and n, where
m = n, are orthogonal on the interval [0, 2π ], relative to the weight function 2π
1
.
3.1.6 Definition
∞
Sequence { pi }i=0 (not necessarily definite), where pi ’s are real polynomials of exactly
degree i, is called orthogonal when all pi s are perpendicular to each other, in other
words:
(1) For each i
3.1 Orthogonal Polynomials 93
where, αi = 0.
(2) For every i = j, we have
pi (x), p j (x) = 0
3.1.7 Example
p0 (x), p1 (x) = 0
1
p0 (x), p2 (x) = ∫ 3x 2 − 1 d x = 0
−1
1
p1 (x), p2 (x) = ∫ 3x 2 − 1 d x = 0
−1
b
μk = ∫ w(x)x k d x, k = 0, 1, ... (3.3)
a
3.1.8 Theorem
Proof Suppose that α0 , ..., αn are arbitrary real numbers such that for every x in the
interval [a, b] we have:
n
Since p(x) = α j p j (x) becomes zero at any point on [a, b], so the equation
j=0
has p(x) = 0 infinite roots. On the other hand, we know that p(x) is a polynomial
of degree n, so p ≡ 0 and the coefficient of each power of x is zero. Every p j is of
exactly degree j and therefore,
n−1
p(x) = αn x n + α j p j (x)
j=0
α0 = · · · = αn = 0
Suppose that f ∈ c[a, b], and pn is a polynomial of at most degree n that minimize
⎛ b ⎞ 21
f − pn 2 = ⎝ ( f (x) − pn (x))2 d x ⎠
a
n
pn (x) = an x n + an−1 x n−1 + · · · + a1 x + a0 = ak x k
k=0
and define:
2
b
n
E(a0 , ..., an ) = ∫ f (x) − ak x k
dx
a
k=0
The goal is to find the real coefficients a0 , ..., an that minimize E. We know that the
necessary and sufficient condition for E to be minimized by the numbers a0 , ..., an
is that:
∂E
= 0, j = 0, ..., n
∂a j
3.1 Orthogonal Polynomials 95
then
∂E b n
b
= −2 ∫ x j f (x)d x + 2 ak ∫ x j+k d x = 0
∂a j a
k=0
a
n
b b
ak ∫ x j+k d x = ∫ x j f (x)d x, j = 0, ..., n
a a
k=0
The above equations are called normal equations. If f ∈ c[a, b] and a = b, the
system of equations always has a unique solution.
3.1.10 Example
As a result:
b b j+k+1 − a j+k+1
∫ x j+k d x =
a j +k+1
the resulted system does not have a proper numerical solution due to the
rounding error, or better to say, the matrix can be ill-conditioned.
(3) We cannot generally claim that the calculating method that used for obtaining
the best nth degree polynomial, i.e., p n , is quite useful for calculating, too.
So, let us consider another method to approximate the least squares as follows.
In this method, as soon as pn is known, pn+1 is easily obtained.
Suppose n be a set of all polynomials of at most degree n and Q j represents
polynomials of degree j. For set {Q 0 , ..., Q n }, the real numbers a0 , ..., an must be
calculated such that the following formula is minimized:
2
b
n
E(a0 , ..., an ) = ∫ f (x) − a − k Q k (x) dx
a
k=0
b
n
b
= ∫( f (x))2 d x − 2 ak ∫ f (x)Q k (x)d x
a a
k=0
n
n
b
+ a j ak ∫ Q j (x)Q k (d x)d x
a
j=0 k=0
∂E
= 0, j = 0, ..., n
∂a j
n
b b
a j ∫ Q j (x)Q k (x)d x = ∫ f (x)Q k (x)d x, k = 0, ..., n
a a
j=0
Now if
3.1 Orthogonal Polynomials 97
b
∫ Q j (x)Q k (x)d x = 0, j = k
a
b b
ak ∫(Q k (x))2 d x = ∫ f (x)Q k (x)d x, k = 0, ..., n
a a
Therefore,
Given the Definition (3.1.6) and w(x) = 1 and the discussion above, we state the
following theorem without proof.
3.1.11 Theorem
n
p(x) = ak Q k (x)
k=0
where
3.1.12 Example
1
Q 0 (x) = √
2π
1
Q 2k (x) = √ cos kx, k = 1, ..., n
π
1
Q 2k−1 (x) = √ sin kx, k = 1, ..., n
π
n
Tn (x) = ak Q k (x)
k=0
where
π
ak = ∫ f (x)Q k (x)d x, k = 0, ..., 2n
−π
2 (−1)k − 1
n
π
Tn (x) = − cos kx
2 π k=0 k2
Chapter 4
Integral Equations
4.1 Introduction
4.1.2 Example
b
(1) y(s) = ∫ k(s, t)x(t)dt, a ≤ s ≤ b
a
b
(2) x(s) = y(s) + ∫ k(s, t)x(t)dt, a ≤ s ≤ b
a
b
(3) x(s) = ∫ k(s, t)x 2 (t)dt, a ≤ s ≤ b
a
b
(4) x(s) = y(s) + ∫ k(s, t, x(t))dt, a ≤ s ≤ b
a
b b
(5) x(s) = y(s) + ∫ ∫ k(s, t)x(t)x(u)dtdu, a ≤ s ≤ b.
a a
In each equation of Example (4.1.2), function x(·) is unknown. Equations (1) and
(2) can also be written in functional form:
where L is a linear operator in the following form that comprises the unknown
function; i.e., it is the operator that acts on the unknown function. For example, for
(1), L is the same integration, and for (2), L is defined as
b
L : I −∫
a
b
L(x(s)) = ∫ k(s, t)x(t)dt
a
b
L(x(s)) = x(s) − ∫ k(s, t)x(t)dt
a
The integral equations, such as Eq. (1), where x(s) (unknown function) appears
only under the integral sign and does not appear anywhere else, are called integral
equations of the first type.
4.1 Introduction 101
Also, equations such as Eq. (2), where the unknown function is both inside and
outside the integral, are called integral equations of the second type.
4.1.4 Definition—Kernel
In each equation of (1) and (2), the function k(s, t) is called the kernel of the equation,
which is a square plane for every a ≤ s ≤ b and every a ≤ t ≤ b.
Obviously, in each of equations that are presented in Example (4.1.2), the kernel
function k(s, t) and y(s) are known.
L(x(s))
b
x(s) = ∫ k(s, t)x(t)dt
a
Volterra equations of second type and Eq. (8) is homogeneous linear Volterra equation
of second type.
The mentioned contents can be expressed another way in the form of the following
definition:
k(s, t) = 0, t > s
b
x(s) = f (s) + λ k(s, t)x(t)dt
a
Equation
⎧
⎨ b
x (t) + p(x)x(t) + y(t) = ∫ k(s, u)x(u)du
a (4.2)
⎩
x(a) = y0
In this section, we want to express the relationship between the integral equation and
the differential equation. Also, we will see that the solution of a differential equa-
tion with an initial value is a solution of Volterra integral equation. The differential
equation can be assumed to be the first order with an initial condition or the second
order with two initial conditions; in both cases, the answer will be a Volterra integral
equation with one integral sign and two integral signs, respectively. On the other
hand, the equations in which there are both differential and integral operators are
also examined.
Before expressing this relationship, we express and prove the following lemma.
4.1.10 Lemma
Then
x
yx = F(x, x) + ∫ Fx (x, u)du
0
x+x x
y(x + x) − y(x) = ∫ F(x + x, u)du − ∫ F(x, u)du
0 0
x x+x
= ∫ F(x + x, u)du + ∫ F(x + x, u)du
0 x
x
− ∫ F(x, u)du
0
x
= ∫(F(x + x, u) − F(x, u))du
0
x+x
+ ∫ F(x + x, u)du
x
1
Now, we multiply both sides of the above relation by x
:
104 4 Integral Equations
If we take the limit of the both sides of the above relationship, we will have:
where f (x, y) is continuous with respect to (x, y). By integrating both sides of the
relation (4.3) on the interval [0, x] and using the change of variables, we have:
x x
∫ y (t)dt = ∫ f (t, y(t))dt
0 0
4.1 Introduction 105
In this case
x
y(x) = y0 + ∫ f (t, y(t))dt (4.4)
0
4.1.11 Lemma
Suppose that f (x, y) is continuous with respect to (x, y). In this case, the solution
of Eq. (4.3) is a nonlinear Volterra integral equation of the second type.
Now, we consider the second-order differential equation and examine its relation-
ship with the integral equation. Suppose that
d2 y
dx2
= y (x) = f (x, y(x))
(4.5)
y(0) = y0 , y (0) = y1
As before, if we integrate both sides of Eq. (4.5) on the interval [0, x], we will
have:
x
y (x) = y1 + ∫ f (t, y(t))dt
0
Multiplying Eq. (4.6) by du and reintegrating on the interval [0, x], we have:
x x x u
∫ y (u)du = ∫ y1 du + ∫ ∫ f (t, y(t))dtdu
0 0 0 0
x
= ∫(x − t) f (t, y(t))dt
0
= h(x)
To prove this claim (g(x) = h(x)), it is sufficient to prove g (x) = h (x), because
in this case
g(x) = h(x) + c
Because, by assuming
u
F(y, u) = ∫ f (t, y(t))dt
0
we have:
x
g (x) = F(y, x) + ∫ Fy (y, u)du
0
x
= ∫ f (t, y(t))dt + 0 (4.8)
0
Because, assuming
We have
x
h (x) = F(x, x) + ∫ F (x, t)dt
0
x
= 0 + ∫ f (t, y(t))dt (4.9)
0
4.1 Introduction 107
So, according to Eqs. (4.8) and (4.9), the sentence is true. Therefore, Eq. (4.7) is
obtained as follows:
x
y(x) = y0 + y1 x + ∫(x − t) f (t, y(t))dt (4.10)
0
4.1.12 Lemma
The solution of the second-order differential equation with the boundary conditions
(4.5) is a nonlinear integral equation of the second type.
We now consider the following second-order differential equation:
y (x) = f (x, y(x))
0≤x ≤l (4.11)
y(0) = α, y(l) = β
First, according to Lemma (4.1.12), the general solution for Eq. (4.11) is as follows
x
y(x) = A + Bx + ∫(x − t) f (t, y(t))dt (4.12)
0
Obviously, Eq. (4.12) holds both in the differential equation and in the boundary
conditions, i.e.,
y(0) = α ⇒ α = A
l
y(l) = β = α + Bl + ∫(l − t) f (t, y(t))dt (4.13)
0
So, by obtaining B from Eq. (4.13) and substituting it in Eq. (4.12), we have:
(β − α)x x
y(x) = α + + ∫(x − t) f (t, y(t))dt
l 0
x l
− ∫(l − t) f (t, y(t))dt
l 0
(β − α)x x
=α+ + ∫(x − t) f (t, y(t))dt
l 0
108 4 Integral Equations
x x x l
− ∫(l − t) f (t, y(t))dt − ∫(l − t) f (t, y(t))dt
l 0 l x
(β − α)x x x(l − t)
=α+ + ∫ (x − t) − f (t, y(t))dt
l 0 l
l x
− ∫ (l − t) f (t, y(t))dt
x l
(β − α)x x t(x − l)
=α+ +∫ f (t, y(t))dt
l 0 l
l x
+ ∫ (t − l) f (t, y(t))dt
x l
l
y(x) = z(x) + ∫ k(x, t) f (t, y(t))dt (4.14)
0
where
t(x−l)
, 0≤t ≤x
k(x, t) = l
x(t−l)
l
, x ≤t ≤l
and
α
β−
z(x) = α + x
, α, β ∈ R
l
Obviously, (4.14) is also a nonlinear Fredholm integral equation of second type.
As it can be seen, the integral kernel, k(x, t), holds in the following equation
4.1.13 Definition
in this case
y (x) = w(x) − p(x)y(x) = f (x, y(x))
y(0), y(l) are known
l
y(x) = z(x) + ∫ k(x, t)(w(t) − p(t)y(t))dt
0
l l
= z(x) + ∫ k(x, t)w(t)dt − ∫ k(x, t) p(t)y(t)dt
0 0
So, assuming
l
T (x) = z(x) + ∫ k(x, t)w(t)dt
0
we have:
l
y(x) = T (x) − ∫ k(x, t) p(t)y(t)dt
0
where k(x, t) p(t) is an asymmetric kernel but k(x, √t) is the same symmetric kernel.
If we multiply both sides of the above equation by p(x), we will have:
l
p(x)y(x) = p(x)T (x) − ∫ p(x)k(x, t) p(x) p(x)y(t)dt
0
Now, assuming
p(t)y(t) = y(t)
we have:
l
y(x) = p(x)y(x) = p(x)T (x) − ∫ p(x)k(x, t) p(x)y(x)dt
0
Since a necessary condition for integration is the continuity of the function under the
integral sign, so we must discuss the continuity of each of functions x(t) and k(s, t).
Also, for the integral of the integral equation to exist, the above functions must be
located in space L 2 . In this section, we assume that x(t) is a complex function in
terms of t on the interval [a, b].
4.2.1 Definition
4.2.2 Definition
The two functions x(t) and y(t) of L 2 -space are almost equal everywhere if the size
of the set A = {t|x(t)
= y(t)} is equal to zero, i.e., m(A) = 0, and we can write
x(t) =0 y(t) and if m(B) = 0 where
B = {t|x(t) = 0}
In this case, the function x(t) is almost equal to zero everywhere, and we can
write x(t) =0 0. We also can say that x(t) is not more than y(t) everywhere, if the
size of the set
C = t|x(t) y(
t)
4.2.3 Definition
21
b
x2 = ∫|x(t)|2 dt
a
4.2.4 Definition
If x(t) is a continuous function on the interval [a, b], then we define xc as follows:
If x(t) and y(t) are functions in the L 2 -space, then x(t) · y(t) is integrable and
b
∫ x(t)y(t)dt ≤ x · y (4.16)
a
and since the right integral of the above inequality exists, so the left integral also
exists. To prove it, it suffices to consider x(t) and y(t) as real and negative functions,
in which case we have:
1 2
x(t) · y(t) ≤ x (t) + y 2 (t)
2
It is easy to obtain the integrability of x(t) · y(t) from the above equation. For
every real λ, we have the following equation:
112 4 Integral Equations
b
0 ≤ ∫(λx(t) + y(t))2 dt
a
b b b
= λ2 ∫ x 2 (t)dt + 2λ ∫ x(t) · y(t)dt + ∫ y 2 (t)dt
a a a
b
= λ2 x2 + 2λ ∫ x(t) · y(t)dt + y2
a
The above equation is a quadratic equation in terms of λ. In order for the above
inequality to exist, or must be negative, because x2 > 0, therefore
2
b
= ∫ x(t) · y(t)dt − x2 · y2 < 0
a
and as a result:
b
∫ x(t)y(t)dt ≤ x · y
a
b
The left side of the equation, i.e., ∫ x(t)y(t)dt, shows the same inner product of
a
the functions x(t) and y(t), which in the more general case can be written as follows
b
x, y = ∫ x(t) · y(t)dt
a
Some properties of inner product can also be easily proved. For example,
b
(I ) |x, y| = ∫ x(t) · y(t)dt
a
b
= ∫ x(t) · y(t)dt
a
= y, x
and
b
(I I ) x, y = ∫ x(t) · x(t)dt
a
4.2 Continuous Functions … 113
b
= ∫|x(t)|2 dt
a
= x2
also
n
b n
(I I I ) x, λi yi = ∫ x(t) λi yi (t)dt
a
i=1 i=1
n
b
= λi ∫ x(t)yi (t)dt
a
i=1
n
= λi x, yi
i=1
4.2.6 Theorem
If x(t) and y(t) are functions in the L 2 -space, then x(t) + y(t) is also a function in
the space L 2 and we have:
Therefore, we have:
b
x + y2 = ∫(x + y)(x + y)dt
a
b
= ∫|x(t) + y(t)|2 dt
a
b
≤ ∫(|x(t)| + |y(t)|)2 dt
a
b b b
= ∫|x(t)|2 dt + ∫|y(t)|2 dt + 2 ∫|x(t)| · |y(t)|dt
a a a
≤ x + y + 2x · y
2 2
so, we have:
Using Definition (4.2.4), we can easily prove the Theorem (4.2.6) for ·c as well,
i.e.,
The relationship between inner product and continuous norm can also be expressed
and proved as follows. We have:
b b
∫ x(t) · y(t)dt ≤ ∫|x(t)| · |y(t)|dt
a a
b
≤ xc · yc dt
a
= (b − a)xc · yc
therefore
According to Eq. (4.19) and Theorem (4.2.6), it can be claimed that if x(t) and
y(t) are functions of L 2 , then each linear combination of them, such as
λx(t) + μy(t), λ, μ ∈ R
is also a function of L 2 . So, all functions of L 2 form a complex vector space. This is
also true for continuous functions. Another property that can be mentioned for the
4.2 Continuous Functions … 115
inner product of two functions is that the inner product of the two functions is linear
in terms of first components but is nonlinear in terms of second components, i.e.,
n
n
λi xi , y = λi xi , y
i=1 i=1
n
n
x, λi xi = λi x, yi
i=1 i=1
b
y(s) = ∫ k(s, t)x(t)dt, a ≤ s ≤ b (4.20)
a
in this case
y = kx ∈ C[a, b]
where y = kx is the compact form of Eq. (4.20). From the continuity of x(t), we
have:
∀ε∃δ∀t∀t t − t < δ ⇒ x(t) − x t < ε
Given the continuity of thee function x(t) and the compactness of [a, b], it can
be said that this function is bounded. That means:
∃m∀t(|x(t)| ≤ m)
d (s, t), s , t = (s − s )2 + (t − t )2 = s − s < δ
and
ε
k(s, t) − k s , t < ε , ε =
(b − a)m
Therefore,
b
y(s) − y s ≤ ∫ k(s, t) − k s , t x(t)dt
a
b
≤ ∫ ε · mdt
a
= ε · m · (b − a) = ε
It can be shown that the continuous kernel satisfies the following property.
because
b
y(s) = ∫ k(s, t)x(t)dt
a
therefore
4.2 Continuous Functions … 117
b
|y(s)| ≤ ∫|k(s, t)| · |x(t)|dt
a
b
≤ ∫ sup |k(s, t)| · xc dt
a s, t
= kc · xc
Now, we have
that means
thus
b
y(s) = ∫ k(s, t)x(t)dt, a ≤ s ≤ b
a
b
I (r ) = ∫ h(r, s)y(s)ds
a
in this case
b
I (r ) = ∫ h(r, s)(k(s, t)x(t)dt)ds
a
b b
= ∫ ∫ h(r, s)k(s, t)x(t)dtds
a a
b b
= ∫ ∫ h(r, s)k(s, t)ds x(t)dt
a a
we assume that
b
I (r ) = ∫ l(r, t)x(t)dt
a
where
b
l(r, t) = ∫ h(r, s)k(s, t)ds
a
= (hk)(r, t) (4.22)
l = hk
4.3 Production of Two Kernels 119
4.3.1 Lemma
b
|l(r, t)| ≤ ∫|h(r, s)| · |k(s, t)|ds
a
b
≤ ∫ sup |h(r, s)| · sup |k(s, t)|ds
a r, s s, t
Then
Given that the above equation holds for every r and every t, therefore
as a result
hk = kh
1
(hk)(r, t) = ∫ h(r, s)k(s, t)ds
0
120 4 Integral Equations
1
= ∫ r s 2 · s 2 tds
0
1
= r · t
5
But
1
(hk)(r, t) = ∫ k(r, s)h(s, t)ds
0
1
= ∫ r 2 s · st 2 ds
0
1
= r2 · t2
3
So, in general, we can say that hk
= kh, unless for t = 0. It can also be shown
that
We prove the part (1) and leave the proof of the other two parts to the reader. We
know:
b
[(hk)l](r, t) = ∫(hk)(r, s)l(s, t)ds
a
So, we have:
b b
[(hk)l](r, t) = ∫ ∫ h(r, z)k(z, s)dz l(s, t)ds
a a
b b
= ∫ h(r, z) ∫ k(z, s)l(s, t)ds dz
a a
b
= ∫ h(r, z)(kl)(z, t)dz
a
= h(kl)(r, t)
4.3.2 Remark
b
∫ k(s, t)x(t)dt = 0
a
∀x(kx = 0 ⇒ k = 0) (4.25)
k1 x = k2 x ⇒ k1 = k2
b b
∫ k(s, t)k(s, t)dt = ∫|k(s, t)|2 dt
a a
If we assume
b
∫|k(s, t)|2 dt = 0
a
k(s, t) = 0
∀t∀s(k(s, t) = 0)
4.3.3 Lemma
∀t∀s(k(s, t)x(t) = 0 ⇒ k(s, t) = 0)
4.3.4 Definition
b b
(1) ∫ ∫|k(s, t)|2 dtds < ∞
a a
b
(2) ∀s ∫|k(s, t)|2 dt < ∞, is measurable
a
b
(3) ∀t ∫|k(s, t)|2 ds < ∞, is measurable
a
We now state the Fubini, Tonley and Hopson theorems without proof.
b
∫ f (s, t)dt
a
and
b b b b
∫ ∫ f (s, t)dsdt = ∫ ds ∫ f (s, t)dt
a a a a
b b
= ∫ dt ∫ f (s, t)ds
a a
4.3.7 Lemma
If x, k ∈ L 2 (a, b) and
b
y(s) = ∫ k(s, t)x(t)dt
a
Then
y ∈ L 2 (a, b)
b
where ∫|k(s, t)|2 dt ∈ L 2 , then
a
b b b b
∫|y(s)| ds ≤ ∫ ∫|k(s, t)| dtds ∫|x(t)|2 dt < ∞
2 2
a a a a
where
21 21
b b b
k2 = ∫ ∫|k(s, t)| dtds , x2 = ∫|x(t)| dt
2 2
a a a
In this section, we want to study the Fredholm integral equation of the second type in
the matrix form. We discuss the availability and uniqueness of the answer, its adjoint
equations, regular values and other properties.
b
x(s) = y(s) + λ ∫ k(s, t)x(t)dt, λ
= 0 (4.26)
a
or
x = y + λkx, λ = 0
therefore
x − λkx = y
then
4.4 Fredholm Integral Equation of the Second Type 125
(I − λk)x = y
x = (I − λk)−1 y
We know that (I − λk)−1 is itself a matrix. In order to write the answer of the
integral equation as another integral equation, suppose
x = (I − λk)−1 y := (I + λh)y
(I − λk)x = y (4.27)
and
x = (I + λh)y (4.28)
(I − λk)(I + λh)y = y
so
(I − λk)(I + λh) = I
thus
I − λk + λh − λ2 kh = I
Then, we have
h − k = λkh
x = (I + λh)(I − λk)x
that means
I = (I + λh)(I − λk) = I + λh − λk − λ2 kh
126 4 Integral Equations
Then
h − k = λhk (4.29)
As a result, provided that the solution of the Fredholm integral equation of the
second type exists, we have:
h − k = λkh = λhk
h λ − k = λh λ k = λkh λ (4.30)
4.4.2 Theorem
h 1 − k = λh 1 k = λkh 1
h 2 − k = λh 2 k = λkh 2
h 1 − h 2 = λ(h 1 − h 2 )k = λk(h 1 − h 2 )
h = λhk = λkh
By forming hh 1 , we have:
4.4 Fredholm Integral Equation of the Second Type 127
hh 1 = h(k + λkh 1 )
= hk + λhkh 1
= hk + hh 1
Then
hk = 0 & h = λhk = 0
So,
h = h1 − h2 = 0
4.4.3 Theorem
x = y + λkx (4.31)
x = y + λhy (4.32)
So, we show that x holds in Eq. (4.31); i.e., it is the answer. Now we show that if
Eq. (4.31) holds, x is in the form of Eq. (4.32).
Now, since x holds in Eq. (4.31), and Eq. (4.32) must be hold, it is unique.
h λ − k = λh λ k = λkh λ
So
h λ = k + λkh λ
= k + λk(k + λh λ k)
= k + λk 2 + λ2 kh λ k
4.5.1 Theorem
where
1
k(s) = ∫|k(s, t)|2 dt 2
1
e(s) = ∫|l(s, t)|2 dt 2
4.5 Continuous Kernels 129
Proof We have
therefore
b
(hx)(u) = ∫ h(u, v)x(v)dv
a
b
= ∫ h(u, v)l(v, t)dv
a
then
b
hx, y = ∫(hx)(u)y(u)du
a
b
= ∫(hx)(u)k(s, u)du
a
b b
= ∫ ∫ h(u, v)l(v, t)k(s, u)dudv
a a
b b
= ∫ ∫ h(u, v)h(v, t)du k(s, u)dv
a a
= khl(s, t)
= e(t)
and
1
y = ∫|y(u)|2 du 2
2
21
= ∫ |k(s, u)| du
130 4 Integral Equations
1
= ∫|k(s, u)|2 du 2
= k(s)
∀ε∃δ∀s∀s ∀t ∀t
2
ε > 0, δ > 0, (s − s ) + (t − t ) < δ ⇒ k(s, t) − k s , t < ε
2
ε
ε= √
2k1 h λ b − a
where
∀s(|k(s)| ≤ k1 )
Because k(s), for every t, is constant and continuous, on a closed and bounded
interval, it takes its limit. Given that:
t − t ≤ (s − s )2 + (t − t )2
s − s ≤ (s − s )2 + (t − t )2
therefore
kh λ k(s, t) − kh λ k s , t ≤ kh λ k(s, t) − kh λ k s, t
+ kh λ k s, t − kh λ k s , t
= kh λ k(s, t) − k s, t
+ kh λ k s, t − k s , t
21
b
≤ h λ k(s) ∫ k(s, t) − k s, t ds 2
a
21
b 2
+ h λ k(s) ∫ k(s, t) − k s , t ds
a
ε ε
≤ h λ k1
2k1 h λ 2k1 h λ
=ε
4.5 Continuous Kernels 131
Therefore, kh λ k is continuous.
Sometimes, the work field is complex numbers, so according to the needs of the
following subjects, we discuss adjoint kernels.
4.6.1 Definition
k ∗ (s, t) = k(s, t)
So, k = k ∗ .
After mentioning and reviewing the above properties, we will examine other
properties of the adjoint kernels.
b
= ∫ k(t, u) · l(u, s)du
a
b
= ∫ l ∗ (s, u)k ∗ (u, t)du
a
∗ ∗
= l k (s, t)
Then, (kl)∗ = l ∗ k ∗ .
It can also be shown that k ∗ x, y = x, ky.
∗
b
∗
b
k x, y = ∫ ∫ k (s, t)x(t)dt y(s)ds
a a
b b
= ∫ ∫ k(t, s)x(t)dt y(s)ds
a a
b b
= ∫ x(t) ∫ k(t, s)y(s)ds dt
a a
= x, ky
4.6.4 Remark
Adjoint equations can be introduced as with adjoint kernels. For the equation x =
y + λkx, the equation u = v + λk ∗ u can also be considered as an adjoint equation,
so that if h is a solver kernel of k corresponding to λ, then h ∗ is the solver kernel of
k ∗ corresponding to λ. Because
4.6 Adjoint Kernels 133
h − k = λkh = λhk
Therefore,
h ∗ − k ∗ = λh ∗ k ∗ = λk ∗ h ∗
b
x(s) = λ ∫ k(s, t)x(t)dt (4.33)
a
x1 = λkx1
x2 = λkx2
therefore
α1 x1 = λk(α1 x1 )
α2 x2 = λk(α2 x2 )
as a result
0 = α1 x1 + α2 x2 = λk(α1 x1 + α2 k2 )
4.6.6 Definition
The Set
x|x = λkx, x ∈ L 2
134 4 Integral Equations
4.6.7 Lemma
b
∫ k(s, t)x(t)dt = 0
a
∀s(x(s) = 0)
4.6.8 Remark
4.6.9 Theorem
x = y + λhy = 0 + λh0 = 0
which is a contradiction.
4.6 Adjoint Kernels 135
4.6.10 Theorem
If x(s) is the characteristic function of the kernel k(s, t) ∈ c[a, b] × [a, b], then x is
continuous.
Proof We have,
b
x(s) = λ ∫ k(s, t)x(t)dt, x ∈ L 2
a
we prove that:
∀ε∃δ∀s∀s s − s < δ ⇒ x(s) − x s < ε
therefore
b b
x(s) − x s = λ ∫ k(s, t)x(t)dt − λ ∫ k s , t x(t)dt
a a
b
= λ ∫ k(s, t) − k s , t x(t)dt
a
as a result
b
x(s) − x s ≤ |λ| · ∫k(s, t) − k s , t |x(t)|dt
a
21
b 2
≤ |λ| · x ∫k(s, t) − k s , t dt
a
Due to the continuity of the kernel k, for every arbitrary ε , there is a δ that for
every s,
(s − s )2 + (t − t )2 < δ
then
k(s, t) − k s , t < ε
therefore,
1
x(s) − x s < |λ| · x (b − a)ε2 2
136 4 Integral Equations
as a result
ε
x(s) − x s ≤ |λ| · x · <ε
1 + |λ| · x
4.6.11 Example
In this case,
1 1
∫ k(s, t)x(t)dt = ∫(2s − 5st)t − 3 dt
1
0 0
1 1
= 2s ∫ t − 3 dt − 5s ∫ t 3 dt = 0, t
= 0
1 2
0 0
4.6.12 Definition
∃ p(s)∀ε∃N ∀n∀s
p(s) ≥ 0 & p(s) ∈ L 2 & n ≥ N ⇒ |xn (s) − x(s)| ≤ εp(s)
4.6 Adjoint Kernels 137
We say that the sequence {xn } convergence to x point wisely, if and only if
4.6.15 Example
Suppose
1
1 , t
= 0, t ∈ [0, 1]
xn (t) = nt 3
0, t = 0
1 , t
= 0
1
p(t) = t 3
0, t = 0
x(s) = 0, s ∈ [0, 1]
1 1
∫ p 2 (s)ds = ∫ s − 3 ds = 3 > 0
2
0 0
1 1 1 −2 3
∫ xn2 (s)ds = ∫ s 3 ds =
0 n 0 n
1
|xn (s) − x(s)| = |xn (s)| = p(s)
n
we have
1
∀ε∃N ∀n n ≥ N ⇒ < ε
n
then
1
∀ε∃N ∀n∀s n ≥ N ⇒ |xn (s) − x(s)| = p(s) ≤ ε p(s)
n
which has relatively uniformly convergence but does not have uniformly conver-
gence, because to prove it, we must prove the following formula:
1
∀ε∃N ∀n∀s n ≥ N ⇒ 1 <ε
nt 3
1 1
= =1<1
N · N1
1
nt 3
which is a contradiction.
If the convergence is relatively uniform, it may not be uniform.
Relatively uniformly convergence can also be expressed by the following
definition.
4.6 Adjoint Kernels 139
4.6.16 Definition
A necessary and sufficient condition for xn (s) to have an uniform relative convergence
to x(s) is that
∀ε∃n 0 ∀n∀m∀s
(n ≥ n 0 & m ≥ n 0 & s ∈ [a, b] ⇒ |xn (s) − x(s)| ≤ εp(s))
4.6.17 Definition
∞
n
Series xi (s) converges relatively uniformly to x(s) if and only if series xi (s)
i=1 i=1
converges relatively uniformly to x(s).
The same convergences hold for kernels.
4.6.18 Definition
4.6.19 Theorem
Proof Given that xn (s) has a relatively uniformly convergence to x(s), therefore:
140 4 Integral Equations
b
ε
where ε = 1+l
and 0 ≤ l = ∫ p(s)|y(s)|ds. As the result, we can write:
a
b
|xn (s), y(s) − x(s), y(s)| = ∫ xn (s) − x(s)y(s)ds
a
b
≤ ∫|xn (s) − x(s)||y(s)|ds
a
b
< ε ∫ p(s)|y(s)|ds
a
εl
=
1+l
<ε
4.6.20 Theorem
Proof Because kn (s, t) has a relatively uniformly convergence to k(s, t), so according
to Definition (4.6.18),
as a result
b b
∫ kn (s, t)x(t)dt − ∫ k(s, t)x(t)dt
a a
b
≤ ∫|kn (s, t) − k(s, t)||x(t)|dt
a
b
≤ ε ∫ p(s, t)|x(t)|dt
a
= ε q(s)
4.6 Adjoint Kernels 141
ε
where q(s) ≥, 0 q ∈ L 2 and ε = 1+q(s) .
The following theorem points out the relatively uniformly convergence of a
sequence of productions of finite kernels.
4.6.21 Theorem
If l ∈ L 2 and kn (s, t) have a relatively uniformly convergence to k(s, t), then kn l has
a relatively uniformly convergence to kl.
Proof Because kn (s, t) has a relatively uniformly convergence to k(s, t), so according
to the definition of (4.6.18),
as a result
b b
∫ kn (s, u)l(u, t)du − ∫ k(s, u)l(u, t)du
a a
b
≤ ∫|kn (s, u) − k(s, u)||l(u, t)|du
a
b
≤ ε ∫ p(s, u)|l(u, t)|du
a
= ε q(s, t)
ε
where q(s, t) ≥ 0, q ∈ L 2 and ε = 1+q(s,t)
.
4.6.22 Theorem
If the assumptions of Theorem (4.6.21) hold, then lkn has a relatively uniformly
convergence to lk.
Proof The burden of the proof is the responsibility of the readers.
The above-mentioned theorems can be stated about the series. For example, if
∞
xi (s) has a relatively uniformly convergence to x(s), and if we assume yn (s) =
i=1
n
xi (s), then yn (s) has a relatively uniformly convergence to y(s). So
i=1
142 4 Integral Equations
∀ε∃n 0 ∀n∀m∀s
(n ≥ n 0 & m ≥ n 0 & s ∈ [a, b] ⇒ |yn (s) − ym (s)| < εp(s))
∀ε∃n 0 ∀n∀m∀s
n
(n > m ≥ n 0 & s ∈ [a, b] ⇒ xi (s) < εp(s))
i=m+1
4.6.23 Theorem
∞
If xi (s) has a relatively uniformly convergence to x(s) and y ∈ L 2 , then
i=1
∞
xi (s), y(s) converges to x(s), y(s).
i=1
That means:
∞
∞
b b
∫ xi (s)y(s)ds = ∫ xi (s) y(s)ds
a a
i=1 i=1
b
is convergent to ∫ x(s)y(s)ds.
a
∀ε ∃n 0 ∀n∀m∀s
n
n > m ≥ n 0 & s ∈ [a, b] ⇒ xi (s) < ε p(s)
i=m+1
We prove that
b
Suppose l = ∫ p(s)|y(s)|ds, then
a
n
xi (s), y(s) < εl
i=m+1
ε
It suffices to assume ε = 1+l
that in this case, the sentence is true.
4.6.24 Theorem
∞
If kn (s, t) has a relatively uniformly convergence to k(s, t) and x ∈ L 2 , then
n=1
∞ b b
∫ kn (s, t)x(t)dt has a relatively uniformly convergence to ∫ k(s, t)x(t)dt.
n=1 a a
∞
Proof Because kn (s, t) has a relatively uniformly convergence to k(s, t), then
n=1
∀ε∃n 0 ∀n∀m∀s∀t
n
n > m ≥ n 0 & s, t ∈ [a, b] ⇒ ki (s, t) < εp(s, t)
i=m+1
4.6.25 Theorem
∞
If kn (s, t) has a relatively uniformly convergence to k(s, t) and l ∈ L 2 , then
n=1
∞
∞
kn l has a relatively uniformly convergence to kl and lkn has a relatively
n=1 n=1
uniformly convergence to lk.
Proof Given that
b
kn l = ∫ kn (s, u)l(u, t)du
a
∞ b
We must prove that ∫ kn (s, u)l(s, t)du has a relatively uniformly convergence
n=1 a
b
to ∫ k(s, u)l(s, t)du. First, suppose that ε > 0 is arbitrary where:
a
∃n 0 ∀n∀m∀s∀t
n
n > m ≥ n 0 & s, u ∈ [a, b] ⇒ ki (s, u) < εp(s, u)
i=m+1
5.1 Introduction
b
x(s) = y(s) + λ k(s, t)x(t)dt
a
∃h λ : h λ − k = λh λ k = λkh λ
then
h λ (I − λk) = k (5.1)
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 147
T. Allahviranloo and A. Esfandiari, A Course on Integral Equations with Numerical
Analysis, Mathematical Engineering, https://doi.org/10.1007/978-3-030-85350-1_5
148 5 Numerical Solution of Integral Equations
(I − λk)h λ = k (5.2)
The Eqs. (5.1) and (5.2) are matrix equations so that the matrix I −λk is invertible,
so according to the Eq. (5.1), it is concluded:
1
h λ = k(I − λk)−1 = k
I − λk
= k I + λk + (λk)2 + · · · = k + λk 2 + λ2 k 3 + · · ·
1
h λ = (I − λk)−1 k = k
I − λk
= I + λk + (λk)2 + · · · k = k + λk 2 + λ2 k 3 + · · ·
That means that in both cases, we can see that h λ is in terms of the above series.
The determination of h λ as the above series can be done in another way:
x = y + λkx
x1 = y, xn+1 = y + λkxn
So
x2 = y + λky
x2 = y + λk(y + λky) = y + λky + λ2 k 2 y
..
.
And finally
xn+1 = y + λky + λ2 k 2 y + · · · + λn k n y
= y + λ k + λk 2 + λ2 k 3 + · · · + λn−1 k n y
in this case, h λ = k + λk 2 + · · · .
The following theorem discusses the relatively uniformly convergence of
Neumann series.
5.2 Neumann Series 149
5.2.1 Theorem
Suppose that λ is a regular value of the kernel k(s, t)and |λ| · k < 1, in which
case, the following series
where
⎛ b ⎞ 21 ⎛ b ⎞ 21
k(s) = ⎝ |k(s, t)|2 dt ⎠ , k(t) = ⎝ |l(s, t)|2 dt ⎠
a a
then
n−1 n
λ k (s, t) ≤ |λ|n−1 · k n−2 · k(s) · k(t)
≤ |λ|n−1 · kn−2 · k(s) · k(t)
suppose
also
|λ|n−1 · kn−2 = an
n−1 n
So n an is convergent. Therefore, it can be concluded that ∞ n=1 λ k (s, t)
has a uniformly convergence to z(s, t). Therefore, it can be written:
n
i−1 i
n
λ k (s, t) ≤ ai p(s, t) < εp(s, t)
i=m+1 i=m+1
as a result
that means
z − k = λkz, z − k = λzk
5.2.2 Example
b
u, v = u(t)v(t)dt = 0
a
b
k (s, t) =
2
k(s, z)k(z, t)dz
a
5.2 Neumann Series 151
b
= u(s)v(z)u(z)v(t)dz
a
b
= u(s)v(t) u(z)v(z)dz
a
=0
then
k 2 = 0, · · · k n = 0, n ≥ 2
That means that the series is zero from the second term onward and does not
depend on λ and h λ = k.
We employ the Neumann method, which we examined in the Sect. (5.1), to solve the
integral Eq. (5.1). We have:
x = y + λky + λ2 k 2 y + · · ·
x0 = y, xn+1 = y + λkxn
We define:
en = xn − x
xn+1 − x = λk(xn − x)
therefore
en+1 = λken
as a result
so
en = en+1 − (xn+1 − xn )
then
therefore
xn+1 − xn
en ≤
1 − |λ| · k
b
n
k(s, t)x(t)dt
w j k s, t j x t j
a j=1
then
n
x1 (s)
y(s) + λ w j k s, t j y t j (5.3)
j=1
n
x2s (s)
y(s) + λ w j k s, t j y t j (5.4)
j=1
n
x1 (ti )
y(ti ) + λ w j k ti , t j y t j (5.5)
j=1
n
n
x2 (s)
y(s) + λ2 wi w j k s, t j k ti , t j y t j
j=1 i=1
n
+λ wi k(s, ti )y(ti )
i=1
b
x(s) = y(s) + λ k(s, t)x(t)dt
a
b
xn+1 = y(s) + λ k(s, t)xn (t)dt
a
n
= y(s) + λ w j k s, t j xn t j , s = si , i = 1, . . . , n (5.6)
j=1
where
(xn )i = xn (ti ), ki j = w j k ti , t j , yi = y(ti )
then
xn+1 = y + λkxn
x R = y + λkx R (5.7)
154 5 Numerical Solution of Integral Equations
x R = y + λkx R
in this case
x R = (I − λk)−1 y
where
en = x R − xn
In this case,
xn+1 − x n
en ≤ (5.8)
1 − λk
We also faced this error in the iterative method obtained from the Neumann series.
Now for the integration rule R, the error will be as follows:
And we have:
b
x R (s) = y(s) + λ k(s, t)x R (t)dt + E R (λk(s, t)x R (t)) (5.9)
a
We also have:
b
x(s) = y(s) + λ k(s, t)x(t)dt (5.10)
a
b
e R (s) = λ k(s, t)e R (t)dt + E R (λk(s, t)x R (t))
a
So, the function e R (s) holds in the second-type integral equation with the same
solver kernel of k(s, t) but with the second side function of λk(s, t)x(t), where s is
a constant and t is a variable.
According to the above, for the integral equation in the form of x = y + λkx, the
answer is
Therefore
E R (λk(s, t)x R (t))
e R ≤ (1 + λH )E R (λk(s, t)x R (t)) ≤ (5.11)
1 − λH
n
x(s) = y(s) + λ w j k s, t j x t j (5.12)
j=1
and also
n
x R (s) = y(s) + λ w j k s, t j x R t j + E R (λk(s, t)x(t)) (5.13)
j=1
n
e R (s) = λ w j k s, t j e R t j + E R (λk(s, t)x(t))
j=1
n
e R (ti ) = λ w j k ti , t j e R t j + E R (λk(ti , t)x(t)) (5.14)
j=1
156 5 Numerical Solution of Integral Equations
e R = λke R + k
e R = λk · e R + k
as a result
k
e R ≤
1 − λk
where xn − x R and x R − x = e R have been calculated in Eqs. (5.8) and (5.11).
Now, we will introduce the Nystrom method.
Suppose that
b
x(s) = y(s) + λ k(s, t)x(t)dt (5.15)
a
And
n
x(s) = y(s) + λ w j k s, t j x t j (5.16)
j=1
n
x(ti ) = y(ti ) + λ w j k ti , t j x t j
j=1
x = y + λkx
In comparison with the Neumann method, we conclude that the Neumann method
also leads to the above equation by introducing iterative methods and writing inte-
gral equations corresponding to the integration methods. But it can be said that the
Neumann method is in fact a generalized Jacobian method for solving the equation
derived from the Nystrom method.
The iterative method can be written as follows:
Therefore
x = y + λkx (5.17)
is a iterative method for Eq. (5.17). But in general, Eq. (5.17) is solved by
direct methods. Of course, sometimes the (5.18) methods are also faster. For one-
dimensional problems, the problem (5.17) works well for an integration method of
Gaussian type with 10–15 points, and it is not necessary to solve (5.18). However,
if the problem is two-dimensional or more, or weak integration methods such as
trapezoid rule are used, a large n should be adopted, and therefore it is necessary to
use iterative methods of (5.18). In equation (I − λk)x = y, a n × n system must
be solved to calculate x. In this system, if λ is a regular value (even if it is almost
regular, i.e., close to a characteristic value), then
and if x2 also satisfies x = y(s) + λkx, it can be easily verified that x2 + μx1 also
satisfies the above equation. So the solution is not unique. Thus, we can say that if λ
value, every multiple of x1 and x2 is a solution to the equation. That is,
is a regular
I − λk is singular for regular or near-regular λ.
We will now briefly review the error.
158 5 Numerical Solution of Integral Equations
In this equation (1 + λk)−1 depends only on the kernel k and the value of λ,
which are the inherent conditions of the problem. But E R (λk(s, t)x R (t)) depends
not only on the kernel k but also on the integration rule and the answer. If a proper
integration rule is chosen, we expect this factor to be as small as desired.
By introducing the following theorem, we give a brief review of Gaussian
numerical integration methods.
5.3.1 Theorem
If k, x, y ∈ c[a, b], and Rn is a sequence of point wise integration rules such that
lim |I f − Rn ( f )| = 0
n→∞
1
π
n
f (x)
√ dx
f (xi )
1 − x2 n i=1
−1
π
n
(2i − 1)π
= f cos
n i=1 2n
as a result:
5.4 Gauss–Chebyshev Method 159
(2i − 1)π
xi = cosθi , θi =
2n
xi
= ±1, θi
= 0, π, i = 1, . . . , n
as a result
⎧
π, i = j = 0
1
Ti (x)T j (x) ⎨
√ = 0, i
= j
−1 1 − x2 ⎩π
2
,i = j > 0
1 1
T0 (x) T0 (x) × 1
√ dx = √ dx
1 − x2 1 − x2
−1 −1
1
T0 (x)T0 (x)
= √ dx
1 − x2
−1
=π
1 1
Tn (x) Tn (x) × 1
√ dx = √ dx
1 − x2 1 − x2
−1 −1
1
Tn (x)T0 (x)
= √ dx
1 − x2
−1
=0
π
N
(2i − 1)nπ
cos = 0, n = k N , k = 1, . . .
N i=1 2N
160 5 Numerical Solution of Integral Equations
Then, if n
= 2k N , therefore
nπ
sin
= 0
2N
as a result
nπ
N N
(2i − 1)nπ nπ (2i − 1)nπ
2 sin cos = 2 sin · cos
2N i=1 2N i=1
2N 2N
N
inπ (i − 1)nπ
= sin − cos
i=1
N N
= sin nπ − sin 0 = 0
thus
N
(2i − 1)nπ
cos =0
i=1
2N
∞
Suppose f (x) = i=0 ai Ti (x), i.e., the function f is produced by orthogonal
polynomials of Ti (x), in which case it has a Chebyshev expansion and we have:
1 ∞
1
f (x) Ti (x)
√ dx = ai √ dx
1 − x2 i=1
1 − x2
−1 −1
therefore
1
π
N
f (x) (2i − 1)π
Ef = √ dx − f cos
1 − x2 N i=1 2N
−1
E Ti = 0, i
= 2k N
5.4 Gauss–Chebyshev Method 161
so
∞
∞
Ef = ai E Ti = a2k N E T2k N
i=0 k=1
thus
1
π
N
T2k N (x)
E T2k N = √ dx − cos((2i − 1)kπ )
1 − x2 N i=1
−1
where
N
−N , oddk
cos((2i − 1)kπ ) =
N , evenk
i=1
also
1
T2k N (x)
√ d x = 0, 2k N
= 0
1 − x2
−1
then
π, oddk
E T2k N =
−π, evenk
therefore
E f = πa2N − πa4N ± · · ·
and as a result
|E f | ≤ π (|a2N | + |a4N | + · · · )
162 5 Numerical Solution of Integral Equations
|E f |
π |a2N |
Now we want to obtain the expansion coefficients because without these coef-
ficients, the function f (x) cannot be known. For this purpose, according to the
Chebyshev expansion, we can write:
∞
f (x) = ai Ti (x)
i=0
therefore
1 ∞ 1
f (x)T j (x) Ti (x)T j (x)
√ dx = ai √ dx
1−x 2
i=0
1 − x2
−1 −1
π
, i = j
= 0
= aj 2
π, i = j = 0
where
21 f (x)T j (x)
π −1
√ d x, j
= 0
aj =
1 1 f√
1−x 2
(x)T j (x)
π −1 1−x 2
d x, j =0
1
π
N
f (x) kπ
√ dx = f cos
1 − x2 N k=0 N
−1
The sign indicates the coefficient 21 in the first and last terms of the summation.
This rule is accurate for polynomials of at most degree 2N − 1. Therefore, we have:
1
π
N
f (x) kπ
Ec f = √ dx − f cos
1 − x2 N k=0 N
−1
E c Ti (x) = 0, i = 2 j N , j ∈ N
therefore
1
π
N
Ti (x) kπ
Ec f = √ dx − Ti cos
1 − x2 N k=0 N
−1
For i = 2 j N we have:
π ikπ
N
E c Ti (x) = 0 − cos
N k=0 N
given that
iπ
2sin
= 0, i
= j N
N
then
164 5 Numerical Solution of Integral Equations
iπ
N N
ikπ iπ ikπ
2 sin cos = 2 sin cos
N k=0 N k=0
N N
iπ iπ
= sin + (−1)i sin
N N
N −1
(k + 1)iπ (k − 1)iπ
+ sin − sin
k=1
N N
iπ iπ
= sin + (−1)i sin
N N
N −1
(k + 1)iπ kiπ
+ sin − sin
k=1
N N
N −1
ikπ (k − 1)iπ
+ sin − sin
k=1
N N
iπ iπ
= sin + (−1)i sin + sin iπ
N N
iπ (N − 1)iπ
− sin + sin + sin 0
N N
iπ iπ
= (−1) sin
i
+ sin iπ −
N N
− sin N + sin N = 0, oddi
iπ iπ
=
sin iπN
− sin iπ
N
= 0, eveni
so
N
ikπ
cos =0
k=0
N
then
E c Ti (x) = 0
i.e.,
π ikπ
N
1
Ti (x)
√ d x = cos
−1 1 − x 2 N k=0 N
π
N
ikπ
E c T2 j N (x) = 0 − cos
N k=0 N
π
N
=− 1
N k=0
= −π
by assuming
∞
f (x) = bi Ti (x)
i=0
we have
1
π
N
f (x) kπ
Ec f = √ dx − f cos
1 − x2 N k=0 N
−1
∞
= bi E(Ti (x))
i=0
⎛ 1 ⎞
∞
N
⎝ Ti (x) π ikπ ⎠
= bi √ dx − f cos
i=0
1 − x2 N k=0 N
−1
∞
= − π b2 j N
j=0
therefore
∞
|E c f | ≤ x b2 j N
π |b2N |
j=1
5.4.3 Theorem
|bi | ≤ c f i −l−1 = c f i − p
because
∞
(2 j N )− p = (2N )− p + (4N )− p + · · ·
j=1
= (2N )− p 1 + 2− p + 3− p + · · ·
⎛ ⎞
∞
= (2N )− p ⎝1 + j −p⎠
j=2
∞
∞
−p dx
j < ,p>1
j=2
xp
1
where
∞ ∞
dx 1
= x−pd x =
xp p−1
1 1
then
∞
1
j−p <
j=2
p−1
therefore
5.4 Gauss–Chebyshev Method 167
∞
−p −p 1
(2 j N ) ≤ (2N ) 1+
j=1
p−1
p
= (2N )− p (5.21)
p−1
where
2− p · p · c f · π
C=
p−1
where
Cn = α − r dn , n = 1, . . . , N
We calculate the least squares solution of this system. Generally, we multiply the
obtained α in λ,2 ≤ λ ≤ 4 and put it equal to log C.
If the weight function does not exist, in order to use the Gauss–Chebyshev method,
we must create the weight function. That means
1 1 √ 1
f (x) 1 − x 2 f (x)
f (x)d x = √ dx = √ dx
1−x 2 1 − x2
−1 −1 −1
168 5 Numerical Solution of Integral Equations
On the other hand, we know that the errors of the Chebyshev’s √ open and closed
methods depend on the continuous derivatives of f (x) = f (x) 1 − x 2 . This func-
tion and its first derivative are not continuous on ±1, so the condition of the theorem
does not hold. Therefore, in the absence of a weight function, these integration
methods are not accurate and cannot be computed for large N . In
∞ such cases,
the Clenshaw–Curtis integration method is used. Suppose f (x) = i=0 bi Ti (x),
therefore:
1 ∞
1
f (x)d x = bi Ti (x)
−1 i=0 −1
∞
1
d x = 2, i = 0
= bi × 1−1
i=0 −1 xd x = 0, i = 1
1 π
Ti (x)d x = cos iθ · sin θ dθ
−1 0
π
1
= (sin(i + 1)θ − sin(i − 1)θ )dθ
2
0
1 cos(i + 1)θ cos(i − 1)θ π
= − +
2 i +1 i −1 0
1 (−1) i+1
(−1)i−1
1 1
= − + + −
2 i +1 i −1 i +1 i −1
0, oddi
= 2
1−i 2
, eveni
1 ∞
2bi
f (x)d x = ,k ∈ N
i=0,i=2k
1 − i2
−1
1
2 f (x)Ti (x)
bi = √ dx
π 1 − x2
−1
5.4 Gauss–Chebyshev Method 169
1
π
N
f (x)Ti (x) kπ kπ
√
f cos Ti cos
1 − x2 N k=0 N N
−1
π
N
kπ kiπ
= f cos cos
N k=0 N N
which assuming
2
N
kπ ikπ
bi
f cos cos
N k=0 N N
we will have
1 ∞
4 1
N
kπ ikπ
f (x)d x
f cos cos
N i=0 1 − i 2 k=0 N N
−1
4 1
N N
kπ ikπ
f cos cos
N i=0 1 − i k=02 N N
N N
4 ikπ 1 kπ
cos · f cos
k=0
N i=0 N 1 − i2 N
N
kπ
= wk f cos
k=0
N
where
4 ikπ
N
1
wk = cos ·
N i=0 N 1 − i2
2 1
dx dx
√ , √
x 1 − x2
0 −1
and also
1
√
xd x
0
because they face no problem in the integration interval, but their derivatives face a
problem in zero, so the error cannot be bounded to a small numerator. Consider the
following integral.
b
w(x) f (x)d x
a
where w(x) is a singular weight function and f (x) is relatively smooth. The following
situations can occur:
(1) The function has fast variations (this happens less often), like the function
b
w(x) f (x)d x
a
In calculating each of the above integrals, we employ a quadrature rule that does
not use the initial and end of the interval.
For the third case, we have two methods:
(1) There is a Gaussian integration rule whose weight function is w(x), like the
Gauss–Chebyshev rule with w(x) = √1−x 1
2
.
(2) We create a Chebyshev-type rule with the weight function w(x) and use
it.
∞ Suppose that the function f (s) has the McLaren expansion of f (x) =
i
i=0 ai x , therefore
5.5 Non-singular Functions 171
b ∞
b
w(x) f (x)d x = ai w(x)x i d x
a i=0 a
∞
= ai m i
i=0
where
b
mi = w(x)x i d x, i = 0, 1, . . .
a
b
By analytically calculating m i , the value of a w(x) f (x)d x can be calculated.
5.5.1 Definition
∞
The sequence of functions { f i (x)}i=1 is called linearly independent whenever
5.5.2 Definition
∞
The sequence of functions {h i (x)}i=1 is said complete whenever
b ∞
b
w(x) f (x)d x = ai w(x)h i (x)d x
a i=0 a
∞
= ai m i
i=0
where
b
mi = w(x)h i (x)d x, i = 0, 1, . . .
a
5.5.3 Example
where
1
2 f (x)Ti (x)
ai = √ dx
π 1 − x2
−1
2
N
ikπ
ai
f cos kπ cos = ai
π k=0 N
N
thus
b ∞
b
w(x) f (x)d x = ai w(x)Ti (x)
a i=0 a
∞
= ai m i
i=0
5.5 Non-singular Functions 173
N
ai mi
i=0
b b b
w(x) f (x)d x = w(x)( f (x) − f (x0 ))d x + w(x) f (x0 )d x
a a a
b b
= w(x)g(x)d x + f (x0 ) w(x)d x
a a
In this case, usually g(x0 ) = 0 and w(x0 ) also face problems due to the behavior of
w(x).
5.5.4 Example
1 1 1
− 21 − 21
s − 2 ds
1
s f (x)ds = s ( f (s) − f (0))ds + f (0)
0 0 0
where
f (s) − f (0) → 0, s → 0
f (s) − f (0)
ks
This does not remove the singularity, but weakens it, and new functions may have
singular derivatives.
5.5.5 Example
∞
The sequence {sini x}i=1 is linearly independent on the interval [0, 2π ] but it is not
complete. Because if f (x) = 1, then
174 5 Numerical Solution of Integral Equations
b 2π
f (x) f i (x)d x = 1 × sini xd x = 0
a 0
While
f (x) = 0
5.5.6 Example
∞
The sequence {cosi x}i=1 is linearly independent on the interval [0, 2π ] but it is not
complete. Because if f (x) = 1, then
b 2π
f (x) f i (x)d x = 1 × cosi xd x = 0
a 0
While
f (x) = 0
It can also be shown by using the contraposition of the Definition (5.4.2) that the
set
Suppose
b
g(s) = x(s) + λ k(s, t)x(t)dt (5.22)
a
5.6 Expansion Method 175
∞
Also assume that sequence {h i (x)}i=1 is linearly independent and complete. Using
the expansion method of
n
x(s)
xn (s) = ai h i (s)∀n (5.23)
i=1
n
n b
g(s)
a j h j (s) + λ aj k(s, t)h j (t)dt
j=1 j=1 a
So
⎛ ⎞
n b
g(s)
a j ⎝h j (s) + λ k(s, t)h j (t)dt ⎠ (5.24)
j=1 a
One of the methods that we can employ is the collocation method, which means the
method of using points. We obtain the system (5.21) for si , i = 1, . . . , n. We will
have
⎛ ⎞
n b
g(si )
a j ⎝h j (si ) + λ k(si , t)h j (t)dt ⎠
j=1 a
Assuming
b
ci j = h j (si ) + λ k(si , t)h j (t)dt, i, j = 1, . . . , n (5.25)
a
we have
n
gi = ci j a j , i = 1, . . . , n (5.26)
j=1
Ca = g (5.27)
where
n
C = ci j i, j=1 , g = [g1 , . . . , gn ], a = [a1 , . . . , an ]
if
b
k(si , t)h j (t)dt, 1 ≤ i, j ≤ n
a
can be calculated analytically, then ci j s will be calculated accurately and the system
of Eq. (5.24) will have an answer as a.
It should be noted that if λ is a regular value, the matrix C must be non-singular.
Also, it is possible that λ is not a regular value, but the matrix C still is non-singular.
It is possible that if C is not close to being non-singular, λ is not a regular value, and
this is due to the converting approximation to equality.
5.7.1 Example
Suppose that
1
10s + 6 = x(s) + 18 (s + t)x(t)dt (5.28)
0
x(s)
x3 (s) = a1 + a2 s + a3 s 2
1
j−1
ci j = si + 18 (si + t)t j−1 dt, i, j = 1, 2, 3
0
Consider points s1 = 0, s2 = 1
2
and s3 = 1. By calculating ci j s, the system (5.27)
is obtained as follows
5.7 Collocation Methods 177
⎡ 11 ⎤⎡ ⎤ ⎡ ⎤
10 7 2
a1 6
⎣ 19 11 31 ⎦⎣ a2 ⎦ = ⎣ 11 ⎦
4
23
28 16 2
a3 16
a1 = a3 = 0, a2 = 1
2
x1 (s) = a2 s = a2 h 2 (s), c21 = s1 + 18
1
(s1 + t)tdt = 10s1 + 6
0
c21 a2 = g(s1 )
As a result, a2 = 1.
For the second solution, the integral in the Eq. (5.25) cannot be easily computed,
so we have to approximate it.
b
m
k(s, t)h j (t)dt
wik k(s, ti )h j (ti ), i = 1, . . . , m (5.29)
a k=1
assuming
178 5 Numerical Solution of Integral Equations
n
bi j = h j (tu ) + λ wik k(tu , ti )h j (ti )
k=1
and
m
B = bi j i, j=1
Ba = g
x(s)
x2 (s) = a1 + a2 s
1
1 1
(s + t)dt = s+4 s+ + (s + 1)
6 2
0
Finally, a1 = 0 and a2 = 1. Since the Simpson rule for functions of at most degree
three is accurate, the answer is accurate. In general, it can be written
x = y + λkx
x = y + kx
as a result
(I − k)x = y
Assuming L = I − k we have
Lx = y
5.7 Collocation Methods 179
To calculate the error and estimate some of its limits, assume that if x
xn , then
rn = L xn − y = L xn − y + (y − L x)
therefore
rn = L(xn − x)
and if εn = xn − x we have
rn = Lεn
therefore
then
rn ≤ (1 + k)εn
as a result
rn
εn ≥
1 + k
given that
rn = (I − k)εn = εn − kεn
so
εn = kεn + rn
then
(1 − k)εn ≤ rn
180 5 Numerical Solution of Integral Equations
therefore
rn rn
≤ εn ≤
1 + k 1 − k
rn
It can be seen that if k is close to 1, 1−k will be large, although if rn is small,
it is not a reason for εn to be small. But if rn → 0, then, εn → 0.
now if we have
b
x(s) = y(s) + k(s, t)xn (t)dt, x(s)
xn (s)
a
therefore
b
rn (s) = y(s) − xn (s) + k(s, t)xn (t)dt
a
then
n
n b
rn (s) = y(s) − ai(n) h i (s)+ ai(n) k(s, t)h i (t)dt
i=1 i=1 a
⎛ ⎞
n b
= y(s) − ai(n) ⎝h i (s) − k(s, t)h i (t)dt ⎠
i=1 a
if
b
ki (s) = k(s, t)h i (t)dt
a
then
So
n
rn = y(s) − ai(n)li (s)
i=1
5.8 Norm Chebyshev 181
And
b
k = max |k(s, t)|dt
a≤s≤b
a
To calculate x, it is not always possible to calculate the maximum, for example,
it is possible that the function is not differentiable. In this case, we consider a series
of points (e.g., equidistant points) and calculate the value of the function at those
points and obtain the maximum absolute value. That is, we assume
A = {si |i = 1, . . . , q}
To calculate k, one method for calculating the integral is the Monte Carlo
method. Since 0 < R N D < 1, therefore 0 < R N D(b − a) < b − a, and as a result
b
b−a
n
f (x)d x
y(si )
n i=1
a
n
min max y(si ) − a (n)
j l (s )
j i
a (n) si ∈A
j=1
182 5 Numerical Solution of Integral Equations
Assuming that this method is familiar to readers, without a basic explanation for the
method, we will examine it directly in the integral equation. This method is for real
functions and
b
n
2
minrn y(s) − ai(n)li (s) ds
a (n) i=1
a
By assuming
b
2
n
I a (n) = y(s) − ai(n)li (s) ds (5.31)
a i=1
It can be written
min (a (n) )
a (n)
∂ I (a (n) )
=0
∂ai
b
n
(n) (n)
I a = y (s) − 2y(s)
2
ai li (s) ds
a i=1
⎛ ⎞
b
n
n
+ ⎝ ai(n)li (s) × a (n) ⎠
j l j (s) ds
a i=1 j=1
therefore
(n)
∂I a b
n b
(n)
= 2 a j l j (s)li (s)ds − 2 y(s)li (s)ds
∂ai j=1
a a
= 0, i = 1, . . . , n
we define
5.9 Least Squares Method … 183
b
(Yls )i = y(s)li (s)ds, (5.32)
a
b
(L ls )i j = li (s)l j (s)ds
a
so we have
n
(L ls )i j · a (n)
j = (Yls )i , i = 1, . . . , n (5.33)
j=1
L ls a (n) = Yls
where
b
li (s) = h i (s) − k(s, t)h i (t)dt
a
5.9.1 Example
with the exact answer of x(s) = sins. Given that x(s) is an odd function, suppose
h 1 (s) = s, h 2 (s) = s 3
Then
x(s)
x2 (s) = a1 s + a2 s 3
184 5 Numerical Solution of Integral Equations
And
π
2
π3 1
(Yls )1 = s 1 − sin s − s ds
96 4
0
5.9 Least Squares Method … 185
0.45835530
π
2
π5 1
(Yls )2 = s s −
2
sin s − s ds
640 4
0
0.6002751
a1
0.9887922, a2
−0.1450618
where according to
s3 s5
x(s) = sins = s − + − ···
3! 5!
We have
1
a1 = 1, a2 = − = −0.16666 . . .
6
b
x(s) = y(s) + k(s, t)x(t)dt, a ≤ s ≤ b (5.34)
a
If we solve the integral existed in the above equation using the corrected composite
trapezoidal quadrature method, we will have
h n−1
x(s) = y(s) + k(s, s0 ) + h k s, s j x s j
2 j=1
186 5 Numerical Solution of Integral Equations
h h2
k(s, sn )x(sn ) + J (s, s0 )x(s0 ) + k(s, s0 )x (s0 )
2 12
h2
− J (s, sn )x(sn ) + k(s, sn )x (sn ) (5.35)
12
where
∂k(s, t)
J (s, t)|t=s j = , j = 0, 1, . . . , n
∂t t=s j
h n−1
x(si ) = y(si ) + k(si , s0 )x(s0 ) + h k si , s j x s j
2 j=1
h h2
+ k(si , sn )x(sn ) + J (si , s0 )x(s0 ) + k(si , s0 )x (s0 )
2 12
h2
− J (si , sn )x(sn ) + k(si , sn )x (sn ) , i = 0, . . . , n (5.36)
12
If we take the derivative of Eq. (5.34) with respect to s, we have
b
x (s) = y (s) + m(s, t)x(t)dt (5.37)
a
where
∂k(s, t)
m(s, t) =
∂s
The answer function of x(s), which satisfies the Eq. (5.34), also holds for the
Eq. (5.36). Consider Eq. (5.37). The following situations can occur:
∂ 2 k(s,t)
(1) ∂s∂t
is not available
∂ 2 k(s,t)
(2) ∂s∂t
= l(s, t) are available.
In the case (1), we solve the Eq. (5.36) using the composite trapezoidal quadrature
method. In this case, we will have
h
h n−1
x (s) = y (s) + m(s, s0 )x(s0 ) + h m s, s j x s j + m(s, sn )x(sn ) (5.38)
2 j=1
2
h
n−1
x (si ) = y (si ) + m(si , s0 )x(s0 ) + h m si , s j x s j
2 j=1 (5.39)
h
+ m(si , sn )x(sn ), i = 0, . . . , n
2
From the system (5.36), we have
h h2
x(si ) = y(si ) + k(si , s0 ) + J (si , s0 ) x(s0 )
2 12
n−1
h h2
+h k(si , s0 )x s j + k(si , sn ) − J (si , sn ) x(sn )
j=1
2 12
h2
+ k(si , s0 )x (s0 ) − k(si , sn )x (sn ) , i = 0, . . . , n (5.40)
12
h
x (s0 ) = y (s0 ) +
m(s0 , s0 )x(s0 )
2
n−1
h
+h m s0 , s j x s j + m(s0 , sn )x(sn )
j=1
2
h
x (sn ) = y (sn ) +
m(sn , s0 )x(s0 )
2
n−1
h
+h m sn , s j x s j + m(sn , sn )x(sn ) (5.41)
j=1
2
h n−1
x (s) = y (s) + m(s, s0 )x(s0 ) + h m s, s j x s j
2 j=1
h h2 (5.42)
+ m(s, sn )x(sn ) + (l(s, s0 )x(s0 ) + m(s, s0 )x(s0 ))
2 12
h2
− l(s, sn )x(sn ) + m(s, sn )x (sn )
12
By substituting s = si in Eq. (5.42), we can write
188 5 Numerical Solution of Integral Equations
h
n−1
x (si ) = y (si ) + m(si , s0 )x(s0 ) + h m si , s j x s j
2 j=1
h h2
+ m(si , sn )x(sn ) + (l(si , s0 )x(s0 ) + m(si , s0 )x(s0 ))
2 12
2
h
− l(si , sn )x(sn ) + m(si , sn )x (sn ) , i = 0, . . . , n (5.43)
12
6.1 Introduction
In general, if the function x exists under the integral sign and its derivatives exist in
the equation, we call it the integral differential equation. For example, the following
equation
s
x (s) = g(s, x(s)) + λ k(s, t, x(t))dt x(a) = α (6.1)
a
b
g(s) = P(s)x (s) + Q(s)x (s) + R(s)x(s) + λ k(s, t)x(t)dt
a
e = C x(r ) + Dx (r ) (6.2)
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 189
T. Allahviranloo and A. Esfandiari, A Course on Integral Equations with Numerical
Analysis, Mathematical Engineering, https://doi.org/10.1007/978-3-030-85350-1_6
190 6 Numerical Methods for Integral–differential Equations
r = (r1 , . . . , rm )T , a ≤ ri ≤ b
x(r ) = (x(r1 ), . . . , x(rm ))T
x (r ) = (x (r1 ), . . . , x (rm ))T (6.3)
and for a p-order problem, the C and D matrices have p × m dimensions, and the
matrix e has p × 1 dimensions. In Eqs. (6.1) and (6.2), K, P, Q and R are known
functions and x is an unknown function. If the kernel in the Eq. (6.2) is as k(s, t, x(t)),
then the integral–differential equation will be nonlinear.
Equations (6.1) and (6.2) represent the boundary value equations. It is clear that
there are no boundary conditions in integral equations, but these conditions appear in
differential integral equations because they are necessary to prove the uniqueness of
the answer, and this is the only difference between integral equations and integral–
differential equations.
There are various solutions for solving integral–differential equations; however,
we will only examine the expansion method in this book.
Consider the following integral–differential equation:
b=1
g(s) = x (s) + λ k(s, t)x(t)dt (6.4)
a=0
x(0) = x(1) = 0
Suppose that
N
x(s) = ai h i (s) (6.5)
i=0
1
r N (s) = x N (s) +λ k(s, t)x N (t)dt − g(s)
0
⎛ ⎞
N 1
= ai ⎝h i (s) + λ k(s, t)h i (t)dt ⎠ − g(s) (6.6)
(i=0) 0
The above equation can be solved by the Galerkin and collocation methods. As
mentioned before, in the collocation method, we consider some points of the interval
[0, 1] according to the number of unknowns in Eq. (6.6) such that
r N (si ) = 0, i = 0, 1, . . . , N (6.7)
6.2 Integral–Differential Equations 191
In Eq. (6.6),
1
ki (s) = k(s, t)h i (t)dt (6.8)
0
6.2.1 Example
1
x (s) − 60 (s − t)x(t)dt = s − 2
0
x(0) = x(1) = 0
which has the real answer as x(s) = s(s − 1)2 . The h i ’s can be obtained such that
they satisfy the boundary conditions. Suppose that
where
h 0 (0) = h 0 (1) = 0
h 1 (0) = h 1 (1) = 0
1
x(s) = ai h i (s) = a0 h 0 (s) + a1 h 1 (s)
i=0
It is clear that
h 0 (s) = 2, h 1 (s) = 6s − 2
As a result
1
r1 (s) = ai (h i (s) − 60ki (s)) − g(s)
(i=0)
192 6 Numerical Methods for Integral–differential Equations
1
1
k0 (s) = (s − t) t (t − 1) dt = (1 − 2s)
12
0
1
1
k1 (s) = (s − t) t 2 (t − 1) dt = (3 − 5s)
60
0
s0 = 0, s1 = 1
So
r1 (0) = r1 (1) = 0
Therefore
3a0 + 5a1 = 2
7a0 + 6a1 = −1
which are linearly independent and satisfy the boundary conditions. These functions
are the same as s 2 , s 3 , s 4 .
We now introduce the El-gendi method for solving integral–differential equations.
1
g(s) = x (s) + R(s)x(s) + λ k(s, t)x(t)dt (6.9)
−1
s s s
g(t)dt = x (t)dt + R(t)x(t)dt
(−1) −1 −1
s 1
+λ dt k(t, u)x(u)du
−1 −1
Assuming s = si , we have
si si
g(t)dt = x(si ) − x(−1) + R(t)x(t)dt
−1 −1
si 1
+λ k(t, u)x(u)dudt (6.10)
−1 −1
N
x(s) x N (s) = a i Ti (s)
i=0
as a result
1 N 1
x N (s)T j (s) Ti (s)T j (s) π
√ ds = ai √ ds = a j
1 − s2 i=0
1−s 2 2
−1 −1
where
1
2 x N (s)Ti (s)
ai = √ ds
π 1 − s2
−1
2 π
N
kπ ikπ
ai · x N cos cos
π N k=0 N N
kπ
sk = cos , k = 0, 1, . . . , N
N
N
g N (t) = α k Tk (t) (6.11)
k=0
s
N s
g N (t)dt = αk Tk (t)dt
−1 k=0 −1
s π
TN (t)dt = cos kθ · sin θ dθ
(−1) s
π
1
= (sin(k + 1)θ − sin(k − 1)θ )dθ
2
s
1 cos(k − 1)θ cos(k + 1)θ π
= −
2 k−1 k+1 s
1 (−1) k−1
(−1) k+1
Tk−1 (s ) Tk+1 (s )
= − − +
2 k−1 k+1 k−1 k+1
Tk+1 (s ) Tk−1 (s ) (−1) k+1
= − + 2 (6.12)
2(k + 1) 2(k − 1) k −1
k 2 −1
is the integration
constant value. For k = 0 and k = 1, we will have respectively
s s
T0 (t)dt = dt = s + 1 = T1 (s) + 1
−1 −1
6.3 El-Gendi Method 195
s s
s2 1 1
T1 (t)dt = dt = − = (T2 (s) − 1) (6.13)
2 2 4
−1 −1
s N +1
N s
g N (t)dt = βr Tr (s) = αj T j (t)dt (6.14)
−1 r =0 j=0 −1
where
N
(−1) j+1 α j 1
β0 = − α1
j=0, j=1
j −1
2 4
αk−1 − αk+1
βk = , k = 1, 2 . . . , N − 2 (6.15)
2k
(Why?) and for 1 < k < N −1, the coefficient of βk is Tk (s). For k = N −1, N , N +1,
βk ’s can be written as follows:
α N −2 − 21 α N
β N −1 =
2(N − 1)
α N −1
βN = (6.16)
2N
1
α
2 N
β N −1 =
2(N + 1)
si si
g(t)dt + e = x(si ) + R(t)x(t)dt
−1 −1
si 1
+λ k(t, u)x(u)dudt (6.17)
−1 −1
N
g N (t) = α i Ti (t)
i=0
196 6 Numerical Methods for Integral–differential Equations
where
2
N
kπ
aj = g N (sk )T j (sk ), sk = cos , j = 0, 1, . . . , N (6.18)
N k=0 N
Using these α j s, β j can be calculated. So, for such a system, we will have
Gg N = ḡ N (6.19)
si ∞
(ḡ N )i = g N (t)dt = wi j g N (si ), i = 0, . . . , N
−1 j=0
where
⎡ ⎤
g N (s0 )
⎢ ⎥
(G)i j = wi j , g N = ⎣ ... ⎦
g N (s N )
1
s N = 1, (ḡ N ) N = g N (t)dt
−1
and finally
If g N (si ) are given, we have αi s, and therefore, we will have βi s, so the system
can be formed. Thus, it is enough to calculate g N (si ).
6.3.1 Example
s
3
g2 (t)dt = βr Tr (s)
−1 r =0
where
1 1 1
β0 = α0 − α1 − α2
2 4 6
1 1
β1 = α0 − α2
2 4
1
β2 = α1
4
1
β3 = α2
12
Also, according to Eqs. (6.18) and (6.20), we have
2
αj = g(sk )T j (sk ), j = 0, 1, 2
k=0
si
2
g2 (t)dt = wi j g s j
−1 j=0
where
1 4 1
w00 = , w01 = , w02 =
3 3 3
1 2 5
w10 = − , w11 = , w12 =
12 3 12
w20 = w21 = w22 = 0
198 6 Numerical Methods for Integral–differential Equations
Now, we use the fast Galerkin method to solve the first-order linear integral–differ-
ential equations:
e = c x(r ) + d T x (r )
T
(6.22)
where
r = (r1 , . . . , rm )T , −1 ≤ ri ≤ 1
c = (c1 , . . . , cm )T , d = (d1 , . . . , dm ) (6.23)
and x(r ) and x (r ) are defined in Eq. (6.23).
Suppose that
∞
x(s) = a j T j (s)
j=0
∞
x (s) = a j T j (s)
j=0
∞
Q(s) = q j T j (s) (6.24)
j=0
∞
R(s) = r j T j (s)
j=0
∞
g(s) = g j T j (s)
j=0
If we substitute Eq. (6.24) in Eq. (6.22) and integrate them, we will have
∞
1 ∞ ∞ 1
Tk (s)Ti (s) Tk (s)T j (s)Ti (s)
gk √ ds aj qk √ ds
k=0
1 − s2 j=0 k=0
1 − s2
−1 −1
6.4 Fast Galerkin Method 199
∞
∞
1
Tk (s)T j (s)Ti (s)
+ aj rk √ ds
j=0 k=0
1 − s2
−1
1
k(s, t)T j (s)Ti (s)
+λ √ ds (6.25)
1 − s2
−1
1 1
Tk (s)T j (s)Ti (s) 1 Tk (s)Ti+ j (s)
√ ds = √ ds
1 − s2 2 1 − s2
−1 −1
1
1 Tk (s)Ti− j (s)
+ √ ds
2 1 − s2
−1
⎧
⎨ π, i = j = k = 0
= π2 δi j , k = 0, i + j > 0
⎩ π
δ
4 k,i+ j
+ δk,|i− j| , k > 0
where
qi+ j + q|i− j|
Qi j =
2
ri+ j + r|i− j|
Ri j =
2
1 1
2 k(s, t)T j (s)Ti (s)
Bi j = √ dsdt, i = 0, . . . , j = 1, . . .
π 1 − s2
−1 −1
200 6 Numerical Methods for Integral–differential Equations
Since Q(s) are given, we can obtain q j s by the closed Gauss–Chebyshev method.
The coefficients a j are related to a j with the following equation:
a j−1 − a j+1
aj = , j = 1, 2, . . .
2j
a (1) = Aa (6.28)
where
⎧
⎪
⎨ 2( j+1) , j = i ≥ 0
1
Ai j = 2(−1 , j =i +2
⎪
⎩
j−1)
0o.w
where
a j−1 a j+1
− + aj × 0 = aj
2j 2j
6.4 Fast Galerkin Method 201
a0 is also obtained from the boundary conditions of (6.25), so that if we write the
boundary conditions as follows
m
ci x(ri ) + di x (r ) = e
i=1
by substituting the Chebyshev expansion of x(ri ) and x (ri ) in the above equation,
we will have:
⎛ ⎞
m ∞ ∞
⎝ci
a j T j (ri ) + di a j T j (ri )⎠ = e
Or
cT T a + d T T a = e (6.29)
where
Ti j = T j (ri ), i = 1, . . . , m, j = 1, 2, . . .
1
Ti0 = , i = 1, . . . , m
2
where
T
1 1
I = ,..., , T (1) = T j, j+1 , i = 1, . . . , m, j = 1, 2, . . .
2 2
e d T + c T T (1) A
μ= , kT = −
1 1
202 6 Numerical Methods for Integral–differential Equations
a = A a + μ (6.32)
where
kt μ
A = ,μ =
A 0
We now substitute Eq. (6.32) in Eq. (6.27), and finally, the following system is
obtained
Q + (R + λB)A a = g1
where
g1 = g − (R − λB)μ
7.1 Introduction
In this chapter, we are going to consider the interval Fredholm integral equations
in which the deriving terms are an interval then the solution must be a function. In
these equations, it is supposed that the kernel is a real-valued function and only the
deriving terms are an interval function.
Since solving the interval integral equations by numerical methods leads to solving a
linear system of equations, then the short description about these systems is brought
here.
⎪
⎪ a x̃ + · · · + a1n x n = ỹ1 + b11 x̃1 + · · · + b1n x̃n ,
⎪ 11 1
⎨
where the coefficient matrix A = (a i j ) and B = bi j , 1 ≤ i, j ≤ n are real
n × n matrices, x̃ t = (x̃1 , . . . , x̃n ) be a n × 1 vector of interval numbers x̃ j and
ỹ t = ( ỹ1 , . . . , ỹn ) be a n × 1 vector of interval numbers in which ỹi is called a dual
interval linear system.
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 203
T. Allahviranloo and A. Esfandiari, A Course on Integral Equations with Numerical
Analysis, Mathematical Engineering, https://doi.org/10.1007/978-3-030-85350-1_7
204 7 Introduction to Interval Integral Equations
7.2.2 Definition
x̃i = x i , x i , 1 ≤ i ≤ n
n
n
n
n
ai j x j = y i + bi j x j , ai j x j = y i + bi j x j
j=1 j=1 j=1 j=1
The following theorem guarantees the existence of an interval solution for general
case. Consider the dual interval linear system (7.1), and transform its n ×n coefficient
matrix A and B into (2n) × (2n) matrices as:
⎧
⎪
⎪ s11 x− + · · · + s1n x− + s1,n+1 (−x̄1 ) + · · · + s1,2n (−x̄n ) = y1 + t11 x− + · · · + t1n x− + t1,n+1 (−x̄1 ) + · · · + t1,2n (−x̄1 )
⎪
⎪ 1 n 1 n
⎪
⎪
⎪
⎪ .
⎪
⎪ .
⎪
⎪ .
⎪
⎪
⎪
⎨ sn1 x− + · · · + snn x− + sn,n+1 (−x̄1 ) + · · · + sn,2n (−x̄n ) = yn + tn1 x− + · · · + tnn x− + tn,n+1 (−x̄1 ) + · · · + tn,2n (−x̄1 )
1 n 1 n
⎪
⎪ sn+1,1 x− + · · · + sn+1,n x− + sn+1,n+1 (−x̄1 ) + · · · + sn+1,2n (−x̄n ) = − ȳ1 + tn+1,1 x− + · · · + tn+1,n x− + tn+1,n+1 (−x̄1 ) + · · · + tn+1,2n (−x̄n )
⎪
⎪
⎪
⎪
1 n 1 n
⎪
⎪ .
⎪
⎪ .
⎪
⎪ .
⎪
⎪
⎪
⎩ s2n,1 x− + · · · + s2n,n x− + s2n,n+1 (−x̄1 ) + · · · + s2n,2n (−x̄n ) = − ȳ1 + t2n,1 x− + · · · + t2n,n x− + t2n,n+1 (−x̄1 ) + · · · + t2n,2n (−x̄n )
1 n 1 n
ai j ≥ 0 → si j = ai j , si+n, j+n = ai j ,
and si j any ti j those which are not determined by (7.2) are zero. Using the matrix
notation, we get
7.2 Interval Fredholm Integral Equations 205
SX = Y + T X (7.3)
therefore, we have:
(S − T )X = Y (7.4)
where S = (si j ) ≥ 0 and T = ti j ≥ 0, 1 ≤ i, j ≤ 2n, and
⎡ ⎤ ⎡ ⎤
x1 y1
⎢ . ⎥ ⎢ . ⎥
⎢ .. ⎥ ⎢ .. ⎥
⎢ ⎥ ⎢ ⎥
⎢ ⎥ ⎢ ⎥
⎢ xn ⎥ ⎢ yn ⎥
X =⎢ ⎥, Y = ⎢ ⎥ (7.5)
⎢ −x 1 ⎥ ⎢ −y 1 ⎥
⎢ . ⎥ ⎢ ⎥
⎢ . ⎥ ⎢ .. ⎥
⎣ . ⎦ ⎣ . ⎦
−x n −y n
For example:
Consider the dual interval linear system
x̃1 + x̃2 = ỹ1 + 2 x̃1 + x̃2
(7.6)
x̃1 + 2 x̃2 = ỹ2 + x̃1 − 2 x̃ 2
⎡ ⎤ ⎡ ⎤
1 0 0 1 2 1 0 0
⎢1 2 0 0⎥ ⎢1 0 0 2⎥
S=⎢
⎣0
⎥, T = ⎢ ⎥
1 1 0⎦ ⎣0 0 2 1⎦
0 0 1 2 0 2 1 0
and
⎡⎤ ⎡ ⎤
0 x1
⎢ 4 ⎥ ⎢ x ⎥
Y =⎢ ⎥ ⎢ 2
⎣ −2 ⎦, X = ⎣ −x 1
⎥
⎦
−7 −x 2
We obtain that the system (7.6) is equivalent to the dual system of equations
SX = Y + TY
Consequently,
206 7 Introduction to Interval Integral Equations
⎡ ⎤ ⎡ ⎤
⎡ ⎤ ⎡ ⎤ ⎡ ⎤
1 0 0 1 ⎢ x− 1 ⎥ 0 2 1 0 1 ⎢ x− 1 ⎥
⎢1 2 0 0⎥ ⎢ x ⎥ ⎢ 4 ⎥ ⎢1 0 0 2⎥ ⎢ ⎥
⎢ ⎥⎢ − 2 ⎥ = ⎢ ⎥ ⎢ ⎥⎢ x− 2 ⎥
⎣0 1 1 0 ⎦⎢ ⎥ ⎣ −2 ⎦ + ⎣ 0 0 2 1 ⎣ −x̄ ⎥
⎦ ⎢
⎣ −x̄1 ⎦ 1⎦
0 0 1 2 −x̄ −7 0 2 1 0 −x̄
2 2
Also,
(S − T )X = Y
where C and E contain the positive entries of A and B, respectively, and D and F the
absolute values of the negative entries of A and B, i.e., A = C − D and B = E = F.
Therefore,
C−E D−F
S−T =
D−F C−E
7.2.3 Theorem
Proof. Assuming that is nonsingular, then the solution vector is found as (7.4).
X = (S − T )−1 Y (7.7)
and
1
7.2.4 Remark
The unique solution X of Eq. (7.7) is an interval vector for arbitrary Y if and only if
(S − T )−1 is nonnegative, i.e.,
(S − T )−1 i j ≥ 0, 1 ≤ i ≤ 2n, 1 ≤ j ≤ 2n.
u i = min x i , x i
u i = max x i , x i
7.2.6 Definition
D ũ, ṽ = max u − v , |u − v|
7.2.7 Definition
where λ > 0, k(s, t) is an arbitrary kernel function over the square a ≤ s, t ≤ b and
f (t) is a function oft : a ≤ t ≤ b. If f (t) is a real function, then the solution of
Eq. (7.11) is real as well. However, if f (t) is an interval function, this equation my
only possesses interval solution. Therefore, we have
b
x̃(s) = f˜(s) + λ k(s, t)x̃(t)dt (7.12)
a
x̃ = f˜ + λK x̃ (7.13)
in truncated form
n
x̃(s) ≈ x̃n (s) = ãi h i (s) (7.15)
i=1
7.3 Interval Fredholm Integral Equation 209
where the set {h i } is complete and orthogonal in 2 (a, b). For finding approximation
solution, we must indicate coefficient ãi .
From Eq. (7.15), we obtain
n
n b
ã j h j (s) = f˜(s) + λ ã j k(s, t)h j (t)dt (7.16)
j=1 j=1 a
We have n unknown parameters in the form ã1 , ã2 , . . . , ãn which for finding them,
we need to n equation, so by using n points s1 , s2 , . . . , sn in interval [a, b]:
n
n b
h j s j ã j = f˜ s j + λ ã j k s j , t h j (t)dt (7.17)
j=1 j=1 a
therefore, we have:
Aã = f˜ + B ã (7.18)
where the coefficients
matrix A = a i j , 1 ≤ i, j ≤ n, and B = bi j , 1 ≤ i, j ≤ n,
are real and f˜ = f˜i , 1 ≤ i ≤ n, is an arbitrary interval number vector, where
b
ai j = h j (si ), bi j = λ k(si , t)h j (t)dt, i, j = 1, . . . , n
a
L x̃ = f˜, L = I − K (7.19)
where Dis any distance function and to compute rn requires no knowledge of x̃ but,
˜
since Ḋ f , L x̃ = 0, we have the identify:
rn = D f˜, L x̃n − D f˜, L x̃ = L D(x̃n , x̃) = Lεn . (7.22)
That is,
rn
εn ≥ (7.24)
1+K
Thus, a small residual is necessary condition for a small error. We would rather
have an upper bound on εn of course; this is harder to provide in general, and we
content ourselves for now with the following.
We rewrite (7.22) as
εn = rn + K εn
whence
and hence if
εn < 1
rn
εn ≤ (7.26)
1−K
f (s) = 0,
f (s) = 4s 3
and kernel
k(s, t) = s + 1, −1 ≤ s, t ≤ 1
7.3 Interval Fredholm Integral Equation 211
x(s) = 0,
x(s) = 4s 3 .
and
s1 = −1, s2 = 1
The fact that after solving this system it is found that ã1 = [a 1 , a 1 ] and ã2 =
[a 2 , a 2 ] are not interval numbers. Therefore, the interval solution of Eq. (7.27) is a
weak interval solution.
References
© The Editor(s) (if applicable) and The Author(s), under exclusive license 213
to Springer Nature Switzerland AG 2022
T. Allahviranloo and A. Esfandiari, A Course on Integral Equations with Numerical
Analysis, Mathematical Engineering, https://doi.org/10.1007/978-3-030-85350-1
214 References
18. G. Hammerlin, K.H. Hoffmann, Numerical Mathematics (Springer, New York, 1991)
19. G. Hammerlin, K.H. Hoffmann, Numerical Mathematics (Springer, New York, 1991)
20. F.B. Hildebrand, Introduction to Numerical Analysis (Dover Publications, New York, 1974)
21. E. Isaacson, H. BishopKeller, Analysis of Numerical methods (Wiley, New York, 1966)
22. E. Isaacson, H. Bishop Keller, Analysis of Numerical Methods (Wiley, New York, 1966)
23. J.J. Buckley, Fuzzy eigenvalues and input–output analysis. FSS 34, 187–195 (1990)
24. J. M. Ortega, Numerical Analysis a Second Course (Siam, 1990)
25. K. Atkinson, An Introduction to Numerical Analysis, 2nd edn (Wiley, 1989)
26. D. Kincaid, W. Cheney, Numerical Analysis: Mathematics of Scientific Computing (Brooks-
Cole, 2002)
27. D. Kincaid, W. Cheney, Numerical Analysis: Mathematics of Scientific Computing (Brooks-
Cole, 2002)
28. A. S. Kronrod, Nodes and Weights of Quadrature Formula (New York, 1965)
29. L.A. Zadeh, The concept of a linguistic variable and its application to approximate reasoning.
Inform. Sci. 8, 199–249 (1975)
30. J. N. Lyness, Notes on the adaptive Simpson quadrature routine. J. Assoc. Comput. Mach. 16,
483–492 (1969)
31. J. N. Lyness, Notes on the adaptive Simpson quadrature routine. J. Assoc. Comput. Mach. 16,
483–492 (1969)
32. M. Friedman, M. Ming, A. Kandel, Fuzzy linear systems. FSS 96, 201–209 (1998)
33. M. Friedman, M. Ma, A. Kandel, Numerical solutions of fuzzy differential equations and
integral equations. Fuzzy Sets Syst. 106, 35–48 (1999)
34. M. Friedman, M. Ming, A. Kandel, Duality in fuzzy linear systems. Fuzzy Sets Syst. 109,
55–58 (2000)
35. C. D. Meyer, Matrix Analysis and Applied Linear Algebra (Siam, 2000)
36. J. M. Ortega, Numerical Analysis: A Second Course (Siam, 1990)
37. P. Diamond, Fuzzy least squares. Inform. Sci. 46, 144–157 (1988)
38. G.M.M. Phillips, P.J. Taylor, Theory and Applications of Numerical Analysis (Elsevier Science
and Technology Books, 1996)
39. R. Goetschel, W. Voxman, Elementary calculus. Fuzzy Sets Syst. 18, 31–43 (1986)
40. A. Ralston, P. Rrabinowitz, A First Course in Numerical Analysis (McGraw-Hill, New York,
1982)
41. A. Ralston, P. Rrabinowitz, A First Course in Numerical Analysis (McGraw-Hill, New York,
1982)
42. S.L. Chang, L.A. Zadeh, On fuzzy mapping and control, IEEE Trans. Syst. Man Cyb. 2, 30–34
(1972)
43. D. Sheen, Introduction to Numerical Analysis (Seoul National University, 2007)
44. J. Stoer, R. Bulirsch, Introduction to Numerical Analysis (Springer, New York, 2002)
45. J. Stoer, R. Bulirsch, Introduction to Numerical Analysis (Springer, New York, 2002)
46. G. Szego, Orthogonal polynomials. Am. Math. (1975)
47. G. Szego, Uber gewisse orthogonal polynomials. Math. Ann. (1935)
48. W. Cong-Xin, M. Ming, Embedding problem of fuzzy number space: Part I. Fuzzy Sets Syst.
44, 33–38 (1991)
49. C.J. Zarowski, An Introduction to Numerical Analysis for Electrical and Computer Engineers
(Wiley , New Jersey, 2004)
50. Z. Gouyandeh, T. Allahviranloo, S. Abbasbandy, A. Armand, A fuzzy solution of heat equation
under generalized Hukuhara differentiability by fuzzy Fourier transform. Fuzzy Sets Syst.
309(15), 81–97 (February 2017)
51. A. Ralston, P. Rabinowitz, A First Course in Numerical Analysis (McGraw-Hill, 1978)
52. J. Stoer, R. Bulirsch, Introduction to Numerical Analysis (Springer, 1993)