Ill-Conditioned Linear Systems

Abstract

For problems in mathematical physics Hadamard [31] postulated three requirements: A solution should exist, the solution should be unique, and the solution should depend continuously on the data. The third postulate is motivated by the fact that in general, in applications the data will be measured quantities and therefore always contaminated by errors. A problem satisfying all three requirements is called well-posed. Otherwise, it is called ill-posed. If A : X-t Y is a bounded linear operator mapping a normed space X into a normed space Y, then the equation Ax = y is well-posed if A is bijective and the inverse operator A-I : Y-t X is bounded (see Theorem 3.24). Since the inverse of a linear operator again is linear, in the case of finite-dimensional spaces X and Y, by Theorem 3.26 bijectivity of A implies boundedness of the inverse operator. Hence, in the sense of Hadamard, nonsingular linear systems are well-posed. However, since one wants to make sure that small errors in the data of a linear system will cause only small errors in the solution, there is an additional need for a measure of the degree of well-posedness, or stability. Such a measure is provided through the notion of the condition number, which we will introduce in this chapter. This will enable us to distinguish between well-conditioned and ill-conditioned linear systems. For the latter, small errors in the data may cause large errors in the solution, and therefore their numerical solution requires special care. Hence, we will continue the chapter with a brief discussion of the singular value cutoff and the Tikhonov regularization as efficient means to deal with ill-conditioned linear systems. Our analysis will be based on the singular value decomposition and will include the introduction of the pseudo-inverse, R. Kress, Numerical Analysis