Iterative Methods for Solving Linear Systems

Abstract

Iterative methods formally yield the solution x of a linear system after an infinite number of steps. At each step they require the computation of the residual of the system. In the case of a full matrix, their computational cost is therefore of the order of n2 operations for each iteration, to be compared with an overall cost of the order of operations needed by direct methods. Iterative methods can therefore become competitive with direct methods provided the number of iterations that are required to converge (within a prescribed tolerance) is either independent of n or scales sublinearly with respect to n.