An iterative method of the Krylov subspace type for solving the linear system of equations Ax=b.

The idea is at each iteration to minimize the residual (the norm of) b-Ay, where y is the current approximate solution. y is required to lie in a space of dimension equal to the iteration number.

To be more precise, at the first iteration y lies in the space spanned by b, in the second in the space spanned by b and A*b, in the third in the space spanned by b, A*b and A²*b and so on...

The algorithm, which involves solving a least squares problem at every iteration can be found in any book on numerical linear algebra.