Methods | Statistics | Clinical | Educational | Industrial | Professional items | World psychology |
The method of iteratively reweighted least squares (IRLS) is used to solve certain optimization problems. It solves objective functions of the form:
IRLS is used to find the maximum likelihood estimates of a generalized linear model, and in robust regression to find an M-estimator, as a way of mitigating the influence of outliers in an otherwise normally-distributed data set. For example, by minimizing the least absolute error rather than the least square error.
Although not a linear regression problem, Weiszfeld's algorithm for approximating the geometric median can also be viewed as a special case of iteratively reweighted least squares, in which the objective function is the sum of distances of the estimator from the samples.
L1 minimization for sparse recovery Edit
IRLS can be used for 1 minimization and smoothed p minimization, p < 1, in the compressed sensing problems. It has been proved that the algorithm has a linear rate of convergence for 1 norm and superlinear for t with t < 1, under the restricted isometry property, which is generally a sufficient condition for sparse solutions.
Lp norm linear regression Edit
where W(t) is the diagonal matrix of weights with elements:
- ↑ (March 31 – April 4, 2008) "Iteratively reweighted algorithms for compressive sensing". IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2008: 3869–3872.
- ↑ I Daubechies et al (2008). Iteratively reweighted least squares minimization for sparse recovery. URL accessed on 2010-11-02.
- ↑ Gentle, James (2007). "6.8.1 Solutions that Minimize Other Norms of the Residuals" Matrix algebra, New York: Springer.
- University of Colorado Applied Regression lecture slides
- Stanford Lecture Notes on the IRLS algorithm by Antoine Guitton
- Numerical Methods for Least Squares Problems by Åke Björck (Chapter 4: Generalized Least Squares Problems.)
- Practical Least-Squares for Computer Graphics. SIGGRAPH Course 11
|This page uses Creative Commons Licensed content from Wikipedia (view authors).|