# Generalized least squares

*34,196*pages on

this wiki

Assessment |
Biopsychology |
Comparative |
Cognitive |
Developmental |
Language |
Individual differences |
Personality |
Philosophy |
Social |

Methods |
Statistics |
Clinical |
Educational |
Industrial |
Professional items |
World psychology |

**Statistics:**
Scientific method ·
Research methods ·
Experimental design ·
Undergraduate statistics courses ·
Statistical tests ·
Game theory ·
Decision theory

In statistics, **generalized least squares (GLS)** is a technique for estimating the unknown parameters in a linear regression model. The GLS is applied when the variances of the observations are unequal (heteroscedasticity), or when there is a certain degree of correlation between the observations. In these cases ordinary least squares can be statistically inefficient, or even give misleading inferences.

## Method outline Edit

In a typical linear regression model we observe data on *n* statistical units. The response values are placed in a vector *Y* = (*y*_{1}, ..., *y*_{n})′, and the predictor values are placed in the design matrix *X* = [[*x*_{ij}]], where *x*_{ij} is the value of the *j*th predictor variable for the *i*th unit. The model assumes that the conditional mean of *Y* given *X* is a linear function of *X*, whereas the conditional variance of the error term given *X* is a *known* matrix Ω. This is usually written as

Here *β* is a vector of unknown “regression coefficients” that must be estimated from the data.

Suppose *b* is a candidate estimate for *β*. Then the residual vector for *b* will be *Y* − *Xb*. Generalized least squares method estimates *β* by minimizing the squared Mahalanobis length of this residual vector:

Since the objective is a quadratic form in *b*, the estimator has an explicit formula:

### PropertiesEdit

The GLS estimator is unbiased, consistent, efficient, and asymptotically normal:

GLS is equivalent to applying ordinary least squares to a linearly transformed version of the data. To see this, factor Ω = *BB′*, for instance using the Cholesky decomposition. Then if we multiply both sides of the equation *Y* = *Xβ* + *ε* by *B*^{−1}, we get an equivalent linear model *Y** = *X*β* + *ε**, where *Y** = *B*^{−1}*Y*, *X** = *B*^{−1}*X*, and *ε** = *B*^{−1}*ε*. In this model Var[*ε**] = *B*^{−1}Ω(*B*^{−1})′ = *I*. Thus we can efficiently estimate *β* by applying OLS to the transformed data, which requires minimizing

This has the effect of standardizing the scale of the errors and “de-correlating” them. Since OLS is applied to data with homoscedastic errors, the Gauss–Markov theorem applies, and therefore the GLS estimate is the best linear unbiased estimator for *β*.

## Weighted least squaresEdit

A special case of GLS called weighted least squares (WLS) occurs when all the off-diagonal entries of *Ω* are 0. This situation arises when the variances of the observed values are unequal (i.e. heteroscedasticity is present), but where no correlations exist among the observed variances. The weight for unit *i* is proportional to the reciprocal of the variance of the response for unit *i*.

## Feasible generalized least squares Edit

.**Feasible generalized least squares (FGLS)** is similar to generalized least squares except that it uses an estimated variance-covariance matrix since the true matrix is not known directly.

The ordinary least squares (OLS) estimator is calculated as usual by

and estimates of the residuals are constructed.

Construct :

Estimate using using weighted least squares

This estimation of can be iterated to convergence given that the assumptions outlined in White^{[citation needed]} hold.

The WLS and FGLS estimators have the following distributions

## See alsoEdit

## ReferencesEdit

This page uses Creative Commons Licensed content from Wikipedia (view authors). |