Psychology Wiki

Linear model

34,135pages on
this wiki
Revision as of 23:18, February 14, 2006 by Lifeartist (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

Assessment | Biopsychology | Comparative | Cognitive | Developmental | Language | Individual differences | Personality | Philosophy | Social |
Methods | Statistics | Clinical | Educational | Industrial | Professional items | World psychology |

Statistics: Scientific method · Research methods · Experimental design · Undergraduate statistics courses · Statistical tests · Game theory · Decision theory

In statistics the linear model is a model given by

Y = X \beta + \varepsilon

where Y is an n×1 column vector of random variables, X is an n×p matrix of "known" (i.e., observable and non-random) quantities, whose rows correspond to statistical units, β is a p×1 vector of (unobservable) parameters, and ε is an n×1 vector of "errors", which are uncorrelated random variables each with expected value 0 and variance σ2. Often one takes the components of the vector of errors to be independent and normally distributed. Having observed the values of X and Y, the statistician must estimate β and σ2. Typically the parameters β are estimated by the method of maximum likelihood, which in the case of normal errors is equivalent (by the Gauss-Markov theorem) to the method of least squares.

If, rather than taking the variance of ε to be σ2I, where I is the n×n identity matrix, one assumes the variance is σ2M, where M is a known matrix other than the identity matrix, then one estimates β by the method of "generalized least squares", in which, instead of minimizing the sum of squares of the residuals, one minimizes a different quadratic form in the residuals — the quadratic form being the one given by the matrix M-1. This leads to the estimator


which is the Best Linear Unbiased Estimator for \beta. If all of the off-diagonal entries in the matrix M are 0, then one normally estimates β by the method of "weighted least squares", with weights proportional to the reciprocals of the diagonal entries.

Ordinary linear regression is a very closely related topic.

Generalizations Edit

Generalized linear models Edit

Generalized linear models, for which rather than

E(Y) = Xβ,

one has

g(E(Y)) = Xβ,

where g is the "link function". An example is the Poisson regression model, which states that

Yi has a Poisson distribution with expected value eγ+δxi.

The link function is the natural logarithm function. Having observed xi and Yi for i = 1, ..., n, one can estimate γ and δ by the method of maximum likelihood.

General linear model Edit

The general linear model (or multivariate regression model) is a linear model with multiple measurements per object. Each object may be represented in a vector.

See also Edit

  • ANOVA, or analysis of variance, is historically a precursor to the development of linear models. Here the model parameters themselves are not computed, but X column contributions and their significance are identified using the ratios of within-group variances to the error variance and applying the F Modell
This page uses Creative Commons Licensed content from Wikipedia (view authors).

Around Wikia's network

Random Wiki