Linear regression
From Psychology Wiki
Community portal · Tasks to do · News · Help
Clinical · Educational · Ind&Org · Other fields · Professional · Transpersonal · World
Assessment |
Biopsychology |
Comparative |
Cognitive |
Developmental |
Language
Personality |
Philosophy |
Research Methods |
Social |
Statistics
Statistics: Scientific method · Research methods · Experimental design · Undergraduate statistics courses · Statistical tests · Game theory · Decision theory
In statistics, linear regression is used for two things:
- to construct a simple formula that will predict a value or values for a variable given the value of another variable.
- to test whether and how a given variable is related to another variable or variables.
Linear regression is a form of regression analysis in which the relationship between one or more independent variables and another variable, called the dependent variable, is modelled by a least squares function, called a linear regression equation. This function is a linear combination of one or more model parameters, called regression coefficients. A linear regression equation with one independent variable represents a straight line when the predicted value (i.e. the dependent variable from the regression equation) is plotted against the independent variable: this is called a simple linear regression. However, note that "linear" does not refer to this straight line, but rather to the way in which the regression coefficients occur in the regression equation. The results are subject to statistical analysis.
Contents |
[edit] Introduction
[edit] Theoretical model
A linear regression model assumes, given a random sample 
, a possibly imperfect relationship between 
, the regressand, and the regressors 
. A disturbance term 
, which is a random variable too, is added to this assumed relationship to capture the influence of everything else on other than . Hence, the multiple linear regression model takes the following form:
Note that the regressors (the X-s) are also called independent variables, exogenous variables, covariates, input variables or predictor variables. Similarly, regressands (the Y-s) are also called dependent variables, response variables, measured variables, or predicted variables. There are p + 1 unknown parameters β0, β1, ..., βp. One of the purpose of linear regression is to determine these unknown parameters and their statistical significance (in other words: the mean value and standard deviation of each βj, j = 0, 1, ..., p).
Models which do not conform to this specification may be treated by nonlinear regression. A linear regression model need not be a linear function of the independent variable: linear in this context means that the conditional mean of is linear in the parameters 
. For example, the model 
is linear in the parameters 
and 
, but it is not linear in 
, a nonlinear function of 
. An illustration of this model is shown in the example, below.
[edit] Data and estimation
It is important to distinguish the model formulated in terms of random variables and the observed values of these random variables. Typically, the observed values, or data, denoted by lower case letters, consist of n values 
.
In general there are 
parameters to be determined, 
. In order to estimate the parameters it is often useful to use the matrix notation
where Y is a column vector that includes the observed values of 
, 
includes the unobserved stochastic components 
and the matrix X the observed values of the regressors
X includes, typically, a constant column, that is, a column which does not vary across observations, which is used to represent the intercept term 
. Matrix X is sometime called the design matrix.
If there is any linear dependence among the columns of X, then the vector of parameters cannot be estimated by least squares unless is constrained, as, for example, by requiring the sum of some of its components to be 0. However, some linear combinations of the components of 
may still be uniquely estimable in such cases. For example, the model
cannot be solved for and independently as the matrix of observations has the reduced rank 2. In this case the model can be rewritten as
and solved to give a value for the composite entity 
.
Note that to only perform a least squares estimation of 
it is not necessary to consider the sample as random variables. It may even be conceptually simpler to consider the sample as fixed, observed values, as we have done thus far. However in the context of hypothesis testing and confidence intervals, it will be necessary to interpret the sample as random variables that will produce estimators which are themselves random variables. Then it will be possible to study the distribution of the estimators and draw inferences.
[edit] Classical assumptions
Classical assumptions for linear regression include the assumptions that the sample is selected at random from the population of interest, that the dependent variable is continuous on the real line, and that the error terms follow identical and independent normal distributions, that is, that the errors are i.i.d. and Gaussian. Note that these assumptions imply that the error term does not statistically depend on the values of the independent variables, that is, that is statistically independent of the predictor variables. This article adopts these assumptions unless otherwise stated. Note that all of these assumptions may be relaxed, depending on the nature of the true probabilistic model of the problem at hand. The issue of choosing which assumptions to relax, which functional form to adopt, and other choices related to the underlying probabilistic model are known as specification searches. In particular note that the assumption that the error terms are normally distributed is of no consequence unless the sample is very small because central limit theorems imply that, so long as the error terms have finite variance and are not too strongly correlated, the parameter estimates will be approximately normally distributed even when the underlying errors are not.
Under these assumptions, an equivalent formulation of simple linear regression that explicitly shows the linear regression as a model of conditional expectation can be given as
The conditional expected value of Yi given Xi is an affine function of Xi. Note that this expression follows from the assumption that the mean of 
is zero conditional on Xi.
[edit] Least-squares analysis
[edit] Least squares estimates
The first objective of regression analysis is to best-fit the data by estimating the parameters of the model. Of the different criteria that can be used to define what constitutes a best fit, the least squares criterion is a very powerful one. This estimate (or estimator, if we are in the context of a random sample), is given by
- WikiTeX: latex reported a failure, namely:
This is pdfeTeX, Version 3.141592-1.21a-2.2 (Web2C 7.5.4)
entering extended mode
(./cff3fa23eb7858f571cbbc5fe8055
LaTeX2e <2003/12/01>
Babel
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Missing } inserted.
}
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Extra }, or forgotten $.
}{\hbox {$\m@th \textstyle...
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Extra }, or forgotten $.
{\hbox {$\m@th \textstyle ...
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Extra }, or forgotten $.
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Missing } inserted.
}
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Extra }, or forgotten $.
}{\hbox {$\m@th \scriptsty...
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Extra }, or forgotten $.
{\hbox {$\m@th \scriptstyl...
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Extra }, or forgotten $.
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Missing } inserted.
}
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Extra }, or forgotten $.
}{\hbox {$\m@th \scriptscr...
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Extra }, or forgotten $.
{\hbox {$\m@th \scriptscri...
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Extra }, or forgotten $.
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Missing } inserted.
}
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Extra }, or forgotten $.
}
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Extra }, or forgotten $.
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Extra }, or forgotten $. \math@atom #1#2->\binrel@ {#1}\binrel@@ {#2}
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Missing $ inserted.
$
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Missing } inserted.
}
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Missing } inserted.
}
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Missing $ inserted.
$
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Missing } inserted.
}
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Missing } inserted.
}
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Missing $ inserted.
$
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Missing } inserted.
}
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Missing } inserted.
}
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Missing $ inserted.
$
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Missing } inserted.
}
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Missing } inserted.
}
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Missing { inserted.
\endgroup
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Missing } inserted.
}
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Missing { inserted.
\endgroup
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Missing } inserted.
}
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Missing { inserted.
\endgroup
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Missing } inserted.
}
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
! Missing } inserted.
}
l.5 \begin{equation*} \boldsymbol\hat
\beta = (\mathbf {X}^T \mathbf {X})^{-1...
[1] (./cff3fa23eb7858f571cbbc5fe8055.aux) ) (see the transcript file for additional information) Output written on cff3fa23eb7858f571cbbc5fe8055.dvi (1 page, 480 bytes).
Transcript written on cff3fa23eb7858f571cbbc5fe8055.log.For a full derivation see Linear least squares.
[edit] Regression inference
The estimates can be used to test various hypotheses.
Denote by 
the variance of the error term (recall we assume that 
for every 
). An unbiased estimate of 
is given by
where 
is the sum of square residuals.
The relation between the estimate and the true value is:
where 
has Chi-square distribution with n − p − 1 degrees of freedom.
The solution to the normal equations can be written as [1]
- WikiTeX: latex reported a failure, namely:
This is pdfeTeX, Version 3.141592-1.21a-2.2 (Web2C 7.5.4)
entering extended mode
(./edf742ca7bf32dc0d0ec0ad72ed08
LaTeX2e <2003/12/01>
Babel
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Extra }, or forgotten $.
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Extra }, or forgotten $.
{\hbox {$\m@th \textstyle ...
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Extra }, or forgotten $.
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Extra }, or forgotten $.
{\hbox {$\m@th \scriptstyl...
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Extra }, or forgotten $.
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Extra }, or forgotten $.
{\hbox {$\m@th \scriptscri...
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Extra }, or forgotten $.
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Extra }, or forgotten $.
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Extra }, or forgotten $. \math@atom #1#2->\binrel@ {#1}\binrel@@ {#2}
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Missing $ inserted.
$
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Missing $ inserted.
$
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Missing $ inserted.
$
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Missing $ inserted.
$
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Missing { inserted.
\endgroup
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Missing { inserted.
\endgroup
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Missing { inserted.
\endgroup
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Extra }, or forgotten $. \mathaccentV ...blethree \frozen@everymath {}$#5$}
\macc@tmp \ifnum \macc@dep...
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Argument of \boldsymbol has an extra }.
\par
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Missing $ inserted.
$
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta=(\mathbf X ^T \mathbf X)^{-1} \mat...
[1] (./edf742ca7bf32dc0d0ec0ad72ed08.aux) ) (see the transcript file for additional information) Output written on edf742ca7bf32dc0d0ec0ad72ed08.dvi (1 page, 488 bytes).
Transcript written on edf742ca7bf32dc0d0ec0ad72ed08.log.This shows that the parameter estimators are linear combinations of the dependent variable. It follows that, if the observational errors are normally distributed, the parameter estimators will follow a joint normal distribution. Under the assumptions here, the estimated parameter vector is exactly distributed,
- WikiTeX: latex reported a failure, namely:
This is pdfeTeX, Version 3.141592-1.21a-2.2 (Web2C 7.5.4)
entering extended mode
(./a7f3c0af802dbf6266e1092314e7b
LaTeX2e <2003/12/01>
Babel
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Missing } inserted.
}
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Extra }, or forgotten $.
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Extra }, or forgotten $.
{\hbox {$\m@th \textstyle ...
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Extra }, or forgotten $.
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Extra }, or forgotten $.
{\hbox {$\m@th \scriptstyl...
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Extra }, or forgotten $.
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Extra }, or forgotten $.
{\hbox {$\m@th \scriptscri...
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Extra }, or forgotten $.
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Extra }, or forgotten $.
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Extra }, or forgotten $. \math@atom #1#2->\binrel@ {#1}\binrel@@ {#2}
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Missing $ inserted.
$
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Missing } inserted.
}
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Missing } inserted.
}
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Missing $ inserted.
$
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Missing } inserted.
}
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Missing } inserted.
}
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Missing $ inserted.
$
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Missing } inserted.
}
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Missing } inserted.
}
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Missing $ inserted.
$
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Missing } inserted.
}
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Missing } inserted.
}
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Missing { inserted.
\endgroup
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Missing } inserted.
}
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Missing { inserted.
\endgroup
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Missing } inserted.
}
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Missing { inserted.
\endgroup
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Missing } inserted.
}
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Missing } inserted.
}
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Extra }, or forgotten $. \mathaccentV ...blethree \frozen@everymath {}$#5$}
\macc@tmp \ifnum \macc@dep...
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Argument of \boldsymbol has an extra }.
\par
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Missing $ inserted.
$
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
! Missing } inserted.
}
l.5 \begin{equation*} \hat\boldsymbol
\beta \sim N ( \boldsymbol\beta, \sigma...
[1] (./a7f3c0af802dbf6266e1092314e7b.aux) ) (see the transcript file for additional information) Output written on a7f3c0af802dbf6266e1092314e7b.dvi (1 page, 604 bytes).
Transcript written on a7f3c0af802dbf6266e1092314e7b.log.where N denotes the multivariate normal distribution.
The standard error of j-th parameter estimator βj (where j = 0, 1, ..., p) is given by
The 100(1 − α)% confidence interval for the parameter, 
, is computed as follows:
The residuals can be expressed as
- WikiTeX: latex reported a failure, namely:
This is pdfeTeX, Version 3.141592-1.21a-2.2 (Web2C 7.5.4)
entering extended mode
(./c7e0030190325e7a86e806c0b0d27
LaTeX2e <2003/12/01>
Babel
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Missing } inserted.
}
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Extra }, or forgotten $.
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Extra }, or forgotten $.
{\hbox {$\m@th \textstyle ...
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Extra }, or forgotten $.
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Extra }, or forgotten $.
{\hbox {$\m@th \scriptstyl...
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Extra }, or forgotten $.
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Extra }, or forgotten $.
{\hbox {$\m@th \scriptscri...
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Extra }, or forgotten $.
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Extra }, or forgotten $.
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Extra }, or forgotten $. \math@atom #1#2->\binrel@ {#1}\binrel@@ {#2}
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Missing $ inserted.
$
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Missing } inserted.
}
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Missing } inserted.
}
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Missing $ inserted.
$
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Missing } inserted.
}
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Missing } inserted.
}
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Missing $ inserted.
$
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Missing } inserted.
}
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Missing } inserted.
}
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Missing $ inserted.
$
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Missing } inserted.
}
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Missing } inserted.
}
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Missing { inserted.
\endgroup
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Missing } inserted.
}
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Missing { inserted.
\endgroup
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Missing } inserted.
}
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Missing { inserted.
\endgroup
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Missing } inserted.
}
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Missing } inserted.
}
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Extra }, or forgotten $. \mathaccentV ...blethree \frozen@everymath {}$#5$}
\macc@tmp \ifnum \macc@dep...
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Argument of \boldsymbol has an extra }.
\par
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Missing $ inserted.
$
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
! Missing } inserted.
}
l.5 ...bf{\hat r = y-X \hat\boldsymbol\beta= y-X}
( \mathbf X^ T \mathbf X)...
[1] (./c7e0030190325e7a86e806c0b0d27.aux) ) (see the transcript file for additional information) Output written on c7e0030190325e7a86e806c0b0d27.dvi (1 page, 588 bytes).
Transcript written on c7e0030190325e7a86e806c0b0d27.log.The matrix 
is known as the hat matrix and has the useful property that it is idempotent. Using this property it can be shown that, if the errors are normally distributed, the residuals will follow a normal distribution with covariance matrix
.
Studentized residuals are useful in testing for outliers.
The hat matrix is the matrix of the orthogonal projection onto the column space of the matrix X.
Given a value of the independent variable, xd, (where d = 1, 2, ..., n) the predicted response is calculated as
Writing the elements 
as 
, the 100(1 − α)% mean response confidence interval for the prediction is given, using error propagation theory, by:
- WikiTeX: latex reported a failure, namely:
This is pdfeTeX, Version 3.141592-1.21a-2.2 (Web2C 7.5.4)
entering extended mode
(./e02fb5ceca23fda0523486709bce3
LaTeX2e <2003/12/01>
Babel
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Extra }, or forgotten $.
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Extra }, or forgotten $.
{\hbox {$\m@th \textstyle ...
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Extra }, or forgotten $.
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Extra }, or forgotten $.
{\hbox {$\m@th \scriptstyl...
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Extra }, or forgotten $.
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Extra }, or forgotten $.
{\hbox {$\m@th \scriptscri...
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Extra }, or forgotten $.
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Extra }, or forgotten $.
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Extra }, or forgotten $. \math@atom #1#2->\binrel@ {#1}\binrel@@ {#2}
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Missing $ inserted.
$
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Missing $ inserted.
$
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Missing $ inserted.
$
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Missing $ inserted.
$
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Missing { inserted.
\endgroup
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Missing { inserted.
\endgroup
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Missing { inserted.
\endgroup
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Extra }, or forgotten $. \mathaccentV ...blethree \frozen@everymath {}$#5$}
\macc@tmp \ifnum \macc@dep...
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Argument of \boldsymbol has an extra }.
\par
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Missing $ inserted.
$
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T\hat\boldsymbol
\beta \pm t_{ \frac{\alpha }{...
[1] (./e02fb5ceca23fda0523486709bce3.aux) ) (see the transcript file for additional information) Output written on e02fb5ceca23fda0523486709bce3.dvi (1 page, 796 bytes).
Transcript written on e02fb5ceca23fda0523486709bce3.log.The 100(1 − α)% predicted response confidence intervals for the data are given by:
- WikiTeX: latex reported a failure, namely:
This is pdfeTeX, Version 3.141592-1.21a-2.2 (Web2C 7.5.4)
entering extended mode
(./15856985aaf2957f4749f835ade2a
LaTeX2e <2003/12/01>
Babel
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Extra }, or forgotten $.
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Extra }, or forgotten $.
{\hbox {$\m@th \textstyle ...
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Extra }, or forgotten $.
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Extra }, or forgotten $.
{\hbox {$\m@th \scriptstyl...
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Extra }, or forgotten $.
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Extra }, or forgotten $.
{\hbox {$\m@th \scriptscri...
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Extra }, or forgotten $.
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Extra }, or forgotten $.
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Extra }, or forgotten $. \math@atom #1#2->\binrel@ {#1}\binrel@@ {#2}
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Missing $ inserted.
$
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Missing $ inserted.
$
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Missing $ inserted.
$
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Missing $ inserted.
$
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Missing { inserted.
\endgroup
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Missing { inserted.
\endgroup
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Missing { inserted.
\endgroup
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Extra }, or forgotten $. \mathaccentV ...blethree \frozen@everymath {}$#5$}
\macc@tmp \ifnum \macc@dep...
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Argument of \boldsymbol has an extra }.
\par
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Missing $ inserted.
$
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
! Missing } inserted.
}
l.5 \begin{equation*}\mathbf z^T \hat\boldsymbol
\beta \pm t_{\frac{\alpha }{...
[1] (./15856985aaf2957f4749f835ade2a.aux) ) (see the transcript file for additional information) Output written on 15856985aaf2957f4749f835ade2a.dvi (1 page, 808 bytes).
Transcript written on 15856985aaf2957f4749f835ade2a.log.[edit] Univariate linear case
We consider here the case of the simplest regression model, 
. In order to estimate 
and 
, we have a sample 
of observations which are, here, not seen as random variables and denoted by lower case letters. As stated in the introduction, however, we might want to interpret the sample in terms of random variables in some other contexts than least squares estimation.
The idea of least squares estimation is to minimize the following unknown quantity, the sum of squared errors:
Taking the derivative of the preceding expression with respect to and yields the normal equations:
This is a linear system of equations which can be solved using Cramer's rule:
The covariance matrix is
The mean response confidence interval is given by
The predicted response confidence interval is given by
The term 
is a reference to the Student's t-distribution. 
is standard error.
[edit] Analysis of variance
In analysis of variance (ANOVA), the total sum of squares is split into two or more components.
The "total (corrected) sum of squares" is
where
is the average value of observed yi. (Here "corrected" means 
has been subtracted from each y-value.) Equivalently
The total sum of squares is partitioned as the sum of the "regression sum of squares" SSReg (or RSS, also called the "explained sum of squares") and the "error sum of squares" SSE, which is the sum of squares of residuals.
The regression sum of squares is
- WikiTeX: latex reported a failure, namely:
This is pdfeTeX, Version 3.141592-1.21a-2.2 (Web2C 7.5.4)
entering extended mode
(./f39b7e7f3f1e19faf55df3bcf78c9
LaTeX2e <2003/12/01>
Babel
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Missing } inserted.
}
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Extra }, or forgotten $.
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Extra }, or forgotten $.
{\hbox {$\m@th \textstyle ...
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Extra }, or forgotten $.
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Extra }, or forgotten $.
{\hbox {$\m@th \scriptstyl...
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Extra }, or forgotten $.
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Extra }, or forgotten $.
{\hbox {$\m@th \scriptscri...
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Extra }, or forgotten $.
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Extra }, or forgotten $.
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Extra }, or forgotten $. \math@atom #1#2->\binrel@ {#1}\binrel@@ {#2}
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Missing $ inserted.
$
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Missing } inserted.
}
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Missing } inserted.
}
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Missing $ inserted.
$
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Missing } inserted.
}
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Missing } inserted.
}
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Missing $ inserted.
$
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Missing } inserted.
}
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Missing } inserted.
}
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Missing $ inserted.
$
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Missing } inserted.
}
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Missing } inserted.
}
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Missing { inserted.
\endgroup
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Missing } inserted.
}
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Missing { inserted.
\endgroup
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Missing } inserted.
}
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Missing { inserted.
\endgroup
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Missing } inserted.
}
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Missing } inserted.
}
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Extra }, or forgotten $. \mathaccentV ...blethree \frozen@everymath {}$#5$}
\macc@tmp \ifnum \macc@dep...
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Argument of \boldsymbol has an extra }.
\par
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Missing $ inserted.
$
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
! Missing } inserted.
}
l.6 = \hat\boldsymbol
\beta ^T \mathbf X^T
[1] (./f39b7e7f3f1e19faf55df3bcf78c9.aux) ) (see the transcript file for additional information) Output written on f39b7e7f3f1e19faf55df3bcf78c9.dvi (1 page, 772 bytes).
Transcript written on f39b7e7f3f1e19faf55df3bcf78c9.log.where u is an n-by-1 vector in which each element is 1. Note that
and
The error (or "unexplained") sum of squares SSE, which is the sum of square of residuals, is given by
- WikiTeX: latex reported a failure, namely:
This is pdfeTeX, Version 3.141592-1.21a-2.2 (Web2C 7.5.4)
entering extended mode
(./d38da04637f079d8793d7b771e0d5
LaTeX2e <2003/12/01>
Babel
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Missing } inserted.
}
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Extra }, or forgotten $.
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Extra }, or forgotten $.
{\hbox {$\m@th \textstyle ...
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Extra }, or forgotten $.
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Extra }, or forgotten $.
{\hbox {$\m@th \scriptstyl...
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Extra }, or forgotten $.
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Extra }, or forgotten $.
{\hbox {$\m@th \scriptscri...
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Extra }, or forgotten $.
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Extra }, or forgotten $.
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Extra }, or forgotten $. \math@atom #1#2->\binrel@ {#1}\binrel@@ {#2}
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Missing $ inserted.
$
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Missing } inserted.
}
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Missing } inserted.
}
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Missing $ inserted.
$
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Missing } inserted.
}
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Missing } inserted.
}
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Missing $ inserted.
$
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Missing } inserted.
}
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Missing } inserted.
}
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Missing $ inserted.
$
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Missing } inserted.
}
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Missing } inserted.
}
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Missing { inserted.
\endgroup
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Missing } inserted.
}
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Missing { inserted.
\endgroup
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Missing } inserted.
}
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Missing { inserted.
\endgroup
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Missing } inserted.
}
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Missing } inserted.
}
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Extra }, or forgotten $. \mathaccentV ...blethree \frozen@everymath {}$#5$}
\macc@tmp \ifnum \macc@dep...
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Argument of \boldsymbol has an extra }.
\par
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Missing $ inserted.
$
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
! Missing } inserted.
}
l.6 = \mathbf y^T \mathbf y - \hat\boldsymbol
\beta^T \mathbf X^T y.\end{equa...
[1] (./d38da04637f079d8793d7b771e0d5.aux) ) (see the transcript file for additional information) Output written on d38da04637f079d8793d7b771e0d5.dvi (1 page, 688 bytes).
Transcript written on d38da04637f079d8793d7b771e0d5.log.The total sum of squares SST is
Pearson's coefficient of regression, R 2 is then given as
If the errors are independent and normally distributed with expected value 0 and they all have the same variance, then under the null hypothesis that all of the elements in β = 0 except the constant, the statistic
follows an F-distribution with p and n − p − 1 degrees of freedom (n is the number of regressands and p + 1 is the number of unknown parameters β0, β1, ..., βp). If that statistic is too large, then one rejects the null hypothesis. How large is too large depends on the level of the test, which is the tolerated probability of type I error; see statistical significance.
[edit] Goodness of fit
Arises in the following situation: there are two statistical models:
and
Which one is better?
Let
be the sum of residuals of the j-th model (j = 1 or 2). Then the ratio
has F-distribution of n − p(1) − 1 and n − p(2) − 1 degrees of freedom, respectively.
Similar but more restricted test is presented in F-test. See also Lack-of-fit sum of squares.
[edit] Example
[edit] Regression analysis
To illustrate the various goals of regression, we give an example. The following data set gives the average heights and weights for American women aged 30–39 (source: The World Almanac and Book of Facts, 1975).
Height (m) 1.47 1.5 1.52 1.55 1.57 1.60 1.63 1.65 1.68 1.7 1.73 1.75 1.78 1.8 1.83 Weight (kg) 52.21 53.12 54.48 55.84 57.2 58.57 59.93 61.29 63.11 64.47 66.28 68.1 69.92 72.19 74.46
A plot of weight against height (see below) shows that it cannot be modeled by a straight line, so a regression is performed by modeling the data by a parabola.
where the dependent variable is weight and the independent variable is height.
Place the observations 
, in the matrix X.
The values of the parameters are found by solving the normal equations
- WikiTeX: latex reported a failure, namely:
This is pdfeTeX, Version 3.141592-1.21a-2.2 (Web2C 7.5.4)
entering extended mode
(./3b7ca9a4a84b81a5cbddf6b7405a7
LaTeX2e <2003/12/01>
Babel
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Missing } inserted.
}
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Extra }, or forgotten $.
}{\hbox {$\m@th \textstyle...
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Extra }, or forgotten $.
{\hbox {$\m@th \textstyle ...
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Extra }, or forgotten $.
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Missing } inserted.
}
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Extra }, or forgotten $.
}{\hbox {$\m@th \scriptsty...
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Extra }, or forgotten $.
{\hbox {$\m@th \scriptstyl...
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Extra }, or forgotten $.
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Missing } inserted.
}
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Extra }, or forgotten $.
}{\hbox {$\m@th \scriptscr...
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Extra }, or forgotten $.
{\hbox {$\m@th \scriptscri...
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Extra }, or forgotten $.
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Missing } inserted.
}
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Extra }, or forgotten $.
}
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Extra }, or forgotten $.
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Extra }, or forgotten $. \math@atom #1#2->\binrel@ {#1}\binrel@@ {#2}
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Missing $ inserted.
$
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Missing } inserted.
}
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Missing } inserted.
}
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Missing $ inserted.
$
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Missing } inserted.
}
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Missing } inserted.
}
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Missing $ inserted.
$
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Missing } inserted.
}
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Missing } inserted.
}
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Missing $ inserted.
$
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Missing } inserted.
}
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Missing } inserted.
}
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Missing { inserted.
\endgroup
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Missing } inserted.
}
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Missing { inserted.
\endgroup
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Missing } inserted.
}
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Missing { inserted.
\endgroup
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Missing } inserted.
}
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
! Missing } inserted.
}
l.5 ...ion*}\mathbf (X^T \mathbf X)\boldsymbol\hat
\beta = \mathbf X^T \mathb...
[1] (./3b7ca9a4a84b81a5cbddf6b7405a7.aux) ) (see the transcript file for additional information) Output written on 3b7ca9a4a84b81a5cbddf6b7405a7.dvi (1 page, 400 bytes).
Transcript written on 3b7ca9a4a84b81a5cbddf6b7405a7.log.Element ij of the normal equation matrix, 
is formed by summing the products of column i and column j of X.
Element i of the right-hand side vector 
is formed by summing the products of column i of X with the column of dependent variable values.
Thus, the normal equations are
- WikiTeX: latex reported a failure, namely:
This is pdfeTeX, Version 3.141592-1.21a-2.2 (Web2C 7.5.4)
entering extended mode
(./c8cae18e031f999f0479bf8f702c9
LaTeX2e <2003/12/01>
Babel
$
l.10
! Display math should end with $$.
\par
l.10
! Missing $ inserted.
$
l.11 \begin{pmatrix}
! Missing $ inserted.
$
l.15 \end{pmatrix}
! Missing $ inserted.
$
l.17 \begin{pmatrix}
! Missing $ inserted.
$
l.21 \end{pmatrix}
! LaTeX Error: Bad math environment delimiter.
See the LaTeX manual or LaTeX Companion for explanation.
Type H
...
l.22 \end{equation*}
! Missing $ inserted.
$
l.22 \end{equation*}
! Display math should end with $$.
\endgroup
l.22 \end{equation*}
[1] (./c8cae18e031f999f0479bf8f702c9.aux) ) (see the transcript file for additional information) Output written on c8cae18e031f999f0479bf8f702c9.dvi (1 page, 816 bytes).
Transcript written on c8cae18e031f999f0479bf8f702c9.log.
(value 
standard deviation)
The calculated values are given by
The observed and calculated data are plotted together and the residuals, 
, are calculated and plotted. Standard deviations are calculated using the sum of squares, S = 0.76.
The confidence intervals are computed using:
with 
= 5%, 
= 2.2. Therefore, we can say that the 95% confidence intervals are:
[edit] Goodness of fit
Assume that the following two models have been proposed for the data in this example:
- A linear model with β(1)0 and β(1)1 as unknown parameters.
- A quadratic model solved above.
Which of these two is better?
The null hypothesis is that there is no difference between these two models. The residual sum of squares are SSE(1) = 7.49 and SSE(2) = 0.76.
The F-distribution with 13 and 12 degrees of freedom, respectively, give for α = 95% F13,12(1 - 0.95) = 2.66. Since Q > 2.66 the zero hypothesis can be rejected on level of 95%.
[edit] Examining results of regression models
[edit] Checking model assumptions
Some of the model assumptions can be evaluated by calculating the residuals and plotting or otherwise analyzing them. The following plots can be constructed to test the validity of the assumptions:
- Residuals against the explanatory variables in the model, as illustrated above. The residuals should have no relation to these variables (look for possible non-linear relations) and the spread of the residuals should be the same over the whole range.
- Residuals against explanatory variables not in the model. Any relation of the residuals to these variables would suggest considering these variables for inclusion in the model.
- Residuals against the fitted values, WikiTeX: latex reported a failure, namely:
This is pdfeTeX, Version 3.141592-1.21a-2.2 (Web2C 7.5.4)
- A time series plot of the residuals, that is, plotting the residuals as a function of time.
- Residuals against the preceding residual.
- A normal probability plot of the residuals to test normality. The points should lie along a straight line.
There should not be any noticeable pattern to the data in all but the last plot.
[edit] Assessing goodness of fit
- The coefficient of determination gives what fraction of the observed variance of the response variable can be explained by the given variables.
- Examine the observational and prediction confidence intervals. In most contexts, the smaller they are the better.
[edit] Other procedures
[edit] Generalized least squares
Generalized least squares, which includes weighted least squares as a special case, can be used when the observational errors have unequal variance or serial correlation.
[edit] Errors-in-variables model
Errors-in-variables model or total least squares when the independent variables are subject to error
[edit] Generalized linear model
Generalized linear model is used when the distribution function of the errors is not a Normal distribution. Examples include exponential distribution, gamma distribution, inverse Gaussian distribution, Poisson distribution, binomial distribution, multinomial distribution
[edit] Robust regression
- Main article: robust regression
A host of alternative approaches to the computation of regression parameters are included in the category known as robust regression. One technique minimizes the mean absolute error, or some other function of the residuals, instead of mean squared error as in linear regression. Robust regression is much more computationally intensive than linear regression and is somewhat more difficult to implement as well. While least squares estimates are not very sensitive to breaking the normality of the errors assumption, this is not true when the variance or mean of the error distribution is not bounded, or when an analyst that can identify outliers is unavailable.
Among Stata users, Robust regression is frequently taken to mean linear regression with Huber-White standard error estimates due to the naming conventions for regression commands. This procedure relaxes the assumption of homoscedasticity for variance estimates only; the predictors are still ordinary least squares (OLS) estimates. This occasionally leads to confusion; Stata users sometimes believe that linear regression is a robust method when this option is used, although it is actually not robust in the sense of outlier-resistance.
[edit] Instrumental variables and related methods
The assumption that the error term in the linear model can be treated as uncorrelated with the independent variables will frequently be untenable, as omitted-variables bias, "reverse" causation, and errors-in-variables problems can generate such a correlation. Instrumental variable and other methods can be used in such cases.
[edit] Applications of linear regression
Linear regression is widely used in biological, behavioral and social sciences to describe possible relationships between variables. It ranks as one of the most important tools used in these disciplines.
[edit] Trend line
- For trend lines as used in technical analysis, see Trend lines (technical analysis)
A trend line represents a trend, the long-term movement in time series data after other components have been accounted for. It tells whether a particular data set (say GDP, oil prices or stock prices) have increased or decreased over the period of time. A trend line could simply be drawn by eye through a set of data points, but more properly their position and slope is calculated using statistical techniques like linear regression. Trend lines typically are straight lines, although some variations use higher degree polynomials depending on the degree of curvature desired in the line.
Trend lines are sometimes used in business analytics to show changes in data over time. This has the advantage of being simple. Trend lines are often used to argue that a particular action or event (such as training, or an advertising campaign) caused observed changes at a point in time. This is a simple technique, and does not require a control group, experimental design, or a sophisticated analysis technique. However, it suffers from a lack of scientific validity in cases where other potential changes can affect the data.
[edit] Epidemiology
As one example, early evidence relating tobacco smoking to mortality and morbidity came from studies employing regression. Researchers usually include several variables in their regression analysis in an effort to remove factors that might produce spurious correlations. For the cigarette smoking example, researchers might include socio-economic status in addition to smoking to ensure that any observed effect of smoking on mortality is not due to some effect of education or income. However, it is never possible to include all possible confounding variables in a study employing regression. For the smoking example, a hypothetical gene might increase mortality and also cause people to smoke more. For this reason, randomized controlled trials are often able to generate more compelling evidence of causal relationships than correlational analysis using linear regression. When controlled experiments are not feasible, variants of regression analysis such as instrumental variables and other methods may be used to attempt to estimate causal relationships from observational data.
[edit] See also
- Anscombe's quartet
- Cross-sectional regression
- Empirical Bayes methods
- Hierarchical linear modeling
- Instrumental variable
- Least-squares estimation of linear regression coefficients
- M-estimator
- Nonlinear regression
- Nonparametric regression
- Multiple regression
- Segmented regression
- Ridge regression
- Lack-of-fit sum of squares
- Truncated regression model
- Censored regression model
[edit] Notes
- ↑ The parameter estimators should be obtained by solving the normal equations as simultaneous linear equations. The inverse normal equations matrix need only be calculated in order to obtain the standard deviations on the parameters.
[edit] References
- Cohen, J., Cohen P., West, S.G., & Aiken, L.S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences. (2nd ed.) Hillsdale, NJ: Lawrence Erlbaum Associates
- Charles Darwin. The Variation of Animals and Plants under Domestication. (1869) (Chapter XIII describes what was known about reversion in Galton's time. Darwin uses the term "reversion".)
- Draper, N.R. and Smith, H. Applied Regression Analysis Wiley Series in Probability and Statistics (1998)
- Francis Galton. "Regression Towards Mediocrity in Hereditary Stature," Journal of the Anthropological Institute, 15:246-263 (1886). (Facsimile at: [1])
- Robert S. Pindyck and Daniel L. Rubinfeld (1998, 4h ed.). Econometric Models and Economic Forecasts,, ch. 1 (Intro, incl. appendices on Σ operators & derivation of parameter est.) & Appendix 4.3 (mult. regression in matrix form).
- Kaw, Autar; Kalu, Egwu (2008), Numerical Methods with Applications (1st ed.), [2], Chapter 6 deals with linear and non-linear regression.
[edit] External links
- http://homepage.mac.com/nshoffner/nsh/CalcBookAll/Chapter%201/1functions.html
- Investment Volatility: A Critique of Standard Beta Estimation and a Simple Way Forward, C.TofallisDownloadable version of paper, subsequently published in the European Journal of Operational Research 2008.
- Scale-adaptive nonparametric regression (with Matlab software).
- Visual Least Squares: An interactive, visual flash demonstration of how linear regression works.
- In Situ Adaptive Tabulation: Combining many linear regressions to approximate any nonlinear function.
- Earliest Known uses of some of the Words of Mathematics. See: [3] for "error", [4] for "Gauss-Markov theorem", [5] for "method of least squares", and [6] for "regression".
- Online linear regression calculator.
- Perpendicular Regression Of a Line at MathPages
- Online regression by eye (simulation).
- Leverage Effect Interactive simulation to show the effect of outliers on the regression results
- Linear regression as an optimisation problem
- Visual Statistics with Multimedia
- Multiple Regression by Elmer G. Wiens. Online multiple and restricted multiple regression package.
- ZunZun.com Online curve and surface fitting.
- CAUSEweb.org Many resources for teaching statistics including Linear Regression.
- Multivariate Regression Python, Smalltalk & Java Implementation of Linear Regression Calculation.
- [7] "Mahler's Guide to Regression"
- Linear Regression - Notes, PPT, Videos, Mathcad, Matlab, Mathematica, Maple at Numerical Methods for STEM undergraduate
- Restricted regression - Lecture in the Department of Statistics, University of Udine
Template:Least Squares and Regression Analysis
| This page uses content from the English-language version of Wikipedia. The original article was at Linear regression. The list of authors can be seen in the page history. As with Psychology Wiki, the text of Wikipedia is available under the GNU Free Documentation License. |
