Nonlinear regression
From Psychology Wiki
Community portal · Tasks to do · News · Help
Clinical · Educational · Ind&Org · Other fields · Professional · Transpersonal · World
Assessment |
Biopsychology |
Comparative |
Cognitive |
Developmental |
Language
Personality |
Philosophy |
Research Methods |
Social |
Statistics
Statistics: Scientific method · Research methods · Experimental design · Undergraduate statistics courses · Statistical tests · Game theory · Decision theory
In statistics, nonlinear regression is a form of regression analysis in which observational data are modeled by a function which is a nonlinear combination of the model parameters and depends on one or more independent variables. The data are fitted by a method of successive approximations.
Contents |
[edit] General
The data consist of error-free independent variables (explanatory variable), x, and their associated observed dependent variables (response variable), y. The ys are modeled as a random variable with mean a nonlinear function f(x,β). Systematic error may be present but its treatment is outside the scope of regression analysis. If the independent variables are not error-free, this is an errors-in-variables model, also outside this scope.
For example, the Michaelis-Menten model for enzyme kinetics
- WikiTeX: latex reported a failure, namely:
This is pdfTeXk, Version 3.141592-1.40.3 (Web2C 7.5.6)
%&-line parsing enabled.
entering extended mode
(./54a66896129ec16b36a8c370aff0a
LaTeX2e <2005/12/01>
Babel
\mathop
l.5 ...= \frac{V_\max[\mbox{S}]}{K_m + [\mbox{S}]}
\end{equation*}
! Missing } inserted.
}
l.5 ...= \frac{V_\max[\mbox{S}]}{K_m + [\mbox{S}]}
\end{equation*}
[1] (./54a66896129ec16b36a8c370aff0a.aux) ) (see the transcript file for additional information) Output written on 54a66896129ec16b36a8c370aff0a.dvi (1 page, 444 bytes).
Transcript written on 54a66896129ec16b36a8c370aff0a.log.can be written as
is the parameter WikiTeX: latex reported a failure, namely:This is pdfTeXk, Version 3.141592-1.40.3 (Web2C 7.5.6) %&-line parsing enabled. entering extended mode (./77d0de90ca8aa710e20af2732cdea LaTeX2e <2005/12/01> Babel,and hyphenation patterns for english, usenglishmax, dumylang, noh yphenation, loaded. (/usr/share/texmf-texlive/tex/latex/base/article.cls Document Class: article 2005/09/16 v1.4f Standard LaTeX document class (/usr/share/texmf-texlive/tex/latex/base/size10.clo)) (/usr/share/texmf-texlive/tex/latex/amsfonts/amssymb.sty (/usr/share/texmf-texlive/tex/latex/amsfonts/amsfonts.sty)) (/usr/share/texmf-texlive/tex/latex/amsmath/amsmath.sty For additional information on amsmath, use the `?' option. (/usr/share/texmf-texlive/tex/latex/amsmath/amstext.sty (/usr/share/texmf-texlive/tex/latex/amsmath/amsgen.sty)) (/usr/share/texmf-texlive/tex/latex/amsmath/amsbsy.sty) (/usr/share/texmf-texlive/tex/latex/amsmath/amsopn.sty)) (/usr/share/texmf-texlive/tex/latex/amsmath/amscd.sty) (/usr/share/texmf-texlive/tex/latex/concmath/concmath.sty) (./77d0de90ca8aa710e20af2732cdea.aux) (/usr/share/texmf-texlive/tex/latex/concmath/ot1ccr.fd) (/usr/share/texmf-texlive/tex/latex/concmath/omlccm.fd) (/usr/share/texmf-texlive/tex/latex/concmath/omsccsy.fd) (/usr/share/texmf-texlive/tex/latex/concmath/omxccex.fd) (/usr/share/texmf-texlive/tex/latex/amsfonts/umsa.fd) (/usr/share/texmf-texlive/tex/latex/amsfonts/umsb.fd) ! Missing { inserted. \mathop l.5 \begin{equation*}V_\max \end{equation*} ! Missing } inserted. } l.5 \begin{equation*}V_\max\end{equation*} [1] (./77d0de90ca8aa710e20af2732cdea.aux) ) (see the transcript file for additional information) Output written on 77d0de90ca8aa710e20af2732cdea.dvi (1 page, 260 bytes). Transcript written on 77d0de90ca8aa710e20af2732cdea.log.
is the parameter 
and [S] is the independent variable, x. This function is nonlinear because it cannot be expressed as a linear combination of the 
s.
Other examples of nonlinear functions include exponential functions, logarithmic functions, trigonometric functions, power functions, Gaussian function, and Lorentzian curves. Some functions, such as the exponential or logarithmic functions, can be transformed so that they are linear. When so transformed, standard linear regression can be performed but must be applied with caution. See Linearization, below, for more details.
In general, there is no closed-form expression for the best-fitting parameters, as there is in linear regression. Usually numerical optimization algorithms are applied to determine the best-fitting parameters. Again in contrast to linear regression, there may be many local minima of the function to be optimized. In practice, estimated values of the parameters are used, in conjunction with the optimization algorithm, to attempt to find the global minimum of a sum of squares.
For details concerning nonlinear data modeling see least squares and non-linear least squares.
[edit] Regression statistics
The assumption underlying this procedure is that the model can be approximated by a linear function.
where 
. It follows from this that the least squares estimators are given by
- WikiTeX: latex reported a failure, namely:
This is pdfTeXk, Version 3.141592-1.40.3 (Web2C 7.5.6)
%&-line parsing enabled.
entering extended mode
(./2d213ed94b412d27fd69c3215e07a
LaTeX2e <2005/12/01>
Babel
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Extra }, or forgotten $.
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Extra }, or forgotten $.
{\hbox {$\m@th \textstyle ...
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Extra }, or forgotten $.
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Extra }, or forgotten $.
{\hbox {$\m@th \scriptstyl...
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Extra }, or forgotten $.
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Extra }, or forgotten $.
{\hbox {$\m@th \scriptscri...
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Extra }, or forgotten $.
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Extra }, or forgotten $.
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Extra }, or forgotten $. \math@atom #1#2->\binrel@ {#1}\binrel@@ {#2}
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Missing $ inserted.
$
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Missing $ inserted.
$
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Missing $ inserted.
$
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Missing $ inserted.
$
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Missing { inserted.
\endgroup
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Missing { inserted.
\endgroup
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Missing { inserted.
\endgroup
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Extra }, or forgotten $. \mathaccentV ...blethree \frozen@everymath {}$#5$}
\macc@tmp \ifnum \macc@dep...
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Argument of \boldsymbol has an extra }.
\par
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Missing $ inserted.
$
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
! Missing } inserted.
}
l.5 \begin{equation*}\hat\boldsymbol
\beta \approx \mathbf { (J^TJ)^{-1}J^Ty}...
[1] (./2d213ed94b412d27fd69c3215e07a.aux) ) (see the transcript file for additional information) Output written on 2d213ed94b412d27fd69c3215e07a.dvi (1 page, 496 bytes).
Transcript written on 2d213ed94b412d27fd69c3215e07a.log.The nonlinear regression statistics are computed and used as in linear regression statistics, but using J in place of X in the formulas. The linear approximation introduces bias into the statistics. Therefore more caution than usual is required in interpreting statistics derived from a nonlinear model.
[edit] Ordinary and weighted least squares
The best-fit curve is often assumed to be that which minimizes the sum of squared residuals. This is the (ordinary) least squares (OLS) approach. However, in cases where the dependent variable does not have constant variance a sum of weighted squared residuals may be minimized; see weighted least squares. Each weight should ideally be equal to the reciprocal of the variance of the observation, but weights may be recomputed on each iteration, in an iteratively weighted least squares algorithm.
[edit] Linearization
Some nonlinear regression problems can be moved to a linear domain by a suitable transformation of the model formulation.
For example, consider the nonlinear regression problem (ignoring the error):
If we take a logarithm of both sides, it becomes
suggesting estimation of the unknown parameters by a linear regression of ln(y) on x, a computation that does not require iterative optimization. However, use of a linear transformation requires caution. The influences of the data values will change, as will the error structure of the model and the interpretation of any inferential results. These may not be desired effects. On the other hand, depending on what the largest source of error is, a linear transformation may distribute your errors in a normal fashion, so the choice to perform a linear transformation must be informed by modeling considerations.
For Michaelis-Menten kinetics, the linear Lineweaver–Burk plot
- WikiTeX: latex reported a failure, namely:
This is pdfTeXk, Version 3.141592-1.40.3 (Web2C 7.5.6)
%&-line parsing enabled.
entering extended mode
(./81f3b7d173c4d299ba95122e50374
LaTeX2e <2005/12/01>
Babel
\mathop
l.5 ...n{equation*} \frac{1}{v} = \frac{1}{V_\max}
+ \frac{K_m}{V_\max[S]}\e...
! Missing { inserted.
\mathop
l.5 ... = \frac{1}{V_\max} + \frac{K_m}{V_\max[S]}
\end{equation*}
! Missing } inserted.
}
l.5 ...max} + \frac{K_m}{V_\max[S]}\end{equation*}
! Missing } inserted.
}
l.5 ...max} + \frac{K_m}{V_\max[S]}\end{equation*}
[1] (./81f3b7d173c4d299ba95122e50374.aux) ) (see the transcript file for additional information) Output written on 81f3b7d173c4d299ba95122e50374.dvi (1 page, 604 bytes).
Transcript written on 81f3b7d173c4d299ba95122e50374.log.of 1/v against 1/[S] has been much used. However, it is very sensitive to data error and it is strongly biased toward fitting the data in a particular range of the independent variable, [S], its use is strongly deprecated.
[edit] See also
[edit] References
- G.A.F Seber and C.J. Wild. Nonlinear Regression. New York: John Wiley and Sons, 1989.
- R.M. Bethea, B.S. Duran and T.L. Boullion. Statistical Methods for Engineers and Scientists. New York: Marcel Dekker, Inc 1985 ISBN 0-8247-7227-X
- K. Schittkowski. Data Fitting in Dynamical Systems. Kluwer, 2002.
Template:Least squares and regression analysis
| This page uses content from the English-language version of Wikipedia. The original article was at Nonlinear regression. The list of authors can be seen in the page history. As with Psychology Wiki, the text of Wikipedia is available under the GNU Free Documentation License. |
