# Consistent estimator

Talk0*34,135*pages on

this wiki

Assessment |
Biopsychology |
Comparative |
Cognitive |
Developmental |
Language |
Individual differences |
Personality |
Philosophy |
Social |

Methods |
Statistics |
Clinical |
Educational |
Industrial |
Professional items |
World psychology |

**Statistics:**
Scientific method ·
Research methods ·
Experimental design ·
Undergraduate statistics courses ·
Statistical tests ·
Game theory ·
Decision theory

In statistics, a sequence of estimators for parameter *θ*_{0} is said to be **consistent** (or **asymptotically consistent**) if this sequence converges in probability to *θ*_{0}. It means that the distributions of the estimators become more and more concentrated near the true value of the parameter being estimated, so that the probability of the estimator being arbitrarily close to *θ*_{0} converges to one.

In practice one usually constructs a single estimator as a function of an available sample of size *n*, and then imagines being able to keep collecting data and expanding the sample *ad infinitum*. In this way one would obtain a sequence of estimators indexed by *n* and the notion of consistency will be understood as the sample size “grows to infinity”. If this sequence converges in probability to the true value *θ*_{0}, we call it a consistent estimator; otherwise the estimator is said to be **inconsistent**.

The consistency as defined here is sometimes referred to as the *weak consistency*. When we replace the convergence in probability with the almost sure convergence, then the sequence of estimators is said to be *strongly consistent*.

## Definition Edit

Loosely speaking, an estimator *T _{n}* of parameter

*θ*is said to be

**consistent**, if it converges in probability to the true value of the parameter:

^{[1]}

A more rigorous definition takes into account the fact that *θ* is actually unknown, and thus the convergence in probability must take place for every possible value of this parameter. Suppose {*p _{θ}*:

*θ*∈ Θ} is a family of distributions (the parametric model), and

*X*= {

^{θ}*X*

_{1},

*X*

_{2}, … :

*X*~

_{i}*p*} is an infinite sample from the distribution

_{θ}*p*. Let {

_{θ}*T*(

_{n}*X*) } be a sequence of estimators for some parameter

^{θ}*g*(

*θ*). Usually

*T*will be based on the first

_{n}*n*observations of a sample. Then this sequence {

*T*} is said to be (weakly)

_{n}**consistent**if

^{[2]}

This definition uses *g*(*θ*) instead of simply *θ*, because often one is interested in estimating a certain function or a sub-vector of the underlying parameter. In the next example we estimate the location parameter of the model, but not the scale:

### Example: sample mean for normal random variables Edit

Suppose one has a sequence of observations {*X*_{1}, *X*_{2}, …} from a normal *N*(*μ*, *σ*^{2}) distribution. To estimate *μ* based on the first *n* observations, we use the sample mean: *T _{n}* = (

*X*

_{1}+ … +

*X*)/

_{n}*n*. This defines a sequence of estimators, indexed by the sample size

*n*.

From the properties of the normal distribution, we know that *T*_{n} is itself normally distributed, with mean *μ* and variance *σ*^{2}/*n*. Equivalently, has a standard normal distribution. Then

as *n* tends to infinity, for any fixed *ε* > 0. Therefore, the sequence *T _{n}* of sample means is consistent for the population mean

*μ*.

## Establishing consistency Edit

The notion of asymptotic consistency is very close, almost synonymous to the notion of convergence in probability. As such, any theorem, lemma, or property which establishes convergence in probability may be used to prove the consistency. Many such tools exist:

- In order to demonstrate consistency directly from the definition one can use the inequality
^{[3]}

the most common choice for function *h* being either the absolute value (in which case it is known as Markov inequality), or the quadratic function (respectively Chebychev's inequality).

- Another useful result is the continuous mapping theorem: if
*T*is consistent for_{n}*θ*and*g*(·) is a real-valued function continuous at point*θ*, then*g*(*T*) will be consistent for_{n}*g*(*θ*):^{[4]}

- Slutsky’s theorem can be used to combine several different estimators, or an estimator with a non-random covergent sequence. If
*T*→_{n}^{p}*α*, and*S*→_{n}^{p}*β*, then^{[5]}

- If estimator
*T*is given by an explicit formula, then most likely the formula will employ sums of random variables, and then the law of large numbers can be used: for a sequence {_{n}*X*} of random variables and under suitable conditions,_{n}

- If estimator
*T*is defined implicitly, for example as a value that maximizes certain objective function (see extremum estimator), then a more complicated argument involving stochastic equicontinuity has to be used._{n}^{[6]}

## Bias versus consistency Edit

### Unbiased but not consistent Edit

An estimator can be unbiased but not consistent. For example, for an iid sample {*x*

**Basa Sunda:**{{{1}}}

*x*

**Basa Sunda:**{{{1}}}

*} one can use*T

*(*X

*) =*x

**Basa Sunda:**{{{1}}}

as the estimator of the mean E[x].

### Biased but consistent Edit

Alternatively, an estimator can be biased but consistent. For example if the mean is estimated by it is biased, but as , it approaches the correct value, and so it is consistent.

## See alsoEdit

- Fisher consistency — alternative, although rarely used concept of consistency for the estimators
- Consistent test — the notion of consistency in the context of hypothesis testing

## Notes Edit

- ↑ Amemiya 1985, Definition 3.4.2
- ↑ Lehman & Casella 1998
- ↑ Amemiya 1985, equation (3.2.5)
- ↑ Amemiya 1985, Theorem 3.2.6
- ↑ Amemiya 1985, Theorem 3.2.7
- ↑ Newey & McFadden (1994, Chapter 2)

## References Edit

- Amemiya, Takeshi (1985).
*Advanced econometrics*, Harvard University Press. - (1998)
*Theory of Point Estimation*, 2nd, Springer. - (1994)
*Large sample estimation and hypothesis testing*, Elsevier Science. - Template:SpringerEOM

This page uses Creative Commons Licensed content from Wikipedia (view authors). |