Asymptotic theory (statistics)

In statistics, asymptotic theory, or large sample theory, is a generic framework for assessment of properties of estimators and statistical tests. Within this framework it is typically assumed that the sample size n grows indefinitely, and the properties of statistical procedures are evaluated in the limit as n → ∞.

In practical applications, asymptotic theory is applied by treating the asymptotic results as approximately valid for finite sample sizes as well. Such approach is often criticized for not having any mathematical grounds behind it, yet it is used ubiquitously anyway. The importance of the asymptotic theory is that it often makes possible to carry out the analysis and state many results which cannot be obtained within the standard “finite-sample theory”.

Overview

Most statistical problems begin with a dataset of size n. The asymptotic theory proceeds by assuming that it is possible to keep collecting additional data, so that the sample size would grow infinitely:

$n \to \infty\,$

Under this assumption many results can be obtained that are unavailable for samples of finite sizes. As an example consider the law of large numbers. This law states that for a sequence of iid random variables X₁, X₂, …, the sample averages $\scriptstyle\overline{X}_n$ converge in probability to the population mean E[X_i] as n → ∞. At the same time for finite n it is impossible to claim anything about the distribution of $\scriptstyle\overline{X}_n$ if the distributions of individual X_i’s is unknown.

For various models slightly different modes of asymptotics may be used:

• For cross-sectional data (iid) the new observations are sampled independently, from the same fixed distribution. This is the standard case of n → ∞ asymptotics.
• For longitudinal data (time series) the sampling method may differ from model to model. Sometimes the data is assumed to be ergodic, in other applications it can be integrated or cointegrated. In this case the asymptotic is again taken as the number of observations (usually denoted T in this case) goes to infinity: T → ∞.
• For panel data, it is commonly assumed that one dimension in the data (T) remains fixed, whereas the other dimension grows: T = const, n → ∞.

Besides these standard approaches, various other “alternative” asymptotic approaches exist:

• Within the local asymptotic normality framework, it is assumed that the value of the “true parameter” in the model varies slightly with n, such that the n-th model corresponds to $\scriptstyle\theta_n\,=\,\theta+h/\sqrt{n}$. This approach lets us study the regularity of estimators.
• When statistical tests are studied for their power to distinguish against the alternatives that are close to the null hypothesis, it is done within the so-called “local alternatives” framework: the null hypothesis is H₀: θ = θ₀, and the alternative is H₁: $\scriptstyle\theta\,=\,\theta_0+h/\sqrt{n}$. This approach is especially popular for the unit root tests.
• There are models where the dimension of the parameter space Θ_n slowly expands with n, reflecting the fact that the more observations a statistician has, the more he is tempted to introduce additional parameters in the model. An example of this is the weak instruments asymptotic.
• In kernel density estimation and kernel regression additional parameter — the bandwidth h — is assumed. In these models it is typically taken that h → 0 as n → ∞, however the rate of convergence must be chosen carefully, usually h ∝ n^−1/5.

Modes of convergence of random variables

Further information: Convergence of random variables

Asymptotic properties

Estimators

• Consistency: a sequence of estimators is said to be consistent, if it converges in probability to the true value of the parameter being estimated:

$\hat\theta_n\ \xrightarrow{p}\ \theta_0$

Generally an estimator is just some, more or less arbitrary, function of the data. The property of consistency requires that the estimator was estimating the quantity we intended it to. As such, it is the most important property in the estimation theory: estimators that are known to be inconsistent are never used in practice.

• Asymptotic distribution: if it is possible to find sequences of non-random constants {a_n}, {b_n} (possibly depending on the value of θ₀), and a non-degenerate distribution G such that

$b_n(\hat\theta_n - a_n)\ \xrightarrow{d}\ G ,$

then the sequence of estimators $\scriptstyle\hat\theta_n$ is said to have the asymptotic distribution G.

Most often, the estimators encountered in practice have the asymptotically normal distribution, with a_n = θ₀, b_n = √n, and G = N(0, V):

$\sqrt{n}(\hat\theta_n - \theta_0)\ \xrightarrow{d}\ \mathcal{N}(0, V).$

• Asymptotic confidence regions.
• Regularity.

Asymptotic theorems

• Law of large numbers
• Central limit theorem
• Slutsky’s theorem
• Continuous mapping theorem