Empirical distribution function
In statistics, the empirical distribution function, or empirical cdf, is the cumulative distribution function associated with the empirical measure of the sample. This cdf is a step function that jumps up by 1/n at each of the n data points. The empirical distribution function estimates the true underlying cdf of the points in the sample and converges with probability 1 according to the GlivenkoâCantelli theorem. A number of results exist to quantify the rate of convergence of the empirical cdf to the underlying cdf.
Contents
[hide]Definition[edit]
Let (x1, âŚ, xn) be iid real random variables with the common cdf F(t). Then the empirical distribution function is defined as [1][2]
where 1{A} is the indicator of event A. For a fixed t, the indicator 1{xi ⤠t} is a Bernoulli random variable with parameter p = F(t), hence is a binomial random variable with mean nF(t) and variance nF(t)(1 â F(t)). This implies that
is an unbiased estimator for F(t).
Asymptotic properties[edit]
By the strong law of large numbers, the estimator converges to F(t) as nââââ almost surely, for every value of t: [1]
thus the estimator is consistent.
This expression asserts the pointwise convergence of the empirical
distribution function to the true cdf. There is a stronger result,
called the GlivenkoâCantelli theorem, which states that the convergence in fact happens uniformly over t: [3]
The sup-norm in this expression is called the KolmogorovâSmirnov statistic for testing the goodness-of-fit between the empirical distribution and the assumed true cdf F. Other norm functions may be reasonably used here instead of the sup-norm. For example, the L²-norm gives rise to the CramĂŠrâvon Mises statistic.
The asymptotic distribution can be further characterized in several different ways. First, the central limit theorem states that pointwise, has asymptotically normal distribution with the standard ân rate of convergence:[1]
This result is extended by the Donskerâs theorem, which asserts that the empirical process , viewed as a function indexed by tâââR, converges in distribution in the Skorokhod space D[ââ,â+â] to the mean-zero Gaussian process GF = BâF, where B is the standard Brownian bridge.[3] The covariance structure of this Gaussian process is
The uniform rate of convergence in Donskerâs theorem can be quantified by the result known as the Hungarian embedding: [4]
Alternatively, the rate of convergence of
can also be quantified in terms of the asymptotic behavior of the
sup-norm of this expression. Number of results exist in this venue, for
example the DvoretzkyâKieferâWolfowitz inequality provides bound on the tail probabilities of
:[4]
In fact, Kolmogorov has shown that if the cdf F is continuous, then the expression converges in distribution to ||B||â, which has the Kolmogorov distribution that does not depend on the form of F.
Another result, which follows from the law of the iterated logarithm, is that [4]
and
See also[edit]
- CĂ dlĂ g functions
- DvoretzkyâKieferâWolfowitz inequality
- Empirical probability
- Empirical process
- KaplanâMeier estimator for censored processes
- Survival function
- Distribution fitting
References[edit]
- ^ Jump up to: a b c van der Vaart, A.W. (1998). Asymptotic statistics. Cambridge University Press. p. 265. ISBN 0-521-78450-6.
- Jump up ^ PlanetMath
- ^ Jump up to: a b van der Vaart, A.W. (1998). Asymptotic statistics. Cambridge University Press. p. 266. ISBN 0-521-78450-6.
- ^ Jump up to: a b c van der Vaart, A.W. (1998). Asymptotic statistics. Cambridge University Press. p. 268. ISBN 0-521-78450-6.
Further reading[edit]
- Shorack, G.R.; Wellner, J.A. (1986). Empirical Processes with Applications to Statistics. New York: Wiley. ISBN 0-471-86725-X.
External links[edit]
![]() |
Wikimedia Commons has media related to Empirical distribution functions. |