Uniformity Topics

1 Pointwise vs.
Uniform Consistency in Level

Let Xi , i = 1, . . . , n be i.i.d. data from some distribution P ∈ P. Suppose
one wishes to test the null hypothesis H0 : P ∈ P0 ( P. To this end, one
may consider a test function φn = φn (X1 , . . . , Xn ) such that controls the
probability of a Type 1 error in some sense. Ideally, we would like the test
to satisfy
EP [φn ] ≤ α for all P ∈ P0 , (1)
but many times this is too demanding a requirement. As a result, we may

settle instead for tests such that
lim sup EP [φn ] ≤ α for all P ∈ P0 . (2)

n→∞
Tests satisfying (1) are said to be of level α for P ∈ P0 , whereas tests

satisfying (2) are said to be pointwise asymptotically of level α for P ∈ P0 .
The hope is that if (2) holds, then (1) holds approximately, at least for large
enough n. But this is not true. All that (2) ensures is that for each P ∈ P0
and > 0 there is an N (P ) such that for all n > N (P )
EP [φn ] ≤ α + .
Importantly, the sample size required for the approximation to work, N (P ),

may depend on P . As a result, it could be the case that for every sample
size n (even, e.g., for n = 1010 ) there could be P = Pn ∈ P0 such that
EP [φn ] α .
Consider the following concrete example of this phenomenon. Suppose

P = {P on R : 0 < σ 2 (P ) < ∞} and P0 = {P ∈ P : µ(P ) = 0}. Let φn
√
be the t-test; that is, φn = I{ nX̄n > σ̂n z1−α }, where z1−α is the 1 − α
quantile of the standard normal distribution. We know that
EP [φn ] → α for all P ∈ P0 ,
but it turns out that the t-test suffers from the problem described above.
In fact, we can show that for every 0 < c < 1 and every sample size n there
1
exists a P = Pn,c such that
EP [φn ] ≥ c .
To see this, let n and c be given. Let P be the distribution that puts mass
1 − p on p and mass p on −(1 − p). We will specify p in a minute, but first
note that for such a distribution P all of the Xi are in fact equal to p > 0
with probability (1 − p)n . For such a sequence of observations, σ̂n = 0 and
√
nX̄n > 0, so φn = 1. The probability of rejection, EP [φn ], is therefore at
least (1 − p)n . Now all that remains is to choose p so that (1 − p)n = c; that
is, p = 1 − c1/n .
To rule this very disturbing possibility out, we need to ensure that the
convergence in (2) is uniform for P ∈ P0 ; that is,
lim sup sup EP [φn ] ≤ α . (3)

n→∞ P ∈P0
Tests satisfying (3) are said to be uniformly asymptotically of level α for

P ∈ P0 . This requirement implies that for each > 0 there is an N (which
does not depend on P ) such that for all n > N
EP [φn ] ≤ α + .
In the case of the t-test, the above example shows us that this is not true
for P = {P on R : 0 < σ 2 (P ) < ∞} and P0 = {P ∈ P : µ(P ) = 0}.
It is possible that this shortcoming is due to the t-test – – perhaps
there are other tests of the same null hypothesis that would behave more
reasonably. Unfortunately, we can show that this is not the case, provided
that P is “‘sufficiently rich”. Formally, we have the following result due to
Bahadur and Savage (1956):
Theorem 1.1 Let P be a class of distributions on R such that
(i) For every P ∈ P, µ(P ) exists and is finite;
(ii) For every m ∈ R, there is P ∈ P such that µ(P ) = m;
2
(iii) P is convex in the sense that if P1 and P2 are in P, then γP1 +(1−γ)P2
is in P for γ ∈ [0, 1].
Let Xi , i = 1, . . . , n be i.i.d. with distribution P ∈ P. Let φn be any test of

the null hypothesis H0 : µ(P ) = 0. Then,
(a) Any test of H0 which has size α for P has power ≤ α for any alternative
P ∈ P.
(b) Any test of H0 which has power β against some alternative P ∈ P has
size ≥ β.
The proof of this result will follow from the following lemma:
Lemma 1.1 Let Xi , i = 1, . . . , n be i.i.d. with distribution P ∈ P, where a

P is the class of distributions on R satisfying (i) - (iii) in Theorem 2.1. Let
φn be any test function. Define
Pm = {P ∈ P : µ(P ) = m} .
Then,
inf EP [φn ] and sup EP [φn ]
P ∈Pm P ∈Pm
are independent of m.
Proof: We show first that supP ∈Pm EP [φn ] does not depend on m. Let m
be given and choose m0 6= m. We wish to show that
sup EP [φn ] = sup EP [φn ] .

P ∈Pm0 P ∈Pm
To this end, choose Pj , j ≥ 1 so that
lim EPj [φn ] = sup EP [φn ] .

j→∞ P ∈Pm
Let hj be defined so that
1 1
m0 = (1 − )m + hj .
j j
3
Choose Hj so that µ(Hj ) = hj . Define
1 1
Gj = (1 − )Pj + Hj .
j j
Thus, Gj ∈ Pm0 . Note that with probability (1 − 1j )n , a sample of size n

from Gj is simply a sample of size n from Pj . Therefore,
1
sup EP [φn ] ≥ EGj [φn ] ≥ (1 − )n EPj [φn ] .
P ∈Pm0 j
But (1 − 1j )n → 1 and EPj [φn ] → supP ∈Pm EP [φn ] as j → ∞. Therefore,
sup EP [φn ] ≥ sup EP [φn ] .

P ∈Pm0 P ∈Pm
Interchanging the roles of m and m0 , we can establish the reverse inequality
sup EP [φn ] ≤ sup EP [φn ] .

P ∈Pm0 P ∈Pm
We could replace φn with 1 − φn to establish that inf P ∈Pm EP [φn ] does not
depend on m.
Proof of Theorem 2.1: (a) Let φn be a test of size α for P. Let P 0 be

any alternative. Define m = µ(P 0 ). Then,
EP 0 [φn ] ≤ sup EP [φn ] = sup EP [φn ] = α .

P ∈Pm P ∈P0
The proof of (b) is similar.
The class of distributions with finite second moment satisfies the require-
ments of the theorem, as does the class of distributions with infinitely many
moments. Thus, the failure of the t-test is not special to the t-test; in this
setting, there simply exist no “reasonable” tests. But this does not mean
that all hope is lost. By restricting the class of distributions some, it is pos-
sible to construct reasonable tests about the mean. In fact, the t-test does
satisfy (3) for certain large classes of distributions that are somewhat smaller
than P0 . See Chapter 11 of Lehmann and Romano (2005) for details.

Uniformity Topics

Uploaded by

Copyright:

Available Formats

Uniformity Topics

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Uniformity Topics

Uploaded by

Copyright:

Available Formats

1 Pointwise vs.

Uniform Consistency in Level

but many times this is too demanding a requirement. As a result, we may

lim sup EP [φn ] ≤ α for all P ∈ P0 . (2)

Tests satisfying (1) are said to be of level α for P ∈ P0 , whereas tests

Importantly, the sample size required for the approximation to work, N (P ),

Consider the following concrete example of this phenomenon. Suppose

EP [φn ] → α for all P ∈ P0 ,

lim sup sup EP [φn ] ≤ α . (3)

Tests satisfying (3) are said to be uniformly asymptotically of level α for

Theorem 1.1 Let P be a class of distributions on R such that

(i) For every P ∈ P, µ(P ) exists and is finite;

(ii) For every m ∈ R, there is P ∈ P such that µ(P ) = m;

Let Xi , i = 1, . . . , n be i.i.d. with distribution P ∈ P. Let φn be any test of

Lemma 1.1 Let Xi , i = 1, . . . , n be i.i.d. with distribution P ∈ P, where a

sup EP [φn ] = sup EP [φn ] .

To this end, choose Pj , j ≥ 1 so that

lim EPj [φn ] = sup EP [φn ] .

Let hj be defined so that

Thus, Gj ∈ Pm0 . Note that with probability (1 − 1j )n , a sample of size n

But (1 − 1j )n → 1 and EPj [φn ] → supP ∈Pm EP [φn ] as j → ∞. Therefore,

sup EP [φn ] ≥ sup EP [φn ] .

Interchanging the roles of m and m0 , we can establish the reverse inequality

sup EP [φn ] ≤ sup EP [φn ] .

Proof of Theorem 2.1: (a) Let φn be a test of size α for P. Let P 0 be

EP 0 [φn ] ≤ sup EP [φn ] = sup EP [φn ] = α .

The proof of (b) is similar.

You might also like