MNPM

A Short Introduction into
Multinomial Probit Models

Univ. Prof. Dr. Reinhard Hujer
Chair of Statistics and Econometrics
University of Frankfurt/M.
The Multinomial Probit Model (1)

Nested Logit models introduced in the previous lecture are one way to avoid the IIA assumption.
Another one is the use of multinomial Probit models.
Recall once again how we introduced multinomial Logit models. Our aim was to model probabilities
for the M different outcomes of the dependent variable yi in such a way that they sum up to unity:
p(yi = 1) + p(yi = 2) + . . . + p(yi = M ) = 1
To meet this restriction we used the following function:
exp(ij )
p(yi = j) = Fj (ij ) = PM
i=1 exp(ij )
where assuming ij = x0ij leads to the conditional and ij = x0ij to the multinomial Logit model.
1. Multinomial Probit Model
Univ. Prof. Dr. Reinhard Hujer, University Frankfurt/M.
The Multinomial Probit model (2)

It can be shown that assuming a latent threshold model:
yij
= ij + i
where the s are independent distributed according to a type I (Gumbel) extreme value distribution
and where the observable dependent variable yi is linked with its latent counterpart yi via:
(
j if yij
= max(yi1
, yi2
, . . . , yiM
)
yi =
0 otherwise
implies that the probability for choosing category j is given by:
exp(ij )
p(yi = j) = Fj (ij ) = PM
i=1 exp(ij )
In this sense these two model formulations are equivalent and lead to the same choice probabilities.

One problem with multinomial Logit models was the IIA assumption imposed by them. This is due
to the fact that the s were assumed to be independent distributed from each other, i.e. from each
other, i.e. the covariance matrix E(0) restricted to be a diagonal matrix.
Although this independence has the advantage that the likelihood function is quite easy to compute,
in most of the cases the IIA assumption leads to unrealistic predictions (recall the famous red-busblue-bus example). One alternative to break down the IIA assumption therefore consists in allowing
the s to be correlated with each other and that is exactly what the multinomial Probit model
does.
Assume again the following model for the latent variable yij
:
yij
= x0ij + ij .
In the multinomial Probit model it is assumed that the is follows a multivariate normal distribution
with covariance matrix where now is not restricted to be a diagonal matrix.

Multinomial Probit models assume that the is follow a multivariate normal distribution and are
correlated across choices:
11 . . . 1M
...
.
M N D(0, ), with = IN and = E(i0i) =
M 1 . . . M M
Category j is chosen if yij

is highest for j, i.e.:
(
j if yij
= max(yi1
, yi2
, . . . , yiM
)
yi = j if yij =
0 otherwise.
The probability to choose category j can then be written as:
p(yi = j|xi) = p(yij

> yi1
, . . . , yij
> yi(j1)
, yij
> yi(j+1)
, . . . , yij
> yiM
)
= p (ij i1) > x0i(1 j ), . . . , (ij i(j1)) > x0i((j1) j ),
(ij i(j+1)) >

4. Multinomial Probit Models
x0i((j+1)
j ), . . . , (ij iM ) >
x0i(M
j ) .
Looking at this probability one can see that only the differences between the yij
s are identified and
hence a reference category has to be assigned as it was the case for the Logit model. As a consequence
the covariance matrix also reduced in its dimension from (M M ) to (M 1) (M 1).
If we define il ij il and il x0i(l j ) for l = 1, . . . , (j 1), (j + 1), . . . , M then the

probability p(yi = j|xi) is given by:
Z
...
i1
i(j1) i(j+1)
Z
...
(
1i, . . . , i(j1), i(j+1), . . . , iM )d
i1 . . . d
i(j1)d
i(j+1) . . . d
iM
iM
And here lies the practical obstacle with multinomial Probit models. There are no closed form
expressions for such high dimensional integrals and hence for M 3 one has to simulate them
using monte carlo simulation techniques.
5. Multinomial Probit Models
Evaluating Integrals Using Monte Carlo Techniques (1)

In the following we will illustrate the basic idea of monte carlo simulation techniques.
Assume for simplicity that we want to calculate the following one-dimensional integral:
Z
I=
f (x)dx.
a
If we rewrite this expression as follows . . .

1
I=
ba
b
a
1
f (x)dx = E(f (x)).
ba
. . . f (x) can be treated as a random variable which is uniformly distributed in the interval [a, b]
and hence the right hand side of the above expression represents the expected value of f (x).

We can therefore rewrite the integral as:
I = (b a)E(f (x)).
Next we take D draws ui from an uniform distribution in the interval [0, 1] and transform it
according to xi = a + (b a) ui. xi will then follow an uniform distribution in the interval [a, b].
Using this series of xi draws we can approximate the expected value of f (x) by averaging the
function f (x) evaluated at the different xis:
D
1 X
f (xi).
E(f (x))
D i=1
Multiplying this expression with (b a) finally we get an approximation for the integral I:
D
1 X
I (b a)
f (xi).
D i=1

Consider as an example the following simple integral:
Z
I=
exdx.
The exact value for this integral is 4.67. In the following we approximated this integral for
D = 10, 20, 50 and 100 draws, respectively. The following table contains the so obtained values
and the absolute deviations from the true value. One can see that with more draws the approximation
gets better.
Draws
I approx.
I exact
Absolute deviation
10
4.58
4.67
0.09
20
4.38
4.67
0.29
50
4.79
4.67
0.12
100
4.75
4.67
0.08

The previously presented proceeding was a simple simulator applied to an univariate integral.
However, estimating a multinomial probit model amounts to evaluate a multidimensional integral
like the following:
Z
...
1
(j1) (j+1)
Z
...
(
1, . . . , (j1), (j+1), . . . , M )d
1 . . . d
(j1)d
(j+1) . . . d
M
Note that we have skipped the individual index for notational convenience. Such integrals reflect the
probabilities for choosing a certain category. The above integral e.g. is the probability for choosing
category j, i.e. p(yi = j|xi).
There are number of different simulators which can be used in this context. Examples include
the Accept-Reject Procedure, Importance Sampling, Gibbs Sampling and the Metropolis-Hastings
Algorithm. One algorithm which has been found to be fast and accurate is the Geweeke-HajivassiliouKeane smooth recursive simulator (GHK-simulator) which will be presented in the following.

We will illustrate the GHK-simulator using a multinomial Probit model with 4 categories, i.e.
y {0, 1, 2, 3}. Since only utility differences matter, the dimension is reduced by one so that the
probability for p(yi = 1|xi) e.g. is given by the following three-fold integral:
p(yi = 1|xi) = p(
0 > 0, 2 > 2, 3 > 3)
Z + Z + Z +
=
3(
0, 2, 3|xi, )d
0d
2d
3
0
with l j l and l x0i(l j ) for l = 0, 2, 3 where the s are distributed according to a

threefold normal distribution with covariance matrix .
The probability p(yi = 1|xi) = p(
0 > 0, 2 > 2, 3 > 3) can be rewritten as a product of one
unconditional and two conditional probabilities according to:
p(
0 > 0)p(
2 > 2|
0 > 0)p(
3 > 3|
2 > 2, 0 > 0).

Since is a positive definite matrix using the Cholesky decomposition, a lower triangular matrix
can be found such that: 0 = . Defining:

1

= 2 with N (0, 1)
3
we can rewrite as = and arrive at the following and arrive at the following relation between
the s and the s:
0 = 111
2 = 121 + 222
3 = 131 + 232 + 333
where ij is the (i, j)-element in the matrix.

Using these relations, the product of unconditional and conditional probabilities can equivalently be
written as:
0
p(yi = j|xi) = p 1 >
11
2 121
0
p 2 >
>
1 11
22
0
3 131 232
2 121
>
p 3 >
,
>
.
1
2
33
11
22
The advantage of this expression is the fact that the s are independent normal distributed random
variables and hence the probability p(yi = j|xi) which we want to evaluate can be equivalently
expressed as a product of independent but conditioned univariate cumulative density functions.

Assume now that 1 and 2 are realizations taken from truncated normal distributions with lower
0
12 1
truncation points 11
and 2
, respectively.
22
Drawing samples from these truncated normal distributions ensures that the conditioning of the
probabilities is taken into account. Plugging 1 and 2 into the previous expression, we can rewrite
it as a product of only unconditional, univariate and independent probabilities:
0
p(yi = j|xi) = p 1 >
11
2 121
p 2 >
22
3 131 232
p 3 >
.
33

The GHK simulator now generates a series of d = 1, . . . , D random observations of 1d and 2d
so that the probability p(yi = j|xi) that we seek for can be approximated by:

N
d
d
d
X
1
0
2 + 121
3 + 131 + 232
p(yi = j|xi) =
.
D i=1
11
22
33
All other probabilities can be calculated in an analogous way. Plugging them into the likelihood
function standard maximization procedures can be applied to get estimates for the parameters.
Quote of the day:
As we know, computers cannot integrate.
Kenneth E. Train (2003)

MNPM

Uploaded by

Copyright:

Available Formats

MNPM

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

MNPM

Uploaded by

Copyright:

Available Formats

A Short Introduction into

Multinomial Probit Models

The Multinomial Probit Model (1)

1. Multinomial Probit Model

Univ. Prof. Dr. Reinhard Hujer, University Frankfurt/M.

The Multinomial Probit model (2)

Univ. Prof. Dr. Reinhard Hujer, University Frankfurt/M.

The Multinomial Probit Model (3)

3. Multinomial Probit Model

Univ. Prof. Dr. Reinhard Hujer, University Frankfurt/M.

The Multinomial Probit Model (4)

Category j is chosen if yij

The probability to choose category j can then be written as:

p(yi = j|xi) = p(yij

= p (ij i1) > x0i(1 j ), . . . , (ij i(j1)) > x0i((j1) j ),

(ij i(j+1)) >

Univ. Prof. Dr. Reinhard Hujer, University Frankfurt/M.

The Multinomial Probit Model (5)

If we define il ij il and il x0i(l j ) for l = 1, . . . , (j 1), (j + 1), . . . , M then the

5. Multinomial Probit Models

Univ. Prof. Dr. Reinhard Hujer, University Frankfurt/M.

Evaluating Integrals Using Monte Carlo Techniques (1)

If we rewrite this expression as follows . . .

6. Multinomial Probit Model

Univ. Prof. Dr. Reinhard Hujer, University Frankfurt/M.

Evaluating Integrals Using Monte Carlo Techniques (2)

Univ. Prof. Dr. Reinhard Hujer, University Frankfurt/M.

Evaluating Integrals Using Monte Carlo Techniques (3)

8. Multinomial Probit Model

Evaluating Integrals Using Monte Carlo Techniques (4)

Univ. Prof. Dr. Reinhard Hujer, University Frankfurt/M.

Evaluating Integrals Using Monte Carlo Techniques (5)

with l j l and l x0i(l j ) for l = 0, 2, 3 where the s are distributed according to a

Univ. Prof. Dr. Reinhard Hujer, University Frankfurt/M.

Evaluating Integrals Using Monte Carlo Techniques (6)

Univ. Prof. Dr. Reinhard Hujer, University Frankfurt/M.

Evaluating Integrals Using Monte Carlo Techniques (7)

12. Multinomial Probit Model

Univ. Prof. Dr. Reinhard Hujer, University Frankfurt/M.

Evaluating Integrals Using Monte Carlo Techniques (8)

13. Multinomial Probit Model

Univ. Prof. Dr. Reinhard Hujer, University Frankfurt/M.

Evaluating Integrals Using Monte Carlo Techniques (9)

Quote of the day:

14. Multinomial Probit Model

As we know, computers cannot integrate.

Kenneth E. Train (2003)

Univ. Prof. Dr. Reinhard Hujer, University Frankfurt/M.

You might also like