Scribd Logo
Upload

Welcome to Scribd!

  • 4 views

    HW4 Solutions

    Uploaded by

    teju1996cool

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    HW4 Solutions

    Uploaded by

    teju1996cool
    4 views2 pages

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    4 views2 pages

    HW4 Solutions

    Uploaded by

    teju1996cool

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    You are on page 1of 2

    APMA 1660 Homework 4 Solutions

    Problem 1. Prove that


    n n  
    ∂ ˜ X X 1
    β) =
    ln (β (Yi − pi ) · Xij = Yi − · Xij = 0, for all j = 1, 2, ..., p.
    ∂βj X iβ )
    1 + exp(−X
    i=1 i=1

    Solution. In order to compute β̂ β MLE , which is the argmax of ˜ln (β


    β ), we compute the first
    ˜ β ) and equate them to zero. We have
    order partial derivatives of ln (β

    n
    X
    ˜ln (β
    β) = [Yi · log pi + (1 − Yi ) · log(1 − pi )]
    i=1
    1
    where pi = 1+exp(−X X iβ ) is the logistic function. Let us compute the partial derivatives of pi
    with respect to βj :

    ∂ −1
    pi = · exp(−X
    X iβ ) · (−Xij )
    ∂βj X iβ ))2
    (1 + exp(−X

    ∂ 1 − pi
    =⇒ pi = −p2i (−Xij ) = pi (1 − pi )Xij
    ∂βj pi

    Thus, after simplifying we get


    n
    ∂ ˜ X 1 1
    β) =
    ln (β [Yi · pi (1 − pi )Xij + (1 − Yi ) · (−pi )(1 − pi )Xij ]
    ∂βj pi 1 − pi
    i=1
    n
    X
    = (Yi − pi ) · Xij
    i=1

    β ) of the logistic model,


    Problem 2. Given the score function sn (β
    ∂
    (a) Calculate the following derivative: β)
    β sn (β
    ∂β
    (b) Write an explicit formula for implementing Newton’s method, i.e., compute the “ob-
    served Fisher information” of the logistic model.
    ∂ ˜
    Solution. Note that the j th -component of sn (β
    β ) is β
    ∂βj ln (β ).
    Hence, we have
    n
    ∂ ˜ X
    β) =
    ln (β (Yi − pi ) · Xij
    ∂βj
    i=1

    To compute the Hessian, we compute the second order derivative as follows:


    n
    ∂2 ˜ X
    β) = −
    ln (β pi (1 − pi )Xij Xik
    ∂βj ∂βk
    i=1

    1
    by using the partial derivatives of pi
    ∂
    pi = pi (1 − pi )Xij
    ∂βj

    Since the observed Fisher information is just the negative of the Hessian, we have the following
    Newton’s method
    n n −1
    (ν+1) (ν) (ν) (ν) (ν)
    X  X
    β
    β̂ β
    = β̂ + (Yi − pi (β̂
    β Xi
    ))X β
    pi (β̂ )(1 − pi (β̂
    β X Ti X i
    ))X , ν = 0, 1, 2, ...
    i=1 i=1

    where care must be taken to remember that X i and β are row vectors. Also note that since
    the Hessian/Fisher information is a sum of outer products, it is positive semidefinite, ensuring
    that it is symmetric.

    You might also like