Scribd Logo
Upload

Welcome to Scribd!

  • 2 views

    MLE

    Uploaded by

    Hicham Atatri

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    MLE

    Uploaded by

    Hicham Atatri
    2 views4 pages

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    2 views4 pages

    MLE

    Uploaded by

    Hicham Atatri

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    You are on page 1of 4

    Maximum Likelihood Estimator (MLE) and Ex-

    amples
    **Steps to Compute the Maximum Likelihood Estimator (MLE)**: 1. **Define
    the Probability Model**: Specify the probability distribution that describes the
    data and its parameter(s) θ.
    2. **Write the Likelihood Function**: For a set of observations x1 , x2 , . . . , xN ,
    the likelihood function is:
    N
    Y
    L(θ) = f (xi ; θ),
    i=1

    where f (xi ; θ) is the probability density (or mass) function.


    3. **Log-Likelihood Function** (Optional for Simplification): Take the nat-
    ural logarithm of the likelihood function:
    N
    X
    ℓ(θ) = log L(θ) = log f (xi ; θ).
    i=1

    4. **Differentiate the Log-Likelihood**: Compute the derivative of ℓ(θ) with


    respect to θ:
    ∂ℓ(θ)
    .
    ∂θ
    5. **Set the Derivative to Zero**: Solve ∂ℓ(θ) ∂θ = 0 to find the critical
    point(s).
    6. **Verify the Maximum**: Use the second derivative test or other criteria
    to ensure the critical point is a maximum.
    7. **Solve for θ̂**: Solve the equation to obtain the MLE, θ̂.
    —

    Examples of MLE for Common Distributions


    1. Binomial Distribution
    n
    
    X ∼ Binomial(n, p), where P (X = k) = k pk (1 − p)n−k .
    Likelihood Function:
    N  
    Y n xi
    L(p) = p (1 − p)n−xi .
    i=1
    x i

    Ignoring constants:
    N
    Y
    L(p) ∝ pxi (1 − p)n−xi .
    i=1

    1
    Log-Likelihood Function:
    N
    X
    ℓ(p) = [xi log p + (n − xi ) log(1 − p)] .
    i=1

    Differentiate and Solve:


    PN PN
    ∂ℓ(p) i=1 xi (n − xi )
    = − i=1 = 0.
    ∂p p 1−p
    PN
    i=1 xi
    p̂ = .
    Nn

    2. Normal Distribution
    X ∼ N (µ, σ 2 ), where

    1 (x−µ)2
    f (x; µ, σ 2 ) = √ e− 2σ 2 .
    2πσ 2
    Likelihood Function:
    N
    Y 1 (xi −µ)2
    L(µ, σ 2 ) = √ e− 2σ 2 .
    i=1 2πσ 2

    Log-Likelihood Function:
    N
    N N 1 X
    ℓ(µ, σ 2 ) = − log(2π) − log(σ 2 ) − 2 (xi − µ)2 .
    2 2 2σ i=1

    Solve for Parameters: - For µ:


    N
    1 X
    µ̂ = xi .
    N i=1

    - For σ 2 :
    N
    1 X
    σ̂ 2 = (xi − µ̂)2 .
    N i=1

    3. Poisson Distribution
    λk e−λ
    X ∼ Poisson(λ), where P (X = k) = k! .
    Likelihood Function:
    N
    Y λxi e−λ
    L(λ) = .
    i=1
    xi !

    2
    Ignoring constants:
    N
    Y
    L(λ) ∝ λxi e−λ .
    i=1

    Log-Likelihood Function:
    N
    X
    ℓ(λ) = [xi log λ − λ − log(xi !)] .
    i=1

    Differentiate and Solve:


    PN
    ∂ℓ(λ) xi
    i=1
    = − N = 0.
    ∂λ λ
    PN
    xi
    λ̂ = i=1 .
    N

    4. Exponential Distribution
    X ∼ Exponential(λ), where f (x; λ) = λe−λx , x ≥ 0.
    Likelihood Function:
    N
    Y
    L(λ) = λe−λxi .
    i=1

    Log-Likelihood Function:
    N
    X
    ℓ(λ) = N log λ − λ xi .
    i=1

    Differentiate and Solve:


    N
    ∂ℓ(λ) N X
    = − xi = 0.
    ∂λ λ i=1

    N
    λ̂ = PN .
    i=1 xi

    5. Pareto Distribution
    X ∼ Pareto(α, xm ), where
    −α−1
    f (x; α, xm ) = αxα
    mx , x ≥ xm .

    Likelihood Function:
    N
    Y
    −α−1
    L(α) = αxα
    m xi .
    i=1

    3
    Log-Likelihood Function:
    N
    X
    ℓ(α) = N log α + N α log xm − (α + 1) log xi .
    i=1

    Differentiate and Solve:


    N
    ∂ℓ(α) N X
    = + N log xm − log xi = 0.
    ∂α α i=1

    N
    α̂ = PN .
    i=1 (log xi − log xm )

    You might also like