Scribd Logo
Upload

Welcome to Scribd!

  • 32 views

    CS-E4820 Machine Learning: Advanced Probabilistic Methods

    Uploaded by

    Gigi
    This document outlines 3 problems related to probabilistic machine learning methods for an advanced course. Problem 1 asks students to derive the posterior distribution of a Poisson-Gamma model. Problem 2 asks students to derive the posterior of a multivariate Gaussian and compare Bayesian and maximum likelihood estimates. Problem 3 asks students to simulate from a Wishart distribution prior and examine how its parameters can be adjusted to target a specific precision matrix.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    CS-E4820 Machine Learning: Advanced Probabilistic Methods

    Uploaded by

    Gigi
    32 views2 pages
    This document outlines 3 problems related to probabilistic machine learning methods for an advanced course. Problem 1 asks students to derive the posterior distribution of a Poisson-Gamma model. Problem 2 asks students to derive the posterior of a multivariate Gaussian and compare Bayesian and maximum likelihood estimates. Problem 3 asks students to simulate from a Wishart distribution prior and examine how its parameters can be adjusted to target a specific precision matrix.

    Original Title

    exercise-03

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    This document outlines 3 problems related to probabilistic machine learning methods for an advanced course. Problem 1 asks students to derive the posterior distribution of a Poisson-Gamma model. Problem 2 asks students to derive the posterior of a multivariate Gaussian and compare Bayesian and maximum likelihood estimates. Problem 3 asks students to simulate from a Wishart distribution prior and examine how its parameters can be adjusted to target a specific precision matrix.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    32 views2 pages

    CS-E4820 Machine Learning: Advanced Probabilistic Methods

    Uploaded by

    Gigi
    This document outlines 3 problems related to probabilistic machine learning methods for an advanced course. Problem 1 asks students to derive the posterior distribution of a Poisson-Gamma model. Problem 2 asks students to derive the posterior of a multivariate Gaussian and compare Bayesian and maximum likelihood estimates. Problem 3 asks students to simulate from a Wishart distribution prior and examine how its parameters can be adjusted to target a specific precision matrix.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    You are on page 1of 2

    CS-E4820 Machine Learning: Advanced Probabilistic Methods

    Pekka Marttinen, Paul Blomstedt, Juho-Kustaa Kangas, Teppo Niinimäki, Homayun


    Afrabandpey, Marko Järvenpää, Nguyen Tranquang (Spring 2018)
    Exercise problems, round 3, Due on Tuesday, 20th February 2018, at 23:55
    Please return your solutions in MyCourses as a single PDF file.
    Problem 1. “Poisson-Gamma.”
    Suppose you have N i.i.d. observations x = { xi }iN=1 from a Poisson(λ) distribution
    with a rate parameter λ that has a conjugate prior
    λ ∼ Gamma( a, b)
    with the shape and rate hyperparameters a and b. Derive the posterior distribution
    λ|x.
    Problem 2. “Multivariate Gaussian.”
    Suppose we have N i.i.d. observations X = {xi }iN=1 from a multivariate Gaussian
    distribution
    xi | µ ∼ N (µ, Σ)
    with unknown mean parameter µ and a known covariance matrix Σ. As prior infor-
    mation on the mean parameter we have
    µ ∼ N ( m 0 , S 0 ).

    (a) Derive the posterior distribution p(µ|X) of the mean parameter µ.


    (b) Compare the Bayesian estimate (posterior mean) to the maximum likelihood es-
    timate by generating N = 10 observations from the bivariate Gaussian
       
    0 1 0
    N , .
    0 0 1
    For this you can use the Python function numpy.random.normal1 , making use of
    the fact that the elements of the bivariate random vectors are independent. In
    the Bayesian case, use the prior with m0 = [0, 0] T and S0 = [ 0.1 0
    0 0.1 ]. Report both
    estimates. Is the Bayesian estimate closer to the true value µ = [0, 0] T ?
    Problem 3. “Wishart distribution.”
    Wishart distribution is the conjugate prior of the precision matrix Λ ∼ Wishart(W, ν)2
    of a multivariate normal distribution N D (µ, Λ−1 ). In Python, the function
    scipy.stats.wishart.rvs can be used to simulate samples from a Wishart distribution 3 .
    Suppose you wish to express as your prior belief that the precision matrix should be
    close to A, given by  
    2 0.3
    A= .
    0.3 0.5
    1 https://docs.scipy.org/doc/numpy/reference/generated/numpy.random.normal.html.
    2 The scale matrix W is a p × p positive definite matrix and the degrees of freedom ν is a real value

    with ν > p − 1.
    3 See documentation https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.

    wishart.html.
    (a) Familiarize yourself with the Wishart distribution, e.g., by reading from Wikipedia
    or any available book. What are the mean and variance of the Wishart distribu-
    tion?

    (b) Specify parameters of the Wishart distribution such that the expected value of
    the Λ matrix would be equal to A. Use e.g. scipy.stats.wishart.rvs(df, scale, size)
    to simulate samples from the distribution you have specified, and confirm by
    averaging over the samples that the expectation indeed is equal to A. Try 1, 10
    and 1000 samples. (The average should converge to A as the number of samples
    increases.)

    (c) How should one select the parameter values to make the matrices simulated
    from the Wishart distribution arbitrarily close to A? Show a few examples of this
    by adjusting the parameters to get increasingly closer to A.

    Your solution should include code (with some comments) that accomplishes (b) and
    (c).

    You might also like