Chapter 1 introduction, exercise 11-20 solutions

11. (*) By setting the derivatives of the log likelihood function
    $$ln p(\boldsymbol{x}|\mu, \sigma^2) = -\frac{1}{2\sigma^2}\sum_{n=1}^{N}(x_n - \mu)^2-\frac{N}{2}ln\sigma^2-\frac{N}{2}ln(2\pi) \tag{1.54}$$

    with respect to $\mu$ and $\sigma^2$ equal to zero, verify the results(1.55) and (1.56)

    $$\mu_{ML} = \frac{1}{N}\sum_{n=1}^{N}x_n \tag{1.55}$$
    $$\sigma^2_{ML} = \frac{1}{N}\sum_{n=1}^{N}(x_n - \mu_{ML})^2 \tag{1.56}$$

    Solution: for $\mu$

    $$\begin{equation} \frac{d}{d\mu}ln p(\boldsymbol{x}|\mu, \sigma^2) = \frac{1}{\sigma^2}\sum_{n=1}^{N}(x_n - \mu) = 0 \\ \mu_{ML}=\sum_{n=1}^{N}x_n \end{equation}$$

    for $\sigma^2$

    $$\begin{equation} \frac{d}{d\sigma^2}ln p(\boldsymbol{x}|\mu, \sigma^2) = \frac{1}{2}(\sigma^2)^{-2}\sum_{n=1}^{N}(x_n - \mu)^2 - \frac{N}{2}(\sigma^2)^{-1} = 0 \\ \sigma_{ML}^2=\frac{1}{N}\sum_{n=1}^{N}(x_n - \mu_{ML})^2 \end{equation}$$
12. (*)www Using the results (1.49) and (1.50), show that
    $$\mathbb{E}[x_{n}x_m] = \mu^2 + I_{nm}\sigma^2 \tag{1.130}$$

    where $x_n$ and $x_m$ denote data points sampled from a Gaussian distribution with mean $\mu$ and variance $\sigma^2$, and $I_{nm}$ satisfies $I_{nm}=1$ if $n==m$ and $I_{nm}=0$ otherwise. Hence porve the results(1.57) and (1.58).