(I am currently studying a high dimensional probability course with very little background knowledge in probability theory as a whole, so I hope it's not annoying that I seem to be oblivious about basic concepts, yet am using more involved ideas. Note: I have a good background understanding of measure theory.)
I am having difficulty understanding how to calculate expectation in the following way:
So, by definition I understand that formally $$\mathbb{E}[X]:=\int_\Omega{}X(\omega)d\mathbb{P}(\omega).$$
And that moment generating function is defined as $M_X(\lambda):=\mathbb{E}[\exp(\lambda{}X)]$, and this is unique, so if two random variables have the same $M_X(\lambda)$ their distributions coincide. Now I am trying to show that the following random variable is normally distributed:
Let $Y$ be a random Gaussian vector and $u\in\mathbb{R}^n$ (each of its components are standard normally distributed). I am trying to show that $\langle Y,u\rangle$~$N(0,\|u\|_2^2)$ (where $\langle\cdot,\cdot\rangle$ is the standard Euclidean scalar product).
I have shown that the mean is 0 and variance is $\|u\|_2^2$ but from my understand this isn't enough. How would I calculate the moment generating function of $\langle Y,u\rangle$ and show that this coincides with that of a normal distribution, or is there an easier way of doing so?
1 Answer
So you know that $$ \langle Y,u\rangle = \sum_{i=1}^n u_i Y_i $$ (that's the usual inner product over $\mathbb{R}^n$), so that for every $\lambda \in \mathbb{R}$ $$ \mathbb{E}[e^{\lambda \langle Y,u\rangle}] = \mathbb{E}[e^{\lambda \sum_{i=1}^n u_i Y_i}]= \mathbb{E}[\prod_{i=1}^n e^{\lambda u_i Y_i}] $$ Now, you can use the fact that the $Y_i$'s are independent (and therefore the $e^{\lambda u_i Y_i}$'s are independent) to get $$ \mathbb{E}[e^{\lambda \langle Y,u\rangle}] = \prod_{i=1}^n \mathbb{E}[e^{\lambda u_i Y_i}] $$ But each $Y_i$ is a standard normal, so you can explicitly compute $\mathbb{E}[e^{t Y_i}]$ for any $t$ (in particular for $t=\lambda u_i$).
Can you take it from there to compute $\mathbb{E}[e^{\lambda \langle Y,u\rangle}]$ and compare the resulting expression that that of the MGF of a (univariate) $\mathcal{N}(0,\lVert u\rVert_2^2)$?
In more detail: recall that the MGF of $Z\sim \mathcal{N}(0,\sigma^2)$ is given by $$ \forall \lambda,\; \mathbb{E}[e^{\lambda Z}] = e^{\frac{1}{2}\lambda^2\sigma^2} \tag{$\dagger$} $$ so that, for every $1\leq i\leq n$, since $Y_i \sim \mathcal{N}(0,1)$ we have $$ \mathbb{E}[e^{\lambda u_i Y_i}] = e^{\frac{1}{2}\lambda^2u_i^2} $$ and therefore $$ \mathbb{E}[e^{\lambda \langle Y,u\rangle}] = \prod_{i=1}^n \mathbb{E}[e^{\lambda u_i Y_i}] = \prod_{i=1}^n e^{\frac{1}{2}\lambda^2u_i^2} = e^{\frac{1}{2}\lambda^2 \sum_{i=1}^nu_i^2} = e^{\frac{1}{2}\lambda^2 \lVert u\rVert_2^2} \tag{$\ddagger$} $$ No, compare $(\ddagger)$ to $(\dagger)$ to conclude about the distribution of $\langle Y,u\rangle$ based on its MGF.
