PDF of the sum of independent random variables

Today I review a result that I'm gonna use in other posts, e.g. the survivor frog problem. Also, it can be useful for you to solve mathematical problems, or to remember a simple but elegant proof. I'll give a simple but intuitive proof for the derivation of the probability density function (pdf) of the sum of independent random variables.

Imagine you have two random variables:

\[\begin{matrix} X \sim f_X(x) \\ Y \sim f_Y(y)  \end{matrix}\]

This notation denotes that each random variable follows a different probability distribution. We will assume that these two variables are independent, and our purpose is to compute the pdf of its sum $Z = X+Y$. To do so, I will start from the cumulative distribution function (cdf):

\[\begin{align} F_Z(z) &= \text{Prob}\left( Z \le z \right)   \nonumber \\ &=\text{Prob}\left( X+Y \le z \right) \nonumber \\ &= \text{Prob}\left( Y \le z - X \right) \nonumber\\ &= \int_{-\infty}^{\infty} \!\!\!\!\! \partial x \int_{-\infty}^{z - x} \!\!\!\!\! f_{XY}(x,y) \partial y \nonumber\\ &= \int_{-\infty}^{\infty} \!\!\!\!\! f_{X}(x) \partial x \int_{-\infty}^{z - x} \!\!\!\!\! f_{Y}(y) \partial y \nonumber \end{align}\]

Now, we want to compute the pdf, i.e. the derivative of the cdf. Some people will freak out here, since we have to derive an integral. No worries, fortunately we have the Leibniz rule, coming into play:

\[f_{Z}(z) = \frac{\partial}{\partial z} F_Z(z) = \int_{-\infty}^{\infty} \!\!\!\!\! f_{X}(x) f_{Y}(z-x) \partial x = f_{X}(z) * f_{Y}(z)\]

For those not familiar, the operator * is extensively used as the convolution. This operation is quite important in signal processing, since the output of linear system is the convolution of the output by the impulse response of the system, and it becomes the product in case of working in the frequency domain (after Fourier transforms). So, to conclude, the result can be read as follows:

“The pdf of the sum of two independent random variables is given by the
convolution of the two pdf's of those variables”

And that's it, folks! If you enjoyed this (short) post please like, and in case of any suggestion, do not hesitate to comment!