Appendix on conditional distributions

This is to give further discussion on conditional distributions. The basic discussion can be found at https://gaomj.cn/probability4/

The following definitions and theorems are NOT stated in a completely rigorous style. Thus the rigorous statements may be referred to other advanced materials on probability theory.

Suppose we have a random vector $(X,Y)$ with joint distribution measure $\mathbb P_{X,Y}$. We first define the marginal distribution:

Definition: Define the projection $\pi_X:(x,y)\mapsto x$, then the marginal distribution of $X$ can be defined as the pushforward measure of $\mathbb P_{X,Y}$:
$\mathbb P_X:=\mathbb P_{X,Y}\circ\pi_X^{-1}.$
With this definition, we have by the property of pushforward:
$\mathbb P(X\in E_x)=\int_{\pi^{-1}_X(E_x)}\mathrm d\mathbb P_{X,Y}=\int_{E_x}\mathrm d\mathbb P_X.$Let $\kappa_{Y,X}$ be a regular conditional distribution of $Y$ given $X$ such that $\kappa_{Y,X}(x,\cdot)=\mathbb P(\cdot\mid X=x)$. Then,

Theorem:

1.
$\mathbb P_{X,Y}(E)=\int\kappa(x,\pi_X^{-1}(x)\cap E)\, \mathrm d\mathbb P_X.$

2.
$\int f(x,y)\, \mathbb P_{X,Y}(\mathrm dx,\mathrm dy)=\int\mathrm d\mathbb P_X\int f(x,y)\, \kappa_{Y,X}(x,\mathrm dy).$

3.
$\mathbb P_Y(E_y)=\int_{\Omega_1}\kappa_{Y,X}(x,E_y)\, \mathrm d\mathbb P_X.$

Therefore, it can often be seen that $\mathbb P_{X,Y}=\mathbb P_X\mathbb P_{Y|X}$.

The following is their informal proof. (1) We write
$\begin{aligned} \mathbb P(E)&=\int \mathrm d\mathbb P_{X,Y}\, \mathbb P((x,y)\in E)\\ &=\int_{\Omega_1}\mathrm d(\mathbb P_{X,Y}\circ\pi_X^{-1})\, \kappa(x,\pi_X^{-1}(x)\cap E)\\ &=\int_{\Omega_1}\mathrm d\mathbb P_X\, \kappa(x,\pi_X^{-1}(x)\cap E). \end{aligned}\begin{aligned} \mathbb P(E)&=\int \mathrm d\mathbb P_{X,Y}\, \mathbb P((x,y)\in E)\\ &=\int_{\Omega_1}\mathrm d(\mathbb P_{X,Y}\circ\pi_X^{-1})\, \kappa(x,\pi_X^{-1}(x)\cap E)\\ &=\int_{\Omega_1}\mathrm d\mathbb P_X\, \kappa(x,\pi_X^{-1}(x)\cap E). \end{aligned}$(2) We derive it from (See Theorem 24 in https://gaomj.cn/probability4/#sec:2.3)
$\int f(Y(\omega))\, \mathbb P(\mathrm d\omega)=\int \mathbb P(\mathrm d\omega)\int f(y)\kappa_{Y,\sigma(X)}(\omega,\mathrm dy).$Then
$\begin{aligned} \int f(X(\omega),Y(\omega))\, \mathbb P(\mathrm d\omega)&=\int \mathbb P(\mathrm d\omega)\int f(X(\omega),y)\kappa_{Y,\sigma(X)}(\omega,\mathrm dy), \\ \int f(x,y)\, \mathbb P_{X,Y}(\mathrm dx,\mathrm dy)&=\int\mathrm d\mathbb P_X\int f(x,y)\kappa_{Y,X}(x,\mathrm dy). \end{aligned}\begin{aligned} \int f(X(\omega),Y(\omega))\, \mathbb P(\mathrm d\omega)&=\int \mathbb P(\mathrm d\omega)\int f(X(\omega),y)\kappa_{Y,\sigma(X)}(\omega,\mathrm dy), \\ \int f(x,y)\, \mathbb P_{X,Y}(\mathrm dx,\mathrm dy)&=\int\mathrm d\mathbb P_X\int f(x,y)\kappa_{Y,X}(x,\mathrm dy). \end{aligned}$

(3) With the result of (2), we have
$\begin{aligned}\mathbb P(Y\in E_y)&=\int_{y\in E_y}\mathrm d\mathbb P_{X,Y}=\int\mathrm d\mathbb P_X\int_{E_y}\kappa_{Y,X}(x,\mathrm dy)\&=\int\mathrm d\mathbb P_X\, \kappa_{Y,X}(x,E_y).\end{aligned}\begin{aligned}\mathbb P(Y\in E_y)&=\int_{y\in E_y}\mathrm d\mathbb P_{X,Y}=\int\mathrm d\mathbb P_X\int_{E_y}\kappa_{Y,X}(x,\mathrm dy)\&=\int\mathrm d\mathbb P_X\, \kappa_{Y,X}(x,E_y).\end{aligned}$