Appendix on conditional distributions

This is to give further discussion on conditional distributions. The basic discussion can be found at https://gaomj.cn/probability4/

The following definitions and theorems are NOT stated in a completely rigorous style. Thus the rigorous statements may be referred to other advanced materials on probability theory.

Suppose we have a random vector (X,Y) with joint distribution measure \mathbb P_{X,Y}. We first define the marginal distribution:

Definition: Define the projection \pi_X:(x,y)\mapsto x, then the marginal distribution of X can be defined as the pushforward measure of \mathbb P_{X,Y}:
\mathbb P_X:=\mathbb P_{X,Y}\circ\pi_X^{-1}.
With this definition, we have by the property of pushforward:
\mathbb P(X\in E_x)=\int_{\pi^{-1}_X(E_x)}\mathrm d\mathbb P_{X,Y}=\int_{E_x}\mathrm d\mathbb P_X.Let \kappa_{Y,X} be a regular conditional distribution of Y given X such that \kappa_{Y,X}(x,\cdot)=\mathbb P(\cdot\mid X=x). Then,

Theorem:

1.
\mathbb P_{X,Y}(E)=\int\kappa(x,\pi_X^{-1}(x)\cap E)\, \mathrm d\mathbb P_X.

2.
\int f(x,y)\, \mathbb P_{X,Y}(\mathrm dx,\mathrm dy)=\int\mathrm d\mathbb P_X\int f(x,y)\, \kappa_{Y,X}(x,\mathrm dy).

3.
\mathbb P_Y(E_y)=\int_{\Omega_1}\kappa_{Y,X}(x,E_y)\, \mathrm d\mathbb P_X.

Therefore, it can often be seen that \mathbb P_{X,Y}=\mathbb P_X\mathbb P_{Y|X}.

The following is their informal proof. (1) We write
\begin{aligned}
\mathbb P(E)&=\int \mathrm d\mathbb P_{X,Y}\, \mathbb P((x,y)\in E)\\
&=\int_{\Omega_1}\mathrm d(\mathbb P_{X,Y}\circ\pi_X^{-1})\, \kappa(x,\pi_X^{-1}(x)\cap E)\\
&=\int_{\Omega_1}\mathrm d\mathbb P_X\, \kappa(x,\pi_X^{-1}(x)\cap E).
\end{aligned}(2) We derive it from (See Theorem 24 in https://gaomj.cn/probability4/#sec:2.3)
\int f(Y(\omega))\, \mathbb P(\mathrm d\omega)=\int \mathbb P(\mathrm d\omega)\int f(y)\kappa_{Y,\sigma(X)}(\omega,\mathrm dy).Then
\begin{aligned}
\int f(X(\omega),Y(\omega))\, \mathbb P(\mathrm d\omega)&=\int \mathbb P(\mathrm d\omega)\int f(X(\omega),y)\kappa_{Y,\sigma(X)}(\omega,\mathrm dy),
\\
\int f(x,y)\, \mathbb P_{X,Y}(\mathrm dx,\mathrm dy)&=\int\mathrm d\mathbb P_X\int f(x,y)\kappa_{Y,X}(x,\mathrm dy).
\end{aligned}

(3) With the result of (2), we have
\begin{aligned}\mathbb P(Y\in E_y)&=\int_{y\in E_y}\mathrm d\mathbb P_{X,Y}=\int\mathrm d\mathbb P_X\int_{E_y}\kappa_{Y,X}(x,\mathrm dy)\&=\int\mathrm d\mathbb P_X\, \kappa_{Y,X}(x,E_y).\end{aligned}