3D Gaussian Distribution with python

- 12월 21, 2021

Multivariate Gaussian distributions are widely used in machine learning. However, that distributions are not easily understood. This post covers how to visualize 2d multivariate distributions with python. 3D Gaussian Distribution can help you to understand that easily.

Case I (independent)

\begin{align}Let \, \mu_x= \begin{bmatrix}0 \\ 0\\ \end{bmatrix}, cov =R= \begin{bmatrix}1 & 0\\ 0 & 1\\ \end{bmatrix}\end {align}

In this case we know multivariate Gaussian distribution is given by $p(x)=\frac{1}{\sqrt{(2\pi)^n}det R} exp(-\frac{1}{2}(x-\mu_x)^TR^{-1}(x-\mu_x))$ . The result represents the result when both $x_1$ and $x_2$ are independent probabilities.

The shape of a circle indicates that the probabilities are randomly distributed. The shape relates to off-diagonal matrix elements. 3d surface indicates probability of combination of two variables such as (0,1), (0,0)-Very high, or (2,2)-very low. Let’s think second case (off-diagonal elements is not zero)

Case II

The definition of covariance is as follows. $Covariance = \mathbb{E}[X-E(X)][Y-E(Y)]$ , its matrix form is as follows. $X_1 \in \mathbb{R}^n$ , $X_2 \in \mathbb{R}^n$

Covariance = \begin{bmatrix} \mathbb{E}[X_1-\mathbb{E}(X_1)][X_1-\mathbb{E}(X_1)]& \mathbb{E}[X_1-\mathbb{E}(X_1)][X_2-\mathbb{E}(X_2)]\\ \mathbb{E}[X_2-\mathbb{E}(X_2)][X_1-\mathbb{E}(X_1)] & \mathbb{E}[X_2-\mathbb{E}(X_2)][X_2-\mathbb{E}(X_2)]\\ \end{bmatrix} \\= \begin{bmatrix} \Sigma_{11}& \Sigma_{12}\\ \Sigma_{21}& \Sigma_{22}\\ \end{bmatrix}

Covariance means main effect interaction physically. In the above figure, ab represents $\Sigma_{12} or \Sigma_{21}. That is, value of [latex] x_1$ affects the distribution of $x_2$ . The case below shows a negative relationship. one value increases, other value tends to decrease.

Let \, \mu_x= \begin{bmatrix}0 \\ 0\\ \end{bmatrix}, cov =R= \begin{bmatrix}1 & -0.5\\ -0.5 & 1\\ \end{bmatrix}