Bayesian Notations

 

Bayes theorem

$ \pi(\theta | \mathbf{x}) = \frac{f(\mathbf{x}, \theta)}{\pi(\mathbf{x})} = \frac{f(\mathbf{x} | \theta) \pi(\theta)}{\pi(\mathbf{x})} = \frac{f(\mathbf{x} | \theta) \pi(\theta)}{\int_{\theta}{f(\mathbf{x} | \theta) \pi(\theta)} d\theta}$

Model

$ likelihood: \ f(\mathbf{x} | \theta_1, \theta_2, \theta_3) $

Prior

$ \pi(\theta_1), \pi(\theta_2), \pi(\theta_3) $

Full posterior

\(\pi(\theta_1, \theta_2, \theta_3 | \mathbf{x}) = \frac{f(\mathbf{x} | \theta_1, \theta_2, \theta_3) \pi(\theta_1, \theta_2, \theta_3)}{\int_{\theta}{f(\mathbf{x} | \theta_1, \theta_2, \theta_3) \pi(\theta_1, \theta_2, \theta_3)} d\theta}\)
\(\quad\quad\quad\quad\quad\quad \propto f(\mathbf{x} | \theta_1, \theta_2, \theta_3) \pi(\theta_1, \theta_2, \theta_3)\)

Full conditional posterior

\(\pi(\theta_1 | \theta_2, \theta_3, \mathbf{x}) = \frac{\pi(\theta_1, \theta_2, \theta_3 | \mathbf{x})}{\pi(\theta_2, \theta_3 | \mathbf{x})} \quad\quad \quad\quad \textit{given $\theta_2, \theta_3, \mathbf{x}$ is constant}\)
\(\quad\quad\quad\quad\quad\quad \propto \pi(\theta_1, \theta_2, \theta_3 | \mathbf{x}) \quad\quad \quad\quad \textit{only $\theta_1$-dependent part is the kernel of conditional posterior}\)
\(\quad\quad\quad\quad\quad\quad \propto f(\mathbf{x} | \theta_1, \theta_2, \theta_3) \cdot \pi(\theta_1, \theta_2, \theta_3)\)
\(\quad\quad\quad\quad\quad\quad \propto f(\mathbf{x} | \theta_1, \theta_2, \theta_3) \cdot \pi(\theta_1) \quad\quad \quad\quad \textit{if $\theta_1$ is independent to $\theta_2, \theta_3$}\)
\(\quad\quad\quad\quad\quad\quad = \prod_i f(x_i | \theta_1, \theta_2, \theta_3) \cdot \pi(\theta_1) \quad\quad \quad\quad \textit{if $\{x_i\}$ are independent}\)


or

\[\begin{equation} \begin{aligned} \pi(\theta_1, \theta_2, \theta_3 | \mathbf{x}) &\propto f(\mathbf{x} | \theta_1, \theta_2, \theta_3) \ \pi(\theta_1, \theta_2, \theta_3) \\ &\propto f(\mathbf{x} | \theta_1, \theta_2, \theta_3) \ \pi(\theta_1 | \theta_2, \theta_3) \ \pi(\theta_2, \theta_3) \\ &\propto f(\mathbf{x} | \theta_1, \theta_2, \theta_3) \ \pi(\theta_1 | \theta_2, \theta_3) \ \pi(\theta_2 | \theta_3) \ \pi(\theta_3) \\ \end{aligned} \end{equation}\]

Here, select one parameter and set rest parameters into given condition.
Then, only the terms related to the one parameter remain on the RHS

\[\begin{equation} \begin{aligned} \pi(\theta_1 | \theta_2, \theta_3, \mathbf{x}) &\propto \pi(\theta_1, \theta_2, \theta_3 | \mathbf{x}) \ \color{blue}{\pi(\theta_2, \theta_3 | x)} \\ &\propto \color{red}{\pi(\theta_1, \theta_2, \theta_3 | \mathbf{x})} \\ &\propto \color{blue}{f(\mathbf{x} | \theta_1, \theta_2, \theta_3)} \ \pi(\theta_1 | \theta_2, \theta_3) \ \color{blue}{\pi(\theta_2 | \theta_3) \ \pi(\theta_3)} \\ &\propto \pi(\theta_1 | \theta_2, \theta_3) \\ \end{aligned} \end{equation}\]