Statistics as Parameter Estimation §

If we interpret the possible parameters for a given probability distribution (eg. $\sigma$ and $\mu$ for the normal distribution, or $n$ and $p$ for the binomial distribution) as hypotheses, then we have a continuous hypothesis space.

This leads to statistical questions of the form “Given the following data, what are the parameters of the distribution from which it is sampled?“.

For example, to determine whether a coin is fair, we have the posterior $P(\theta \mid Observation)$ for the “success parameter” $\theta \in [0,1]$ and the observed data from $N$ trials. To compute this, we have

$$P(\theta \mid n,N)=\frac{P((n,N)\mid \theta )P(\theta )}{P((n,N))}$$

Where $(n,N)$ is the number of successes $n$ in the number of trials $N$. Computing $P((n,N)\mid \theta )$ as

$$\prod _{i=1}^{N}P(x_i\mid \theta )$$

requires an assumption that all the trials are independent.

As $P(x_i\mid \theta )$ is $\theta$ when $x_i$ is a success, and $(1-\theta )$ otherwise, This formula can be expanded and simplified to give that

$$P((n,N)\mid \theta ,I)={\theta }^n(1-\theta {)}^{N-n}.$$