Sampling

Typically, it is impractical or impossible to record data on each subject in a population (a census). As such, we must draw conclusions on the population from data on a subset thereof.

The population can be viewed as a probability distribution with mean $\mu$ and variance ${\sigma }^2$.

Sample Statistics §

We can calculate the Measures of Central Tendency and Measures of Spread for individual samples, and use these to infer information about the whole population. The sample mean ($\hat{m}$) and sample variance (${\hat{s}}^2$) are calculated from the sample as usual.

Relationships Between Samples and their Populations §

The sample mean is an unbiased estimator of the population mean:

$$\mathcal{E}(\hat{m})=\mu .$$

The sample variance is not an unbiased estimator of the population variance:

$$\mathcal{E}({\hat{s}}^2)=\frac{N-1}{N}{\sigma }^2.$$

This can be rearranged to find an unbiased estimator of the population variance:

$${\hat{s}}_{unbiased}^2=\frac{N}{N-1}{\hat{s}}^2=\frac{N}{N-1}\sum _{i=1}^N(x_i-\hat{m}{)}^2.$$

As such, the sample variance is of little use to us, so we change the notation:

• $\hat{s}$ is the unbiased estimator.
• ${\hat{s}}_b$ is the biased estimator.

Large Samples §

The law of large numbers states that the empirical mean converges to the population mean as the sample size increases:

$$\underset{N\to \infty }{\lim }\hat{m}=\mu$$

This law also gives us that, for all $ϵ>0$ ,

$$\text{Prob}(|\hat{m}-\mu |\ge ϵ)\le \frac{{\sigma }^2}{N{ϵ}^2}$$

Combining Random Variables §

For any $x\sim p_x$ and $y\sim p_y$,

• Mean of $x+y$ is ${\mu }_x+{\mu }_y$.
• Variance of $x+y$ is ${\sigma }_x^2+{\sigma }_y^2$.
• Standard deviation of $x+y$ is $\sqrt{{\sigma }_x^2+{\sigma }_y^2}$. The first fact always holds, while the latter two require $x$ and $y$ to be independent.

Central Limit Theorem §

The central limit theorem states that the distribution of sample means is well approximated by a normal distribution for large sample sizes.

This means that for samples of size $N$ drawn randomly from a parent distribution with mean $\mu$ and standard deviation $\sigma$, the distribution of sample means is a normal distribution with mean $\mu$ and standard deviation $\frac{\sigma }{\sqrt{N}}$, where $N$ is large.