central-limit-theorem

Scripts to visualize the central limit theorem in action.

The central limit theorem (CLT) is an important theorem in probability and statistics. While there exist rigorous proofs of CLT, I thought it'd be nice to visualize it in action without going into the mathematical nitty-gritties involving characteristic functions.

Here is Wolfram's explanation of CLT, but that can be somewhat esoteric to understand.

I like to explain CLT as follows:

We are given a population with any distribution, with a well-defined mean and variance . We then take samples from the population, with each sample containing data points each.

The sample mean for a given sample is defined as

There are samples, so we now can calculate sample means. We can treat as a random variable itself. This is called the sampling distribution of the sample means.

The central limit theorem tells us that the sampling distribution of the sample means () is approximately a normal distribution when m is large (m > 35 is usually good enough) with:

• Mean
• Variance

where and are the population mean and population standard deviation respectively.

Keep in mind that the population may have any distribution, so long as that distribution has a well-defined mean and variance . So it is pretty remarkable that the sampling distribution of the sample means always obeys a normal distribution for such a distribution. No wonder the normal distribution is so important!

Refer: Central limit theorem, Sampling distribution of sample means

Examples

Exponential distribution

The exponential distribution with looks pretty different from the normal distribution.

The sampling distribution of the mean looks like a normal distribution. The mean of the sampling distribution of the sample means is pretty close to the actual population mean . Also, is equal to the population variance .

Custom distribution

Here is another distribution that doesn't follow any particular formula, and is far from normal, but has a well-defined mean and variance.

The sampling distribution looks like a normal distribution, and also has skew and excess kurtosis values close to 0 (as do normal distributions). The mean of the sampling distribution of the sample means is equal to the actual population mean . Also, is pretty close to the population variance .