Sampling and the central limit theorem

You never observe a distribution. You observe a handful of draws from it and try to work backwards. Every return figure, every volatility estimate, every correlation in this course is a sample statistic standing in for a population parameter nobody can see.

The distinction, and the notation

Population (unknown)	Sample (measured)
$\mu$ — true mean return	$\bar{x}$ — average of what you saw
$\sigma$ — true volatility	$s$ — measured volatility
$\rho$ — true correlation	$r$ — measured correlation

A sample statistic is itself a random variable: run history again and you would get a different number. The question is how different.

The central limit theorem

For a sample of $n$ independent draws from any distribution with finite variance, the distribution of the sample mean approaches a normal distribution as $n$ grows:

$\bar{X} \sim N\!\left(\mu,\ \frac{\sigma^2}{n}\right)$

Two claims in there, and the second is the useful one.

It does not matter what the underlying distribution is. Skewed, fat-tailed, discrete — the sample mean still tends to normality. This is why the normal distribution keeps appearing even where the data is clearly not normal, and it is a stronger result than it first sounds.

The spread of the sample mean shrinks as $\sqrt{n}$. The standard error:

$SE = \frac{\sigma}{\sqrt{n}}$

What $\sqrt{n}$ costs you

The square root is the brutal part. To halve your uncertainty you need four times the data; to divide it by ten you need a hundred times.

Work the problem. A fund with $s = 22\%$ over $n = 5$ years:

$SE = \frac{22}{\sqrt{5}} = 9.8\%$

The measured 14% carries a standard error of nearly 10 percentage points. A two-standard-error range — roughly 95% confidence, chapter 9 — runs from about −6% to +34%.

Five years of data cannot distinguish a very good fund from a losing one. That is not a complaint about the fund; it is arithmetic. To get the standard error down to 2% would need around 120 years of returns, by which time the fund, its manager and its mandate are all different things.

This is the real content behind SEBI requiring every performance disclosure to carry the line that past performance may or may not be sustained. The regulator is not being lawyerly. The sample is too small to support the inference most readers draw from it.

Three ways the theorem does not apply

The CLT assumes independent, identically distributed draws with finite variance. Financial data violates all three.

Not independent. Volatility clusters: turbulent days follow turbulent days. Chapter 13 measures this. Dependence means the effective sample size is smaller than the count of observations, so the true standard error is larger than $\sigma/\sqrt{n}$ — the formula flatters you.

Not identically distributed. The market that produced 2008's returns is not the market of 2024. Averaging across regimes estimates a parameter that no longer exists.

Tails are heavy. Convergence to normality is slow when the underlying has fat tails, and for some distributions the sample mean never settles at all. Chapter 6's point, arriving again.

The bias no sample size fixes

Survivorship bias is not sampling error, and this is the key distinction: more data does not help, because the data is systematically wrong rather than noisy.

Funds that performed badly are merged away or closed. A list of funds available today is a list of the ones that survived, so an average computed across it overstates what an investor would actually have experienced. The same mechanism inflates index backtests, strategy backtests and any study of "companies that did X".

Sampling error is random and shrinks with $n$. Selection bias is systematic and does not. Collecting ten times more of a biased sample gives a more precise estimate of the wrong number.

What this changes in practice

• Treat any performance figure from under a decade as approximately uninformative about skill.
• Prefer long samples, and then distrust them for a different reason — regimes change.
• Ask what is missing from a dataset before asking what it says.
• When two funds differ by less than a standard error, they have not been distinguished.

The point

Every statistic in finance is a sample standing in for something unobservable. The central limit theorem says the sample mean is approximately normal around the truth with a standard error of $\sigma/\sqrt{n}$ — so five years of a 22%-volatility fund gives a 95% range roughly 40 percentage points wide. Dependence, changing regimes and fat tails all make that understate the real uncertainty, and survivorship bias is a separate problem that more data cannot fix.