Confidence intervals

Chapter 8 showed that a measured average is uncertain. A confidence interval is the standard way of saying how uncertain, and it is the most useful single habit in this subject: quote a range, not a point.

Construction

$\bar{x} \pm z \times \frac{s}{\sqrt{n}}$

where $z$ comes from the normal distribution for the confidence level wanted:

Confidence	$z$
90%	1.645
95%	1.960
99%	2.576

For small samples — under about 30 — the $t$ distribution replaces the normal, with slightly wider multipliers because $\sigma$ is itself estimated. With five observations the 95% multiplier is 2.776 rather than 1.96, which widens an already wide interval by 40%.

Applied to the fund: $14\% \pm 2.776 \times 9.8\% = 14\% \pm 27.2\%$, an interval from −13.2% to +41.2%.

The sentence that interval supports is something like: "over five years this fund returned 14% a year, which is consistent with anything from a substantial loss to an exceptional gain as its long-run behaviour." That is a far weaker claim than the factsheet implies, and it is the honest one.

What 95% confidence actually means

This is the part almost everyone gets wrong, and it is worth being exact.

Wrong: "there is a 95% probability the true mean lies in this interval."

Right: "if this procedure were repeated on many samples, 95% of the intervals it produced would contain the true mean."

The distinction is that the true mean is a fixed number, not a random one. It is either in your particular interval or it is not. The 95% is a property of the method, not of the interval in front of you.

Why this matters practically rather than philosophically: the guarantee is only as good as the assumptions behind the procedure. If the data is not independent, or the distribution has fat tails, the method does not deliver 95% — and no amount of staring at the interval will reveal that. A confidence interval reports sampling error and nothing else. It says nothing about a biased sample, a changed regime, or a model that was wrong to begin with.

Why financial intervals are wider than they look

Three reasons, all from chapter 8, all in the same direction:

• Dependence means the effective $n$ is below the observation count, so the true interval is wider than $\sigma/\sqrt{n}$ suggests.
• Fat tails mean the $z$ multipliers, which come from the normal distribution, understate the extremes.
• Non-stationarity means the parameter being estimated moved while you were estimating it.

Every one of these widens the honest interval. The computed interval is a lower bound on the real uncertainty, which is a useful thing to remember when one looks uncomfortably wide already.

Where this changes a decision

Comparing two funds. If their intervals overlap substantially, the data has not distinguished them. Picking the higher number is then a preference, not a finding.

Reading a backtest. A strategy returning 18% against a benchmark's 12% over three years is inside the noise. The interval on a three-year mean is enormous.

Projecting a retirement corpus. A plan built on an 11% expected return should be tested at 7% and 15%, because both are inside the interval the history supports. Chapter 5 of the Retirement subject's insistence on flexibility over precision is this point in a different vocabulary.

Assessing a manager. Distinguishing skill from luck needs decades, which is longer than most careers and far longer than most track records.

The discipline

Quote the range. When somebody gives you a single number for anything estimated from data, the useful question is not "is that right" but "how wide is the interval around it, and what was assumed to compute it".

The point

A confidence interval is a point estimate plus a multiple of the standard error, and it says the procedure captures the truth 95% of the time — not that this interval has a 95% chance of containing it. For financial data the computed interval is a floor on real uncertainty, because dependence, fat tails and shifting regimes all widen it further. Overlapping intervals mean two things have not been told apart.