What does a p-value actually mean? I keep getting the interpretation wrong on practice exams.

The p-value is the most misinterpreted statistic in all of finance and science. Let's nail down what it actually means.

Correct Definition:

The p-value is the probability of obtaining a test statistic at least as extreme as the one observed, assuming the null hypothesis is true.

What a p-value of 0.03 means:

"If the null hypothesis were true, there would be only a 3% chance of seeing data this extreme or more extreme purely by random chance."

What a p-value of 0.03 does NOT mean:

• NOT: "There's a 3% chance the null hypothesis is true" (this is the most common error)
• NOT: "There's a 97% chance the alternative is true"
• NOT: "The result is 97% reliable"

Decision Rule:

• If p-value <= significance level (α) --> Reject the null hypothesis
• If p-value > α --> Fail to reject the null hypothesis

Example:

An analyst at Beacon Asset Management tests whether a fund's average return exceeds 0% (one-tailed test).

• H₀: μ <= 0% (fund doesn't beat zero)
• H₁: μ > 0% (fund beats zero)
• Significance level: α = 0.05
• Computed p-value: 0.018

Since 0.018 < 0.05, she rejects H₀ and concludes the fund's average return is statistically significantly greater than zero.

Important nuances:

1. Statistical vs. practical significance: A p-value of 0.001 for a fund returning 0.01% annually is statistically significant but practically meaningless
2. p-values depend on sample size: With enormous samples, even trivially small effects produce low p-values
3. "Fail to reject" ≠ "Accept": You never accept the null; you just don't have enough evidence to reject it

Exam tip: The CFA exam tests whether you know the correct interpretation. If an answer choice says "the probability that the null hypothesis is true is 3%" — that's always wrong. The p-value is about the data, not about the hypothesis.

Join our CFA community for more statistics discussions.