Binomial Test Explained: How to Make, Read and Report It
The binomial test is an exact test of whether the proportion of "successes" in a sample differs from a hypothesised value. It needs no normal approximation, so it is valid for small samples.
When to use it
One binary outcome (e.g. improved / not improved), a fixed number of independent trials n, and a hypothesised probability p0. The sign test is a special case with p0 = .50.
The mathematics
P(X = k) = C(n, k) p0k (1 − p0)n − k
The p-value is the probability, under H0, of a result at least as extreme as observed. Assumptions: independent trials, a constant success probability, and a fixed n.
Original example (illustrative)
A pilot study of 20 patients finds that 15 improved after a new exercise programme. H0: p = .50; H1: p ≠ .50.
With p0 = .50, P(X ≥ 15) = (15504 + 4845 + 1140 + 190 + 20 + 1) / 220 = 21700 / 1,048,576 = .0207. Because the null distribution is symmetric, the two-sided p-value is 2 × .0207 = .0414. The observed proportion is .75, and the exact (Clopper–Pearson) 95% confidence interval is approximately [.51, .91]. The effect size is Cohen's g = .75 − .50 = .25 (Cohen, 1988).
The result is statistically significant at α = .05, but the interval is wide and its lower bound lies close to .50. In a pilot study with no control group, the improvement could reflect natural recovery or placebo effects, so the test does not demonstrate that the programme causes improvement.
How to make it in StatFAS
Open the Binomial Test Calculator. Typical inputs are number of successes, n and p0, or a raw binary column.
APA 7 reporting
An exact binomial test indicated that the proportion of patients who improved (.75, 95% CI [.51, .91]) was higher than .50, p = .041 (two-sided), n = 20.
Common mistakes and alternatives
Treating repeated measures on the same person as independent trials, choosing p0 after seeing the data, using the normal approximation with tiny n, and ignoring the interval. For paired binary data use McNemar's test; for larger samples a z-test for proportions is acceptable; for continuous data see the t-test calculator.
Conclusion
The binomial test gives an exact answer for small binary samples, but its conclusion is only as good as the independence and design behind the data.
1.
statfas-binomial-test-interface.png, alt: "StatFAS Binomial Test Calculator interface"2.
statfas-binomial-test-example-data.png, alt: "Example data entered in the StatFAS Binomial Test Calculator"3.
statfas-binomial-test-settings.png, alt: "Binomial Test Calculator settings in StatFAS"4.
statfas-binomial-test-result.png, alt: "Binomial Test Calculator result generated in StatFAS"References
- Clopper, C. J., & Pearson, E. S. (1934). The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika, 26(4), 404–413.
- Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Erlbaum.