Proportion differences using the beta-binomial distribution

Abstract

Scope and Method of Study: When estimating the difference between two proportions with overdispersion due to correlation within the trials, the usual asymptotic confidence interval based on the maximum likelihood estimators generally has lower than desired coverage rates for small sample sizes. Consequently, the purpose of this study is to construct confidence intervals in this setting that exhibit near nominal coverage even for small sample sizes. The beta-binomial model is one possible way to model correlated 0-1 data. This model is used to develop two new intervals, referred to as the Haldane and Jeffreys-Perks intervals. The paper then compares these two new intervals with four existing competitors and evaluates their performance via simulations.

Findings and Conclusions: The usual asymptotic interval based on the maximum likelihood estimators is discouraged for cases when the sample sizes are small or the proportions are close to zero or one. The Haldane interval is an improvement over the usual interval but still has many cases with less than desirable coverage probability. The Jeffreys-Perks interval provides significant improvement over the usual interval as do the existing intervals referred to as the extended Beal, extended Newcombe, and extended Peskun intervals. In particular, the Jeffreys-Perks interval is generally the best choice in terms of coverage probability for cases where the difference between the proportions is large. In specific cases when the two proportions are equal, or close to equal, the extended Newcombe and extended Beal generally have the best results. In many other cases, the extended Newcombe, extended Beal, and the Jeffreys-Perks intervals provide very similar results.