Supplementary Exercise 6.7 of IPS7e ----------------------------------- Coverage of confidence intervals (i.e., whether a confidence interval (CI) includes the true value) in an agricultural field trial. A total of n=7 CIs are calculated. (a) The number (X) of CIs out of 7 that cover/include the true value will follow a binomial distribution B(7,0.95). This is because with the assumed independence of the CIs the situation corresponds to a binomial setting. We can therefore find P(X=7) in Table 1 of S, or calculate P(X=7) = 0.95^7 = 0.698. It is seen that the probability is substantially below the 0.95 for each CI. The 0.698 (or 69.8%) is the simultaneous confidence level. (b) The probability that at least 6 out of 7 intervals cover the true value can similarly be calculated from B(7,0.95), using here the table: P(X>=6) = P(X=6)+P(X=7) = 0.257+0.698 = 0.955. We could also have calculated, P(X=6) = 7*0.95^6*(1-0.95) = 0.257. Added note (not part of question): ---------------------------------- The fact that the probability for at least 6 of the 7 intervals to cover the true value is so close to 0.95 is a coincidence and does not reflect any general rule. We need to use adjustments for multiple testing/comparisons to achieve a simultaneous coverage of 0.95. Due to the independence of the intervals, in this case we simply need to solve 0.95 = p_adj^7, or p_adj=0.95^(1/7) = 0.9927, meaning that if each of the confidence intervals have coverage 99.27%, then the simultaneous coverage is 0.95. For independent intervals/tests, this is (slightly) better than using the Bonferroni method, which will tell us to use an error level of 0.05/7=0.0071 in each interval, and hence a coverage of 1-0.0071=0.9929 or 99.29% in each interval.