Reading Questions.

- What is a “null hypothesis?” Why does it play a special role in hypothesis testing?
- Why is it good to have a hypothesis test with a low “significance level?”
- Why is it good to have a hypothesis test with a high “power?”
- What is a p-value?
- Why are there two kinds of potential errors in hypothesis testing, Type I and Type II?

Prob 13.01. Which of the following are legitimate possible outcomes of a hypothesis test? Mark as “true” any legitimate ones.

- (a)
- True or Falseaccept the alternative hypothesis
- (b)
- True or Falseaccept the null hypothesis
- (c)
- True or Falsereject the alternative hypothesis
- (d)
- True or Falsefail to reject the alternative hypothesis
- (e)
- True or Falsereject the null hypothesis
- (f)
- True or Falsefail to reject the null hypothesis
- (g)
- True or Falseindeterminate result

Pick one of the illegitimate outcomes and explain why it is illegitimate.

Prob 13.02. Consider the two conditional sampling distributions shown in the figure.

Imagine that you set rejection criteria so that the power was 99%. What would be the significance of the test? (Choose the best answer.)

A
| About 0.05. |

B
| About 0.30 |

C
| About 0.95 |

D
| Can’t tell (even approximately) from the information given. |

We sometimes speak of the probability of Type I or Type II errors. We want to know what sort of probability these are. To simplify notation, we’ll define the following outcomes:

- N
- The Null Hypothesis is correct.
- A
- The Alternative Hypothesis is correct.
- Fail
- Fail to reject the Null Hypothesis
- Reject
- Reject the Null Hypothesis.

What is the probability of a Type I error?

A
| A joint probability: p(Reject and N) |

B
| A joint probability: p(Fail and A) |

C
| A conditional probability: p(Reject given N) |

D
| A conditional probability: p(Reject given A) |

E
| A marginal probability: p(Reject) |

F
| A marginal probability: p(Fail) |

What is the probability of a Type II error?

A
| A joint probability: p(Reject and A) |

B
| A joint probability: p(Fail and N) |

C
| A conditional probability: p(Fail given N) |

D
| A conditional probability: p(Fail given A) |

E
| A marginal probability: p(N) |

F
| A marginal probability: p(A) |

Prob 13.14. Consider these two sampling distributions:

Suppose that you insisted that every hypothesis test have a significance of 0.05. For the conditional sampling distributions shown above, what would be the power of the most powerful possible one-tailed test?

A
| About 5% |

B
| About 50% |

C
| About 95% |

D
| Can’t tell from the information given. |

What would be the power of the most powerful possible two-tailed test?

A
| About 10% bigger than the one-tailed test. |

B
| About 10% smaller than the one-tailed test. |

C
| About 50% bigger than the one-tailed test. |

D
| About 50% smaller than the one-tailed test. |

E
| Can’t tell from the information given. |

Prob 13.15. The Ivory-billed woodpecker has been thought to be extinct; the last known observation of one was in 1944. A new sighting in the Pearl River forest in Arkansas in 2004 became national news and excited efforts to confirm the sighting. These have not been successful and there is skepticism whether the reported 2004 sighting was correct. This is not an easy matter since the 2004 sighting was fleeting and the Ivory-billed woodpecker is very similar to a relatively common bird, the Pileated woodpecker.

Pileated (left) and Ivory-billed Woodpeckers
| |

Drawings by Ernest S Thompson & Rex Brasher
| |

This problem is motivated by the controversy over the Ivory-billed sighting, but the problem is a gross simplification.

Ivory-billed woodpeckers are 19 to 21 inches long, while Pileated woodpeckers are 16 to 19 inches. Ivory-bills are generally glossy blue-black, whereas Pileated woodpeckers are generally dull black, slaty or brownish-black. Ivory-bills have white wing tops while Pileated woodpeckers have white underwings. These differences would make it easy to distinguish between the two types of birds, but sightings are often at a distance in difficult lighting conditions. It can be difficult, particularly in the woods, to know whether a distant bird is flying toward the observer or away.

Imagine a study where researchers display bird models in realistic observing conditions and record what the observers saw. The result of this study can usefully be stated as conditional probabilities:

Alternative | Null | ||

Observed | Code | Ivory-billed | Pileated |

Short & Dull | A | 0.01 | 0.60 |

Long & Dull | B | 0.10 | 0.13 |

Short & Glossy | C | 0.04 | 0.20 |

Long & Glossy | D | 0.60 | 0.05 |

Short & White Back | E | 0.05 | 0.01 |

Long & White Back | F | 0.20 | 0.01 |

For simplicity, each of the six possible observations has been given a code: A, B, C, and so on.

The Bayesian Approach The table above gives conditional probabilities of this form: given that the bird is an Ivory-billed woodpecker, the probability of observation D is 0.60.

In the Bayesian approach, you use the information in the table to compute a different form of conditional probability, in this form: given that you observed D, what is the probability of the bird being an Ivory-billed.

In order to switch the form of the conditional probability from “given that the bird is ...” to “given that the observation is ...”, you need some additional information, called the prior probability. The prior probability reflects your view of how likely a random bird is to be an Ivory-billed or Pileated before you make any observation. Then, working through the Bayesian calculations, you will find the posterior probability, that is, the probability after you make your observation.

Suppose that, based on your prior knowledge of the history and biology of the Ivory-bill, that your prior probability that the sighting was really an Ivory-bill is 0.01, and the probability that the sighting was a Pileated is 0.99. With this information, you can calculate the joint probability of any of the 12 outcomes in the table.

Then, by considering each row of the table, you can calculate the marginal probability of Ivory-bill vs Pileated for each of the possible observations.

- (a)
- What is the joint probability of a sighting being both D and Ivory-billed?
(Pick the closest one.)
impossible 0.006 0.01 0.05 0.60
- (b)
- What is the joint probability of a sighting being both D and Pileated? (Pick the
closest one.)
impossible 0.006 0.01 0.05 0.60
- (c)
- What is the conditional probability of the sighting being an Ivory-billed GIVEN
that it was D? (Pick the closest one.)
0.01 0.05 0.10 0.60
- (d)
- Which of the possible observations would provide the largest posterior probability
that the observation was of an Ivory-bill?
A B C D E F
- (e)
- What is that largest posterior probability? (Pick the closest one.)
0.02 0.08 0.17 0.73

The Hypothesis-Testing Approach The hypothesis-testing approach is fundamentally different from the Bayesian approach. In hypothesis testing, you pick a null hypothesis that you are interested in disproving. Since the Pileated woodpecker is relatively common, it seems sensible to say that the null hypothesis is that the observation is a Pileated woodpecker, and treat the Ivory-billed as the alternative hypothesis.

Once you have the null hypothesis, you choose rejection criteria based on observations that are unlikely given the null hypothesis. You can read these off directly from the conditional probability table given above.

- 1.
- The two least likely observations are E and F. If observing E or F is the
criterion for rejecting the null hypothesis, what is the significance of the
test?
0.01 0.02 0.05 0.25 0.60 0.65
What would be the power of this test?

0.01 0.05 0.25 0.60 0.65 - 2.
- Now suppose the rejection criteria are broadened to include D, E, and
F.
What would be the significance of such a test?

0.01 0.02 0.05 0.07 0.10 0.20 0.25 0.60 0.85 otherWhat would be the power of such a test?

0.01 0.02 0.05 0.07 0.10 0.20 0.25 0.60 0.85 other

Comparing the Bayesian and hypothesis-testing approaches, explain why you might reject the null hypothesis even if the observation was very likely to be a Pileated woodpecker.