Fisher's exact test

Fisher's exact test is a quantitative research method that tests the hypothesis that two categorical variables possess different proportions of another categorical variable. If certain assumptions are congruent with the experimental data, this biostatistical procedure allows for an exact calculation of the probability that experimental data would arise. This is performed by adding up the probabilities of the experimental data and other more extreme deviations assuming the frequency distributions between one categorical variable are unrelated and equal.

History

This test was created by Ronald Fisher, who reportedly developed the test after Muriel Bristol claimed to be able to taste whether tea or milk were first added to the cup.¹ Fisher setup an experiment with Bristol, constructing a matrix called a 2 x 2 contingency table where one binary categorical variable is whether tea or milk were the first to be added to the cup. The second binary categorical variable is whether Bristol thought tea or milk were added first. The data within the contingency table contains the number of events that fall in each binary categorial variable combination. Importantly, Bristol knows how many cups have tea or milk added first and will guess so there is a correct number in each category. Following these conditions, Fisher shows that the frequencies will follow a hypergeometric distribution that in brief describes the probability of a number of successes while sampling without replacement from a finite, known size of samples. Since we know all possible combinations, we can determine the probability of arriving at that frequency distribution between the two categorial variables using equations specified for the hypergeometric distribution.

Example

We will borrow an example from John H McDonald² that uses data from van Hood et al. (2013) which studied patients with C. difficile infections.³ One variable was the treatment: vancomycin or fecal transplant. The other variable was outcome: cured or not cured. The null hypothesis for a fisher test between these two variables is that the frequency distribution of outcome should be the same for both types of treatment.

Consider the following 2 x 2 contingency table:

	Fecal	Vancomycin
Sick	3	9
Cured	13	4

There are 16 total fecal transplant patients, 13 total vancomycin patients, 12 total sick patients, and 17 total cured patients. Using equations for hypergeometric distributions, the probability of these exact numbers is 0.00072.

For a one sided fisher’s exact test, you would now calculate the probabilities for all the more extreme ways the data could be distributed for sick patients while maintaining the amount of patients in each treatment category.

	Fecal	Vancomycin
Sick	2	10
Cured	14	3

	Fecal	Vancomycin
Sick	1	11
Cured	15	2

fecal vancomycin Sick 0 12 Cured 16 1

	Fecal	Vancomycin
Sick	0	12
Cured	16	1

The probabilities for these three tables are 0.000661, 0.0000240, and 0.000000251, respectively. The result of the one sided fisher’s exact test for the data is then calculated by summing all the probabilities listed, arriving at P = 0.00840. However, in most cases we use the two sided which also considers extreme data deviations int he opposite way that have a probability less than our data. This would include:

	Fecal	Vancomycin
Sick	11	1
Cured	5	12

	Fecal	Vancomycin
Sick	12	0
Cured	4	13

The probabilities for these tables are 0.000035 and 0.00109, respectively, and adding these two the one sided probability yields a two sided probability of 0.00953.

Assumptions

We assume that the individual events are independent of one another.

An important condition is that the total amounts of each categorial variable are known before the experiment is conducted. For example, with Fisher and Bristol’s tea experiment Bristol knew how many cups had tea or milk added first, and would have guessed a combination that fit these totals. We just don’t know what number would have been correct. These known totals, referred to as the data being “conditioned”, are an assumption that rarely fits biological data. In our C Diff treatment example, we know the amount of patients treated with fecal transplants and vancomycin, but we wouldn’t know what total amount eventually is still sick or cured. In these cases, the fisher’s exact test is not exact but a conservative estimation - it makes it harder to reject the null hypothesis and easier to miss a real difference between frequency distributions.

Purpose

Fisher’s exact test is useful because given that the data meets the required assumptions, you can arrive at an exact P value for any sample size. Even if the data is not conditioned, the conservative nature of the test means it can still be used and if anything will underreport true differences. Comparing two binary categorial variables for association and having small sample sizes are both common scenarios in healthcare research, making Fisher's exact test a useful tool.

Advantages:

• Calculate an exact P value, not an estimate
• Good for small sample sizes (n < 100) since other tests (e.g. chi-squared test, g-test) approximate a distribution that might not be accurate for small samples
• Widely used and accepted

Disadvantages

• Only appropriate for 2x2 contingency tables and not higher dimensions
• Can be computationally expensive to compute combinations for large sample sizes
• Can be too conservative when data is not conditioned

References

1. Fisher, Sir Ronald A. (1956) [The Design of Experiments (1935)]. "Mathematics of a Lady Tasting Tea". In James Roy Newman (ed.). The World of Mathematics, volume 3. Courier Dover Publications. ISBN 978-0-486-41151-4.

1. van Nood, E., Vrieze, A., Nieuwdorp, M., et al. (13 co-authors). 2013. Duodenal infusion of donor feces for recurrent Clostridium difficile. New England Journal of Medicine 368: 407-415.

1. McDonald, J. H. (2022, April 24). 2.7: Fisher's exact test. Statistics LibreTexts. Retrieved May 2, 2023, from https://stats.libretexts.org/Bookshelves/Applied_Statistics/Book%3A_Biological_Statistics_(McDonald)/02%3A_Tests_for_Nominal_Variables/2.07%3A_Fisher's_Exact_Test#:~:text=I%20recommend%20using%20Fisher's%20exact,sample%20size%20less%20than%201000.

Submitted by Sidharth Sengupta