The McNemar test (NcNemar (1947)1)) is used to verify the hypothesis determining the agreement between the results of the measurements, which were done twice and of an feature (between 2 dependent variables and ). The analysed feature can have only 2 categories (defined here as (+) and (–)). The McNemar test can be calculated on the basis of raw data or on the basis of a contingency table.
The test statistic is defined by:
This statistic asymptotically (for large frequencies) has the Chi-square distribution with a 1 degree of freedom.
The Continuity correction for the McNemar test
This correction is a more conservative test than the McNemar test (a null hypothesis is rejected much more rarely than when using the McNemar test). It guarantees the possibility of taking in all the values of real numbers by the test statistic, according to the distribution assumption. Some sources give the information that the continuity correction should be used always, but some other ones inform, that only if the frequencies in the table are small.
The test statistic with the continuity correction is defined by:
McNemar's exact test
A common general rule for the asymptotic validity of the McNemar chi-square test is the Rufibach assumption, which is that the number of incompatible pairs is greater than 10: 2) when this condition is not satisfied, then we should base the exact probability values of this test 3). The exact probability value of the test is based on a binomial distribution and is a conservative test, so the recommended exact value of the mid-p McNemar test is also given in addition to the exact value of the MnNemar test.
If the study is carried out twice for the same feature and on the same objects – then, odds ratio for the result change (from to and inversely) is calculated for the table.
The odds for the result change from to is , and the odds for the result change from to is .
Odds Ratio () is:
Confidence interval for the odds ratio is calculated on the base of the standard error:
Additionally, for small sample sizes, the exact range of the confidence interval for the Odds Ratio can be determined4).
The settings window with the
Bowker-McNemar test can be opened in
Statistics menu →
NonParametric tests →
Bowker-McNemar or in ''Wizard''.
The Bowker test of internal symmetry (Bowker (1948)5)) is an extension of the McNemar test for 2 variables with more than 2 categories (). It is used to verify the hypothesis determining the symmetry of 2 results of measurements executed twice and of feature (symmetry of 2 dependent variables i ). An analysed feature may have more than 2 categories. The Bowker test of internal symmetry can be calculated on the basis of either raw data or a contingency table.
where , , , so and are the frequencies of the symmetrical pairs in the table
The test statistic is defined by:
This statistic asymptotically (for large sample size) has the Chi-square distribution with a number of degrees of freedom calculated using the formula: .
Two different surveys were carried out. They were supposed to analyse students' opinions about the particular academic professor. Both the surveys enabled students to give a positive opinion, a negative and a neutral one. Both surveys were carried out on the basis of the same sample of 250 students. But the first one was carried out the day before an exam done by the professor, and the other survey the day after the exam. There are some data below – in a form of raw rows, and all the data – in the form of a contingency table. Check, if both surveys give the similar results.
where, for example, changing the opinion from positive to negative one is symmetrical to changing the opinion from negative to positive one.
Comparing the p-value for the Bowker test (p-value<0.0001) with the significance level it may be assumed that students changed their opinions. Looking at the table you can see that, there were more students who changed their opinions to negative ones after the exam, than those who changed it to positive ones after the exam. There were also students who did not evaluate the professor in the positive way after the exam any more.
If you limit your analysis only to the people having clear opinions about the professor (positive or negative ones), you can use the McNemar test:
If you compare the p-value, calculated for the McNemar test (p-value < 0.0001), with the significance level , you draw the conclusion that the students changed their opinions. There were much more students, who changed their opinions to negative ones after the exam, than those who changed their opinions to positive ones. The possibility of changing the opinion from positive (before the exam) to negative (after the exam) is eleven times greater than from negative to positive (the chance to change opinion in the opposite direction is: ).