LIS 504 - Statistical tests

Chi-square test

The chi-square2) test is a test of whether the differences observed between actual counts in a sample and those expected given certain assumptions are significant, or whether they could be due to chance. Using the chi-square test is made a little more complex by the need to calculate expected as well as observed values.

For example, in a series of experiments, people were asked to write abstracts using computer assistance. Before they wrote the abstracts, they were asked how how much experience they had with the computer operating environment. Afterwards, the abstracts were examined to see if they contained any spelling errors. The results can be summarized in the following table:
Experience with operating environment No misspellings One or more misspellings Total
Little or none 6 17 23
Some or a lot 18 17 35
Total 24 35 58
It looks as though people with more experience are more likely to avoid spelling errors. But is the difference statistically significant?

To use the chi-square test, we need expected counts, too. In this case, these are calculated on the assumption that experience and spelling are not correlated at all. The formula to use for each expected value is
(Row Total) * (Column Total)
(Grand Total)
For example, the expected value for the top left cell is
(23) * (24) = 9.517
Applying the formula gives the following table, which keeps the same totals:
Experience with operating environment No misspellings One or more misspellings Total
Little or none 9.517 13.483 23
Some or a lot 14.483 20.517 35
Total 24 35 58

Applying the CHITEST function in Excel gives a result of about 0.055, which is just short of the critical value of 0.050 for minimal statistical significance. Even though a relationship between experience and spelling is plausible, therefore, for these data, based on the chi-square test, we should accept the null hypothesis that the observed differences could have arisen by chance.

Like most statistical tests, the chi-square test provides only an approximate value, especially for small numbers of observations.

T test

Sometimes called Student's t, the t test is a test of whether an observed difference in the means of two samples is significant, or whether the two samples could have come from the same population by chance.

For example, in a series of experiments, people were asked to write abstracts of an article. The distributions of the lengths of the abstracts produced by males and females respectively looked like this:

It seems that the males might be tending on average to produce slightly shorter abstracts than the females. The mean length of males' abstracts is about 1686.8 bytes, while that for females' abstracts is about 1843.6. But could this just be an accident, a result of sampling error?

Excel was used to run a two-tailed t test (the usual kind) on the original lists of abstract lengths for males and females respectively. The result was about 0.696*, indicating no statistical significance, and supporting the null hypothesis that the difference is due to sampling error.

*For unequal variance, the safer option, which also makes sense here since the variances are quite different, at 3,568,221 and 904,922 respectively.
The t test actually assumes that the data are normally distributed, which is very often not really the case. For example, the distributions of abstract lengths illustrated above have rather longer tails to the right than to the left.

Other tests

For a table that allows you to test the statistical significance of a correlation coefficient, see Palys, p. 405.

Last updated July 4, 2001.
This page maintained by Prof. Tim Craven
E-mail (text/plain only):
Faculty of Information and Media Studies
University of Western Ontario,
London, Ontario
Canada, N6A 5B7