In the design of experiments , two or more "treatments" are applied to estimate the difference between the mean responses for the treatments. For example, an experiment on baking bread could estimate the difference in the responses associated with quantitative variables, such as the ratio of water to flour, and with qualitative variables, such as strains of yeast. Experimentation is the step in the scientific method that helps people decide between two or more competing explanations – or hypotheses . These hypotheses suggest reasons to explain a phenomenon, or predict the results of an action. An example might be the hypothesis that "if I release this ball, it will fall to the floor": this suggestion can then be tested by carrying out the experiment of letting go of the ball, and observing the results. Formally, a hypothesis is compared against its opposite or null hypothesis ("if I release this ball, it will not fall to the floor"). The null hypothesis is that there is no explanation or predictive power of the phenomenon through the reasoning that is being investigated. Once hypotheses are defined, an experiment can be carried out and the results analysed to confirm, refute, or define the accuracy of the hypotheses.
There are a number of important considerations when using the Chi-Square statistic to evaluate a crosstabulation. Because of how the Chi-Square value is calculated, it is extremely sensitive to sample size – when the sample size is too large (~500), almost any small difference will appear statistically significant. It is also sensitive to the distribution within the cells, and SPSS gives a warning message if cells have fewer than 5 cases. This can be addressed by always using categorical variables with a limited number of categories (., by combining categories if necessary to produce a smaller table).
In order to avoid a lot of spurious positives, the alpha value therefore needs to be lowered to account for the number of comparisons being performed. This is a correction for multiple comparisons. There are many different ways to do this. The simplest, and the most conservative, is the Bonferroni correction . In practice, more people are more willing to accept false positives (false rejection of null hypothesis ) than false negatives (false acceptance of null hypothesis ), so less conservative comparisons are usually used.