The overall fit of a linear model is tested using the F-statistic. The F-statistic is used to test whether groups are significantly different and then specific model parameters (the bs) are used to show which groups are different.

The F-statistic gives an associated p-value as well. A p-value which is smaller than 0.05 (or any set alpha) stands for a significant difference between the group means. The downside of the F-test is that it does not tell us which groups are different. Associated t-tests can show which groups are significantly different.

The null hypothesis if the F-statistic is that the group means are equal and the alternative hypothesis is that the group means are not equal. If the null hypothesis is true, then the b-coefficients should be zero. The F-statistic can also be described as the ratio of explained to the unexplained variation.

The total sum of squares is the total amount of variation within the data. This can be calculated by using the following formula:

It is the difference between each observed data point and the grand mean squared. The grand variance is the total sum of squares of all observations. It is the variation between all scores, regardless of the group from which the scores come.

The model sum of squares is calculated by taking the difference between the values predicted by the model and the grand mean. It tells us how much of the variation can be explained using the model. It uses the following formula:

It is the difference of the group mean and the grand mean squared. This value is multiplied with the number of participants in this group and these values for each group are added together.

The residual sum of squares tells us how much of the variation cannot be explained by the model. It is calculated by looking at the difference between the score obtained by a person and the mean of the group to which the person belongs. It uses the following formula:

It is the squared difference between the participant’s score (xig) and the group mean and this is done for all the participants in all the groups. The residual sum of squares can also be denoted in the following way:

One other way of denoting the residual sum of squares is the following formula:

It is the variance of a group multiplied by one less than the number of people in that group and this value is added together for all the groups. The average sum of squares (mean squares) is calculated by dividing the model sum of squares with the degrees of freedom (N-k).

ASSUMPTIONS WHEN COMPARING MEANS
There are several assumptions when comparing means:

1. Homogeneity of variance
   The assumption is that the variance of the outcome is steady as the predictor changes. It means that the variance of all the groups are equal. This assumption could be violated if the sample sizes are not equal.
2. Normality
   The assumption is that the data is normally distributed.
3. Independence
   The assumption is that the cases are independent.

If the assumption of equal variance is not met, then the Welch’s F or the Brown-Forsythe F could be used which adjusts F and the residual degrees of freedom to combat problems from violating this assumption.

PLANNED CONTRASTS (CONTRAST CODING)
When doing t-tests to see which groups are significantly different, the type-I error rate is inflated. Besides that, using dummy codes (if someone is in the group ‘short’ they have a 1 on short and a 0 in the group for ‘long’ and people in the group ‘long’ have it the other way around) does not always lead to comparison of all the groups you want to compare.

Contrast coding is a way of assigning weights to groups in dummy variables to carry out planned contrasts (planned comparisons). It controls for the type-I error rate. Planned contrasts are used to test specific hypotheses and post-hoc tests are used when there are no specific hypotheses.

There are three rules of thumb for designing planned contrasts:

1. The control group is what you want to compare any other group to
2. Each contrast must compare only to ‘chunks’ of variation
3. Once a group has been singled out in a contrast it can’t be used in another contrast

There are k-1 contrasts possible. There are a few rules for assigning values to the dummy variables to obtain the contrast you want:

1. Choose sensible contrasts
2. Groups coded with positive weights will be compared against groups with negative weights
3. If you add up the weights of a contrast, the result should be 0
4. If a group is not involved, assign it a weight of zero
5. For a given contrast, the weight assigned to the group in one chunk of variation should be equal to the number of groups in the opposite chunk of variation

If the products of the weight also adds up to zero, then the contrasts are independent or orthogonal. The polynomial contrast tests for trends in the data. It can test for linear trends, quadratic trends (curve in the line), cubic trends (two changes in the direction of the trend) and quartic trends (three changes of direction). In SPSS, there are several standard contrasts available:

1. Deviation (first)
   This compares the effect of each category (except the first) to the overall experimental effect.
2. Deviation (last)
   This compares the effect of each category (except the last) to the overall experimental effect
3. Simple (first)
   Each category is compared to the first category.
4. Simple (last)
   Each category is compared to the last category
5. Repeated
   Each category (except the first) is compared to the previous category
6. Helmert
   Each category (except the last) is compared to the mean effect of all subsequent categories
7. Difference (reverse Helmert)
   Each category (except the first) is compared to the mean effect of all previous categories.

POST HOC PROCEDURES
Post hoc tests consist of pairwise comparisons that are designed to compare all different combinations of the treatment groups. It is taking every pair of groups and performing a separate test on each. Post-hoc tests control for the inflated type-I error rate by correcting the level of significance for each test such that the overall type-I error rate across all comparisons remains at 0.05.

The Bonferroni method has more power when the number of comparisons is small, whereas the Tukey method has more power when the number of comparisons is larger. The Tukey method is also useful when the sample sizes are equal. The Hochberg’s GT2 should be used when the sample sizes are very different.

CALCULATING THE EFFECT SIZE
The effect size can be calculated in the following way:

The omega squared is used to estimate the effect size in the population. It is not purely based on sums of squares from the sample. It uses the following formula:

The degrees of freedom can be obtained by looking at the output in SPSS. Omega is an unbiased estimate of r.