Statistics, the art and science of learning from data by A. Agresti (fourth edition) – Chapter 14 summary

ONE-WAY ANOVE: COMPARING SEVERAL MEANS
The inferential method for comparing means of several groups is called analysis of variance, also called ANOVA. Categorical explanatory variables in multiple regression and in ANOVA are referred to as factors, also known as independent variables. An ANOVA with only one independent variable is called a one-way ANOVA.

Evidence against the null hypothesis in an ANOVA test is stronger when the variability within each sample is smaller or when the variability between groups is larger. The formula for the F (ANOVA) test is:

When the null hypothesis is true, the mean of the F-distribution is approximately 1. If the null hypothesis is wrong, then F>1. This also increases if the sample size increases. The larger the F-statistic, the smaller the P-value. The F-distribution has two degrees of freedom values:

 and

The ANOVA test has five steps:

1. Assumptions
   A quantitative response variable for more than two groups. Independent random samples. Normal population distribution with equal standard deviation.
2. Hypotheses
   
   
3. Test statistic
   y
4. P-value
   This is the right-tail probability of the observed F-value.
5. Conclusion
   The null hypothesis is normally rejected if the P-value is smaller than 0.05.

If the sample sizes are equal, the within-groups estimate of the variance is the mean of the g sample variances for the g groups. It uses the following formula:

If the sample sizes are equal, the between-groups estimate of the variance uses the following formula:

The ANOVA F-test is robust to violations if the sample size is large enough. If the population sample sizes are not equal, the F test works quite well as long as the largest group standard deviation is no more than about twice the smallest group standard deviation. Disadvantages of the F-test are that it tells us whether groups are different, but it does not tell us which groups are different.

ESTIMATING DIFFERENCES IN GROUPS FOR A SINGLE FACTOR
The F-test only tells us if groups are different, not how different and which groups are different. Confident intervals can. A confidence interval for comparing means uses the following formula:

The degrees of freedom for the confidence interval is:

If the confidence interval does not contain 0, we can infer that the population means are different. Methods that control the probability that all confidence intervals will contain the true differences in means are called multiple comparison methods. Multiple comparison methods compare pairs of means with a confidence level that applies simultaneously to the entire set of comparisons rather than to each separate comparison. This can be done by using the Tukey method.