Multivariate Data Analysis

Week 1: Multiple Regression Analysis

Multivariate means exploring the dynamics between 3 or more variables.

Multiple regression analysis (MRA) can be done when all variables are interval level (e.g. weight, height, IQ score).

The research question for MRA is: can Y be predicted from X1 and/or X2?

What are the linearity, homoscedasticity and normality of residuals?

Linearity: the relationship between the independent and dependent is linear. Can be tested with scatterplots.

Homoscedasticity: variance of residuals is constant across values predictors. Can be seen on the scatterplot.

Normality: approximate straight line on P-P plot.

What does multicollinearity mean?

That there is a high intercorrelation between predictors. There is multicollinearity when the tolerance is higher than 0.10 and the VIF is lower than 10.

Are there outliers, influential points, or outliers on the predictors?

Outliers: on dependent variable Y: Residuals, between −3 and 3.

Influential points: Cook’s distance smaller than 1.

Outliers on the predictors: on independent variable(s) X: Leverage, smaller than 3(k+1)/n

What are the null and the alternative hypothesis to test the regression model?

H_o: b^*₁ =b^∗₂ =···=b^∗_k =0 (No relation between Y and X1, X2)

Ha: :at least one b^∗_j =/= 0

When can the null hypothesis be rejected?

When the hypothesis of no relation between variables can be rejected (no relation would mean all variables equal 0)

What are the null and the alternative hypothesis to test the individual coefficients?

H₀: b₁=0 and H_a = b₁=/=0

H₀: b₂=0 and H_{a =} b₂=/=0

What are the unstandardized and standardized regression equations?

Unstandardized (MRA): b₀+b₁X₁+b₂X₂  

Standardized (MRA_st): β₁X_1st+ β₂X_2st

How much variance of Y is explained in total by X1 and X2?

The R² gives the value for the explained data (X₁ and X₂).

How much variance of Y is uniquely explained by X1? How much variance of Y is uniquely explained by X2? What is the best predictor?

We look at part coefficient. r²x(1.2) and r²y(2.1) gives the answer. Whichever the highest value is, that’s the best predictor.

Week 3: Analysis of covariance (ANCOVA)

ANCOVA can be used when one of the independent variables (factor X) is nominal level,

while the other one (covariate) is interval level.

The goal of ANCOVA is the reduction of error variance and removal of systematic bias.

The research question is: What is the effect of X on Y after correction for C?

Is there a significant effect of x method (e.g. teaching method) on posttest? (report test statistic, df, p value and a suitable measure of effect size). If yes, interpret that effect.

Here we look at the table tests of Between-Subjects Effects. We need the eta squared value, the F statistic and the p value.

The eta squared can be calculated by dividing the Sum of Square (SS) of the method by the SS of the total.

The F is the Mean Square (MS) of method, divided by the MS of error. This value can also be seen in the table. The two degrees of freedom used are the one for method (in the example given below, 2) and for error (in the example: 177). The p value is seen in the table under significance.

• An example of how the test statistic should be reported: F(2, 177)=13.371, p<0.001.

If the answer is: yes there is a significant effect of method on posttest, then we look at the multiple comparisons table. We look at the mean difference for all methods and see if there is any that stand out or are significantly different. For example, if the mean difference of method B with all other methods is a lot larger than mean differences without method B, then we can say B is significantly different.

Is there a significant correlation between pretest and post-test in all groups?

Here we look at the within-group correlations table and see if all are significant (p < .001)

Are there significant differences between the groups at pretest? If yes, interpret these differences.

Here we look at the Tests of Between-Subjects Effects table, with the dependent variable: pretest. If the F test is significant, there are differences between the groups. Then, in the multiple comparisons table look at mean differences.

Example of reporting what we found:

Significant group differences at pretest, F(2,177) = 10.221,p<.001.

No difference between methods A and B (Mdif f= -2.67,p= .339), but Method C differs from both method A (Mdif f= -5.72,p= .008) and B (Mdif f= -8.38,p<.001).

Do you think that adding the covariate might lead to reduction of error, reduction of bias, neither, or both?

If there are large and significant within-group correlations→Possible reduction of error

If there are significant differences between group means→Possible elimination of bias

Week 5: MANOVA and Descriptive DA

The goal of MANOVA and of Descriptive DA is the optimal prediction of differences between group means on several interval variables.

Why? Often very natural to compare groups on more than one variable, for example:

• quality and quantity of task performance;

• different stress reaction (e.g. emotions, physiological measures).

Compared to ANOVA:

ANOVA: one dependent variable -->(univariate)

MANOVA: two or more dependent variables -->(multivariate)

Check the assumption of equality of the covariance matrices and discuss robustness ofthe MANOVA against violations of the assumption of multivariate normality.

To check for the equality of covariance matrices, in the table: Test of Equality of Covariance Matrices, look at Box’s M. If higher than 0.05 then not significant.

To check for robustness of non-normality: number of participants per group has to be ≥20:

The design is balanced when robust to unequal covariance matrices.

Consider the mean values in the descriptive statistics. Which groups differ a lot on characteristic x (e.g. physical complaints)? Which groups differ only a little?

Look at descriptive statistics table and the appropriate box, in this case: physical complaints. Compare the values under Mean.

Given your crude assessment in the previous question, is it plausible to expect a multivariate effect?

If there are differences under sample group means, then yes, a multivariate effect is expected.

Example: Is there a significant effect of occupation on physical complaints, experience of hostility, and/or dissatisfaction?

We look at the multivariate tests table, in the Occupation box. Then, we report F statistic, df, and p value. If the tests are higher than p≤.05, there is a significant effect.

For which variables is the univariate effect significant?

We look at the Tests of Between-Subjects Effects table. Then, you report the tests were the p<0.05.

What is the answer to the previous question be if we apply a Bonferroni correction for multiple testing?

For Bonferroni, we divide the Alpha by number of categories (dependent variables). For example, if there are three categories (e.g.: hostility, physical complaint, dissatisfaction). The alpha then becomes smaller and we have to check if the p values are still smaller. Often, they aren’t or only few remain significant. Example:

Before Bonferroni:

Physical complaints F(2,177) = 3.196,p=.043 -->significant

Hostility F(2,177) = 3.405,p=.035 ---> significant

Dissatisfaction F(2,177) = 4.511,p=.013 --> significant

After Bonferroni, Withα=.05/3 =.0167:

Only Dissatisfaction F(2,177) = 4.511,p=.013 -->significant.

Interpret the significant effect(s) (after Bonferroni correction) using the table with multiple comparisons. Why is the Tukey HSD correction applied?

The Tukey test is invoked when you need to determine if the interaction among three or more variables is mutually