Kahn (2011). Multilevel modelling: Overview and applications to research in counselling psychology.” - Article summary

Multilevel modelling (MLM) is becoming the standard for analysing nested data.

The unit of analysis problem refers to when the moderator variable exists at a different level than the independent and dependent variable (e.g. moderator variable is on the level of the course itself and the independent and dependent variables are on the individual level). Multilevel modelling (MLM) or hierarchical linear modelling is designed to analyse data that exists at different levels.

When individuals exist in natural groups such as schools, there is a hierarchical or nested structure. Ignoring the nested structure of the data could have adverse consequences. Person-analyses with nested data ignore the fact that individuals sharing a common environment and are more similar to each other than they are to individuals from another environment. Nested data may thus violate the assumption of independence of observations. This leads to an increase of type-I errors due to the small standard errors.

In a typical regression analysis, the slope and the intercept are fixed as the same parameters apply to each case in the sample. However, these parameters may vary as a function of group membership in the case of nested data, thus needing the need for a different analysis.

The formula for the level one, multilevel analysis model is the following:

Yij=β0j+β1jXij-Xj+rij

The letter i refers to the level 1 unit (e.g. individuals). The letter j refers to the level 2 unit (e.g. group). Yij is the level 1 dependent variable (e.g. individual score in a given group). Xij is the score on the level 1 independent variable. This formula makes use of a form of centring, as the score on the level 1 independent variable is subtracted from the mean of the group. β0j is the intercept for a given group (e.g. predicted level of a score in a particular group). This can be interpreted as the mean level of a dependent variable for all people in a particular group. β1j is the slope for a given group. It is the predicted increase in the dependent variable per 1-unit increase in the independent variable within a given group. rij is the residual at the individual level.

The level two model addresses what the average intercept and slope across groups is (i.e. fixed effects) (1), how much intercepts and slopes vary across groups (i.e. random effects) (2) and how useful group-level variables for predicting group intercepts and slopes are (3). The level two model uses the following formula:

The level two model describes the difference between group level and not between individuals within the groups. The first formula represents the level two model for predicting the group intercepts (i.e. group means). The second formula represents the level two model for predicting the group slopes. Wj represents a level two independent variable (i.e. characteristic of the level two unit that might be predictive of between-groups variability in the intercept).

The level two independent variables are often grand-mean centred such that 0 reflects the mean of all groups. In this case, a score of 0 on a predictor would mean that Y00 can be interpreted as the mean level of the variable, adjusted for individuals in that group, for that group. Y10 is the level two slope and can be interpreted as the change in that group’s mean for the dependent variable per 1-unit increase in the independent variable (i.e. ‘W’). u0j reflects the unique effect of group j on the grand mean of Y.

In the second formula, Y10 represents the predicted slope between the independent and the dependent variable for groups with a mean of 0. Y11 refers to the change in the slope per 1-unit increase in the independent variable. It expresses the relationship between a group-level variable and the slope between two individual-level variables. It thus represents a cross-level interaction. It is a test of moderation.

It is necessary to conduct MLM with nested data, rather than multiple analyses focused on a single level because this would increase the error in the data, as there are multiple sources of error in the data when there are more levels. Therefore, it is important to model the error in MLM models.

Multilevel models are often estimated using maximum likelihood procedures which is an iterative process. Different groups are given different weight to determine how much the data from the group itself should contribute to the parameter versus data from the entire sample. This weight is based on the reliability of the group estimate. This reliability will be the highest when the sample size is large (1) and the variability among groups is high (2).

There are four often used MLM models. The one-way ANOVA with random effects tests whether the groups differ on a level one variable. In this design, groups are treated as random and not fixed. This tests whether there is sufficient variability in group intercepts to predict with a level two independent variable. The intraclass correlation refers to the percentage of the total variance that is between group. This reflects the degree to which scores on the dependent variable are due to the nested factor.

The means-as-outcomes model (i.e. intercepts-as-outcomes model) assesses whether group-level variables can be used to predict group means. Compared to the one-way ANOVA with random effects, it adds a group-level predictor which is multiplied by the slope. A significant slope indicates that the prediction of the dependent variable is improved by including the group-level independent variable.

Random coefficients regression assesses whether one or more level one variables predict another level one outcome. It is a simple regression analysis with the possibility of randomly varying intercepts and slopes across level two modelled.  It can demonstrate whether slopes and intercepts vary significantly across groups.

The intercepts- and slopes-as-outcomes model assesses whether the varying slopes between level one independent and dependent variables can be predicted by one or more level two independent variables. In this model, the varying intercepts and slopes are predicted whereas in the means-as-outcomes model only the varying intercepts were predicted.

There are three potential applications of MLM to psychotherapy:

1. Researchers can disentangle therapist and client factors on outcome (i.e. determine how much variance is due to the therapist and how much variance is due to the client).
2. Researchers can examine processes and outcomes of group psychotherapy.
3. Process and process-outcome research benefit from MLM. This means that clients with missing data do not need to be discarded from the data set and the relationship between client-level variables and group/therapist level variables with regards to the variability and change in a process or outcome can be addressed.

It may be possible to have a better test of career-related hypotheses (e.g. theories about person-job fit) using MLM. This could promote career development techniques. Furthermore, it may be effective in training graduates as graduates are nested within programme.

It is also possible that MLM could provide better tools to conduct culture research as individuals may be nested within culture. Research on individual differences that are rather stable could benefit from daily repeated measures and this allows them to be subjected to MLM analysis.