Discovering statistics using IBM SPSS statistics by A. Field (5th edition) a summary
- 11137 reads
Statistics
Chapter 18
Exploratory factor analysis
In factor analysis, we take a lot of information (variables) and a computer effortlessly reduces this into a simple message (fewer variables).
Latent variable: something that cannot be accessed directly.
Measuring what the observable measures driven by the same underlying variable are.
Factor analysis and principal component analysis (PCA) are techniques for identifying clusters of variables.
Three main uses:
If we measure several variables, or ask someone several questions about themselves, the correlation between each pair of variables can be arranged in a table.
Factor analysis attempts to achieve parsimony by explaining the maximum amount of common variance in a correlation matrix using the smallest number of explanatory constructs.
Explanatory constructs are known as latent variables (or factors) and they represent clusters of variables that correlate highly with each other.
PCA differs in that it tries to explain the maximum amount of total variance in a correlation matrix by transforming the original variables into linear components.
Factor analysis and PCA both aim to reduce the R matrix into a smaller set of dimensions.
Graphical representation
Factors and components can be visualized as the axis of a graph along which we plot variables.
The coordinates of variables along each axis represent the strength of relationship between that variable and each factor.
In an ideal world a variable will have a large coordinate for one of the axes and small coordinates for any others.
Factor loading: the coordinate of a variable along a classification axis.
If we square the factor loading for a variable we get a measure of its substantive importance to a factor.
Mathematical representation
A component ins PCA can be described as:
Componenti = b1Variable1i + b2Variable2i + … + bnVariableni
There is no intercept in the equation because the lines intersects at zero.
There is no error because we are simply transforming the variables.
The bs in this equation represent the loadings.
Ideally, variables would have very high b-values for one component and very low b-values for all other components.
The factors in factor analysis are not represented the same way as components.
A factor is defined as:
Variables = VariableMean + (Loadings X Common Factor) + Unique Factor
Common factors: factors that explain the correlations between variables
Unique factors: factors that cannot explain the correlations between variables
In PCA we predict components from the measured variables
In factor analysis we predict the measured variables form the underlying factors.
Both factor analysis and PCA are linear models in which loadings are used as weights.
In both cases, these loadings can be expressed as a matrix in which the columns represent each factor and the rows represent the loadings of each variable on each factor.
Factor scores
Having discovered which factor exists, and estimated the equation that describes them, it should be possible to estimate a person’s score on a factor, based on their scores for the constituent variables.
These are known as factor scores (or components in PCA).
The scales of measurement will influence the resulting scores, and if different variables use different measurement scales, then factor scores for different factors cannot be compared.
There are several techniques for calculating factor scores that use factor score coefficients as weights rather than the factor loadings.
Factor score coefficients can be calculated in several ways
Tot obtain the matrix of factor score coefficients (B) we multiply the matrix of factor loadings by the inverse (R-1) of the original correlation or R-matrix.
Using the regression technique, the resulting factor scores have a mean of 0 and a variance equal to the squared multiple correlation between the estimated factor scores and the true factor values.
Factor scores: a composite score for each individual on a particular factor.
There are several uses of factor scores
Choosing a method
There are two things to consider
Assuming we want to explore, we need to consider whether we want to apply our findings to the sample collected (descriptive method) or to generalize our findings to a population (inferential method).
Certain techniques assume that the sample used is the population and results cannot be extrapolated beyond that sample.
A different approach assumes that participants are randomly selected but that the variables measured constitute the population of variables in which they are interested.
By assuming this, it is possible to generalize from the sample to a larger population, but with the caveat that any findings hold true only for the set of variables measured;
Communality
The total variance of a variable in the R-matrix will have two components:
Thus, a variable that has no unique variance would have a communality of 1.
A variable that shares none of its variance with any other variable would have a communality of 0.
Factor analysis tries to find common underlying dimensions within the data and so is primarily concerned with the common variance.
We want to find out how much of the variance in our data is common.
To solutions
Factor analysis or PCA?
Factor analysis derives a mathematical model from which factors are estimated.
Principal component analysis decomposes the original data into a set of linear variates.
Only factor analysis can estimate the underlying factors and it relies on various assumptions for these estimates to be accurate.
PCA is concerned only with establishing which linear components exists within the data and how a particular variable might contribute to a given component.
Theory behind PCA
Principal component analysis works in a very similar way to MANOVA and discriminant function analysis.
We take the correlation matrix and calculate the variances.
There are no groups of observations, so the number of variates calculated will always equal the number of variables measured (p).
The variates are described by the eigenvectors with the correlation matrix.
The elements of the eigenvectors are the weights of each variable on the variate. The values are the loadings.
The largest eigenvalue associated with each of the eigenvectors provides a single indicator of the substantive importance of each component. The basic idea is that we retain components with relatively large eigenvalues and ignore those with relatively small eigenvalues.
Factor analysis works differently, but there are similarities.
Factor extraction: eigenvalues and the scree plot
In both PCA and factor analysis, not all factors are retained.
Extraction:the process of deciding how many factors to keep.
Eigenvalues associated with a variate indicate the substantive importance of that factor.
Retain only factors with large eigenvalues.
Scree plot: plotting each eigenvalue (Y-axis) against the factor with which it is associated (X-axis).
It is possible to obtain as many factors as there are variables and each has an associated eigenvalue.
By graphing the eigenvalues, the relative importance of each factor becomes apparent.
Typically there will be a few factors with quite high eigenvalues, and many factors with relatively low eigenvalues.
This graph as a very characteristic shape, there is a sharp descent in the curve followed by a tailing off.
An alternative to the scree plot is to use the eigenvalues, because these represent the amount of variation explained by a factor.
You set a criterion value that represents a substantial amount of variation and retain factors with eigenvalues above this criterion.
Two common criteria
The three criteria often provide different answers.
In these situations consider the communalities of the factors.
In both PCA and factor analysis we determine how many factors/components to extract and then re-estimate the communalities. The factors we retain will not explain all the variance data and so the communalities after extraction will always be less than 1.
The factors retained do not map perfectly onto the original variables, they merely reflect the common variance in the data.
The closer the communalities are to 1, the better our factors are at explaining the original data.
The communalities are good indices of whether too few factors have been retained.
Factor rotation
Once factors have been extracted, it is possible to calculate the degree to which variables load onto these factors.
Factor rotation: a technique used to discriminate factors.
If we visualize our factors as an axis along which variables can be plotted, then factor rotation effectively rotates these axes such that variables are loaded maximally to only one factor.
Factor rotation amounts to rotating the axes to try to ensure that both clusters of variables are intersected by the factor to which they relate most.
After rotation, the loadings of the variables are maximized on one factor and minimized on the remaining factor(s).
If an axis passes through a cluster of variables, then these variables will have a loading close to zero on the opposite axis.
There are two flavours of rotation
SPSS implements three methods or orthogonal rotation
SPSS has two methods of oblique rotation
The choice of orthogonal or oblique rotation depends on:
Factor transformation matrix: used to convert the unrotated factor loadings into the rotated ones. Values in this matrix represents the angle through which the axes have been rotated, or the degree to which factors have been rotated.
Interpreting the factor structure
Once a structure factor has been found, it needs to be interpreted.
Loadings are a gauge of the substance of a given variable to a given factor.
We use these values to place variables with factors. Every variable will have a loading on every factor, so we’re looking for variables that load highly on a given factor. Once we’ve identified these variables, we look for a theme within them.
It is possible to assess the statistical significance of a loading, but the p-value depends on sample size.
So, instead we can gauge importance by squaring the loadings to give an estimate of the amount of variance in a factor for by a variable.
When reporting factor analysis, provide readers with enough information to make an informed opinion about what you’ve done.
Be clear about your criteria for extracting factors and the method of rotation used.
Provide a table of the rotated factor loadings of all items and flag values about a criterion level.
Report the percentage of variance that each factor explains and possibly the eigenvalue too.
Measures of reliability
If you’re using factor analysis to validate a questionnaire, it is useful to check the reliability of your scale.
Reliability: that a measure should consistently reflect the construct that it is measuring.
Cronbach’s alpha, α: a measure that is loosely equivalent to creating two sets of items in every way possible and computing the correlation coefficient for each split. The average of these values.
Α = (N2 * mean(cov))/ (Σs2item+Σcovitem)
For each item in our scale we cal calculate two things.
We can construct a variance-covariance matrix of all items.
The top half of the equation is the number of items (N) squared multiplied by the average covariance between items.
The bottom half is the sum of all the item variances and item covariances.
Interpreting Cronbach’s α: some cautionary tales
The value of α depends on the number of items on the scale.
Alpha should not be used as a measure for unidimensionality (the extent to which the scale measures one underlying factor or construct).
Reverse items will affect α.
How to report reliability analysis
Report the reliabilities in the text using the symbol α.
JoHo can really use your help! Check out the various student jobs here that match your studies, improve your competencies, strengthen your CV and contribute to a more tolerant world
Statistics
Chapter 1
Why is my evil lecturer forcing me to learn statistics?
Initial observation: finding something that needs explaining
To see whether an observation is true, you need to define one or more variables to measure that quantify the thing you’re trying to measure.
Generating and testing theories and hypotheses
A theory: an explanation or set of principles that is well substantiated by repeated testing and explains a broad phenomenon.
A hypotheses: a proposed explanation for a fairly narrow phenomenon or set of observations.
An informed, theory-driven attempt to explain what has been observed.
A theory explains a wide set of phenomena with a small set of well-established principles.
A hypotheses typically seeks to explain a narrower phenomenon and is, as yet, untested.
Both theories and hypotheses exist in the conceptual domain, and you cannot observe them directly.
To test a hypotheses, we need to operationalize our hypotheses in a way that enables us to collect and analyse data that have a bearing on the hypotheses.
Predictions emerge from a hypotheses. A prediction tells us something about the hypotheses from which it derived.
Falsification: the act of disproving a hypotheses or theory.
Collecting data: measurement
Independent and dependent variable
Variables: things that can change
Independent variable: a variable thought to be the cause of some effect.
Dependent variable: a variable thought to be affected by changes in an independent variable.
Predictor variable: a variable thought to predict an outcome variable. (independent)
Outcome variable: a variable thought to change as a function of changes in a predictor variable (dependent)
Levels of measurement
The level of measurement: the relationship between what is being measured and the number that represent what is being measured.
Variables can be categorical or continuous, and can have different levels of measurement.
A categorical variable is made up of categories.
It names distinct entities.
In its simplest form it names just two distinct types of things (like male or female).
Binary variable: there are only two categories.
Nominal variable: there are more than two categories.
Ordinal variable: when categories are ordered.
Tell us not only that things have occurred, but also the order in which they occurred.
These data tell us nothing about the differences between values. Yet they still do not tell us about the differences between point scale.
Continuous variable: a variable that gives us a score for each person and can take on any value on the measurement scale that we are using.
Interval variable: to say that data are interval, we must certain that equal intervals on the scale represents equal differences in
Statistics
Chapter 2
The spine of statistics
What is the spine of statistics?
The spine of statistics: (an acronym for)
Testing hypotheses involves building statistical models of the phenomenon of interest.
Scientists build (statistical) models of real-world processes to predict how these processes operate under certain conditions. The models need to be as accurate as possible so that the prediction we make about the real world are accurate too.
The degree to which a statistical model represents the data collected is known as the fit of the model.
The data we observe can be predicted from the model we choose to fit plus some amount of error.
Scientists are usually interested in finding results that apply to an entire population of entities.
Populations can be very general or very narrow.
Usually, scientists strive to infer things abut general populations rather than narrow ones.
We collect data from a smaller subset of the population known as a sample, and use these data to infer things about the population as a whole.
The bigger the sample, the more likely it is to reflect the whole population.
Statistical models are made up of variables and parameters.
Parameters are not measured an are (usually) constants believed to represent some fundamental truth about the relations between variables in the model.
(Like mean and median).
We can predict values of an outcome variable based on a model. The form of the model changes, but there will always be some error in prediction, and there will always be parameters that tell us about the shape or form of the model.
To work out what the model looks like, we estimate the parameters.
The mean as a statistical model
The mean is a hypothetical value and not necessarily one that is observed in the data.
Estimates have ^.
Assessing the fit of a model: sums of squares and variance revisited.
The error or deviance for a particular entity is the score predicted by the model for that entity subtracted from the corresponding observed score.
Degrees of freedom (df): the number of scores used to compute the total adjusted for the fact that we’re trying to estimate the population value.
The degrees of freedom relate to the number of observations that are free to vary.
We
.....read moreStatistics
Chapter 6
The beast of bias
Bias: the summary information is at odds with the objective truth.
An unbiased estimator: one estimator that yields and expected value that is the same thing it is trying to estimate.
We predict an outcome variable from a model described by one or ore predictor variables and parameters that tell us about the relationship between the predictor and the outcome variable.
The model will not predict the outcome perfectly, so for each observation there is some amount of error.
Statistical bias enters the statistical process in three ways:
An outlier: a score very different from the rest of the data.
Outliers have a dramatic effect on the sum of squared error.
If the sum of squared errors is biased, the associated standard error, confidence interval and test statistic will be too.
The second bias is ‘violation of assumptions’.
An assumption: a condition that ensures that what you’re attempting to do works.
If any of the assumptions are not true then the test statistic and p-value will be inaccurate and could lead us to the wrong conclusion.
The main assumptions that we’ll look at are:
Additivity and linearity
The assumption of additivity and linearity: the relationship between the outcome variable and predictor is accurately described by equation.
The scores on the outcome variable are, in reality, linearly related to any predictors. If you have several predictors then their combined effect is best described by adding their effects together.
If the assumption is not true, even if all the other assumptions are met, your model is invalid because your description of the process you want to model is wrong.
Normally distributed something or other
The assumption of normality relates in different ways to things we want to do when fitting models and assessing them:
Statistics
Chapter 7
Non-parametric models
Sometimes you can’t correct problems in your data.
This is especially irksome if you have a small sample and can’t rely on the central limit theorem to get you out of trouble.
The four most common non-parametric procedures:
All four tests overcome distributional problems by ranking the data.
Ranking the data: finding the lowest score and giving it a rank 1, then finding the next highest score and giving it the rank 3, and so on.
This process results in high scores being represented by large ranks, and low scores being represented by small ranks.
The model is then fitted to the ranks and not to the raw scores.
There are two choices to compare the distributions in two conditions containing scores from different entities:
Both tests are equivalent.
There is also a second Wilcoxon test that does something different.
Theory
If you were to rank the data ignoring the group to which a person belonged from lowest to highest, if there’s no difference between the groups, ten you should find a similar number of high and low ranks in each group.
If you were to rank the data ignoring the group to which a person belonged from lowest to highest, if there’s a difference between the groups, ten you should not find a similar number of high and low ranks in each group.
The Mann-Whitney and Wilcoxon rank-sum test use the principles above.
Statistics
Chapter 8
Correlation
The data we observe can be predicted from the model we choose to fit the data plus some error in prediction.
Outcomei = (model) + errori
Thus
outcomei = (b1Xi)+errori
z(outcome)i = b1z(Xi)+errori
z-scores are standardized scores.
A detour into the murky world of covariance
The simplest way to look at whether two variables are associated is to look whether they covary.
If two variables are related, then changes in one variable should be met with similar changes in the other variable.
Covariance (x,y) = Σni=1 ((xi-ẍ)(yi-ÿ))/N-1
The equation for covariance is the same as the equation for variance, except that instead of squaring the deviances, we multiply them by the corresponding deviance of the second variable.
A positive covariance indicates that as on variable deviates from the mean, the other variable deviates in the same direction.
A negative covariance indicates that as one variable deviates from the mean, the other deviates from the mean in the opposite direction.
The covariance depends upon the scales of measurement used: it is not a standardized measure.
Standardization of the correlation coefficient
To overcome the problem of dependence on the measurement scale, we need to convert the covariance into standard set of units → standardization.
Standard deviation: a measure of the average deviation from the mean.
If we divide any distance from the mean by the standard deviation, it gives us that distance in standard deviation units.
We can express the covariance in a standard units of measurement if we divide it by the standard deviation. But, there are two variables and hence two standard deviations.
Correlation coefficient: the standardized covariance
r = covxy/(sxsy)
sx is the standard deviation for the first variable
sy is the standard deviation for the second variable.
By standardizing the covariance we end up with a value that has to lie between -1 and +1.
A coefficient of +1 indicates that the two variables are perfectly positively correlated.
A coefficient of -1 indicates a perfect negative relationship.
A coefficient of 0 indicates no linear relationship at all.
We can test the hypothesis that the correlation is different from zero.
There are two ways of testing this hypothesis.
We can adjust r so that its sampling distribution is normal:
zr = ½ loge((1+r)/(1-r))
The resulting zr has a standard error given by:
Sezr = 1/(square root(N-3))
We can adjust r
.....read moreStatistics
Chapter 9
The linear model (regression)
The linear model with one predictor
outcome = (b0+b1xi) +errori
This model uses an unstandardised measure of the relationship (b1) and consequently we include a parameter b0 that tells us the value of the outcome when the predictor is zero.
Any straight line can be defined by two things:
These parameters are regression coefficients.
The linear model with several predictors
The linear model expands to include as many predictor variables as you like.
An additional predictor can be placed in the model given a b to estimate its relationship to the outcome:
Yi = (b0 +b1X1i +b2X2i+ … bnXni) + Ɛi
bn is the coefficient is the nth predictor (Xni)
Regression analysis is a term for fitting a linear model to data and using it to predict values of an outcome variable form one or more predictor variables.
Simple regression: with one predictor variable
Multiple regression: with several predictors
Estimating the model
No matter how many predictors there are, the model can be described entirely by a constant (b0) and by parameters associated with each predictor (bs).
To estimate these parameters we use the method of least squares.
We could assess the fit of a model by looking at the deviations between the model and the data collected.
Residuals: the differences between what the model predicts and the observed values.
To calculate the total error in a model we square the differences between the observed values of the outcome, and the predicted values that come from the model:
total error: Σni=1(observedi-modeli)2
Because we call these errors residuals, this is called the residual sum of squares (SSR).
It is a gauge of how well a linear model fits the data.
The least SSR gives us the best model.
Assessing the goodness of fit, sums of squares R and R2
Goodness of fit: how well the model fits the observed data
Total sum of squares (SST): how good the mean is as a model of the observed outcome scores.
We can use the values of SST and SSR to calculate how much better the linear model is than the
.....read moreStatistics
Chapter 10
Comparing two means
Categorical predictors in the linear model
If we want to compare differences between the means of two groups, all we are doing is predicting an outcome based on membership of two groups.
This is a linear model with one dichotomous predictor.
Independent t-test: used when you want to compare two means that come from conditions consisting of different entities (this is sometimes called the independent-measures or independent-means t-test)
Paired-samples t-test: also known as the dependent t-test. Is used when you want to compare two means that come from conditions consisting of the same or related entities.
Rationale for the t-test
Both t-tests have a similar rationale:
Most test statistics have a signal-to-noise ratio: the ‘variance explained by the model’ divided by the ‘variance that the model can’t explain’.
Effect divided by error.
When comparing two means, the model we fit is the difference between the two group means. Means vary from sample to sample (sampling variation) and we can use the standard error as a measure of how much means fluctuate. Therefore, we
Statistics
Chapter 11
Moderation, mediation, and multi-category predictors
The conceptual model
Moderation: for a statistical model to include the combined effect of two or more predictor variables on an outcome.
This is in statistical terms an interaction effect.
A moderator variable: one variable that affects the relationship between two others.
Can be continuous or categorical.
We can explore this by comparing the slope of the regression plane for X ad low and high levels of Y.
The statistical model
Moderation is conceptually.
Moderation in the statistical model. We predict the outcome from the predictor variable, the proposed variable, and the interaction of the two.
It is the interaction effect that tells us whether moderation has occurred, but we must include the predictor and moderator for the interaction term to be valid.
Outcomei = (model) + errori
or
Yi = (b0 + b1iX1i + b2iX2i + … + bnXni) + Ɛi
To add variables to a linear model we literally just add them in and assign them a parameter (b).
Therefore, if we had two predictors labelled A and B, a model that tests for moderation would be expressed as:
Yi = (b0 + b1Ai + b2Bi + b3ABi) + Ɛi
The interaction is ABi
Centring variables
When an interaction term is included in the model the b parameters have a specific meaning: for the individual predictors they represent the regression of the outcome on that predictor when the other predictor is zero.
But, there are situation where it makes no sense for a predictor to have a score of zero. So the interaction term makes the bs for the main predictors uninterpretable in many situations.
For this reason, it is common to transform the predictors using grand mean centring.
Centring: the process of transforming a variable into deviations around a fixed point.
This fixed point ca be any value that you choose, but typically it’s the grand mean.
The grand mean centring for a given variable is achieved by taking each score and subtracting from it the mean of all scores (for that variable).
Centring the predictors has no effect on the b for highest-order predictor, but will affect the bs for the lower-order predictors.
Order: how many variables are involved.
When we centre variables, the bs represent the effect of the predictor when the other predictor is at its mean value.
Centring is important when your model contains an interaction term because it makes the bs for lower-order effects interpretable.
There are good reasons for not caring about the lower-order effects when the higher-order
Statistics
Chapter 12
Comparing several independent means
ANOVA: analysis of variance
the same thing as the linear model or regression.
In designs in which the group sizes are unequal, it is important that the baseline category contains a large number of cases to ensure that the estimates of the b-values are reliable.
When we are predicting an outcome from group membership, predicted values from the model are the group means.
If the group means are meaningfully different, then using the group means should be an effective way to predict scores.
Predictioni = b0 + b1X + b2Y + Ɛi
Control = b0
Using dummy coding ins only one of many ways to code dummy variables.
The F-test is an overall test that doesn’t identify differences between specific means. But, the model parameters do.
Logic of the F-statistic
The F-statistic tests the overall fit of a linear model to a set of observed data.
F is the ratio of how good the model is compared to how bad it is.
When the model is based on group means, our predictions from the model are those means.
F tells us whether the group means are significantly different.
The same logic as for any linear model:
Statistics
Chapter 13
Comparing means adjusted for other predictors (analysis of covariance)
The linear model to compare means can be extended to include one or more continuous variables that predict the outcome (or dependent variable).
Covariates: the additional predictors.
ANCOVA: analysis of covariance.
Reasons to include covariates in ANOVA:
For example:
Happinessi = b0 + b1Longi + b2Shorti + b3Covariatei + Ɛi
We can add a covariate as a predictor to the model to test the difference between group means adjusted for the covariate.
With a covariate present, the b-values represent the differences between the means of each group and the control adjusted for the covariate(s).
Independence of the covariate and treatment effect
When the covariate and the experimental effect are not independent, the treatment effect is obscured, spurious treatment effects can arise, and at the very least the interpretation of the ANCOVA is seriously compromised.
When treatment groups differ on the covariate, putting the covariate into the analysis will not ‘control for’ or ‘balance out’ those differences.
This problem can be avoided by randomizing participants to experimental groups, or by matching experimental groups on the covariate.
We can see whether this problem is likely to be an issue by checking whether experimental groups differ on the covariate before fitting the model.
If they do not significantly differ then we might consider it reasonable to use it as a covariate.
Homogeneity of regression slopes
When a covariate is used we loot at its overall relationship with the outcome variable:; we ignore the group to which a person belongs.
We assume that this relationship between covariate and outcome variable holds true for all groups of participants: homogeneity of regression slopes.
There are situations where you might expect regression slopes to differ across groups and that variability may be interesting.
What to do when assumptions are violated
But bootstrap won’t help for the F-tests.
There is a robust variant of ANCOVA.
The main analysis
The format of the ANOVA table is largely the same as without the covariate, except that there is an additional
.....read moreStatistics
Chapter 14
Factorial designs
Factorial design: when an experiment has two or more independent variables.
There are several types of factorial designs:
We can still fit a linear model to the design.
Factorial ANOVA: the linear model with two or more categorical predictors that represent experimental independent variables.
The general linear model takes the following general form:
Yi =b0 + b1X1i+b2X2i+... +bnXni+Ɛi
We can code participant’s category membership on variables with zeros and ones.
For example:
Attractivenessi = b0+b1Ai+b2Bi+b3ABi+Ɛi
b3AB is the interaction variable. It is A dummy multiplied by B dummy variable.
Behind the scenes of factorial designs
Calculating the F-statistic with two categorical predictors is very similar to when we had only one.
Therefore, the sum of squares gets further subdivided into
Total sum of squares (SST)
We start of with calculating how much variability there is between scores when the ignore the experimental condition from which they came.
The grand variance: the variance of all scores when we ignore the group to which they belong.
We treat the data as one big group.
The degrees of freedom are: N-1
SST = s2Grand(N-1)
The model sum of squares (SSM)
The model sum of squares is broken down into the variance attributable to the first independent variable, the variance attributable to the second independent variable, and the variance attributable to the interaction of those two.
The model sum of squares: the difference between what the model predicts and the overall mean of the outcome variable.
Statistics
Chapter 15
Repeated measures designs
Repeated measures: when the same entities participate in all conditions of an experiment or provide data at multiple time points.
Repeated measures can also be considered as a variation of the general linear model.
For example.
Ygi = b0i +b1iXgi +Ɛgi
b0i = b0 + u0i
b1i = b1 + u1i
Ygi for outcome g within person i from the specific predictor Xgi with the error Ɛgi
g is the level of treatment condition
i for the individuals
u0i for the deviation of the individual’s intercept from the group-level intercept
The way that people typically handle repeated measures in IBM SPSS is to use a repeated-measures ANOVA approach.
The assumption of sphericity
The assumption that permits us to use a simpler model to analyse repeated-measures data is sphericity.
Sphericity: assuming that the relationship between scores in pairs of treatment conditions is similar.
It is a form of compound symmetry: holds true when both the variances across conditions are equal and the covariances between pairs of conditions are equal.
We assume that the variation within conditions is similar and that no two conditions are any more dependent than any other two.
Sphericity is a more general, less restrictive form of compound symmetry and refers to the equality of variances of the differences between treatment levels.
For example:
varianceA-B = varianceA-C = varianceB-C
Assessing the severity of departures from sphericity
Mauchly’s test: assesses the hypothesis that the variances of the differences between conditions are equal.
If the test is statistically significant, it implies that there are significant differences between the variances of differences and, therefore, sphericity is not met.
If it is not significant, the implication is that the variances of differences are roughly equal and sphericity is met.
It depends upon sample size.
What’s the effect of violating the assumption of sphericity?
A lack of sphericity creates a loss of power and an F-statistic that doesn’t have the distribution that it’s supposed to have.
It also causes some complications for post
Statistics
Chapter 16
Mixed designs
Situations where we combine repeated-measures and independent designs.
Mixed designs: when a design includes some independent variables that were measured using different entities and others that used repeated measures.
A mixed design requires at least two independent variables.
Because by adding independent variables we’re simply adding predictors to the linear model, you can have virtually any number of independent variables if your sample size is gin enough.
We’re still essentially using the linear model.
Because there are repeated measures involved, people typically use an ANOVA-style model. Mixed ANOVA
All the sources of potential bias in chapter 6 apply.
You can apply the Greenhouse-Geisser correction and forget about sphericity.
Effect sizes are more useful when they summarize a focused effect.
A straightforward approach is to calculate effect sizes for your contrasts.
Statistics
Chapter 17
Multivariate analysis of variance (MANOVA)
Multivariate analysis of variance (MANOVA) is used when we are interested in several outcomes.
The principles of the linear model extend to MANOVA in that we can use MANOVA when there is one independent variable or several, we can look at interactions between outcome variables, and we can do contrasts to see which groups differ.
Univariate: the model when we have only one outcome variable.
Multivariate: the model when we include several outcome variables simultaneously.
We shouldn’t fit separate linear models to each outcome variable.
Separate models can tell us only whether groups differ along a single dimension, MANOVA has the power to detect whether groups differ along a combination of dimensions.
Choosing outcomes
It is a bad idea to lump outcome measures together in a MANOVA unless you have a good theoretical or empirical basis for doing so.
Where there is a good theoretical basis for including some, but not all, of your outcome measures, then fit separate models: one for the outcomes being tested on a heuristic and one for the theoretically meaningful outcomes.
The point here is not to include lots of outcome variables in a MANOVA just because you measured them.
A matrix: a grid of numbers arranged in columns and rows.
A matrix can have many columns and rows, and we specify its dimensions using numbers.
For example: a 2 x 3 matrix is a matrix with two rows and three columns.
The values within a matrix are components or elements.
The rows and columns are vectors.
A square matrix: a matrix with an equal number of columns and rows.
An identity matrix: a square matrix in which the diagonal elements are 1 and the off-diagonal elements are 0.
The matrix that represents the systematic variance (or the model sum of squares for all variables) is denoted by the letter H and is called the hypothesis sum of squares and cross-products matrix (or hypothesis SSCP).
The matrix that represents the unsystematic variance (the residual sums of squares for all variables) is denoted by the letter E and called the error sum of squares and cross-products matrix (or error SSCP).
The matrix that represents the total amount of variance present for each outcome variable is denoted by T and is called the total sum of squares and cross-products matrix (or total SSCP).
Cross-products represent a total value for the combined error between two variables.
Whereas the sum of squares of a variable is
Statistics
Chapter 18
Exploratory factor analysis
In factor analysis, we take a lot of information (variables) and a computer effortlessly reduces this into a simple message (fewer variables).
Latent variable: something that cannot be accessed directly.
Measuring what the observable measures driven by the same underlying variable are.
Factor analysis and principal component analysis (PCA) are techniques for identifying clusters of variables.
Three main uses:
If we measure several variables, or ask someone several questions about themselves, the correlation between each pair of variables can be arranged in a table.
Factor analysis attempts to achieve parsimony by explaining the maximum amount of common variance in a correlation matrix using the smallest number of explanatory constructs.
Explanatory constructs are known as latent variables (or factors) and they represent clusters of variables that correlate highly with each other.
PCA differs in that it tries to explain the maximum amount of total variance in a correlation matrix by transforming the original variables into linear components.
Factor analysis and PCA both aim to reduce the R matrix into a smaller set of dimensions.
Graphical representation
Factors and components can be visualized as the axis of a graph along which we plot variables.
The coordinates of variables along each axis represent the strength of relationship between that variable and each factor.
In an ideal world a variable will have a large coordinate for one of the axes and small coordinates for any others.
Factor loading: the coordinate of a variable along a classification axis.
If we square the factor loading for a variable we get a measure of its substantive importance to a factor.
Mathematical representation
.....read moreStatistics
Chapter 19
Categorical outcomes: chi-square and loglinear analysis
Analysing categorical data
Sometimes we want to predict categorical outcome variables. We want to predict into which category an entity falls.
With categorical variables we can’t use the mean or any similar statistic because the mean of a categorical variable is meaningless: the numeric values you attach to different categories are arbitrary, and the mean of those numeric values will depend on how many members each category has.
When we’ve measured only categorical variables, we analyse the number of things that fall into each combination of categories (the frequencies).
Pearson’s chi-square test
To see whether there’s a relationship between two categorical variables we can use the Pearson’s chi-square test.
This statistic is based on the simple idea of comparing the frequencies you observe in certain categories to the frequencies you might expect to get in those categories by chance.
X2 = Σ(observedij-modelij)2 / modelij
i represents the rows in the contingency table
j represents the columns in the contingency table.
As model we use ‘expected frequencies’.
To adjust for inequalities, we calculate frequencies for each cell in the table using the column and row totals for that cell.
By doing so we factor in the total number of observations that could have contributed to that cell.
Modelij = Eij = (row totali x column totalj) / n
X2 has a distribution with known properties called the chi-square distribution. This has a shape determined by the degrees of freedom: (r-1)(c-1)
r = the number of rows
c = the number of columns
Fischer’s exact test
The chi-square statistic has a sampling distribution that is only approximately a chi-square distribution.
The larger the sample is, the better this approximation becomes. In large samples the approximation is good enough not to worry about the fact that it is an approximation.
In small samples, the approximation is not good enough, making significance tests of the chi-square statistic inaccurate.
Fischer’s exact tests: a way to compute the exact probability of the chi-square statistic in small samples.
The likelihood ratio
An alternative to Pearson’s chi-square.
Based on maximum-likelihood theory.
General idea: you collect some data and create a model for which the probability of obtaining the observed set of data is maximized, then you compare this model to the probability of obtaining those data under the null hypothesis.
The resulting statistic is based on comparing observed frequencies with those predicted by the
Here is a short explanation how to do tests in SPSS. These are the tests needed for the third block of WSRt and psychology at the second year of the uva.
This magazine contains all the summaries you need for the course WSRt at the second year of psychology at the Uva.
Discovering statistics using IBM SPSS statistics
Chapter 20
Categorical outcomes: logistic regression
This summary contains the information from chapter 20.8 and forward, the rest of the chapter is not necessary for the course.
Logistic regression is a model for predicting categorical outcomes from categorical and continuous predictors.
A binary logistic regression is when we’re trying to predict membership of only two categories.
Multinominal is when we want to predict membership of more than two categories.
The linear model can be expressed as: Yi = b0 + b1Xi + errori
b0 is the value of the outcome when the predictors are zero (the intercept).
The bs quantify the relationship between each predictor and outcome.
X is the value of each predictor variable.
One of the assumptions of the linear model is that the relationship between the predictors and outcome is linear.
When the outcome variable is categorical, this assumption is violated.
One way to solve this problem is to transform the data using the logarithmic transformation, where you can express a non-linear relationship in a linear way.
In logistic regression, we predict the probability of Y occurring, P(Y) from known (logtransformed) values of X1 (or Xs).
The logistic regression model with one predictor is:
P(Y) = 1/(1+e –(b0 +b1X1i))
The value of the model will lie between 1 and 0.
You need to test for
Multinomial logistic regression predicts membership of more than two categories.
The model breaks the outcome variable into a series of comparisons between two categories.
In practice, you have to set a baseline outcome category.
JoHo kan jouw hulp goed gebruiken! Check hier de diverse studentenbanen die aansluiten bij je studie, je competenties verbeteren, je cv versterken en een bijdrage leveren aan een tolerantere wereld
There are several ways to navigate the large amount of summaries, study notes en practice exams on JoHo WorldSupporter.
Do you want to share your summaries with JoHo WorldSupporter and its visitors?
Field of study
Add new contribution