What is a confidence interval in null hypothesis significance testing?

Statistics
Chapter 1
Why is my evil lecturer forcing me to learn statistics?

The research process

Initial observation: finding something that needs explaining

To see whether an observation is true, you need to define one or more variables to measure that quantify the thing you’re trying to measure.

Generating and testing theories and hypotheses

A theory: an explanation or set of principles that is well substantiated by repeated testing and explains a broad phenomenon.

A hypotheses: a proposed explanation for a fairly narrow phenomenon or set of observations.
An informed, theory-driven attempt to explain what has been observed.

A theory explains a wide set of phenomena with a small set of well-established principles.
A hypotheses typically seeks to explain a narrower phenomenon and is, as yet, untested.
Both theories and hypotheses exist in the conceptual domain, and you cannot observe them directly.

To test a hypotheses, we need to operationalize our hypotheses in a way that enables us to collect and analyse data that have a bearing on the hypotheses.
Predictions emerge from a hypotheses. A prediction tells us something about the hypotheses from which it derived.

Falsification: the act of disproving a hypotheses or theory.

Collecting data: measurement

Independent and dependent variable

Variables: things that can change

Independent variable: a variable thought to be the cause of some effect.

Dependent variable: a variable thought to be affected by changes in an independent variable.

Predictor variable: a variable thought to predict an outcome variable. (independent)

Outcome variable: a variable thought to change as a function of changes in a predictor variable (dependent)

Levels of measurement

The level of measurement: the relationship between what is being measured and the number that represent what is being measured.

Variables can be categorical or continuous, and can have different levels of measurement.

A categorical variable is made up of categories.
It names distinct entities.
In its simplest form it names just two distinct types of things (like male or female).
Binary variable: there are only two categories.
Nominal variable: there are more than two categories.

Ordinal variable: when categories are ordered.
Tell us not only that things have occurred, but also the order in which they occurred.
These data tell us nothing about the differences between values. Yet they still do not tell us about the differences between point scale.

Continuous variable: a variable that gives us a score for each person and can take on any value on the measurement scale that we are using.
Interval variable: to say that data are interval, we must certain that equal intervals on the scale represents equal differences in the property being measured.
Ratio variables: in addition to

Statistics
Chapter 6
The beast of bias

What is bias?

Bias: the summary information is at odds with the objective truth.

An unbiased estimator: one estimator that yields and expected value that is the same thing it is trying to estimate.

We predict an outcome variable from a model described by one or ore predictor variables and parameters that tell us about the relationship between the predictor and the outcome variable.
The model will not predict the outcome perfectly, so for each observation there is some amount of error.

Statistical bias enters the statistical process in three ways:

• things that bias the parameter estimates (including effect sizes)
• things that bias standard errors and confidence intervals
• things that bias test statistics and p-values

Outliers

An outlier: a score very different from the rest of the data.

Outliers have a dramatic effect on the sum of squared error.
If the sum of squared errors is biased, the associated standard error, confidence interval and test statistic will be too.

Overview of assumptions

The second bias is ‘violation of assumptions’.

An assumption: a condition that ensures that what you’re attempting to do works.
If any of the assumptions are not true then the test statistic and p-value will be inaccurate and could lead us to the wrong conclusion.

The main assumptions that we’ll look at are:

• additivity and linearity
• normality of something or other
• homoscedasticity/ homogeneity of variance
• independence

Additivity and linearity

The assumption of additivity and linearity: the relationship between the outcome variable and predictor is accurately described by equation.
The scores on the outcome variable are, in reality, linearly related to any predictors. If you have several predictors then their combined effect is best described by adding their effects together.

If the assumption is not true, even if all the other assumptions are met, your model is invalid because your description of the process you want to model is wrong.

Normally distributed something or other

The assumption of normality relates in different ways to things we want to do when fitting models and assessing them:

• Parameter estimates.
  The mean is a parameter and extreme scores can bias it.
  Estimates of parameters are affected by non-normal distributions (such as those with outliers).
  Parameter estimates differ in how much they are biased in a non-normal distribution.
• Confidence intervals
  We use values of the standard normal distribution to compute the confidence interval around a parameter estimate. Using values of he standard normal distribution makes sense only if the parameter estimates comes from

Statistics
Chapter 8
Correlation

Modeling relationships

The data we observe can be predicted from the model we choose to fit the data plus some error in prediction.

Outcome_i = (model) + error_i
Thus
outcome_i = (b₁X_i)+error_i

z(outcome)_i = b₁z(X_i)+error_i

z-scores are standardized scores.

A detour into the murky world of covariance

The simplest way to look at whether two variables are associated is to look whether they covary.
If two variables are related, then changes in one variable should be met with similar changes in the other variable.

Covariance (x,y) = Σⁿ_i=1 ((x_i-ẍ)(y_i-ÿ))/N-1

The equation for covariance is the same as the equation for variance, except that instead of squaring the deviances, we multiply them by the corresponding deviance of the second variable.

A positive covariance indicates that as on variable deviates from the mean, the other variable deviates in the same direction.
A negative covariance indicates that as one variable deviates from the mean, the other deviates from the mean in the opposite direction.

The covariance depends upon the scales of measurement used: it is not a standardized measure.

Standardization of the correlation coefficient

To overcome the problem of dependence on the measurement scale, we need to convert the covariance into standard set of units → standardization.
Standard deviation: a measure of the average deviation from the mean.
If we divide any distance from the mean by the standard deviation, it gives us that distance in standard deviation units.
We can express the covariance in a standard units of measurement if we divide it by the standard deviation. But, there are two variables and hence two standard deviations.

Correlation coefficient: the standardized covariance

r = cov_xy/(s_xs_y)

s_x is the standard deviation for the first variable
s_y is the standard deviation for the second variable.

By standardizing the covariance we end up with a value that has to lie between -1 and +1.
A coefficient of +1 indicates that the two variables are perfectly positively correlated.
A coefficient of -1 indicates a perfect negative relationship.
A coefficient of 0 indicates no linear relationship at all.

The significance of the correlation coefficient

We can test the hypothesis that the correlation is different from zero.
There are two ways of testing this hypothesis.

We can adjust r so that its sampling distribution is normal:

z_r = ½ log_e((1+r)/(1-r))

The resulting z_r has a standard error given by:

Se_zr = 1/(square root(N-3))

We can adjust r into a z-score

z = z_r/Se_zr

Statistics
Chapter 10
Comparing two means

Categorical predictors in the linear model

If we want to compare differences between the means of two groups, all we are doing is predicting an outcome based on membership of two groups.
This is a linear model with one dichotomous predictor.

The t-test

Independent t-test: used when you want to compare two means that come from conditions consisting of different entities (this is sometimes called the independent-measures or independent-means t-test)
Paired-samples t-test: also known as the dependent t-test. Is used when you want to compare two means that come from conditions consisting of the same or related entities.

Rationale for the t-test

Both t-tests have a similar rationale:

• two samples of data are collected and the sample means calculated. These might differ by either a little or a lot
• If the samples come from the same population, then we expect their means to be roughly equal. Although it is possible for the means to differ because of sample variation, we would expect large differences between sample means to occur very infrequently. Under the null hypothesis we assume that the experimental manipulation has no effect on the participant’s behaviour: therefore, we expect means from two random samples to be very similar.
• We compare the difference between the sample means that we collected to the difference between the sample means that we would expect to obtain (in the long run) if there were no effect. We use the standard error as a gauge of the variability between sample means. If the standard error is small, then we expect most samples to have very similar means. When the standard error is large, large differences in sample means are more likely. If the difference between the samples we have collected is larger than we would expect based on the standard error then one of two things has happened:
  • There is no effect but sample means form our population fluctuate a lot and we happen to have collected two samples that produce very different means.
  • The two samples come from different populations, which is why they have different means, and this difference is indicative of a genuine difference between the samples.
• the larger the observed difference between the sample means, the more likely it is that the second explanation is correct.

Most test statistics have a signal-to-noise ratio: the ‘variance explained by the model’ divided by the ‘variance that the model can’t explain’.
Effect divided by error.
When comparing two means, the model we fit is the difference between the two group means. Means vary from sample to sample (sampling variation) and we can use the standard error as a measure of how much means fluctuate. Therefore, we can use the standard error of the differences between the

Statistics
Chapter 12
Comparing several independent means

Using a linear model to compare several means

ANOVA: analysis of variance
the same thing as the linear model or regression.

In designs in which the group sizes are unequal, it is important that the baseline category contains a large number of cases to ensure that the estimates of the b-values are reliable.

When we are predicting an outcome from group membership, predicted values from the model are the group means.
If the group means are meaningfully different, then using the group means should be an effective way to predict scores.

Prediction_i = b₀ + b₁X + b₂Y + Ɛ_i

Control = b₀

Using dummy coding ins only one of many ways to code dummy variables.

• an alternative is contrast coding: in which you code the dummy variables in such a way that the b-values represent differences between groups that you specifically hypothesized before collecting data.

The F-test is an overall test that doesn’t identify differences between specific means. But, the model parameters do.

Logic of the F-statistic

The F-statistic tests the overall fit of a linear model to a set of observed data.
F is the ratio of how good the model is compared to how bad it is.
When the model is based on group means, our predictions from the model are those means.

• if the group means are the same then our ability to predict the observed data will be poor (F will be small)
• if the means differ we will be able to better discriminate between cases from different groups (F will be large).

F tells us whether the group means are significantly different.

The same logic as for any linear model:

• the model that represents ‘no effect’ or ‘no relationship between the predictor variable and the outcome’ is one where the predicted value of the outcome is always the grand mean
• we can fit a different model to the data that represents our alternative hypotheses. We compare fit of this model to the fit of the null model
• the intercept and one or more parameters (b) describe the model
• the parameters determine the shape of the model that we have fitted.
• in experimental research the parameters (b) represent the differences between group means. The bigger the differences between group means, the greater the difference between the model and the null model (grand mean)
• if the differences between group means are large enough, then the resulting model will be a better fit to the data than the null model

Statistics
Chapter 14
Factorial designs

Factorial designs

Factorial design: when an experiment has two or more independent variables.
There are several types of factorial designs:

• Independent factorial design: there are several independent variables or predictors and each has been measured using different entities (between groups).
• Repeated-measures (related) factorial design: several independent variables or predictors have been measured, but the same entities have been used in all conditions.
• Mixed design: several independent variables or predictors have been measured: some have been measured with different entities, whereas others used the same entities.

We can still fit a linear model to the design.
Factorial ANOVA: the linear model with two or more categorical predictors that represent experimental independent variables.

Independent factorial designs and the linear model

The general linear model takes the following general form:

Y_i =b₀ + b₁X_1i+b₂X_2i+... +b_nX_ni+Ɛ_i

We can code participant’s category membership on variables with zeros and ones.

For example:

Attractiveness_i = b₀+b₁A_i+b₂B_i+b₃AB_i+Ɛ_i

b₃AB is the interaction variable. It is A dummy multiplied by B dummy variable.

Behind the scenes of factorial designs

Calculating the F-statistic with two categorical predictors is very similar to when we had only one.

• We still find the total sum of squared errors (SS_T) and break this variance down into variance that can be explained by the model/experiment (SS_M) and variance that cannot be explained (SS_R)
• The main difference is that with factorial designs, the variance explained by the model/experiment is made up of not one predictor, but two.

Therefore, the sum of squares gets further subdivided into

• variance explained by the first predictor/independent variable (SS_A)
• variance explained by the second predictor/independent variable (SS_B)
• variance explained by the interaction of these two predictors (SS_AxB)

Total sum of squares (SS_T)

We start of with calculating how much variability there is between scores when the ignore the experimental condition from which they came.

The grand variance: the variance of all scores when we ignore the group to which they belong.
We treat the data as one big group.
The degrees of freedom are: N-1

SS_T = s²_Grand(N-1)

The model sum of squares (SS_M)

The model sum of squares is broken down into the variance attributable to the first independent variable, the variance attributable to the second independent variable, and the variance attributable to the interaction of those two.

The model sum of squares: the difference between what the model predicts and the overall mean of the outcome variable.
What the model predicts is the group mean.
We

Statistics
Chapter 16
Mixed designs

Mixed designs

Situations where we combine repeated-measures and independent designs.

Mixed designs: when a design includes some independent variables that were measured using different entities and others that used repeated measures.
A mixed design requires at least two independent variables.

Because by adding independent variables we’re simply adding predictors to the linear model, you can have virtually any number of independent variables if your sample size is gin enough.

We’re still essentially using the linear model.
Because there are repeated measures involved, people typically use an ANOVA-style model. Mixed ANOVA

Assumptions in mixed designs

All the sources of potential bias in chapter 6 apply.

• homogeneity of variance
• sphericity

You can apply the Greenhouse-Geisser correction and forget about sphericity.

Mixed designs

• Mixed designs compare several means when there are two or more independent variables, and at least one of them has been measured using the same entities and at least one other has been measured using different entiteis.
• Correct for deviations from sphericity for the repeated-measures variable(s) by routinely interpreting the Greenhouse-Geisser corrected effects.
• The table labelled Tests of Within-Subject Effects shows the F-statistic(s) for any repeated-measures variables and all of the interaction effects. For each effect, read the row labelled Greenhouse-Geisser or Huynh-Feldt. If the values in the Sig column is less than 0.05 then the means are significantly different
• The table labelled Test of Between-Subjects Effects shows the F-statistic(s) for any between-group variables. If the value in the Sig column is less than 0.05 then the means of the groups are significantly different
• Break down the mean effects and interaction terms using contrasts. These contrasts appear in the table labelled Tests of Within-Subjects Contrasts. Again, look at the column labelled sig.
• Look at the means, or draw graphs, to help you interpret contrasts.

Calculating effect sizes

Effect sizes are more useful when they summarize a focused effect.

A straightforward approach is to calculate effect sizes for your contrasts.

Statistics
Chapter 18
Exploratory factor analysis

In factor analysis, we take a lot of information (variables) and a computer effortlessly reduces this into a simple message (fewer variables).

When to use factor analysis

Latent variable: something that cannot be accessed directly.

Measuring what the observable measures driven by the same underlying variable are.

Factor analysis and principal component analysis (PCA) are techniques for identifying clusters of variables.
Three main uses:

• To understand the structure of a set of variables
• to construct a questionnaire to measure an underlying variable
• to reduce a data set to a more manageable size while retaining as much of the original information as possible.

Factors and components

If we measure several variables, or ask someone several questions about themselves, the correlation between each pair of variables can be arranged in a table.

• this table is sometimes called the R-matrix.

Factor analysis attempts to achieve parsimony by explaining the maximum amount of common variance in a correlation matrix using the smallest number of explanatory constructs.
Explanatory constructs are known as latent variables (or factors) and they represent clusters of variables that correlate highly with each other.

PCA differs in that it tries to explain the maximum amount of total variance in a correlation matrix by transforming the original variables into linear components.

Factor analysis and PCA both aim to reduce the R matrix into a smaller set of dimensions.

• in factor analysis these dimensions, or factors, are estimated form the data and are believed to reflect constructs that can’t be measured directly.
• PCA transforms the data into a set of linear components. It doesn’t estimate unmeasured variables, it just transforms measured ones.

Graphical representation

Factors and components can be visualized as the axis of a graph along which we plot variables.
The coordinates of variables along each axis represent the strength of relationship between that variable and each factor.
In an ideal world a variable will have a large coordinate for one of the axes and small coordinates for any others.

• this scenario indicates that this particular variable is related to only one factor.
• variables that haver large coordinates on the same axis are assumed to measure different aspects of some common underlying dimension.

Factor loading: the coordinate of a variable along a classification axis.

If we square the factor loading for a variable we get a measure of its substantive importance to a factor.

Mathematical representation

A component ins PCA can be described as:

Component_i = b₁Variable_1i

Here is a short explanation how to do tests in SPSS. These are the tests needed for the third block of WSRt and psychology at the second year of the uva.

Correlation analysis (two continuous variables)

1. Open the data
2. Go to analyse, correlate, bivariate
3. Place the variables of which you want to know the correlation under ‘variables’
4. Click on ‘paste’ and run the syntax

Partial correlation (three continuous variables and you want to know the correlation between two variables, corrected for a third variable)

1. Open the data
2. Go to analyse, correlate, partial
3. Place the variable of which you want to know the correlation under ‘variables’
4. Place the variable for which you want to control under ‘controlling for’
5. Click on ‘options’
   Select ‘zero-order correlations’ (this is the correlation without controlling for one variable)
6. Click on ‘continue’
7. Click on ‘paste’ and run the syntax

Multiple regression analysis

1. Open the data
2. Go to analyse, regression, linear
3. Place the dependent variable under ‘dependent’
4. Place the independent variables under ‘independent’
   If you want to run more models, you can put the first variable under ‘independent’, click on ‘next’ and put the next variable under ‘independent’ (this way you can compare the models)
5. Click on ‘statistics’ and select:
   Model fit
   R squared change (if you have multiple models)
   Descriptives
   Part and partial correlations
   Collinearity diagnostics
6. Click on ‘plots’
   Put ZPRED under Y
   Put ZRESID under X
   (This is for testing homoscedasticity)
7. Click on ‘save’ and select:
   Unstandardised
   (for expected values)
   Mahalanobis
   Cook’s
   Leverage values
   (for outliers)
8. Click on paste and run the syntax

Principal component analysis

1. Open the data
2. Go to analyse, dimension-reduction, Factor
3. Put the items which you want to analyse under ‘variables’
4. Click on ‘descriptives’ and select:
   Univariate descriptives
   Initial solution
   Coefficients
   Significance levels
   Anti-image (for assumptions)
   KMO and Bartlett’s test of sphericity (also for assumptions)
5. Click on Extraction
   Chose Principal component analysis
   Select:
   Scree plot
   Chose for an eigenvalue bigger than 1
6. Click on rotation and select:
   Varimax
7. Click on options and select:
   Suppress

Discovering statistics using IBM SPSS statistics
Chapter 20
Categorical outcomes: logistic regression

This summary contains the information from chapter 20.8 and forward, the rest of the chapter is not necessary for the course.

What is logistic regression?

Logistic regression is a model for predicting categorical outcomes from categorical and continuous predictors.

A binary logistic regression is when we’re trying to predict membership of only two categories.
Multinominal is when we want to predict membership of more than two categories.

Theory of logistic regression

The linear model can be expressed as: Y_i = b₀ + b₁X_i + error_i

b₀ is the value of the outcome when the predictors are zero (the intercept).
The bs quantify the relationship between each predictor and outcome.
X is the value of each predictor variable.

One of the assumptions of the linear model is that the relationship between the predictors and outcome is linear.
When the outcome variable is categorical, this assumption is violated.
One way to solve this problem is to transform the data using the logarithmic transformation, where you can express a non-linear relationship in a linear way.

In logistic regression, we predict the probability of Y occurring, P(Y) from known (logtransformed) values of X₁ (or Xs).
The logistic regression model with one predictor is:
P(Y) = 1/(1+e ^{–(b0 +b1X1i)})
The value of the model will lie between 1 and 0.

Testing assumptions

You need to test for

• Linearity of the logit
  You need to check that each continuous variable is linearly related to the log of the outcome variable.
  If this is significant, it indicates that the main effect has violated the assumption of linearity of the logic.
• Multicollinearity
  This has a biasing effect

Predicting several categories: multinomial logistic regression

Multinomial logistic regression predicts membership of more than two categories.
The model breaks the outcome variable into a series of comparisons between two categories.
In practice, you have to set a baseline outcome category.