Variance of a single variable represents the average amount that the data vary from the mean. The cross-product deviation multiplies the deviation for one variable by the corresponding deviation for the second variable. The average value of the cross-product deviation is the covariance. This is an averaged sum of combined deviation. It uses the following formula:

A positive covariance indicates that if one variable deviates from the mean, the other variable deviates in the same direction. A negative covariance indicates that if one variable deviates from the mean, the other variable deviates in the opposite direction.

Covariance is not standardized and depends on the scale of measurement. The standardized covariance is the correlation coefficient and is calculated using the following formula:

A correlation coefficient of values  0.1 represents a small effect. Values of  0.3 represent a medium effect and values of  0.5 represent a large effect.

 In order to test the null hypothesis of the correlation, namely that the correlation is zero, z-scores can be used. In order to use the z-scores, the distribution must be normal, but the r-sampling distribution is not normal. The following formula adjusts r in order to make the sampling distribution normal:

The standard error uses the following formula:

This leads to the following formula for z:

The null hypothesis of correlations can also be tested using the t-score with degrees of freedom N-2:

The confidence intervals for the correlation uses the same formula as all the other confidence intervals. These values have to be converted back to a correlation efficient using the following formula:

CORRELATION
Normality in correlation is only important if the sample size is small (1), there is significance testing (2) or there is a confidence interval (3). The assumptions of correlation are normality (1) and linearity (2).

The correlation coefficient squared (R2) is a measure of the amount of variability in one variable that is shared by the other. Spearman’s correlation coefficient (rs) is a non-parametric statistic that is sued to minimize the effects of extreme scores or the effects of violations of the assumptions. Spearman’s correlation coefficient works best if the data is ranked. Kendall’s tau, denoted by τ, is a non-parametric statistic that is used when the data set is small with a large set of tied ranks.

A biserial or point-biserial correlation is used when a relationship between two variables is investigated when one of the two variables is dichotomous (e.g. yes or no). The dichotomous variable can be discrete or continuous. It is discrete when there are only two options and no in between options (e.g. dead or alive). Margins do not matter here. It is continuous if there are two options, but there are in between options as margins matter (e.g. passing or failing a test is dichotomous, but depends on margins, you can fail hard or fail just barely).

The point-biserial correlation coefficient (rpb) is used when one variable is a discrete dichotomy. The biserial correlation coefficient (rb) is used when one variable is a continuous dichotomy.

A semi-partial correlation expresses the unique relationship between two variables as a function of their total variance. It can be used to control for the effect of a third variable. It is a correlation that accounts for the effect of a third variable on only one variable in the original correlation. The partial correlation expresses the unique relationship in terms of variance in variable Y left over when other variables have been considered. It is a correlation that accounts for the effect of a third variable on both variables in the original correlation.

COMPARING CORRELATION
In order to compare independent correlations, the z-score of the difference can be calculate using the following formula:

This z-score can be used to see if there is a significant difference. In order to compare correlations that come from the same entities, and thus are dependent, the t-score of the difference can be used:

This t-score uses a degrees of freedom of N-3.