A conceptual introduction to psychometrics, development, analysis, and application of psychological and educational tests, by G. J. Mellenberg (first edition) – Summary chapter 5

Measurement precision consists of information (1) and reliability (2). Information applies to the test score of a single person. It is the within-person aspect of measurement precision. Reliability applies to a population of persons. It is the between-person aspect of measurement precision. The true score is the score with a perfect measurement instrument. Measurement error is the distortion of the true score because the measurement instrument is not perfect. They are unsystematic influences.

The test score of a person is the true score of that person plus the measurement error:

The true score equals the mean test score over infinite test administrations. The expected value of the measurement error over infinite trials equals zero. Test taker j’s standard error of measurement is the square root of the within-person variance. The information on test taker j’s true score is the inverse of the within-person error variance and uses the following formula:

ere are several guidelines for reliability. A parallel test is a different test with exactly the same properties as the original test. Reliability is the correlation between two parallel test scores. Reliability can be calculated by splitting the test in two halves and treating them both as parallel tests. The reliability of each part equals the correlation between the two parts. It makes use of the following formula:

It is two times the correlation between part one and part two divided by one plus the correlation between part one and part two. Cronbach’s alpha is used to calculate the lower-bound of the reliability of the full test. It uses the following formula:

It is the number of items divided by number of items minus one times (one minus the sum of item variances divided by the test variance). The reliability depends on the number of items. A large test is usually more reliable than a shorter test. The correlation between the test scores is not equal to the correlation between the constructs, as the test score is the true score plus measurement error. The correlation between the test score is smaller than the correlation between true scores. The correction for attenuation formula corrects for this difference and uses the following formula:

It is the correlation between scores of test A and the scores of test B divided by the square root of the reliability of test A times the reliability of test B. The reliability of a test after adding a number of parallel tests containing a number of items can be calculated using the Spearman-Brown formula:

RelS(K) = KRel(S)1+K-1Rel(S) =

It is the number of parallel parts times the reliability of one parallel part divided by one plus (the number of parallel parts minus one) times the reliability of one parallel part.

The number of items needed for a particular reliability can be calculated. It uses the following formula:

It is one minus the reliability of the test times the desired reliability divided by one minus the desired reliability times the reliability of the test. Rel(S) denotes the reliability of the test. Rel{S(K)} denotes the desired reliability. K denotes the number of parts that should be added to the test.

The higher the reliability the smaller the standard error of measurement. The lower the reliability the larger the standard error of measurement. It can be calculated by using the reliability of the test:

It is the square root of the variance of the test times one minus the reliability of the test. It can also be calculated using the within-person variance:

It is the square root of the sum of the score of test taker j on test 1 minus the score of test taker j on test 2 squared divided by the number of test takers. The standard error of measurement can be used to construct confidence intervals of the true score.