Observed test scores - a summary of chapter 4 of A conceptual introduction to psychometrics by G, J., Mellenbergh

A conceptual introduction to psychometrics
Chapter 4
Observed test scores

The aim of testing is to yield scores of test takers’ maximum or typical performance.
Two main types of test scores are distinguished

• Observed test
  Computed after the separate test items are scored.
  Derived from the item scores by taking the unweighted or weighted sum of the item scores.
  The latent variable is unobserved, and in general, the laten variable is not a simple sum of item scores.
• Latent variable (construct) scores
  To compute the latent variable score, a model is needed that specifies the relation between the latent variable and item responses.
  The latent variable score is derived from the item responses under the assumption of a latent variable item response model.

Item scoring by fiat

Conventionally, items are scored by assigning ordinal numbers to the responses.
The scoring differs slightly between maximum and typical performance tests.

• Maximum performance items are scored by assigning 0 to the lowest category, and consecutive rank numbers to subsequent categories.
• Typical performance items are indicative or contra-indicative of the latent variable that is measured by the test, and the scoring of contra-indicative item has to be reversed with respect to the scoring of indicative items.
• Dichotomous indicative typical performance items are scored assigning 0 to the ‘no’ (don’t agree), and 1 to the yes (agree) categorie.
  Whereas contra-indicative items are scored by assigning 0 to the ‘yes’, and 1 to the ‘no’ category.
• The categories of ordinal-polytomous items are scored by assigning rank numbers to the categories.
• Bounded-continuous items ares cored in measurement units, such as centimeters.

Measurement by fiat: the item scores are assigned to a test taker’s responses without any theoretical justification.
(for example, scores 0 and 1 are assigned to a correct and incorrect answer, ad the scores 1, - 5 are based on convention (by fiat) and are not based on psychometric theory)

The sum score

The score of the j^th test taker on the k^th item is indicated by X_jk. The conventional test score of the j^th test taker on a n-item test is the unweighed sum of his (or her) item scores:

Us_j = X_j1 + X_j2 +… + X_jn

It may be argued that items differ in imporance, and that they should be weighted differently.
The weighed sum score of the j^th item on an n-item test is:

Ws_j = w₁X_j1 + w₂X_j2 + … + w_nX_jn

w₁ is the weight assinged to the first item and so on.
A problem with the weighted sum score is that the weights have to be determined before the weighted sum score can be computed.
The weighted and unweighted sum score will be highly correlated if the weights of the weighted sum do not differ substantially form each other.

In modern testing, the unweighed sum score is mostly preferred.

The (un)weighted sum score combines the scores of n different test items.
A sum score only makes sense if the items predominantly reflect the same attribute.

Item writing based on a conceptual framework intends to write items that measure the attribute of interest.
Conceptually based item writing strives for a unidimensional test or a test of unidimensional subtests.
But, conceptually based item writing does not guarantee that the test is unidimensional.

The sum score also requires that each of the items is of sufficient quality.

The sum score is based on item scores that were assigned by fiat.

The observed test score distribution

S is used to indicate both the unweighed and weighted sum score when the analysis or use are the same for both types of score.
US and WS are used when the analysis are different.

If a test is administered to a population of N test takers, the frequency distribution of the observed test scores can be computed.
A frequency distribution can be characterized by different parameters.

• Mean
• Range
• Variance
• Standard deviation