Test development consists of several steps:

1. The construct of interest
   The latent variable (construct) that the test is supposed to measure has to be specified. Latent variables can vary in scope (1), content (2) and between educational and psychological variables (3).
2. The measurement mode
   The test has to specify how the latent variable will be measured. This includes three modes: self-performance mode (1), self-evaluation mode (2) and other-evaluation mode (3). Modes can be reactive or nonreactive. In a reactive mode, test takers can deliberately distort their construct value. In a non-reactive mode, test takers cannot deliberately distort their construct value.
3. The objectives of the test
   The test has to specify the objectives of the test. Tests can be used for practical and scientific purposes and objectives can be at the level of an individual test taker or at the level of a group of test takers. Objectives distinguish between description, diagnosis and decision making.
4. The population
   The target population, the set of persons to whom the test has to be applied has to be specified. This often includes inclusion and exclusion criteria.
5. The conceptual framework
   This is the theoretical framework on which the test is based. It is used to make conceptual distinctions and organize ideas.
6. The item response mode
   The test has to specify how test takers respond to the items. This can be a free- or constructed response mode, which includes short-answer items and essay items. It can also be a choice or selected response mode. It can make use of frequency or intensity response scales and of endorsement response scales.
7. The administration mode
   The test has to specify how the test is administered. Administration mode can be oral (1), paper and pencil (2), computerized (3) and computerized adaptive test administration (4). Computerized adaptive test administration adjusts the difficulty of the test according to the level of the test taker.

Response scales can be dichotomous (two ordered categories), partial ordinal-polytomous (more than two ordered categories) and ordinal-polytomous (completely ordered categories). There are several item-writing guidelines:

1. Focus on one relevant aspect
   This means that items should not test two aspects at the same time.
2. Use independent item content
   Items should be independent of each other, but this is not necessary if the questions are based on a reading passage.
3. Avoid overly specific and overly general content
   Overly specific and overly general content lead to ambiguity in the answers.
4. Avoid items that deliberately deceive test takers
   Items that distract test takers’ attention from the problem that they have to solve should be avoided.
5. Keep vocabulary simple for the population of test takers
   For native speakers, the items should not require the reading skill beyond that of a twelve year old.
6. Put item options vertically
7. Minimize reading time and avoid unnecessary information
   Unnecessary information obscures

Giving scores to different responses (e.g. agree = 3) is called scoring by fiat and has no theoretical justification. The observed test score is derived from the item scores by taking the unweighted or the weighted sum of the item scores. The construct score is derived from the item responses under the assumption of a latent variable response model.

The unweighted sum of item scores is the sum of all the item scores of a person j. It uses the following formula:

The weighted sum of item scores can be used because some items need to weigh heavier than others. This uses the following formula:

‘w’ denotes the weight of an item. The formula for the population mean of observed scores is the following:

It is the sum of the (un)weighted score divided by the number of test takers. ‘ε’ denotes expectation and ‘p’ denotes that the expectation is taken with respect to the population of test takers. The formula for the test variance is the following:’

It means the sum of (the test score of person j – the expected score) squared divided by the number of test takers. The expected score is the population mean of observed scores. The formula for the population standard deviation is the following:

It is the square root of the test variance. The population mean test score is estimated by using the sample mean and uses the following formula:

It is the sum of the observed test scores divided by the number of test takers. The item mean of item k uses the following formula:

It is the sum of item score on item k divided by the number of test takers. For dichotomous items, the item mean is equal to the proportion of that item. The formula for item variance of item k is the following formula:

It is the sum of the item score k minus the mean of item k squared. The item test variance uses the same formula, except it uses test scores of test taker j, instead of item scores. The item standard deviation is the square root of the item variance. Item variance for item k for dichotomous items can be calculated using the following formula:

The correlation between item k and item l can be calculated in the following way:

It is the sum of item score k minus the mean of item k times item score l minus the mean of item l divided by

An item score distribution can be described by location (1), dispersion (2) and shape (3). Item difficulty is a parameter in maximum performance tests. More test takers fail on more difficult items. Item attractiveness is a parameter in typical performance tests. More test takers choose attractive items. The item difficulty / item attractiveness is equal to the item mean.

Items with small variances do not contribute much to the overall variance. There is a danger of small variances due to floor / ceiling effects in Likert scales (e.g. items with low attractiveness). Large item correlations result in high reliability. Item discrimination refers to how well a given item can distinguish between people that differ on the underlying construct.

The item-test correlation is the correlation between the scores on a given item and the test scores. Items that discriminate well have a high item-test correlation. It uses the following formula:

It is the sum of item score k for test taker j minus the mean for item k times the test score for test taker j minus the mean of the test score divided by the square root of the sum of the item score k for test taker j minus the item mean for item k squared times the sum of the test score for test taker j minus the mean of the test score squared. In other words, it is the covariance between item k and the test score divided by the standard deviation of item k times the standard deviation of the test score.

The item-rest correlation is the correlation between the scores on a given item and the rest score, the score without that item. It is used because in the item-test correlation, correlation is biased upwards as you are correlating an item with itself. It uses the following formula:

It is the sum of the item score k for test taker j minus the mean for item k times the test score for test taker j without item k minus the mean of the test score without item k divided by the square root of the sum of the item score k minus for test taker j minus the mean for item k squared times the sum of the test score for test taker j without item k minus the mean of the test score without item k squared. In other words, it is the covariance between item k and the test score without that item (rest score) divided by the standard deviation of item k times the standard deviation of the test score without item k (rest score standard deviation).

The item-reliability index uses the following formula:

It is the correlation between item k and the test score times the standard deviation of item k. It uses the item-test correlation and not the item-rest