Validity

Validity: if test-results can be interpreted in terms of the construct the test tries to measure.

The nomological network: the system of hypothetical relations around the construct.
This can be a part of the theory.

Forms of validity:

Impression-validity: an subjective judgment of the usability of an measurement-instrument on the base of the direct observable properties of the test-material.

Content-validity: the judgment about the representativeness of the observations, appointments, and questions for a certain purpose.

Criterium-validity: the (cor)relation between test-score and a psychological or social criterium.

• Predictive criterium-oriented validity: the criterium lies in the future. Criterium-performance aren’t measured at the same time as test-performance, but later.
• Concurrent criterium-oriented validity: the criterium is measured at the same time as the test. The criterium lays in the now or past.

Process-validity: the manner on which the response is established.

Construct-validity: A part of the similarities between the strictly formulated, hypothetical relations between the measured construct, and other constructs and otherwise empirical proved relations between instruments which should measure those constructs.

• The multitrait-multimethod approach of validation: a process in which with separated independent measurement-procedures at different traits is sought to construct-validity of a test.
• Convergence: a tests is cohesive with other measurements of the same construct or related constructs.
• Divergence: the test isn’t cohesive with other non-related constructs.

Reliability

Internal consistence-reliability: mutual cohesion of items that form a scale or sub-tests.

Repeated reliability: repeated measures with the same instrument

Local reliability: an impression of the reliability of the measurement within a certain wide of scores.

The homogeneity or consistency-reliability: the cohesion between the different (items) of a scale. With psychological measurement, it is assumed the the items are repeated, independent measures of a trait.

The reliability of the prediction: the repeatability of the prediction on a certain point of time.

Stability of the prediction: the repeatability of the prediction in the course of time.

Hits and misses

Base rate: the proportion of people in the population that possesses a particular trait, behaviour, characteristic, or attribute.c
Criterium-group: a, for the users-goal of the test, representative group of which all the members have the same criterium-behaviour and of which all the criterium-scores are known.

Hits

Hit: a correct classification

Hit rate: the proportion of people that an assessment tool accurately identifies as possessing or exhibiting a particular trait, ability, behaviour, or attribute

Misses

Miss: an incorrect classification

Miss rate: the proportion of people that an assessment tool inaccurately identifies as possessing or exhibiting a particular trait, ability, behaviour, or attributeReturn to investment: the ratio of benefits to costs.

Prediction-error or classification-error: the percentage wrongfully submitted cases by the test.

False positive: a specific type of miss whereby an assessment tool falsely indicates that the test-taker possesses or exhibits a particular trait, ability, behaviour, or attribute

False negative: a specific type of miss whereby an assessment tool falsely indicates that the test-taker does not possess or exhibit a particular trait, ability, behaviour, or attribute

Sensitivity and specificity

Sensitivity or predictive accuracy: the percentage rightfully submitted cases that actually has the trait (hits).

Specificity: the percentage of cases that is rightfully not submitted and that also doesn’t have the trait.

Predictive values

Positive predictive value (PPV): the percentage that is rightfully detected with the trait by the test of the total persons that the test said has the trait.

Negative predictive value (NPV): the percentage which the test rightfully said didn’t have the trait of the total of people the test said didn’t have the trait.

Prediction or description

Measurement-model: about what the constructor wants to measure

Structure-model: about what the constructor wants to predict

Prediction: first the test-score is established, then the criterium-score

Postdiction: first is the criterium-score established, then the test-score

Interpretation

The reflective interpretation: the measured attribute is conceptualized as the common cause of the observables

Formative interpretation: the measured attribute is seen as the common effect of the observables.

Utility

Utility of an instrument: the use of an instrument as becomes apparent from a costs-bate analysis.

Utility analysis: a family of techniques that entail a family of techniques that entail a cost-benefit analysis designed to yield information relevant to a decision about the usefulness and/or practical value of a tool of assessment.

Utility gain: an estimate of the benefit of using a particular test or selection method.

Norm-group: the group of people that forms the norm

Taylor-Russell tables: increase the base rate of successful performance that is associated with a particular level of criterion-related validity.

Naylor-Shine tables: tells us the likely average increase in criterion performance as a result of using a particular test or intervention: also provides selection ratio needed to achieve a particular increase in criterion performance.

The expectancy table or chart: tells us the likelihood that individuals who score within a given range on the predictor will perform successfully on the criterion.

Items

A domain: a wide area of more or less coherent properties.

Difficulty: the attribute of not being easily accomplished, solved, or comprehended.

Discrimination: the degree to which an item differentiates among people with higher levels or lower levels of the trait, ability, or whatever it is being measures.

Local dependence: items are all dependent on some factor that is different from what the test as a whole is measuring. Items are locally dependent if they are more related to each other than to the other items on the test.

Dichotomous or polytomous

Dichotomous test items: test items or questions that can be answered with only one or two alternative responses.

Polytomous test items: test items or questions with three or more alternative responses, where only one is scored as being consistent with a targated trait or other construct.

Inter and intra individual differences

Inter-individual differences: differences between people

Intra-individual differences: differences within people

Tools

Cross-validation: control of instability of outcomes

Method-variance: systematic variance as a result of the measurement-procedure with which the trait is measured.

Rasch model: an IRT model with very specific assumptions about the underlying distribution.

• Information in IRT: the precision of measurement.

Item characteristic curve (ICC), an item response curve, a category response curve, or an item trace line: the expression in graphic form of the probabilistic relationship between a test-taker’s response to a test item and that test-taker’s level on the latent construct being measured.

Assumptions

The unidimensionality assumption: the set of items measures a single continuous latent construct. This construct is referred to by the Greek letter theta (θ).

• Theta level: a reference to the degree of the underlying ability or trait that the test-taker is presumed to bring to the test.

The assumption of local independence: a) there is a systematic relationship between all of the test items and b) that relationship has to do with the theta level of the test-taker. When the assumption is met, it means that differences in responses to items are reflective of differences in the underlying trait or ability.

The assumption of monotonicity: the probability of endorsing or selecting an item response indicative of higher levels of theta should increase as the underlying level of theta increases.