What is psychometrics? - Chapter 1

Psychometrics play an important role in daily life. Whether you are a student, teacher, parent, psychologist, mathematician, or physicist; everyone has to deal with psychological tests. Psychological tests can have influenced your educational career, career, health, prosperity and so on. Psychometrics can even influence questions about life and death. For example in some countries, people with a severe cognitive impairment (significantly below average) can not receive the death penalty. But what is significantly below average? And how can we determine if the intelligence of an individual is below this limit? These kind of difficult questions can be answered through psychological tests. All in all, we can therefore say that psychometrics extends beyond psychological research. Psychometrics play a role in daily life, everyone has to deal with it.

How are psychological characteristics measured?

Psychologists use instruments to measure observable situations in the physical world. Sometimes psychologists measure a certain type of behavior solely because they are interested in that specific behavior in itself. But it is mainly behavioral scientists that measure human behavior to measure unobservable psychological attributes. We then identify a certain observable behavior and assume that this represents a certain unobservable psychological process, attribute or attitude. You must ensure that what you are measuring is also what you are aiming to measure. In social science, theoretical concepts such as short-term memory are often used to explain differences in human behavior. Psychologists call these theoretical concepts hypothetical constructs or latent variables. They are theoretical psychological properties, attributes, processes or states that cannot be observed directly. The procedures or actions with which they measure these hypothetical constructs are called operational definitions.

What is a psychological test?

According to Cronbach, a psychological test is a systematic procedure for comparing the behavior of two or more people. This test must meet three conditions:

1. the test must have behavioral samples;
2. the behavioral samples must be collected in a systematic manner and;
3. the purpose of the test must be to measure the behavior of two or more people (inter-individual differences).

It is also possible that we measure the behavior of an individual at different times, in which case we speak of intra-individual differences.    

Different types of testing

You can distinguish between different tests in the field of content. Which type of answer is used (open or closed) and which methods are used in the measurement. 

A distinction is also made between the different objectives of testing: the criterion referenced and norm referenced. Criterion referenced tests (also called domain referenced tests) are most common in situations where a statement must be made about a certain skill of a person. One pre-determined cutoff score is used to divide people into two groups: people whose score is higher than the cutoff score and people whose score is lower than the cutoff score.       

Norm-referenced tests are mainly used to compare the scores of a person with scores from the norm group. Nowadays, it is difficult to make a distinction between the benchmark tests and the benchmark tests. 

Another well-known distinction between tests is the distinction between the so-called speed tests (speed) and power tests. Speed tests are time-bound tests. It often happens that not all questions can be answered in a questionnaire. We look at how many questions you can answer correctly in the given time. Power tests are not time-bound tests. Here it is highly likely that one can answer all questions in a questionnaire. These questions often become more difficult and it is checked here how many questions people have answered correctly.        

Finally, the difference between reflective or effect indicators and formative or causal indicators is briefly discussed. An example of reflective / effect indicators are scores on intelligence or personality tests. These scores are usually considered as a reflection, or consequence, of a person's intelligence level. There are formative / causal indicators against this. Socio-economic status (SES) can, for example, be quantified by combining different indicators such as income, education level, and occupation. In this case, the indicators are not caused by SES. In contrast, the indicators are, in part, what SES defines. In this book the focus is on test scores, derived from reflective / effect indicators, which is typical of most tests and measurements in psychology.            

What is psychometrics?

With psychometrics, the focus is on the attributes of testing. Just as psychological tests are designed to measure psychological attributes of people, psychometry is the science where people are concerned with the attributes of psychological tests. There are three attributes that are important: the type of data (mainly scores), the reliability and validity of the psychological tests. Psychometrics is about the procedures with which test attributes are estimated and evaluated.

Psychometrics is based on two important foundations. The first foundation is the practice of psychological testing and measurement. The use of formal tests to measure skills of an individual (of any kind) goes back 2,000 years, or maybe even 4,000 years. Especially in the last 100 years there has been a huge increase in the number, type, and application of psychological tests. The second foundation is the development of static concepts and procedures. From the beginning of the nineteenth century, scientists became increasingly aware of the importance of static concepts and procedures. This led to an increase in knowledge about how quantitative data resulting from psychometric tests can be understood and analyzed. Pioneers in this field are Charles Spearman, Karl Pearson, and Francis Galton .       

Francis Galton was obsessed with measurements, mainly the so-called 'anthropometry'. Anthropometry contains measurements of human characteristics such as the size of the head, the length of an arm and the physical strength of the body. These properties have, according to Galton, psychological characteristics. He called these measurements of mental traits 'psychometrics'. Galton was primarily interested in the ways in which people differ. Galton's point of view was known as differential psychology , or the study of individual differences.   

Psychometrics is the collection of procedures that are used to measure variability in human behavior and subsequently combine these measurements into psychological phenomena. Psychometrics is a relatively young, but rapidly developing, scientific discipline. 

What are the challenges in psychometrics?

Many sciences are very similar but behavioral science has its challenges. 

One of those challenges is to try to identify and capture the important aspects of different types of human psychological attributes in a single number. 

A second challenge is participant reactivity. When participants know that they are being tested and they know why, this in itself influences the responses of the participant. For example, if a participant knows that it is being tested whether he or she is a racist but does not want this to appear in the test, this influences his or her reactions. Examples of participant reactivity are demand characteristics (influenced by what the participant thinks is the goal of the researcher), social desirability (responding to the wishes of the outside world), and malingering (wanting to leave a bad impression).       

A third challenge is that psychologists rely on so-called composite scores. This means that scores that have something in common are combined. For example, in a questionnaire with ten questions about extraversion, the scores on these questions are combined.  

A fourth challenge in psychological measurement is the problem of score sensitivity. Sensitivity refers to the possibility of a measurement to distinguish meaningful dimensions. For example, a psychologist wants to know if a patient's mood has changed. But if the psychologist uses an instrument that is not sensitive enough to measure small changes, the psychologist may miss important changes.  

The final challenge is the lack of attention to important information in psychometrics. Knowledge about psychometrics increases the chance of developments in testing. And test takers should at least use psychometrically good instruments. 

These challenges should make us aware of the data collected through psychological measurements. For example, we must be aware of the fact that participant reactivity can influence the reactions of the participants in a test.

What is the purpose of measuring in psychology? 

The theme that links the chapters of this book is related to the fact that the ability to identify and characterize psychological differences is the basis of all methods used to evaluate tests.

The purpose of measuring in psychology is to identify and quantify psychological differences that exist between people, over time or in different situations.