Classical analysis of item scores - a summary of chapter 6 of A conceptual introduction to psychometrics by G, J., Mellenbergh

A conceptual introduction to psychometrics
Chapter 6
Classical analysis of item scores

The conventional way of scoring items is by assigning ordinal numbers to the response categories.
Usually, these item scores are ordered with respect to the attribute that the item is assumed to measure. But, these assignment of these ordinal numbers lacks a theoretical justification.

Usually, the analysis of test scores is supplemented by an analysis of the item scores.

Item score distributions

The scores of a given item have a distribution in a population of N persons.

  • Location: the place of the scale where item scores are centered
  • Dispersion: the scatter of the item scores
  • Shape: the form of the distributions

Classical item difficulty and attractiveness

The location of the item score distribution is used to define the classical item difficulty (maximum performance tests) and classical item attractiveness (typical performance tests) concepts.

  • Classical item difficulty: a parameter that indicates the location of the item score distribution in a population of persons.
  • Classical item attractiveness: a parameter that indicates the location of the item score distribution in a population of persons.

The two definitions are the same.

Classical item difficulty and attractiveness are defined in a population of persons.
Population-dependent and may differ between populations.

The mean in mainly used for this.
The mean of a dichotomously scored item is called the item p-value.

Item score variance and standard deviation

The most common parameters that are used in classical item score analysis are the variance and the standard deviation of the item scores.

Items that have a small item score variance, have little effect on the test score variance.

The variance of dichotomous item scores is a function of the item p-value.
For a given sample size, the variance has its maximum value at p=.5.

Classical item discrimination

Location and dispersion parameters yield useful information on the items of a test.
But, these parameters do not indicate the extent to which an item contributes to the aim of a test to assess individual differences in the attribute that is measured by the test.

Classical item discrimination: a parameter that indicates the extent to which the item differentiates between the true test scores of a population of persons.
Defined in a population of persons, may vary between different populations.

The item-test and item-rest correlations

An appropriate index for discrimination between the true scores would be the product moment correlation between the item score and the true score in the population of persons.
Test taker j’s observed score is the estimator of his true score.
The population item-test correlation is estimated by the sample correlation.

Item-rest correlation: the product moment correlation between the item scores and the rest scores of the test, where the studied item is deleted.

The item reliability index

Item reliability index: the kth item reliability index is the product of the item-test correlation and item score standard deviation of the kth item in a population of persons.

Distractor analysis

A maximum performance item presents the test taker with a problem that has to be solved.
The response mode of this type of item is either the free(constructed) response or the choice (selected) response mode.
The most common response mode is multiple choice.

  • A multiple choice itme has one correct option and a number of distractors. If the test taker selects the correct option his item score is 1, if he selects a distractor his item score is 0.
    Classical item difficulty and discrimination indices are used for this type of dichotomous scoring, but the test takers’ distractor choices contain information on the multiple choice item.

Item distractor popularity

Classical difficulty of a dichotomously scored multiple-choice item: the proportion of persons of the population who selected the correct answer to the item.

Item distractor popularity: the proportion of persons of a population who selected the distractor.

The item difficulty and the item distractor popularities are estimated in a sample of test takers by the proportions of test takes of the sample who selected the correct answer and the distractors, respectively.

The distractor popularities yield information on the appropriateness of the distractors. An unpopulair distractor is selected by only a small proportion of test takers.
Apparently, most test takers know that the distractor is an incorrect answer to the item, which means that the item can do without this distractor.

Item distractor-item correlations

The usual way of scoring multiple-choice items is by assigning a 1 to the choice of the correct option, and assigning a 0 to the choice of a distractor.

Item distractor-rest correlations: the product moment correlations of the separate dichotomous correct answer/distractor variables and the rest score.

The item distractor-rest correlations yield detailed information on item quality.
A positive distractor-rest correlation indicates that the distractor tends to attract test takers who have lower true scores than test takers who selected the correct answer.
A negative distractor-rest correlatoin indicates that the distractor discriminates in the wrong direction because the distractor tends to attract test takers who have higher true scores than test takers who selected the correct answers.

The internal structure of the test

The observed test score is the unweighted or weighted sum of the item scores.

The basic idea is that in a population of test takers the items that measure the same attribute are more highly associated among each other than associated with items that measure another attribute.
The association between items can be assessed by different coefficients.

  • The conventional approach is to use a correlation coefficient to assess inter-item association. The internal structure of the test is studied by searching for clusters of items that are highly correlated within clusers and less correlated between clusters.

Analysis of inter-item product moment correlations

The conventional approach is to compute product moment correlations between the item scores, and to look for clusters of items that are highly correlated within the cluster and less correlated with items of other clusters.

The product moment correlation between the scores of two items in a population of persons is estimated by the sample product moment correlation.

Phi coefficient: the product moment correlation coefficient between two dichotomous variables.

The phi coefficient between two items can only reach a maximum of 1 if the p-values are equal.

Image

Access: 
Public

Image

Join WorldSupporter!
Search a summary

Image

 

 

Contributions: posts

Help other WorldSupporters with additions, improvements and tips

Add new contribution

CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Image CAPTCHA
Enter the characters shown in the image.

Image

Spotlight: topics

Check the related and most recent topics and summaries:
Activities abroad, study fields and working areas:
Countries and regions:

Image

Check how to use summaries on WorldSupporter.org

Online access to all summaries, study notes en practice exams

How and why use WorldSupporter.org for your summaries and study assistance?

  • For free use of many of the summaries and study aids provided or collected by your fellow students.
  • For free use of many of the lecture and study group notes, exam questions and practice questions.
  • For use of all exclusive summaries and study assistance for those who are member with JoHo WorldSupporter with online access
  • For compiling your own materials and contributions with relevant study help
  • For sharing and finding relevant and interesting summaries, documents, notes, blogs, tips, videos, discussions, activities, recipes, side jobs and more.

Using and finding summaries, notes and practice exams on JoHo WorldSupporter

There are several ways to navigate the large amount of summaries, study notes en practice exams on JoHo WorldSupporter.

  1. Use the summaries home pages for your study or field of study
  2. Use the check and search pages for summaries and study aids by field of study, subject or faculty
  3. Use and follow your (study) organization
    • by using your own student organization as a starting point, and continuing to follow it, easily discover which study materials are relevant to you
    • this option is only available through partner organizations
  4. Check or follow authors or other WorldSupporters
  5. Use the menu above each page to go to the main theme pages for summaries
    • Theme pages can be found for international studies as well as Dutch studies

Do you want to share your summaries with JoHo WorldSupporter and its visitors?

Quicklinks to fields of study for summaries and study assistance

Main summaries home pages:

Main study fields:

Main study fields NL:

Follow the author: SanneA
Work for WorldSupporter

Image

JoHo can really use your help!  Check out the various student jobs here that match your studies, improve your competencies, strengthen your CV and contribute to a more tolerant world

Working for JoHo as a student in Leyden

Parttime werken voor JoHo

Statistics
2530