A conceptual introduction to psychometrics by G, J., Mellenbergh - a summary
- 4606 keer gelezen
A conceptual introduction to psychometrics
Chapter 2
Developing maximum performance tests
Seven elements
The test developer must specify the latent variable of interest that has to be measured by the test.
Latent variable is a general term. The term construct is used when a subsantive interpretation is given of the latent variable.
The latent variable (construct) is assumed to effect test makers’ item responses and test scores.
Constructs can vary in many different ways.
A good way to start a test development project is to define the construct that has to be measured by the test.
This definition describes the construct of interest, and distinguished it from other, related, constructs.
Usually, the literature on the construct needs to be studies before the definition can be given. Frequently the definition can only be given when other elements of the test development plan are specified.
Different modes can be used to measure constructs.
The test developer must specify the objectives of the test. Tests are used for many different purposes.
Target population: the set of persons to whom the test has to be applied.
The test developer must define the target population, and must provide criteria for the inclusion and exclusion of persons.
A target population can be split into distinct subpopulations. The test developer must specify whether subpopulations need to be distinguished. And, if so, they need to define the subpopulations, and to provide criteria to include persons in subpopulations.
Test development starts with a definition or description of the construct that has to be measured by the test. But, the definition or description is usually not concrete enough to write test items.
A conceptual framework: gives the item writer a handle to write items.
In the literature, examples of conceptual frameworks are available.
The item response mode needs to be specified before item writing starts.
Dsitinction:
Free response items are further divided into:
Different types of choice modes are used in achievement and ability testing:
A test can be administered to test takers in different ways:
The responses to free- (constructed-) response items have to be grated by raters.
Important guidelines:
Standard practice is that item writers produce a set of concept items and pilot studies are done to test the quality of these concept items.
Generally, at least half of the concept items do not survive the pilot studies, and items that survive are usually revised several times.
Expert’s and test takers’ pilot studies need to be done for both free-response and multiple-choice items.
For free-response items pilot studies need to be done on the ratings of test takers’ responses to the items.
Expert’s pilots
The concept items have to be reviewed before they are included in a test.
Items are reviewed on their content, technical aspects, and sensitivity.
Each of the concept items is discussed by a panel of experts.
A good start for the discussion of a multiple-choice item is to look for distractors that panel members could defend as (partly) correct answers.
The reviewing of the items yields qualitative information that is used to rewrite items or to remove concept items that cannot be repaired.
Revised items should be reviewed again by experts until further rewriting is not needed.
The sensitivity of items also needs to be reviewed.
Usually, the panel for the sensitivity review of the items consists of person not on the panel reviewing the content and technical aspects of the items.
The sensitivity review panel is composed of members of different groups.
The panel has to be trained to detect aspects of the items and the tests that may be sensitive to subpopulations.
The sensitivity review provides qualitative information that also could lead to rewriting or removal of concept items.
Test takers’ pilots
The concept items are individually administered to a small group test takers from the population of interest.
Each of the test takers is interviewed on their thinking while working on an item.
Two versions of the interview can be applied
Protocols of the interviews are made and the information is used to rewrite or remove concept items.
The concept items that survived the pilot studies are used to compile a concept version of the test that includes instructions for the test takers.
Usually, the instruction contains some example items that test takers have to answer to ensure that they understand the test instructions.
The concept test may consist of a number of subtest that measure different aspects of the ability or achievement.
The conventional way of assembling a maximum performance test is to start with easy items and to end with difficult items.
The concept test is submitted o a group of experts.
The group can be the same as the group that was used in the experts’ pilot study on item quality. The group has expertise in:
The experts evaluate two different properties of the concept test.
The comments of the experts are used to compile the first draft of the test.
The first draft of the test is administered in a try-out to a sample of at least 200 test takers from the population of interest.
Join with a free account for more service, or become a member for full access to exclusives and extra support of WorldSupporter >>
This is a summary of the book A conceptual introduction to psychometrics by G, J., Mellenbergh. The summary contains chapter 1 to 6, and focusus on developing psychological tests.
The first chapter of this summary is for free, but to support worldsupporter and Joho,
...There are several ways to navigate the large amount of summaries, study notes en practice exams on JoHo WorldSupporter.
Do you want to share your summaries with JoHo WorldSupporter and its visitors?
Main summaries home pages:
Main study fields:
Business organization and economics, Communication & Marketing, Education & Pedagogic Sciences, International Relations and Politics, IT and Technology, Law & Administration, Medicine & Health Care, Nature & Environmental Sciences, Psychology and behavioral sciences, Science and academic Research, Society & Culture, Tourisme & Sports
Main study fields NL:
JoHo can really use your help! Check out the various student jobs here that match your studies, improve your competencies, strengthen your CV and contribute to a more tolerant world
2639 | 1 |
Add new contribution