Cohen on the science of psychological measurement” - Article summary

A utility analysis refers to a family of techniques that entail a cost-benefit analysis designed to yield information relevant to a decision about the usefulness and/or practical value of a tool of assessment. It is done in order to see whether the benefits of using a test outweigh the costs of that test. The objective of a utility analysis determines the required information (1) and the specific methods that have to be used (2).

One method of utility analysis is expectancy data. This is converting the test data to an expectancy table. It can provide a likelihood that a test taker will score within some interval of scores on a criterion measure. Taylor-Russel tables provide an estimate of the extent to which inclusion of a particular test in the selection system will improve selection. It gives an increase in base rate of successful performance that is associated with a particular level of criterion-related validity.

The selection ratio is a numerical value that reflects the relationship between the number of people to be hired and the number of people available to be hired. The base rate refers to the percentage of people hired under the existing system.

Top-down selection is a process of awarding available positions to applicants whereby the highest scorer is awarded the first position. A downside of top-down selection is that this may lead to unintended discriminatory effects.

Hit

A correct classification

Miss

An incorrect classification

Hit rate

The proportion of people that an assessment tool accurately identifies as possessing or exhibiting a particular trait, ability, behaviour or attribute.

Miss rate

The proportion of people that an assessment tool inaccurately describes as possessing or exhibiting a particular trait, ability, behaviour or attribute.

False positive

A specific type of miss whereby an assessment tool falsely indicates that the test taker possesses a trait.

False negative

A specific type of miss whereby an assessment tool falsely indicates that the test taker does not possess a trait.

One limitation of the Taylor-Russell tables is that the relationship between the predictor and the criterion must be linear. Naylor-Shine tables make use of the differences between the means of the selected and unselected groups to derive an index of what the test is adding to the established procedures. It determines the increase in average score on a criterion measure.

The Brogden-Cronbach-Gleser formula calculates the dollar amount of utility gain resulting from using a specific selection instrument. It uses the following formula:

utility gain=NTrxySDyZm-NC

N represents the number of applicants selected per year. T represents the average length of time in the position. Rxy represents the criterion-related validity coefficient for the given predictor and criterion. SDy represents the standard deviation of performance of employees. Zm represents the standardized mean score on the test for selected applicants. C represents the cost of the test for each applicant.

Productivity gain refers to an estimated increase in work output. It helps estimate the percent increase in output expected through the use of a particular test.

Instrument

Advantage

Disadvantage

Expectancy table or chart

Easy to use graphical display. It aids in decision making regarding a specific individual or a group of individuals scoring in a specific range.

Dichotomizes performance into successful and unsuccessful

Taylor-Russell table

It shows the relationship between selection ratio, criterion-related validity and existing base-rate. It aids decision making with regard to test use and recruitment to lower the selection ratio.

Relationship between predictor and criterion must be linear and it does not indicate the likely average increase in performance with use of test. It is difficult to identify a criterion value to separate successful and unsuccessful performance.

Naylor-Shine table

It provides information needed to use the BCG formula and does not dichotomize criterion performance. It can show average performance gain or show selection ratio needed for a particular performance gain.

It overestimates utility unless top-down selection is used. It can be difficult to interpret in practical terms as it is based on standardized units.

 

Image

Access: 
Public

Image

Join WorldSupporter!
This content is used in:

Scientific & Statistical Reasoning – Summary interim exam 5 (UNIVERSITY OF AMSTERDAM)

Scientific & Statistical Reasoning – Article summary (UNIVERSITY OF AMSTERDAM)

Search a summary

Image

 

 

Contributions: posts

Help other WorldSupporters with additions, improvements and tips

Add new contribution

CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Image CAPTCHA
Enter the characters shown in the image.

Image

Spotlight: topics

Check the related and most recent topics and summaries:
Institutions, jobs and organizations:
Activities abroad, study fields and working areas:
This content is also used in .....

Image

Check how to use summaries on WorldSupporter.org

Online access to all summaries, study notes en practice exams

How and why use WorldSupporter.org for your summaries and study assistance?

  • For free use of many of the summaries and study aids provided or collected by your fellow students.
  • For free use of many of the lecture and study group notes, exam questions and practice questions.
  • For use of all exclusive summaries and study assistance for those who are member with JoHo WorldSupporter with online access
  • For compiling your own materials and contributions with relevant study help
  • For sharing and finding relevant and interesting summaries, documents, notes, blogs, tips, videos, discussions, activities, recipes, side jobs and more.

Using and finding summaries, notes and practice exams on JoHo WorldSupporter

There are several ways to navigate the large amount of summaries, study notes en practice exams on JoHo WorldSupporter.

  1. Use the summaries home pages for your study or field of study
  2. Use the check and search pages for summaries and study aids by field of study, subject or faculty
  3. Use and follow your (study) organization
    • by using your own student organization as a starting point, and continuing to follow it, easily discover which study materials are relevant to you
    • this option is only available through partner organizations
  4. Check or follow authors or other WorldSupporters
  5. Use the menu above each page to go to the main theme pages for summaries
    • Theme pages can be found for international studies as well as Dutch studies

Do you want to share your summaries with JoHo WorldSupporter and its visitors?

Quicklinks to fields of study for summaries and study assistance

Main summaries home pages:

Main study fields:

Main study fields NL:

Follow the author: JesperN
Work for WorldSupporter

Image

JoHo can really use your help!  Check out the various student jobs here that match your studies, improve your competencies, strengthen your CV and contribute to a more tolerant world

Working for JoHo as a student in Leyden

Parttime werken voor JoHo

Statistics
2200