Evaluating Theories - summary of an article by Dennis & Kintsch

Critical thinking
Article: Dennis & Kintsch
Evaluating Theories

Introduction

A theory is a concise statement about how we believe the world to be.
Theories organize observations of the world and allow researchers to make predictions about what will happen in the future under certain conditions.

Science is about the testing of theories, and the data we collect as scientists should either implicitly or explicitly bear on theory.

The characteristics that lead a theory to be successful from those that make it truly useful:

  • Descriptive adequacy:
    Does the theory accord with the available data?
  • Precision and interpretability:
    Is the theory described in a sufficiently precise fashion that other theorists can interpret it easily and unambiguously?
  • Coherence and consistency:
    Are there logical flaws in the theory? Does each component of the theory seem to fit with the others in to a coherent whole? Is it consistent with theory in other domains?
  • Prediction and falsifiability:
    Is the theory formulated in such a way that critical tests can be conducted that could reasonably lead to the rejection of the theory?
  • Postdiction and explanation:
    Does the theory provide a genuine explanation of existing results?
  • Parsimony:
    Is the theory as simple as possible?
  • Originality:
    Is the theory new or is it essentially a restatement of an existing theory?
  • Breadth:
    Does the theory apply to a broad range of phenomena or is it restricted to a limited domain?
  • Usability:
    Does the theory have applied implications?
  • Rationality:
    Does the theory make claims about the architecture of mind that seem reasonable in the light of the environmental contingencies that have shaped or evolutionary theory?

Criteria on which to evaluate theories

Descriptive adequacy

The extent to which it accords with data.
In psychology, the most popular way of comparing a theory against data is null hypothesis significance testing.

Determining whether a theory is consistent with data is not always as straightforward as it may at first appear.

Some of the the subtleties involved in determining the extent to which a theory accords with data

  • Using null hypothesis significance testing, it is not possible to conclude that there is no difference. A proponent of a theory that predicts a list-length effect can always propose that a failure to find the difference was a consequence of lack of power of the experimental design.
  • Null hypothesis significance testing encourages a game of 20 questions with nature. A study proceeds by setting up one or more dichotomies. Rather than develop a cohesive theory and use hypothesis testing to evaluate it, researchers will generate an endless set of issues, each of which is only loosely coupled with previous work, and little cumulative progress will be achieved.

Precision and interpretability

When evaluating a theory, pay close attention to how well the constructs in the theory and the relationships between them are defined. Can you be confident that you could apply the theory in a related domain unambiguously?
Conversely, when articulating a theory, ask yourself what implicit assumption you may be making that may not be shared by your readers.

Coherence and consistency

Although it may seem obvious that theories should be free form logical flaws, it isn’t easy to spot these flaws, particularly when theories are presented in verbal form.
Another common problem that one must look for in evaluating theories is circularity.

Beyond ensuring that theories are free form logical flaws, it is important to ask how consistent a theory is, both with other theories within psychology and also with theory outside psychology.
Our current understanding remains our best guess of how the world operates, and so theories that are consistent with this approximation are more likely to endure.

Prediction and falsifiability

Ideally, one should be able to make unambiguous predictions based on the theory and conduct empirical tests that could potentially bring the theory into doubt.

Although falsification provides the most useful information in advancing scientific knowledge, it is sometimes the case that verifying predictions can increase our confidence in a theory.
However, not all predictions are equally diagnostic.

  • Predictions that seem to violate our intuitions and yet turn out to be the case provide more support for a theory than predictions that are unsurprising.

Postdiction and explanation

Prediction is possible under well-controlled laboratory conditions, but hardly ever under natural conditions.
Postdictive explanations are weaker than predictive explanations, but they are still explanations.

Prediction cannot be our goal.
Explanation after the fact (postdiction) is possible and worthwhile.

To predict we need to understand what is going on and have sufficient control over the relevant variables.
Postdiction also implies understanding, though not control.

The true goal of science is understanding, not necessarily prediction.

Postdiction can be based on formal theories as much as prediction.

Parsimony (Occam’s Razor)

The principle of parsimony states that theories should be as simple as possible.
In choosing between models of a given phenomenon, we would like to favor the one that fits the data well and is as simple as possible.

Breadth

Theories should attempt to be as broad as possible, while maintaining the other criteria such as descriptive adequacy and the ability to provide genuine explanations of phenomena.

Originality

Great care must be taken when comparing theories against each other, even when they are stated formally.

Usability

Good scientific theories should be useful in addressing societal problems.

Rationality

Does the theory make claims about the architecture of mind that seem reasonable in the light of the environmental contingencies that have shaped our evolutionary history?

Image

Access: 
Public

Image

Join WorldSupporter!
This content is used in:

WSRt, critical thinking - a summary of all articles needed in the second block of second year psychology at the uva

Search a summary

Image

 

 

Contributions: posts

Help other WorldSupporters with additions, improvements and tips

Add new contribution

CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Image CAPTCHA
Enter the characters shown in the image.

Image

Spotlight: topics

Check the related and most recent topics and summaries:
Institutions, jobs and organizations:
Activity abroad, study field of working area:
Countries and regions:
This content is also used in .....

Image

Check how to use summaries on WorldSupporter.org

Online access to all summaries, study notes en practice exams

How and why use WorldSupporter.org for your summaries and study assistance?

  • For free use of many of the summaries and study aids provided or collected by your fellow students.
  • For free use of many of the lecture and study group notes, exam questions and practice questions.
  • For use of all exclusive summaries and study assistance for those who are member with JoHo WorldSupporter with online access
  • For compiling your own materials and contributions with relevant study help
  • For sharing and finding relevant and interesting summaries, documents, notes, blogs, tips, videos, discussions, activities, recipes, side jobs and more.

Using and finding summaries, notes and practice exams on JoHo WorldSupporter

There are several ways to navigate the large amount of summaries, study notes en practice exams on JoHo WorldSupporter.

  1. Use the summaries home pages for your study or field of study
  2. Use the check and search pages for summaries and study aids by field of study, subject or faculty
  3. Use and follow your (study) organization
    • by using your own student organization as a starting point, and continuing to follow it, easily discover which study materials are relevant to you
    • this option is only available through partner organizations
  4. Check or follow authors or other WorldSupporters
  5. Use the menu above each page to go to the main theme pages for summaries
    • Theme pages can be found for international studies as well as Dutch studies

Do you want to share your summaries with JoHo WorldSupporter and its visitors?

Quicklinks to fields of study for summaries and study assistance

Main summaries home pages:

Main study fields:

Main study fields NL:

Follow the author: SanneA
Work for WorldSupporter

Image

JoHo can really use your help!  Check out the various student jobs here that match your studies, improve your competencies, strengthen your CV and contribute to a more tolerant world

Working for JoHo as a student in Leyden

Parttime werken voor JoHo

Statistics
3415