Evaluating theories - Dennis & Kintsch - 2008 - Article

Introduction

According to Popper all theories are false, so this should mean that evaluating theories is very straightforward. But some theories are more false than others and some theories help in the advance of scientific knowledge because of their characteristics. A theory is a concise statement about how we believe the word to be. These theories are used to organize the observations of the word and give researchers the opportunity to make predictions about what will happen in the future. In recent years, more interest has grown towards how theories can be tested with formal models.

Criteria on which to evaluate theories

Descriptive adequacy

To judge a theory on the extent to which it accords with data is probably the most important criterion. Different domains have different ways they employ data, so it’s an issue to understand what the “right” data is. But across all these domains, data is still very important and theories that are consistent with data are preferred. In psychology, using null hypothesis significance testing is a popular way to compare a theory against data. This involves creating two competing hypotheses, one would be true if the theory is correct and the other one would be false.

A difficulty with this hypothesis testing is that it is not possible to conclude that there is no difference. Another difficulty is that it is possible that researchers will generate an endless set of issues and achieve little cumulative progress. The advantage of the formal model is that they can precisely say how closely it approximates the data and give us additional information about the nature of the relationship between variables.

Precision and interpretability

For a theory it is important that it is described in a precise fashion and it can be interpreted easily and unambiguously. Many theories are often described vaguely, which makes it unclear how data would invalidate such theories.

Coherence and consistency

Another criterion for a good theory is its coherence and consistency. There shouldn’t be any logical flaw or circularity. It is also important to ask how consistent a theory is, both with theories within psychology and outside psychology.

Prediction and falsifiability

The formulation of the theory has to be in such way that critical tests can be conducted that could lead to the rejection of the theory. Even though falsification provides the useful information in moving forward in scientific knowledge, sometimes confirming predictions can increase our confidence in a theory. Surprising predictions, that do not seem to get along with our intuitions and still turn out to be the case, provide more support for a theory than unsurprising predictions.

Postdiction and explanation

The theory has to provide a genuine explanation of existing results. Postdictive explanations are not as strong as predictive explanations, but they are still explanations. Our explanations of behavior are often postdictive. In psychology there is no reason to think we will ever have all the information needed to make a precise prediction of his or her future acts. So only in limited circumstances prediction can be our goal.

Parsimony (Occam’s Razor)

Theories should be as simple as possible. Only the things needed to explain a phenomenon should be included. It is also important to consider the range of data sets the theory can fit.

Breadth

Theories should try to be as broad as possible, while still maintaining the other criteria that were discussed, such as descriptive adequacy.

Originality

Even though theories may look completely different, and are different in their broader implications, it can be impossible to differentiate them when it comes to a specific set of data. So we have to be very careful when we are comparing theories against each other.

Usability

Good scientific theories should be useful in addressing societal problems. The best research contributes to scientific understanding, while also fulfilling a societal need.

Rationality

The claims that are made with the theory should seem reasonable. The environment has adapted the cognitive system in many years through evolution. It is adapted to the way information is distributed in the environment. So the information should seem reasonable in this shaped environment. It is not easy to get the relevant environmental statistics, but when it is possible, it provides convincing support for a theory.

Conclusions

We have to take multiple considerations into account in the evolution of theories. Every theory is different and this creates different demands on the reader, so the weighting of the points discussed depends on many factors. For every case factors have to be weighted carefully.