Kraemer et al. (2003). Measures of clinical significance.” - Article summary

There are three important questions when assessing the relationship between variables:

  1. Statistical significance
  2. Effect size
  3. Practical significance

The practical significance (i.e. clinical significance) can be assessed by considering factors such as clinical benefit (1), cost (2) and side effects (3). In order to assess the practical significance, the strength of the association (i.e. ‘r’) (1), magnitude of the difference between the difference between treatment and comparison (i.e. ‘d’) (2) and measures of risk potency (3) can be used.  Risk potency can be assessed using the odds ratio (1), risk ratio (2), relative risk reduction (3), risk difference (4) and number needed to treat (5).

The p-value refers to the probability that the found outcome or more extreme is found, given that the null hypothesis is true. It is possible to have statistical significance by chance and outcomes with lower p-values are sometimes interpreted as having stronger effect sizes. A non-significant result also does not tell us anything about the truth of the null hypothesis.

There are several different effect size measures:

  1. The ‘r’ family
    This is expressing effect size in terms of strength of association (e.g. correlation).
  2. The ‘d’ family
    This is an effect size that can be used when the independent variable is binary and the dependent variable is ordered. It can be computed by subtracting the mean of the second group from the mean of the first group and divide it by the pooled standard deviation of both groups. The effect size ranges from minus to plus infinity.
  3. Measures of risk potency
    This is an effect size that can be used when both the dependent and independent variables are binary. The odds ratio and risk ratio vary from 0 to infinity and 1 indicates no effect. Risk relative reduction and risk difference vary between -1 and 1 with 0 indicating no effect. Number needed to treat (NNT) ranges from 1 to plus infinity with very large values indicating no treatment effect.
  4. AUC
    This is an effect size that can be used when the independent variable is binary but the dependent variable is either binary or ordered. It ranges from 0% to 100% with 50% indicating no effect.

This standard is relative and it should be noted that ‘larger than typical’ should be used rather than ‘large’. However, it might be best to find information about typical effect sizes in the context of a particular research field.

The disadvantages to the ‘d’ and the ‘r’ as measures of clinical significance are that they are relatively abstract (1), they were not intended as measures of clinical significance (2) and they are not readily interpretable in terms of how much individuals are affected by treatment (3).

There is no consensus on the externally provided standards for the clinical significance of treatments. Clinical significance could be defined as a change to normal functioning due Phi is not a good measure for clinical significance.

When continuous data are dichotomized (e.g. success or failure), there is a loss of information (1), it can result in arbitrary effect size indexes (2) and it can result in inconsistent effect size indexes (3), mostly due to different choices for the cut-off point of failure.

A limitation of the odds ratio is that the magnitude of the odds ratio may approach infinity if the outcome is very rare or very common. This can also happen if the outcome is near random. The magnitude of the cut-off point varies strongly with the choice of cut-point. Therefore, the interpretation of the odds-ratio is arbitrary and very difficult.

The risk ratio is obtained by dividing the failure or success rate of the comparison group by the failure or success rate of the treatment group.  The choice of cut-off point changes the magnitude of the risk ratio which makes it difficult to interpret.

Relative risk reduction is computed by subtracting the treatment group failure rate from the comparison group failure rate and dividing it by the comparison group failure rate or by using the success rates. There are no agreed upon standards for assessing the magnitude of this.

Risk difference is computed by subtracting the percentage of failures in the treatment group from the percentage of failures in the comparison group or by using the successes. The risk difference is often near zero when the odds ratio and the risk ratios are very large.

Number needed to treat refers to the number of patients who must be treated to generate one more success or one less failure than would have resulted if all people had been given the comparison treatment. In risk studies, NNT is the number who would need to be exposed to the risk factor to generate one more case than if none had been exposed. NNT can only be interpreted relative to the comparison.

AUC represents the probability that a randomly selected subject in the treatment group has a better result than one in the comparison group. AUC can be computed by using clinical judgement alone.

 

Image

Access: 
Public

Image

Join WorldSupporter!
This content is used in:

Evidence-based Clinical Practice – Full course summary (UNIVERSITY OF AMSTERDAM)

Evidence-based Clinical Practice – Article overview (UNIVERSITY OF AMSTERDAM)

Search a summary

Image

 

 

Contributions: posts

Help other WorldSupporters with additions, improvements and tips

Add new contribution

CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Image CAPTCHA
Enter the characters shown in the image.

Image

Spotlight: topics

Check the related and most recent topics and summaries:
Institutions, jobs and organizations:
Activity abroad, study field of working area:
This content is also used in .....

Image

Check how to use summaries on WorldSupporter.org

Online access to all summaries, study notes en practice exams

How and why use WorldSupporter.org for your summaries and study assistance?

  • For free use of many of the summaries and study aids provided or collected by your fellow students.
  • For free use of many of the lecture and study group notes, exam questions and practice questions.
  • For use of all exclusive summaries and study assistance for those who are member with JoHo WorldSupporter with online access
  • For compiling your own materials and contributions with relevant study help
  • For sharing and finding relevant and interesting summaries, documents, notes, blogs, tips, videos, discussions, activities, recipes, side jobs and more.

Using and finding summaries, notes and practice exams on JoHo WorldSupporter

There are several ways to navigate the large amount of summaries, study notes en practice exams on JoHo WorldSupporter.

  1. Use the summaries home pages for your study or field of study
  2. Use the check and search pages for summaries and study aids by field of study, subject or faculty
  3. Use and follow your (study) organization
    • by using your own student organization as a starting point, and continuing to follow it, easily discover which study materials are relevant to you
    • this option is only available through partner organizations
  4. Check or follow authors or other WorldSupporters
  5. Use the menu above each page to go to the main theme pages for summaries
    • Theme pages can be found for international studies as well as Dutch studies

Do you want to share your summaries with JoHo WorldSupporter and its visitors?

Quicklinks to fields of study for summaries and study assistance

Main summaries home pages:

Main study fields:

Main study fields NL:

Follow the author: JesperN
Work for WorldSupporter

Image

JoHo can really use your help!  Check out the various student jobs here that match your studies, improve your competencies, strengthen your CV and contribute to a more tolerant world

Working for JoHo as a student in Leyden

Parttime werken voor JoHo

Statistics
2207