FOTO Rehab Outcomes Blog

Clinical vs Statistical Significance

Written by Selena Horner | May 1, 2017 10:00:00 AM

 Quite frequently research focusing on comparing treatment interventions relies on a patient reported outcome measure. Sadly, it seems that some researchers haven't quite made the connection that the all important p-value determining change in scores isn't the best choice in determining significance. A p-value definitely helps in knowing statistical significance. A p-value doesn't bring meaning into whether the change in scores is relevant to the patient. Enter into the picture the idea of clinical significance. Minimal clinically important difference is what matters in determining if the change in score matters to the patient. 

Whenever you are reading research that includes a patient reported outcome measure, don't focus on p-value. Know the minimal clinically important difference when looking at change in scores and comparing groups. Is there a clinically relevant difference between the groups? (The main way to know is if the difference between scores is at least the minimal clinically important difference.) 

FOTO builds science into its reports. A risk adjustment process allows for predicting the outcome for the episode of care. Clinicians are also provided with the minimal clinically important difference to immediately know how much the score needs to change to be relevant to the patient. Very recently, FOTO included the patient acceptable symptom state as an option for clinicians to use.  

Below you will find a quick view of the abstract. 

Research Pearls: The Significance of Statistics and Perils of Pooling.Part 1: Clinical Versus Statistical Significance.

 

Abstract

Patient-reported outcomes (PROs) are increasingly being used in today's rapidly evolving health care environment. The value of care provision emphasizes the highest quality of care at the lowest cost. Quality is in the eye of the beholder, with different stakeholders prioritizing different components of the value equation. At the center of the discussion are the patients and their quantification of outcome via PROs. There are hundreds of different PRO questionnaires that may ascertain an individual's overall general health, quality of life, activity level, or determine a body part-, joint-, or disease-specific outcome. As providers and patients increasingly measure outcomes, there exists greater potential to identify significant differences across time points due to an intervention. In other words, if you compare groups enough, you are bound to eventually detect a significant difference. However, the characterization of significance is not purely dichotomous, as a statistically significant outcome may not be clinically relevant. Statistical significance is the direct result of a mathematical equation, irrelevant to the patient experience. In clinical research, despite detecting statistically significant pre- and post-treatment differences, patients may or may not be able to perceive those differences. Thresholds exist to delineate whether those differences are clinically important or relevant to patients. PROs are unique, with distinct parameters of clinical importance for each outcome score. This review highlights the most common PROs in clinical research and discusses the salient pearls and pitfalls. In particular, it stresses the difference between statistical and clinical relevance and the concepts of minimal clinically important difference and patient acceptable symptom state. Researchers and clinicians should consider clinical importance in addition to statistical significance when interpreting and reporting investigation results.

Arthroscopy. 2017 Apr 25. pii: S0749-8063(17)30181-0. doi: 10.1016/j.arthro.2017.01.053. [Epub ahead of print]