fbpx

Liars, relationships and statistical reporting – Chad Cook (Pre-congress conference : Physio360 2020)

This blog is a brief summary of the key points discussed during this conference. In order to go more in depth on the topics discussed, I suggest you listen to it in its entirety. Please refer to the Ordre des Physiothérapeutes du Québec website to verify its online availability.

By Marc-Olivier Dubé

 

This lecture highlighted some limitations of the research or at the very least reiterated the importance of keeping a critical eye when reading the scientific literature.

Lies

From the outset, Dr. Cook discussed the many lies and subterfuge that can be encountered in the scientific world, despite the rigor and objectivity typically attributed to this field.

First of all, there are now more and more retracted publications, even in the biggest journals. Indeed, he mentioned that at the time of his presentation recording, there were already 33 articles on COVID-19 that had been retracted. What is a retraction: it is the act of withdrawing a publication when it is shown that it gave erroneous results (by mistake or intentionally).

Among the lies that were published and then retracted, the presenter cited the article suggesting that vaccines cause autism, the one mentioning that drinking 2 glasses of wine a day had a positive effect on heart health as well as studies supporting the prescription of Vioxx (NSAID) when it had a very high possibility of inducing negative side effects on cardiovascular health.

Since the research community is one that is very money driven, publishing many articles with interesting and innovative results is one way for a researcher to gain some financial security. Therefore, some in the community allow themselves to modify or embellish their results in order to ensure that their funding is maintained or that their notoriety is increased. Obviously, it is important not to fall into generalization and these dishonest individuals make up only a very small part of the scientific population.

Further reading suggested by Chad Cook:

Ioannidis JP. Why most published research findings are false. PLoS Med. 2005;2(8):e124. doi:10.1371/journal.pmed.0020124

Relationships

He then went on to compare human relationships to research by mentioning that they are both influenced by our emotions, although one should keep an objective eye on them. He used the example of a criminal’s parents who would be more inclined to blame themselves or the system rather than incriminating their own child. On the research front, clinicians and researchers may tend to be blinded by their therapeutic allegiance. This can lead to an unbalanced view and lead the individual to support an intervention deemed useful or not, as long as it corresponds to the values ​​advocated by his school of thought.

To support this point, he presented a study that found that 62% of clinical practice guideline creators had a conflict of interest or some relationship to the intervention or diagnosis they suggested.

Statistical reporting

In the statistics section, he discussed several elements that can influence the results and their interpretation. Indeed, the presenter mentioned that it is not always the most significant results that are presented and that the results of a search can be spinned to make them appear more important or more positive. Without listing all the elements that Dr Cook covered, here are some of them:

He also discussed a potential central explanation for patients’ improvement, regardless of the intervention used. These shared mechanisms between the different interventions evaluated in physiotherapy would mainly refer to the globality of individuals and their bio-psycho-social characteristics rather than to specific mechanisms that are generally attributed to the interventions.

In conclusion, he suggested 3 things to consider when reading a scientific article discussing the effectiveness of an intervention:

  1. If a study reports a larger effect size (multiple standard deviations) than other studies with the same methodology, you should question the article results.
  2. If the study results cannot be reproduced with a similar methodology by independent researchers, you should question the article results.
  3. If it sounds too good to be true and it is not what you encounter in your clinical practice, it is probably too good to be true.

Further reading suggested by Chad Cook:

Cook CE, George SZ, Keefe F. Different interventions, same outcomes? Here are four good reasons. BJSM. 2018;52(15):951-952. doi:10.1136/bjsports-2017-098978

Subscribe to our newsletter

Content according to your interests will be offered!

© 2021 Bia Formations Inc. All rights reserved.