By Zoe Greenberg ’23
Staff Writer
Common guidance for researching scientific information on the internet is to avoid social media, be wary of news coverage and only trust studies published in peer-reviewed journals. However, while the peer review system increases the likelihood that information presented in studies will be accurate, peer-reviewed journals should not be considered infallible sources of information. The peer-review process is indeed flawed and allows for mistakes.
This is the premise of social psychologist Stuart Ritchie’s recently published book, “Science Fictions: How Fraud, Bias, Negligence, and Hype Undermine the Search for Truth.” Over the course of the book, Ritchie reveals that inaccurate scientific findings, such as falsified data and misleading statistics, routinely make it into peer-reviewed journals.
According to Ritchie and several other researchers who have documented corruption in the scientific community over the years, studies cannot be assumed to be credible simply because they were published in a peer-reviewed journal. When reading any scientific studies, readers should be wary of possible misinformation, whether or not an article has been peer-reviewed.
The first piece of advice Ritchie gives for accessing the accuracy of a paper is to look at the reputability of the authors’ employers and the outlet the study is published in. If a website looks unprofessional, it may be one of the many untrustworthy journals that exist online. Reputability alone is not proof enough, however — several well-known researchers and publications have been caught publishing false information.
Readers should also look for proof that participants registered to participate in a study, and for the data collected for the study. Ritchie recommends considering the sample size, as it is easier for researchers to claim that small effects are significant when pulling from a small pool. Larger sample sizes can increase the statistical validity of a study.
We should also think about both statistical significance and how the study stacks up against other established research, Ritchie says. If a study is so groundbreaking it seems implausible, that may be a sign that it should not be trusted.
As another metric, Ritchie recommends paying attention to the language used in a study. For example, researchers should only conclude causation if the study is an experiment. The most researchers can conclude from observational studies is a correlation; therefore, Ritchie says, a scientist who claims causation in an observational study is incorrect.
Other clues found in the language of a study may allude to bias, such as words that exaggerate the findings. Bias also can be detected by looking into the funding behind the study, to see if a social or political group might benefit from the results.
How plausible findings are — especially in studies that use human participants — and whether the study has been replicated by other researchers are two other metrics that Ritchie recommends readers use to measure the trustworthiness of studies. He suggests looking in news publications for quotes from third party scientists who have weighed in on the research and whether their judgment tends to be positive or negative.
Students and faculty at Mount Holyoke College depend on online scientific information for their own research. The Mount Holyoke News Health and Science section is committed to providing advice on how the community can assess the accuracy of research in addition to practicing these principles in our efforts to provide the most reliable information on the latest health and science news.