Forensic Metascience, the GRIM test, and technology for checking papers | James Heathers

James Heathers is the Founder and Director of the Medical Evidence Project, a venture of The Center for Scientific Integrity. He aims to reduce medical harm and improve patient outcomes by identifying and publicizing errors and miscondcut in the medical literature. He uses forensic meta-analytical techniques to detect and deconstruct errors arising from low-quality science and fraudulent work in areas that involve large numbers of patients.

CONTACT RANDY:

metasciencematters@gmail.com

EPISODE LINKS:

The original GRIM test paper:

http://www1.psych.purdue.edu/~gfranci...

Machine-readable documents:

https://journals.sagepub.com/doi/10.1...

Chaos in the Brickyard: https://www.science.org/doi/10.1126/s...

Books:

The Man Who Only Loved Numbers:

https://en.wikipedia.org/wiki/The_Man...

The Emperor of All Maladies:

https://en.wikipedia.org/wiki/The_Emp...

OUTLINE:

0:00 - Introduction

5:27 - How James has been the last couple days, and his radio voice

6:40 - Writing the first book on Forensic Metascience

10:47 - James' PhD work on heart rate variability

14:12 - Starting to work on metascience

18:40 - The GRIM test

26:40 - Programmatically scaling statistical checks

33:25 - Centering quantitative results and peripheralizing narrative in scientific papers

39:36 - AI for metascience, unearthing data underneath narrative

48:11 - Pursuing cases of misconduct

52:45 - Founding and Directing the Medical Evidence Project

56:10 - Incentives for positive data, post-publication review, challenging the binary of positive and negative data

1:05:41 - Advice and resources for listeners

1:12:40 - Optimism over pessimism

1:16:58 - Outro