25: Misunderstanding p-values

Episode 25 · August 27th, 2016 · 55 mins 2 secs

About this Episode

P-values are universal, but do we really know what they mean? In this episode, Dan and James discuss a recent paper describing the failure to correctly interpret p-values in a sample of academic psychologists.

Some of the topics discussed:

  • Common p-value misconceptions
  • James tests Dan on his p-value knowledge
  • p-values vs. effect size
  • The problem of sample size with p-value interpretation
  • The Facebook mood manipulation study
  • Data peeking
  • Equivalent p-values do not represent equivalent results
  • Meta-analytical thinking
  • Using significance as a categorical factor
  • Statistical vs. clinical significance
  • Clinical trial registration and 'secondary outcome creep'
  • Dan and James answer listener questions
  • Science communicator vs. scientist
  • Grant titles and the 'Pub test'
  • NASA and social media


  • The article


  • Geoff Cumming's book (we got the name completely wrong - sorry Geoff!)


The story on research passing the 'pub' test


Real scientists


Facebook page


Twitter account


Support Everything Hertz

Episode Comments