Matt Gedye

Notes on Thinking Scientifically

I recently listened to episode 398 of Peter Attia’s podcast titled, Thinking Scientifically: why it’s hard, why it matters, and a practical toolkit. I found it so useful, and think that this concept is so important, that I summarised it for myself.

See below:

Thinking scientifically is difficult for a few reasons. Firstly, it’s a discipline not a destination. It takes continual practice, just like training a muscle, lest it get weaker over time from neglect. Additionally, the practice itself involves a type of thinking that is counter-intuitive to human psycho-biology.
So, if we have people in the world whose job it is to be a scientist, perform experiments, and report findings for the benefit of the community, why do non-scientists need to be able to think scientifically? The multi-billion dollar health and wellness industry is a pretty strong indicator that many are interested in this information. Unfortunately, these days, the sheer abundance and ease with which scientific findings can be circulated by anybody, makes for a perfect storm of misinterpretation, misunderstanding and distrust. Therefore, for those who are interested in their health and well-being, it pays to be better informed, but it costs us time and effort. Fortunately, as with any skill, it can be learned and improved upon over time and it’s absolutely worth it. Below are a series of three guidelines (adapted from the five outlined by Peter Attia in his podcast) that can be used to help improve scientific thinking for those who are interested in it.

1. Treat Certainty as a Cue to Slow Down and Notice When Identity Is Doing Your Thinking for You

“A person is smart. People are dumb, panicky, dangerous animals and you know it.”
- Kay (Men in Black, 1997)

A bit of mindfulness is required here at the outset, because it necessitates cultivating an awareness and treading cautiously around a claim you might immediately feel strongly about. Start by pausing and asking yourself, “why do I feel this way?” Your answers will likely fall into two broad buckets. First, a social basis for the belief, whereby a health claim may come with any of the following:
- Likes/engagements on social media.
- Friends and family tend to agree.
- The claim carries a strong, confident tone.
- We want the claim to be true.

The second bucket is the evidential basis which is categorised as making a decision based on:
- Seeing and understanding the data.
- Considered alternatives.

Most of us tend to fall into the first bucket, and that’s through no fault of our own. We are biologically hard-wired to agree with the masses - to let identity do our thinking for us. The basis for this lies in our evolutionary history. Humans are primates and therefore highly social animals. Fitting in with our tribe was integral to our survival (and our success as a species), while not doing so, often resulted in exile and a death sentence. Therefore, to consider a scientific claim from an evidential, rather than a social basis is not just hard, it’s counter-intuitive. But we need to remember that no group whether scientific, political, religious, etc. is always right, so it pays to question your certainty and even question your questioning.

“We get better at knowing what we don’t know and that awareness is worth a lot.”
- Peter Attia

2. Trust the Process, Not the Conclusion and Outsource Your Thinking Carefully

With increased mindfulness around a specific claim, we can begin to think about a process that led to a scientific claim, rather than fixating solely on the conclusion. First, resist the urge to simply ask, “is this true” or “do I agree?” Instead, we should ask a series of additional questions to better understand how the conclusion was reached. These can include, but are not limited to:
- How did this person arrive at this conclusion?
- What is the evidence and how strong is it?
- What alternatives were considered?
- What do the critics say and have they engaged with those criticisms?

“When you start asking those questions, something shifts. You stop evaluating claims as things to agree or disagree with, and you start evaluating them as products of a process, and the quality of that process tells you far more than the conclusion itself.”
- Peter Attia

However, this is where things can start to unravel. Because at this point we shift from taking something at face value, which is easy, to critical evaluation, which is more difficult. Furthermore, most of us are simply never going to read, or perhaps even have enough of an understanding of the nuance of the specifics of the method that produced the result in order to evaluate it.
Thankfully, by simply deciding that we want to better understand the process, we can still apply the principles of trusting the process (i.e asking the above questions) by outsourcing our thinking to experts. This, unsurprisingly, also involves repeated questioning.
First, we need to simply ask who the person is and what their professional background is. We’re more likely to trust that someone knows what they’re talking about, if they’ve been formally trained and spent many years working in a specific field, compared to someone on social media masquerading as an expert by regurgitating an interesting claim they learned on YouTube last week.
We should also be thinking about how they communicate their science and what their incentives are. Perceived confidence, charisma, heavy use of technical jargon and incentives that are purely financial or engagement based are red flags compared to someone who can cite studies, discuss limitations and identify key results within an enormous body of literature.
We should also consider how frequently someone engages in criticisms and whether they’ve changed their mind over time. The social cost of someone publicly changing their mind is enormous, but speaks volumes to whether they care more about getting it right than simply being right.
Finally, beware of experts over-extending into fields in which they’re not experts. It’s not uncommon for experts in one field to make bold (and sometimes outlandish) claims in another.

3. Don’t Confuse Criticism with Understanding

“A lie will travel halfway round the world while the truth is still putting on its shoes.”
- Mark Twain

Another type of ‘expert’ to watch out for is the ever-contrarian who only ever criticises a study. Anyone can pick holes in any type of study (e.g. “the sample size could have been larger” or “the follow-up period could have been longer”). But this affords no value. Synthesising evidence, however, by looking at the cumulative evidence of multiple studies and then asking “are they informative despite their limitations,” is a much more constructive way to move the needle towards greater understanding.
Interestingly, this process can sometimes result in previously held beliefs or dogmas being updated. Sadly, this is often used as an attack on science itself (like the debate on whether eggs are bad or good). But this is not a flaw or weakness in science. Rather, it is a sign that the very nature of producing science and the peer-review process, is working exactly as it should be.

To summarise, scientific thinking is hard. It’s counter-intuitive and it takes ongoing practice to get better at it. But we live in a world today that is rife with misinformation and therefore understanding how to scrutinise scientific findings ourselves and knowing who to trust on the internet, rather than taking advice at face value, is an important skill.

I highly recommend listening to the whole podcast. Attia does a much better job of explaining this than I do and he utilises many interesting stories and anecdotes that I chose not to summarise above.

P.S What I’m doing now.