Brain scan diagnoses misunderstanding of diagnosis

There have been a lot of media stories in the past week about a study from the US military supposedly showing that a new form of brain scan can diagnose post-traumatic stress disorder (PTSD) in army veterans. Although interesting, the study doesn’t show any such thing and this is an example of a common misconception that regularly appears as a form of ‘new biological test diagnoses mental disorder’ story.

The study used a form of brain scan called MEG, essentially a high-tech form of EEG that picks up magnetic fluctuations from the brain’s electrical activity rather than the electrical signals themselves, and found that the coherence of signals across the resting brain was reliably different in vets diagnosed with PTSD by interview, compared to healthy people without mental illness.

Crucially, the scan didn’t pick out cases of PTSD among people with a range of mental illnesses, it just found a difference between people with PTSD and healthy people. But this is not a diagnosis, it’s just a difference.

If you’re not clear on this distinction, imagine that I claimed I found a new way of diagnosing malaria in under 2 seconds – I just measure body temperature and if the person has a fever, I decide they have malaria.

I hope you would point out that this is ridiculous, because people with flu can have fever, as can people with typhoid, mumps, dengue and so on.

My test would genuinely distinguish between people with malaria and healthy people, but in no way is it a diagnosis.

And this is the same situation with this new PTSD study. The difference could be due to levels of anxiety, common in many mental disorders, or to people who’ve experienced life threatening situations, regardless of whether they have PTSD or not, or any other factor I’ve not accounted for.

In other words, like with my fever example, it could be common to many different problems and not specific to the diagnosis I’m studying and I would need to make sure my method made a differential diagnosis – i.e. specifically ‘picked out’ the disorder among many – to be a useful diagnostic tool.

However, this latest PTSD story follows a common format in mental health news. I’ve lost count over how many reports I’ve read on how a ‘new test’ could diagnose schizophrenia based entirely on the fact that a study has found a difference between people with schizophrenia and people who don’t have it.

From reading these stories, I suspect it’s often the researchers who are at fault in describing their research.

When asked about to publicly justify their work, I suspect researchers often go for the easy “it could help diagnose the disorder”, which sounds immediately useful, as compared to the more truthful “it’s a small piece of knowledge in a very large area and we won’t know if it is reliable until it is replicated and if so, we may not fully understand its significance for many years to come. However, these small incremental advances are all useful even if they prove to be dead ends as they help us understand the problem from all angles”.

In this case, the researchers wrongly suggest in their scientific article that their findings “can be used for differential diagnosis” and so we can hardly blame the media for picking up on the hype.

So the next time you read a ‘new test diagnoses mental illness’ story, check to see whether it is genuinely picking out the problem among many others, or whether it’s just reporting a non-diagnostic difference.

Link to PubMed entry for new study.