David Baxter PhD
Late Founder
How a New Blood Test for Depression is like Apple Recognition
The Carlat Psychiatry Blog
April 16, 2015
Four years ago I wrote a blog post about the MDDScore blood test for depression. That was before there were any peer-reviewed publications describing it. Now there are at least two. The latest came out a couple of months ago in the Journal of Clinical Psychiatry, and you can access the article, along with two interesting commentaries, for free.
While I won't go into the article in any detail, suffice it to say that the overall accuracy of the test for diagnosing depression was between 91% to 94%, depending on the group studied. Based on this, the authors report that the test "has excellent performance in confirming a diagnosis of MDD (major depressive disorder)."
The article is a classic example of the pitfalls of focusing on glitzy-sounding statistics while downplaying the actual clinical usefulness, which in this case is close to nil, as both of the Journal's commentators agreed.
I recently discussed the same problem in an article I wrote for CCPR about the NEBA EEG test for ADHD. Like the MDDScore, the NEBA test promises to aid in the diagnosis of a psychiatric illness. The NEBA's accuracy is high, with a positive predictive value for ADHD of 96% for kids, and 81% for adolescents. But no matter how accurate it is, the crucial question is whether it adds value above and beyond the standard psychiatric interview. Neither the MDDScore nor the NEBA do.
In my article, I used a hypothetical analogy of a new test to diagnose apples:
"Let’s imagine that there’s a new apple-recognizing device on the market called the “Apple Rec,” which uses various technologies to measure the wavelength of light reflected by an object, its mathematical curvature, etc. The manufacturer provides impressive data showing that the Apple Rec has 100% sensitivity and 100% specificity for diagnosing (recognizing) an object as being an apple. Given these dazzling statistics, would you buy the Apple Rec? No, because even though it’s exquisitely accurate, it provides you with no useful diagnostic information beyond what you can obtain by looking at the apple yourself. However, if the Apple Rec provided you with added value, you might consider it a good investment. For example, if, in addition to correctly recognizing it as an apple, it also calculated its sweetness and crispness, the Apple Rec suddenly becomes a useful tool, because these are qualities that you would otherwise struggle to ascertain."
The apple principal applies to diagnostic tests in psychiatry. Before you refer your patients to an expensive test that diagnoses ADHD, depression, or anything else, you need to make sure that it does something that you can’t easily do yourself.
The Carlat Psychiatry Blog
April 16, 2015
Four years ago I wrote a blog post about the MDDScore blood test for depression. That was before there were any peer-reviewed publications describing it. Now there are at least two. The latest came out a couple of months ago in the Journal of Clinical Psychiatry, and you can access the article, along with two interesting commentaries, for free.
While I won't go into the article in any detail, suffice it to say that the overall accuracy of the test for diagnosing depression was between 91% to 94%, depending on the group studied. Based on this, the authors report that the test "has excellent performance in confirming a diagnosis of MDD (major depressive disorder)."
The article is a classic example of the pitfalls of focusing on glitzy-sounding statistics while downplaying the actual clinical usefulness, which in this case is close to nil, as both of the Journal's commentators agreed.
I recently discussed the same problem in an article I wrote for CCPR about the NEBA EEG test for ADHD. Like the MDDScore, the NEBA test promises to aid in the diagnosis of a psychiatric illness. The NEBA's accuracy is high, with a positive predictive value for ADHD of 96% for kids, and 81% for adolescents. But no matter how accurate it is, the crucial question is whether it adds value above and beyond the standard psychiatric interview. Neither the MDDScore nor the NEBA do.
In my article, I used a hypothetical analogy of a new test to diagnose apples:
"Let’s imagine that there’s a new apple-recognizing device on the market called the “Apple Rec,” which uses various technologies to measure the wavelength of light reflected by an object, its mathematical curvature, etc. The manufacturer provides impressive data showing that the Apple Rec has 100% sensitivity and 100% specificity for diagnosing (recognizing) an object as being an apple. Given these dazzling statistics, would you buy the Apple Rec? No, because even though it’s exquisitely accurate, it provides you with no useful diagnostic information beyond what you can obtain by looking at the apple yourself. However, if the Apple Rec provided you with added value, you might consider it a good investment. For example, if, in addition to correctly recognizing it as an apple, it also calculated its sweetness and crispness, the Apple Rec suddenly becomes a useful tool, because these are qualities that you would otherwise struggle to ascertain."
The apple principal applies to diagnostic tests in psychiatry. Before you refer your patients to an expensive test that diagnoses ADHD, depression, or anything else, you need to make sure that it does something that you can’t easily do yourself.
