David Baxter PhD
Late Founder
Are mental health apps totally useless?
by Tom Chivers, UnHerd
Feb 5, 2020
The NHS1 is recommending cheap tech that's barely been tested
If you go to the App Store on your iPhone and type “depression” into the search bar, you’ll find dozens of apps purporting to screen for or help alleviate depression. One study claims there are 10,000 such apps, another some 350 or so.
Depression is a problem. The World Health Organisation calls it one of the “leading causes of disability” worldwide, and says there are 260 million sufferers — about one in every 30 people globally. The number of sufferers has gone up by about 50% since 1990, according to the Global Burden of Diseases, Injuries, and Risk Factors Study 2017, although since the global population has also increased by about the same percentage in that time, the prevalence hasn’t really changed. If we look at all mental health issues together, almost a billion people — more than one person in eight — are affected.
This is a serious burden, whether or not it is getting worse. But it goes largely untreated. The same WHO article says that between 76% and 85% of people with depression in developing countries receive no help (although they base this claim on data from 2007 so it may be outdated). In the developed West, of course, the figures are less — but still, here in Britain we are facing an NHS funding shortfall, and not everything that we would like to fund can be funded. It’s against that background that we see the proliferation of depression-treatment apps.
In theory, at least, an app doesn’t need a doctor, or an appointment; it’s fast and scalable and cheap. So if you can farm out a decent fraction of treatment to them, then you’ll provide a huge amount of benefit to a large swath of society, and save the NHS millions as you do so.
The trouble is that there doesn’t seem to be a great deal of evidence that they actually work.
A study from November last year looked at 293 mental health apps and found that only 3.41% — that is, ten of them (!) — even claimed (!) to have evidence showing their effectiveness. Of those 10, only three were backed by independent research, i.e. research not carried out by the people who made it.
The apps — things like Destressify, “a complete program for developing the practices that permanently rewire the brain” and give you “20% less stress in four weeks” in “as little as 10 minutes per day”, or MoodMission, an app which gives you short “missions” to improve your mood, such as running on the spot — take ideas from therapy, such as CBT or mindfulness, and offer it to you in app form. Those are in fact two of the 10 which have some research evidence, but even they are small, un-preregistered studies which (knowing what I know about the state of psychological research) I would be cautious to trust.
That doesn’t mean that the apps are worthless. You could argue that you don’t need to test your app if it’s just offering well-evidenced CBT therapy, although of course getting CBT therapy from an actual human may be very different from getting it from a screen, in the same way that French lessons from an actual French teacher are probably more effective than Duolingo. But only 30% of the apps tested even claimed to have experts helping with the design, meaning that “over two-thirds of apps for treating depression and/or anxiety were developed without any professional input”. Other studies have found similar shortages of expert input and evidence; this one found that 89% of apps have exactly no research evidence.
A more recent study, covered in this excellent post on the mental-health blog Mental Elf, found that even the evidence that exists is somewhat problematic. The study, a meta-analysis, looked at 18 earlier papers studying mental health apps, to see what percentage of users dropped out – they thought that would be a good proxy for how well-engaged the users were. The study found that more than a quarter of participants dropped out before the study was completed; and once they accounted for publication bias, that figure went up to almost half. Disturbingly, real apps fared no better than “placebo” ones. For comparison, only about 14% of people using “non-app controls”, e.g. face-to-face therapy or some other more traditional method, drop out.
It also found that the rate of drop-out wasn’t affected by what particular kind of therapy the app offered, but by whether or not there was an actual human involved. If some real therapist is looking at your app inputs and responding to them, you are less likely to give up on the whole process, which probably isn’t surprising. A psychologist I asked about this said “I would predict that these apps are only useful insofar as they connect people to actual services or actual people, or destigmatise seeking help.”
1 The NHS is the National Health Service in the UK.
by Tom Chivers, UnHerd
Feb 5, 2020
The NHS1 is recommending cheap tech that's barely been tested
If you go to the App Store on your iPhone and type “depression” into the search bar, you’ll find dozens of apps purporting to screen for or help alleviate depression. One study claims there are 10,000 such apps, another some 350 or so.
Depression is a problem. The World Health Organisation calls it one of the “leading causes of disability” worldwide, and says there are 260 million sufferers — about one in every 30 people globally. The number of sufferers has gone up by about 50% since 1990, according to the Global Burden of Diseases, Injuries, and Risk Factors Study 2017, although since the global population has also increased by about the same percentage in that time, the prevalence hasn’t really changed. If we look at all mental health issues together, almost a billion people — more than one person in eight — are affected.
This is a serious burden, whether or not it is getting worse. But it goes largely untreated. The same WHO article says that between 76% and 85% of people with depression in developing countries receive no help (although they base this claim on data from 2007 so it may be outdated). In the developed West, of course, the figures are less — but still, here in Britain we are facing an NHS funding shortfall, and not everything that we would like to fund can be funded. It’s against that background that we see the proliferation of depression-treatment apps.
In theory, at least, an app doesn’t need a doctor, or an appointment; it’s fast and scalable and cheap. So if you can farm out a decent fraction of treatment to them, then you’ll provide a huge amount of benefit to a large swath of society, and save the NHS millions as you do so.
The trouble is that there doesn’t seem to be a great deal of evidence that they actually work.
A study from November last year looked at 293 mental health apps and found that only 3.41% — that is, ten of them (!) — even claimed (!) to have evidence showing their effectiveness. Of those 10, only three were backed by independent research, i.e. research not carried out by the people who made it.
The apps — things like Destressify, “a complete program for developing the practices that permanently rewire the brain” and give you “20% less stress in four weeks” in “as little as 10 minutes per day”, or MoodMission, an app which gives you short “missions” to improve your mood, such as running on the spot — take ideas from therapy, such as CBT or mindfulness, and offer it to you in app form. Those are in fact two of the 10 which have some research evidence, but even they are small, un-preregistered studies which (knowing what I know about the state of psychological research) I would be cautious to trust.
That doesn’t mean that the apps are worthless. You could argue that you don’t need to test your app if it’s just offering well-evidenced CBT therapy, although of course getting CBT therapy from an actual human may be very different from getting it from a screen, in the same way that French lessons from an actual French teacher are probably more effective than Duolingo. But only 30% of the apps tested even claimed to have experts helping with the design, meaning that “over two-thirds of apps for treating depression and/or anxiety were developed without any professional input”. Other studies have found similar shortages of expert input and evidence; this one found that 89% of apps have exactly no research evidence.
A more recent study, covered in this excellent post on the mental-health blog Mental Elf, found that even the evidence that exists is somewhat problematic. The study, a meta-analysis, looked at 18 earlier papers studying mental health apps, to see what percentage of users dropped out – they thought that would be a good proxy for how well-engaged the users were. The study found that more than a quarter of participants dropped out before the study was completed; and once they accounted for publication bias, that figure went up to almost half. Disturbingly, real apps fared no better than “placebo” ones. For comparison, only about 14% of people using “non-app controls”, e.g. face-to-face therapy or some other more traditional method, drop out.
It also found that the rate of drop-out wasn’t affected by what particular kind of therapy the app offered, but by whether or not there was an actual human involved. If some real therapist is looking at your app inputs and responding to them, you are less likely to give up on the whole process, which probably isn’t surprising. A psychologist I asked about this said “I would predict that these apps are only useful insofar as they connect people to actual services or actual people, or destigmatise seeking help.”
1 The NHS is the National Health Service in the UK.
