David Baxter PhD
Late Founder
Meta analysis debunks psychopathy-violence link
By Karen Franklin, Ph.D.
September 3, 2010
No clear winner among violence risk tools
If you are looking for the best tool to assess someone's risk for violence, the array may seem confusing. Lots of acronyms, lots of statistical data about AUC's (Areas Under the Curve) and the like. What do do?
No worries. As it turns out, they're pretty much interchangeable. That is the bottom-line finding of a groundbreaking meta-analytic study in the APA journal Psychological Bulletin by three academic researchers from the United Kingdom.
The University of Nottingham researchers used sophisticated statistical tools to meta-analyze multiple studies on the accuracy of nine leading violence risk assessment tools. All nine turned out to have similarly moderate predictive accuracy, with none clearly leading the pack. And none - the scholars warned - were sufficiently accurate for courts to rely upon them as a primary basis for decision-making in forensic cases requiring "a high level of predictive accuracy, such as preventive detention."
Widely touted PCL-R's "Factor 1" a bust
In a result with potentially momentous implications for forensic practitioners, the researchers found that Factor 1 of the Psychopathy Checklist-Revised (PCL-R) does not predict violence. As you know, Factor 1 purports to measure the core constellation of a psychopathic personality (superficial charm, manipulativeness, lack of empathy, etc.). When introduced in court, evidence of psychopathy has an enormously prejudicial impact on criminal offenders.
But, the PCL-R's much-ballyhooed ability to predict certain types of violence owes only to the instrument's second factor, according to the meta-analysis by researchers Min Yang, Steve Wong, and Jeremy Coid. And that's no surprise. After all, Factor 2 measures the criminogenic factors (criminality, irresponsibility, impulsivity, history of delinquency, etc.) that even a fifth-grader knows are bad signs for future law-abiding citizenship. It's like saying that a criminal past predicts a criminal future.
In my experience, the Factor 1 items - the ones purporting to measure an underlying personality profile - are the ones more likely to be inflated by some evaluators. That's because many of these items are pretty subjective. Glib? Superficially charming? If you don't like a guy - and/or he doesn't like you - you are more likely to rate these negative items as present. That's one of my hypotheses for the large evaluator differences and partisan allegiance effects found with the PCL-R in forensic practice.
Cumulatively, the emerging PCL-R findings beg the question:
Why introduce the Psychopathy Checklist in court if other violence risk tools work just as well, without the implicitly prejudicial effect of labeling someone as a "psychopath"?
Psychopathy evidence skyrocketing in juvenile cases
Despite (or perhaps because of, in some cases) its prejudicial impact, the construct of psychopathy is increasingly being introduced in court cases involving juveniles. It is often used to infer that a youth should get a longer sentence because he or she is dangerous and not amenable to treatment.
Source: Viljoen et al, Psychology, Public Policy, and Law (2010)
The first systematic review, published in the current issue of Psychology, Public Policy, and Law, found the use of psychopathy evidence against juveniles skyrocketing in both Canada and the United States. Psychopathy evidence is typically introduced when juveniles are being sentenced as adults and in sex offender commitment cases. It is also introduced in a variety of other cases, including ones involving disputed confessions, competency to stand trial, and criminal responsibility, report authors Jodi Viljoen, Emily MacDougall, Nathalie Gagnon, and Kevin Douglas.
In one egregious case showing how judges may improperly use evidence of psychopathy, a Canadian judge reasoned that a youth's "psychopathic device [sic] score" showed that under his "shy and unassuming" exterior lurked "a monster" that "at any time ... may well come alive." As a result, the judge sentenced this minor to an adult penitentiary.
Such inferences of unremitting danger and untreatability are improper. A large proportion of youths measured high in psychopathy score lower on psychopathy instruments once they mature. And so-called psychopathic youths are far from untreatable; in one recent study by Michael Caldwell and colleagues, after intensive treatment youths who scored high in psychopathy were actually less likely to recidivate than a comparison group in a juvenile jail.
"[T]he introduction of psychopathy evidence into juvenile forensic contexts has been somewhat rushed and premature at times," the authors conclude.
Have risk prediction tools hit the ceiling?
Researchers have been toiling for almost five decades to perfect risk prediction tools. Unfortunately, they keep running into an insurmountable obstacle: A large proportion of violence is situational. It's affected by environmental context, not just qualities internal to the individual. And not only that, but it is always extremely hard to predict a rare event.
Based on their meta-analytic findings, the UK researchers say maybe it's time to stop searching for the holy grail. Maybe we've reached the ceiling of predictive efficacy.
Dr. Wong, former Research Director at the Regional Psychiatric Centre in Saskatoon, Saskatchewan, studied psychopathy and high-risk offenders for 25 years and developed the Violent Risk Scale and the Violence Risk Scale - Sexual Offender Version before becoming a special professor at the Institute of Mental Health at the University of Nottingham. Dr. Yang is a professor of medical statistics with the Faculty of Medicine and Health Sciences at the University of Nottingham. And Dr. Coid, Director of the Forensic Psychiatry Research Unit, is principal investigator of the UK Home Office’s Prisoner Cohort Study and also studies the epidemiology of violent and criminal behavior at the population level.
Articles cited:
By Karen Franklin, Ph.D.
September 3, 2010
No clear winner among violence risk tools
If you are looking for the best tool to assess someone's risk for violence, the array may seem confusing. Lots of acronyms, lots of statistical data about AUC's (Areas Under the Curve) and the like. What do do?
No worries. As it turns out, they're pretty much interchangeable. That is the bottom-line finding of a groundbreaking meta-analytic study in the APA journal Psychological Bulletin by three academic researchers from the United Kingdom.
The University of Nottingham researchers used sophisticated statistical tools to meta-analyze multiple studies on the accuracy of nine leading violence risk assessment tools. All nine turned out to have similarly moderate predictive accuracy, with none clearly leading the pack. And none - the scholars warned - were sufficiently accurate for courts to rely upon them as a primary basis for decision-making in forensic cases requiring "a high level of predictive accuracy, such as preventive detention."
Widely touted PCL-R's "Factor 1" a bust
In a result with potentially momentous implications for forensic practitioners, the researchers found that Factor 1 of the Psychopathy Checklist-Revised (PCL-R) does not predict violence. As you know, Factor 1 purports to measure the core constellation of a psychopathic personality (superficial charm, manipulativeness, lack of empathy, etc.). When introduced in court, evidence of psychopathy has an enormously prejudicial impact on criminal offenders.
But, the PCL-R's much-ballyhooed ability to predict certain types of violence owes only to the instrument's second factor, according to the meta-analysis by researchers Min Yang, Steve Wong, and Jeremy Coid. And that's no surprise. After all, Factor 2 measures the criminogenic factors (criminality, irresponsibility, impulsivity, history of delinquency, etc.) that even a fifth-grader knows are bad signs for future law-abiding citizenship. It's like saying that a criminal past predicts a criminal future.
In my experience, the Factor 1 items - the ones purporting to measure an underlying personality profile - are the ones more likely to be inflated by some evaluators. That's because many of these items are pretty subjective. Glib? Superficially charming? If you don't like a guy - and/or he doesn't like you - you are more likely to rate these negative items as present. That's one of my hypotheses for the large evaluator differences and partisan allegiance effects found with the PCL-R in forensic practice.
Cumulatively, the emerging PCL-R findings beg the question:
Why introduce the Psychopathy Checklist in court if other violence risk tools work just as well, without the implicitly prejudicial effect of labeling someone as a "psychopath"?
Psychopathy evidence skyrocketing in juvenile cases
Despite (or perhaps because of, in some cases) its prejudicial impact, the construct of psychopathy is increasingly being introduced in court cases involving juveniles. It is often used to infer that a youth should get a longer sentence because he or she is dangerous and not amenable to treatment.
Source: Viljoen et al, Psychology, Public Policy, and Law (2010)
The first systematic review, published in the current issue of Psychology, Public Policy, and Law, found the use of psychopathy evidence against juveniles skyrocketing in both Canada and the United States. Psychopathy evidence is typically introduced when juveniles are being sentenced as adults and in sex offender commitment cases. It is also introduced in a variety of other cases, including ones involving disputed confessions, competency to stand trial, and criminal responsibility, report authors Jodi Viljoen, Emily MacDougall, Nathalie Gagnon, and Kevin Douglas.
In one egregious case showing how judges may improperly use evidence of psychopathy, a Canadian judge reasoned that a youth's "psychopathic device [sic] score" showed that under his "shy and unassuming" exterior lurked "a monster" that "at any time ... may well come alive." As a result, the judge sentenced this minor to an adult penitentiary.
Such inferences of unremitting danger and untreatability are improper. A large proportion of youths measured high in psychopathy score lower on psychopathy instruments once they mature. And so-called psychopathic youths are far from untreatable; in one recent study by Michael Caldwell and colleagues, after intensive treatment youths who scored high in psychopathy were actually less likely to recidivate than a comparison group in a juvenile jail.
"[T]he introduction of psychopathy evidence into juvenile forensic contexts has been somewhat rushed and premature at times," the authors conclude.
Have risk prediction tools hit the ceiling?
Researchers have been toiling for almost five decades to perfect risk prediction tools. Unfortunately, they keep running into an insurmountable obstacle: A large proportion of violence is situational. It's affected by environmental context, not just qualities internal to the individual. And not only that, but it is always extremely hard to predict a rare event.
Based on their meta-analytic findings, the UK researchers say maybe it's time to stop searching for the holy grail. Maybe we've reached the ceiling of predictive efficacy.
Violent behavior is the result of the individual interacting with the immediate environment. Although it may be possible to improve on our understanding and predicting what an individual may do in hypothetical situations, it will be much more difficult to predict the situation that an individual actually encounters in the open community. Even predicting violence within an institutional environment is difficult, where the assessor has much more information about that environment.
Instead, they say, it is time to turn our attentions to interventions that can reduce risk:
Building a better model of violence prediction should not be the sole aim of risk prediction research, which is just one link in the risk assessment-prediction-management triad that aims to achieve violence reduction and improved mental health…The risk, need and responsivity principles derived from the theory of the psychology of criminal conduct provide a useful theoretical framework for risk reduction intervention. Appropriate risk assessment can identify high-risk individuals in need of more intensive management and intervention…. Using tools with dynamic risk predictors to assess risk can identify appropriate changeable treatment targets linked to violence.
The studies included in the meta-analysis were from six countries: the United Kingdom (11), Canada (9), Sweden (3), the United States (3), Holland (2), and Germany (1). The instruments included the PCL-R, the PCL:SV, the HCR-20, the VRAG, the OGRS, the RM2000V, the LSI/LSI-R, the GSIR, and the VRS, as well as seven instrument subscales: PCL-R Factor 1 and Factor 2, the 10-item Historical subscale, the five-item Clinical subscale, and the five-item Risk Management subscale of the HCR-20; and the Static and Dynamic scales of the VRS.
Dr. Wong, former Research Director at the Regional Psychiatric Centre in Saskatoon, Saskatchewan, studied psychopathy and high-risk offenders for 25 years and developed the Violent Risk Scale and the Violence Risk Scale - Sexual Offender Version before becoming a special professor at the Institute of Mental Health at the University of Nottingham. Dr. Yang is a professor of medical statistics with the Faculty of Medicine and Health Sciences at the University of Nottingham. And Dr. Coid, Director of the Forensic Psychiatry Research Unit, is principal investigator of the UK Home Office’s Prisoner Cohort Study and also studies the epidemiology of violent and criminal behavior at the population level.
Articles cited:
- Yang, M., Wong, S.C.P., & Coid, J. (2010). The efficacy of violence prediction: A meta-analytic comparison of nine risk assessment tools. Psychological Bulletin, 136, 740-767. Correspondence to: Min Yang.
- Viljoen, J.L., MacDougall, E.A.M., Gagnon, N.C., & Douglas, K.S. (2010). Psychopathy evidence in legal proceedings involving adolescent offenders. Psychology, Public Policy, and Law, 16, 254-283. Correspondence to: Jodi Viljoen.
- Boccaccini, M.T., Turner, D.B., & Murrie, D.C. (2008). Do some evaluators report consistently higher or lower PCL-R scores than others? Findings from a statewide sample of Sexually Violent Predator evaluations. Psychology, Public Policy, and Law, 14, 262-283. Correspondence to: Marcus Boccaccini.
- Murrie, D.C., Boccaccini, M.T., Johnson, J.T., & Janke, C. (2008). Does interrater (dis)agreement on Psychopathy Checklist scores in Sexually Violent Predator trials suggest partisan allegiance in forensic evaluations? Law & Human Behavior, 32, 352-362. Correspondence to: Marcus Boccaccini.
