Advertisement
Page 1 of 2 12 LastLast
Results 1 to 10 of 18

Validity of the Psychopathy Checklist

Antisocial Personality Disorder, Psychopathy, and Conduct Disorder - Personality Disorders: Psychopath guru blocks critical article Sunday, May 30, 2010 Will case affect credibility of PCL-R test in court? Despite recent ...

 

  1. #1

    Validity of the Psychopathy Checklist

    Psychopath guru blocks critical article
    Sunday, May 30, 2010

    Will case affect credibility of PCL-R test in court?

    Despite recent evidence that scores on the Psychopathy Checklist-Revised (PCL-R) vary widely in adversarial legal contexts depending on which party retained the evaluator, the test has become increasingly popular in forensic work. In Texas, indeed, Sexually Violent Predator (SVP) evaluators are required by statute to measure psychopathy; almost all use this test. It is not surprising that prosecutors find the PCL-R particularly attractive: Evidence of high psychopathy has a powerfully prejudicial impact on jurors deciding whether a capital case defendant or a convicted sex offender is at high risk for bad conduct in the future.

    But a current effort by the instrument's author, Robert Hare, to suppress publication of a critical article in a leading scientific journal may paradoxically reduce the credibility of the construct of psychopathy in forensic contexts.

    That's the opinion of two psychology-law leaders, psychologist Norman Poythress and attorney John Petrila of the University of South Florida (two authors of a leading forensic psychology text, Psychological Evaluations for the Courts), in a critical analysis of Dr. Hare's threat to sue the journal Psychological Assessment. The contested article, Is Criminal Behavior a Central Component of Psychopathy? Conceptual Directions for Resolving the Debate, is authored by prominent scholars Jennifer Skeem of UC Irvine and David Cooke of Glasgow University. The study remains unpublished.

    "[T]he threat of litigation constitutes a serious threat to academic freedom and potentially to scientific progress," write Poythress and Petrila in the current issue of the International Journal of Forensic Mental Health. "Academic freedom rests on the premise that advances in science can only occur if scholars are permitted to pursue free competition among ideas. This assumes that scholars have the liberty to do their work free from limitations imposed by political or religious pressure or by economic reprisals."

    According to Poythress and Petrila, after the critical article passed the peer-review process and was accepted for publication, Dr. Hare's lawyer sent a letter to the authors and the journal stating that Dr. Hare and his company would "have no choice but to seek financial damages from your publication and from the authors of the article, as well as a public retraction of the article" if it was published. The letter claimed that Skeem and Cooke's paper was "fraught with misrepresentations and other problems and a completely inaccurate summary of what amounts to [Hare's] life's work" and "deliberately fabricated or altered quotes of Dr. Hare, and substantially altered the sense of what Dr. Hare said in his previous publications."

    In general, defamation claims must prove that a defendant made a false and defamatory statement that harmed the plaintiff's reputation. Truth is an absolute defense. Critical opinions are also protected from defamation actions, as are "fair comments" on matters of public interest.

    In this case, the contents of Skeem and Cooke's contested article have not been made public. However, it is hard to see how critical analysis of a construct that is enjoying such unprecedented popularity and real-world impact would NOT be of public interest.

    Poythress and Petrila express concern that defamation claims against opposing researchers, while traditionally rare, may be becoming more common, leading to a potentially chilling effect on both individual researchers and the broader scientific community. Like so-called SLAPPS -- Strategic Lawsuits Against Public Participation -- used by corporations and other special interest groups to impede public participation, even meritless defamation lawsuits extract heavy penalties in terms of lost time and money and emotional distress.

    Judges have been critical of pretextual deployment of defamation lawsuits, Poythress and Petrila report; a judge in one case warned that "plaintiffs cannot, simply by filing suit and crying 'character assassination!,' silence those who hold divergent views, no matter how adverse those views may be to plaintiffs' interests. Scientific controversies must be settled by the methods of science rather than by the methods of litigation."

    Potential negative effects of defamation threats against scientific researchers include:


    1. Researchers avoid conducting critical research out of fear of lawsuits.
    2. Academics decline to serve as volunteer peer reviewers for academic journals due to loss of anonymity in defamation suits.
    3. Journal editors self-censor on controversial topics.


    As Poythress and Petrila conclude:

    Because publication of the article by Professors Skeem and Cooke has effectively been long delayed, if not ultimately suppressed, one clear impact of this threat to sue is that researchers who may have been willing to investigate alternative models of psychopathy that might have been derived from the Skeem and Cooke article are not able to do so, simply because the article is unavailable. Because science progresses, in part, both by confirming viable models and disconfirming nonviable ones, the suppression of information relevant to constructing candidate models for empirical evaluation can be viewed as impeding the progress of science….


    It seems clear from our review that such threats strike at the heart of the peer review process, may have a chilling effect on the values at the core of academic freedom, and may potentially impede the scientific testing of various theories, models and products. In our view it is far better to debate such matters in peer review journals rather than cut off debate through threats of litigation.

    In court, meanwhile, the effects of Dr. Hare's threat may prove paradoxical. Attorneys whose clients could be prejudiced by introduction of the Psychopathy Checklist may be able to discredit the instrument by pointing to the suppression of critical literature about the underlying construct of psychopathy.

    The full article is PCL-R Psychopathy: Threats to Sue, Peer Review, and Potential Implications for Science and Law. A Commentary, by Norman Poythress and John P. Petrila, in the current issue of the International Journal of Forensic Mental Health. The abstract is available HERE; the full article requires a subscription.

    Related:


    • "The Dark Side of Peer Review," by Stephen D. Hart, also in the current issue of the International Journal of Forensic Mental Health (abstract HERE
    • "Does interrater (dis)agreement on Psychopathy Checklist scores in Sexually Violent Predator trials suggest partisan allegiance in forensic evaluations?" by Murrie, D.C., Boccaccini, M.T., Johnson, J.T., & Janke, C. (2008). Law & Human Behavior, 32, 352-362 (abstract HERE

  2. Advertisement
  3. #2

    Re: Questions raised about the validity of the Psychopathy Checklist

    I have been arguing pretty much the same thing for years. The PCL does not measure psychopathy. It attempts to do so but fails. What it measures is criminal propensity, not the same thing at all but something probably more closely resembling the criteria for the DSM-IV-TR Antisocial Personality Disorder diagnosis.

    The problem in essence is this:


    1. The PCL-R breaks down into two factors. Factor 1, which attempts to measure the "core personality features" of psychopathy (basically along the lines of Cleckley's criteria); and Factor 2, which measures criminal behavior and criminal personality.
    2. Factor 1 is a weak factor in terms of predictive validity, interrater reliability, etc. Thus, the part of the PCL which tries to measure the essence of psychopathy is pretty much a failure.
    3. Factor 2, which is heavily influenced by historical data regarding past antisocial and criminal behavior, is much stronger, both in terms of predicting future criminal behavior (not surprisingly, given that all things being equal the best predictor of future behavior is past behavior) and in terms of interrater reliability.


    This is not exhaustive: There are other problems with the PCL-R. But this is the fundamental problem with the scale in a nutshell.

    Conclusion: The PCL-R is a measure of antisocial/criminal potential and/or "criminal personality". It is decidedly NOT a very good measure of psychopathy. Scoring high on the PCL-R does NOT make an individual a psychopath.

    Now, of course Dr. Hare is going to object to the publication of additional data pointing this out. He has made his reputation on the PCL-R and, additionally, has a financial interest in its propagation and popularity.

  4. #3

    Re: Questions raised about the validity of the Psychopathy Checklist

    Psychopath researcher threatens to sue critics
    by Vaughan, MindHacks
    May 31, 2010

    Robert Hare is a psychologist who studies psychopaths and is best known for developing the "Hare Psychopathy Checklist" or PCL-R, a standard diagnostic tool for assessing offenders. He is currently threatening to sue two psychologists who wrote an article critical of the theory underlying the checklist, as well as the academic journal, Psychological Assessment, that accepted the piece for publication after it was peer-reviewed.

    There's an account of the affair over at the excellent forensic psychology blog, In the News, who notes that the article was authored by respected researchers Jennifer Skeem and David Cooke and was titled Is Criminal Behavior a Central Component of Psychopathy? Conceptual Directions for Resolving the Debate. As a result of the legal threat the article has never come to light.

    The letter from Hare's lawyers apparently claimed that the he would:

    "have no choice but to seek financial damages from your publication and from the authors of the article, as well as a public retraction of the article" if it was published. The letter claimed that Skeem and Cooke's paper was "fraught with misrepresentations and other problems and a completely inaccurate summary of what amounts to [Hare's] life's work" and "deliberately fabricated or altered quotes of Dr. Hare, and substantially altered the sense of what Dr. Hare said in his previous publications."

    It's probably worth noting that the PCL-R is big business. At current prices, each assessor who uses the checklist needs their own copy of the manual ($123) and the rating booklet ($68.50) and each individual assessment requires an interview guide at $5 each and a scoring form at about $3 each.

    However, to use the assessment, each person needs to attend a training workshop at about $350 per person and workshops can easily involve 100 people at a time. Additionally, there is a follow-up correspondence course, price unspecified.

    Because the assessments are used in the legal system, it is important that no-one (like an opposing lawyer in court) can find fault in the process and attending the 'official' training from the PCL-R company is considered the gold standard.

    Recently, the affair has caught the attention of two lawyers and legal scholars who have just published their own analysis of the situation in the International Journal of Forensic Mental Health.

    They express regret that Hare has chosen to use legal threats to counter his critics rather than to refute any points he felt were unfair in print himself, but also note that his strategy may actually undermine the usefulness of the PCL-R in court as opposing lawyers "may attempt to discredit that testimony by arguing that the literature relevant to evaluating the PCL-R has been tainted".

  5. #4

    Re: Questions raised about the validity of the Psychopathy Checklist

    More coverage of psychopathy censorship controversy
    Tuesday, June 1, 2010

    The controversy over Robert Hare's attempt to block publication of a peer-reviewed article critical of his psychopathy construct is getting more attention since Sunday's blog post.

    Among the online coverage:




    Overwhelmingly, opinion is that Dr. Hare shot himself in the foot by threatening legal action against the researchers and the journal. Hopefully, this debacle will serve as a cautionary tale for others whose research undergoes critical scrutiny due to forensic or other public-policy implications.

  6. Re: Questions raised about the validity of the Psychopathy Checklist

    Hi Dr Baxter,

    Assuming that your critique of the PCL-R is correct (I am familiar with the PCL-R, but not the literature critiquing it) and that the PCL-R does not measure psychopathy very well, my question to you is what does measure psychopathy well?

    I appreciate that the PCL factor 2 list is a reasonable approximation of antisocial personality disorder, and that its useful and predictive of reoffending. I'm sure I've read somewhere that Hare claims that there are three personality types measured by his scale:

    High Factor 1's only: The successful psychopath, who is selfish and callous but behaviourally controlled
    High Factor 2's only: The antisocial individual, who is not actually psychopathic, but may be criminal and will likely meet the criteria for ASPD.
    High in both factors: Criminal psychopath, most likely incarcerated.

    I appreciate that other researchers have suggested a 3 factor analysis of the PCL - suggesting selfishness, narcissism and criminality.

    Just to repeat though, if factor 1 is invalid or lacks inter-rater reliability, what does measure psychopathy?

    Also, might factor 1 lack validity precisely because it is a personality scale, and psychopaths tend to lie about such things? In contrast, factor 2 is about lifestyle, for which the evidence can't be denied.

  7. #6

    Re: Questions raised about the validity of the Psychopathy Checklist

    Quote Originally Posted by Tom Michael View Post
    Assuming that your critique of the PCL-R is correct (I am familiar with the PCL-R, but not the literature critiquing it) and that the PCL-R does not measure psychopathy very well, my question to you is what does measure psychopathy well?
    I don't know of a psychometric test which even comes close. That was why there was initial excitement and enthusiasm about the PCL (similarly to the initial enthusiasm and susbsequent disappointment with the MCMI for personality disorders in general). It would be nice to have a valid instrument to "diagnose" psychopathy. To date, we don't have one.

    Quote Originally Posted by Tom Michael View Post
    I appreciate that the PCL factor 2 list is a reasonable approximation of antisocial personality disorder, and that its useful and predictive of reoffending. I'm sure I've read somewhere that Hare claims that there are three personality types measured by his scale:

    High Factor 1's only: The successful psychopath, who is selfish and callous but behaviourally controlled
    High Factor 2's only: The antisocial individual, who is not actually psychopathic, but may be criminal and will likely meet the criteria for ASPD.
    High in both factors: Criminal psychopath, most likely incarcerated.
    That's all rather circular, though. First, the validity of Factor 1 is, as I've stated, poor, so high Factor 1 scores don't really mean anything at all, or they may mean any number of other things besides psychopathy. Second, Factor 2 does measure criminality - not very surprising since it is based on the presence or absence of a history of criminal antisocial acts. But really what does it tell us that we don't already know from looking at criminal record?

    Quote Originally Posted by Tom Michael View Post
    Also, might factor 1 lack validity precisely because it is a personality scale, and psychopaths tend to lie about such things? In contrast, factor 2 is about lifestyle, for which the evidence can't be denied.
    I don't think lying is the problem - that's a problem for any scale and any population, but it is possible to construct scales with (a) validity scales to detect lying and (b) low transparency or face validity. Additionally, the PCL is a rating scale rather than an objective or self-report test.

    Rather, the problem with the PCL, especially Factor 1, is that it's a poorly constructed scale with high face validity but poor interrater reliability and poor predictive validity.

  8. Re: Questions raised about the validity of the Psychopathy Checklist

    Thanks for your reply David. My work is mainly to do with frontal lobe brain injury, but I have an interest in psychopathy due to the similarities between orbitofrontal brain injury and ASPD, and early amygdala injury and psychopathy. What I have initially liked about the PCL is that it did appear to distinguish those two types.

    If the PCL has low inter-rater reliability then it does fail as a measure, although that could be one reason that the company that market it insist on training the PCL raters (lack of experience is one reason for poor inter-rater reliability). I guess my second question to you would be this; If we lack any valid psychometric test or measure, can we really be sure that psychopathy exists?

    I'm reasonably certain that it does, as there are physiological markers, such as low skin conductance responses to violent images, and decreased frontal activity in brain imaging. Also, there do exist individuals who seem to be totally self centred and who have a callous disregard for others. These individuals will generally lie and make out that they are sincere and caring, although I'm sure I've read this in work by Hare and others, and it does tend to make one a little paranoid, seeing psychopaths everywhere.

    Can we really only detect psychopaths in a qualitative manner? Is there any theoretical obstacle to producing a valid rating scale? or just practical barriers such as the person being rated lying? I appreciate there are tests of malingering for psychometric tests, but that is a different type of deception than pretending you are a caring and sincere person.

    Kind regards

    Tom Michael

  9. #8

    Re: Questions raised about the validity of the Psychopathy Checklist

    I would not say that the failure of the PCL-R is any reason to assume that psychometric measures of psychopathy are not possible or that psychopathy does not exist. I have met a few classic psychopaths (i.e., Cleckely's criteria) in my career and they are unmistakable - like lokking into the eyes of a shark.

    But psychopathy does not equate to criminality. Not all psychopaths have a criminal rfecord and not all criminals are psychopaths (indeed, most criminals are not). I think using Cleckley's criteria is a good starting point but creating a psychometric test from that basis is obviously not as easy as it looks. I personally don't think that a 3 point rating scale founded on a subjective estimate or judgement of whether an individual has or does not have a specific characteristic is the way to go.

    Add to that the problem of diagnosing via cut-off (e.g., if the PCL-R score is equal to or greater than 30, he is a psychopath; if his score is 29, he is not a psychopath), and the lack of agreement even on what that cut-off should be (the research literature changes the cut-off seemingly on a whim to create better group sizes), and the fundamental concept of the PCL-R seems unsound to me.

    As for training, I can tell you that the PCL-R is in widespread use in many venues, especially correctional facilities, parole offices, and the like, where "training" generally involves reading the manual and some of the research literature.

  10. Re: Questions raised about the validity of the Psychopathy Checklist

    So then, if we agree that psychopathy is a valid construct, and that they can be identified, is there any way we can quantitively measure their traits with reliability? I'm sure that you have correctly identified those that you have met, but I guess that this must have been a qualitative process, using your knowledge and Cleckley's criteria.

    I think the dirty secret of a lot of psychometric measures (as opposed to tests) is that whether we assign a 0, 1 or 2 to a trait on a 3 point Likert scale effectively IS a qualitative judgement. I use the Dysexecutive questionnaire (a measure of symptoms of frontal lobe/dysexecutive brain injury), which has good inter-rater reliability amongst psychologists, but I know from personal experience that this reliability drops off dramatically if you as family members to use it to rate their relatives brain injury symptoms.

    What I think this means is that creating a psychopath trait measure is possible, but that unless the raters are well trained, its reliability is questionable. You mention that the PCL is used by many non psychologists, which could be one reason why inter-rater reliability is so poor. I strongly suspect that the factor 1 poor reliability is because it is rating internal personality traits, which can be hard to judge objectively as a third party unless you know that person very well.

    What criteria might you replace factor 1 with, in order to form a more reliable list? Might you be in a position to test and validate a new checklist, for use only by clinical or forensic psychologists?

  11. #10

    Re: Questions raised about the validity of the Psychopathy Checklist

    What I think this means is that creating a psychopath trait measure is possible, but that unless the raters are well trained, its reliability is questionable.
    I don't think a rating scale is the way I would go at all. I'd go with something a lot less subjective, and a lot less transparent, along the lines of the MMPI.

    You mention that the PCL is used by many non psychologists, which could be one reason why inter-rater reliability is so poor.
    No. I said that in my experience it is used by a lot of psychologists who purchase the test and read the manual but who do not have the training. I would add that I'm not sure how much difference training would make in any case. Ultimately, this kind of test is highly subject to rater bias.

    I strongly suspect that the factor 1 poor reliability is because it is rating internal personality traits, which can be hard to judge objectively as a third party unless you know that person very well.
    And see above: If you believe John Doe to be a psychopath, your ratings are biased in that direction before you even begin to complete a PCL.

Page 1 of 2 12 LastLast

Similar Threads

  1. Check your checklist - Your needs belong on top
    By David Baxter in forum Positive Psychology
    Replies: 1
    Last Post: February 12th, 2010, 08:58 AM
  2. Validity, Gullibility, and the Barnum Effect
    By David Baxter in forum Psychology, Psychiatry, and Mental Health
    Replies: 1
    Last Post: November 20th, 2009, 12:14 PM
  3. Assertiveness Checklist
    By Daniel in forum Family and Friends
    Replies: 0
    Last Post: July 17th, 2009, 07:47 PM
  4. Depression Checklist
    By Jazzey in forum Depression, Dysthymia, Seasonal Affective Disorder
    Replies: 0
    Last Post: December 22nd, 2008, 08:37 PM

Tags for this Thread

Bookmarks

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

PsychLinks is not responsible for the content
of posts or comments by forum members.
This website is certified by Health On the Net Foundation. Click to verify. Psychlinks complies with the HONcode standards
for health trustworthy information: verify here.


Additional Forum Web Design by PsychLinks and Wilder Tweedale Web Design
© All rights reserved.

HOSTING BY




PSYCHLINKS RECOMMENDS