Abstract
The American Academy of Psychiatry and the Law (AAPL) was organized in 1969, in large part through the efforts of Dr. Jonas Rappeport. The founders of AAPL emphasized that an important purpose of the organization was to advance knowledge in the area of psychiatry and the law. The science of forensic psychiatry has since been vigorously debated. In 2005, Congress enacted a statute authorizing the National Academy of Sciences (NAS) to conduct a study on the state of the forensic sciences in the United States. As a result of this legislation, a forensic science committee was formed, and the report, “Strengthening Forensic Science in the United States: A Path Forward,” was produced, emphasizing the need for research in the forensic disciplines, particularly those that rely on more subjective assessments. The committee also identified two important factors relevant to standards of evidence admissibility: the scientific methodology used and the impact of bias on the interpretation of data. In this article, I apply the NAS committee's findings to the field of forensic psychiatry, with specific recommendations to assist educators in achieving more objective assessment methodologies, critical in forensic education and the search for truth.
On March 12, 1968, Dr. Jonas Rappeport mailed a letter that he had written to directors of forensic psychiatry fellowship programs to request a meeting. Two months later, several forensic fellowship directors met in the Andover Room at the Boston Sheraton Hotel. Although the meeting lasted for less than two hours, the results were to endure for years to come. On May 5, 1969, 13 forensic psychiatrists came together at the Bel Harbor Hotel in Miami Beach to organize an association of forensic psychiatrists. According to the minutes, the mission of the newly formed group was defined as follows:
The purpose of this organization would be to advance the body of knowledge in the area of psychiatry and the law, to act as an agency of exchange of information, knowledge and ideas between members and at the interface between psychiatry and the law, and to indicate and study where contributions to the legal and penal system could be made by the behavioral sciences. It was emphasized that training of psychiatrists in how better to testify in court was not the major function of group and membership, which is suggest [sic] to be by invitation, was to include only those who shared the goals outlined above [Ref. 1, p I].
The group chose the name, the American Academy of Psychiatry and the Law (AAPL), for this newly formed educational organization, and the seeds of future forensic psychiatric education were thus planted.1
The first AAPL Annual Meeting was held in Baltimore at the Friendship International Hotel on Sunday, November 16, 1969. The two panels discussed topics related to the diagnosis and disposition of the dangerous offender. The 10th Annual Meeting in 1979 ended the decade much as it had begun, with the meeting held in Baltimore and a lively debate focusing on whether psychiatrists could determine future dangerousness.1 AAPL's educational mission expanded in October 1980, when Dr. Richard Ciccone, in his role as Chair of the AAPL Education Committee, established the AAPL Board Review Course and Update. The first session was held at the Del Coronado Hotel in San Diego in 1980 (Ciccone R, personal communication, September 24, 2012). The success of the AAPL course established by Dr. Ciccone has continued under the leadership of Dr. Phillip Resnick and is currently offered over a three-day period before the annual meeting.
A review of the history shows that a seminal moment for AAPL and its members occurred on October 22, 1982, at the 13th Annual Meeting in New York. In his keynote address, “The Ethical Boundaries of Forensic Psychiatry: A View From the Ivory Tower, ” Dr. Alan Stone, Professor of Law and Psychiatry at Harvard University, expressed his concerns regarding the ethics of psychiatrists' testifying in the courtroom and commented that these concerns kept him out of legal settings. He raised important questions about the lack of any clear ethics guidelines for forensic psychiatrists who testify and he wondered whether a forensic psychiatrist has any truth to tell in court. A written version of Stone's 1982 presentation was published in The Bulletin of the American Academy of Psychiatry and the Law in 1984, and the issue was devoted to responding to his views of psychiatric testimony in court.2
Subsequently, Dr. Paul Appelbaum formulated his theory of ethics as related to forensic psychiatry, which he presented as the focus of his 1996 AAPL presidential address. His theory has been referenced as the Standard Position, or truth-oriented stance. He outlined two broad principles governing the ethics of forensic work: first, the forensic psychiatrist has an obligation for both subjective and objective truth-telling. In other words, not only must the psychiatrist present information that he believes to be true, but the information presented also must accurately reflect the current scientific evidence and consensus of the field. Second, a forensic psychiatrist's search for the truth must be balanced against the evaluee's rights and dignity.3 Numerous forensic psychiatrists, in addition to highly regarded ethicists, have made meaningful contributions to the field of ethics and forensic psychiatry. I highlight Appelbaum's Standard Position because of its particular relevance to the subsequent points that this article addresses.
In 2008, The Journal republished Stone's 1984 article verbatim. In it, he commented on a view put forth by another forensic psychiatrist, his friend and colleague Dr. Loren Roth, that the ethical forensic psychiatrist is guided by a commitment to the standards of science. In response to Roth's “scientific standard” as an appropriate ethics guide, Stone remarked, “… if forensic psychiatrists limited themselves to the standards of bench scientists, not only would they not testify about ultimate legal questions, but also their lips would be sealed in the courtroom” (Ref. 4, p 170).
Forensic psychiatric educators should take this statement seriously and strive to answer the following questions that stem from Stone's response to Roth:
Are Stone's concerns regarding expert testimony in court unique to forensic psychiatry, or do bench scientists and other forensic disciplines confront similar problems?
How should AAPL and forensic psychiatric educators address such concerns to accomplish Appelbaum's Standard Position, which emphasizes a subjective and objective search for truth as an ethics-based approach to the practice of forensic psychiatry?
This article reviews early origins of forensic science and evidence admissibility, criticisms by the National Academy of Sciences (NAS) regarding the current state of forensic sciences in the United States, and two key areas in which I believe forensic psychiatry should advance to remain a meaningful member of the larger forensic science community.
Origins of Forensic Science and Its Development in the United States
The word forensic derives from the Latin, forensis, meaning of or before the forum. Under early Roman law, a person facing a criminal charge had his case publicly presented before a group in the forum. The accused and the accuser delivered a speech presenting their versions of the facts. The individual with the best argument and delivery prevailed.5 In ancient times, investigations and trials relied primarily on forced confessions or witness statements. One of the first documented uses of forensic evidence in a criminal trial traces back to a book from the Sung Dynasty in China titled Xi Yuan Lu (translated as Washing Away of Wrongs) written by Song Ci. The author describes a local Chinese peasant who was stabbed to death in his village in approximately 1235 A. D. According to Song Ci's account, investigators determined that a hand sickle must have caused the fatal wound. All villagers who possessed sickles were assembled in the town square and told to place their harvesting tools on the ground and stand behind them. Within minutes, shiny, metallic green flies swarmed around a single sickle, the one that had remnants of blood remaining on it, suggesting that the owner was the murderer. He quickly became the focus of attention and immediately confessed and begged for mercy.6
Over subsequent centuries, forensic scientific evidence introduced during legal proceedings increased in complexity and sophistication. Eventually, the United States legal system developed tests governing the admissibility of scientific evidence. In the 1923 case of Frye v. United States, a defendant attempted to demonstrate that he was innocent by seeking to admit into evidence a crude lie detector test that measured systolic blood pressure.7 The D.C. Court of Appeals ruled that this new technology was not admissible, stating:
Just when a scientific principle or discovery crosses the line between the experimental and demonstrable stages is difficult to define. Somewhere in this twilight zone the evidential force of the principle must be recognized, and while courts will go a long way in admitting expert testimony from a well-recognized scientific principle or discovery, the thing from which the deduction is made must be sufficiently established to have gained general acceptance in the particular field to which it belongs [Ref. 7, p 1014].
The Frye test became known as the general acceptance test and for decades served as the primary standard for governing admissibility of scientific evidence in the courtroom.
In 1975, more than 50 years after the Frye test was formulated, Congress passed a law establishing the Federal Rules of Evidence to guide the admissibility of evidence in criminal and civil litigation. Federal Rule 702 outlined the following standard for determining the admissibility of expert witness testimony:
If scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training, or education, may testify thereto in the form of an opinion or otherwise.8
In 1993, the U. S. Supreme Court reviewed these two standards and, in the case of Daubert v. Merrell Dow Pharmaceuticals, Inc., held that Rule 702 governs the admissibility of evidence in federal courts. In addition, the Daubert Court noted that the “trial judge must ensure that any and all scientific testimony or evidence is not only relevant, but reliable” and the Court provided several factors that the judge could use in making this determination.9
Scientific evidence routinely admitted under Frye, Federal Rule 702, and Daubert includes testimony derived from fingerprint analysis, ballistics, and DNA. These forensic sciences have well-established methodologies and standardized procedures, which it is assumed increase the reliability of their results. However, questions have recently arisen regarding some of the standards governing these forensic sciences, particularly in regard to fingerprint analysis.
As a result of these questions, Congress enacted a statute in 2005 authorizing the National Academy of Sciences (NAS) to conduct a study on the state of forensic science in the United States.10 As a result, the NAS Forensic Science Committee was formed and given specific tasks to address. The committee consisted of 17 members described as having extensive experience in forensic analysis and practice. The academic degrees represented among committee members included 10 PhDs, 2 MDs, 5 JDs, and 1 Master of Science in Chemistry (some members had more than one degree). Over a 12-month period, this committee met eight times, heard a great deal of expert testimony, and reviewed volumes of material related to the state of forensic science in the United States.
The NAS committee issued a report, “Strengthening Forensic Science in the United States: A Path Forward,”11 that outlined their task:
… [T]he aim of our committee is to chart an agenda for progress in the forensic science community and its scientific disciplines. Because the work of forensic science practitioners is so obviously wide-reaching and important—affecting criminal investigation and prosecution, civil litigation, legal reform, the investigation of insurance claims, national disaster planning and preparedness, homeland security, and the advancement of technology—the committee worked with a great commitment and spent countless hours deliberating over the recommendations that are included in the report. These recommendations, which are inexorably interconnected, reflect the committee's strong views on policy initiatives that must be adopted in any plan to improve the forensic science disciplines and to allow the forensic science community to serve society more effectively [Ref. 11, p xix].
The committee commented that the term forensic science encompasses a broad range of forensic disciplines with a range of techniques, methodologies, reliability, types and numbers of potential errors, general acceptability, and related publications. Forensic psychiatry was mentioned once in this document and was defined simply as “similar to forensic pathology, with residency in psychiatry” (Ref. 11, p 220). A review of the NAS committee findings and their applicability to forensic psychiatry is relevant and important to review.
The concern that biases could affect the reliability of scientific evidence, even in fingerprint analysis, was a particular focus of the NAS committee report.11 Although fingerprint analyses are often presented as an objective science that meticulously matches swirls and patterns, the identification of similar visual patterns depends on human experience and judgment. Rather than determining whether two fingerprints are identical, expert fingerprint examiners evaluate whether two different images are similar enough to conclude that they are from the same person.12 Many view fingerprint evidence as infallible.13 Reality demonstrates that this belief is not always true. In fact, there are known cases in which prints from two different people were so similar that the person identified as the fingerprint source was not the actual source.
The mistaken identification of an individual in the case of the Madrid Bomber illustrates how confidence in a scientific technique or practice does not automatically equate with accuracy. On March 11, 2004, 10 explosions occurred on four commuter trains in Madrid, killing 191 people and wounding 1,800. Fingerprints on a bag containing detonating devices were initially identified by the FBI as belonging to Brandon Mayfield, a Muslim attorney who lived in Oregon. The fingerprint match was noted as 100 percent verified. On May 6, 2004, the FBI arrested Mr. Mayfield as a material witness in connection with the Madrid attacks. Spanish authorities later discovered that the fingerprints belonged to an Algerian national named Ouhnane Daoud. Mr. Mayfield was released from prison the next day.11,12 How was this mistake made in the first place? The experts analyzing the fingerprints were already convinced that Mr. Mayfield was the guilty party. As a result, their bias influenced their judgment of the actual data and resulted in an erroneous finding.12
The NAS committee report specifically cited the Mayfield case as a cause for concern regarding the state of forensic sciences in the United States. The committee recognized that forensic science had produced meaningful and valuable evidence relevant to the exoneration of innocent people and successful prosecution of criminals. However, the committee also expressed concerns regarding the admission of faulty forensic science into legal proceedings. One key finding reads:
The simple reality is that the interpretation of forensic evidence is not always based on scientific studies to determine its validity. This is a serious problem. Although research has been done in some disciplines, there is a notable dearth of peer-reviewed, published studies establishing the scientific bases and validity of many forensic methods [Ref. 11, p 8].
The committee emphasized that research is desperately needed, particularly in the forensic disciplines that rely more on subjective assessments. It outlined two factors that should govern the admissibility of forensic evidence in court:
The extent to which a particular forensic discipline is founded on a reliable scientific methodology that gives it the capacity to accurately analyze evidence and report findings;
The extent to which practitioners in a particular forensic discipline rely on human interpretation that could be tainted by error, the threat of bias, or the absence of sound operational procedures and robust performance standards [Ref. 11, p 9].
The NAS committee's concerns foreshadow further scrutiny in our own field. If reliability of fingerprint evidence is now called into question, what might this suggest for the scientific admissibility of forensic psychiatric evidence? These two NAS committee guidelines are directly applicable to the science of forensic psychiatric evidence and can be used as a foundation on which forensic psychiatric educators may build a framework for the future.
Scientific Methodology and Forensic Psychiatry
The medical field continues to advance as a result of scientific investigation and clinical experience. However, today's scientific truths often transition to tomorrow's discarded beliefs. Leeches, bloodletting, and the treatment of various ailments with mercury were considered state-of-the-art medicine during their respective times. Forensic psychiatry has had its share of respected practices that have subsequently fallen out of favor. For example, Franz Joseph Gall, a German physician who rose to prominence in the 1790s, developed the science of phrenology.14 Gall believed that the underlying brain molded the skull's shape. According to his theory, prominent portions of the brain were associated with a cranial protuberance or bump. Gall palpated the heads of psychiatric patients, artists, and criminals and determined that there were 27 aspects or personality traits, each associated with a discrete organ in the brain with an associated palpable bump on the head. Particular cranial bumps of likely interest to a forensic psychiatrist included those associated with an instinct to kill, a desire to own property, a tendency to steal, and, on the opposite spectrum, moral consciousness and goodness. Although Gall's detailed dissections contributed to the work of neuroscience, his work eventually lost the respect of the scientific community and was dismissed as meaningless by the late 1890s.
Numerous challenges to the science of current psychiatric assessment have appeared in various publications, such as Ziskin's book, Coping with Psychiatric and Psychological Testimony,15 which was updated in 2012 by David Faust.16 Both editions highlight speculative expert witness testimony that strays from scientifically established principles. To avoid having their discipline viewed as modern-day phrenology, forensic psychiatric educators bear a responsibility for identifying and teaching the most reliable scientific methodologies in accordance with the NAS committee's guideline that addresses evidence admissibility. Two specific areas where standardized methodologies are particularly important in forensic psychiatry are the assessment of violence and sexual risk and the assessment of malingering.
Forensic Psychiatric Education and Violence and Sexual Risk Assessment Methodology
In his oft-quoted 1981 review of mental health clinicians' ability to predict violence, John Monahan wrote:
[P]sychiatrists and psychologists are accurate in no more than one out of three predictions of violent behavior over a several-year period among institutionalized populations that had both committed violence in the past (and thus had high base rates for it) and who were diagnosed as mentally ill [Ref. 17, pp 47–9].
Although the U.S. Supreme Court permitted psychiatrists to testify about future dangerousness despite the state of violence prediction at that time,18 a psychiatric assessment skill deemed wrong two out of three times is hardly satisfactory, from either a professional or scientific standpoint.
Over the past three decades, a great deal of research has focused on improving violence and sexual risk assessments, a core competency for forensic psychiatry. Skeem and Monahan19 opined that the current risk assessment process involves a continuum of rule-based structure rather than a simple clinical-actuarial dichotomy. On one pole of this continuum are completely unstructured risk assessments (known as the clinical judgment approach), whereas on the opposite pole are completely structured assessments (known as the actuarial approach). Four components that may or may not be present in the continuum of risk assessment approaches include the following: identifying risk factors, measuring risk factors, combining risk factors, and producing a final risk estimate.19
The authors defined five approaches based on the number of structured components included in the assessment process. The five approaches, from least to most structured, are:
No structured component. The clinical judgment approach to risk assessment has no structured component. According to the authors, in this approach, “the clinician selects, measures, and combines risk factors and produces an estimate of violence risk solely according to his or her clinical judgment” (Ref. 19, p 39).
One structured component. This approach involves assessment according to a standard list of risk factors, with identification of these factors being the only structured component. The authors note that this risk assessment approach is conducted by referencing “… a standard list of risk factors that have been found to be empirically valid (e.g., age, past violence), such as the lists provided in psychiatric texts…. Such lists function as memory aids to help clinicians identify which risk factors to attend to in conducting their assessments, but such lists do not further specify a method for measuring these risk factors” [Ref. 19, p 39].
Two structured components. The authors described this approach as Structured Professional Judgment (SPJ) and provide the HCR Historical-Clinical-Risk Management-20 (HCR-20) assessment scheme as an example of the SPJ approach.20 They wrote that the HCR “structures two components of the process: both the identification and the measurement of risk factors…. Structured professional judgments do not go further to structure how the individual risk factors are to be combined in clinical practice” (Ref. 19, p 39).
Three structured components. The authors identified two risk assessment schemes with three structured components, which are the Classification of Violence Risk (COVR)21 and the Level of Service Inventory (LSI).22 According to the authors, “these instruments structure the identification, measurement, and combination of risk factors (via a classification tree design or summing scores)” (Ref. 19, p 39). Although these evaluation schemes combine risk factors to provide estimates of risk levels, evaluators are permitted to modify the overall score based on their clinical impression.
Four structured components. The authors noted that the Violence Risk Appraisal Guide (VRAG)23 is the best-known instrument that structures all components of the violence risk assessment process. In describing the VRAG, the authors wrote, “This instrument not only structures the identification, measurement, and combination of risk factors; it also specifies that once an individual's violence risk has been actuarially characterized, the risk assessment process is complete” (Ref. 19, p 39).
Forensic science demands that the most current and objective approaches be standard practice. An examination of the five approaches outlined shows that the unstructured clinical approach has the least empirical support.19 Mossman24 noted that predictions based only on a person's violent history are about as accurate as statistical predictions, thereby giving some support for the standard list of risk factors approach. The scientific literature, however, indicates significant support for the use of more structured methods, with the primary debate centering on the predictive efficiency of the SPJ method (such as the HCR-20) compared with more actuarial approaches.19 Court admissibility of structured risk assessments varies by jurisdiction and appears to be influenced by the statistical evidence supporting the reliability and accuracy of these instruments, along with an established methodology regarding their scoring procedures.25
I believe the important conclusion to be drawn from this abundant research is that residency training programs must dedicate the necessary resources to train forensic psychiatrists on the appropriate and ethical use, administration, and interpretation of standard violence risk assessment methods. In addition, because the Hare Psychopathy Checklist-Revised (PCL-R)26 is included in numerous violence risk schemes, I believe competency in the use of this instrument is also necessary if the psychiatrist wishes to achieve competency in other assessment tools that incorporate the PCL-R.
Recommending that an evaluator become knowledgeable about the PCL-R does not suggest that this instrument is without significant limitations. Teaching its limitations is as important as teaching its strengths. According to their analysis of the PCL-R total scores from data sets of North American Male Offenders and U.K. male prisoners, Cooke and Michie wrote, “On the basis of empirical findings, statistical theory, and logic, we conclude that predictions of future offending cannot be achieved in the individual case with any degree of confidence” (Ref. 27, p 259). The authors emphasize that evaluators must be extremely cautious regarding determining the risk potential of an individual client based on a PCL-R score, particularly as the precision of numerical scores they observed was less than previously described.
Despite identified problems with the PCL-R and other structured instruments, there are several reasons why training on specific risk assessment instruments in forensic psychiatric education remains important.
First, the scientific evidence indicates that violence and sexual risk assessments are often improved with more structured risk approaches when the selected assessment is appropriately matched to the evaluee. There will always be isolated individual cases where clinical judgment will and should trump more structured schemes due to a unique set of circumstances or a particularly case-specific high-risk factor. However, the evidence that more objective and structured methodologies improve violence and sexual risk prediction in many cases is substantial. It would be difficult to argue that more accurate approaches should not be taught to or used by forensic psychiatrists. However, the evaluator must also be aware of the emerging literature outlining limitations of such assessment tools. For example, in their systematic review and meta-analysis regarding the use of risk assessment instruments in 24,287 individuals from 73 samples, Fazel et al. concluded that these instruments appear to identify low-risk individuals with high levels of accuracy, but “their use as a sole determinant of detention, sentencing, and release is not supported by the current evidence” (Ref. 28, p 1).
Second, statutes and regulations increasingly require specialized assessments to distinguish high-risk individuals for detention and low-risk individuals for release, and structured instruments are increasingly used to accomplish this goal.17 For example, in certain Canadian provinces, parole boards explicitly consider PCL-R scores, and Texas mandates by statute an assessment of psychopathy as a component of sexual predator evaluations.29
In 2006, California legislatively mandated individual sex offender risk assessment as part of the Sex Offender Punishment, Control and Containment Act.30 A state committee on risk assessment, known as the State Authorized Risk Assessment Tool for Sex Offenders (SARATSO) Review Committee was established to choose the official risk assessment tools authorized for use in California. This law requires that a risk assessment instrument chosen by the Review Committee “reflect the most reliable, objective and well-established protocols for predicting sex offender risk of recidivism, has been scientifically validated and cross-validated, and is, or is reasonably likely to be, widely accepted by the courts.”31 Starting in 2012, California began using three evidence-based risk instruments that evaluate risk of reoffense by adult males. A specific risk instrument to assess male juveniles' risk of sexual reoffense has also been selected. With an increasing number of states emphasizing the use of more structured assessments, forensic psychiatrists' inability to understand or use such approaches may result in their not being regarded as adequately informed to conduct examinations.
A required testing protocol should never be the determinant of whether a forensic expert is qualified to perform an examination. Qualifications are determined by knowledge, skills, expertise, and training. An unqualified psychiatrist can and should inform the court that he cannot perform the legislatively required protocol. Some may argue that the default position can always be for the psychiatrist to consult with psychology on these specific assessment tools. This approach is certainly acceptable. However, should forensic psychiatric educators not even attempt to render future psychiatrists qualified in this area? If so, why?
Third, forensic psychiatrists who collaborate with a testing psychologist, either in a treatment setting or as part of a forensic evaluation, should have a practical working knowledge of risk continuum approaches. Some forensic psychiatric hospitals are moving in the direction of requiring structured risk assessments as part of the treatment-planning process. For example, the California Department of State Hospitals (DSH) is planning to have a range of accepted risk assessment instruments as a standard component of patient assessment and risk management (Warburton K, personal communication, October 14, 2012).
Forensic psychiatrists who are not skillfully trained in the use of these instruments will have a lesser ability to coordinate the assessment effectively with the evaluating psychologist and to participate in their patient's treatment and release decisions. Furthermore, if the instruments are used or scored incorrectly, then the treating psychiatrist may well be unaware of their misuse and unable to challenge potentially spurious findings. How can a forensic psychiatrist have an intelligent conversation in a forensic language that he cannot understand or speak?
Fourth, the structured risk assessment instruments are standard components of violence and sexual risk research. The NAS committee report lists 13 specific recommendations that are important in the future development of forensic science in the United States. Three highlight the importance of research, and one specifically notes, “Research is needed to address issues of accuracy, reliability, and validity in the forensic science disciplines” (Ref. 11, p 22). In the past, forensic psychology has been at the forefront in the development of and research on risk assessment instruments. Over the past several years, it has begun to incorporate many of the more structured assessment approaches in both training and publications. For example, The Journal of the American Academy of Psychiatry and the Law (JAAPL) has increasingly emphasized the importance of empirical research in its publication. Likewise, my forensic psychiatry residency training program at the University of California, Davis, routinely provides structured and supervised training on assessment instruments and their use in designing research protocols. I believe that for forensic psychiatrists to become leaders in risk assessment and prevention research, they must have an in-depth knowledge in this area.
Fifth, forensic psychiatrists should play vital leadership roles as administrators overseeing the delivery of care in forensic settings. In this role, the psychiatric administrator must have an understanding of these assessment tools, to include both their strengths and weaknesses and the appropriate populations for their use. Without such knowledge, the administrator must either proceed blindly or delegate the responsibility to an informed surrogate.
Sixth, forensic psychiatrists are likely to encounter other experts who have used structured violence and sex risk assessment approaches in their case analyses, and they must be trained in this subject area to understand the methodology and appropriateness of the assessment instrument used by other evaluators.
The recommendation that forensic psychiatrists achieve competency in administering certain assessment tools does not suggest that they can or should administer all forms of psychological assessments and should not be taken out of context as suggesting so. Rather, it is important to acknowledge that there are several instruments appropriate for psychiatrists to administer, consistent with each instrument manual's guidelines and commensurate with the knowledge and skills of the forensic psychiatrist. In fact, nearly all of the manuals associated with the assessment tools mentioned in this article specifically identify psychiatrists as appropriate professionals to administer the test.
I believe that forensic psychiatric educators should provide training on a limited number of evaluation tools that includes each instrument's:
Ethical and appropriate use;
Limitations;
Applicable populations in its development;
Research application and utility;
Reliability and validity;
Governing statistical principles;
Administration procedures;
Relevant legal issues and restrictions; and
Role in a comprehensive forensic evaluation process.
Training in this area should be rigorous and comprehensive, with established standards to determine and document competency.
Forensic psychologists are an invaluable resource to provide this education as part of forensic psychiatry residency programs and other educational efforts. In addition, AAPL should seriously consider how the organization might provide education to its members regarding the assessment instruments that are appropriate and relevant for its members to administer and available resources for achieving competency in them.
Forensic Psychiatric Education and Malingering Assessment Methodology
Malingering assessments are a core component of many forensic evaluations and represent an area where forensic psychiatrists should demonstrate particular competency. In his 1973 article, “On Being Sane in Insane Places,” Rosenhan32 forwarded the view that psychiatrists cannot reliably distinguish individuals with genuine mental illness from those without. In his study, eight individuals who had never demonstrated symptoms of a psychiatric disorder and were functioning well were told to go to the admissions office of a psychiatric hospital and complain that they were hearing voices. These pseudopatients were instructed that when asked what the voices said, they were to reply that the voices were often unclear but sounded like they said, “empty,” “hollow,” and “thud.”
All eight individuals were admitted to the hospital. Although these pseudopatients reported no further symptoms and functioned normally on the unit, all were diagnosed as schizophrenic with the exception of one, who was diagnosed as manic-depressive.32
This study is sometimes cited as evidence that psychiatrists cannot reliably detect malingering, even though this is a task that they are frequently asked to do. There are obvious flaws in this outdated study, which include a very small sample size and the fact that individuals were seeking mental health assistance rather than being evaluated for forensic purposes. Even though this study is sometimes used to cross-examine psychiatrists about their limited ability to evaluate malingering, one cannot make any general conclusion about current psychiatric assessments of malingering from these findings. Nevertheless, the question of how forensic psychiatrists evaluate malingering remains relevant.
Resnick has substantially advanced the psychiatric literature on the clinical assessment of malingering and has detailed numerous symptoms that help distinguish genuine from feigned symptoms.33 He has also significantly improved the quality of clinical assessments of malingering through his workshops and his teaching efforts at the AAPL Annual Forensic Review Course. In addition to the development of improved clinical guidelines to assess malingering, structured assessment instruments have emerged that specifically address malingering of psychiatric symptoms.
Because of the complexity involved in identifying someone as a malingerer and the consequences of doing so, such instruments are often a useful, if not critical, component of this evaluation. As with risk assessment instruments, there are also several instruments appropriate for psychiatrists to administer consistent with each instrument manual's guidelines and commensurate with the knowledge and skills of the practitioner.
Examples of tools for assessment of malingering include the Test of Memory Malingering (TOMM),34 the Structured Inventory of Malingered Symptomatology (SIMS),35 the Structured Interview of Reported Symptoms (SIRS),36 and the Miller Forensic Assessment of Symptoms Test (M-FAST).37
Many of the same reasons that forensic psychiatrists should be educated regarding structured risk assessment instruments support the need to be trained on structured malingering assessments. I believe that educators should train forensic psychiatrists on the ethical and appropriate use of specific instruments to assess malingering consistent with the recommended teaching requirements of the structured risk assessment approaches outlined above.
Forensic Psychiatric Education Regarding Potential Biases
Biases include beliefs or extraneous factors that influence decision-making, often without the evaluator's awareness. Psychiatric assessments have been characterized as more subjective when compared with other forensic disciplines (such as DNA or fingerprint analysis) and therefore at potentially greater risk to be influenced by various biases. However, problems identified with latent fingerprint analyses indicate that even this serious science is vulnerable to a variety of outside influences that may support faulty findings. The NAS Committee's report highlights the need for research in this field and states among their final recommendations:
The National Institute of Forensic Science (NIFS) should encourage research programs on human observer bias and sources of human error in forensic examinations. Such programs might include studies to determine the effects of contextual bias in forensic practice (e.g., studies to determine whether and to what extent the results of forensic analyses are influenced by knowledge regarding the background of the suspect and the investigator's theory of the case) [Ref. 11, p 24].
Recent research on cognitive biases in fingerprint analysis provides a useful framework for understanding and addressing potential biases in the field of forensic psychiatry and is worth reviewing.
Dror et al.38 hypothesized that extraneous information would influence the forensic analysis of fingerprints by experts. To test this hypothesis, they obtained fingerprints that had been examined and assessed five years earlier by five latent print experts to make positive identification of suspects. These same fingerprints were submitted again to the same experts. However, on this second presentation, the experts were provided information that suggested that the fingerprints were a non-match. In particular, they were told that the pair of prints provided were the ones that were erroneously matched by the FBI to those of the Madrid bomber.
This extraneous information created a context that the prints were a non-match, when in fact the experts had previously identified the fingerprints as a match. Four of the five experts changed their identification decisions from their earlier decisions, and only one participant judged the prints to be a match. The authors concluded that the extraneous context in which fingerprint examinations occur could determine the identification decision.38
A follow-up study confirmed these findings and noted that fingerprint examiners could be biased in both directions, toward exclusion or identification, based on more ordinary contextual information.39 Contextual information shown to bias the fingerprint analysis included telling the examiner something about the suspect, such as “the suspect has an alibi” or the “suspect confessed to the crime.”33
As a result of their research, Dror et al.38 provided four recommendations to improve the accuracy of forensic examinations. I believe these recommendations are relevant to most forensic science disciplines, including forensic psychiatry:
improved initial selection and screening of experts;
appropriate training and professional development;
adoption of methodological procedures; and
application of cognitive research to forensic analysis to maintain the objectivity of the scientific method and minimize biases and extraneous contexts.
Forensic psychiatry has made significant gains over the past several decades in achieving the goals outlined by the first and second recommendations. The establishment of nationally accredited training programs has resulted in an increased number of trained forensic psychiatrists, eligible for a national board certification offered by the American Board of Psychiatry and Neurology (ABPN). AAPL has also achieved significant advances related to the recommendation to adopt methodological procedures as demonstrated by the publication of national practice guidelines. I addressed the need for additional training on the methodology of structured assessment instruments in the prior section.
Regarding the fourth recommendation, the AAPL Ethics Guidelines focus on the influence of potential biases on an expert's objectivity. They specifically state:
The adversarial nature of most legal processes presents special hazards for the practice of forensic psychiatry. Being retained by one side in a civil or criminal matter exposes psychiatrists to the potential for unintended bias and the danger of distortion of their opinion. It is the responsibility of psychiatrists to minimize such hazards by acting in an honest manner and striving to reach an objective opinion.40
The Guidelines provide the standard for what forensic psychiatrists should be striving for and why it is important. Forensic psychiatric education must also provide specific skills for identifying and correcting such biases.
Biases and Forensic Psychiatry
Bias represents a preference that influences impartial judgment. Forensic experts must be aware of the potential impact of bias on their case analyses and opinions. The list of potential biases that may affect forensic science is extensive. Several publications have defined eight biases for the evaluator to consider:
Anchoring bias: “the tendency for information received early in the diagnostic process to be remembered better and used more than information received later in the process” (Ref. 41, pp 33–4).
Attribution bias: “discounting contextual factors accounting for behavior and imputing it instead to a permanent characteristic of an individual” (Ref. 42, p 325).
Confirmation bias: “giving more weight to information that is consistent with [the evaluator's] own beliefs” (Ref. 41, p 34).
Conformity bias: the tendency to “conform to the perceptions, beliefs, and behavior of others” (Ref. 43, p 20).
Halo effect: “the tendency for a general evaluation of a person, or an evaluation of a person on a specific dimension, to be used as a basis for judgments of that person on other specific dimensions” (Ref. 44, p 220).
Hindsight bias: “when people who are aware of how an incident turns out believe that the outcome was more likely than objective prediction would indicate” (Ref. 41, p 35).
Observer bias: “the … thoughts, feelings, experiences, and expectations of people, including scientists, influence [the expert's] perceptions and conclusions” (Ref. 41, p 35).
Overconfidence bias: “when the clinician feels certain of his or her conclusions and therefore assumes they are valid” (Ref. 41, p 35).
This article will address five of the common biases that may affect forensic psychiatry: anchoring, confirmation, attribution, observer, and hindsight. There are many available illustrations of each, but for brevity only one or two examples will be provided.
Anchoring Bias
In his classic study of the impact of anchoring bias, Asch45 presented descriptions of two people and asked for comments on their personalities. The simple question, “What do you think of Alan and Ben?” was posed, with answers to be based on the following attributes:
Alan: intelligent, industrious, impulsive, critical, stubborn, envious;
Ben: envious, stubborn, critical, impulsive, industrious, intelligent.
Most people viewed Alan more favorably than they viewed Ben, even though identical descriptors were provided for each. Asch's research demonstrated that the initial traits presented change the meaning of subsequent traits. Furthermore, the personality traits that appeared later in the sequence were not equally considered in deciding how to view the individual's personality.
Similar findings have also demonstrated how quickly psychiatrists may form an opinion on the patients whom they are evaluating. For example, Gauron and Dickinson46 requested psychiatrists to provide their diagnostic impressions of patients shown to them on videotape. Many of the psychiatrists made their diagnoses within 30 to 60 seconds, and their opinions rarely changed, even when they were provided contradictory evidence. Forensic psychiatrists should consider and consciously address potential influences of early impressions on their ultimate opinions.
Confirmation Bias
Confirmation bias is the favoring of information that supports a preferred view or a decision that has already been reached. A desire to please the retaining party, negative feelings toward the retaining party, or a wish to receive additional financial compensation for a particular opinion may influence an expert in the review of the available evidence. However, even without such obvious sources of potential bias, an expert who arrives at an early opinion regarding a case may fall prey to confirmation bias, even if he is not consciously aware of the influence.
Structured assessment tools, such as the PCL-R, may also be subject to biases that include confirmation bias. The PCL-R26 is an accepted instrument to measure the construct of psychopathy. High PCL-R scores have also been described as a potential risk factor for future criminal recidivism,47 even though the PCL-R was not designed as an actuarial risk assessment instrument. A potential advantage of using an assessment instrument with defined scoring guidelines (such as the PCL-R) is the ability to examine rater agreement among independent raters. Under research circumstances, the PCL-R has, in fact, demonstrated a high intraclass correlation coefficient for a single rating (0.86).26 Do the reliable scores achieved among trained researchers represent the reliability of PCL-R scores achieved in real-world forensic settings, such as evaluations of sex offenders for purposes of sexually violent predator (SVP) hearings? If not, does this diminish its practical utility?
There are surprisingly few studies examining how PCL-R scores obtained by various evaluators correlate in actual forensic assessments. In one study, Murrie et al.48 examined 23 cases in which defendants being considered for commitment as SVPs were assessed with the PCL-R by both defense and state evaluators. Of note, the difference in the PCL-R scores obtained by opposing evaluators was higher than would have been expected, considering the instrument's standard error of measurement (±3 points). In general, the score differences were most often in the direction that supported the side retaining the evaluator. These results raise concerns that even instruments with structured guidelines and scoring systems may be influenced by potential partisan allegiance. If these results are generalized, and scores depend on the scorers, should they be allowed? A subsequent study suggested that judges should be informed how to adjust such PCL-R scores on the basis of which side hired the evaluator.49
The lack of reliable information arising from any assessment instrument seriously challenges both the science and ethics of our field. However, a good assessment instrument can be used badly, either by deliberate distortion or as a result of a more subtle confirmation bias, where information supporting a particular opinion is more heavily weighted. Forensic psychiatric education must not only provide rigorous training on the correct administration of assessment instruments but also on the influences that may adversely affect the instrument's accuracy.
Attribution Bias
Attribution bias includes extraneous factors or beliefs that may influence an evaluator's decision. The possibility that the referring party may selectively provide the most favorable information to support its view of the case is a contextual bias familiar to many trained forensic psychiatrists. Contextual biases may be less obvious, yet they influence forensic opinions.
For example, Esses and Webster50 researched the potential contextual bias of physical attractiveness on ratings of dangerousness of sexual offenders. In their study, 284 adults were given information about a hypothetical offender, including a facial photograph and a conviction record. Questions regarding potential dangerousness were drawn from the Canadian Criminal Code's Dangerous Offender criteria. The authors found that sexual offenders perceived as physically unattractive were significantly more likely to be assessed as meeting the Dangerous Offender criteria when compared with average-looking and attractive sexual offenders. Furthermore, unattractive sexual offenders were seen as less likely to restrain their behavior in the future. Although the evaluators in this study were adults from the general population rather than forensic professionals, the findings raise questions as to the potential negative bias against unattractive persons in ratings related to future dangerousness.
Gold51 noted that unique biases may also arise in sexual harassment claims, to include gender biases, diagnostic biases, sociopolitical biases, and biases that arise from lack of knowledge regarding sexual harassment or a lack of formal psychiatric training. Other potential contextual biases not specific to sexual harassment include a person's age, social class, and even empathy for the evaluee.52,53
Although poorly understood and probably more covert in its underpinnings, disparities in mental health diagnoses, services, and outcomes from persons in racial minority groups are well documented and pose concerning questions of racial bias during psychiatric evaluations. As an example, African Americans are disproportionately diagnosed as having schizophrenia.54 The research indicates that evaluators use different symptom criteria when making a schizophrenia diagnosis among African Americans,55 and this difference in diagnostic assessment is equally prevalent among white and African-American clinicians.56 Furthermore, clinicians assess African Americans as more paranoid and suspicious, factors commonly regarded as elevating a person's violence risk.57,58
To improve understanding of factors that contribute to this overrepresentation of schizophrenia diagnoses affecting the African-American community, Eack et al.58 used data from the 1995 MacArthur Violence Risk Assessment Study in examining the impact of racial differences in sociodemographic characteristics, clinical presentation, and perceived honesty. African Americans were more than three times as likely as whites to receive a diagnosis of schizophrenia. Mediator analyses confirmed that interviewer-perceived honesty was the only consistent mediator of the relationship between race and diagnosis of schizophrenia.
Such racial biases, whatever the underlying etiology, have profound implications for minorities facing forensic evaluations in both the criminal and civil justice systems.
Observer Bias
Characteristics and experiences unique to the examiner may influence his decision. Extraneous factors, such as mood or working environment may also play a role in biasing the evaluator. For example, cognitive research indicates that asking someone to render an opinion repeatedly over a defined period strains executive functioning. Such cognitive strain affects subsequent decisions that a person makes and results in a simplification of one's overall reasoning.59 This process has been referred to as decision fatigue and may cause a practitioner to make a decision that is the most commonly accepted, or default, decision. An example in forensic psychiatry may include an evaluator who conducts several violence risk assessments on forensic psychiatric patients in one day. Decision fatigue would predict that, as the day progresses, the default reasoning pattern would prevail and the examiner would be more likely to advise continued hospitalization if this recommendation was the usual one.
Can decision fatigue affect important forensic decisions, such as which prisoners are appropriate for parole consideration? Danziger et al.60 attempted to answer this question in reviewing judges' decisions involving a prisoner's request for release. The authors hypothesized that as judges advance through their sequence of cases, they are more likely to accept the status quo outcome, which is to deny a prisoner's request. Researchers reviewed 1,112 judicial rulings and recorded the judges' two daily food breaks. They found that the likelihood of a favorable ruling was greater at the very beginning of the workday or after a food break than later in the sequence of cases. The study indicated that when judges made repeated rulings regarding whether to release a prisoner, they showed an increased tendency to rule in favor of the status quo, which was not to release.61 However, the judge's mood might also explain this finding. If his mood after a break was more positive, then the judge could have a bias toward generosity or leniency, indicating a mood-congruency bias unique to the judge.
Hindsight Bias
Hindsight bias is the process by which knowledge of an outcome influences the perceived likelihood of that outcome.61,62 This bias arises particularly in reviews of psychiatric care for medical review panels, administrative and quality assurance committees, and medical malpractice evaluation referrals.
A study of hindsight bias among psychiatrists conducted by Le Bourgeois et al.63 provides an excellent potential resource for standardized training of forensic psychiatrists. In this study, 235 general and forensic psychiatrists reviewed hypothetical cases in which patients with suicidal or homicidal ideation presented for psychiatric care. One half of the participants were informed that a suicide or homicide occurred shortly after the patients were released from care (hindsight group) whereas the other participants (control group) were not provided information about the outcome. Subjects were asked to estimate the likelihood that suicide or homicide would occur at the time of release and whether the standard of care had been met.
The authors found that hindsight bias played a role in the estimate of risk in this study, with those knowing the outcome estimating a higher risk of suicide or homicide. However, hindsight bias did not play a role in opinions regarding negligence. In addition, psychiatrists who were AAPL members demonstrated less hindsight bias when compared with American Psychiatric Association (APA) non-AAPL members. The authors concluded that belonging to a professional organization (such as AAPL) that promotes research on topics related to forensic psychiatric practice and the ethics of forensic psychiatry provides a protective buffer against hindsight bias.63
Forensic Educational Approaches to Improve Bias Awareness
The importance of understanding and identifying potential bias in forensic psychiatric examinations has been well recognized. The AAPL Ethics Guidelines, which include caveats about biases and an emphasis on striving for honesty and objectivity, are routinely incorporated into forensic psychiatry training programs and national presentations.40 However, a simple verbal or written warning to oneself or others may be inadequate to counter the bias effect.
For example, in his study examining the impact of hindsight bias, Fischoff61 found that examiners who were specifically instructed to consider alternative explanations to the available information failed to counter hindsight bias. Therefore, the development and implementation of specific teaching methodologies to ensure that psychiatrists not only understand the impact of bias but also develop approaches to keep such biases at bay are critical.
Wills64 presents the CHESS model as one example of a more structured approach to help forensic experts checkmate bias in their forensic analyses. She proposes five steps based on the acronym: C, the claim (preliminary opinion) is formed after collecting the data; H, hierarchies of supporting evidence are established; E, examinations of evidence are made for potential vulnerabilities in cross-examination; S, studies of the evidence are conducted to look for potential weaknesses and determine whether additional information is needed; and S, synthesis of a revised opinion is made with stronger supporting evidence.
Shuman and Zervopoulos65 comment that Wills' approach to addressing potential bias is overly focused on defending the opinion than on the opinion's development. They recommend a more assertive approach to address empathy bias specifically. However, their approach is relevant to identifying many other biases as well. In particular, they propose that forensic experts must first generate plausible alternatives to the data that they consider and then challenge how each plausible explanation matches the data, until a best match between the explanation and data is reached. To help achieve this goal, they provide a six-stage model to aid the expert in reaching an objective opinion. This approach could be adopted as a teaching strategy for forensic educators to implement in their training programs as well as a component of educational teaching strategies for forensic mental health organizations such as AAPL.
In addition to these two models, an educational curriculum specific to bias recognition could be formalized in forensic psychiatry training programs and by national education committees. In view of the NAS committee's strong recommendation for research on human observer bias and sources of human error, bias-detection-correction training would help address serious concerns in this area. In addition to a review of the key literature on this subject, the curriculum could incorporate the type of vignettes used in research studies that address both racial and hindsight bias.
For example, having participants assign diagnoses and violence risk in vignettes with or without racial identifiers would help expose potential bias toward minorities, with training on how disparities in outcomes could be a consequence. Likewise, educators could present psychiatric assessment and treatment decisions in a case format where ultimate negative outcomes were and were not included, to highlight the effects of hindsight bias on forensic opinions regarding standard of care. Developing standardized bias-detection-correction training would not only address many concerns regarding forensic psychiatry, but would provide a national teaching strategy for other forensic science disciplines to assess potential biases.
Summary
AAPL was founded primarily as an organization to advance the body of knowledge in the area of psychiatry and the law. Its original mission statement specifically stated that training psychiatrists in how to improve their court testimony was not the major function of the group and membership; the major function was and is education in the field of forensic psychiatry. The education provided should be as accurate and objective as possible and should be based on the current science of the field. The NAS Committee report provides specific guidance as to how forensic psychiatry, as a member of the forensic science community, should strive toward the scientific search for truth. These guidelines emphasize that the forensic discipline must have a reliable scientific methodology that provides a capacity for accurate analysis of evidence in addition to procedures and standards to minimize the risk that evidence will be tainted by error or bias.
Forensic psychiatric education can help achieve that goal by providing increased training on standardized forensic assessment instruments and specific training regarding the impact of biases on assessment methodology and opinion formation. When teaching the assessment tools, the educator must provide a balanced approach regarding the strengths and limitations of assessment tools. In other words, just because a structured instrument may improve an assessment in some situations does not mean that its use is always warranted. Forensic psychiatrists who blindly advocate that tests always be used may fall prey to the same impaired reasoning as those who opine that psychiatrists should never use them. For both groups, believing doesn't make it so.
Nearly 45 years ago Dr. Jonas Rappeport wrote a letter. That letter set into motion a series of events that ultimately led to the creation of the American Academy of Psychiatry and the Law. That one letter changed the lives of many. Over the years, AAPL has contributed significantly to the advancement of forensic psychiatry as a scientific field through the establishment of national annual training sessions, national practice guidelines, peer review procedures, and the creation of institutes of research and education. As a result, more objective methodologies in forensic psychiatric assessments have been realized. Although significant work remains to advance the field further, the challenge can be met. There are many letters left to write.
Footnotes
Disclosures of financial or other potential conflicts of interest: None.
- © 2013 American Academy of Psychiatry and the Law
References
- 1.↵
- 2.↵
- 3.↵
- 4.↵
- 5.↵
- 6.↵
- 7.↵
- 8.↵
- 9.↵
- 10.↵
- 11.↵
- 12.↵
- 13.↵
- 14.↵
- 15.↵
- 16.↵
- 17.↵
- 18.↵
- 19.↵
- 20.↵
- 21.↵
- 22.↵
- 23.↵
- 24.↵
- 25.↵
- 26.↵
- 27.↵
- 28.↵
- 29.↵
- 30.↵
- 31.↵
- 32.↵
- 33.↵
- 34.↵
- 35.↵
- 36.↵
- 37.↵
- 38.↵
- 39.↵
- 40.↵
- 41.↵
- 42.↵
- 43.↵
- 44.↵
- 45.↵
- 46.↵
- 47.↵
- 48.↵
- 49.↵
- 50.↵
- 51.↵
- 52.↵
- 53.↵
- 54.↵
- 55.↵
- 56.↵
- 57.↵
- 58.↵
- 59.↵
- 60.↵
- 61.↵
- 62.↵
- 63.↵
- 64.↵
- 65.↵
