Abstract
Despite major advances in the prediction of violence and risk management, risk evaluations are necessarily imperfect. This article focuses on the role of the evaluator in such assessments and addresses the consequences incurred by clinicians involved in conducting risk assessments. We discuss the risks of overpredicting and underpredicting violence with respect to how these risks can influence an evaluator's opinion. Then, we review how law and psychology inform the clinician's work. Finally, in response to the compelling tendency toward overprediction of violence, we propose a preliminary self-assessment guide as a method of examining influences on the evaluator in assessing violence risk.
Violence risk assessment has long been the business of forensic mental health practitioners. Moreover, it has been historically tumultuous. Serious concerns about violence prediction emerged during Baxstrom v. Heral d,1 when the court released or shifted to minimum security 966 patients who had been committed to maximum security hospitals. In their follow-up, Steadman and Cocozza2 found that only 20 percent of the patients were reconvicted. Similar concerns arose with testimony of the notorious “Dr. Death,” including his infamous prediction of 100 percent likelihood of future violence as described in Barefoot v. Estelle. 3 These and other events led to distrust of violence predictions, including those of distinguished commentators within the field.2,4 In 1983, the American Psychiatric Association presented an amicus curiae brief to the effect that psychiatrists were so inaccurate at predicting violence that courts should not hear psychiatric testimony on the subject.3
Despite (or perhaps because of) this gloomy early history, mental health professionals have responded with great energy to cultivate a new era of risk prediction.5 Actuarial and structured professional judgment risk assessment measures have entered the field; we are now in the third generation of risk measurement.6 While the field has made significant improvements in methodology, several difficulties still exist. Perhaps the most notable problem is base rates. The behaviors that evaluators try to predict (future violence) vary widely among relevant populations, and the variability diminishes the accuracy of risk assessment tools when applied to samples outside of the validation studies.7 Even when given direct information about base rates, lay people and professionals alike still markedly overpredict violence.8 No clear guidelines exist regarding how to best communicate risk, and different presentations of the same information may be interpreted differently by consumers.9 Further, identifying correlates of violence risk yields limited useful information to clinicians. The nature of the relation between these factors and future outcomes (e.g., as moderators, mediators, or risk markers) is still unclear.10
An additional difficulty in risk assessment is the large number of “moderate risk” evaluees.11 Justice Blackmun, in his dissent in Barefoot, stated that “no reputable expert would be able to predict with confidence that the defendant will not be violent” (Ref. 3 p 934; emphasis in original). The point is well stated, for even the most innocuous of evaluees would still be described to present some level of risk. One task for the clinician in conducting risk assessments may be conceptualized as a conflict between protection of an individual's civil liberties and protection of public safety. Some observers have gone so far as to assert that “concern for civil liberties has sometimes obscured the need to place public protection at the forefront” (Ref. 11, p 104). Although society may find this individual restriction of rights to be acceptable, the question for the clinician remains as to whether ethics-related obligations of promoting accuracy and truthfulness in the practice of psychology and respecting people's rights12 have been satisfied in conducting the assessment and reaching a prediction estimate.13
This weighing of individual rights versus public safety has left characteristics of the evaluator out of the debate. Although one might hope that evaluators are interchangeable and would make no difference in the outcome of an evaluation, this is more of an aspirational objective than a practical reality.14,15 The present paper focuses on the role of the evaluator in risk assessments, particularly with regard to the personal and professional consequences of being wrong. We address the following questions: What are the perceived risks to the evaluator? What information do evaluators draw on to help in the decision-making process? Where do we go from here? We propose a method of self-guided assessment to assist mental health professionals in assessing influences on their violence predictions.
The Risks of Inaccuracy
Although there has been debate over the best way to report an individual's risk level (such as categorical risk, odds ratios, or probabilities over a given time period), researchers generally agree that a level of descriptive and explanatory detail beyond a simple yes/no answer is necessary.16 However, consumers of risk assessments do not necessarily interpret evaluator findings with the intended level of nuance, as evidenced by legal statutes that rely on dichotomous decisions such as whether to impose additional sanctions.7 The following discussion will focus only on categorical predictions of high and low risk.
An inaccurate prediction of high risk when the evaluee is actually unlikely to commit future violence would result in a restriction of that person's rights (through civil or criminal commitment or a longer sentence). The label “high risk” or a similar label from clinical terminology may attach additional stigma to the individual. In the end, however, the evaluator's influence on the restriction of rights is tempered by the fact that the court, parole board, or other criminal justice entity makes the ultimate decision on confinement or release. In other words, even if the mental health professional's opinion influenced the decision to confine the evaluee, the justice system has jurisdiction over the case and will be the body responsible for deciding about incarceration or when confinement will end.
Consider the consequences to the clinician when a person predicted not to be violent does, indeed, commit a violent act. Negative publicity is likely to occur as soon as the connection between a given violent incident and the offender's release from a mental health or correctional facility is made. One published case example demonstrated a major impact on a forensic hospital in the form of fired administrators, patient and staff confusion, and day-to-day changes in policy after a patient escaped, even though the patient caused no harm while he was in the community.17 Another case study demonstrated lasting negative effects on hospital staff in terms of lower morale, fear of personal litigation, and psychological symptoms of distress years after a released patient killed a family member.18 These examples do not bode well for a facility that would release an individual who is subsequently violent. Such publicity may not only affect the institution, but also the individual clinician(s) who conducted the assessment and the administrator(s) who made the final decision for release.
Closely related to resulting negative publicity is public outrage and subsequent loss of trust of mental health professionals. Indeed, predictive inaccuracy from expert mental health witnesses may poison the relationship between clinicians and court personnel.19 In more immediate terms, the high risk individual may harm others after release. Finally, the potential exists for civil litigation. Research suggests that the possibility of litigation leads professionals to recommend a more secure placement than predictive data alone indicate, suggesting that clinicians do take such potential consequences into account when making conclusions in a risk evaluation.20
Overprediction in Context
A common perception is that it is safer for evaluators to overpredict violence than to underpredict it. However, risk assessments are not equal, and both external and internal pressures may influence the consequences to the predictor of being inaccurate. With regard to external forces, violence risk assessments can take place in different contexts, such as release from a forensic hospital, civil commitment proceedings, and sentencing hearings. The degree to which rights are restricted will vary across contexts.13 Situational factors in each case will have an impact as well, most notably in the example of a high profile case. Community and media scrutiny would increase, leading to a wider audience to the evaluator's inaccuracy.
Internal pressures refer to evaluator-specific factors that may impact the consequences of prediction. Personal biases, attitudes, and values are examples. While mental health professionals are specifically taught during their training to remain objective, overt or hidden preconceptions may nonetheless leak into their work.21 Adversarial allegiance, or the tendency to lean in favor of the hiring side, is an additional risk.15 Further, confidence or level of experience may influence the consequences to the predictor. In one prospective study, clinicians were generally confident in their risk assessments, although some findings suggest that higher levels of confidence were associated with lower accuracy in predicting future aggression.22
Contextual factors in risk assessments can be divided into categories based on whether the evaluation occurs before or after confinement. Civil commitment exemplifies the “before” context, in that confinement usually has not yet taken place. Moreover, no bodily harm has to have taken place. Most statutes require an overt act or evidence of imminent danger,23 requirements that are met by patients voicing suicidal or homicidal ideation. The applications of therapeutic jurisprudence and parens patriae often make such hearings less adversarial,23 and thus the resulting restriction of liberty may be seen as minimal because it is presumably in the patient's best interest.
Another assessment that takes place before confinement is conducted for a presentence hearing. In contrast to the civilly committed, the convicted individual is less likely to be viewed favorably.24 The individuals who are likely to be most vocal in favor of severe punishment are the victim, victim's family, and the community, particularly when there is public outrage over a heinous crime. However, psychological testimony is not overly influential in presentence hearings,23 decreasing the stress on the evaluator to be “right” as public pressures push the evaluator for harsher outcomes. Both civil commitment and presentence situations contain pulls toward overpredicting violence, whether it is for the patient's benefit or because of societal outrage.
On the other hand, evaluations for release from a forensic setting occur after confinement (and some illegal act) has already taken place. In one study of 115 schizophrenic patients in a court clinic and in a jail, patients' limited insight was associated with severity of disorder and violence.25 Such findings open up consideration of, whether clinical variables were successfully treated as a component of the risk assessment (e.g., specifics of the confining facility). Several risk assessment instrument manuals emphasize that the information source indicating the most criminal or violent incidents (whether it be from records or from self-report) is to be given the most weight,26,–,28 subtly pushing evaluators toward a higher estimate of risk.
One situation that does not fit neatly into the “before” and “after” confinement categories is risk assessments for sexually violent predator (SVP) hearings. Many states have an SVP statute that permits the civil commitment of sex offenders after their prison sentences are completed.23 Such individuals are similar to parolees in that the retributive purpose of the criminal justice system has been served; however, sex offenders are particularly disliked and stigmatized by the public.29 This community repulsion brings with it the involvement of groups that advocate against leniency for any sex offender. SVP hearings are also similar to standard civil commitment proceedings, in that the commitment is not portrayed explicitly as punishment,23 but rather an acceptable restriction of individual rights in the interest of public safety and rehabilitation. The contextual factors in SVP commitments again seem to push the mental health professional to overpredict the evaluee's level of risk.
Both external and internal factors influence the consequences to the predictor. A continuum of predictor risk depends on the type of risk evaluation being conducted. The risk of negative consequences may well be a product of the subjective understandings of the predictor combined with the severity of the potential consequences. For example, the consequences of underestimating the risk of a one-time burglar may be less severe than underestimating the risk of someone with a history of aggravated assault. Alternatively, a more parsimonious explanation is that the degree of predictor risk is determined solely by the level of public visibility. We have identified several contexts in which the community reaction and public interest may add external pressures toward overpredicting risk. However, little empirical work has been conducted to directly assess consequences for the predictor of inaccurate predictions. Nonetheless, some areas of the literature do inform the work of mental health professionals in this arena.
What Informs Our Work So Far?
Precedent from both law and psychology provide direction to the mental health professional with regard to prediction of risk of violence. The main goal of the United States criminal justice system is retributive, though the pendulum has started to swing back toward more rehabilitative approaches for juvenile offenders. Consistent with retributive principles, many states have enacted three-strikes laws in which a third felony conviction results in a substantially higher penalty than it might ordinarily carry.23 Sex offender statutes are heavily informed by the U.S. Supreme Court decision in Kansas v. Hendricks,30 which upheld civil commitment after imprisonment as long as the offenders have a mental abnormality that makes them “dangerous beyond their control” (Ref. 23, p 283). The mental illnesses allowed under “mental abnormality” in these statutes are much greater in scope than the limited disorders historically accepted under not guilty by reason of insanity pleas.
The Supreme Court in Foucha v. Louisiana acknowledged that a “pressing public concern” may override the general rule that the dangerousness of an individual cannot form the sole basis for confinement.31 Here the law again weighs in on the side of public safety over individual rights. Finally, the Tarasoff ruling demonstrates that an attempt to get a high risk individual committed or detained is not necessarily enough; mental health professionals in fact have a duty to protect identifiable third parties from potential harm.32 Such legal precedents suggest that, where there is doubt, a higher level of stated risk is the preferred professional conclusion.
Other psychological factors also influence risk assessments. The cognitive literature identifies mental heuristics that people use to better manage the information received every day. Many such heuristics play an important role in legal decision making and at times lead to more distorted conclusions. Perlin33 identified examples relevant to evaluators of risk, including confirmation bias (seeking information to support a preconceived conclusion while ignoring information that contradicts it) and illusory correlations (assuming two things are related in an anticipated direction because they happened simultaneously).
Pre-existing attitudes also have the potential to influence evaluators. For example, a clinician's personal orientation toward punishment or rehabilitation of offenders will affect how that assessor views the criminal justice system and its players.34 Past experiences, both professional and personal, may influence risk decisions. Some research suggests that attitudes toward sex offenders are more positive among those professionals who work with such individuals than among similar professionals who do not work with sex offender populations.35 Professional training experiences can promote looking for psychopathology where it is supposed to exist.36 Further, witnessing consequences that befall a colleague who inaccurately predicts risk may be internalized through observational learning.
The concept of adversarial allegiance suggests the retaining side may have weight with the psychologist's opinion.15 Some have suggested the use of clinical judgment in conjunction with purely actuarial assessments may be an avenue for one's fear of making an error to influence risk assessment decisions.21 Finally, clinical lore passed down from supervisors and colleagues seems to suggest that it is better to “err on the side of safety,” with safety belonging to the public.36 Despite such lore, the overall evidence from psychology is not always indicative of overprediction. Rather, personal attitudes and experiences can be found at either extreme, and cognitive heuristics would simply encourage continuation in existing directions of thinking.
Evaluator Self-Assessment
The reality is that factors outside of the individual risk assessment case can influence evaluators' opinions. Just like any other ethics-related concern in the work of mental health professionals, the gray area should be examined on a case-by-case basis. Knowing one's own tendencies toward over- or underprediction may lead to more sound assessments. We propose a way to think about self-assessment of the influences on violence prediction (Table 1). This first step is preliminary and is intended to open up the topic for professional discussion and for successive iterations toward a more formal checklist.
Influences on Violence Prediction: A Self-Assessment Guide
This preliminary guide proposes consideration of potentially influential sources with accompanying questions to promote discussion. Review of the final form of this guide may be most useful on an annual basis and as a frame of reference during particularly difficult cases, to reflect on the pushes and pulls of forensic risk evaluations. Although designed as a self-assessment, it may have a future use as a supervision tool for review with a consultant or colleague.
The guide incorporates three broad domains of potential bias. First are those influences specific to each individual evaluator, which may arise out of personal experiences or attitudes. Examined next are influences resulting from the interaction of individual and contextual factors in a particular case. For example, a given defendant or the facts of a crime may arouse emotions or otherwise touch on one's strongly held internal beliefs. Of course, different cases will contain various degrees of ambiguity, which may lead to differing levels of confidence or uncertainty in one's conclusions. Every case will differ in the legal and psychological boundaries of the evaluation and may have external pressures, such as media involvement. Finally, the context in which an evaluator works may have an influence through organizational culture and social pressure. While it is impossible to address every influence on one's work, these considerations of individual, environmental, and case-specific factors may allow for a thoughtful review of potentially biasing sources.
Numerical ratings are not suggested in the guide. In the future, it may be desirable to quantify the degree of influence of any given variable on the prediction of violence. There are six subcategories: one individual, one contextual, and four case-specific. They easily could be self-rated on a three-point scale, in which the anchoring points of zero refer to no influence and two to definite influence. Numerical ratings could be employed to encourage a formal processing of the influences on one's own work product, perhaps resulting in a more thorough and individualized assessment of the pressures on the risk evaluator.
These self-checks are an organized method of supervision or assessment of one's own competency, a method that is especially relevant for professionals who do not have a meaningful and attentive hierarchy of clinical consultants or supervisors in their workplace. The idea of a self-guided assessment is consistent with other areas of psychology in which tracking and testing automatic thoughts will serve to reduce stress.37 Further, periodic self-assessments can be seen as part of staying current in the field by being aware of the influences on one's work product.
Conclusions
The potential for bias in risk assessments should not and does not automatically deter the clinician from performing such evaluations. As stated by Holloway, “the very worst clinical practice does not involve making wrong decisions (however these might be defined), but the failure to make any decisions at all” (Ref. 11, p 144). Risk evaluations should be conducted, and mental health professionals must be willing and able to rise to the challenge. The proposed guide seeks to assist the evaluator in working against the forces that pull toward overprediction.
Footnotes
-
Disclosures of financial or other potential conflicts of interest: None.
- © 2011 American Academy of Psychiatry and the Law
