Letters

Editor:

Newton and associates¹ published a retrospective case-control study of patients who committed either an act of serious violence or three acts of less serious violence during admission to the acute care unit of John Umstead Hospital, Butner, North Carolina. They emphasize in the paper's abstract that, by using easily collected clinical data, clinicians can correctly categorize 80 percent of patients as either having or not having tendencies toward difficult-to-manage violence. Although it is clear that the combination of risk factors they describe can be used to define a group of patients at a significantly increased probability of violence, we would like to make two points of clarification.

First, the proportion of patients correctly classified by a risk assessment tool is not necessarily a helpful measure. For example, a risk assessment that categorizes every patient as at a low risk of violence would be correct 95 percent of the time in a population with a base rate of violence of 5 percent.

The most widely accepted measure of the ability of a risk assessment to discriminate between high- and low-risk individuals is the area under the receiver operating curve (AUC), which is the probability that a randomly selected violent patient will have a higher risk score than will a randomly selected nonviolent patient.² In the footnote to Table 2, Newton and associates report an impressive AUC of 0.881, indicating that the variables they examined could strongly statistically differentiate populations of violent and nonviolent patients. One feature of the AUC is that it is not affected by base rate considerations, which are central to our second point about the recent study. In contrast to the proportion of correctly classified patients and the AUC, it is the proportion of high-risk patients who go on to be violent that is the central test of the clinical usefulness of a high-risk categorization.^3,4 This proportion, the positive predictive value (PPV), can be calculated by using sensitivity, specificity, and base rate.

A sensitivity of 0.74 and a specificity of 0.85 can be derived from the data reported by Newton and associates. John Umstead Hospital is a very large, state-run mental health facility, and we understand from the authors of the recent paper that the acute unit had more than 10,000 admissions during the study period. Assuming that this figure included more than 2,000 individual patients, the base rate of difficult-to-manage violent patients was below five percent. With a base rate of 5 percent and the reported sensitivity and specificity, a PPV of 20 percent can be estimated. This means that approximately 80 percent of patients who are regarded as high-risk will not become violent, whereas 20 percent of high-risk patients will become violent. If the base rate of violence were lower than five percent, the degree of certainty in the high-risk categorizations, expressed as the PPV, would be lower still.

During the mid-17th century, the English clergyman Thomas Bayes considered the degree of certainty that an observer can have in the probability of future events after observing nothing more than their previous occurrences and non-occurrences.⁵ Part of Bayes' answer, now immortalized as Bayes' Theorem, was that belief in contingent probability (in this case the contingent probability of a high-risk categorization) depends on belief in prior probability (in the present case, the incidence of difficult-to-manage violence in the population of patients). In contemporary terms, the positive predictive value of a risk assessment depends not only on its psychometric properties (measured by the AUC or another indicator of effect size) but on the base rate.^3,4 It follows that the usefulness of a risk assessment can never be separated from base rate considerations. Newton et al. have illustrated that even a powerful statistical test of future violence has a limited utility when rare and more severe acts of violence are considered.

Violence against fellow patients and staff is a major problem that faces psychiatric hospitals all over the world. However, there is an inverse relationship between the severity of violence and its incidence. Very severe violence resulting in permanent injury or even death is fortunately rare,⁶ while more minor violence can be regarded as common. Furthermore, base rates of violence vary over time and between settings and can be known with certainty only in retrospect. It follows that the predictive value of risk categories for severe violence is always going to be both low and, to some degree, uncertain. After an episode of severe violence, it is sometimes assumed that the event could have been anticipated and avoided. However, risk assessment cannot provide certain or accurate predictions of rare and severe harm. Instead, as Bayes suggested, the purpose of a risk assessment is to modify our prior beliefs about future harm with systematically collected data.