This study evaluated the Brief Rating of Aggression by Children and Adolescents–Preliminary Version (BRACHA 0.8), an actuarial method of assessing the risk of aggressive behavior by hospitalized children and adolescents. Licensed psychiatric social workers used a 16-item questionnaire to assess all patients seen in the emergency department (ED) of a major urban children's hospital. Over a six-month period, 418 patients (age range, 3.5–19.0 years) underwent psychiatric hospitalization after ED evaluation. The hospital nursing staff recorded the inpatients' behavior, with the Overt Aggression Scale (OAS). Inpatients were deemed aggressive if, during the first six days of their hospital stay, they scored one or higher on any OAS subscale. We evaluated questionnaire properties, items, and demographic covariates (e.g., age, sex, and living situation) by using factor analyses, logistic regression models, and receiver operating characteristic (ROC) methods. A total of 292 aggressive acts were committed by 120 (29% of 418) patients. Fourteen of the 16 items predicted (p < .007) inpatient aggression and showed good internal consistency (Cronbach's α = 0.837). Age was inversely related to probability of aggression and was incorporated into the final assessment instrument. Predictive power was comparable with other published risk assessment instruments (ROC areas of .75 for any aggression and .82 for aggression toward others). BRACHA 0.8 shows promise in rapidly assessing risk of inpatient aggression, but further studies are needed to establish the reliability and validity of the instrument.
Managing aggressive patients is a significant concern for child and adolescent inpatient psychiatric units.1–4 Shorter hospital stays, reduced funding for mental health services, and an increasing number of aggressive patients have made assaults against psychiatrists and nursing staff a significant occupational health risk.5,6 A prospective study of a state psychiatric hospital found that, over a two-month period, 33 percent of the hospitalized children and adolescents were aggressive to others.7 Assaults by patients may result in physical and/or emotional injury to hospital personnel.8,9 Seclusion and restraint, two common responses to episodes of inpatient aggression,10 can be psychologically traumatizing11 and occasionally result in injury or fatality.12
Although aggression on child and adolescent psychiatric units is troublingly frequent, only a few studies provide clues that might help clinicians to assess their patients' risk of violence. Several features limit the usefulness of these studies: relatively small sample sizes,2–4,13 predominantly male or white samples,2,3,14 unstandardized lengths of hospital stay,3 lack of differentiation between verbal and interpersonal aggression,13 retrospective designs,3,15 weekly rather than daily aggression ratings,4 and samples with restricted age ranges (6–12 years13,16; 12–18 years17).
Violence risk assessment tools for adults have received extensive attention,18,19 but to our knowledge, no studies have been conducted to examine tools applicable to in-hospital aggression by child and adolescent psychiatric inpatients. The Structured Assessment of Violence Risk in Youth (SAVRY) evaluates risks of recidivism in juvenile offenders, but its focus is on long-term outcomes for adolescent outpatient populations.20–22 Research applicable to aggression by child and adolescent inpatients is scarce, because most studies look at longer term outcomes in residential2 and forensic settings.17,23
A screening instrument that would quickly identify patients at elevated risk of being aggressive might lower the risk of injuries, help hospital staff implement measures aimed at averting violence, and reduce the use and hazards of seclusion and restraint. This article describes the development to date and predictive effectiveness of the Brief Rating of Aggression by Children and Adolescents–Preliminary Version (BRACHA 0.8), a tool for assessing risk of aggression on inpatient psychiatric units.
Before the present study, we developed a 67-item list of potential risk factors for physical or verbal aggression during hospitalization. We selected the 67 items from risk factors for aggression identified in previous studies2,3,7,13,16,24 and from risk and protective factors suggested by a committee of psychiatric clinicians (including psychiatrists, nurses, and social workers) at Cincinnati Children's Hospital Medical Center (CCHMC).
We piloted the 67 items with 289 patients in a quality-of-care project, to select the strongest risk indicators for aggression and violence. (As a quality-of-care project, this pilot study was deemed exempt by the CCHMC Institutional Review Board [IRB].) Forty percent (115/289) of the patients exhibited some form of aggression, and 14 of the 67 items were significantly associated (p < .05) with aggression. (More information about the pilot phase is available from the first author upon request.) Despite the lack of significance, two historical questions on sexual and physical abuse were retained in a second version of the scale, because clinical experience and some of the literature have identified maltreatment as a risk factor for aggression.3 Thus, the versions of the BRACHA evaluated in the current study consisted of 16 questions with yes or no responses (Table 1), plus easily obtained demographic data (birth date, sex, race, ethnicity, living arrangements, and type of insurance, a proxy for socioeconomic status) that we thought might be related to aggression (Table 2). Before continuation of the project, the CCHMC IRB reviewed the study again and reapproved it with a waiver of signed consent. Specific consent for BRACHA administration or OAS scoring was not required of patients or their guardians, because these instruments had become part of routine clinical care, and clinicians were using the instruments to make treatment and management decisions.
Licensed psychiatric social workers administered the BRACHA to all patients during the emergency department (ED) admission process at CCHMC. Social workers received training on the instrument using live patient interviews of children, adolescents, and their guardians. Interviewers obtained answers to BRACHA questions primarily from the patients' parents or guardians, although information sometimes also came from patients or collateral sources.
The hospital where patients underwent treatment has 67 inpatient psychiatric beds divided into four units. The children and adolescents were placed on separate units divided by age (3–7 years, 8–11 years, and 12+ years). Before admission and bed assignment, the patients were classified into three risk groups based on their 16-item scores. Some adolescent patients in the highest risk category (those with scores of ≥12 of a possible 16) were placed on a safety unit that had higher levels of staffing. For safety reasons, unit staff members were not masked to patients' risk categories.
Hospital staff members used the Overt Aggression Scale (OAS)25 to record inpatients' behavioral outcomes. The OAS characterizes an individual's verbal and physical aggression toward self, other persons, and objects. Using the OAS, a rater records the presence or absence of four categories of aggression: verbal aggression, physical aggression against self, physical aggression against other people, and physical aggression against objects. If an incident of aggression occurs, the rater assigns a score on a four-point severity scale anchored with behavioral descriptors. The OAS was developed for children and adults in the original studies, which established reliability with intraclass correlation coefficients (ICC) of 0.50 for verbal aggression and 0.81–1.00 for physical aggression.25 A subsequent study confirmed the usefulness and appropriateness of the OAS for assessing aggression on a child psychiatric unit.26 The OAS has been used in child and adolescent inpatient studies27,28 and adult inpatient studies29 to record and measure aggression.
In this article, we use the term aggression to designate any threatening verbal or physical behavior toward self, other people, or objects that would generate a score of 1 or higher on any subscale of the OAS. Examples of behavior that would earn a score of 1 for aggression include “shout angrily,” “hit self,” or “slam door.” We use the term violence to refer to actions that would generate a score of 1 or above on the “physical aggression toward other people” subscale of the OAS. Examples of behavior that would earn a score of 1 for violence include “make threatening gesture” or “swing at people.”25
Hospital staff members were trained to use the OAS with vignettes. An OAS was completed for each patient after day (7 a.m.–3 p.m.) and evening (3 p.m.–11 p.m.) shifts and also after any aggressive episode. (OAS ratings were not obtained on the overnight shift because the expected incidence of aggression during this time was low.) At discharge, board-certified child and adolescent psychiatrists working on the inpatient units provided diagnoses for each patient based on Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision (DSM-IV-TR)30 criteria using all available clinical information (interviews, historical information, observation, psychological testing, and collateral information from caregivers and outside treatment providers).
This study reports on findings for patients admitted during a six-month period beginning April 22, 2006, the first day the 16-item BRACHA questionnaire went into use. Data collection ended on October 31, 2006. During this period, CCHMC admitted 458 children and adolescents to psychiatric units. Nineteen patients were dropped from the analyses because of missing information about their living arrangements. Of the 145 patients for whom staff members documented aggressive episodes, 120 (82.8%) had their first aggressive incidents in the first six days of hospitalization. Based on this finding and on the clinical reality that most psychiatric inpatients have relatively short durations of stay, we limited analyses to aggression occurring in the first six hospital days.
During a six-day hospitalization, staff members should have completed at least 12 OASs (2 per day) for each patient. However, not all the OASs were completed. To avoid bias that might arise from having multiple missing OASs, we excluded from our analyses those patients who were missing six or more OASs. We chose six as the exclusion threshold, because during a six-day stay, a maximum of six OASs could be missing for legitimate reasons. For example, a patient who arrived at the hospital late at night, had weekend passes, or had an early morning discharge would not have an OAS for these periods. Twenty-one patients were excluded because six or more OAS ratings were missing, leaving a final sample of 418 patients. We found no evidence that the BRACHA 0.8 scores of the 21 excluded subjects differed from the scores of the other 418 subjects (p = .23, Wilcoxon two-sample test). Patients ranged in age from 3.5 to 19.1 years, with a mean ± SD age of 13.5 ± 3.4 years. One hundred ninety-one (45.7%) patients in the sample were female. Most patients (n = 258, 61.7%) were Caucasian (Table 2). All 418 patients in the sample had been admitted through the ED at CCHMC. No patient underwent more than one admission during the study period.
Data Collection and Analysis
All BRACHA and OAS data were entered by TeleForm. We evaluated each of the 16 questionnaire items and the patient's demographic information as potential predictors of aggression and violence (as previously defined). We treated age as a continuous variable, because inspection of the receiver operating characteristic (ROC) curve revealed a gradually decreasing slope. Data recording forms included several options concerning living arrangements (Table 2). Though studies indicate that caregiver burden in single-parent homes is higher than in two-parent households, we found that differences in aggression rates between patients who lived with one versus two biological parents did not differ significantly (χ2 = 5.06, df = 1, p = .024) after taking multiple comparisons into consideration. We found a clearer difference, however, when we compared aggression rates of patients who lived with at least one biological parent with aggression rates for patients in other living situations (χ2 = 8.88, df = 1, p < .0029). This being the case, we treated living arrangement as a dichotomous variable in subsequent data analyses (that is, living with one or two biological parent(s) = 1, other living arrangements = 0). Literature indicating that separation from a parent and lack of parental support are predictive of aggressive behavior supports this approach.13,31 We used the chi-square and Fisher's exact tests to learn whether sex, race, and various diagnoses predicted aggression or violence.
Ninety-eight percent of the patients (n = 410) had at least one primary or secondary diagnosis in the following categories: anxiety disorders and adjustment disorders (28.5%, n = 119), attention deficit hyperactivity disorder and impulse control disorders (37.1%, n = 155), disruptive behavior disorders (24.9%, n = 104), mood disorders (66.%, n = 276), developmental disorders (3.6%, n = 15), schizophrenia and other psychotic disorders (14.6%, n = 61), and mental retardation (4.3%, n = 18) (Table 3). Anxiety disorders and adjustment disorders were categorized together, since they often overlap and are difficult to distinguish during hospitalizations after acute stressors. The remaining eight patients had a substance use disorder, an eating disorder, or both.
Aggressive Versus Nonaggressive Patients
Table 2 summarizes relationships between demographic factors and occurrence of any aggression. During the first six days of hospitalization, 28.7 percent (120/418) of the patients committed a total of 292 aggressive acts. Of the 292 incidents, 102 (34.9%) were verbal, 81 (27.7%) were toward objects, 63 (21.6%) were toward others, and 46 (15.8%) were toward self. Forty (89%) of those patients who engaged in aggression toward self also engaged in other types of aggression, which suggests that in the hospital setting, harm to self shares some common risk factors with outwardly directed aggression. Patients with diagnoses of attention deficit hyperactivity disorder or other impulse control disorders were more likely to be aggressive (p < .0001), as were patients with disruptive behavior disorders (p = .0013). Associations with other conditions (e.g., mood disorders) were not significant after adjustment of significance levels for multiple comparisons (Table 3).
All questionnaire items were significantly associated with aggression except for the two questions about personal abuse history (questions 4 and 5; Table 1). The 16 items had good internal consistency (Cronbach's α = 0.818), and internal consistency improved when the two nonpredictive items were deleted (Cronbach's α = 0.837). Although these values suggest that the scale is unidimensional, we conducted factor analyses on the 14 predictive items. Exploratory factor analysis with an orthogonal or varimax rotation indicated a three-factor solution: personal history (questions 1–3, 6, and 12–16); past violence (questions 9–11); and impulsiveness in the ED (questions 7 and 8). Kaiser's Measure of Sampling Adequacy (MSA) was 0.82, a good indicator that the data are appropriate for the common factor model. Internal consistency for each factor was good, with Cronbach's α = 0.79 for Factor 1, α = 0.84 for Factor 2, and α = 0.78 for Factor 3.
Evaluation of Risk Assessment Potential
As Table 2 indicates, several demographic variables were clearly associated with aggression during hospitalization. Thus, an important threshold problem in evaluating our questionnaire was to see whether the responses to the 14 items added any predictive power to what one might conclude from demographic data that are quickly and reliably obtainable. We decided not to evaluate race as a potential predictive item, in part because we thought that apparent race-related differences in aggression rates might reflect race-related differences in interpreting and scoring aggressive incidents. Using stepwise, backward-elimination logistic regression models, we evaluated the potential predictive role of the remaining demographic factors. Results showed that when age and responses to the 14-item scale were included, the other three demographic factors added no predictive value. We therefore evaluated prediction models that used only patients' ages and questionnaire results to gauge the risk of aggression or violence.
The following equation describes the BRACHA 0.7 prediction model: (1)
As Equation 1 shows, one computes a BRACHA 0.7 score by subtracting a patient's age from 20 plus the total number of questionnaire items answered yes. (The constant 20 simply assures that the total score will be positive.) BRACHA 0.7 has 14 items, and the patients' age range is 3 to 19 years. BRACHA 0.7 scores thus range from 1 to 31, with aggression risk increasing as the score increases. Notice that BRACHA 0.7 gives roughly equal predictive weight to a patient's age (which has a range of 16 years) and the 14-item score; age is subtracted in the computation, because age is inversely related to likelihood of aggression.
The second model, BRACHA 0.8, takes advantage of the three component factors discussed above, while aiming to avoid overfitting that might result from using β-weights generated from a logistic regression using our 418-patient sample. One calculates a BRACHA 0.8 score thus: (2)
Equation 2 implies that a yes answer to a given questionnaire item contributes to the prediction equation in inverse proportion to the number of items in the factor of which the given item is an element. For example, Item 1 is one of nine elements of Factor 1, and so a yes in response to Item 1 adds five-ninths of a point to the total score. Similarly, a yes answer for one of the two items making up Factor 3 adds 2 points to the total score. Scores on the BRACHA 0.8 can range from 1 to 32. One multiples the three factor sums by 5 so that, as with BRACHA 0.7, age and questionnaire answers contribute roughly equally to the total risk assessment score.
We used receiver operating characteristic (ROC) analysis to evaluate the performance of BRACHA 0.7 and 0.8 and to compare their accuracy with two simpler predictors: patients' ages and patients' total scores on the 14 items from the questionnaire. ROC analysis is the appropriate method for evaluating the performance of a risk assessment tool for which possible outcomes are arrayed along a continuous or ordinal scale.32 A ROC graph plots test sensitivity (equal to the true-positive rate, tpr) as a function of the false-positive rate (fpr = 1 − specificity). The area under the ROC curve (AUC) is a useful summary measure of test performance. In this context, AUC equals the probability that the instrument will assign a patient randomly chosen from the aggressive subgroup a higher score (i.e., a higher likelihood of aggression) than a patient randomly chosen from the nonaggressive subgroup. Given such a pair of patients, an instrument with no discriminative power would perform no better than flipping a coin to classify them and would have an AUC of 0.5. An instrument that always assigned a higher score to the aggressive patient would have an AUC of 1.0. Adult studies examining performance of actuarial risk assessment instruments typically report AUCs of 0.65 to 0.80.19,32 A recent study of actuarial risk assessments for violent offending in 74 teenage outpatients reported an AUC range of 0.71 to 0.73.24
We calculated AUCs for BRACHA 0.7 and 0.8 by using ROCKIT software.33 This package permits comparison of correlated ROC curves, an important consideration when, as in the present study, accuracy parameters are derived from applying two or more assessment methods to the same subjects. Table 4 describes the performance of BRACHA 0.7 and 0.8 in assessing risk of aggression and violence and shows pair-wise comparisons of these scales' performances with what one would expect by simply using age or the 14 items by themselves as risk assessment tools. Figures 1 and 2 show the ROC curves for age alone, the 14 items, and BRACHA 0.7, and 0.8.
In assessing risk of aggression toward others, age alone was a good predictor (AUC = 0.7691) but not significantly greater than the 14-item score (AUC = 0.7154). However, combining age and the 14 items yielded predictive formulae with AUCs above 0.8 and, in the case of BRACHA 0.8, accuracy was significantly higher than the 14-item score (p < .01) or age alone (p < .05).
AUC values provide a global measure of discrimination power, but they do not directly help clinicians grasp the practical effect of using an assessment method. One way to appreciate what an assessment tool might do is to suppose that a practitioner picked a single cutoff score to differentiate between patients at high and low risk for aggressive or violent behavior. Methods of cutoff selection include maximizing sensitivity plus specificity or utility (benefits minus costs). However, the former method is insensitive to base rate considerations, and the latter method requires establishing precise values for true positives and true negatives and precise costs for false positives and false negatives. Establishing these precise amounts usually is impossible, but a reasonable, practicable alternative is to find and evaluate the consequences of using the cutoff that yields maximum diagnostic information (MDI).
To select the MDI cutoff for any form of aggression, we used the base rate in our sample (120/418 = 0.287), binormal accuracy parameters generated by the ROCKIT software, and the cutoff location procedure described by Somoza and Mossman.34 The (fpr, tpr) pair that maximized diagnostic information was (0.431, 0.802; Fig. 1), which corresponds to a BRACHA 0.8 score of 13. This operating point would divide the patients into two groups: a 224-member higher-risk group with a 43 percent rate of aggression (that is, a group containing 96 actually violent and 128 actually nonviolent patients) and a 194-member lower-risk group with a 12 percent rate of aggression (170 actually nonviolent and 24 actually violent patients). Using the same process to find the MDI cutoff for violence (base rate = 63/418 = 0.151) led to a MDI cutoff at (fpr, tpr) = (0.327, 0.822; Fig. 2), which corresponds to a BRACHA 0.8 score of 14. This operating point would divide patients into a 168-member higher risk group in which 52 (31%) patients were actually violent and 116 were not, and a 250-member, lower risk group of whom 9 (3.6%) patients were actually violent and 241 were not.
Readers should note that, in this article, we have used the MDI cutoff primarily for demonstration purposes. An interesting consequence of using either 13 or 14 as a cutoff for the BRACHA 0.8 is that children ages 7 and under automatically fall into the high-risk group. While such a categorization reflects the strong inverse relationship between age and aggression risk, using a single cutoff for all ages might not be useful in actual clinical contexts.
Our findings suggest that the BRACHA can help ED clinicians rapidly categorize children and adolescents into groups with distinctly higher and lower risks for aggression and violence during hospitalization, which in turn would help ED clinicians convey an empirically validated assessment of risk for inpatient aggression and violence to the inpatient unit that will receive the patient. Previous research has found that several demographic, clinical, and historical factors are indicative of future aggressive behavior, but these factors do not necessarily predict inpatient aggression during an acute inpatient stay. To our knowledge, this study is the first to find that a younger age is associated with a higher risk for inpatient aggression and violence.
Past research has found that certain diagnoses, including disruptive behavior disorders,2 conduct disorder,18 specific developmental disorders,18 and mental retardation,35 increase the risk of aggression. Other studies suggest mixed effects for certain diagnoses. For example, Knox and colleagues36 suggest that an internalizing disorder (e.g., depression or anxiety) reduces the likelihood of aggression by males, but may be a risk factor for aggression by females. Although the present study supports previous findings about disruptive behavior disorders and aggression, we intentionally omitted diagnostic status from our predictive model. Our reasoning was that child psychiatric diagnoses have limited accuracy and reliability when based solely on information available in the ED. Also, discharge diagnoses may have reflected information contained in OAS (aggression) scores, which could have the effect of inflating the association between particular diagnoses and aggression. We instead examined information and potential predictive factors that clinicians can assess objectively even during crises that precipitate psychiatric hospitalization.
A history of aggressive behavior is one of the most commonly cited risk factors for aggression and violence,32,35,37–39 and impulsiveness combined with aggression is a strong predictor of future aggression.9,35,40 In adults, adolescents, and children, aggressive acts committed just before admission predict violence during subsequent psychiatric hospitalization.3,13,17,38 The present study confirms these findings: all BRACHA items relating to interpersonal violence or hostility were strong predictors of in-hospital aggression, as was a history of property destruction.
Other investigators have found that victimization via physical or sexual abuse has a modest correlation with future aggression in referred youths,40 but we did not find an association between these elements of inpatients' histories and inpatient aggression.
We note several limitations to our conclusions. First, inpatient clinicians used responses from the 16-item questionnaire (particularly the total of positive items) throughout the study period as an aid in clinical management. Although we calculated BRACHA 0.7 and 0.8 scores after data collection, staff members who made OAS ratings were not blind to the number of items answered “yes.” Therefore, patients' risk scores may have influenced reactions, perceptions, and data recording by staff members.
Second, some adolescent patients with 12 or more “yes” responses on the 16-item questionnaire were placed on a safety unit with increased staffing based on the expectation that the scale had predictive validity. Although this precaution appeared to be ethically required, it may have prevented or reduced aggressiveness in higher risk patients, which would have had the effect of reducing apparent accuracy of the assessment instrument. Despite implementation of placement precautions and other inpatient safety measures, we still found a clear association between the number of items answered yes and subsequent inpatient aggression. This association might have been even stronger had clinicians not used information contained in our assessment tool.
Third, compliance checks for OAS recording and training in use of the assessment questionnaire were not ongoing during the study. Regarding OAS compliance, missing data prevented the inclusion of some otherwise eligible patients; for other patients, we expect that data were not recorded as often as they should have been. These omissions may have occurred because patients had no aggressive incidents to record, but we cannot be sure. Although we have no reason to suspect that these omissions introduced systematic bias, it is possible that they did.
Fourth, hospital unit placements for children and adolescents depended on their age category. Although assignment of patients to separate child and adolescent units comports with reasonable current practice, the strong inverse relationship between age and aggression may partly reflect differences in how units evaluated and rated incidents.
Fifth, our study took place at a single institution with a unique referral pattern. The concordance between risk factors suggests that these findings may be generalizable to other similar settings, but the degree to which this is true is unknown. We hope that future studies will include other locations and settings to let us evaluate the generalizability of the BRACHA.
Sixth, the relatively good predictive power of BRACHA 0.7 and especially BRACHA 0.8 may reflect a fortuitous weighting of the elements that happens to fit the data. Of course, any combination of data elements requires some weighting; even using unit weights reflects a choice about weighting. For BRACHA 0.7, we simply combined age with the 14-item score and hoped that this simple formula would be robust to other data sets. In BRACHA 0.8, each factor is weighted identically, and questionnaire items contribute in inverse proportion to the number of constituents in the factor to which they belong. Yet, the factor structure derived from our current data may not apply to future data sets, and our methods of combining items need additional evaluation.
Seventh, we recognize that several questionnaire items lack simple operational definitions and could be interpreted differently by individual clinicians. To compensate, we are currently conducting additional research, including an interrater reliability study using a version of the BRACHA with improved wording and a scoring system that allows evaluators to give specified, graded responses.
Finally, the current study focused on how well BRACHA 0.7 and 0.8 assessed the likelihood of aggression or interpersonal violence occurring within the first six days of admission. Although diagnostic performance was very good, we did not evaluate whether BRACHA 0.7 or 0.8 could gauge the frequency and severity of aggressive incidents, which have practical consequences just as important as knowing whether aggression will occur. Having reached the findings described here, we plan future studies with stricter compliance monitoring to learn whether the BRACHA can make meaningful assessments about potential frequency and severity of aggression.
Despite these limitations, our study suggests the potential usefulness, accuracy, and practicality of the BRACHA as an assessment tool. A combination of readily assessable background factors, behavior patterns, and age appears to sort child and adolescent psychiatric inpatients into different levels of risk for post-admission aggression. If future research confirms our initial findings, the BRACHA may ultimately help clinicians improve safety in hospitals, reduce the use of seclusion and restraint, and focus interventions on reducing aggression-related risk.
Disclosures of financial or other potential conflicts of interest: None.
- American Academy of Psychiatry and the Law