The Performance of Incarcerated Juveniles on the MacArthur Competence Assessment Tool-Criminal Adjudication (MacCAT-CA) ======================================================================================================================= * Susan LaVelle Ficke * Kathleen J. Hart * Paul A. Deardorff ## Abstract The MacArthur Competency Assessment Tool-Criminal Adjudication (MacCAT-CA) is a structured interview that assesses abilities related to an individual's competency to stand trial (CST). In the present study, we examined the performance of 247 juvenile offenders on the scales of the MacCAT-CA (Understanding, Reasoning, and Appreciation), along with several other variables (age, IQ, achievement level, experience with the juvenile justice system, and a screen for psychopathology) that may be related to CST. In general, results suggest that performance on the MacCAT-CA varied with age, with younger participants performing significantly worse than older juveniles. When compared with the normative data, the juveniles in the present sample demonstrated deficits in court-related skills measured by the MacCAT-CA across all age ranges. In addition, several other variables, including achievement level, intelligence level, and psychopathology, were differentially related to the three scales of the MacCAT-CA. The original mission of juvenile courts was to rehabilitate and treat troubled youths, but many researchers1 and observers2 have argued that several factors over the past 40 years have altered fundamentally the mission and consequences of juvenile court proceedings. Beginning with the procedural changes that came with the Supreme Court decision rendered in *In re Gault*,3 juvenile courts arguably have drifted from the informal meetings designed to counsel and help youths to highly formal proceedings that are barely distinguishable from “adult” court. In the 1990s, public concern about increases in juvenile crime brought further changes to juvenile courts, including legislative and other changes that allow for longer sentences for juveniles and more aggressive prosecution of juvenile crime in many jurisdictions.4 In response, defense attorneys appear to be questioning youths' competence to stand trial (CST) with greater frequency in some jurisdictions.5 Until recently, this topic received relatively little research attention. CST for juvenile and adult defendants is based on the legal standard established by the U.S. Supreme Court decision in *Dusky v. U.S.*6 This standard requires that a trial defendant have an “understanding of the proceedings against him” and be able to “consult with his lawyer with a reasonable degree of understanding,” to be deemed as having CST. Not all state courts have explicitly acknowledged that the right to be competent extends to juveniles,7 but rulings (such as *Kent v. U.S.*8 and *In re Gault*3) that extend other due process rights to juveniles are assumed to include the need for CST for juveniles in most jurisdictions. How CST should be understood in juvenile proceedings is less well articulated, however. Some courts9 assume that age-related considerations are implicit in juvenile court proceedings. In addition, although most states acknowledge mental retardation and mental illness as reasons underlying a defendant's incompetence,7 there is limited court precedence regarding the role that cognitive maturity should play when considering CST. Some scholars have argued that cognitive and psychosocial immaturity are the most likely causes of lack of CST in juveniles. For example, Grisso10 has identified age as a feature that should raise questions about CST. Of the available studies on CST in juveniles, several have examined the relationship between age and CST-related skills, and there is a consistent finding that age is related to CST judgments and abilities.11–16 For example, McKee16 explored the differences in CST abilities between adults and juveniles ages 7 to 16 who were ordered by courts to undergo competency evaluations. He found that 85 percent of the youths had been judged to be competent, compared with 96 percent of an adult sample. Age effects were also found, with the older adolescents demonstrating better understanding and appreciation of legal issues than the younger ones. In addition, the older adolescents (15–16 years old) were more likely to be judged competent (94.6%) than the younger (12 and younger) group (50%). Unfortunately, this study (like many of the early studies on CST) relied on the court's judgment of CST as a primary variable, and this judgment may well be influenced by the variables identified as significant. That is, finding that younger children are more commonly found to be incompetent may be due to the court's determination (based on the evaluator's suggestion) that age is a relevant factor; it is not an independent finding that age is, in fact, related to impairment in competence-related skills. Some studies have evaluated CST in a less circular manner. Cooper17 administered the Georgia Court Competence Test-Juvenile Revision to a sample of adjudicated youths of ages 11 to 16. In addition to finding a significant relationship between age and CST, she found that a great majority (89.2%) of the sample exhibited significant deficits in CST-related knowledge. Those youths with impairments underwent training in trial-related matters and were retested. Although the post-test scores improved significantly over the pre-training scores, only 11 percent of the participants obtained scores that indicated “competent” level understanding or skills based on the criteria established by Cooper. Another recent study18 found similar court-related knowledge deficits in a sample of urban youths, a large percentage of whom (25%) had had contact with the juvenile court. Certainly, the results of these studies suggest that there is reason to question the CST-related skills of many youths. Age is often used as a variable in developmental studies, but in the case of CST, we believe that age serves as a substitute for the “true” variable of interest, cognitive maturity. Here, we use cognitive maturity to describe a variety of thinking abilities that change over the course of development, including the ability to anticipate the consequences of one's actions or decisions, reasoning, decision-making, and judgment. Others19 have used the term “psychosocial maturity” to encompass other related skills such as assessment of risk, conformity to peers, and time perspective that may also have relevance in CST. We believe it likely that both cognitive maturity and psychosocial maturity are related to intelligence, and research has consistently supported the relationship between IQ and CST in both adult20 and adolescent samples.17,21 Other variables have been found to relate to CST, legal decision-making and court knowledge in children and adults to varying degrees and with varying consistency. These include the variables of psychopathology, juvenile justice history, and academic achievement skills.12–17,22,23 Several assessment devices of varying structures and lengths have been developed in an attempt to quantify and measure CST-related abilities. The MacArthur Competence Assessment Tool-Criminal Adjudication (MacCAT-CA)24 was developed to correct what have been viewed as the deficiencies of most other instruments. It is a measure rooted in theory, with standardized administration and scoring and strong psychometric properties, and involves the assessment of abilities beyond just legal knowledge. Although relatively new, it is backed by strong research in its development.25 Administration involves the presentation of a hypothetical crime situation followed by subsequent structured questions tapping three areas (Understanding, Reasoning, and Appreciation), which yield three separate scores. The first section, Understanding, consists of questions regarding factual knowledge of trial information and the roles of those involved. The Reasoning section asks the defendant to choose the more relevant between two pieces of information related to the hypothetical court case and to make a plea decision for the main character. The examinee's response is scored based on his or her choice and on the reasoning supplied for that choice. This score provides information relevant to the examinee's ability to consult with an attorney. The Appreciation section assesses the examinee's ability to recognize how aspects of the legal system apply to his or her particular case, and attempts to assess the defendant's implausible or delusional thinking related to his or her case. The MacCAT-CA offers norms based on a sample of adult defendants,24 but its use with juvenile samples is limited to four very recent studies. Redlich *et al.*26 evaluated several features affecting competent court participation of a small sample of juveniles (*n* = 18; ages 14–17 years) and young adults (*n* = 17; ages 18–25 years). Of relevance to the present study is their examination of the relationship of suggestibility, age, school grades, and frequency of police contacts to performance on the MacCAT-CA. None of these variables was significantly related to any of the scores of the MacCAT-CA at *p* < .05. The authors computed a “total” competence score from the MacCAT-CA and found that having below-average grades and a tendency to change answers after receiving negative feedback (a measure of suggestibility) were related to lower total scores on the MacCAT-CA. These findings must be interpreted cautiously, however, in light of the small sample size and the use of the “total” MacCAT-CA score, which is not described in the manual.24 Burnett and colleagues27 examined the MacCAT-CA scores of a sample of 110 youths ages 10 to 17 years. Seventy (64%) of the participants were awaiting adjudication in juvenile court, with the remaining 40 (36%) recruited from the community. They examined the scores by breaking the sample into age groups, but the number of youths in some groups was very small (e.g., among the 10–12-year-olds, there were only three who were awaiting adjudication), and the community and adjudication groups differed significantly in intellectual level. Both of these factors confound the interpretation of their findings. Nonetheless, they found that the Understanding and Reasoning scores of both the adjudication and community samples were significantly lower than the adult norms; the adjudication group also obtained significantly lower Appreciation scores than did the adult norms. When evaluating their sample by age, they found that youths below the age of 17 obtained significantly lower Understanding and Reasoning scores than did the adult normative sample. The 17-year-old group did not differ from the adult normative sample on any of the scores. On the Appreciation score, only the 10- to 12- and 13- to 14-year-old groups obtained scores that were significantly lower than the adult normative group. The presence of psychopathology is a common reason for lacking CST in adults, but has received little attention in studies of juveniles. Warren and colleagues23 examined 120 youths ages 10 to 17 years who were psychiatrically hospitalized; over half (*n* = 67) reported previous contact with the juvenile justice system. All youths were administered the MacCAT-CA and measures of intelligence (the K-BIT) and psychopathology (psychiatric diagnosis, the Brief Psychiatric Rating Scale, and the Massachusetts Youth Screening Instrument). Consistent with previous findings, they found significant correlations between IQ and all three scores on the MacCAT-CA; age was significantly correlated with Understanding and Reasoning, but not Appreciation. In addition, they found that diagnoses of Learning Disorder and Mental Retardation were correlated with Understanding and Reasoning scores on the MacCAT-CA. Notably, only four variables correlated with the Appreciation score: verbal and nonverbal IQ scores, the Psychoticism rating from the BPRS, and a diagnosis of Mental Retardation. In their sample (which, notably, had mean IQ scores in the average range), the mean scores on the three portions of the MacCAT-CA were in the No Impairment or Mild Impairment range, indicating that many of the youths in their sample performed at a level comparable with competent adults. There was significant variability in performance, however, with almost half of those youths under 14 years of age demonstrating some degree of difficulty on the MacCAT-CA; even among the 14- to 17-year-olds, the mean score on the Appreciation scale was in the Mildly Impaired range, indicating some degree of difficulty with this feature of the instrument. In the largest study completed to date in this area, Grisso and colleagues28 administered a number of measures related to court participation (including the MacCAT-CA) to a large sample of adolescents (*n* = 927) and young adults (*n* = 466) who were recruited through detention centers and jails and from the community. They found significant differences in performance on all three sections of the MacCAT-CA across the four age groups (11–13, 14–15, 16–17, and 18–24 years). The general trend of those differences was that the youngest group (11–13) performed more poorly than the other groups on all MacCAT-CA sections; the performance of the 14- to 15-year-olds was significantly lower than both of the older groups on the Reasoning and Appreciation sections, respectively. Notably, the effect size for the differences was moderate for the 11- to 13-year-olds when compared with that of the young adults, and the effect sizes for the differences between the 14- to 15-year-olds and the other groups were small. Grisso and colleagues interpreted their findings to suggest that youths under the age of 15 are at risk for competence-related deficits, at least in part because of the high prevalence of IQ scores between 60 and 89, which they found to be “associated with a significant risk of being incompetent to stand trial because of impaired Understanding or Reasoning or both” (Ref. 28, p 350). In separate analyses, they also found significant correlations between MacCAT-CA scores and intelligence test scores, but these scores did not correlate significantly with prior experience in the juvenile justice system or most screening measures of mental health problems. In summary, the CST-related abilities of juveniles remain poorly understood, particularly as they are related to other features common in juvenile offenders, such as young age, low intellectual functioning, school failure, and history of psychiatric disorder. We examined the relationship between the Understanding, Reasoning, and Appreciation scales of the MacCAT-CA with several variables potentially affecting CST (age, IQ, academic achievement, juvenile justice history, and psychopathology), utilizing a sample of incarcerated youths ages 9 to 18 years. The present study expands on previous studies by including younger offenders, a nonhospitalized sample, and measures of academic ability and psychiatric symptoms. The present study also examined how the juveniles' scores on the MacCAT-CA related to impairment categories based on the normative data.24 Based on both the previous studies on CST and well-accepted theories of cognitive development, we expected that MacCAT-CA scores would increase with age. Many youths involved in the juvenile justice system also present features that may reasonably be expected to affect CST, such as poor school achievement and psychopathology, so these features were also measured. In addition, we included a young age group (9–12 years), as some jurisdictions prosecute children at these ages; most studies have not included children of this age level. ## Materials and Methods ### Participants We recruited 247 youths, ages 9 to 18, who were incarcerated at a detention center in a midsized city in the Midwest. The sample did not differ from the general population of the facility in race, but girls were under-represented in the sample (19%) versus the girls served by the facility (25%). Mental health screening of youths when they enter the facility indicates that approximately 20 percent self-report symptoms of anger, depression, or anxiety that warrant further evaluation. About 13 percent of girls and 6 percent of boys indicate suicidal ideation; fewer than 10 percent endorse some symptoms related to thought disorder, rarely at a high level.29 Severely psychiatrically disturbed youths are typically not housed at this facility. Eighty-one percent of the sample was male, which was slightly higher than the general census of the facility, which is 75 percent male. Sixty-six percent of the sample was African American, 31 percent was white, and 2.8 percent was of other racial backgrounds. These percentages did not differ from the general census of the facility. The participants ranged in age from 9 to 18 years with a mean age of 14.7 years (SD = 1.8). For the purposes of some analyses and for ease of data presentation, participants are presented by age groupings (i.e., 9–12, 13–14, 15–16, and 17–18 years) that are based on developmental research and prior competence studies.13,15 We did not eliminate participants on the basis of IQ to create a sample that reflects the full range of juvenile offenders. Issues related to the range of IQ scores will be discussed in the Results section. Further demographic information and test scores are presented by age group in Table 1. One-way analyses of variance (ANOVA) were computed to examine any significant difference among the groups. These results are also presented in Table 1. As can be seen, the groups differed significantly on several variables, including estimated IQ, math and reading skills, and number of charges. Specifically, the oldest group had a higher IQ score than did the three younger groups, which did not differ from each other; the same trend was seen for reading and math skills. The youngest group had significantly higher BPRS-C Externalization scores and significantly lower WRAT-3 Reading scores, but the other three groups did not differ from each other. As would be expected, the oldest groups had significantly more charges than the younger groups. View this table: [Table 1](http://jaapl.org/content/34/3/360/T1) **Table 1** Demographic Variables and Test Scores by Age Group ### Measures #### The MacArthur Competence Assessment Tool-Criminal Adjudication As previously described, the MacCAT-CA24 is a structured interview that assesses an individual's CST-related abilities. This measure presents a scenario about a crime situation followed by questions about the legal system and legal decisions based on that scenario. The measure yields three scores: Understanding, Reasoning, and Appreciation. The MacCAT-CA manual contains normative comparisons for these three scales based on a standardization sample of adult defendants.24 No norms specific to juveniles currently exist. The MacCAT-CA takes approximately 35 to 45 minutes to administer and has demonstrated sound psychometric properties.20 #### Estimated Intelligence To obtain estimates of intelligence, the Vocabulary and Block Design subtests of the Wechsler Intelligence Scale for Children—Third Edition (WISC-III)30 (for youths 16 years and younger) or the Wechsler Adult Intelligence Test—Third Edition (WAIS-III)31 (for 17- and 18-year-olds) were administered to participants. For the present study, the participants' subtests standard scores were converted to estimated full scale IQ scores (mean of 100 and standard deviation of 15) using tables provided by Sattler and Ryan32 for the WAIS-III and WISC-III.33 Use of these tables to provide estimates of intellectual functioning is well supported, particularly for research purposes.32 #### Wide Range Achievement Test-3 The WRAT-334 is a brief measure of academic achievement that assesses basic reading, spelling, and arithmetic skills. It was used in the present study to estimate the participants' academic achievement skills. This test yields standard scores for each of the three sections. The standard scores (mean of 100 and a standard deviation of 15) were used in the present analyses. #### Brief Psychiatric Rating Scale for Children The BPRS-C35 is designed to assess a wide variety of psychiatric symptoms in children 5 to 18 years of age. It has increasingly been used in research and clinical settings.36 Derived from the Brief Psychiatric Rating Scale,37 the BPRS-C consists of 21 symptoms or behaviors rated by the examiner on a seven-point Likert scale. A recent study36 found the BPRS-C to have substantial reliability and concurrent validity, but the authors recommended using three factor-derived scores (Internalization, Externalization, and Developmental Maladjustment) rather than the original seven subscales. Based on this recommendation, these factor scores were used in the present study. The examiners used behavioral observations made during the testing session and information gathered during a brief clinical interview to complete the BPRS-C. #### Demographic Information Data on sex, age (in years and months), race, and grade level were gathered from the participants and verified with information from their records. Participants' current and past offenses were obtained from their records. ### Procedure Permission to conduct this study was granted by the superintendent of the Hamilton County Juvenile Court Youth Center and the Hamilton County Juvenile Court Judges. Xavier University's Institutional Review Board also approved the project. Participants were selected from the daily roster of youths housed in the facility. All youths who were housed in the facility during the 18-month period in which the data were collected were eligible for recruitment. Permission to participate was granted by each youth's parent or guardian, and each youth also provided his or her assent. The participants did not receive any compensation for participation. Of the 382 parents or guardians contacted, 52 (13.6%) declined to participate. Of the 295 youths approached, 48 (16.3%) declined or ceased participation. We found no differences in basic demographic data between those who declined to participate and those who agreed. The testing sessions were conducted by doctoral students trained in test administration in interview rooms on the participant's unit. The sessions generally lasted 60 to 90 minutes. The MacCAT-CA, subtests of the WISC-III or WAIS-III, and WRAT-3 were presented in a counterbalanced order to minimize order effects. A brief interview followed these measures for each participant. Examiners completed the BPRS-C based on this interview and behavioral observations made during testing. ## Results Inter-rater reliability was examined by having 10 percent (25) of the protocols independently scored by a second examiner. Pearson correlation coefficients between the raters' scores (*r* = .96 for Understanding, .93 for Reasoning, and .77 for Appreciation) are consistent with those in past research.20 These reliabilities are generally higher than those obtained in the study of juveniles by Grisso *et al*.28 They found intra-class correlations that ranged across time from .60 to .91 for the Understanding and Reasoning scores and from .17 to .86 for Appreciation. We also examined inter-rater consistency by looking at the agreement of scores within plus or minus 1 point. Using this method, we found 88 percent agreement on Understanding, 84 percent on Reasoning, and 76 percent on Appreciation. Of note, inter-rater reliability and consistency seem to be consistently lower for the Appreciation score across all previous studies. Before we computed other analyses, we compared the performance of the boys and the girls on the MacCAT-CA by performing three ANOVAs, with the Understanding, Reasoning, and Appreciation scores serving as the dependent variables. There were no significant differences between the boys and the girls: Understanding, *F =* .66, *p* = .42; Reasoning, *F =* .39, *p* = .533; and Appreciation, *F =* .008, *p* = .93. Therefore, we computed all further analyses with boys and girls combined. Figure 1 graphically depicts the mean scores of our sample on the MacCAT-CA by age. As shown, the average performance of those 13 years of age or younger was in a range indicative of at least mild impairment on all three portions of the MacCAT-CA. The average score of the Appreciation scale remained in the mild impairment range through age 17. The average Reasoning score was in the minimal or no impairment range after age 14, but the average Understanding score is in or near the mild impairment range through age 16. This level of performance on the Understanding portion is notable, as it suggests that many youths are not able to benefit from the teaching feature of the MacCAT-CA in acquiring basic court-related information. The line depicted is not straight, and suggests a curvilinear relationship between age and MacCAT-CA scores. ![Figure 1](http://jaapl.org/https://jaapl.org/content/jaapl/34/3/360/F1.medium.gif) [Figure 1](http://jaapl.org/content/34/3/360/F1) **Figure 1** Mean MacCAT scores by age. The cut-off for clinically significant impairment is 7 or lower for the Understanding Scale and 8 or lower for the Reasoning and Appreciation Scales. The range of possible scores is 0 to 16 for Understanding and Reasoning and 0 to 12 for Appreciation. We also examined the correlations among the variables used in the present study; the correlation coefficients are presented in Table 2. There was a small but significant relationship between age and estimated IQ, most likely due to the significantly higher estimated IQ score of our oldest age group (see Table 1). Age also correlated significantly with achievement skills and number of charges. The latter relationship is probably related to the higher IQ scores of our older juveniles. Age also correlated significantly with all three of the MacCAT-CA scores. Subsequent regression analyses (using the square of age) indicated that the relationship between age and MacCAT-CA performance is curvilinear: The strongest relationship occurs at younger ages, with the relationship generally leveling off at older ages. Not surprisingly, estimated IQ correlated significantly with academic skills and with all three MacCAT-CA scores. Notably, the Externalizing factor score of the BPRS-C correlated negatively with both age and IQ, indicating that older youths and/or those with higher IQ scores were less likely to manifest behaviors such as overactivity and impulsivity. Most of these correlations are in the small to moderate range. The MacCAT-CA scores correlated moderately with each other (*r* = .45−.53). View this table: [Table 2](http://jaapl.org/content/34/3/360/T2) **Table 2** Correlation Coefficients and Probability Levels for the Study Variables Table 1 presents descriptive information about the performance of our sample on the study measures for participants grouped into four age ranges. There were significant differences between the groups on estimated IQ. *Post hoc* analyses indicated that the oldest age group (17–18 years) had significantly higher IQ scores than the other groups, which did not differ from one another. A similar pattern was seen for the WRAT Reading and Arithmetic scores. There were also significant differences in the BPRS-C Externalization factor score across the age groups. The youngest age group (9–12 years) generally demonstrated greater symptoms of overactivity and impulsivity relative to the other age groups. To examine differences in performance on the MacCAT-CA across age groups, three one-way Analyses of Covariance (ANCOVA) were performed with estimated Full-Scale IQ serving as the covariate. Results indicate a significant difference across age groups for Understanding, Reasoning, and Appreciation (Table 3). *Post hoc* analyses with the Least Significant Difference (LSD) test found that the 9- to 12-year-old group scored significantly lower than all other age groups on all three scales. There were no significant differences between the remaining age groups on any of the scales. View this table: [Table 3](http://jaapl.org/content/34/3/360/T3) **Table 3** MacCAT-CA Scale for Each Age Group We examined the differential impact of several variables on juvenile offenders' performance on the scales of the MacCAT-CA by computing multiple regression equations for each scale (Understanding, Reasoning, and Appreciation). Since the relationship between age and MacCAT-CA scores did not appear to be linear (Fig. 1), we opted to use the square of age to test this feature of the relationship in these analyses. The BPRS-C Internalizing, Externalizing, and Developmental Maladjustment factor scales; age (in months) squared; estimated IQ; WRAT-3 scores (Reading, Spelling, and Arithmetic); and number of charges served as predictor variables, with the Understanding, Reasoning, and Appreciation scales of the MacCAT-CA serving as criterion variables. The results of the final multiple regression analyses are presented in Tables 4, 5, and 6. View this table: [Table 4](http://jaapl.org/content/34/3/360/T4) **Table 4** Multiple Regression Results: Predicting Scores on the MacCAT-CA View this table: [Table 5](http://jaapl.org/content/34/3/360/T5) **Table 5** Multiple Regression Results: Predicting Scores on the Reasoning Scale of the MacCAT-CA View this table: [Table 6](http://jaapl.org/content/34/3/360/T6) **Table 6** Multiple Regression Results: Predicting Scores on the Appreciation Scale of the MacCAT-CA When the Understanding scale of the MacCAT-CA served as the criterion variable, age, WRAT Arithmetic, WRAT Reading, estimated IQ, and the BPRS-C Externalization factor score accounted for 40.1 percent of the total variance (*F =* 17.64, *p* = .001) for the full model. For the Reasoning scale, 37.1 percent of the total variance (*F =* 15.55, *p* = .001) was accounted for by age, WRAT Reading, WRAT Arithmetic, estimated IQ, and the BPRS-C Externalization and Developmental Maladjustment factor scores. Finally, 23.1 percent of the total variance (*F =* 7.93, *p* = .001) of the Appreciation scale was accounted for by the number of charges, WRAT Arithmetic, estimated IQ, and the BPRS-C Internalization and Externalization factor scores. To further understand these findings, we computed regression analyses that involved only the significant variables (i.e., a reduced model), to determine the contribution of just these variables to the MacCAT-CA scores. For Understanding, the reduced model accounted for 38 percent of the variance; for Reasoning, it accounted for 36 percent of the variance; and for Appreciation, 23 percent of the variance. These findings indicate that these variables alone account for nearly all of the variance captured by the full model, and an important portion of a youth's competence-related skills. Generalizing across these analyses, it appears that the combination of age, intelligence, learning skills, and behavior problems contribute to deficits in court-related knowledge, reasoning about legal issues, and understanding one's role in court proceedings. We also examined the scores with reference to the cut-off scores that have been developed based on the performance of adults.24 We computed the percentage of each age group whose score fell in the categories established to interpret the scores: minimal or no impairment, mild impairment, or clinically significant impairment. The results are presented in Table 7. As expected, the youngest group had the largest portion of youths with scores in the clinically significant impairment range (61.5% on the Understanding portion, 50% on the Reasoning portion, and 65% on the Appreciation portion). When combined with those youths whose scores were in the mild impairment range, 73.08 percent of 9- to 12-year-olds demonstrated some degree of impairment on the Understanding scale; 84.62 percent on the Reasoning portion; and 80.76 percent on the Appreciation scale. These numbers dropped dramatically as age increased, with reference to the Understanding and Reasoning portions, but a large number of youths obtained scores in the clinically significant or mild impairment ranges on the appreciation scale up through the oldest juveniles. With reference to the oldest group (17- to 18-year-olds), over three-quarters had scores in the minimal or no impairment range on the Understanding and Reasoning portions, but only 42.55 percent had scores in this range for the Appreciation portion. View this table: [Table 7](http://jaapl.org/content/34/3/360/T7) **Table 7** Percentage of Youth Whose Scores Fall in the Minimal or No Impairment, Mild Impairment, or Clinically Significant Impairment Ranges on the MacCAT-CA As mentioned earlier, we chose not to eliminate any juveniles on the basis of estimated IQ scores, as we wanted to produce a sample that reflected the full range of youths who become involved with the juvenile justice system. However, the normative MacCAT-CA sample24 and Grisso *et al.*28 eliminated individuals with IQs below 60. As a result of our failure to exclude such individuals, the estimated IQ of our sample was fairly low (M = 76.17, *SD* = 15.84) and much lower than the IQ scores of previous studies, such as Grisso *et al.* whose juvenile offender's average IQ (M = 86, SD = 12.95) was 10 points higher than that of our sample. To allow comparison with other studies, we examined the performance of the 19 percent of our sample with IQs below 60. This involved 4 (15%) of the 9- to 12-year-olds, 17 (23%) of the 13- to 14-year-olds, 27 (27%) of the 15- to 16-year-olds, and none of the 17- to 18-year-olds. We graphically present the percentages of youths who obtained scores in the clinically significant impairment range on the MacCAT-CA scales by age and IQ level in Figures 2, 3, and 4. As these figures demonstrate, larger percentages of those youths with IQ scores below 60 obtained scores on the MacCAT-CA that raise concern about their competence-related skills. A combination of young age and lower intellectual functioning appears to increase the risk of impaired court-relevant skills. ![Figure 2](http://jaapl.org/https://jaapl.org/content/jaapl/34/3/360/F2.medium.gif) [Figure 2](http://jaapl.org/content/34/3/360/F2) **Figure 2** Percentage of youth with IQ scores above or below 60 whose scores on the Understanding scale were in the clinically significant impairment range. No youth in the 17- to 18-year age group had an IQ score below 60. ![Figure 3](http://jaapl.org/https://jaapl.org/content/jaapl/34/3/360/F3.medium.gif) [Figure 3](http://jaapl.org/content/34/3/360/F3) **Figure 3** Percentage of youth with IQ scores above and below 60 whose Reasoning score was in the clinically significant impairment range. ![Figure 4](http://jaapl.org/https://jaapl.org/content/jaapl/34/3/360/F4.medium.gif) [Figure 4](http://jaapl.org/content/34/3/360/F4) **Figure 4** Percentage of youth with IQ scores above and below 60 whose Appreciation scores were in the clinically significant impairment range. No youth in the 17- to 18-year age group had an IQ score below 60. ## Discussion The present study examined the performance of incarcerated juvenile offenders on the MacCAT-CA in several ways, including the relationship between age, IQ, academic achievement, number of charges, and psychopathology with scores on the MacCAT-CA. Not surprisingly, our results indicate that age is related to the level of competence-related abilities. The youngest group of participants (ages 9–12) performed significantly worse on all three scales of the MacCAT-CA than the older three age groups, which did not differ significantly from each other. It seems safe to conclude that juveniles 12 and younger have less well-developed knowledge about court proceedings, demonstrate less well-developed reasoning about court proceedings, and do not appreciate their cases in the context of the larger court system, when compared with older juveniles. Most of the previous studies23,27,28 in this area have reached similar conclusions. Overall, the findings to date are consistent with Grisso's10 suggestion that the question of CST be raised in situations in which the defendant is 12 years old or younger, but also can be seen as supporting a recent position by the National Council of Juvenile and Family Court Judges,38 who recommended that a juvenile's competence to stand trial should be “explored through additional questioning during the detention or initial hearing” if, among other criteria, the youth is under the age of 15 (Ref. 38, p 93). Although, as we have described earlier, the relationship between age and competence-related skills is curvilinear, our data suggest that a substantial group of youths through age 16 display difficulties in this area. Nearly 40 percent of our 15- to 16-year-old sample had scores indicative of mild (or greater) impairment in legal reasoning and court knowledge. It is generally assumed that the deficits of juveniles are due, at least in part, to immature reasoning skills, which typically do not reach adult capacity until adolescence, if then. It should also be noted that the youths who find themselves before the juvenile court are, as a group, more likely than their non-involved peers to have a mental disorder or lower than average intelligence, which also increases the likelihood of impairments in reasoning. A discussion of the three portions of the MacCAT-CA helps elucidate the abilities of the youths in this sample. The Understanding scale of the MacCAT-CA focuses on the first part of the *Dusky* standard for CST, which is that a defendant must have an “understanding of the proceedings against him.” Thus, the Understanding section assesses factual knowledge of trial information and the roles of those involved. Our finding that age and IQ were related to the Understanding score is consistent with past research. Academic scores from the WRAT correlated moderately with Understanding, at least in part because of their overlap with IQ. However, the WRAT also measures how one learns and understands new information, which the Understanding scale intends to assess. Past research has reached inconsistent findings regarding the relationship between CST and school achievement, specifically concerning the relationship between a history of remedial education and CST.2,11 The results of the present study, however, indicate a strong relationship between current achievement skills and CST, even after allowance for general intelligence. It is our view that the academic screening measures provide an indication of the benefit youths derive from traditional instruction. The MacCAT-CA involves a “teaching” component, and the youths' difficulties with learning in school may account for their failure to derive sufficient benefit from the instruction provided in the MacCAT-CA. Perhaps this feature of juveniles' functioning should be given more consideration in future research and practice, including an exploration of how they might best be instructed about court-related matters, and what characteristics or test scores may inform opinions about whether a youth is “restorable” to competence. The Externalization factor score of the BPRS-C was significantly negatively related to scores on the Understanding scale. This suggests that problematic behavior (i.e., being uncooperative, hostile, or manipulative) and motor excitation (i.e., hyperactivity, distractibility, and pressured speech) are related to poorer legal knowledge and the ability to learn new legal information. In contrast, the variables related to experience with the juvenile justice system (number of charges) were not related to the Understanding score. While this finding replicates other results,26 it is a finding that bears mention. We repeatedly find that court personnel (attorneys, magistrates, judges) assume that experience with the juvenile court system increases factual knowledge of legal and trial-related material. Our finding contributes to the many studies that do not support this assumption. The Reasoning section of the MacCAT-CA is most closely related to the second part of the *Dusky* standard for CST, which is that a defendant must be able to “consult with his lawyer with a reasonable degree of understanding.” The same variables that were associated with Understanding were also related to the Reasoning score: age, estimated IQ, WRAT Reading and Arithmetic scores, and the Externalization factor of the BPRS-C. The BPRS-C Developmental Maladjustment scale was also significantly associated with the Reasoning score. From these results, it appears that similar factors affect both features of the *Dusky* standard and/or there is some overlap in the skills needed to perform well on both scales. Of note, the analyses failed to show a significant relationship between the Internalization and Developmental Maladjustment factor scores of the BPRS-C, suggesting that the problems measured by these factors (i.e., depression, withdrawal, anxiety, disorientation, deviant speech, hallucinations, or delusions) are not strongly linked to CST-related abilities in juveniles. Failure to find this relationship is most likely due to the low representation of these symptoms in this group. Certainly, these types of symptoms can be related to CST abilities in individual juveniles. The Appreciation portion of the MacCAT-CA is designed to assess a defendant's implausible or delusional thinking related to legal issues so as to determine the examinee's ability to appreciate how aspects of the legal system apply to his or her particular case. As was the case with Understanding and Reasoning scores, IQ, WRAT Arithmetic, and the BPRS-C Externalization factor were significantly related to the Appreciation score, although WRAT Reading and age were not. In contrast to the findings of the Reasoning and Understanding scales, the number of previous criminal charges and the Internalization factor of the BPRS-C were significantly related to the Appreciation score. This latter finding suggests that the Appreciation scale may assess some aspect of CST affected by experience with the juvenile justice system. The results of the present study suggest that several variables should be considered when examining CST in a juvenile population. Age, IQ, academic skills, and mental health conditions involving externalizing behavior were significantly related to CST abilities, at least as measured by the newly developed MacCAT-CA. According to the results, hyperactivity and behavior problems, which are symptoms of disorders such as Attention Deficit Hyperactivity Disorder and Conduct Disorder, play a meaningful role in a youth's ability to participate in court proceedings. In our experience, these are common disorders among juvenile offenders. An area in need of future research is the further examination of the impact of psychopathology and childhood behavioral and mental disorders on CST-related abilities. The multiple regression equations resulted in significant findings, but the amount of variance remained fairly small (ranging from 23.1% to 40.1% for the full models). Obviously, many unmeasured factors account for a large portion of the variance in competence-related abilities. Therefore, even though age, IQ, achievement, and psychopathology demonstrated a relationship to CST abilities in the present study, there are many more untapped factors playing a role. With reference to the performance of our sample compared with the normative data, our data suggest that many juveniles have deficits in understanding the roles of basic court personnel and basic court procedures; have difficulty thinking logically about the potential legal implications of available information (this is seen as being especially critical in assisting counsel in formulating a defense); and show deficits in the manner in which they appraise their own legal situation. In addition to the differences observed across age groups, there were some notable differences in the performance of the present juvenile sample on the MacCAT-CA when compared with the adult norms.24 For the 9- to 12-year-old group, the mean Understanding, Reasoning, and Appreciation scores all fell in the clinically significant impairment range for adults. The 13- to 14-year-old and 15- to 16-year-old groups' mean scores fell in the mild impairment range for the Understanding and Reasoning scales and in the clinically significant impairment range for the Appreciation scale. The 17- to 18-year-old group's scores fell in the minimal to no impairment range for the Understanding and Reasoning Scales and the mild impairment range for the Appreciation scale. However, when the percentage of youths at each age group whose scores indicated impairment is examined, most of the youngest group (9- to 12-year-olds) and many of the 13- to 14-year-olds obtained scores that suggest some impairment in CST-related skills. Of the 15- to 16-year-olds, about 40 percent demonstrated impairment in Understanding and Reasoning. Among the oldest group (17- to 18-year-olds), only about 25 percent obtained scores at a level that would suggest limitations in Understanding and Reasoning. Overall, the pattern of findings suggests that even middle adolescents (13- to 14-year-olds and 15- to 16-year-olds) are more likely than adults to perform at levels indicative of impairment on the MacCAT-CA. Although it is difficult to compare our data directly with the data presented by Grisso and colleagues,28 a larger percentage of our sample appears to have exhibited impairment relative to their sample, perhaps because we included younger offenders and those with lower IQ scores. Performance on the Appreciation scale deserves particular mention in the current use with a juvenile sample. Very high percentages of the present sample (e.g., over 80% of 9- to 12-year-olds) obtained scores indicative of impairment on this scale. Even among the oldest group, over 50 percent obtained scores in the impaired ranges. This portion of the MacCAT-CA is designed to measure delusional or irrational responses. However, poor scores are also given if the participant cannot provide reasoning for his choice. As a result, in addition to delusional thinking, the final score may reflect a respondent's hesitation to venture a guess or explanation for a choice, or his limited understanding of what the items ask him to do. In light of the fact that the typical age of onset for psychotic disorders like schizophrenia is older than most of the participants in the present study, the poor performance of juvenile participants on this scale is likely to be a result of factors other than delusional thinking. Indeed, our experience in administering this measure to juveniles suggests that many did not seem to comprehend the questions being posed, as the questions are fairly complex grammatically and address topics that seemed unfamiliar to our sample. Both our experience and our results have left us feeling that this scale does not measure what it intends to measure when used with juveniles. We concur with Grisso and colleagues28 in questioning the value of this scale for juveniles, especially those without a history of serious mental illness. We also suggest a review of the scoring criteria used for juveniles so as to differentiate delusional or disordered thinking from poor understanding (although this, too, could arguably represent an impediment to competence). Certainly, scores in the impaired ranges should not be interpreted as indicators of disordered thinking without other information. We offer a note of caution in interpreting the findings of the MacCAT-CA. It allows for systematic assessment of competence-related skills, but it is not intended to be the “final word” in a competence evaluation, and findings of studies such as this one should not be seen as reflecting the full range of competence skills in the youths we assessed. The authors of the MacCAT-CA24,25 have stated that this is a tool to be used with other information in rendering an opinion about CST. Likewise, findings such as ours should be understood with that caveat in mind: This study does not definitively identify those youths who would be (or should be) found competent (or not competent) to proceed. Certainly, very low scores on the MacCAT-CA seem to make an opinion of incompetence more likely, but not all information that is relevant in rendering a psycholegal opinion is gathered by using the MacCAT-CA alone. Although the evolving research on juveniles' CST-related skills seems to point to age differences in ability, further research in this area would help clarify these findings. Thus far, studies demonstrating age differences have been based on comparisons of cohorts that may differ in some fundamental ways. For example, in our study, the oldest group had a significantly higher average IQ than did the younger groups. Longitudinal designs would solve this confound and could provide us with specific information about the changes in thinking, symptoms of mental illness, or relationship to authority figures (among other things) that may be connected to CST abilities. In many ways, the findings from the current study (and others reviewed in this article) are not surprising: juveniles, especially those younger than 16, generally have less knowledge about the world around them and would be expected to have greater difficulty managing legal decisions. Based on findings about cognitive development, we can expect juveniles' abilities to reason and process information also to be less well-developed than those of adults. The challenge is to determine what we should do with these findings. Juvenile courts were designed to provide a way to rehabilitate youths. On the one hand, if youths are not competent to participate in court proceedings, they will not be able to avail themselves of the rehabilitative services offered by the juvenile court. On the other hand, many observers feel that the lengthy sentences available to juvenile courts, including “blended” sentences that allow for incarceration into adulthood, require that juveniles function competently in legal proceedings. In 1997, 1,700 juveniles were implicated in 1,400 murders, and nearly 60,000 juveniles 14 years of age and younger were adjudicated for crimes against persons such as homicide, forcible rape, and assault.39 If most of these youths are not competent to proceed, courts (and society) will face public safety issues. The evolving research findings in this area pose troubling social questions. Is there a way to maintain procedural safeguards while recognizing the cognitive and experiential limitations that youths present? Can we, as a society, find a way to protect youths' rights, while still protecting society and assisting law-breaking youths? These questions certainly extend beyond the scope of the current findings, but present a broader challenge to our society and to the mental health professionals who work in this area. ## Acknowledgments The authors gratefully acknowledge the support and cooperation offered by the Hamilton County Juvenile Court Youth Center and the Hamilton County Juvenile Court, especially the Honorable Sylvia Sieve Hendon and the Honorable Thomas Lipps. We also greatly appreciate the assistance of Jennifer Bartos Scott in data collection and Angie Siegler and Nicole Pattitucci in data management. ## Footnotes * This study was partially funded by the Hamilton County Juvenile Court Youth Center, Cincinnati, OH. Portions of this paper served as the doctoral dissertation of Dr. Ficke. * American Academy of Psychiatry and the Law ## References 1. Woolard JR, Reppucci ND: Researching juvenile's capacities as defendants, in Youth on Trial. Edited by Grisso T, Schwartz RG. Chicago: The University of Chicago Press, 2000, pp 173–91 2. Bonnie RJ, Grisso T: Adjudicative competence and youthful offenders, in Youth on Trial. Edited by Grisso T, Schwartz RG. Chicago: The University of Chicago Press, 2000, pp 73–104 3. In re Gault, 387 U.S. 1 (1967) 4. Snyder HN, Sickmund M: Juvenile Offenders and Victims: 1999 National Report. Washington, DC: Office of Juvenile Justice and Delinquency Prevention, 1999 5. Heilbrun K, Hawk G, Tate DC: Juvenile competence to stand trial: research issues in practice. Law Human Behav 20:573–8, 1996 6. Dusky v. U.S., 362 U.S. 402 (1960) 7. Melton G, Petrila J, Poythress N, *et al*: Psychological Evaluations for the Courts (ed 2). New York: Guilford Press, 1997 8. Kent v. U.S., 383 U.S. 541 (1966) 9. In the matter of D.G., 698 N.E.2d 533 (Ohio Misc. 2d 1998) 10. Grisso T: Forensic Evaluation of Juveniles. Sarasota, FL: Professional Resource Press, 1998 11. Savitsky JC, Karras D: Competency to stand trial among adolescents. Adolescence 74:349–58, 1984 12. Peterson-Badali M, Abramovitch R: Grade related changes in young people's reasoning about plea decisions. Law Hum Behav 17:537–52, 1993 13. Cowden VL, McKee GR: Competency to stand trial in juvenile delinquency proceedings: cognitive maturity and the attorney-client relationship. J Fam Law 33:629–61, 1995 14. Peterson-Badali M, Abramovitch R, Duda J: Young children's legal knowledge and reasoning ability. Can J Criminol 39:145–70, 1997 15. Baerger DR, Griffin EF, Lyons JS, *et al*: Competency to stand trial in preadjudicated and petitioned juvenile defendants. J Am Acad Psychiatry Law 31:314–20, 2003 [Abstract/FREE Full Text](http://jaapl.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFhcGwiO3M6NToicmVzaWQiO3M6ODoiMzEvMy8zMTQiO3M6NDoiYXRvbSI7czoyMDoiL2phYXBsLzM0LzMvMzYwLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 16. McKee GR: Competency to stand trial in preadjudicatory juveniles and adults. J Am Acad of Psychiatry Law 26:89–99, 1998 17. Cooper DK: Juveniles' understanding of trial-related information: are they competent defendants? Behav Sci Law 15:167–80, 1997 18. Dreyer CS: Competence to stand trial related abilities in non-delinquent juveniles. Unpublished dissertation, Xavier University, Cincinnati, OH, 2001 19. Steinberg L, Cauffman E: Maturity of judgment in adolescence: psychosocial factors in adolescent decision making. Law Hum Behav 20:249–72, 1996 20. Otto RK, Poythress NG, Nicholson RA, *et al*: Psychometric properties of the MacArthur Competence Assessment Tool-Criminal Adjudication. Psychol Assess 10:435–43, 1998 [CrossRef](http://jaapl.org/lookup/external-ref?access_num=10.1037//1040-3590.10.4.435&link_type=DOI) [Web of Science](http://jaapl.org/lookup/external-ref?access_num=000077959900013&link_type=ISI) 21. McKee GR, Shea SJ: Competency to stand trial in family court: characteristics of competent and incompetent juveniles. J Am Acad Psychiatry Law 27:65–73, 1999 [Abstract/FREE Full Text](http://jaapl.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFhcGwiO3M6NToicmVzaWQiO3M6NzoiMjcvMS82NSI7czo0OiJhdG9tIjtzOjIwOiIvamFhcGwvMzQvMy8zNjAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 22. Nicholson RA, Kugler K: Competent and incompetent criminal defendants: a quantitative review of comparative research. Psychol Bull 109:355–70, 1991 [CrossRef](http://jaapl.org/lookup/external-ref?access_num=10.1037/0033-2909.109.3.355&link_type=DOI) [PubMed](http://jaapl.org/lookup/external-ref?access_num=2062978&link_type=MED&atom=%2Fjaapl%2F34%2F3%2F360.atom) [Web of Science](http://jaapl.org/lookup/external-ref?access_num=A1991FK21700001&link_type=ISI) 23. Warren JI, Aaron J, Ryan E, *et al*: Correlates of adjudicative competence among psychiatrically impaired juveniles. J Am Acad Psychiatry Law 31:299–309, 2003 [Abstract/FREE Full Text](http://jaapl.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFhcGwiO3M6NToicmVzaWQiO3M6ODoiMzEvMy8yOTkiO3M6NDoiYXRvbSI7czoyMDoiL2phYXBsLzM0LzMvMzYwLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 24. Poythress NG, Nicholson R, Otto RK, *et al*: The MacArthur Competence Assessment Tool-Criminal Adjudication: Professional Manual. Odessa, FL: Psychological Assessment Resources, Inc., 1999 25. Poythress NG, Bonnie RJ, Monahan J, *et al*: Adjudicative Competence: The MacArthur Studies. New York: Kluwer Academic/Plenum Publishers, 2002 26. Redlich AD, Silverman M, Steiner H: Pre-adjudicative and adjudicative competence in juveniles and young adults. Behav Sci Law 21:393–410, 2003 [PubMed](http://jaapl.org/lookup/external-ref?access_num=12808697&link_type=MED&atom=%2Fjaapl%2F34%2F3%2F360.atom) 27. Burnett DMR, Noblin CD, Prosser V: Adjudicative competence in a juvenile population. Crim Just Behav 31:438–62, 2004 [Abstract](http://jaapl.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToic3BjamIiO3M6NToicmVzaWQiO3M6ODoiMzEvNC80MzgiO3M6NDoiYXRvbSI7czoyMDoiL2phYXBsLzM0LzMvMzYwLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 28. Grisso T, Steinberg L, Woolard J, *et al*: Juveniles' competence to stand trial: a comparison of adolescents' and adults' capacities as trial defendants. Law Hum Behav 27:333–63, 2003 [CrossRef](http://jaapl.org/lookup/external-ref?access_num=10.1023/A:1024065015717&link_type=DOI) [PubMed](http://jaapl.org/lookup/external-ref?access_num=12916225&link_type=MED&atom=%2Fjaapl%2F34%2F3%2F360.atom) [Web of Science](http://jaapl.org/lookup/external-ref?access_num=000183331200001&link_type=ISI) 29. Leisgang NA: Emotional disturbance, substance abuse and violence among incarcerated youth. Unpublished dissertation, Xavier University, Cincinnati, OH, 2004 30. Wechsler D: Manual for the Wechsler Intelligence Scale for Children (ed 3). New York: The Psychological Corp., 1991 31. Wechsler D: WAIS-III Administration and Scoring Manual. San Antonio, TX: The Psychological Corp., 1997 32. Sattler JM, Ryan JJ: Assessment of Children Revised and Updated 3rd Edition: WAIS-III Supplement. San Diego: Jerome Sattler, Publisher, Inc., 1999 33. Sattler JM: Assessment of Children: WISC-III and WPPSI-R Supplement. San Diego: Jerome Sattler, Publisher, Inc., 1992 34. Wilkinson GS: Wide Range Achievement Test Administration Manual. Wilmington, DE: Wide Range, 1993 35. Overall JE, Pfefferbaum B: The Brief Psychiatric Rating Scale for Children. Psychopharm Bull 18:10–6, 1982 [PubMed](http://jaapl.org/lookup/external-ref?access_num=7111598&link_type=MED&atom=%2Fjaapl%2F34%2F3%2F360.atom) 36. Lachar D, Randle SL, Harper RA, *et al*: The Brief Psychiatric Rating Scale for Children (BPRS-C): validity and reliability of an anchored version. J Am Acad Child Adolesc Psychiatry 40:333–40, 2001 [CrossRef](http://jaapl.org/lookup/external-ref?access_num=10.1097/00004583-200103000-00013&link_type=DOI) [PubMed](http://jaapl.org/lookup/external-ref?access_num=11288775&link_type=MED&atom=%2Fjaapl%2F34%2F3%2F360.atom) [Web of Science](http://jaapl.org/lookup/external-ref?access_num=000167200000013&link_type=ISI) 37. Overall JE, Gorham DR: The Brief Psychiatric Rating Scale. Psychol Rep 10:799–812, 1962 [CrossRef](http://jaapl.org/lookup/external-ref?access_num=10.2466/PR0.10.3.799-812&link_type=DOI) [Web of Science](http://jaapl.org/lookup/external-ref?access_num=A1962CBK8600036&link_type=ISI) 38. National Council of Juvenile and Family Court Judges: Juvenile delinquency guidelines: improving court practice in juvenile delinquency cases. Reno, NV: National Council of Juvenile and Family Court Judges, 2005 39. Stahl A, Finnegan T, Kang W: Easy access to juvenile court statistics: 1985–2000. Available at [http://ojjdp.ncjrs.org/ojstatbb/ezajcs](http://ojjdp.ncjrs.org/ojstatbb/ezajcs). Accessed September 13, 2005