Abstract
Ethics guidelines recommend that forensic mental health professionals begin in-person assessments by explaining the nature and purpose of the examination. To learn whether evaluees have understood and can give consent, forensic practitioners may ask evaluees to paraphrase the explanation. This article explores how a forensic evaluee's disclosure response (DR) reveals substantive information relevant to the purposes of a forensic examination. We examined archival data from 255 reports on competence to stand trial (CST) that a Midwest public sector hospital had previously submitted to courts. We classified each evaluee's DR at one of three levels: DR = yes (accurate paraphrasing), DR = no (inability to paraphrase or provide a relevant response), or DR = other (an intermediate level implying a less-than-accurate response). None of the 28 DR = no evaluees was CST, and only 7 (17%) of the 48 DR = other evaluees were CST. Thus, a CST evaluee who cannot paraphrase an examiner's explanation is likely to be incompetent to stand trial, and an examiner would need to adduce a strong argument to support any opinion to the contrary.
In its Ethics Guidelines for the Practice of Forensic Psychiatry, the American Academy of Psychiatry and the Law (AAPL) tells psychiatrists, “At the outset of a face-to- face evaluation, notice should be given to the evaluee of the nature and purpose of the evaluation and the limits of its confidentiality.”1 Similar instructions appear in specialty guidelines for forensic psychologists, who should “strive to inform service recipients about the nature and parameters of the services to be provided” (Ref. 2, p 12). The obligation to provide notice flows from a commitment to the doctrine of informed consent, “one of the core values of the ethical practice of medicine and psychiatry. It reflects respect for the person, a fundamental principle in the practices of psychiatry and forensic psychiatry.”1 In medical practice generally, requiring consent prevents clinicians from imposing their views on patients and gives patients “the privacy to make decisions in line with their own values” (Ref. 3, p 356).
To obtain a forensic evaluee's genuine assent or consent, an examiner needs to know that the evaluee has understood the examiner's explanation of the examination's purpose and appreciates its significance. For this reason, forensic practitioners may invite evaluees to ask questions after describing the nature and purpose of an examination.4 Examiners may also ask evaluees to paraphrase or answer questions about the information disclosed (as is done in obtaining informed consent for medical care)5,6 and then correct any misconceptions that evaluees hold.
The disclosure process for an evaluation of adjudicative competence, more often termed competence to stand trial (CST), usually includes an explanation of the reason for the evaluation, the nontreating nature of the examination, who has appointed or retained the examiner, the lack of confidentiality, who will receive the examiner's report, and the right of the evaluee not to answer particular questions, with a warning that the examiner may have to report such noncooperation or refusal.7 To paraphrase this information satisfactorily, the evaluee must hear it, appreciate its bearing on the situation, and explain it rationally to the examiner; that is, the evaluee must exercise the very sorts of mental faculties required to be CST. How much information about the defendant's competence, therefore, is revealed by the quality of a defendant's responses during the CST examination's disclosure process? Is the disclosure response (DR) itself an indicator or “test” (as it were) of a defendant's CST?
In an effort to answer these questions, this article describes a study of archival material developed under actual CST examination conditions and taken from actual CST reports submitted to criminal courts. Before undertaking the study, we anticipated that these reports would contain data that would validate or fail to prove the hypothesis that a defendant's responses during a CST examination's disclosure process provide substantive information about whether that defendant is competent or not.
Method
Setting
This study used archival data gathered from CST reports prepared at Summit Behavioral Healthcare, an Ohio public sector hospital operated by the Ohio Department of Mental Health and Addiction Services (OhioMHAS). Among the services provided by Summit Behavioral Healthcare are assessments of criminal defendants sent by courts for evaluations of their adjudicative competence8 or for treatment to restore defendants found incompetent to stand trial (IST).9 Pursuant to Ohio statutory requirements,10 the hospital sends reports of the competence assessments to the region's criminal courts and retains copies of the reports in patients' records.
Sample
Study data came from 255 reports that Summit Behavioral Healthcare had sent to courts during the years 2010–2015. These reports represented all the CST evaluations conducted by four psychiatrists who, when the evaluations took place, were forensic psychiatry fellows at an academic medical center. During their training, the fellows had examined hospitalized patients and prepared reports that served as the hospital's opinion on CST.
The study sample included 54 women, a proportion (21% of the full sample) fairly similar to recent statistics about arrest rates across the United States11 and in Ohio.12 All sample members were adults whose mean (SD) age was 39.4 (12.9) years. The majority (n = 141; 55%) were African American, 106 (42%) were white, and 8 (3%) were of other backgrounds (Latino, Asian, or African ancestry). These proportions are consistent with the ethnic make-up of the hospital's catchment area13,14 and with the arrest ratio of African-American to white defendants for Ohio generally.12 The sample's mean education level was 10.9 ± 2.4 years; 143 (56%) of the evaluees had either completed high school or had passed a General Educational Development (GED) test.
The evaluees' most serious charges ranged from second-degree misdemeanors (e.g., obstructing official business and resisting arrest), to higher level felonies (e.g., felonious assault, rape), to homicides. Nearly one-fourth (n = 62; 24%) of the sample's most serious charges were assaults, a proportion that rose to nearly one-third (n = 83; 33%) if one includes domestic violence charges. More than half (n = 137; 54%) the evaluees faced misdemeanor charges only. In Ohio, incompetent misdemeanor defendants may spend no longer than 60 days undergoing competence restoration; for felony defendants, restoration may last up to one year.15 Also, 23 (9%) of the evaluees had undergone hospitalization for purposes of a 20-day inpatient CST evaluation.16 These factors account for the relatively short median length of stay (42 days; intraquartile range, 28–118 days) at the time of evaluation.
The four psychiatric conditions that most commonly underlay evaluees' possible or adjudicated incompetence to stand trial were schizoaffective disorder-bipolar type (n = 77; 30%), schizophrenia (n = 72, 28%), bipolar disorders (n = 50; 20%), and other psychoses (n = 17; 7%). Substance use disorders was in fifth place (n = 8; 3%), but 49 evaluees (19%) had substance use disorders in addition to the principal psychiatric condition related to possible incompetence. Only three (1%) of the evaluees were thought to be malingering, a rate much lower than is reported for initial CST evaluations.17,18 In our sample, however, all the evaluees had undergone at least one prehospitalization CST evaluation by other clinicians at local forensic centers, which likely had filtered out many court-referred defendants who were malingering.
Measurement
In response to our inquiries, the four original psychiatric examiners confirmed that at the outset of every CST evaluation, they had tried conscientiously to complete a disclosure process that included a description of the examination's purpose and nonconfidentiality, the parties that would receive the CST report, the potential use of the report in determining case disposition, and the nontreatment purpose of the examination; and asking the evaluee to paraphrase this explanation to assess understanding of the description.
The original examiners confirmed that they had accurately summarized evaluees' DRs in all their CST reports. These reports also included narrative summaries of evaluees' personal background, educational attainment, employment history, legal history, psychiatric and other medical history, hospital course, mental status findings, and responses to inquiries specific to CST. Further, examiners documented their findings from administrations of assessment instruments (e.g., the Georgia Court Competency Test (GCCT)19 or MacArthur Competence Assessment Tool-Criminal Adjudication (MacCAT- CA)20) that they had used. The reports concluded with the examiners' opinions on psychiatric diagnosis and CST; the latter was assessed against the jurisdiction's statutory and case law criteria for competence to stand trial.21,22 In Ohio, a defendant is presumed competent to stand trial and only found incompetent to stand trial if “the defendant is incapable of understanding the nature and objective of the proceedings against the defendant or of assisting in the defendant's defense.”
Data Collection Procedure
We undertook this archival study after receiving approvals from the Institutional Review Boards of the University of Cincinnati and Summit Behavioral Healthcare. From each CST report prepared by the original four examiners, three of this article's authors [R.B., B.C.L., C.M.] extracted data that included: evaluees' demographic features (e.g., age, sex, race/ethnicity, educational attainment); reason for hospitalization (i.e., for CST evaluation or restoration); length of hospital stay when the examination occurred; offense type for the most serious charge; the examiner's diagnoses; scores on assessment instruments (e.g., the GCCT); the examiner's opinion on CST; and a classification (described in the next subsection) of the evaluee's DR.
Data were recorded on an Excel spreadsheet with identifiers removed such that no information could be linked to any individual evaluee. To verify that the extractors' classifications were reasonable, the first author [D.M.] reviewed (but in no case changed) the original data on which extractors' had based “DR = no” and “DR = other” ratings. (The DR = yes ratings came from reports with highly consistent descriptions of evaluees' responses.)
Data Analysis
We compared the evaluees whom examiners had opined CST and IST along several demographic, treatment, and assessment variables. We also compared evaluees who fell into each of the three DR classifications described above. For ordered values, we used Mann-Whitney U tests; for categorical values, we used chi-square tests or (when expected categorical values were below 10 subjects) Fisher's exact test. In evaluating significance, we corrected for multiple comparisons using the Holm-Bonferroni method.23
To quantify what the evaluees' DRs said about their CST status, we needed to examine the DR mathematically as though it were a diagnostic test of CST. As a first step, we assigned each evaluee's DR to one of three categories:
DR = yes. This category designated evaluees whom the examiners described as having paraphrased the disclosure statement satisfactorily before agreeing to participate. Typically, the examiners reported this by writing something very close to the following: “To see whether Mr. A had understood, I asked him to paraphrase what I had told him about the nature and purpose of the interview. Aided by some prompting questions, he did so accurately, and he agreed to speak with me.” Here, “prompting questions” referred to portions of the disclosure that the evaluee had not paraphrased spontaneously. If, for example, the evaluee had not mentioned who would receive the report, the examiner might then ask, “Did I say anything about who gets my report?” and await the evaluee's response. An evaluee who (for example) responded, “Oh yeah, you said the court, my lawyer, and the prosecutor” would then be described in the report as having paraphrased accurately.
DR = no. This category contained evaluees who, examiners reported, offered little or no response, who gave responses that were irrelevant or too disorganized or chaotic to understand or who could not remember or paraphrase more than an element or two of the disclosure information. Examples of examiner descriptions that led to DR = no categorizations included:
“[The evaluee] did not acknowledge my presence. I interpreted his ignoring me as an indication that he would not participate in an interview.”
“[The evaluee] asked, ‘What am I going to court for?’ …I could not elicit [further] responses.”
“[The evaluee] responded, ‘I need to meet and then go home.’”
“[The evaluee] laughed briefly, but he did not talk, gesture, or make any other response.”
DR = other. This category contained evaluees for whom we could not neatly categorize responses as clearly DR = yes or DR = no. Responses in this category tended to indicate problems with grasping information disclosed or the presence of symptoms (e.g., paranoia) that affected the evaluee's participation in an interview. DR = other responses did not indicate complete inability to grasp the interview's purpose, however. To illustrate, we provide a few examples taken from the examiners' reports:
“He could not restate the information, but the answers he provided to my questions showed some grasp of what I had explained.”
“He could not restate the information, but his answers to my questions showed adequate understanding of this information.”
“She asked me what court and where the court was located. She asked me to repeat my answers a few times. When she restated the disclosure, she did it accurately and agreed to speak with me.”
“She said, ‘It's for competent to stand trial; it's between me and you.’ I again explained that the evaluation was not confidential, and she agreed to participate.”
“He began to speak rapidly and tangentially about his case, and said, ‘If they don't show up, throw the case out of court.’ After a few minutes, I asked him again if he could tell me why we were meeting. Mr. A said that it was because he was found incompetent to stand trial, and he agreed to participate in the evaluation.”
We assigned the numbers 1 to DR = no, 2 to DR = other, and 3 to DR = yes to reflect our assumption that this order represented decreasing likelihood that the evaluee was IST.
We wrote OpenBUGS24 code to generate Bayesian estimates of accuracy parameters, examining how the DR performed as a diagnostic test of CST from three perspectives:
Opinion as truth, which treated the examiners' CST opinions, formulated based on all data available to the examiner, as the criterion for whether the evaluee was indeed CST or IST.
Opinion as an imperfect gold standard, which treated the examiners' CST opinions as a very accurate but not perfect criterion (or gold standard) for the true CST-IST status of the evaluees.
Agnostic, which adopted latent class analysis techniques use in other recent studies of forensic data25,26 and are applicable when data include test results from subjects who have undergone evaluation for a condition with more than one diagnostic method. Here, four findings potentially differentiated between IST and CST evaluees: the evaluees' DRs, the examiners' CST opinions, whether the evaluee had a diagnosis of schizophrenia or schizoaffective disorder (designated Σ+) or not (Σ−),27 and evaluees' scores on the GCCT.19,28
We calculated accuracy indices for the DR categorizations and, for comparison purposes, the Σ+/Σ− distinction and GCCT scores. These indices included the area under the receiver operating characteristic (ROC) curve (AUC), which is a global measure of classificatory performance. In this context, the AUC equals the probability that, if an IST defendant and a CST defendant were chosen at random from all IST and CST defendants, the diagnostic method would correctly classify the defendants. (This interpretation of the AUC assumes that if the two randomly chosen evaluees had tied scores, the tie would be resolved by flipping a coin to decide which way to classify the evaluees.)
We also calculated another accuracy index: the stratum-specific likelihood ratio (SSLR). One formulation of Bayes' theorem expresses the posterior odds of being IST as the product of the pretesting or prior odds of being IST and the SSLR associated with Tk, a test result in category k, or ODDSpost = ODDSprior × SSLRk. The SSLR equals the slope of the portion of the ROC curve that corresponds to a particular test result category. On an ROC graph, test result categories are demarcated by points that represent (fpr, tpr) pairs of false-positive and true-positive rates. One calculates SSLRk from the (fpr, tpr) pairs as follows:
where c = {1, …, K}, c = {1, … K −1} correspond to the K −1 (fpr, tpr) pairs that delimit the K result categories, fprc = 0 and tprc = 0 when c = K, and fprc−1 = 1 and tprc−1 = 1 when c = 1.
The rationale for undertaking the imperfect gold standard and agnostic analyses was to evaluate explicitly the accuracy of the examiners' opinions on CST and IST. These two analyses showed, however, that the examiners were almost perfectly accurate. Further, all three methods for quantifying diagnostic performances of the DR, the Σ+/Σ− distinction, and GCCT scores produced similar results. The Results section that follows therefore presents just the findings from the “opinion as truth” analyses.
Results
Table 1 presents univariate comparisons of the evaluees whom the examiners thought were CST and IST. IST evaluees tended to be older, to have lower scores on forensic assessment instruments, to be more likely to have diagnoses of schizophrenia and schizoaffective disorder, and to have had trouble paraphrasing the initial disclosure.
Comparisons of Defendants Opined CST or IST
Although taking an antidepressant medication appeared to be an indicator of being CST, a statistical adjustment for multiple comparisons suggested that this association was coincidental.
Table 2 describes the diagnostic performance of three types of examination findings in differentiating between competent and incompetent evaluees: having schizophrenia or schizoaffective disorder (Σ+) or not (Σ−), GCCT scores (which, as Table 1 shows, were available for almost 70% of evaluees), and DR. We present median values with 95 percent Bayesian credible intervals in Table 2, rather than means ± SDs, because the Bayesian posterior distributions for the SSLRs were highly skewed, which made medians better estimators of central tendency. Table 2 shows that around 95 percent of the time, a randomly chosen IST evaluee would have a lower GCCT score than a randomly chosen CST evaluee. Similarly, about three-fourths of the time, a randomly chosen defendant who turns out to be incompetent will have a poorer DR than a randomly chosen defendant who turns out to be competent.
Examination Findings Construed as Diagnostic Tests of Incompetence to Stand Trial
We illustrate use of the SSLRs in Table 2 with some numerical examples. Suppose that Mr. Jones, a CST evaluee, gave a response to the disclosure that would be classified DR = other. Recent reports estimate that about one-fourth of defendants referred for CST evaluations are actually incompetent.29 If this estimate appeared applicable to the population from which Mr. Jones came, then the prior odds of being IST would be 1:3. Table 2 shows that for DR = other, the SSLR was 5.2. Thus, the postdisclosure odds of being IST would be 5.2 × 1/3 = 1.73, or 1.73:1, and the postdisclosure probability of being CST = 
≈ 0.63, or 63 percent. Similar calculations show that a rating of DR = no would make it highly likely that the evaluee was IST. Finally, a rating of DR = yes would reduce the odds of being IST by about one-half, so that the posttest odds of being IST would be 1:5.77, and the posttest probability of being IST = 
≈ 0.15, or 15 percent.
Finally, Table 3 compares the members of the three DR groups. Although the DR = yes group appeared slightly younger than the other two groups, this difference probably is not significant when corrected for multiple comparisons. The only significant differences are those that involve the diagnoses rendered by the examiners and the evaluee's GCCT score. That DR = yes was associated with having bipolar disorder makes sense, in that persons with affective illnesses tend to have less cognitive impairment and better rates of competence restoration than do persons with schizophrenia-spectrum disorders.21,27,30 Similarly, one would expect that individuals who could do relatively well on a forensic assessment instrument such as the GCCT would assimilate the DR and paraphrase it satisfactorily.
Comparisons of Defendants According to Their DR
Discussion
This study showed that the disclosure process that begins a CST examination foreshadows later findings from the examination and the evaluee's CST status itself. In particular, an evaluee's inability to paraphrase the examiner's disclosure is a strong indicator of incompetence to stand trial. Thus, an examiner learns much about the evaluee's competence, even before the evaluee has agreed to disclose information relevant to CST.
Based on our findings, the DR appears to be a fairly specific indicator of being IST, though not a sensitive one. That is, less-than-satisfactory paraphrasing of the disclosure greatly heightened the odds that the evaluee was IST, but (contrary to what we had expected) many evaluees who could paraphrase a disclosure satisfactorily still were not CST. Of course, a poor disclosure response should not be the sole basis for an examiner's opinion on adjudicative competence. Yet our findings suggest that examiners should provide substantial data to support and documentation to justify any opinion that an evaluee who paraphrased poorly was nonetheless competent to stand trial.
Our findings come with several important qualifications. Our efforts at quantification were limited by our study's retrospective design and our use of archival data that were not collected with research goals in mind. Although fellowship supervisors had instructed the original four examiners to describe evaluees' DRs faithfully in their reports, the examiners did not use any prespecified scoring system or description formats for the DR that may be evaluated for interrater reliability. Nor did the examiners use standardized wording for their disclosures; consistent with good forensic practice, examiners individualized how they conveyed information to optimize individual evaluees' comprehension. Although the article's first author reviewed his coauthors' DR = no and DR = other ratings to confirm that their classifications were reasonable, we did not conduct formal tests of interrater reliability among the data gatherers.
Despite these limitations, we believe our findings constitute a useful view of what a poor DR signifies: likely incompetence. Supporting this conclusion are findings from recent research showing that the Montreal Cognitive Assessment,32 the Repeatable Battery for the Assessment of Neuropsychological Status,33 and the Bender Gestalt Test34 identify individuals whose cognitive impairments limit their competence and capacity to be restored. Moreover, our findings should not surprise individuals who are familiar with the disclosure process for a forensic CST examination, although the low sensitivity and high specificity of the DR may. After all, navigating the disclosure process successfully requires that a CST evaluee exercise mental faculties that also are relevant to adjudicative competence, including attention, orientation, verbal comprehension, memory, reasoning, executive functioning, and aspects of social cognition.35 Reports by CST examiners may therefore include a poor DR as one of several findings that support an opinion that a defendant-evaluee is IST.
A not-yet-mentioned limitation relates to our using the original four examiners' CST opinions as the criterion for whether an evaluee actually was CST or IST. Yet, this choice makes sense in light of reasons articulated in a previous study of CST restoration27: in all but a handful of cases, the criminal courts accepted the hospital's opinion based on the examiner's report; using the examiner's opinions provided a truth criterion that was more uniform across evaluees than would be the opinions from dozens of judges; judges (not examiners) could be wrong; and judges' opinions may differ because of posthospitalization changes in the evaluees' mental status. Further, our imperfect gold standard and agnostic analyses showed that treating the examiners' opinions as the truth did not distort estimates of accuracy.
Readers may wonder whether evaluees' DRs led examiners to reach premature closure about evaluees' competence. If this were the case, then hearing evaluees' DRs at the outset of the examination would have caused rather than merely presaged the examiners' final opinions; the examiners would have succumbed to what cognitive psychologists call confirmatory bias.36 Our findings tend to dispel this notion, however. As Table 3 shows, obvious features of the evaluees known to the examiners before the interview predicted neither how examiners viewed the DRs nor their ultimate points of view on CST: their age, sex, ethnicity, length of hospital stay, and charges. The GCCT score, obtained after the disclosure, tended to confirm what the DR suggested. The GCCT also was a much better gauge of CST than was the disclosure response, as implied by its much higher AUC (see Table 2).
If an evaluee is too impaired to understand why a CST examination is taking place or otherwise cannot give consent to the examination, a court order or the express permission of the evaluee's defense attorney provides acceptable legal authorization for a CST examination to continue.7 In all the CST examinations that generated our study data, examiners had acted pursuant to court authorizations grounded in Ohio's statutory provisions.
Nonetheless, our findings show that seeking evaluees' consent undermined the ethical rationale for disclosure (fostering and respecting self-determination regarding use of one's personal information) by eliciting clinically and legally significant findings relevant to the evaluees' mental functioning before they had actually agreed to have this information exposed. For evaluees who preferred not to convey such information, the examiners' disclosures and efforts to obtain consent contravened the evaluees' wishes. Statutory provisions usually provide legal justification for CST evaluations of defendants who cannot give valid consent to participate, and by helping courts to identify defendants who are too impaired to assist counsel or understand their legal proceedings, forensic examiners provide an ethically valuable service to those defendants themselves. But when performing other types of forensic evaluations, undertaking a careful consent process may help examiners identify defendants who should not undergo further assessment absent an express authorization from the court or the evaluee's attorney.
Footnotes
Disclosures of financial or other potential conflicts of interest: None.
- © 2018 American Academy of Psychiatry and the Law
