A Primer for Increasing Competency in Forensic Psychiatry Research

• forensic psychiatry research
• competence restoration
• systematic review
• empirical research

Forensic psychiatry research is all-encompassing, ranging from case reports and epidemiological investigations to treatment outcome studies and neurobiological experiments. The challenge in understanding the essence of forensic psychiatry research is that it involves medical professionals operating within the multidisciplinary sphere of criminal justice, social control, and empirical sciences in a social context.¹ To highlight their importance, ethics considerations have been specifically proposed for forensic psychiatry research² and publishing.³ Forensic psychiatrists belong to a large cadre of forensic mental health professionals, including psychologists, nurses, social workers, occupational therapists, pharmacologists, toxicologists, sociologists, criminologists, neurobiologists, and others, all of whom engage in the study of persons with mental disorders who perpetrate illegal or violent behaviors.⁴ While forensic psychiatry may lag behind some of these other disciplines in advancing empirical forensic mental health research, numerous examples abound of high-quality empirical research conducted by forensic psychiatrists that enhance the credibility and recognition of our subspecialty. The systematic review on the quality of competence restoration research⁵ provided by Drs. Philip J. Candilis and George F. Parker and published in this edition of The Journal is but one example of such work.

Competence to stand trial evaluations, also known as “adjudicative competence,” “fitness to stand trial,” or “competence to proceed with adjudication,”⁶ are hailed as “the most significant mental health inquiry pursued in the system of criminal law” (Ref. 7, p 200) and are the most common. With a recent estimate that there are 94,000 such evaluations conducted annually in the United States,⁸ it is understandable that this psycholegal mechanism has garnered the attention of forensic mental health researchers.

Three meta-analyses hold possible answers about what the scientific literature tells us about competence restoration. Nicholson and Kugler⁹ calculated Pearson correlation coefficients as measures of effect sizes averaged across studies between 1967 and 1989, reflecting the magnitude of the relationship between competency status and defendant characteristics. Findings revealed that the most robust predictors of incompetency were poor performance on psychological tests designed to analyze defendants’ legally relevant functional abilities, a psychotic diagnosis, and psychiatric symptoms suggestive of severe psychopathology. A subsequent meta-analysis by Pirelli and colleagues¹⁰ that used more contemporary meta-analytic methods examined studies published between 1967 and 2008 to compare competent and incompetent defendants on several demographic, psychiatric, and criminological variables. Results were similar to the previous meta-analysis: defendants diagnosed with a psychotic disorder were eight times more likely to be adjudicated incompetent than defendants without a psychotic disorder; unemployed defendants were approximately twice as likely to be found incompetent compared with employed defendants; and competent defendants performed much better than incompetent defendants on competency instruments. An attempt at a third meta-analysis by Pirelli and Zapf¹¹ was ultimately abandoned because of significant limitations of the research base. The main objective had been to quantitatively synthesize competency restoration samples that comprised defendants who had undergone competency restoration treatment. Studies were considered if they included defendants whose competency had been restored or was ordered to be restored and if at least one data point or estimate was present that could produce an effect size estimate. Where possible, the investigation sought to compare defendants engaging in restoration programs with their restored or competent counterparts. In the end, the authors concluded that the data could not be analyzed in any statistically meaningful way and that the restoration literature did not currently lend itself to meta-analysis. Pirelli and Zapf lamented that “the most important group comparisons — between defendants who have had their competency restored and those who have not — have essentially not been investigated in any meaningful way over the past half-century” (Ref. 11, p 153).

Candilis and Parker, two widely published forensic psychiatrists, undertake the task of assessing the quality of competence restoration research by employing another high-quality design: a systematic review of the empirical literature using National Institutes of Health (NIH) quality metrics¹² that highlight study design, sample size, and statistical methods. Several aspects of the methodology utilized by the authors to conduct their analysis are laudable. For example, they followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) protocol,¹³ which is the gold standard process for conducting systematic reviews. This strategy enabled the authors to derive empirical articles from several relevant databases that conformed to their search terms centered on adult inpatient competence restoration. The PRISMA protocol ultimately yielded 1,187 studies from 1981–2022 that were uploaded to the Covidence platform (covidence.org/reviews), which is the screening and data extraction tool utilized by Cochrane review authors.¹⁴ Candilis and Parker then reviewed the titles and abstracts of all studies in the initial round of screening, with the aim of removing publications not related to competence restoration. As described, each author entered a vote for every study, as required by the Covidence platform, and conflicts between researchers were resolved via conference call. The authors next reviewed the abstracts of the remaining articles to ensure that they were related to adult inpatient competence restoration. Finally, the full text of each remaining article was reviewed by each author for relevance to adult inpatient competence restoration. In the end, 76 studies met inclusion criteria.

Candilis and Parker also categorized each study using the NIH toolkit, which prompts reviewers to determine whether a study has the elements of design, analysis, and description associated with a lower risk of bias and greater likelihood of high quality. The authors explain that by encouraging reviewers to evaluate the adequacy of sample descriptions and selection, sample and measure justification, and statistical adjustment of confounding variables, the NIH tools offer a guide to a dialogue on the strength of a study’s methods and ultimately an approach to assess the quality of human subject research. Candilis and Parker lastly provided an overall study quality rating of good, fair, or poor for each study. As agreed upon by the authors a priori, study design, sample size, and statistical analysis were prioritized objective elements that could affect the overall study quality rating. The characteristics of each study were then recorded, which included the type of study, the sample size, the demographics of the study population (if available), the success of competence restoration (if available), the statistical methods, if the authors initially agreed on the overall quality rating, and the final quality rating (this information is available in Appendix C of the article).

Among the 76 articles included in the systematic review, only eight (11%) were classified as good quality, and most articles (72%) were case series. Indeed, among the eight good quality articles, six were case series. This is not necessarily problematic, explain Candilis and Parker, as case studies can be considered good quality if their sample size (N) is sufficiently large and the statistical analyses are sophisticated. Overall, however, the systematic review revealed that retrospective, fair-quality studies with modest sample sizes currently dominate the adult competence literature.

Perhaps even more salient than the results of the systematic review itself are the implications for future research proffered by the authors. Candilis and Parker exhort that individuals wishing to conduct high-quality research in adjudicative competence or “forensic research in general” will need to consider pursuing larger databases or developing prospective studies with sizeable Ns. Regarding the latter point, the field of forensic psychiatry has begun to respond with large-scale prospective trials of differing forensic mental health service platforms on the horizon.^15–16 Candilis and Parker also draw attention to the importance of expert statisticians providing input along the entire research project continuum, from assisting with power analyses at study inception to analyzing data with robust modeling techniques. While most forensic psychiatrists have limited experience with complex statistics, they may be affiliated with academic departments or public sector agencies that have access to individuals with the required level of expertise. These affiliations may also hold relevance for clinical practice, as one recent study found that when considering overall excellence of clinical forensic psychiatry services, strong university and research links appear to matter.¹⁷ Candilis and Parker point out that, in an effort to provide statistical advice for researchers without these connections and spur on the development of more empirical research, the AAPL Research Committee cultivated a network of researchers several years ago that could consult with investigators seeking statistical guidance.

Candilis and Parker rightfully acknowledge that only a minority of studies in their systematic review enrolled females (36%) and that such male-dominated samples largely sidestep questions of unique symptom presentations by females or challenges presented during different reproductive stages among females in forensic settings. Stratification by race, ethnicity, socioeconomic status, and neighborhood was also not possible in most studies, as reported by Candilis and Parker, owing to small sample sizes. It remains to be seen whether gender as a nonbinary category or sexual minority status will be investigated in the future as variables contributing to competency restoration status.

We need to consider where the field goes from here. If Candilis and Parker have their druthers, subspecialty training programs in forensic psychiatry would focus their research efforts on teaching robust research methods and developing empirical projects that span multiple fellowship years with the intended consequence of expanding the literature, instructing cohorts of trainees, and emphasizing methods for large-scale collaborations and publication of meta-analyses. Other avenues for increasing uptake of empirical forensic psychiatric research could include targeted recruitment of MD/PhD students who show an early interest in the field or establishment of more clinician-scientist stream residency programs, where it would be feasible for residents inclined toward a career in forensic psychiatry and research to pursue a postgraduate degree in tandem with their clinical training. To increase options for research training beyond residency, creation of more programs like the Columbia/New York State Psychiatric Institute Research Fellowship in Forensic Psychiatry or tailoring other research fellowships that equip learners with a tangible skill set (e.g., genetic or epidemiological data analysis) and then have them apply it to the study of forensic psychiatric patient populations could be envisaged.

Interspersed throughout the Candilis and Parker paper are examples of how the AAPL Research Committee aims to nurture more empirical research among members of the larger organization. Indeed, we are here to help. One recent initiative involves the Research Committee coordinating with the AAPL Institute for Education and Research (AIER) to assist members with fine-tuning their grant applications to set them up for the greatest chance of success. Another idea that has been discussed by the Research Committee is the development of a compilation of landmark research studies in forensic mental health science (akin to the series of landmark cases fellows are taught) that could be disseminated to forensic fellowship training programs as a means of increasing research literacy among trainees. We always welcome the addition of new members to the committee, whether they be residents, fellows, early career psychiatrists, or experienced practitioners, to help maintain our relevance to the organization.

In summary, empirical forensic psychiatric research poses unique challenges but is vital for our profession to thrive and flourish. While the systematic review by Candilis and Parker revealed a dearth of high-quality studies in the area of competency restoration, their article still provides a model template of how to conduct a first-rate systematic review, which can be of benefit to researchers wishing to systematically review other areas of forensic psychiatry research. Similarly, the high-quality publications identified by the review can also serve as exemplars for researchers who aim to expand the evidence base of competency restoration in a meaningful way or contribute to other areas of empirical forensic psychiatry research.

• © 2024 American Academy of Psychiatry and the Law