Research Literacy Is Crucial in Training Forensic Psychiatrists

• research
• training
• curriculum development
• forensic psychiatry
• violence
• bias

Forensic psychiatry as a discipline requires not only excellence in psychiatric evaluation and written and oral communication skills with courts, but also the ability to distill complex concepts, including from research, to a lay audience. Our field demands we be lifelong learners and should want our fellows and future colleagues to have research literacy and to engage in scholarly activities. In addition to report writing, landmark case seminars, and mock trials, I argue that a research component to forensic psychiatry fellowship training is crucial for the development of our future leaders. Here, I am referring to what psychiatrists pursuing forensic careers (outside of research) should know, in order to be able to testify about a topic in court or use findings in their clinical forensic practice. I am not suggesting the level of rigorous training of a research fellowship or of a PhD, but rather that critical analysis of the empirical literature and problem-based learning should be infused into the curriculum throughout the course of the fellowship. To demonstrate that it is readily achievable, I describe what one program has done for half a decade to address research literacy in forensic psychiatry training, which has required only several additional hours of fellow effort per month.

When I worked overseas in New Zealand for seven years, teaching trainees was an important part of my role. Training programs in New Zealand follow guidelines from the Royal Australia New Zealand College of Psychiatrists (RANZCP),¹ similar in many respects to the requirements in American training. Still, there are some marked differences, including, for example, that the fellowship training is at least two years after completing general psychiatry training and that a scholarly research project of some heft is required.

According to the RANZCP, one of the premises behind the requirement of completing a research project in fellowship is that “[t]he practical experience of scholarship is a fundamental part of postgraduate training and this necessarily entails a critique and assimilation of scientific evidence as it applies to their practice” (Ref. 2, Section 1). Included in the learning objectives are conducting a critical appraisal of the literature regarding a topic and then formulating a scholarly question or hypothesis, completing a project to address the question, and presenting the results, discussion, and limitations.

Two years affords a deeper understanding of forensic topics and the forensic patient population. As well, it allows for trainees to complete research projects from start to finish. While I was in New Zealand, trainees completed a vast array of scholarly articles under supervision on topics of their interest, including Pacific Islander forensic mental health, juvenile competency to stand trial, geriatric sexual offending, mental health review tribunals, and the benefits of mock trials in our training program, advancing the field’s empirical knowledge and even winning awards.^3,–,10 (At the American Academy of Psychiatry and the Law (AAPL) 2024 annual meeting debate¹¹ about the length of forensic training, New Zealand colleagues may have been flummoxed about how the wealth of information and research learning could be gathered in the United States in a single year.)

On my return to the United States and Case Western six years ago, when we were reviewing the curriculum, I advocated for the addition of specific research didactics and a regular Journal Club, while mindful that the primary goals in forensic psychiatry training traditionally include evaluations, report writing, and testimony. (For decades, completion of scholarly projects during fellowship has been required, often taking the form of scholarly presentations and Legal Digest writing.)

In the United States, the Accreditation Council for Graduate Medical Education (ACGME) requires scholarly activity, indicating (albeit widely interpreted) that “fellows must participate in developing new knowledge or evaluating research findings” (Ref. 12, Section IV.D.3.a). Elsewhere in the forensic fellowship ACGME requirements, case conferences and seminars about law, ethics, and research are to regularly occur, and it is noted that “journal clubs should be specifically designed to complement the clinical experiences” (Ref. 12, Section Iv.C.5.a).

Recently, Candilis and Parker considered the quality of research on competency restoration.¹³ In their article, they also made an important argument about the need for research training in forensic fellowship, “where coordinated projects and curricula can encourage systematic approaches to forensic research” (Ref. 13, p 153). Candilis and Parker wrote, “While case review remains a crucial element of the subspecialty, high quality research is equally important, for it is research that justifies how evaluations are conducted and which outcomes or interventions are recommended” (Ref. 13, p 159). In his accompanying commentary, Kolla reminded readers that “Forensic psychiatry research is all-encompassing, ranging from case reports and epidemiological investigations to treatment outcome studies and neurobiological experiments” (Ref. 14, p 161).

The general psychiatry residency training literature describes various efforts to integrate research literacy into general training for all residents (separate from specific research tracks). Forehand and colleagues described the “challenge to teach research literacy to psychiatry residents in a palatable manner without any clear ACGME-sanctioned definitions of the core research competencies” (Ref. 15, p 505). Cogswell noted that general psychiatry residents should be “knowledgeable of basic research principles, critically appraise the research literature, apply evidence-based research findings to patient care, and engage in their own scholarly activities” (Ref. 16, p 406). Forehand and colleagues¹⁵ noted that their program determined six research competencies to address: methodology and design, hypothesis development, statistical analysis, literature review, scholarly writing, and research ethics. Cogswell noted that the challenge of the forensic psychology program he directed “was to develop research literacy in a small training program without the extensive research infrastructure in place in some larger programs” (Ref. 16, p 406). Such change is possible even for small programs, in forensics as well as general residencies.

By way of example, Table 1 describes the various components in the research portion of the curriculum at Case Western, each of which adds to the learning experience and which has been well-received by fellows and faculty. Such a research literacy program can be experienced elsewhere along a similar template. Early in the academic year, after seminars have begun which review landmark cases, we begin our review of six historical landmark research articles. Lectures cover reviewing manuscripts, including for journals, with later opportunities to perform a peer review under supervision; and publishing, including about predatory journals and plagiarism. Lectures are followed by Journal Clubs and, of course, mentoring regarding academic presenting, writing, and research.

In our program, six articles are reviewed as historical landmark research.^17,–,22 Although I am not suggesting these are an authoritative list of historical landmark articles, six was a manageable number to add to the didactic curriculum. They were carefully chosen based on my research background and by querying several other forensic psychiatrists. Each article has important teaching points. These articles are discussed in a format similar to a Journal Club, teaching about expectations of critical thinking regarding research articles, prior to beginning Journal Clubs for more recent articles.

Through this lens, discussion of Appleby et al.’s¹⁷ landmark article about suicide utilizing the UK’s National Confidential Inquiry into Suicide and Homicide by People with Mental Illness allows for consideration of the benefits of national databases overseas, the difference in coronial systems between the United States and other countries, and differences in malpractice law and cultural factors, which may contribute to findings and cause difficulty with generalization about suicide. We discuss the potential of bias in physicians’ reflections about whether their own patients’ suicides were preventable. We pause to consider how often people who die by suicide had not been in contact with mental health services in considering what proportion of suicides are actually preventable. In addition to discussing methods and results, we also consider the effects such a study examining all suicide deaths in hospital could have regarding prevention, the ramifications for clinicians when completing suicide risk assessments, and for forensic suicide malpractice evaluations.

In discussing Taylor and Gunn’s early work,¹⁸ which considered men remanded in prison and their rates of psychosis, we talk about the state of the literature at the time and the progress made since this classic paper. We discuss the power of the study and review the inherent biases in considering a population of those who have been remanded, as those who have not been apprehended may be less likely to have mental health problems, for example. We review how data would have been gathered for such a study and whether screening tests or self-report, clinician, or researcher diagnosis would work best in measurement. We also spend significant time discussing that, although a high prevalence of psychiatric disorder was found among violent offenders, “violence was much more commonly committed by psychiatrically normal than by disturbed people” (Ref. 18, p 1947). We also discuss that although the title of the paper is “Violence and psychosis, 1. Risk of violence among psychotic men,” it describes epidemiological data rather than prospectively discussing risk and that titles may not always be fully representative.

Discussion of the work of Fazel and colleagues¹⁹ allows an entry into both rates of serious mental illness among incarcerated populations and the value of systematic reviews in light of many extant smaller surveys on the topic. Learners consider whether studies that include different nations and study samples of prisoners (remand and sentenced, separation by gender) provide information that is generalizable to the United States. Consideration of the different prevalences of mental illness among male and female samples (with females having more burden of mental illness in these populations) is noted, in addition to reviewing simple statistical concepts, such as confidence intervals, and their real-world meaning in our field.

As we reflect on violence risk predictions, Doug Mossman’s²⁰ Guttmacher award-winning and groundbreaking article about “being accurate about accuracy” has unanimously been considered the most challenging of the six topics by fellows. Still, each group has been able to master the information with review and discussion. We review, as Mossman describes, that a study would be “methodologically ideal (if ethically and legally questionable)” if its data were not affected by “influence of treatment decisions on behavior, limited knowledge of actual behavior, or ambiguity in the definition of violence” (Ref. 20, p 784). We review sensitivity and specificity and their tradeoff in quotidian decision-making about violence risk and of course the area under the receiver operating characteristic (ROC) curve. In the analyzed studies, the vast range of time periods during which violence was considered and the type of population studied are notable. Finally, we review the conclusion that “Because clinicians cannot avoid making mistakes, they have to choose what kind of mistakes they prefer to make [regarding violence risk]” (Ref. 20, p 790).

We next discuss in detail the executive summaries of the two MacArthur studies. The book regarding competency to consent to treatment²¹ and the Guttmacher award-winning book²² regarding the MacArthur Violence Risk Assessment Study are also referred to but are too large to be required reading.

In reviewing the MacArthur Treatment Competence Study (executive summary available from author upon request), we first discuss the historical pendulum of rights for people with mental illness. As fellows will recall from their consultation-liaison rotations, most patients with mental illness have the capacity to consent, and we discuss the four abilities described in decision-making competence. The strengths of the study, such as its size and its careful design, are reviewed. We discuss the salient points from the study and some other areas separate from hospital treatment, where the judgment of people with mental illness has been questioned as a whole, for example, regarding voting rights²³ and the capacity to consent to research.²⁴

In considering the MacArthur Violence Risk Assessment Study (executive summary available from author upon request), we discuss the prior serious gaps in research methodology identified by the research group: studying a limited range of potential risk factors, weak measures of violence, lack of careful definitions of violence, single site of study, and limited segment of the patient population. (These gaps still occur in more recent studies.) The groundbreaking elements of the MacArthur study included the diversity of the 134 potential risk factors studied; a large number of subjects at various sites; and, perhaps most importantly, the triangulation of the outcome measures to include violence by self-report, a collateral informant, and official records from hospital and police. We review the meaning of a bivariate relationship of many considered variables to violence and how this differs from multivariate analysis, which allows for a discussion of race versus socioeconomic status and neighborhood. Fellows also reflect that, although these results might be generalizable to others released from the civil psychiatric hospital, this is a different population than the forensic population, in which we are often considering release decisions during their fellowship.

We note the availability of the MacArthur Violence Risk Assessment Study database to researchers for secondary analysis. This allows us to move to discussing MacArthur progeny, including two projects undertaken by former fellows subsequent to fellowship. One considered psychiatric symptoms and alcohol use among those exhibiting community violence,²⁵ and another used the database to consider violence risk of parents toward their children, finding that parents with mental illness who had been treated as inpatients were at lower risk than those in the general community of being violent toward their children.²⁶

In addition to reviewing these six research projects, didactic lectures are used to teach fellows about general concepts in research and writing. Teaching about how to review a manuscript as a reviewer²⁷ allows for critical reasoning about articles and for giving back to our field in the future. (Later in the fellowship, fellows may review a manuscript under mentorship.) The process of publishing is discussed, as are various publishing opportunities that are accessible for fellows in addition to empirical research articles, such as media reviews and Legal Digest articles in The Journal, as well as the Fellow’s Corner of the AAPL Examiner (newsletter). We discuss how to reach one’s target audience when publishing regarding a topic. (For example, an article primarily intended to educate general psychiatrists about a forensic topic versus an in-depth consideration of a forensic topic intended for forensic psychiatrists.) While teaching about publishing, we also discuss predatory journals and their emails to “esteemed colleagues.” Plagiarism (and its avoidance) is also discussed, and I describe my experience of being plagiarized, for example.²⁸

For Journal Club, fellows choose articles to critically review and fellows lead the discussion under my supervision, each presenting articles several times during the year. They choose the articles because of clinical questions and experiences, which may or may not be related to their grand rounds topic or other inquiry. Articles must be empirical research (rather than legal research) and must have been published in the previous five years. The Journal is a frequent source of these articles. Fellows also choose articles of varying quality from other journals, and articles from predatory journals may even be selected (which has led to a discussion about paying to publish and how predatory journals work). Trainees are expected in the Journal Club to lead a critical discussion of the goals of the study, its methods, the results, implications, and limitations. They consider the relevance to clinical forensic questions of interest. Research is explored in a supportive atmosphere (like a child learning to swim from the parent to the wall), and by the end of the year, fellows feel more appropriately confident in discussing what the empirical literature can and cannot tell us about key topics in our field.

Frequently discussed topics are research methods, whether a sample of convenience was used, what the optimal sample would be to answer such questions, and what the hypothesis and research question were versus ad hoc findings and why this difference matters. We consider what a specific article is able to show or not, for example, whether it demonstrates that a concept has validity or whether it demonstrates that there is reliability in psychiatrists’ understanding of a concept after a workshop (e.g., extreme overvalued beliefs²⁹). If trainees are struggling to choose their first article, I have recently pointed them to Dror et al.,³⁰ an article that was the focus of much debate in forensic pathology about bias. Throughout the Journal Club, the linkage between the literature and the forensic question under consideration is stressed. The goal is for fellows to be able to intelligently synthesize the importance and findings of studies in our field, to separate the wheat from the chaff.

Forehand and colleagues noted “By motivating residents to engage in independent investigation, [Problem-Based Learning] may help psychiatry residents cultivate critical appraisal skills, use research findings in clinical decision-making, and produce clinically relevant, patient-oriented outcomes” (Ref. 15, p 505), which is relevant for Journal Clubs and individual projects. Regarding psychiatry resident Journal Clubs, Cogswell noted “In addition to concrete factors (e.g., holding regular meetings; teaching residents how to read the literature), many programs cite adult learning theory (ALT) as underpinning their journal clubs” (Ref. 16, p 406). Using ALT principles included understanding why they were learning about the research, taking responsibility for their own learning, and using their own clinical experience to help pursue knowledge,¹⁶ consistent with experience leading forensic psychiatry fellow Journal Clubs. Cogswell noted that, for psychiatry residents, they held regular required Journal Clubs during didactic time that had consistent faculty leadership and “place[d] an emphasis on teaching commonly encountered methodological topics (e.g., contrasting types of studies) and statistics (e.g., number needed to treat)” (Ref. 16, p 407). This is very similar to the model now used in our fellowship.

In the Case Western fellowship, a didactic session about statistics is not given. Rather, during landmark study review and Journal Club, we discuss why certain methods were used in a particular study and consider basic statistical concepts in small segments over the course of the year, such as power analysis in designing a study, interpreting p values or why t tests were used, or logistic regression (or ROC curves). We consider, as elsewhere in medical training, the importance of knowing when one is out of one’s depth, here regarding statistics. We discuss what studies have the power to reveal versus how they have been misinterpreted. Similarly, there is no exam given on research knowledge, but teaching is through engaged discussion with all participating. Ideally, the results of such encouragement and teaching should lead to inquisitive forensic psychiatrists who are able to grasp nuances from the research literature as well as to collaborate in future research.

Finally, as always, fellows continue to be mentored on scholarly projects, such as academic presentations at grand rounds or local or national presentations, research, and writing. (The interested reader is referred to articles about the New Zealand experience supervising forensic psychiatry trainee research projects.^31,32) Fellows are encouraged to publish in their areas of interest, with mentorship, in media reviews, legal digest, fellows corner, research projects, and online educational materials.

Kolla offered a challenge to forensic psychiatry:We need to consider where the field goes from here. If Candilis and Parker have their druthers, subspecialty training programs in forensic psychiatry would focus their research efforts on teaching robust research methods and developing empirical projects that span multiple fellowship years with the intended consequence of expanding the literature, instructing cohorts of trainees, and emphasizing methods for large-scale collaborations and publication of meta-analyses (Ref. 14, p 163).

He noted that AAPL’s Research Committee seeks to nurture further empirical research by AAPL members and has considered compiling a list of landmark research studies, and I look forward to the fruits of this endeavor.

In considering research experiences for fellows, Candilis and Parker offer this recommendation:It would be salutary for training programs to focus their research efforts on robust research methods and projects, even though such projects may extend across multiple fellowship years. By doing so, a training program will advance the literature, teach cohorts of trainees, and underscore methods that can be applied broadly in future collaborations and meta-analyses (Ref. 13, p 159).

Structuring all fellowships’ research programs this way is an amazing possibility, although one for which not all programs may find themselves prepared. For fellowships that cannot be structured in such a way, perhaps fellows can choose their own mentored research topics.

A year is a limited amount of time to complete an empirical research project and to learn about forensic psychiatry. Often, when a research project is completed during the fellowship, it needed to be started prior to fellowship, because of time requirements, like my own long-ago research projects completed in fellowship.^33,–,36 Although having trainees learn from ongoing research projects at their fellowship’s institution is a highly valuable approach, another consideration is the benefit to trainees from running a project from start to finish (of course with significant supervision). It has been my experience through decades of teaching trainees both in forensic psychiatry and reproductive psychiatry that trainees are more likely to engage in future research and writing if they are able to choose the topic of their research project (as they would choose the topic of their grand rounds). Pursuing my own projects based on both my clinical interests and the lack of literature to answer burning questions, and seeking appropriate mentors, propelled my loves of research and writing throughout my own career, and it has guided my teaching to others as well.

In summary, in addition to report writing training, landmark case seminars, and mock trials, a research component to a forensic psychiatry fellowship training is crucial for the development of our future leaders. Scholarly inquiry should be fully integrated into fellowship. A template has been herein described for building research literacy, which has been used for six years. Similar additions are possible across programs, without significant additional time commitment by fellows or faculty. Fellowship presents a wonderful opportunity to awaken a love of research learning.

• © 2025 American Academy of Psychiatry and the Law