Abstract
Psychiatrists without specialty forensic training routinely encounter forensic questions and treat justice-involved patients, which underscores the importance of adequate forensic training in general psychiatry residency. Some programs may face challenges providing adequate forensic instruction due to a lack of local forensic psychiatrists or other forensic resources. Novel training approaches are needed to fill this gap. This article describes the development, dissemination, and preliminary impact of two online learning modules designed to teach general psychiatry residents about basic forensic psychiatry principles: confidentiality and the duty to third parties. The modules are based on adult learning theory and synthesize clinically relevant vignettes from historically significant legal cases. We disseminated the modules nationally with built-in pre-tests and post-tests. The module responses demonstrate three significant themes: the modules reached learners across the United States; even advanced residents had relatively low subject matter knowledge prior to module completion; and module completion was associated with a significant improvement in resident learners' knowledge of these two topics. This work shows one potential avenue for filling gaps in forensic education within general psychiatry training, although further studies are needed to appreciate the impact of such educational interventions on clinical practice.
The practice of forensic psychiatry is not limited to forensic settings. Psychiatrists, in general and in subspecialty settings, routinely encounter forensic questions.1 Statutes and legal precedents regulate numerous aspects of the clinical practice of psychiatry, including confidentiality, informed consent, civil commitment, and involuntary medication administration, among others.2,–,4 Further, many patients with mental illness interface with the criminal justice system5 and require treatment by psychiatrists knowledgeable about relevant laws and clinical concerns pertinent to justice-involved patients.1,6,7 At present, however, the number of subspecialty-trained psychiatrists is insufficient to meet these needs effectively.8,9 The growing numbers of individuals with mental illness who are involved in the criminal justice system and the movement of forensic patients from institutions to communities means that general psychiatrists are increasingly likely to work with justice-involved individuals.10 Therefore, all psychiatrists must be familiar with basic forensic principles applicable to clinical practice.6
The American Council for Graduate Medical Education (ACGME) requires exposure to forensics in general psychiatry training,11 although the actual exposures vary considerably across programs.1,6 The literature on novel methods for teaching forensics to general psychiatrists is developing,2,12,–,15 including evidence that both clinical and didactic exposure are important and may be complementary.6,16 But with only 2,240 U.S. psychiatrists ever having been board-certified in forensic psychiatry,17 it is unlikely that there are sufficient forensic psychiatrists to provide direct education to all residents. Further, the number of practicing forensic psychiatrists currently available to teach trainees is even lower, given retirements, deaths, and those who may not be engaged with academic training programs.
To further visualize the distribution of forensic psychiatrists available to teach general residents, we used U.S. census data to calculate the number of forensic psychiatrists per 100,000 residents in each state and U.S. census region (Table 1).18 This revealed wide inter-state variation, with the number of forensic psychiatrists per capita notably higher in New England, in states along the Atlantic coast, and in states with forensic fellowship programs. These numbers suggest an uneven distribution of potential forensic faculty resources, which may contribute to a dearth of forensic education resources in some regions.
National Distribution of Board-Certified Forensic Psychiatrists
Many residency training programs lack a forensic psychiatry department, and some may lack board-certified forensic psychiatrists on their faculty. It follows that some programs may face challenges developing a forensic curriculum and struggle with providing residents with adequate exposure to core legal and ethics principles. This concern is supported by a Canadian survey in which residents reported lack of experience and significant discomfort with forensic psychiatry6 and by another study that reported a significant lack of knowledge and improper application of local commitment laws in theoretical cases.19
Given these training concerns, it behooves forensic educators to approach the development and dissemination of educational content creatively. In both undergraduate and graduate medical education, educators are studying and implementing innovative approaches to augment or replace traditional classroom environments.20 Studies support many of these techniques as productive teaching methods, including problem-based learning, simulations, and online learning networks.15,21,22 Educators are employing these training methods as both primary and supplementary learning experiences in many other specialties, including internal medicine, pediatrics, and anesthesiology.23,–,25
Central to the design of these educational experiences is the recognition of trainees as adult learners who possess internal motivation and autonomy, have prior clinical experiences to draw upon, and seek problem-solving approaches in their education that can be applied directly to their clinical work.26 Using these principles, innovative curricula aim to teach principles and explore clinical skills using cases that allow the learner to apply the new information directly to clinical practice.
Online learning modules make use of technology that allows dissemination of educational content to a wider audience and therefore may be ideally suited for meeting the needs of programs with fewer forensic resources. Further, the use of online modules provides an opportunity to ensure consistent and high-quality forensic education to residents.
In this article, we describe the development, dissemination, and initial impact of two online learning modules designed to teach general psychiatry residents about two basic forensic psychiatry principles: confidentiality and the duty to third parties (i.e., Tarasoff warnings). We have incorporated adult learning theory by synthesizing historically significant legal cases into clinical vignettes, which help contextualize and highlight relevance for important legal principles. The modules incorporated pre-test and post-test assessments to evaluate learners' knowledge. We hypothesized that subjects' test performance would improve following completion of each module and that these results would not differ significantly based on gender, postgraduate year in training, or geographic region.
Methods
We created two interactive online modules to introduce general psychiatry residents to core concepts of confidentiality and duties to third parties. The modules introduced core principles of each topic in a clinically relevant and succinct format, relying on principles from adult learning theory.27 Malcolm Knowles popularized the concept of andragogy, which is the art and science of helping adults learn, and posited a set of assumptions that distinguished adult learners from child learners.28 Knowles asserted that adult learners are increasingly self-directed, draw on their accumulated reservoir of life experiences, are problem-centered and interested in applying their learning in real time and to real world scenarios, and more greatly motivated by internal factors. Educators of adult learners should create a cooperative climate for learning; direct their teaching to the learners' specific needs; develop learning objectives based on the learners' specific needs, interests, and skills; work collaboratively with learners; and evaluate the quality of the learning experience for ongoing improvement.28
With these theories in mind, we designed interactive modules based on case vignettes that are problem-focused, are directed toward the specific learning needs of resident psychiatrists, incorporate interactive components, and include pre- and post-module evaluations. The case vignettes place the learner in the role of a treating psychiatrist and ask the learner to consider how to approach the clinical scenario in a “choose your own adventure” format. The vignettes are presented through a combination of written and interactive static visual content. Upon completion of the clinical case portion of the module, the learner is provided the opportunity for self-directed learning about the actual legal case and the case's historic significance in written and video format. Finally, each module concludes with an interactive presentation that introduces the learner to the relevant core clinical concepts for each topic. Didactic content is focused on core concepts, rather than on reviewing all current state-specific regulations. The modules highlight these dynamic elements in regulations, including interstate and historical variation, and identify resources for determining local state laws. The modules incorporate an assessment of participants' knowledge of core concepts prior to and after completing each module (i.e., a pre-test and a post-test) to assess the modules' effectiveness in improving subject matter knowledge. The modules were developed utilizing Articulate Software.29 Each module requires approximately 20–30 minutes to complete.
The modules were freely available to any interested learner. We shared them with other residency programs using the American Association of Directors of Psychiatric Residency Training (AADPRT) and the American Academy of Psychiatry and the Law (AAPL) Resident Forensic Training Committee listservs. The modules incorporated a request that the learner provide informed consent for collection of demographic and pre-test/post-test scores. Residents' consent for data collection was voluntary and did not affect their ability to participate in the learning activity (i.e., they could decline to have their data collected but still use the module). This work was deemed exempt from review by the Yale University Institutional Review Board.
For both modules, the best possible score on both the pre-test and post-test was achieved by answering all 10 questions correctly. The primary outcome measure assessed in this study was the degree of improvement between participants' pre-test and post-test scores following completion of the module. We also collected limited demographic information about participants and controlled for this in the statistical analysis (i.e., gender, residency program's U.S. census geographic region,18 and year in training) (Table 2).
Demographic Information for Subjects
Results
Sample Characteristics
A total of 120 residents from 39 residency programs completed the confidentiality module. The gender split was 47 percent male, 48 percent female (6% of participants did not identify their gender). Residents in their fourth postgraduate year (PGY-4) made up the highest proportion of participants at 26 percent, followed by PGY-2 residents at 23 percent. Eight of 10 geographic census regions were represented, with New England having the greatest representation (39%), followed by the Pacific (20%) and Middle Atlantic (14%) regions. The East North Central and East South Central regions had the lowest representation (i.e., 0 participants each).
A total of 90 residents from 20 residency programs completed the duties to third parties module. There was a slight gender shift toward female participants (47% female, 42% males; 11% did not identify their gender). PGY-2 and PGY-4 residents were again the highest proportion of participants (24% each). Seven of eight geographic regions were represented, with the New England (49%), Pacific (17%), and Middle Atlantic (14%) regions again having the highest representation, and the East North Central, East South Central, and Puerto Rico regions with the lowest (i.e., 0 participants each).
Pre-Test Performance
Table 3 summarizes the mean pre-test performance of all participants on each module overall and broken down by gender, year in training, and geographic region (excluding participants who declined to answer these demographic questions). For both modules, there were no statistically significant differences on the pre-test results by gender (confidentiality: t(110.67) = 1.89, P = .061; duties to third parties: t(75.84) = 0.11, P = .912) or year in training (confidentiality: F(5,94) = .91, P = .481; duties to third parties module: F(5,69) = 0.84, P = .530). The confidentiality module's mean pre-test score varied significantly by region (F(7,111) = 2.56, P = .018), but the duties to third parties module mean pre-test score did not (F(6,80) = 0.69, P = .656).
Module Pre-Test Mean Results
Post-Test Performance
The confidentiality module's average post-test score was 7.4, representing a 2.83-point mean improvement from the average pre-test score of 4.57, which was statistically significant (t(119) = 16.51, P < .001; Fig. 1). Improvement on the post-test did not differ significantly by gender (t(111) = 1.06, P = .290) or geographic region (F(8,111) = 1.19, I = .314).
Pre- and post-test results on each module.
Improvement results did significantly differ between training years (F(6,113) = 2.58, P = .022; Table 4). Based on observed means, a post hoc analysis was performed to compare the difference between the average PGY-1 post-test improvement and all other training years (excluding those who declined to answer). This analysis demonstrated that PGY-1 residents' post-test improvement was significantly greater than all other years of training (t(96) = 2.92, P = .004).
Confidentiality Module Post-Test Improvement by Year in Training
For the duties to third parties module, the average post-test score was 7.19, representing a 2.3-point improvement from the average pre-test score of 4.89, which was also statistically significant (t(89) = 11.60, P < .001; Fig. 1). For this module, post-test improvement did not differ significantly by gender (t(78) = −0.75, P = .453), geographic region (F(7,82) = 1.73, P = .113), or year of training (F(6,83) = 0.19, P = .979).
Discussion
This study adds to recent literature on forensic education by evaluating the potential impact of online modules. This online, interactive, clinical case–based educational modality allows for broad distribution and represents a potential mechanism to address the forensic educational needs of residents around the country. Such technology-based approaches have been successfully implemented in other medical specialties,21,–,25 and our study supports similar efforts in forensic education for general psychiatry residents.
As anticipated, residents' performance on post-tests significantly improved after completion of the online educational modules. This result is not surprising, given the common experience of many educators that focused educational interventions, including those that are technology based, can lead to short-term improvements in content knowledge acquisition.21,23,25
The study results also support the use of online educational approaches to reach vast geographic areas. The confidentiality module reached 39 individual residency programs spanning eight of 10 national U.S. census regions, and the duties to third parties module reached 20 individual programs and seven U.S. regions.
Interestingly, pre-test scores did not differ significantly between training years on either module, with average scores below 50 percent on both pre-tests. Anecdotally one might expect more advanced residents to perform better on the pre-tests than their junior colleagues given their greater training, completion of the required forensic exposure, and clinical experience. This finding highlights the insufficient forensic training across general psychiatry residency, as well as the lack of basic forensic knowledge possessed by advanced residents who will soon be entering practice. This further supports the need to develop additional forensic training interventions in general residency programs, including novel educational programs and resources (such as those described here).
Pre-test scores for the confidentiality module varied significantly by geographic region, whereas pre-test scores for the duties to third parties module did not; this difference was most likely attributable to the advanced stage of training of respondents from one region (West South Central). Similarly, New England residents, on the basis of their pre-test scores, did not seem to benefit from the greater number of available forensic psychiatrists in the region, suggesting that potential access to specialists alone may not correspond with access to high-quality forensic training. This highlights that additional factors, such as local forensic psychiatrists' teaching interest and aptitude, may influence residents' learning.
Although not part of our hypothesis, it is not surprising that PGY-1 residents' performance improved significantly more on the confidentiality module post-test compared with other years of training. Although not significant, the average pre-test score on this module for PGY-1 residents was lower than any other year's (i.e., PGY-1 was 3.92, PGY-2 was 4.79). One potential explanation is that PGY-1 residents are still acclimating to the physician role and are the least experienced with confidentiality concepts in psychiatry. Residents' clinical experience with confidentiality in psychiatric settings typically increases in the second postgraduate year, which is often spent on inpatient psychiatry rotations. Residents showed relatively uniform lack of initial knowledge and significant improvement in post-test scores regardless of year for the duties to third parties module.
This study has several limitations on the basis of the sample size, which limits the power of analyses, and the sampling method. Study respondents from New England residency programs were over-represented in this study compared with other regions (39% and 49% of participants, respectively). Considering that New England–based programs make up only 20 of 271 (7%) of ACGME accredited adult psychiatry residency programs, this region was significantly over-represented in the sample. This finding, along with the small or absent sample sizes from several other geographic regions, raises questions about the generalizability of the results nationally. It is somewhat reassuring, however, that the score improvements did not differ when controlling for geographic region. Another potential limitation was the dissemination of the modules through the AADPRT listserv, which may have contributed to a self-selection bias in two respects that may further limit generalizability. First, it may have selected for program directors with particular interest in making use of the modules, either because of a baseline interest in forensics or because of an identified need for additional forensic programming. Second, residency directors likely made module completion voluntary, which may have further self-selected for residents with a greater interest or desire for education in these topics. In addition, only two modules were tested, and it is uncertain whether similar results could be replicated across other knowledge content areas. Finally, the post-test occurred immediately after completing the module, so it is unclear whether knowledge acquisition was maintained over time and or how it affected clinical practice, which is the educational gold standard. Moving forward, it will be important to develop mechanisms to assess both in greater detail.
Conclusion
We have described the development, dissemination, and impact of two online learning modules designed to teach general psychiatry residents about basic forensic psychiatry principles of confidentiality and duty to third parties. Our data show that the modules reached learners spread across the United States, that even advanced residents had relatively low subject matter knowledge prior to module completion, and that completion was associated with a statistically significant improvement in residents' knowledge. Further work will be required to fully understand longer-term knowledge retention and clinical impact of such education, but, given the high demand for general psychiatrists to work with forensic patients, resident discomfort with forensic psychiatry, and variable forensic offerings among training programs, this work presents one potential avenue for improving the foundational forensic education of general psychiatry residents.
Footnotes
This work was supported by an educational grant from the AAPL Institute for Education and Research (AIER). Dr. Wasser acknowledges the support of the Connecticut Department of Mental Health and Addiction Services. Dr. Wasser is involved in the editorial leadership of The Journal; however, he did not participate in any aspect of this article's review and acceptance.
Disclosures of financial or other potential conflicts of interest: None.
- © 2020 American Academy of Psychiatry and the Law
