Abstract
Forensic psychiatry fellowship programs recruit applicants through a nonstandardized process that differs by program. Although there are deadlines, informal guidance, and more recent communication guidelines, perceived differences in recruitment practices persist between geographic regions, small and large programs, and newer and more well-established programs. In the wake of a survey of fellowship applicants that found mixed opinions surrounding the application process, U.S. forensic fellowship directors undertook a mixed method quantitative-qualitative survey of their colleagues to assess interest in a match as a potential improvement and factors influencing that interest (e.g., program size, age, and unfilled positions). With responses from all 46 active U.S. programs, results indicated broad support for principles of fairness, transparency, and minimizing pressure on applicants, with an almost perfectly divided interest in a match. Respondents supported the use of a centralized database to standardize the application process and favored certain exceptions for internal applicants. Hypotheses about the reasons underlying program directors’ attitudes toward a match did not yield significant results, with only the size of a program approaching significance. This novel comprehensive survey of forensic fellowship directors offers a model for assessing and monitoring the evolution of application processes for medical subspecialties interested in expanding and improving their recruitment.
When recruiting for the 2022-2023 academic year, fellowship programs in 75 medical subspecialties used the National Resident Matching Program (NRMP) to fill over 11,500 positions across the United States and Canada.1,2 Three of psychiatry’s five American Council on Graduate Medical Education (ACGME)-accredited subspecialties participated in this process: addiction psychiatry, child and adolescent psychiatry, and consultation-liaison psychiatry. Despite widespread adoption of the NRMP, forensic psychiatry has not participated, preferring instead to allow fellowship programs to curate their recruitment activities to their unique needs. As a result, the forensic psychiatry fellowship recruitment process is not standardized or centralized. Programs have consequently varied in their application timelines, requirements, and selection processes.3,4
Prior to 1997, when the American Board of Psychiatry and Neurology (ABPN) recognized forensic psychiatry as a subspecialty and the ACGME began accrediting forensic programs, there were fewer than half the number of programs there are today. Sixteen forensic programs received accreditation in 1997. At that time, there was an informal agreement that program directors would not require responses to offers before November 1, the idea being that many interviews would take place at the annual meeting of the American Academy of Psychiatry and the Law (AAPL) in late October.
With ABPN recognition of the subspecialty, many institutions started new forensic training programs, so the number of available fellowship slots increased dramatically in subsequent years. This increase did not result in a proportionate increase in applicants. In the 2001-2002 academic year, there were 38 accredited programs and 73 on-duty forensic fellows; in the 2022-23 academic year, there were 49 accredited programs offering over 110 positions and 75 on-duty fellows. Thus, as more programs were accredited, the number of unfilled training slots increased.5 This increased the competition for applicants, and the informal agreement between program directors broke down as programs started interviewing applicants earlier and earlier, and some programs made offers with the condition that the offer be accepted within a short period (so-called exploding offers). This led to considerable criticism of the recruitment process and calls for change.4 There was also limited geographic diversity, with a higher density of programs in the Northeastern states. Former AAPL president Richard Frierson underscored the lack of geographic diversity among programs in his 2019 presidential address, highlighting the lack of data on unfilled positions because of the absence of standardized recruitment processes.6
In 2019, after years of dissatisfaction with the traditional recruitment approach but no consensus about a match, the Association of Directors of Forensic Psychiatry Fellowships (ADFPF), an organization of program directors and associate program directors, undertook the creation of a semistandardized recruitment system. ADFPF developed common application forms and requirements, a common timeline for interviews, and guidelines for how offers could be made. These procedures were revised annually based on feedback from program directors at ADFPF meetings. For the 2022-2023 fellowship year, the interview season began on April 1, 2021. Program directors were prohibited from extending time-limited offers that might encourage applicants to accept a position in haste and cancel subsequent interviews. Once an offer was made, applicants could not be pressed to accept or decline prior to June 1, 2021, although they could accept an offer sooner if they were not pressured to do so. Internal applicants were exempted from the ADFPF guidelines; program directors could interview and offer them positions at any time.
Despite attempts at improving the recruitment process, several challenges remained. A 2021 survey of forensic psychiatry applicants across three fellowship recruitment cycles found that 30 percent of respondents felt pressured to accept an offer before the earliest date set by ADFPF, and 58 percent of respondents were not satisfied with the existing process.7 No consensus existed about how to remedy the situation, with 49 percent of respondents supporting a match as a potential solution and 51 percent opposing it.
After completing recruitment for the 2022-2023 academic year, ADFPF formed a special committee to design a survey of program directors that tested hypotheses raised by the applicant survey. Committee members developed the hypotheses by consensus based on the applicant survey results and members’ familiarity with years of discussion during ADFPF meetings. Specific hypotheses tested whether programs that favored a match were younger, smaller, and unfilled and had less confidence in the current process.
A committee subgroup (authors PC, PA, RB, and OC) subsequently designed and circulated a survey among program directors to solicit feedback on the existing application process, to gauge the degree of consensus about the use of a match, and to identify common reasons for and against a match. An additional objective was to identify strategies to streamline the recruitment process for applicants and programs. These goals were considered crucial to expanding and invigorating the subspecialty and to filling more fellowship positions over time.
Methods
In March 2022, the ADFPF special committee distributed a SurveyMonkey instrument via the organization’s listserv to gather the views of all program directors of U.S. forensic psychiatry training programs that were accredited by ACGME and that were actively recruiting fellows (n = 46). The Institutional Review Board of Saint Elizabeths Hospital and Department of Behavioral Health in Washington, DC exempted the project from review. Following three email reminders, a committee cochair (PA) collected responses from a program director or associate program director at each program, achieving a 100 percent response rate. Program directors described their support for the existing semistandardized approach to recruitment, including the timeline for interviews, prohibition on pressuring applicants to accept offers quickly, and exemption of internal candidates. Survey questions also queried support for a match. Questions were designed on a five-point Likert scale (strongly support, support, neutral, oppose, and strongly oppose) and included both closed and open-ended responses. When a survey question asked about pros and cons of a problem, respondents were permitted to provide up to three pro and three con responses.
In the data analysis, four hypotheses were tested. Based on discussions during ADFPF meetings, the committee had hypothesized that smaller programs would support a match at a higher rate because they believe a match could help them compete with larger programs. To test this hypothesis with a small sample, fellowship programs were divided into two categories: those offering two or fewer positions and those offering three or more.
Second, the committee hypothesized that older, established programs would offer less support for a match because the current recruitment process is working well for them. Two indicator variables were created to represent the length of time a program was in existence: fewer than 10 years versus 10 or more years and fewer than 20 years versus 20 or more years. Each indicator variable was tested separately when evaluating the relationship with support for a match.
A third hypothesis tested whether a program’s support for the match was associated with its belief that the match would result in other programs behaving more ethically during the recruitment process. Programs that chose the “neutral” or “not closely” options in response to the question, “How closely do other programs adhere to the ‘no pressure’ guideline?” were grouped together and examined for their support for a match.
Finally, the hypothesis that programs with more unfilled positions would favor a match was tested, essentially positing that a match could overcome the traditional advantage of established programs that filled quickly.
To test each of the four hypotheses, the chi-square test, a test of independence of two variables, was performed. Statistical significance was set at P < 0.05. Because of the small sample size and a spread of responses across Likert scales, response options were ultimately collapsed into three categories: strongly support or support, strongly oppose or oppose, and neutral. SAS 9.4 software was used for all analyses.8
Open-ended responses were coded by a three-member coding team (OC, MD, PC). Qualitative analyses began with the team’s familiarization with the material through repeated readings. Subsequently, the coding was conducted by examining responses with a common meaning within the text.9,10 The analysis focused on the manifest content of responses with the wording of the respondents applied in creating codes. Moreover, to achieve credibility, a triangulation method (“analyst triangulation,” use of a multiperson team to review findings and reduce bias) was applied as the coding was created and reviewed.11
Results
All active accredited forensic psychiatry fellowship programs in the United States in 2021 completed the survey (n = 46). Twenty-four programs were in existence 20 years or more (53%), eight programs between 10 and 20 years (18%), and 13 programs less than 10 years (29%). One program did not respond to the question on longevity. Thirty-two programs offered one or two fellowship positions, with 14 offering three or more. Table 1 illustrates the age and size of fellowship programs at the time of the survey (March 2022).
Demographics of Forensic Psychiatry Fellowship Programs
In response to preliminary questions, most programs (65%) expressed support for a centralized application database (perhaps similar to the Electronic Residency Application Service (ERAS)) because it could streamline numerous applications.12 When asked about the recruitment timeline, respondents indicated the greatest support for August 1 as the date for beginning the interview season (28 of 46, 61%) and October 3 for the earliest date that offers could be made to applicants (24 of 46, 52%). The exception for interviewing and accepting internal applicants outside these timelines was supported overwhelmingly (41 of 46, 89%). Two-thirds of programs (67%) favored the implementation of explicit communication guidelines to uphold the “no pressure principle” (i.e., not to pressure applicants to commit to a program ahead of consensus timelines). In related open-ended responses, programs expressed a desire to enhance fairness and equity in the recruitment process by utilizing a common system and applying the “no pressure” principle.
Respondents were evenly split in their support for a match (Fig. 1). Seventeen of 46 respondents (37%) were in favor of some form of match, with 17 opposed (37%) and 12 neutral (26%). Eighty-three percent were in favor of a match algorithm that allowed for an exception for internal applicants, whereas 51 percent of program directors opposed a model that resembled the existing psychiatry residency match, largely because it precluded extending early offers to internal applicants. Respondents indicated that an application process that lacked an exception for internal candidates would lead to significant recruitment challenges because a large proportion of applicants matriculate to forensic fellowships from their own residency programs (10 responses, e.g., “impacts internal candidates who want to stay”; “no option for internal applicants to opt out”).
Figure 1. Support for some type of fellowship match.
There was no statistically significant association between program size and attitude toward the match process (chi-square(2) = 1.106, P = 0.575); the distribution was equivalent by size of program.
Neither age analysis (setting program age at 10 or 20 years) resulted in a statistically significant association with support for the match (10 years: chi-square(2) = 3.676, P = 0.159; 20 years: chi-square (2) = 0.365, P = 0.833). Programs that had been in existence for less than 10 years, however, were more likely to support a match (57%) compared with those more than 10 years old (28%, chi-square: 3.676 (df = 2), P = 0.159). There was less of a difference in support for a match when testing the 20-year parameter, comparing match preferences for programs that were less than 20 years old with those more than 20.
Although nearly all respondents (43 of 46, 93%) believed their own program adhered to ADFPF guidelines regarding pressure-free applications, only 15 percent (7 of 46) had the same confidence in other programs’ adherence. In support of their skepticism about other programs’ behavior, respondents cited information from other program directors and applicants themselves who described receiving time-limited offers and offers expiring before the deadline (12 responses, e.g., “applicants tell me they have been pressured by other programs,” “as we heard in our meeting other programs admitted to withdrawing their acceptance offer early”). Respondents believed adherence could be improved by educating applicants about the rules, so it was clear which programs followed them and which did not (nine responses; e.g., “wider dissemination of the rules to applicants,” “a bill of rights for applicants”), whereas others recommended public disclosure of the programs that did not adhere to guidance (nine responses; e.g., “publicizing programs…to other members,” “making violations public”). There were no statistically significant associations between respondents’ interest in a match and their concerns with programs following “no-pressure” guidelines (chi-square(4) = 2.801, P = 0.592). Overwhelmingly, respondents favored keeping the “no-pressure” guidelines, as demonstrated in Figure 2.
Figure 2. Agreement with the implementation of explicit communication guidelines to adhere to a “no pressure” principle.
The hypothesized association to unfilled positions did not meet traditional criteria for accepting the chi-square result because the sample size was too small.13 The expected values of half the chi-square cells in the cross-tabulation of match support and unfilled positions were less than five, leading to unreliable results, so by rule, the third hypothesis could not be evaluated.13 This was confirmed by use of Fisher’s exact test.
In open-ended responses to the question about “pros and cons for some type of match,” respondents appreciated the benefits of a standardized process (20 responses; e.g., “a uniform process,” “consistent and predictable”), general fairness (12 responses; e.g., “more equitable,” “fairness for all parties,” “equal opportunities for all applicants”), and decreased pressure on applicants (11 responses; e.g., “takes pressure off applicants,” “less pressure for early acceptances”). Cons described deleterious effects on flexibility and innovation in recruiting (23 responses; e.g., “rigid,” “less ability to be innovative”), cost and administrative burdens (22 responses; e.g., “additional costs,” “more admin work”), and effects on applications (19 responses; e.g., “more applications per applicant,” “need to interview more applicants”).
Discussion
The current survey represents the most comprehensive review to date of forensic psychiatry fellowship directors’ views on the recruitment process, with 100 percent of U.S. programs responding. Respondents generally agreed on the important principles for recruitment, including fairness, equity, and transparency. They also agreed upon conducting interviews during a two-month period beginning in August each year and requiring programs to allow applicants sufficient time to consider their options before accepting a position. Respondents generally agreed that utilizing a centralized application system (e.g., Electronic Residency Application Service (ERAS)) would improve the current system.
Respondents strongly favored a system that treats internal and external applicants differently, with a large proportion (89%) favoring acceptance of internal applicants outside the timelines and guidelines developed for external candidates. Program directors supported their preference by indicating that such a process is easier for internal applicants who have family and geographic needs and that it is important for institutions to prioritize applicants already within their systems. Yet the exception for internal applicants amounts to a privilege for candidates from residency programs with a forensic psychiatry fellowship, raising equity concerns about the recruitment process. This was a minority view among the survey respondents, who generally stated that this element of the system did not require adjustment.
Programs’ distrust of others’ adherence to the established ADFPF guidelines, with only 15 percent having confidence in their colleagues, stands in sharp contrast to their assessment of their own behavior, with 93 percent reporting full confidence in their own adherence to recruitment guidelines. It is unclear what accounts for this disparity. Nonetheless, a more united, transparent, and collegial presentation by programs could improve the recruitment experience for future colleagues entering the profession.
The lack of consensus on whether to utilize a match aligned with results from the 2021 survey of fellowship applicants: respondents were evenly split between those who favored a match and those who did not.7 There are no directly comparable data from surveys about the match in other medical specialties or empirical support for concerns that a match affects creativity or innovation in the recruitment process. Program Directors reported anecdotally during ADFPF meetings that applications increased following the adoption of a common application process and virtual interviewing. These reports are consistent with data from medical specialties that use the NRMP match, which found a substantial increase in applications over time, especially in competitive fields with more applicants than positions.14,15 Thus, survey respondents’ concerns about an increased burden on program directors stemming from a formal match may be well founded.
Although the current survey explored correlations between opinions on a match and characteristics of fellowship programs (program size, program age, number of unfilled positions, and attitudes about other programs’ adherence to recruitment guidelines), none of these factors had a statistically significant association with attitudes on the match, although program age less than 10 years approached significance. This result may support the hypothesis that younger programs feel that a match “levels the playing field” and allows them to compete with older, more established programs. Further study is complicated by the small number of programs and the need for large differences in attitudes to achieve significance in a small sample. Enlarging future samples to include associate program directors may result in a broader range of opinions and improved statistical significance.
The study has some limitations. Despite a 100 percent response rate, the sample size was still too small to obtain statistically significant associations in support of four specific hypotheses; there was too much of a spread in responses and insufficient cell sizes to allow the collapsing of cells. In addition, one response was solicited from each fellowship program, so any disagreement between leaders in the same program (e.g., between program director and associate program director(s)) would not be captured. Finally, the study was conducted in March 2022, so it reflects program directors’ views at that time. Effective January 11, 2023, ADFPF made changes to the recruitment guidelines for the 2024-2025 fellowship year, including shifting the interview season from April to September, instituting a three-week “no offers or acceptance” period, and developing a centralized database to report accepted offers, as outlined in Table 2. Thus, the survey results should be interpreted as a reflection of program directors’ opinions in 2022, rather than a contemporaneous view of evolving forensic fellowship recruitment.
Timeline for 2024-2025 Forensic Fellowship Year
Conclusion
Forensic psychiatry fellowship recruitment continues to develop, with the current study demonstrating that, as of March 2022, there was no consensus among program directors about whether a match would improve the process. This lack of consensus is similar to a 2021 survey of fellowship applicants, who were also evenly split about preferring a match to the current process. Nonetheless, a level of dissatisfaction and mistrust among both program directors and applicants necessitates continued review of fellowship recruitment using tools such as this mixed-method quantitative-qualitative survey.
Based on the current survey, program directors prefer incremental improvements over widespread reform. Program directors expressed specific support for developing a centralized electronic application system, utilizing consistent time frames and communication guidelines for offering positions, and allowing flexibility in the recruitment of internal applicants. This information will guide the future efforts of ADFPF in refining the recruitment process and developing a system that better meets the needs of fellowship programs and applicants.
Footnotes
Disclosures of financial or other potential conflicts of interest: None.
- © American Academy of Psychiatry and the Law
