Abstract
Organizing frameworks that affect medical education include requirements for residency education set out by the Accreditation Council for Graduate Medical Education (ACGME), board certification requirements of the American Board of Psychiatry and Neurology (ABPN), and requirements of the Accreditation Council for Continuing Medical Education (ACCME) for awarding continuing medical education (CME) credits. Physicians rely on a variety of sources to meet these requirements, including medical journals. It is unclear, however, whether journals actually assist physicians in meeting these educational requirements. In this review of articles, we make the first known attempt at a systematic comparison of an academic journal's content areas to national standardized educational indices for physicians. Findings from the 2008 to 2012 content of The Journal of the American Academy of Psychiatry and the Law demonstrated that The Journal's articles cover the gamut of board certification examination topics, although content areas are not weighted in parallel with the examination. Some overlap and differences were seen when comparing journal content with ACGME topic areas. The Journal appears to meet identified gaps in knowledge that CME can address. The importance of balancing readers' educational needs with the ability to provide a resource for unique topics is discussed, along with other implications of these findings.
One of the main goals of the American Academy of Psychiatry and the Law (AAPL) is to promote scientific and educational activities in forensic psychiatry.1 To accomplish this goal, AAPL facilitates the exchange of ideas and practical clinical experience through publications and regularly scheduled national meetings at which continuing medical education is offered. The two main meetings include the annual meeting of the organization and the semiannual meeting that takes place during the annual meeting of the American Psychiatric Association (APA). AAPL also supports regional meeting activities, develops practice guidelines, publishes ethics guidelines, contributes to national conversations within organizations of medicine and psychiatry, and stimulates education and research through the provision of financial grants by the AAPL Institute for Education and Research (AIER). The Journal of the American Academy of Psychiatry and the Law, the main publication of AAPL, is intended to be a forum for the exchange of multidisciplinary ideas. Its content includes correctional psychiatry; psychiatric evaluations of persons involved with the criminal or civil legal system; ethics; the philosophy of law; legal regulation of psychiatric practice; education and training in the field; and research into the causes and treatment of behavioral problems that manifest themselves in individuals in contact with the legal system (Page ii of any volume of The Journal).
A professional journal is often measured by its impact factor, which relates to how often its articles are cited elsewhere in scientific publications. The impact factor seems to correlate with physician satisfaction with a journal's content.2 However, forensic journals have been found to have low impact factors, which may be related to having a smaller following in a field where scientific research is not as voluminous.3 Thus, although impact factors can be important in scientific publishing, they do not address whether the content of a journal meets the professional educational needs of a readership.
Because professional journals are a common way to learn the more recent information about a field of medicine, it is important to ensure that the topics covered by a journal relate to what a practicing physician needs to learn. The knowledge base necessary for a physician to be competent in practicing or teaching forensic psychiatry is somewhat elusive, but there are two potential sources for definition: The American Board of Psychiatry and Neurology (ABPN) and the Accreditation Council for Graduate Medical Education (ACGME). The ABPN, which is one of many boards that are overseen by the American Board of Medical Specialties (ABMS), is the entity that is responsible for initial board certification and maintenance of certification (MOC) for forensic psychiatrists in the United States. Part of the certification or recertification process involves a written examination for which ABPN has developed a content outline, a topic list outlining the appropriate skills and knowledge necessary for a competent forensic psychiatrist to engage in the practice and teaching of the subspecialty. The ACGME accredits all medical residencies and fellowships in the United States, including fellowships in forensic psychiatry. One of its components, the Residency Review Committee (RRC) for Psychiatry, develops program requirements for all psychiatric specialties, including forensic psychiatry fellowships. These program requirements contain a didactic topic list, an additional source of information for what a forensic psychiatrist should know to practice competently.
The Accreditation Council for Continuing Medical Education (ACCME) has as a mission to identify, develop, and promote quality continuing medical education for “maintenance of competence and incorporation of new knowledge to improve quality medical care for patients and their communities.”4 The ACCME is not focused on psychiatry or forensic psychiatry as a whole (nor does it create a topic list of what should be taught in any specialty), but it asks that entities that seek to provide CME credits for their educational activities comply with the Council's standards. It also asks that these entities identify practice gaps that the educational activities seek to fill. For example, when this requirement was promulgated, the AAPL Education Committee identified three broad professional practice gaps that the organization's educational activities should address:
Not practicing psychiatry at the highest level attainable based on current knowledge of the fundamentals of the field;
Lacking the knowledge of content or technique to teach psychiatrists the fundamentals of forensic psychiatry in the most effective ways;
Lacking the ability to conduct or assess research in forensic psychiatry.
Each overarching gap was attached to an identified need, such as improvement of knowledge in areas of civil, criminal, and correctional forensic psychiatry; gaining knowledge of new content and effective ways of teaching forensic psychiatry; and understanding how to conduct research, analyze the outcomes, and apply the outcomes to forensic practice.
Because these various national oversight bodies coexist and shape directions for enhancing knowledge of physicians in particular ways and on particular content areas, it is important to ensure that the activities of an organization such as AAPL, whose main mission is education, match the national requirements. Moreover, these national standards can be a helpful tool in assessing how well the main educational products of the organization, such as its academic journal, meet its stated educational goals.
The purpose of this article is to compare the content of The Journal over the past five years (2008–2012) to the 2013 ABPN MOC Examination content outline and the current ACGME program requirements of fellowships in forensic psychiatry, to identify strengths and weaknesses in this dimension of The Journal's educational performance. In reviewing the content, we also discuss the relationship of The Journal to the educational mission of AAPL overall, its efforts to assist AAPL members with ABPN maintenance of certification (MOC), and its efforts to assist in skill development, as seen through a recent emphasis on the development and publication of practice guidelines.
Method
All Special, Regular, and Analysis and Commentary articles published in The Journal from 2008 to 2012 were selected for review. One of the authors (R.F.) took the role of primary reviewer and consulted with the other author if a particular article was difficult to classify. Each reviewed article was categorized into one section of the ABPN Content Outline for the Maintenance of Certification Examination in Forensic Psychiatry and the ACGME program requirements for forensic psychiatry. The authors specifically chose these organizational educational requirements as a framework for review, as opposed to comparing The Journal to other journals, because these requirements have been vetted over the years and are recognized educational benchmarks in the field. Commentaries on Special or Regular Articles, Reflections, Editorials, Memorials, Legal Digest submissions (i.e., case reviews), and Book and Media Reviews were not included in the review because they were not considered primary content areas, but reflections on content or other published material. In addition, practice guidelines were not considered, but will be discussed later in this report.
The ABPN MOC Examination Content Outline
The content outline for the ABPN MOC Examination can be found in Table 1. Its makeup is controlled by the ABPN Committee on Certification of Added Qualifications in Forensic Psychiatry.5 Within each broad category, there are several subcategories, and the Committee for Certification can modify content areas as they deem appropriate to meet current standards. For example, recently, the subcategory of Corrections was expanded significantly.
The ACGME Program Requirements for Fellowships in Forensic Psychiatry
The current program requirements for forensic psychiatry fellowships have been in effect since 2003. At the time of this writing, they were undergoing revision by the ACGME. Unlike the ABPN MOC Examination Content Outline, the ACGME does not assign weighted percentages devoted to each part of the curriculum, in part to allow programmatic flexibility among fellowship programs. Therefore, some fellowship programs may be heavily weighted toward correctional psychiatry, and others may be more weighted toward civil forensic topics. However, core didactic curriculum topics are required of all fellowship programs.6 The topics contained in the program requirements are summarized in Table 2.
Results
The results of the review of article topics in The Journal by volume in comparison to the ABPN MOC Examination's content outline are summarized by year in Table 3. The articles span the full breadth of the topics required by the ABPN for the MOC Examination. The overall percentage weightings of articles in each topic area do not correlate specifically with the ABPN weightings. Criminal Forensics, Corrections, Special Issues, and Legal Issues/Basic Law (if one includes the Legal Digest articles) all are weighted more heavily than in the ABPN topic areas, with Criminal Forensics and Special Issues weighted most heavily, accounting for just over 50 percent of the topic content for The Journal over the past five years. Legal Regulation of Psychiatry, Civil Concerns, Death Penalty, and Children/Families were topics with lower coverage relative to the percentage weighting used for the MOC Examination by the ABPN. Of those, Children/Families and the Death Penalty were covered least frequently. Of note, there was a trend toward an increasing percentage of articles in the ABPN area of the legal regulation of psychiatry, and the 2012 edition of The Journal saw an increase in the number of articles related to children/families than in previous years. Other topic areas showed variable frequencies, with some years having several articles and other years when there may have been no articles on a given topic.
Not including the Legal Digest case reviews, Book and Media Reviews, and Editorials, The Journal published between 38 and 47 articles each year from 2008 through 2012. In a given year during the same period, on average, 40.6 substantive articles were published. This volume of articles represents an opportunity for individual forensic psychiatrists and other readers of The Journal to read at least three to four topical articles per month to help maintain knowledge, thereby filling specific gap areas identified by members of AAPL.
The review of article topics by volume of The Journal in comparison to the ACGME program requirements is found in Table 3. Thirty articles could not be classified in any category, a reflection of how narrow the ACGME didactic topic list is in comparison to the ABPN MOC Examination Content Outline (18 articles were unclassifiable in relation to the Outline's topics). For example, the ACGME list contains no requirement for didactic training in malingering or in neuroimaging and other special procedures. Essentially, many of the articles assigned to the ABPN special issues category, could not be assigned in the ACGME topic list. There are areas on the ACGME list in which no articles have been published in The Journal during the years studied: Eyewitness Testimony, Use of a Law Library, Civil Procedure, Jurisdiction, Tort Law, Children's Rights, Family Law, Juvenile Correctional Systems, Conservators and Guardians, Termination of Parental Rights, Child Abuse, Testamentary Capacity, and Competence to be Executed. There are abundant articles on Corrections, Sex Offenders, Risk Assessment, Ethics, the Legal Regulation of Psychiatric Practice, and Competence to Stand Trial.
Discussion
To the authors' knowledge, this review represents the first systematic effort to compare an academic journal's content to national standardized indices of content areas that physicians in a particular subspecialty must master to maintain board certification status, enhance their continuing education, and maintain proficiency of their skills. Limitations of this review include the potential bias of the primary and secondary reviewers that may be reflected in sorting the articles into categories and the possibility that some articles may qualify for more than one category. There was no similar methodology in existing studies on which to rely, but this attempt may be a first step for other professional journals to consider in thinking about choice of content for publication.
Despite some limitations of the categorical sorting, the findings of the systematic review revealed many strengths of The Journal, including the number of substantive articles published each year and the breadth of topics covered, particularly in criminal forensic psychiatry and in special issues within forensic psychiatry.
The findings also pointed out that the weightings of topics identified by the ABPN for the MOC Examination did not fully match the weighting of topics of The Journal's articles from 2008 through 2012, and several of the ACGME didactic content areas were not covered. It should be noted, however, that our findings were limited by the number of years we selected to review. Expanding the number of years under review may have led to different results and could have shown articles covering topics not included in those five years.
The Journal, as a product of AAPL, is intended to help fulfill the organization's main mission to educate its members as well as the broader readership. In meeting the educational needs of the members overall, as articulated by the gaps noted for ACCME accreditation purposes, it is clear that the topic areas meet the main learning goals of the members. Specifically, the content areas covered by The Journal can help fill the gap in basic knowledge of the fundamentals of forensic psychiatry as well as knowledge of how to conduct research.
Although there are national guidelines of content areas that physicians should master and maintain through lifelong learning, the fact that the frequencies of content areas of The Journal do not specifically align with the weighted content areas of the ABPN or of the full range of topics taught in ACGME-approved fellowships does not and should not detract from the value of The Journal as an educational tool for the members and readership. Some journals are specifically designed to help the readership gain information covered on board examinations (e.g., the Focus journals, published by American Psychiatric Publishing).
The editors of The Journal did not set as an overarching goal the publication of content in the areas of a board certification examination. There may be several reasons that certain content areas are covered less frequently. For example, the relative lack of articles in The Journal related to child forensic psychiatry probably reflects that these articles are more likely to be published in journals in the specific area of child psychiatry and that they cover areas that are emerging and for which new knowledge and literature will evolve over time. Finally, there was an absence of articles on topics such as basic law, civil procedure, and the like. Basic knowledge areas have been emphasized as important in residency training and may be more suited for textbooks than for a higher level journal geared for a readership consisting of professionals who are already practicing in their subspecialty field.7,8
One could argue that a specialty publication such as The Journal, which is intended for a sophisticated professional audience, should publish articles covering more nuanced areas within forensic psychiatry, such as those that fell in the special topic areas and those that were difficult to classify altogether. Such articles, in fact, may be one of the best strengths of The Journal overall, since it clearly meets this goal. That said, it is unclear which activity should lead in setting the direction for active learners and consumers of these products. Should a journal that has a peer review process and publishes the latest in a field of scientific and clinical depth, such as forensic psychiatry, drive the proportionality of the content area for the national guidelines? Or, should the national educational content area guidelines, which are informed by a peer consensus-driven process among practicing senior forensic psychiatrists who serve on a committee of the ABPN or ACGME, drive the content areas of The Journal? At a minimum, at least one process involving knowledge content selection should inform the other. By evaluating and comparing journal content to national standards, our findings represent a step in this direction.
It should also be noted that apart from publishing The Journal, the main live educational activities of the AAPL organization are its annual meeting and forensic review course and the new self-assessment CME examination. The live activities offer a place to gather with colleagues but also to earn continuing medical education credits by attendance at workshops and other training sessions. These activities must comport with the overall requirements of the ACCME, but may not specifically follow the content areas of the ABPN examination, although the Forensic Review Course is often attended by those whose focus is on taking the ABPN examination. Both the AAPL Annual Meeting and The Journal depend on the individuals who submit the articles and presentation proposals that drive the overall content areas selected. Of course, policies could be established that support a more consistent weighting of themes that mirror the ABPN requirements or other educational requirements. If such a policy were to be adopted, reviewers could keep it in mind in evaluating various submissions, giving preference to the most needed content areas. As a step in that direction, the AAPL Education Committee recently decided to adopt the ABPN content areas as those to be submitted at the AAPL Annual Meeting. If the goal is to match the national educational content area standards, such a practice might also allow for a planned solicitation of articles or presentations that would bolster areas that are not currently as actively covered for the members who would seek to use these activities to enhance their potential to succeed on the MOC Examination. Although this approach may seem positive, it too would have a downside in limiting flexibility and creativity for articles and submissions as the organization works to fit its efforts into a more structured approach. In addition, negative aspects of soliciting articles and having peer reviewers keep a content policy in mind may lead to biases that would favor publication by certain institutions or on certain topics, crowding out creative submissions and thus decreasing opportunities for innovation.
Assisting members with Maintenance of Certification (MOC) and skill acquisition has been a major focus of AAPL as an organization.9 The MOC activities required for ongoing board certification include positive professional standing, as demonstrated by unrestricted medical licensure, completion of Performance in Practice (PIP) modules, Self-Assessment CME and additional CME, and the successful completion of a cognitive examination every 10 years. To date, AAPL has engaged in a progressive and systematic plan to assist its members in achieving MOC. The associated activities have included an effort to educate members about these requirements through a webinar, multiple presentations on MOC requirements (presented in collaboration with ABPN staff) given at the annual meetings, and most recently, an informational booth at the AAPL meeting. In addition, AAPL has developed several MOC-related products for its members. These include a Self-Assessment CME examination, which was compiled with the help of a task force of the Education Committee and many members' contributions to a question bank. In addition, the MOC Task Force recently developed forms to assist with the clinical and feedback modules of the Performance in Practice requirements.10
In the discussions of how to help members achieve MOC, The Journal continues to be a rich resource that provides the references and content on which the MOC products are developed. For example, it has published four practice guidelines to date, covering insanity defense evaluations,11 videotaping of forensic interviews,12 competence to stand trial evaluations,13 and disability assessments.14 Another guideline on forensic assessment is in progress. These practice guidelines have informed both the Self-Assessment CME examination and the Performance in Practice modules. In this way, beyond the content areas identified, The Journal has had a clear influence on educational activities that are viewed as key to achieving lifelong certification in forensic psychiatry.
Forensic publishing is a growing area, with several journals now available for authors and for readers who are interested in the intersection between law and psychiatry or psychology. Each journal is designed and managed for different readerships and purposes. The nature of a journal is such that it offers on-demand learning for the busy professional, and it is important that journals meet the needs of their readership while ensuring high-quality content that has undergone rigorous peer review. As physicians are increasingly held accountable for maintaining and demonstrating a level of proficiency through lifelong learning, certification, and recertification, it is important for them to have access to identified resources that will help them maintain the level of knowledge and quality needed throughout a career. The Journal and educational offerings at AAPL meetings represent just this sort of valuable resource. From the esoteric to the more routine, the articles presented in The Journal cover a breadth and depth critical to the advancement of the field. That said, an opportunity for self examination of The Journal itself, such as the one conducted here, can provide a forum for further conversation and shaping of the landscape of knowledge promoted through this important publication.
Footnotes
Disclosures of financial or other potential conflicts of interest: None.
- © 2014 American Academy of Psychiatry and the Law