Abstract
Bias can vitiate the quality and credibility of a mental health professional’s forensic evaluations as well as scientific and scholarly contributions to the forensic process in forensic psychiatry publications. Our attention here is on this latter influence of bias, although the genres of bias identified here can as well occur in forensic practice and writings. Attention is given to multiple forms of bias in peer review: ad hominem, ideological, confirmatory, hindsight, the halo effect, gender, publication, conflict of (financial) interest, political, religious, nationality or country of origin, esthetic or linguistic, racial or ethnicity, and herding. No doubt much bias in peer review goes undetected and no absolute purification process exists. Nonetheless, as with almost any problem, the first step toward a remedy is recognition.
- peer review
- bias in peer review
- forensic psychiatry publication
- confirmatory bias
- herding bias
- publication bias
A serious concern for both the quality and fairness of the peer review process is the possibility of bias, which in turn can diminish inter-reviewer reliability.1 Guarding against bias, i.e., “systematic deviation from the truth” (Ref. 2, p 92), serves to ensure that research reports, meta-analyses, and other scientific reports are valued.2 Dozens of different types of biases in research potentially exist (Ref. 3, citing Sackett 1979, Ref. 4), and it is the task of peer review to ferret them out. The existence and extent of every genre of bias identified here is not firmly established, yet concern for these potential biases in the interest of fairness and quality is reason enough to address them. As noted by Kassirer and Campion in 1994, and seemingly equally true today, “There is no consensus on how to evaluate or assess the relative importance of these many kinds of bias” (Ref. 3, p 96). Yet the effort must continue. Here, we examine bias in the peer review process, commenting on several of the growing number of types of bias recognized in peer review if not also in editorial decisions.
Our search of the literature identified no prior professional articles on bias in peer review for forensic psychiatry publications. Space did not allow inclusion of this important aspect of the peer review process in our earlier article on the subject.5 Therefore, this present review is a follow-up to our earlier article.
Tennant and Ross-Hellauer6 addressed deficiencies in understanding peer review processes by identifying core themes to be examined, among which is the subjectivity and bias of peer reviewers. It cannot be overemphasized that bias is universal, ever present, and characteristic of all of us. Because of the absence of studies and commentaries on biases in peer review for forensic psychiatry publications, “trans-application” of reports and reviews on peer review bias in other medical and scientific fields is necessary. This is done with awareness that the exercise of biases may well be of limited comparability across disciplines. In reviewing the biases presented below, we bear in mind that the identification and analysis of biases can also be biased. Analysts who examine and assess for bias, including ourselves, can be influenced by their own biases, and the analytic process may be biased.
The following are perceived potential and actual genres of biases in peer review for scientific and medical publications that warrant consideration as well for peer review in forensic psychiatry and psychology publications. In this study, the authors conducted an updated and comprehensive literature search in September 2023 to identify relevant articles and research papers. The search strategy included various combinations of MeSH terms, namely, citation bias, religious bias, publication bias, peer review process, forensic psychiatry, forensic psychology, psychiatry, and psychology. The purpose of this expanded search was to ensure the inclusion of the most recent and pertinent studies, thereby enhancing the comprehensiveness and validity of the research findings.
Genres of Bias
Ad Hominem Bias
Tvina et al.7 identified three biases in peer review: ad hominem bias, affiliation bias, and ideological bias, the three genres of bias “most pertinent to issues surrounding peer review” (Ref. 7, p 1081), described and discussed in great detail earlier by Shatz.8 Ad hominem bias is where the reviewer has some personal feelings or preconception about the author(s) of a submission that color the reviewer’s assessment of the quality of the paper. Ad hominem bias can be positive, such as from a supportive friendship, or negative, as can arise through professional competition or envy. Where the reviewer is moved by competitive feeling with the author, ad hominem bias may be more likely among subject matter experts (see Ref. 8, pp 37-39).
Some journals invite or encourage submitting authors to recommend a few individuals to review their manuscript. This can be useful where the topic is within a narrow niche, for which there are few subject matter experts known to the editor. Research from the Australian Research Counsil found this procedure to be the “only major systemic bias” in peer review in comparison with the more common sources of academic rank, age, experience, gender, nationality, research team composition, and university. According to Marsh et al., author-selected reviewers resulted in “inflated, unreliable, and invalid ratings” (Ref. 9, p 160).
Affiliation Bias
If aware of the author’s institutional affiliation, the reviewer may be influenced by the prestige or lack thereof of the institution (e.g., Ross et al.10). Tvina et al.7 indicate that editors may be favorably predisposed by a prestigious institutional affiliation, believing that this will add to the status of the journal. An experimental study examining reviewer assessment of scientific submissions to a conference on web search and data mining showed that single-blinded reviews preferentially rated submissions from top universities and companies, whereas double-blinded reviews were less likely to favor submission from top universities and companies and famous authors.11 Further empirical support for affiliation bias was presented in the study by von Wedel et al.,12 although the accuracy of these findings was questioned in an exchange of published letters.13,14 Marsh et al.9 raise the contrary possibility that any institutional association with successful applications represents a validity source premised on stronger researchers being from the more prestigious institutions rather than reflecting bias.9 For selection of research and development projects in Korea, Jang et al.15 found evidence for selection bias toward research proposals by alumni from the same university as the evaluator. Peters and Ceci16 provide potential explanations for affiliation bias, including the possibility that certain research communities “supply the largest proportion of the peer reviewers for its journals” (Ref. 16, p 192). In his amplified and research-supported discussion of affiliation bias, Shatz raises the possibility of a “bend over backwards bias,” a counterbias to affiliation, as well as gender and other genres of recognized biases that can undermine the objectivity and fairness of peer review (Ref. 8, pp 39-42).
Ideological Bias
The third bias of concern to Tvina et al.7 and Shatz8 is whether the reviewer has strong personal views about research that can be positive, such as where authors agree with assumptions and findings that are consistent with those of the reviewer’s own research, or negative, such as when the reviewer plans to publish on the same topic. Especially relevant to publications in forensic psychiatry, this definition could be expanded to include the political biases, discussed separately below. Tvina et al. have expanded the definition to embrace content-related bias, which includes “ego bias” influenced by citation of the reviewer’s work and “cognitive cronyism,” where reviewers adhere to the theoretical framework of the author; conservatism with excessive skepticism of innovation; conflict of interest; and bias against interdisciplinary research, which should be minimal for law and psychiatry journals that are interdisciplinary by nature. Yet, on topics such as the treatment of paraphilias and delusional disorders, the psychological and psychiatric literature is siloed. Naturally, separate disciplines will examine the same disorders from their own disciplinary framework and cite research from their own field. Such compartmentalization of viewpoints and sources can lead to contrary conclusions with advocates of pharmacotherapy of paraphilias dismissing psychotherapeutic approaches and authors promoting psychotherapy claiming pharmacotherapy to be ineffective. A clear example of this is the contrary therapeutic recommendations of the American Psychiatric Association and the American Psychological Association in their respective amici curiae to the U.S. Supreme Court in Sell v. U.S.17 as to whether antipsychotic medication ought to be prescribed to a treatment-refusing defendant with delusional disorder for trial competency restoration. Disagreement is perhaps expected in advocacy writing, such as amicus curiae, but such ideological bias based upon discipline bias has been observed in the confidential process of peer review and selection of scientific manuscripts for publication. As Shatz pointed out, ideological bias can overlap with confirmatory bias. It can also take the form of a bias for or against the contemporary prevailing and most widely accepted paradigm in the field.8
Confirmatory Bias
Mahoney defined confirmatory bias in research publication as “the tendency to emphasize and believe experiences which support one’s views and to ignore or discredit those which do not” (Ref. 18, p 161). This has also been designated more aptly as “disconfirmation bias,” because it is the neglect or denial of information that would contradict one’s assumptions. This should not be confused with the tested confirmation and disconfirmation procedures integral to the scientific method, clinical assessment, and judicial fact-finding.
“Self-citation” bias2 can be considered a closely related subtype of confirmatory bias. This should probably be qualified as “excessive” or “inappropriate” self-citation, as omitting the most pertinent literature, including one’s own contribution, could also represent scholastic negligence or the “bending over backwards” bias of Shatz. Especially relevant to the peer review process is when the authors cite the reviewers’ work, making reviewers favorably disposed toward the submission. Although the field of study was quite apart from forensic psychiatry and psychology, a study of the review process of two conferences concerning machine learning and algorithmic economics found that citation of a reviewer appears to favor the likelihood of acceptance.19 One cannot dismiss the possibility, however, that the author of a thorough literature review would have cited the work of a subject matter expert who, because of expertise in the area, was selected to conduct the peer review.
A psychological explanation for confirmatory bias, supported by research, is Heider’s assimilation contrast theory20: information concordant with one’s assumption is accepted, whereas discordant information is not.21 His research supported the existence of confirmatory bias in the assessment of purported research reports. The importance of striving to limit confirmatory bias is important in forensic practice22 and should apply as well to peer review of manuscripts in general, not only research reports.
Hindsight Bias
A number of authors have described a genre of bias wherein the author claims only after the results of research have been obtained to have had a hypothesis that explains the results.23 Kerr24 proposed the acronym HARKing (hypothesizing after the results are known) to capture the essence of this process. Research results can suggest hypotheses to be tested with future research, but this needs to be made clear and disambiguated from false post hoc hindsight. Hindsight bias, like other author-generated biases, may influence peer reviewers as well.
The Halo Effect
Thorndike in 1920 described the psychological observation that most people are reluctant to disagree with someone whom they consider to be important.25,26 He observed that those who are already famous tend to receive disproportionate credit for their accomplishments, a reward system that he termed the “Matthew effect.” This pattern of giving disproportionately more recognition to those already well recognized is named after the biblical passage in the book of Matthew: “For unto everyone that hath shall be given, and he shall have abundance: but from him that hath not shall be taken away even that which he hath” (King James version, Mt 25:29). More than simply a halo effect, the Matthew effect may have some merit in quality selection, such as when a customer selects products of a trusted brand name. Nonetheless, it is unfair to the lesser known scientists whose work is just as contributory, and it detracts from the more logical and empirical methods of quality assessment. Peters and Ceci16 provided empirical support for the Matthew effect influencing publication decisions by editors and reviewers.
Although not tested through empirical peer review research, the Brehms reactance theory27 posits a phenomenon opposite that of the Thorndike halo effect. Some people, perhaps to impress others, tend to take the opposite view of a well-known authority.27 Brehm’s reactance theory has also been posited as a tendency that can bias an individual in peer reviewing a manuscript.28 We might add that, with the anonymity of blinded peer review, any personal pride that could come from impressing others is diminished. This anonymity, however, may enable expression of an “antiauthority” bias by providing greater freedom to decide with minimal adverse consequences.
Gender Bias
Out of concern for the possibility of gender bias, several social science journals changed to a policy of having the reviewers blinded to authors’ identities.29,–,31 In the 1980s, women’s acceptance rates for the Publication of the Modern Language Association, except for 1986, equaled or surpassed that for men for the five years examined.31 A recent study of 145 journals, 1.7 million authors, and 740,000 reviewers found that manuscripts authored by women were reviewed more favorably by editors and reviewers.32 A study of the peer review process for the Journal of the American Medical Association found several gender differences in the characteristics of editors and reviewers.29 Nonetheless, no gender bias was found in the content reviewer recommendations or in the rates of final acceptance.29 A more recent study showed difference in review scores or acceptance rates based upon the author’s apparent gender in four studies. In contrast, in four studies in which the author’s gender was anonymized, no difference was demonstrated.33
In a recent study of assessment of abstracts describing a clinical study, raters found that research rigor on men and women was not significantly different and the scientific contribution on female subjects was deemed to be superior. Yet the same research on men was nearly two times more likely to be recommended for publication than that on women.34 This suggests bias about which gender is studied, not about authors’ gender. A study of editors and reviewers from the Frontiers journal found that women were underrepresented and that both men and women were subject to same-gender preference, a homophilic bias that would persist even if gender parity were numerically attained.35
The use of the term “gender” in studies on publication bias has not caught up with contemporary usage pertaining to gender identity, which identifies additional demographically separate groups that are subject to discriminatory practices. Gender-sensitive terminology reverts to the long tradition of using “sex” to designate birth assignments, dichotomously based on physical characteristics, as male or female, whereas gender pertains to how individuals experience themselves in terms of sexual identity, a distinction that ought to be particularly relevant, occasionally the central psycholegal concern, in forensic mental health practice. Non-cis, and especially transgender, individuals constitute not only vulnerable minorities but numerically very small minorities, more challenging to distribute proportionately, as editors, editorial board members, and reviewers, than for the underrepresentation of females, for example.
As with what we should be calling sex bias, the existence and extent of subtypes of gender bias cannot be known without deblinding and “outing” the specific genders of authors, thereby violating the very measures that should minimize this and other types of biases. Self-report surveys of both reviewers and authors could shed light on this question.
Publication Bias
Genres of peer review bias can have similar types of bias in research itself. A famous example of this, also representing confirmatory bias, is Mendel’s exclusion of data that did not support his expected findings. His error is overlooked because his theory turned out to be correct, but biased research can obviously lead to incorrect conclusions.36
The chemist Robert Boyle is said to have been the first to describe the details of his experiments on air and the necessary measures to replicate his studies.25 From his publications in 1680 through the 1800s, typical scientific reports described both positive and negative (“nil”) results.25 By the mid-1800s, scientific reports were written in a more concise style,37 and by the latter half of the 20th century, the bias toward publishing positive results became increasingly recognized.
Also called the “file drawer problem,” studies with positive results and Type I errors were presumably published to the exclusion of those with negative results and Type II errors,38,39 resulting in a skewed representation of the results of quality research. Publication bias has been demonstrated in various subfields of psychiatry and psychology and especially in drug trials for treatment of mental disorders.40 The tendency toward publication of positive over negative results begins with the investigation and ends with the editorial decision to publish but is probably also influenced by extrapublication factors, such as pharmaceutical and other research funding interests. Within this context, it should be of no surprise that publication bias, i.e., “The bias that is created when publication of study results is based on the direction or significance of the findings” (Ref. 41, p 1385), and specifically of positive findings, has been demonstrated in the peer review process as well as through experimental paradigms.16 Not all studies confirmed existence of publication bias.40 Failure to submit research reports with negative results may be a much more important, albeit latent, factor than editorial publication bias.40
Conflict of (Financial) Interest Bias
The potential that scientists can be influenced by their employer or by the financial sponsor for their research is well recognized. Authors are familiar with requirements for disclosure of conflicts of interest. Although financial support can be essential for research, it can also drive the selection of topics to be studied, the research design, and the presentation of the results.42 Funding source can influence the aforementioned publication bias and even incentivize the alteration of results, affecting the quality and integrity of the research.43 Understudied but worthy of consideration is the potential of conflict of interest, not of the investigator, but peer reviewers themselves. Employer and funding source loyalty could at least conceptually reinforce one’s feelings of solidarity or competitiveness with the authors of a submission and their source(s) of support.
Political Bias
Not mentioned in the literature on peer review but perhaps playing an even greater role in reviewing manuscripts in forensic psychiatry is the matter of political bias. As topics in forensic psychiatry manuscripts concern not only science but also public policy, the latter will most certainly be colored or shaped by the author’s political views, not necessarily to be discouraged. A recent example is provided in forensic authorship and peer review of manuscripts on international and domestic terrorism,44 wherein editors and peer reviewers can hold fundamentally different concepts of what terrorism is, depending on their political beliefs. Political bias can affect one’s preferred definition of terrorism, the study of the phenomenon, selection of what and who is to be studied, what studies are called, how they are carried out, and how the results are presented.
Perhaps any of the many public policy concerns that rightfully should, if not must, be addressed by forensic psychology and psychiatry can draw highly personal interpretation and comments that dilute otherwise well-reasoned and objective scholarship. The presentation of science can become an articulation of one’s poorly bridled political biases, even through gratuitous politically tainted comments. And the review process, involving reviewers and editors, may be similarly affected and only selectively corrective.
Religious Bias
If political beliefs can bias the peer review process, perhaps religious beliefs can as well. Both could represent subsets of ideological bias that we have reserved for disciplinary or theoretical bias within the scientific sphere. Conceivably, one’s most fundamental cultural beliefs could influence how a peer reviewer judges the merits of a submission or, regardless of one’s beliefs, even if antireligious or atheistic, how one perceives the presumed religion of the author(s), which in turn affects the assessment of the scholarly paper. Although there is a wealth of research reports on various ways in which religious and scientific methods and content interact, little is known about how religious or antireligious bias may affect peer review of scientific manuscripts, especially in the realm of forensic psychiatry, which addresses public policies that are shaped by more than scientific knowledge.
“The Einstein effect” refers to, for example, the rather universal acceptance that E = mc 2 without understanding how this formula was derived. A large international study (n = 10,195 from 24 countries) provided evidence that people would more readily believe meaningless statements from a scientist than from a spiritual guru.45 But this raises the question of the credibility of meaningful-to-them statements from clerical authorities within their own religion and denomination. Results of other studies suggest that “religiosity” could undermine science literacy.46 Yet empirical evidence as to how one’s spiritual views actually affect peer review in scientific publication is lacking. One could hypothesize that religious emphasis on fairness and selflessness serve to control one’s other potential biases or that religious scientists successfully compartmentalize their views based on faith and empiricism, respectively, such that one sphere has virtually no effect on the other in the realm of peer review. Nonetheless, it remains important to limit the possibility of bias based upon belief systems, whether political or religious, in peer review of scientific and forensic publications.
Nationality and Country-of-Origin Bias
Results of several recent studies provide evidence of country-of-origin (COO) bias in peer review. Ross and colleagues10 compared the acceptance rate of abstracts submitted for presentation in the scientific sessions of the American Heart Association meetings from 2000 to 2001 (which were open reviews) with those accepted for meetings from 2002 to 2004, for which reviews were blinded. Bias in the open review favored acceptance from the United States, other anglophonic countries, and prestigious academic institutions (see institutional bias above10).
Although not involving scientific or mental health publications, a recent study in the post-COVID-19 business field provided evidence that nationality bias can distort peer evaluations. The investigators tested COO effect in the peer evaluations conducted within 6,634 global virtual teams (GVTs), which included 33,271 individuals representing 79 countries who collaborated on business projects. Results indicated that personal bias toward COO (prestige and level of economic development) affected team member ratings more than the members’ actual contributions supported by the objective measures of skill, competence, English proficiency, technical ability, cultural intelligence, and national per-capita education expenditure.47
Acknowledging, as other studies may not, the limitations of nonblinded reviews and lack of quality assessment of submission, a study of peer review for Gastroenterology found that U.S. authored submissions were preferred over non-U.S. authored submissions, especially if the reviewers were from the United States.48 For legal and forensic journals, submissions on domestic jurisprudence, for example, may more appropriately address a predominantly national readership.
A study of journals on physical medicine and rehabilitation, with Germany, Sweden, United Kingdom, and United States representing location of publications, demonstrated an association between journal and author nationality.49 This association was strongest for the United States. It is unknown if some contributions were solicited or the submissions were uniformly unsolicited and peer reviewed. Again, association does not necessarily equal bias, but disproportionate national representation can, as the authors suggest, result in false or skewed scientific views shaped by one’s national preference.
Esthetic and Linguistic Bias
Although a number of studies point to COO bias, Lee et al.50 observe that some Americans may be more critical of fellow American authors and more accepting of non-American authors.9 Cronin51 interprets what appears to be nationality bias as being related to poor quality of prose, whereas Herrera52 found problematic peer review where the writing style was not flawed.
Poor linguistic expression in the written language of the journal can be a legitimate concern. Negative reviews and rejection based upon inept linguistic expression, usually in English, has been referred to as “lingua franca publication bias” (Ref. 53, p e41). Regardless of how well double-blinded the process may be, reviewers and editors must decide whether to downgrade a submission based upon its unacceptable presentation or encourage or assist to support advancement of an otherwise scientifically worthy submission, as recommended by Santos et al.53 Blinding would not help, and reviewers must retain responsibility for addressing presentation as well as substance in ensuring quality of accepted publications.
There are now several commercial companies that assist authors preacceptance in improving their grammar and written English expression.51 Some journals direct editors and submitting authors to such a service. Such linguistic services may not fully correct for a journal’s style or a specialized field’s unique terminology. Although we are unaware of empirical studies on the results of commercial editing and linguistic services, clarity and semantic expression is an important parameter of quality. This is a consideration for manuscript improvement, but not to eclipse the scientific substance in ultimate acceptance decisions.
Mahoney’s addition of “esthetic bias” completes Shatz’s catalogue of the most salient biases in scientific publication. The smoothness and flawlessness of the manuscript typeset (e.g., free of typographical errors) certainly must affect a reviewer’s impressions. To what extent style should weigh against substance may deserve further analysis.
Race and Ethnicity Bias
As concerns are expressed and claims are made that racial, ethnic, and national biases disadvantage members of certain groups, it is remarkable that there has been rather little research to determine if such biases affect judgments of peer reviews for professional journals. Understandably, when authors have an unfavorable peer review outcome, they may wonder about such biases. Stereotypical assumptions can be made based upon an author’s name or country of origin. On the other hand, even if outcome discrepancies are demonstrated, correlation does not necessarily establish causation. Lack of proven bias, however, need not obviate the need to examine and correct irrational and unfair outcomes.
Recent research on race in publication has focused on the underrepresentation of persons of color on editorial boards and in editorships.54 Results raise, without answering, the question of the role racial bias plays in the peer review process, a determination difficult to measure empirically without breaking the very method of neutralizing potential racial and other biases, double-blind reviews, to render race of authors unrecognizable.
Editors of most psychological, and presumably forensic, journals are white, as are most editorial board members and reviewers.54 This raises speculation that racial bias could affect peer review and submission selection for publication.55 If anonymity has been used to minimize the possibility of racial and ethnic discriminatory practices, an initiative is now underway to further investigate the race and ethnicity of scientists, with questions coming from 50 publishers representing 15,000 journals internationally.56
As we await this and other studies concerning the existence and extent of racial and ethnic biases, the possibility of such biases in peer review is a reasonable concern. Like gender bias, identity biases in general can be discriminatory against or for, and such biases could conceivably relate to the reviewer’s own racial, ethnic, or national identity and related personal experiences (e.g., homophilic bias). If so, one approach may be to strive for more racial, ethnic, and national diversity among the pool of reviewers. This may be a worthy objective for equality of opportunity, and it may possibly dilute these genres of biases or it could “redistribute” the biases, with each reviewer practicing favoritism based upon the reviewer’s own identity features. For the review of some manuscripts, it may seem more “fair” for editors to assign the submitted manuscript to reviewers known to be of the same or similar background as the authors. This practice, however, could introduce a bias of favoritism. Similar background assignments are more appropriate for areas of specialization, not demographic characteristics. In general, however, the best generic remedy for treating individual submissions fairly is to optimize double anonymity.
Herding Bias
The phenomenon of one’s behavior being influenced strongly from awareness of peers’ behavior much more than from one’s personal dispositions is known as herding.35 In scientific work and publication, herding can result in convergence on a specific methodology or paradigm that can be correct, but it can also be false. If herding affects peer review of research manuscripts, it can no doubt affect the peer review of other genres of manuscripts, such as scholarly legal analysis, also submitted to forensic journals. Peer review in scientific publication should serve to correct or mitigate the effects of herding, but the assessments and decisions of peer reviewers and editors can also be influenced by herding. The reliability of scientific publications can be reduced by the popularity of the research theme.57 Perhaps herding among peer reviewers and editors is a factor in the claim or observation that prestigious journals are more likely to publish less reliable findings that are eventually retracted.58 Speculatively, prestigious journals may somehow be more influenced by herding. Or, alternatively, they may be more attentive to the possibility and open to acknowledging it.
As reviewers strive for objectivity, their decisions become less influenced by their subjective dispositions that can potentially lead to erroneously skewed decisions. Research into this process recommends allowing peer reviewers to express subjective opinions, with moderate degree of subjectivity being optimal, to counter the potentially harmful bias of herding.38
Discussion
Because this is, to our knowledge, the first review of potential biases in forensic psychiatry and psychology publications and the existence, extent, and impact of specific genres of bias in other scientific publications is questionable, crucial empirical questions arise that can be usefully addressed. The first is whether each of the genres of biases identified here actually exists in forensic mental publications and, if so, what the prevalence and impact are of these biases. Either of two approaches could be useful in this endeavor.
The first approach would be to list all such journals in the English language (there are not many) and examine recent issues methodically with data retrieval outlines to identify the existence and extent of each genre of bias. The second approach would focus on a single journal, perhaps as a quality improvement project, and examine more deeply for evidence of bias from authors, reviewers, editors, and perhaps even production staff. Either approach resulting in open disclosure of findings could tarnish journal reputations by adverse findings or enhance the esteem of the journal by its open attempt at self-correction and improvement. Consequences to individual journals could be minimized by anonymizing the journals in the first approach and by maintaining privacy and nondisclosure of findings in the second approach, in other words handling the inquiry as an internal quality improvement audit. (But then no general good would come from the study outside of only the one journal.)
Before embarking on a study to determine which of the genres of bias discussed in this review are present and consequential in forensic psychiatry and psychology publications, some thought should be given to the purpose of this empirical search. Two overarching values in identifying the presence and extent of the various potential biases are to ensure quality and to promote fairness. These purposes can go hand in hand, where, for example, unfair practices and procedures result in diminished quality or vice versa. Depending on one’s conception of fairness and the interests involved, fairness intended to promote one group of people or ideology could diminish fairness for another group or a contrary ideology.
In our previous review,5 we identified the following domains as potentially useful areas of effort and attention if peer review is to promote quality: selection of peer reviewers, recruitment and retention of peer reviewers, and desired qualities of peer-reviewed ratings. Based upon results of that review, we offered the following measures for improving the quality of peer review: select reviewers based upon “their experience in writing and editing in mental health” (Ref. 5, p 312); “provide regular reviewers with written guidelines that explain how the manuscript rating form is to be used, pitfalls to be avoided, and important qualities of good reviews” (Ref. 5, p 312); “The journal should design the manuscript review form to include clearly written, practical measures of manuscript quality, using those already recognized in peer review literature as well as items specific to the subspecialty of forensic mental health publications” (Ref. 5, p 312); reward guest reviewers appropriately for their work; “adopt a mechanism for monitoring the performance of regular reviewers and providing them with periodic feedback” (Ref. 5, p 313); share with peer reviewers copies of reviews by other reviewers of the same manuscript (Ref. 5, p 313); and review appointments of regular reviewers periodically and stagger the turnover (Ref. 5, p 313). Although not mentioned in this review, one can easily imagine methods of reducing adverse bias by attending to the domains of the peer review process developing domain-specific measures for quality improvement.
This, however, leaves unanswered the question of which genres of bias should be ferreted out and avoided. Better than nothing, an item in the peer review form asking if the manuscript is free of bias at least directs the reviewer to this consideration. Specific example(s) of bias in the manuscript would be even more helpful. A question about whether ethics concerns were adequately addressed in the manuscript is an example of a bias for which most would presumably favor selection. This would not, however, address whether the reviewers themselves are influenced by their biases.
The important effort to promote fairness can be more complicated than pure quality control, requiring consideration of the target and purpose of fairness. In this review, we mentioned the dilemma of assessing presentation, i.e., linguistic propriety and clarity, as disfavoring authors whose first language is not that of the journal or whose legal system seems of little relevance to the majority of the readership.
More complicated than seeking, monitoring, and correcting biases that adversely affect scientific quality is the matter of perceived or actual unfairness. International representation of journal reviewers and editorial boards may reduce perceived or actual international bias, but it will diminish any effort at mirror representation of the population of the readership, whether an organization, profession, nation, geographical region, or linguistic realm. A bias toward one group even for the sole purpose of fairness means a bias at least in outcome of other groups. Strictly for quality, more statisticians may be desired; for quality and diverse perspective, epidemiologists; and for perspective alone, perhaps members of selected ethnic groups. One must bear in mind the possibility that a constellation of biases different from the status quo could conceivably replace one set of biases with another, hence the overriding importance of addressing actual biases that compromise quality. Recognizing the universality of perspective and bias, the primary effort should be to reduce and prevent clearly unfair bias (e.g., excluding involvement based upon factors that are irrelevant to purpose and quality of the publication) and bias that compromises scientific accuracy and integrity.
Conclusions
Identifying and controlling bias in peer review for forensic psychiatry publications supports fairness of process and quality of product. This effort begins with the recognition of common and potentially consequential genres of bias in scientific publications. Measures to minimize bias, only touched on here, must be analyzed for potential “tradeoffs” that may not in every respect favor fairness and quality. From this review, we also believe that empirical review of potential bias in forensic psychiatry publication would inform aspirational efforts toward controlling unfair or unscientific bias.
Footnotes
Disclosures of financial or other potential conflicts of interest: None.
- © 2025 American Academy of Psychiatry and the Law
