Abstract
The use of videoconferencing technology to conduct forensic psychiatric and forensic psychological evaluations remotely has grown considerably in the last decade. This commentary addresses a number of points made by Recupero regarding the use of remote technology to conduct forensic psychiatric evaluations. These points include the research supporting telepsychiatry and its generalizability to forensic assessment, the error rate associated with remote forensic assessment, and its general acceptance in the field. The commentary also considers the inclusion of psychological testing and specialized forensic measures in forensic assessment and describes criteria for considering tests and measures that can reasonably be included in a remotely conducted forensic assessment.
The use of videoconferencing technology to conduct forensic psychiatric and forensic psychological evaluations remotely has grown considerably in the last decade. Doubtless its frequency was accelerated during the COVID-19 pandemic; from March 2020 until at least a year later, access to secure facilities and personal contact with nonincarcerated individuals were limited substantially by public health considerations. Whether remotely conducted forensic evaluations will continue as this pandemic recedes is partly dependent on whether courts will accept and value them relative to evaluations conducted in person. This question is considered by Recupero1 in detail using the framework of Daubert v. Merrill Dow Pharmaceuticals2 to identify the concerns that courts might consider on the question of admissibility. This commentary considers some of Recupero’s points and also addresses additional aspects of this analysis that arise when psychological testing is a part of the remotely conducted evaluation.
Although Recupero largely focuses on the question of the admissibility of remotely conducted forensic evaluations, an important point is made that we have not yet seen a successful Daubert challenge to such evaluations because they are remote. Accordingly, forensic psychiatrists and forensic psychologists should be at least as attentive to the weight of the evidence generated when using remote technology as its admissibility. It is worth noting, in this vein, that there are some significant advantages to using such remote technology in forensic evaluations. It has been suggested that such using such technology can enhance evaluator impartiality,3 a particularly important consideration given the evidence that retention bias4 can influence the scoring of even a highly structured measure such as the Psychopathy Checklist-Revised.5
There are other potential advantages as well. Remote technology allows access to a pool of more specialized experts when travel time is prohibitive. Even when an evaluator is willing to travel a significant distance, this is costly; remote technology use can limit travel costs for the retaining party and travel time for the evaluator. It can also reduce the travel time for evaluees who are not incarcerated. Evaluators who now conduct collateral interviews by telephone might use videoconferencing to see and hear collateral interviewees. Finally, our society is witnessing the maturation of a generation that has grown up with the Internet and social media accessed by smartphones, tablets, and computers. Such individuals, whether they are evaluators, evaluees, or collateral interviewees, are likely to be comfortable with this technology. There is some evidence that incarcerated evaluees are more interested and attentive when questions are presented on a tablet rather than the traditional approach used in psychological testing: paper and pencil.6 In this respect, the forensic psychiatric use of remote technology reflects a larger societal trend that seems likely to expand.
Daubert Criteria and Remote Assessment
I now turn to Recupero’s analysis of the considerations for forensic assessment in the Daubert areas: testability, peer review and publication, error rate, and general acceptance. Those weighing whether to use remote technology in conducting a forensic psychiatric or forensic psychological assessment are well advised to consider this careful analysis. Indeed, they might look further to consider the four relevant sources of authority that have been identified in the forensic assessment field: law, science, ethics, and practice.7 Using Daubert to guide an analysis of the legal questions of admissibility (and evidentiary weight) seems reasonable, although evaluators in non-Daubert jurisdictions might pay most attention to the “general acceptance in the …field” criterion from Frye (Ref. 8, p 1014).
Regarding the scientific evidence that may be relevant, Recupero makes two particularly important points. First, there is limited empirical evidence on the broad question of whether remote evaluations are comparable to those conducted in person in forensic contexts. There is substantial evidence, however, that clinical telepsychiatry is a reliable and valid approach for providing clinical services. To the extent that there are common elements such as interviewing, establishing a working relationship, asking questions and observing responses, and diagnosing those that are seen in both forensic assessment and clinical psychiatry, it is fair to conclude that the evidence for the effectiveness of remotely conducted clinical services provides support for remote forensic assessment. Second, there are also a few studies9,10 that focus more directly on forensic assessment, with encouraging findings suggesting that remotely conducted evaluations of competence to stand trial are comparable with those provided in person.
The strong research base for telepsychiatry also suggests to Recupero that the “peer reviewed” Daubert criterion has been reasonably addressed. Publication in well-regarded journals that use peer review certainly demonstrates the support for using telepsychiatry for clinical purposes. It is important, however, for the evaluator facing a Daubert challenge in this area to both identify this research base and provide appropriate translation from the empirically supported tasks of clinical telepsychiatry to the similar procedures used in forensic psychiatric assessment.
The Daubert criterion of “error rate” can be applied to a specific test or technique (considering the psychometric properties of reliability and validity of a given psychological test) and there is meta-analytic evidence that neuropsychological tests (for example) can be administered remotely with comparable results.11 But there are any number of factors that can influence reliability and validity when the unit of analysis is the overall forensic assessment: the amount of information obtained, sources of the information, accuracy of these sources, and how well the evaluator succeeds in avoiding the influence of retention bias or other cognitive biases.12 Without carefully controlling these other sources of influence, it is very difficult to gauge the effect of the remote versus in-person modality. Yet, empirical research that does carefully control alternative influences creates the risk of providing evidence that is not readily generalizable to the world of actual practice.
On the question if remotely conducted forensic evaluations are generally accepted in the field, Recupero cites several relevant indicators for such acceptance: support by the American Psychiatric Association for telepsychiatry, a growing recognition of the value of remote technology for both clinical and forensic purposes, the presence of multiple sets of clinical practice guidelines, and the provision of guidance in the use of remote technology by professional organizations (e.g., the American Telemedicine Association). Taken together, these led Recupero to conclude that a Daubert challenge is very unlikely to exclude evidence simply because the evaluation was conducted remotely. I agree. Yet I see a greater risk for evaluators who are not familiar with the remote technology and the associated scientific and practice literatures: the weight assigned to the evidence they gather remotely may diminish. There are problems that arise in the course of remotely conducted forensic assessments, including a poor Internet connection, leading to difficulty in seeing and hearing the evaluee; risks to privacy stemming from the use of software that is not Health Insurance Portability and Accountability Act compliant; and difficulty in determining who else might be present in the room with the evaluee as well as noise and other distractions. Without recognizing these as potential problems and appraising their presence and effect, the evaluator could be unpleasantly surprised on cross-examination.
I turn next to a discussion of forensic assessment instruments (FAIs). These are considered in the context of remote evaluations.
Psychological Testing and FAIs
There is a very substantial literature on the use of psychological testing13 and FAIs14 in the context of forensic assessment. For present purposes, the distinction between the two refers to whether they were initially developed for the clinical purposes of diagnosis, symptom description, and treatment planning, or to provide information about functional capacities relevant in a given forensic evaluation. There are several considerations to the decision to use a given test or specialized measure in a remote forensic assessment. Some tests have used computerized administration for some time and have manuals and available scoring software that also offer assistance in interpretation. Such tests have an accompanying manual describing the supportive research and the practical details from administration to scoring, and can be readily incorporated into a remotely administered forensic assessment. By contrast, tests that require visual-motor actions by the evaluee (typical in the appraisal of intellectual functioning, for example) cannot be readily administered remotely.
Whether a specific psychological test or specialized measure should be included in an in-person forensic assessment can be summarized by applying two criteria to the measure (relevance and reliability) and one to the evaluator (competence to administer and interpret). The additional considerations regarding whether such a measure can be included in a remotely conducted forensic assessment are precedent and practicality. Precedent refers to the process for remote administration that has been developed to facilitate such usage in a way that is reasonably similar to the conditions under which the test was derived. Practicality addresses the presence of challenges to the administration or interpretation that would keep the measure from being used as intended, thus providing results that are not credible.
Certainly, there are many psychological tests that would not be appropriate for use in a forensic context, whether administered in person or remotely. A test without psychometric properties described in a manual does not have the demonstrated reliability (a legal term encompassing both reliability and validity) to add meaningfully to the information being gathered in a forensic context. A test that measures a construct or symptom constellation not relevant to the areas being investigated would not be selected for use. Accordingly, the question is not how we consider the universe of available psychological tests and its appropriateness for use in remotely conducted forensic assessment. Rather, the question is how we identify tests or measures that already satisfy the relevance and reliability criteria and apply the precedent and practicality criteria to their potential use in remote forensic assessments.
Conclusions
The use of remote technology in forensic assessment is not straightforward. There are technical and practical challenges associated with it. It would be preferable to have a stronger research base addressing how much we can rely on the translation of favorable empirical evidence in one context (e.g., in-person forensic assessment and remote clinical treatment) to remote forensic assessment. I anticipate that this research base will be strengthened considerably over the next decade. The COVID-19 pandemic, in my view, forced the field to confront questions about the effective use of existing technology in a professional activity, namely, forensic assessment, which has often moved slowly in the incorporation of such technology. As this technology improves and becomes more affordable, as citizens who grew up with it mature, and assuming our research and practice continue to support its effectiveness, it seems prescient to use reviews like Recupero’s to improve the effectiveness and efficiency of psychiatric evaluations in legal contexts.
Footnotes
Disclosures of financial or other potential conflicts of interest: None.
- © 2022 American Academy of Psychiatry and the Law