Daubert Considerations in Forensic Evaluations by Telepsychiatry

Testable Hypothesis

In a Daubert analysis, the inquiry as to whether a hypothesis is testable relates to the scientific concept of falsifiability.^21,22,33 As Shapiro and colleagues explain, this prong of a Daubert analysis poorly fits the work that forensic mental health professionals perform: “what is the testable hypothesis … in a child custody evaluation?—that the tests used by the evaluator can really measure ‘parenting capacity’?” (Ref. 33, p 150). Supposing a court does attempt to evaluate testimony based on telepsychiatric evaluation under the falsifiability prong, the testable hypothesis may concern the diagnostic reliability of psychiatric evaluations performed via VCT (i.e., whether telepsychiatric assessments achieve results equivalent to those obtained through traditional, face-to-face evaluations, and, if so, whether forensic evaluations achieve the same results). Unfortunately, there are few scientific studies specifically investigating the reliability and validity of forensic telepsychiatry. Furthermore, the studies that do address reliability in teleforensic assessments are focused on specific questions (such as competency to stand trial), have small sample sizes, and may be relevant to only individual factors within a specific subset of forensic cases. Although forensic psychiatric evaluations (conducted for the purpose of informing a legal question) and clinical psychiatric assessments (conducted for the purpose of diagnosis and treatment planning) are different, empirical evidence supporting clinical telepsychiatry may be relevant to the court’s inquiry in a hypothetical Daubert challenge, given the dearth of empirical research on forensic telepsychiatry.

Clinical telepsychiatry is one of the oldest and best studied forms of telemedicine.^34,35 Numerous research studies, including several meta-analyses, have shown clinical telepsychiatry to be a valid, reliable, and well-accepted method for psychiatric diagnosis and treatment, with results comparable with those of in-person practice.^34,36,–,41 The research base on the feasibility and efficacy of telepsychiatry or telemental health is supported by high-quality randomized controlled trials.³⁴ Compared with other forms of telemedicine, telepsychiatry enjoys a stronger empirical research base.⁴²

While the research supporting the use of telepsychiatry for forensic evaluations and hearings is not quite as extensive as that supporting its use for clinical applications,^4,42,43 there are some encouraging findings to note. In a randomized controlled study, Manguno-Mire and colleagues found high levels of agreement between live, in-person forensic evaluations and remote, telepsychiatry-based assessments for competency to stand trial.⁴⁴ Using the Georgia Court Competency test⁴⁵ and two Board-certified forensic psychiatrists, evaluations were conducted on 21 forensic inpatients in a maximum-security forensic hospital. The researchers found that while providers were generally more satisfied with live interviews, evaluees had no preference between the two types of assessment, and ultimate scores and findings were generally consistent between face-to-face and telepsychiatry-based evaluations. Thus, telepsychiatric assessments of competency to stand trial do appear to achieve results equivalent to traditional, in-person evaluations applying the same procedures.

In another study, Lexcen and colleagues tested the Brief Psychiatric Rating Scale—Anchored Version⁴⁶ and the MacArthur Competence Assessment Tool—Criminal Adjudication⁴⁷ in forensic evaluations of 72 forensic inpatients.⁴⁸ They found “good to excellent” interrater reliability between three interview conditions: in one group, an in-person evaluator administered the scales, which were then observed and scored remotely; in a second group, scales were administered by a remote evaluator, with in-person observation; in the third group, the interviewer and observer were both present in person with the evaluee. They concluded that “providers can expect remote interviews to provide clinical information similar to that obtained by in-person interviews” (Ref. 48, p 715). Here, again, telepsychiatric assessments appear to achieve results equivalent to in-person evaluations within a forensic context.

Bayne and colleagues found that “telephonic psychiatric evaluations produce comparable results to in-person evaluations” in forensic assessments for asylum seekers (Ref. 49, p 1). Their study found a “small, but not statistically significant” (Ref. 49, p 2) difference in the quality of affidavits based on the telephonic versus in-person evaluations. This difference appears to be related primarily to the absence of visual data for appearance and motor-activity assessments in the mental status examination, as the telephonic evaluations were not conducted via VCT. The fact that these assessments with auditory data alone produced results comparable with in-person assessments lends further support to the validity of VCT-based assessments, which would provide even more data (i.e., visual information) in the psychiatric interview.

How a judge might apply a Daubert-based analysis of falsifiability to testimony based on forensic assessment conducted through telepsychiatry will likely depend upon numerous factors, including the level of the court (e.g., federal versus state, trial versus appellate), the judge’s training and experience, and specific variables in the case at hand. In a national survey regarding Daubert and its applicability to judicial gatekeeping responsibilities for expert witness testimony, Gatowski and colleagues found that few judges understood the scientific meaning of falsifiability despite stating that it is “a useful guideline for determining the merits of proffered scientific evidence” (Ref. 22, p 444). One testable hypothesis for forensic telepsychiatry is that VCT-based assessments achieve results roughly equivalent to those of traditional, face-to-face modalities, assuming the same tests and procedures are applied in both conditions. For evaluating competency to stand trial, VCT does appear to achieve results comparable with in-person assessments, and preliminary data suggest the likelihood of similar findings for evaluating asylum eligibility.

Peer Review and Publication

The Supreme Court in Daubert noted that “Publication (which is but one element of peer review) is not a sine qua non of admissibility; it does not necessarily correlate with reliability” (Ref. 18, p 593). The Court noted the importance of publication as a forum for peer review, however: “submission to the scrutiny of the scientific community is a component of ‘good science,’ in part because it increases the likelihood that substantive flaws in methodology will be detected” (Ref. 18, p 593). Unfortunately, forensic telepsychiatry remains an under-researched area, with few peer-reviewed studies and publications compared with those supporting clinical telepsychiatry.⁵⁰ Courts may therefore inquire as to whether clinical telepsychiatry has been subjected to peer review and publication.

One of the reasons scientific studies are published in academic journals is so that their results can be validated (replicated) or, alternatively, falsified, by other researchers. The publication of numerous studies showing the reliability and efficacy of clinical telepsychiatry for both diagnosis and treatment in well-regarded, peer-reviewed medical journals lends strong support to the statement that the videoconferencing-based psychiatric evaluation for clinical purposes has been subjected to extensive peer review.^34,36,–,41 In fact, early studies did bring substantive flaws to light, such as the inability to detect movement, subtle facial expressions, and other appearance-related concerns when using poor connection speeds (e.g., 128 kbps or less).⁴¹ Anecdotal data suggest that the use of high-bandwidth services produces superior results,⁵¹ although even relatively low-bandwidth telepsychiatry was found nearly equivalent to face-to-face, in-person psychiatric assessments in one meta-analysis.³⁸ Recent years have shown a steady refinement and improvement in the techniques of both clinical and forensic telepsychiatry. Telepsychiatry has been subject to expert consensus as clinical practice and has been subject to peer review in clinical publications. In a typical forensic evaluation, the essential element of the face-to-face psychiatric assessment is the examiner’s observation of the evaluee, posing questions to the evaluee, and observing responses. These are common aspects of psychiatric assessment and examination regardless of whether the purpose is clinical or forensic.

Known or Potential Rate of Error

In evaluating scientific testimony, the Court in Daubert directed judges to consider “the known or potential rate of error” as well as “the existence and maintenance of standards controlling the technique’s operation” (Ref. 18, p 594). With respect to forensic psychiatry in general, the potential rate of error may relate to the likelihood of agreement between forensic experts. Unfortunately, there are few empirical studies investigating the reliability and validity of forensic psychiatric assessments in general and even fewer studies demonstrating the reliability and validity of VCT-based forensic evaluations.¹² As with falsifiability, the scientific concept of “error rate” is poorly suited to analysis of forensic mental health evaluations,³³ and the majority of judges in a national survey showed an understanding of the concept that was “questionable at best” (Ref. 22, p 447). Furthermore, “[t]he Court [in Daubert] did not give any indication as to what rate of error would be unacceptably high, although courts almost uniformly have found that error rates of 50 percent or more reflect a lack of reliability” (Ref. 52, p 30). Numerous variables may affect the potential for error in forensic telepsychiatry, most of which are present regardless of whether the clinical interview is conducted in person or through VCT.

One potential concern is the absence of some sensory data that may be available in an in-person psychiatric examination. These data may include smell (e.g., an odor of alcohol on the evaluee’s breath or foul body odor which can signify poor hygiene), subtle visual cues (e.g., slight tremor, bruises or swelling on body parts not shown on camera, jaundice that is not apparent due to color distortion or correction in the video transmission), and auditory information (e.g., if the audio quality is poor, the psychiatrist may not be able to hear a shaking voice or labored breathing).^13,14,53 Additionally, technology-specific variables may interact with the evaluee’s symptoms (e.g., disorganized thought processes or delusions about technology).¹³ In a study of VCT for competency evaluations, several examiners reported communication difficulties with defendants caused by technical problems such as poor bandwidth and audio delay.⁵³ Adjorlolo and Chan have suggested that administering a test via VCT that was normed in an in-person setting may compromise the test’s reliability and validity,⁵⁰ but there are data supporting the reliability and validity of remote administration of numerous standardized tests.⁵⁴

Authentication of the patient or evaluee can be a concern in telepsychiatry and may require some careful attention on the part of the evaluating psychiatrist. For example, it may be difficult to know or ensure that the person viewed via VCT or webcam is, in fact, the intended evaluee and not, for example, his brother. Furthermore, the psychiatrist may not know whether the evaluee is responding independently to the examiner’s questions, being coached by an unseen third party, or conducting internet-based research simultaneously.⁴ The examiner can take steps to address these concerns, such as requesting a photograph of the evaluee beforehand or taking a screenshot or video recording during the interview. Anticipating potential risks and taking corrective steps may further decrease the potential rate of error, thereby increasing the likelihood of one’s testimony withstanding a challenge on these grounds.

The existence and adoption of Clinical Practice Guidelines (CPGs) to regulate and guide critical aspects of telepsychiatry demonstrate that there are standards controlling its application. In addition to the CPGs that guide traditional forensic psychiatric evaluations, the specialist who conducts an examination via VCT should follow the recommendations of organizations like the American Telemedicine Association (see Table 1). When an evaluation follows these practice guidelines and is based upon techniques shown to be valid and reliable, a court would have difficulty excluding such testimony on the basis of error rate or lack of standardization merely because the examination had been conducted remotely rather than in person. Peer-reviewed literature demonstrates that the potential rate of error associated with telepsychiatry is similar to that of traditional, face-to-face psychiatric practice.

General Acceptance

The Daubert Court noted that: “[w]idespread acceptance [of a theory or technique] can be an important factor in ruling particular evidence admissible” (Ref. 18, p 594). Some states still apply only the general acceptance standard for the admissibility of expert witness testimony.²⁰ Furthermore, general acceptance, as one of the four criteria, remains relevant in jurisdictions that apply the Daubert standard; there is some indication that judges in state-level courts have continued to focus on general acceptance even after the adoption of rules based on the Federal Rules of Evidence²² and that the importance of general acceptance may have increased after the Daubert decision.¹⁷ Endorsement of a method by a professional scientific organization is an important indicator of the technique’s acceptance by the medical profession. The APA has publicly acknowledged the value of forensic telepsychiatry since the late 1990s:Telepsychiatry is appropriate for a variety of forensic uses, including patient assessment for involuntary commitment … and for conducting commitment hearings. Indeed, in the latter case it may enable family members to give testimony and emotional support who might be unable to attend otherwise (Ref. 55, p 3).

Despite its early adoption by some, telepsychiatry’s growth was initially slow, largely due to cost.³⁵ Computer and telecommunications systems sophisticated enough to allow high-speed and high-quality information exchange were quite expensive in the early days of the internet. Several federal legislation packages, such as the American Recovery & Reinvestment Act,⁵⁶ allocated funds to support improvements in broadband internet access and telemedicine. As the cost of high technology decreased, telepsychiatry’s use grew correspondingly.⁵⁷ From 2010 to 2017, the use of VCT by mental health facilities in the United States nearly doubled.⁵⁸ In recent years, telepsychiatry has gained more widespread acceptance for clinical and forensic purposes.^7,50,59

General acceptance may also be indicated by the adoption and dissemination of CPGs, as noted above. There are today several sets of CPGs regarding the use of telemedicine or telepsychiatry, some of which are listed in Table 1, above. Another indication that telemedicine and videoconferencing have achieved general acceptance in psychiatry is the American Psychiatric Association’s Position Statement on Telemedicine in Psychiatry, which states that “[t]elemedicine in psychiatry, using video conferencing, is a validated and effective practice of medicine” (Ref. 73, p 1).

Increasing reimbursement of telemedicine and telemental health by CMS and private insurers is another important proxy for its general acceptance. Notably, CMS authorized reimbursement for clinical telepsychiatry⁷⁴ and actively encouraged its increased use to address the opioid crisis through Medicaid programs⁷⁵ well before the COVID-19 outbreak. Since March 2020, a broad swath of requirements and limitations on reimbursement have been lifted to encourage more providers to offer remote services. Furthermore, several bills pending in Congress would further lift restrictions on telepsychiatry or make temporary COVID-19 allowances permanent.⁷⁶ These trends illustrate that the use of clinical telepsychiatry has indeed achieved general acceptance within the scientific community. It bears noting, however, that these developments, such as CMS reimbursement of telepsychiatry, may not apply to forensic psychiatric evaluations.

In studies of judicial gatekeeping post Daubert, researchers have found that judges tend to focus on two prongs of the Daubert criteria in particular: peer review and publication, and general acceptance.^17,22,30 Falsifiability (testable hypothesis) and known or potential rate of error are rarely mentioned.³³ Of the four criteria, judges appear to be focusing most on general acceptance,^17,30 an inquiry which has an established history in case law through the Frye standard. Focusing on general acceptance rather than falsifiability or error rate, particularly for disciplines like forensic psychiatry, is an application of Daubert that is consistent with the Supreme Court’s reasoning.³³ Evidence supporting the notion that VCT has achieved general acceptance among forensic psychiatrists may therefore lessen the likelihood of a successful Daubert challenge based solely on the psychiatrist’s use of VCT as a modality to conduct the forensic interview.

Of particular importance to this article was a finding by Shapiro and colleagues “that if an expert did not conduct an in-person interview of the defendant or plaintiff, the testimony is more likely to be excluded due to reliability” (Ref. 33, p 152). In interpreting their results, one should note that their review spanned cases dating back to the late 1990s, a time when the necessary technology for high-quality telepsychiatry was prohibitively expensive for many, if not most, mental health professionals. Technological advancements since that time have led to more widespread adoption of telepsychiatry and its growing acceptance as a valid modality for diagnosis and treatment in both clinical and forensic contexts, as evidenced by the growing body of literature in peer-reviewed journals as well as the proliferation of CPGs and similar publications regarding its use. With increasing adoption of telepsychiatry in the context of the COVID-19 pandemic, judges may be more inclined to recognize VCT-based psychiatric evaluations as relevant and consistent with generally accepted practice among forensic professionals.

Furthermore, even more important than general acceptance is the question of whether the proffered testimony is likely to assist the trier of fact.^21,33,77 In a detailed review of published civil and criminal cases involving challenges to the admissibility of expert witness testimony, the most common determinative factors controlling admission or exclusion of testimony were whether it would assist the trier of fact, reliability, relevance, and the expert’s qualifications.³³ The Federal Rules of Evidence were cited more often in these determinations than were the Daubert criteria.³³ In an analysis that discusses the relevance of several Rules of Evidence as well as Daubert, Faust and colleagues noted: “if there is a mantra that judges repeat when deciding admissibility of expert testimony, it is that the evidence must ‘fit’ and be ‘relevant’ to the issues in the case, ‘reliable’ as outlined in Daubert/Kumho Tire and Rule 702, and ‘helpful’ to the jury” (Ref. 21, p 56).