Abstract
Although individuals who use illicit drugs are a potentially vulnerable population, there have been no objective evaluations of the effectiveness of standard informed consent procedures in assuring that prospective participants entering drug abuse treatment trials fully understand the nature of the research and treatments in which they have agreed to participate. Young, marijuana-dependent adults referred by the criminal justice system who were enrolling in a randomized treatment trial were asked to complete a multiple-choice quiz concerning basic elements of the trial before providing written informed consent. Participants were assigned to standard drug counseling or motivational interviewing/skills-building therapy, delivered alone or with incentives for attending sessions and submitting marijuana-free urine specimens. Only 55 percent of the 130 participants correctly answered all four questions, and 20 percent incorrectly answered a question concerning their right to refuse to participate. An unexpected finding was that quiz scores were modestly associated with marijuana use outcome measures. These preliminary findings highlight the importance of systematically evaluating the understanding of research participants, particularly those in vulnerable populations, of their rights and key aspects of the trials in which they agree to participate.
Participants’ provision of informed consent is a cornerstone of clinical research. Particularly crucial to the informed-consent process is the extent to which the participant understands the potential risks and benefits associated with the study, as well as their rights as research participants.1 Prospective participants often receive a large and even confusing amount of information regarding the study and participation, including verbal explanations from staff, study brochures, and consent forms of various length and clarity, and the extent to which prospective participants are fully informed of these procedures is infrequently addressed in many trials.1–3
Concerns regarding informed consent are particularly salient in studies involving vulnerable populations such as children or individuals whose cognitive functions are compromised. One particularly vulnerable population includes those individuals who may feel pressured to participate, including prisoners and individuals who are involved with the court system. Among users of illicit drugs, treatment is often prompted or even mandated by the criminal justice system, and a high proportion of substance users are involved with both the substance abuse treatment and criminal justice systems.4 These pressures, especially those exerted on persons whose involvement in treatment is prompted by the criminal justice system, may interfere with the individual's understanding of the voluntary nature of research participation and their ability to withdraw from trials.3, 5–7
In several studies the informed-consent process has been evaluated in vulnerable populations or participants’ understanding of the process has been systematically assessed. Wirshing and colleagues8 reported that among 49 schizophrenics given a quiz concerning details of the study, the initial median score was 80 percent correct. However, only 10 percent received a perfect score on the first try, 53 percent answered all questions correctly after taking the test a second time after the relevant sections of the consent form were reviewed again, and the remaining 37 percent required three or more attempts at the test to answer all questions correctly. Agre and Rapkin9 compared the effectiveness of three different types of media (video, computer, and pamphlet) in providing information to 441 study participants recruited in a surgical day hospital waiting area. A 12- to 15-question quiz showed no significant differences between the three types of presentation media in terms of participants’ comprehension of the study; the participants had an average score of 69.4 percent correct.
Although users of illicit drugs are a vulnerable population and many researchers report that they routinely assess prospective participants’ understanding of clinical trials,2 there has been virtually no research to date on the effectiveness of the informed-consent process with populations of substance users who are referred to treatment by the criminal justice system. Although many investigators have begun to assess systematically the participants’ understanding of research or clinical protocols by using strategies such as objective tests (e.g., Refs. 6, 10) or directly asking participants about key aspects of research trials, no study has, to our knowledge, reported on the utility of those tests in terms of participants’ understanding of the trial provided by standard informed-consent procedures. Of particular interest in this population is the crucial matter of individuals’ perceptions of their right not to participate in or to withdraw from the trial, particularly among those who have been referred by the criminal justice system, who represent an increasingly large proportion of the substance abuse treatment population.
In considering the problem of participants’ perception of their rights in volunteering for a study, as a preliminary step, we used data from a randomized clinical trial in which behavioral therapies were evaluated for young marijuana-dependent adults11 and in which a brief multiple-choice questionnaire was used to assess the participants’ understanding of a few very basic elements of the trial, including their appreciation of the voluntary nature of their participation and the right to withdraw. In addition, we explored the relationship between the participants’ understanding of the study and their baseline characteristics and treatment outcome.
Methods
This research was approved by the Yale University Human Investigations Committee. Data were drawn from an eight-week randomized controlled trial in which participants were randomized to one of four treatment conditions: manualized drug counseling alone, motivational enhancement combined with cognitive behavioral therapy (MET/CBT) alone, drug counseling plus voucher-based contingency management with reinforcement for attending sessions on time and submitting marijuana-negative urine samples, and MET/CBT with vouchers.11 Participants were individuals aged 18 to 25 who were referred for treatment for marijuana dependence by the Office of Adult Probation to the Substance Abuse Treatment Unit in New Haven, CT, and who met criteria for current marijuana dependence. Participants were referred, but not mandated, to treatment by their probation officers. That is, individuals were asked to participate in treatment as a component of probation but would not necessarily receive consequences such as violation of probation if they did not complete treatment. Of 208 individuals screened, 174 met eligibility criteria and provided written informed consent; of those, 38 withdrew before completing the assessment process.
During the intake/screening session, prospective participants received a detailed explanation of the study, its risks and benefits, and their rights and responsibilities from a member of the research staff and were asked to provide written informed consent. Informed consent procedures consisted of a review of the study by a research assistant (who had completed the extensive training in human subjects protection offered through the Yale University Human Investigations Committee) including detailed review of the consent form by the research assistant while the individual followed along on a copy of the form. The protections and potential limitations associated with the Federal Certificate of Confidentiality obtained for the trial were also described. Participants were given ample opportunity to ask questions. The research staff emphasized that participation in the study would not be viewed as either positive or negative by the probation officers and that the risks and benefits of participation were commensurate with standard treatment at the clinic. Next, because of our concerns that the individuals might not fully comprehend or attend to the informed consent procedures, their knowledge of some of the most basic but critical aspects of the trial was assessed via a four-question, multiple-choice quiz.
The quiz was used as a tool to facilitate the participants’ understanding of the trial. Thus, after the quiz was completed, the research staff reviewed the participant's responses and clarified any question the participant had not answered correctly, reviewing the relevant sections of the consent form until it was clear that the participant understood the concerns raised. The quiz was intentionally brief to reduce the participants’ burden and to highlight some of the points that participants most frequently misunderstand or forget based on our experience in the pilot phase of the trial.12 The first question asked the participants to identify the number of treatment sessions offered. The second assessed whether they understood that participation was voluntary and that they could withdraw from the study at any time. Question three asked them to identify the types of treatment offered in the study. The last question asked them to identify the names of the investigators who could be contacted if they had questions or concerns about the study.
Substance use was monitored throughout the study by weekly urinalysis (Roche TestCup 5 with additional tests for adulterants) in addition to self-report of substance use (including marijuana, cocaine, alcohol, methamphetamines, opioids, benzodiazepines, and other illicit drugs) by recording usage on a Substance Abuse Calendar that used the Timeline Followback method, which has been demonstrated to be reliable and valid for assessing day-to-day substance use in several trials.13–17 Current and lifetime diagnoses of substance use and psychiatric disorders were assessed with the Structured Clinical Interview for DSM-IV, Patient Edition (SCID)18; substance use, and related problems were assessed via the Addiction Severity Index.19
Regarding data analyses, simple correlations were determined to evaluate the extent to which participants’ understanding of the study scores was related to baseline demographic and drug use characteristics of the sample, including age, educational level, severity of substance use and legal problems, and the primary outcome measures (retention in treatment, percentage marijuana-positive urine samples submitted, self-reported days of marijuana use during treatment). The second quiz question was of particular interest for this study, as it concerned the participant's understanding of the voluntary nature of the study and the ability to withdraw at any point. Thus, relationships between this single item and the primary outcomes were evaluated by using point bi-serial correlations.
Results
Participant Characteristics
The mean age of the sample of 136 individuals who provided informed consent and were randomized to treatment was 21 (SD 2.1). Ninety percent of the participants were male. Sixty percent identified themselves as African American, 13 percent as Latin American, and 23 percent as European American. Nearly half (48%) had not completed high school, 35 percent were high school graduates, and 18 percent had completed some college-level work, but none was a college graduate. Most (96%) had never been married, relatively few held a full-time job (21%), and about half (51%) were unemployed. The sample reported an average of five previous arrests (with a mean age of 16 at the first arrest). Participants also reported that they had been incarcerated an average total of nine months during their lifetimes.
Regarding substance use and comorbid psychopathology, the participants reported that they had first used alcohol at age 15 and marijuana at age 14. They reported using marijuana a mean of 13.0 days (SD 10.3) and using alcohol a mean of 2.8 days (SD 4.3) of the 28 days before baseline assessment. Five percent met criteria for a current DSM-IV alcohol use disorder (24.4% lifetime), 11 percent met criteria for a lifetime diagnosis of a depressive disorder, 22 percent met criteria for a lifetime anxiety disorder, and 43 percent met criteria for antisocial personality disorder.
Quiz Responses
Of the 136 individuals who provided informed consent and were randomized to treatment, 130 completed the informed consent quiz. Of these, 72 (55.4%) answered all four questions correctly, 40 (30.8%) answered three, 15 (11.5%) answered two, and the remaining 3 (2.3%) answered only one. As shown in Table 1, almost all (97.7%) of the participants correctly identified the number of treatment sessions that the study offered. Most (81.5%) correctly understood their rights to participate voluntarily and withdraw from the study whenever they wanted. The most frequently incorrect response was to the question regarding the types of treatment offered in the trial, with 74.6 percent answering that question correctly. The question concerning whom the individuals could ask for more information was answered correctly by 85.4 percent of the participants.
Informed-Consent Quiz: Item Response Frequencies
Performance on the informed consent quiz was significantly associated with only two of the many baseline demographic and substance use variables assessed (including quantity/frequency of marijuana use, ASI composite scores and measures of motivation). Quiz scores correlated positively with the number of years the individual had been drinking (r = 0.19, p = .03) and the diagnosis of an antisocial personality disorder (ASPD) (r = 0.19 p = .04): participants who reported regular alcohol use and who met criteria for ASPD received higher scores on the quiz (these variables also correlated significantly (r = 0.660, p = .003)). Education was not highly related to quiz score (r = −.093, p = .290). The question regarding the voluntary nature of treatment and withdrawal did not correlate significantly with any baseline demographic or substance use variable assessed, including severity of legal problems, motivation for treatment as assessed by the URICA (University of Rhode Island Change Assessment),20 or frequency of marijuana use.
As shown in Table 2, quiz scores did not correlate significantly with retention in treatment or percentage of drug-free urine specimens submitted; however, quiz scores correlated significantly with the longest continuous period of marijuana use during treatment (r = −0.224, p = .013) and total days of marijuana use during treatment (r = −0.188, p = .037).
Relationships Between Informed-Consent Quiz Scores and Treatment Outcome Indicators
The single item assessing the participants’ understanding of their right to withdraw from the study was also significantly associated with several of the outcome measures, including the percentage of urine specimens positive for marijuana (r = −0.191, p = .03), the longest continuous period of abstinence from marijuana (r = −0.254, p = .004), days of marijuana use (r = −0.239, p = .007), and percentage of days abstinent from all illicit drugs during treatment (r = 0.197, p = .03). These relationships held even after controlling for years of alcohol use (the only baseline variable significantly related to quiz score) and treatment assignment.
Discussion
In this preliminary report on the utility of a multiple-choice quiz for assessing prospective participants’ understanding of the nature of research in which they have agreed to participate by standard informed consent procedures, we found the following. First, only about half of the participants were able to answer correctly all four multiple-choice questions about the study immediately after a detailed review of the protocol and consent form. Moreover, certain key aspects were not well understood by a proportion of the participants. Almost one-fifth did not understand their right to withdraw at any time during the study, highlighting the possibility of perceived coercion despite multiple assurances of confidentiality and the voluntary nature of treatment. Roughly a quarter could not correctly identify the nature of the treatment conditions to which they could be assigned. These findings thus highlight the potential utility of systematic evaluation of participants’ understanding of study procedures and their rights, particularly with vulnerable populations and comparatively poorly educated populations such as the young probation-referred, marijuana-dependent sample described herein. An unexpected and intriguing finding was that quiz scores, particularly the question related to the voluntary nature of the research, were moderately associated with substance use outcomes.
The finding that education level was not related to quiz score adds to the mixed findings regarding the relationship between education and understanding of informed consent.9, 21, 22 However, it should be noted that this could be related to the restricted variability in educational level in this sample. The unexpected finding of statistically significant correlations between both quiz scores and the participants’ understanding of the right to withdraw with marijuana use outcomes raises the possibility that individuals’ understanding of the studies in which they agree to participate may have implications for their response to treatment. A possible explanation for this unanticipated relationship is that court-referred individuals who were going through the motions because of perceived pressure may have been less likely to invest energy in understanding and participating in the study. Similarly, is it possible that the participants’ lack of understanding of the study's risks, benefits, and obligations would undercut the efficacy of treatment or their willingness to become fully invested in treatment and hence to benefit from it.
The primary limitation of this study is the correlational nature of the findings. That is, it is possible that other factors may account for the apparent relationship between quiz score and outcome as well as the brevity of the quiz itself, which was composed of only four items. Nevertheless, there has been virtually no systematic examination of substance users’ ability to understand informed consent, and this represents, to our knowledge, the first report on the relationship between informed consent and treatment outcome. Although the quiz on informed consent consisted of only four questions, it covered the most basic aspects of the study and the consent process: the duration and content of the treatments provided, the right to deny consent or to withdraw from participation, and the person whom they could contact with concerns or questions about the study. Moreover, the very short time between review of the consent form and taking the quiz suggests that the quiz was a reasonable assessment of the effectiveness of the initial informed consent process. Similarly, while the specialized nature of the sample (young, unemployed, drug users) limits the generalizability of the findings, referrals from the criminal justice population represent most drug abuse treatment referrals, making this sample's understanding of the protocol through the consent process of interest. While preliminary, these findings underline the importance of exercising great care in informed consent procedures and the potential utility and importance of including formal assessment of understanding such as this, particularly in vulnerable populations when perceived coercion is a possibility.
- American Academy of Psychiatry and the Law
