Abstract
Substance-related disorders (SRD) are common psychiatric morbidities among adolescents within youth correctional systems. Identification and treatment of SRDs is critical for successful reformation and reintegration. Lack of simple, structured, valid, brief screening instruments that can be easily administered and scored by lay workers militates against screening for SRDs. We present the results of the reliability and concurrent validity of the CRAFFT (acronym for Car, Relax, Alone, Forget, Friends, and Trouble) substance abuse screening instrument among residents of youth correctional facilities in Lagos, Nigeria. Adolescents who screened positive on CRAFFT were further assessed with the Kiddie Schedule for Affective Disorders and Schizophrenia (K-SADS) to determine whether they met diagnostic criteria for SRDs. The mean CRAFFT scores for all the adolescents (n = 178) was 0.66 (SD ± 1.45). A total of 23 (12.9%) had CRAFFT scores of >1.00. The CRAFFT instrument has good internal consistency (Cronbach's α = 0.85) and 2-week test reliability (Spearman correlation = 0.979; p < .001). At a cutoff point of >1.00, CRAFFT had the best sensitivity and specificity (area under the curve = 0.889; 95% confidence interval 0.765–1.000) among the participants. As validated, the CRAFFT is a reliable instrument for screening for SRDs in incarcerated youth.
Nigeria has some of the poorest human development indices in the world.1 In addition, social indicators for children are very poor.2 Children who grow up in the midst of adversity are at increased risk of coming in contact with the juvenile justice system.3 It is therefore not surprising that the number of children and adolescents in youth correctional institutions in Nigeria has been on the rise.4 Research from different parts of the world,5,–,7 including Nigeria,8,9 have documented that alcohol- and substance-related disorders are common mental health problems of young inmates in correctional facilities. Alcohol- and substance-related disorders among adolescents in general have been linked with morbidity and mortality. For instance, adolescents and young persons accounted for up to 320,000 of the 2.5 million deaths per year worldwide attributed to harmful use of alcohol, and about 9 percent of deaths in this age group.10 In addition, the abuse of alcohol and other psychoactive substances is associated with significant health and social problems among adolescents, with attendant negative socioeconomic impact on the family and communities at large. In the youth correctional setting, when left unaddressed, comorbid alcohol and substance abuse is known to reduce the chance of successful reformation and rehabilitation.11
According to reports from other parts of the world, such as a national survey from the United States,12 the number of adolescents within youth correctional services who have access to treatment for alcohol- and substance-related disorders is very small. The situation is likely to be worse in poorly resourced regions such as sub-Saharan Africa, where child and adolescent mental health services are generally scarce and low in priority, even for youth outside of the juvenile justice system.13,14 The chances that alcohol- and substance-related disorders among adolescents within juvenile justice institutions will be addressed is a function of the degree of incorporation of alcohol and other substance screening into intake assessments.15,16
Unfortunately, going by recent observations, there is yet to be any form of substance abuse screening and treatment in youth correctional institutions in Nigeria.8 A recent study identified the absence of a simple, structured, valid, and brief instrument that can be easily administered and scored by lay workers as a key factor militating against substance abuse screening and intervention within these settings.17 There are a few instruments that meet such criteria and have been widely used in other countries. These include quantitative substance abuse screening instruments with mnemonics or acronyms such as AUDIT (Alcohol Use Disorder Identification Test),18 POSIT (Problem Oriented Screening Instrument for Teenagers),19 and CRAFFT (Car, Relax, Alone, Forget, Friends, and Trouble).20 Each of these instruments has its strength and weaknesses when compared with the others. For instance, aside from the fact that the AUDIT test was primarily designed for use among adults, it screens for only alcohol use disorders but not for other substances of abuse. In contrast, POSIT and CRAFFT were designed to be developmentally appropriate for teenagers, and they screen for both alcohol and other substance-related disorders. However, although POSIT is also a developmentally valid assessment tool for substance-related disorders among adolescents,20 it takes a much longer time to administer than CRAFFT (17 items versus 6 items). Moreover, CRAFFT has distinct advantages other than its brevity (taking only one to two minutes to administer). It has an easily remembered mnemonic21 and can be self-administered and easily computerized, making it perhaps more adolescent friendly.22
Complementary to these unique strengths, a systematic review and meta-analysis of validation studies conducted in different countries and settings between 1999 and 2010 further showed that the CRAFFT instrument has adequate psychometric properties for detecting alcohol- and other substance-related disorders among adolescents.23 For instance, meta-analysis showed that the CRAFFT instrument has sensitivity, specificity, and internal consistency which ranged from 0.61 to 1.00, 0.33 to 0.97, and 0.65 to 0.86, respectively. The instrument has also been found to have a specificity and sensitivity that is comparable with both POSIT and AUDIT among adolescents.21 Unfortunately, although the psychometric properties of CRAFFT have been tested in several countries,23 its cross-cultural validity (in terms of cross-national measurement invariance) has not been explored. It has been noted that determination of the psychometric properties of CRAFFT across different settings and contexts, especially in the under-represented regions of Africa and East Asia, is the next step in the globalization of the valid use of CRAFFT.
Aside from the regional imbalance in the evaluation of the psychometric properties of CRAFFT, it has hardly ever been tested among high-risk and underserved populations, such as youth within juvenile justice systems anywhere in the world. This is a critical omission, as validation of an instrument such as CRAFFT with all its inherent advantages (brevity, simplicity, and validity) has a potential to revolutionize alcohol- and substance-abuse screening and treatment, especially in a resource-constrained region such as sub-Saharan Africa. This study therefore aimed to determine the psychometric properties of CRAFFT and appropriate cutoffs among residents of youth correctional facilities in Lagos, Nigeria.
Materials and Methods
Institutional Review Board approval was obtained at Lagos State University Teaching Hospital (LASUTH), Ikeja, Lagos.
Settings
This work is part of a larger project seeking to provide rationale and framework for incorporating mental health screening and interventions into juvenile justice services in Lagos.24 The study was conducted in all five youth correctional facilities being operated within the city. Each of the five institutions represents an initial separation of the adolescents according to differences in age, gender, and judicial category, such that each institution houses adolescents of the same gender, similar age, and similar stage of judicial processing. The adolescents who are resident in these institutions often fall under three administrative–legal categories: juvenile offenders, adolescents adjudged to be beyond parental control, and adolescents in need of care and protection.
Sampling
To determine the number of cases of substance-related disorder we had to have for the intended analyses, we undertook a priori estimation of sample size. Given the small population size of adolescents in the juvenile justice system in Lagos, we used a sampling estimation guided by the literature.25 This was carried out with the use of medcalculator (downloaded from https://www.medcalc.org/manual/sampling_ROC1.php). The Type I and Type II errors were 0.10 and 0.20, respectively, and the area under the curve was 0.80. The ratio of the sample sizes in the negative and positive groups was 2, and the null hypothesis of 0.5 was used. The result of the calculation estimated the minimum required positive cases to be 8, whereas that of negative cases was 16, yielding a minimum sample size of about 24. To increase the power of the study, we projected a sample size that was at least five times the estimated sample size. To achieve this within the limited resources, two-thirds of all the adolescents in each of the five youth correctional institutions were recruited by using random sampling techniques (simple balloting with two-thirds of the ballots being marked “yes”). In the end, of a total of 185 eligible adolescents, 7 were excluded from the study because of inability to comprehend the interview for reasons including severe mental disorder and apparent intellectual disability, among other reasons. All 178 adolescents included in the study agreed to participate, and were all interviewed.
Measures
Basic sociodemographic data on age, gender, and reason for admission in the correctional facility were obtained.
CRAFFT
The CRAFFT questionnaire is a behavioral health screening tool for assessing levels of problematic alcohol and drug use among adolescents in the past 12 months.20 It consists of a series of six questions developed to screen adolescents for high-risk alcohol and other drug use disorders simultaneously (see Table 1). It is capable of providing information about both pattern and extent of use. The response to each of the six questions that assesses the extent of use can either be yes (1 point) or no (0 points). A minimum score of zero and a maximum score of six are thereby generated for each respondent who reported a 12-month history of use of alcohol or any other substance. In the original validation study for CRAFFT,20 a score of two and above was suggestive of a problematic pattern of use (abuse or dependence). Further details about CRAFFT are accessible online.26 In the present study, we used a version of CRAFFT that has been culturally adapted to a Nigerian setting and used among adolescents in the country.27 Initial cultural adaptation involved translation to and back-translation from a Nigerian language, following the guidelines for the translation and cultural adaptation of patient-reported outcome measures.
The Six Crafft Questions20
Kiddies' Schedule for Affective Disorders and Schizophrenia
The K-SADS is a semistructured diagnostic interview designed to assess psychopathology in children and adolescents in accordance with the criteria in the Diagnostic and Statistical Manual, Fourth Edition (DSM-IV).28 It can be used by trained professionals to assess the presence of DSM-IV psychiatric disorders, including substance-related disorders. The alcohol and substance related disorder schedule of the K-SADS was administered to adolescents in face-to-face interviews.
Procedure
The participants first completed the CRAFFT (period 1). They were then divided into two groups based on their scores on CRAFFT. The first group comprised persons with CRAFFT scores of two or more, and the second group consisted of those with scores less than two. Two trained psychiatrists, blind to the CRAFFT scores, used K-SADS to assess all the participants in the first group and a randomly selected 10 percent of those in the second group for the DSM-IV criteria for substance use disorder. The two trained psychiatrists conducted the clinical interviews separately and their inter-rater reliability, which was measured with Cohen's κ, was 0.89. Two weeks later (period 2), all the respondents completed the CRAFFT. The ethics of the procedure were in accordance with the recommendations of the Borstal Institution and Remand Centre Act,29 a major law that guides research and services in youth correctional facilities in Nigeria. The Ethics and Research Committee of LASUTH approved the study protocol. Approvals were granted by the Ministry of Youth and Social Development. The principals and personnel in various correctional facilities consented to assist with recruitment of juveniles in their schools. In view of the difficulty in tracing parents of the participants in this study, the authorities of the Lagos State Ministry of Youth Development, as well as the principals of the correctional facilities acted in loco parentis and gave consent, whereas written assents were obtained from the participants. The sample frame included all the adolescents in the five correctional facilities in Lagos. The potential respondents were all assured of their liberty to accept and decline participation without any penalty. As part of the project, substance abuse counseling was provided for adolescents identified as having substance-related disorders.
Statistical Analyses
The Statistical Package for the Social Sciences (SPSS, ver. 20) was used for statistical analysis. Participants were classified as cases or noncases of alcohol/substance use based on their K-SADS diagnosis. Results were calculated as frequencies (%), means, and standard deviations. The reliability of diagnoses was evaluated using κ. To calculate the differences between the groups, independent-samples t test and chi-square were used. All tests were two-tailed, and the level of significance was set at p < .05. Screening parameters including sensitivity, specificity, predictive values, and likelihood ratios were calculated for CRAFFT scores. The internal consistency of CRAFFT was measured by Cronbach's α. The Spearman correlation was used to establish the test–retest reliability of CRAFFT. The psychometric performance of CRAFFT was compared against the K-SADS diagnosis using the receiver operating characteristic (ROC) curve. The area under the curve (AUC) was also calculated.
Results
A total of 178 adolescents (age range, 13–17 years) completed the study. The mean age of the sample was 15.19 ± 1.98 years. There were more boys (n = 110; 61.8%) than girls (n = 68; 38.2%), with the boys being significantly older than the girls (15.43 ± 1.94 versus 14.79 ± 1.98; t = −2.2; p = .035). These participants had lived in the correctional institutions for a period that ranged from 1 week to 120 months, with a median length of stay of 12 months. Regarding the categories of offense or reasons for admission in the youth correctional facilities, 34 (19.1%) (male, 88.2%; female, 11.4%) were young offenders; 13 (7.3%) (male: (61.5%), female: (38.5%)) were adolescents who had been declared beyond parental control, and 131 (73.6%) (male: (55.0%), female: (45.0%)) were status offenders (mostly runaways, underage children found hawking goods or services on the streets, or those who were lost or wandering in the city).
Psychometric Performance of CRAFFT
The mean CRAFFT scores for all the adolescents (n = 178) was 0.66 (SD ±1.45). A total of 23 (12.9%) had CRAFFT scores >1.00. In addition to the randomly selected 10 percent of those who had CRAFFT scores ≤1 (n = 16), 39 participants had the clinical (K-SADS) interview. There was no significant difference in age and CRAFFT scores between the randomly selected 10 percent and the other 90 percent in the second group. The mean CRAFFT score for the interviewed group was 2.51 (SD ±2.09). For the participants with alcohol- and substance-related disorders (diagnosed by K-SADS) in the interviewed group, the mean CRAFFT score was 3.70 (SD ±1.46) and, for those without alcohol- and substance-related disorders, the mean score was 0.81 (SD ± 1.64). The difference was statistically significant (t = −5.77; df = 37; p < .001; CI = −3.90 to −1.87). The CRAFFT score correlated strongly with the K-SADS diagnostic classification (Spearman ρ = .687; p < .001). The leading K-SADS symptom criteria that were most prominently reported and used in reaching a positive diagnosis of a substance-use disorder include frequent reinstatement after voluntary cessation of use, tolerance, using more than planned, continuing to use despite evidence of negative physical consequences, and experience of withdrawal symptoms, among others.
The internal consistency of questions within the CRAFFT estimated by Cronbach's α in this study was 0.85. The sensitivity (proportion of adolescents with alcohol- and substance-related disorders correctly identified by CRAFFT), specificity (the proportion of adolescents without alcohol- and substance-related disorder correctly identified by CRAFFT), positive predictive value (PPV; the proportion of adolescents screened positive by CRAFFT who actually had substance-related disorders), and negative predictive value (NPV; the proportion of adolescents screened negative by CRAFFT who actually did not have a substance-related disorder) at various cutoff scores are shown in Table 2 and summarized in an ROC curve (Fig. 1). To determine the two-week test–retest reliability of CRAFFT, the adolescents who had clinical interview again completed the CRAFFT two weeks after the first test. The mean CRAFFT score at period 2 was 0.69 (SD ±1.47). There was good correlation between the period 1 and 2 scores (ρ = .979, p < .001).
Psychometric Properties of Crafft in Screening for Substance Use Among Adolescents in Youth Correctional Institution
The receiver operating characteristic curve for CRAFFT.
Discussion
This study, to our knowledge, is the first to evaluate the psychometric properties of CRAFFT and determine the appropriate threshold to screen for substance-related disorders among residents of youth correctional facilities in sub-Saharan Africa. We found CRAFFT to be an effective screening instrument for substance use among adolescents in the study setting. The CRAFFT had good internal consistency and two-week test–retest reliability. The ROC is the preferred method to examine the cutoff values for an instrument and it displays the relationship between sensitivity (true positive rate) and 1 − sensitivity (false-negative rate) in a sample.25 The validity of CRAFFT was supported by a good AUC. Evidence abounds that AUC values that are greater than 0.5 indicate better-than-chance classification, whereas an AUC of 0.8 or greater suggests that the scale is useful.30 We also found CRAFFT to be reliable in this study. For a self-report scale to be reliable, a Cronbach's α of at least 0.6 is recommended,31 and the CRAFFT in our study achieved 0.85 which shows that among residents of youth correctional institutions in Lagos, it is reliable. This finding is similar to previous findings among the general population of adolescents.32,–,35
The reliability of CRAFFT in our study is also supported by the diagnostic likelihood ratio (DLR) analysis. The DLR is a function of both the sensitivity and the specificity of a test, and it indexes how much the test result will change the odds of having a disorder. Its use is therefore recommended by many statisticians in evidence-based medicine.36,37 One main advantage of DLR is that it is independent of the prevalence rate. Although the calculated DLR will be valid in another sample with a different prevalence rate, this consistency is not true of sensitivity, specificity, PPV, or NPV.
In this study, we found that a total CRAFFT score of more than one is optimal for detecting juvenile offenders with substance-related disorders and that this is sufficiently discriminating in this sample. With the positive likelihood ratio of 4.856, there is a small to moderate increase in the likelihood of substance-related disorders when the CRAFTT score is more than one. Given a positive test result on CRAFFT, the odds of having a substance-related disorder (with a score of more than one) in a resident of a youth correctional facility is 1.4. Correspondingly, the negative likelihood ratio of 0.107 shows there is a moderate decrease in the likelihood of having a substance-related disorder when there is a score of one or less. Given this negative test result, an adolescent resident in the study settings has a one in one odds of being free of substance-related disorders. We therefore suggest that, a health worker can be reasonably reassured when the CRAFFT score is one or lower, but a resident of a youth correctional facility should be further assessed when the score is above one.
Although conducted within a special setting, the optimal cutoff point of >1 in the present study aligns with previous validation studies of CRAFFT among adolescent populations. For instance, Knight and colleagues35 suggested a cutoff of two, whereas others recommended a cutoff of two or higher.32,34 We did not find any similar studies in a juvenile justice population anywhere, with which we could have compared our findings.
The results of this study have a few policy implications. First, it will eliminate one of the major barriers to substance abuse screening and treatment in youth correctional settings: lack of a valid screening tool. Second, early identification of substance use disorders will facilitate treatment planning. In this study, we used CRAFFT, which has been adapted to be culturally and developmentally appropriate for this and other regions.27 Screening with CRAFFT that has been culturally adapted and validated has been found to be more accurate in comparison with subjective or even clinical judgments in identifying the needs and the level of risk of an adolescent.38 This study therefore adds to the scant (if any) literature providing evidence-based knowledge with regard to CRAFFT as an effective screening instrument for substance abuse in a youth correctional setting. It has also shown that a brief but reliable substance-abuse screening tool such as the CRAFFT can be easily validated for use among juvenile justice populations. It can actually help bridge the global dearth of substance use screening and intervention services among this vulnerable but underserved population.12
Limitations
The results and recommendations of this study should be interpreted in light of its limitations. First, although statistically sufficient for the analyses, the total sample size of adolescents with substance-related disorders in the present study was small. This size limitation is evident in the fairly wide confidence intervals of the parameter estimates derived from the ROC analysis. Furthermore, the small sample size precluded any form of subanalyses, which may have yielded different cutoff points across demographic differences. We are aware that, given the heterogeneity of juvenile justice populations, a single instrument is not likely to meet the high reliability and validity standards across all categories of youth (i.e., ages, genders, ethnicities, offense histories, and cognitive and developmental capacities). Another limitation is that the K-SADS used in the study was based on DSM-IV definitions of substance related disorders and there were major changes in The Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5),39 based on concern about the validity of an abuse-dependence distinction. Future studies may want to re-evaluate within the DSM-5 framework. Finally, we had to assume the fidelity of the self-report, as there was no biochemical confirmation of substance use. Therefore, even when the present study established good psychometric properties of CRAFFT and recommends the same for use as a screening tool, we are not able to provide evidence of the fidelity of the self-report. The literature has established concerns about the fidelity of self-reports of alcohol and substance use among adolescents who are not presenting for treatment.40 Some studies have, in fact, found fairly significant inconsistencies in adolescent self-reports of substance use and actual biochemical analyses.41 The degree of inconsistency has even been reported to be higher among adolescents who are within the custody of juvenile justice institutions.42 Future validation of CRAFFT in this population will do well to include biochemical validation of the self-report so as to be able to make more reliable statements on its fidelity.
Conclusion
We have shown that the CRAFFT is a valid instrument for screening for substance-related disorders among adolescents in youth correctional institutions in Lagos. It provides an important opportunity for incorporating a culturally relevant and valid substance related disorder screening into the service package for adolescents in the Nigerian youth correctional systems in line with international best practice.43,44 It will also enhance the onset of needed collaborations between the public mental health sector and juvenile justice systems in Nigeria.
Acknowledgments
This research was funded by the Humphrey Fellowship Alumni Impact Award under the Humphrey Fellowship Program, with support from the Mental Health Desk of the Ministry of Health, Lagos State, Nigeria.
Footnotes
Disclosures of financial or other potential conflicts of interest: None.
- © 2017 American Academy of Psychiatry and the Law
