ARPA accounts for the effect of hidden interactions better than alternative methods, and is independent of the type of data and of the type of data distribution. Furthermore, results supplied by tree-based analytics are easy to interpret visually and logically. Therefore, to generate the most comprehensive and parsimonious classificatory model to predict the susceptibility to disruptive behaviors, we applied ARPA using a set of different modules implemented in the Salford Predictive Modeler® software, namely, Classification and Regression Trees , Random Forest, and TreeNet . One important advantage of SPM when compared to other available data mining software is its ability to use raw data with sparse or empty cells, a problem frequently encountered in genetic data. Briefly, CART is a non-parametric approach whereby a series of recursive subdivisions separate the data by dichotomization. The aim is to identify, at each partition step, the best predictive variable and its best corresponding splitting value while optimizing a splitting statistical criterion, so that the dataset can be successfully split into increasingly homogeneous subgroups. We used a battery of different statistical criteria as splitting rules to determine the splitting rule, maximally decreasing the relative cost of the tree while increasing the prediction accuracy of target variable categories. The best split at each dichotomous node was chosen by either a measure of between-node dissimilarity or iterative hypothesis testing of all possible splits to find the most homogeneous split . Similarly,hydroponic shelving we used a wide range of empirical probabilities to model numerous scenarios recreating the distribution of the targeted variable categories in the population.
Following this iterative process, each terminal node was assigned to a class outcome. To avoid finishing with an over-fitted CART predictive model , and to ensure that the final splits were well substantiated, we applied tree pruning. During the procedure, predictor variables that were close competitors were pruned to eliminate redundant commonalities among variables, so the most parsimonious tree would have the lowest mis-classification rate for an individual not included in the original data. Additionally, we applied the Random Forest methodology using a bagging strategy to exactly identify the most important set of variables predicting disruptive behaviors. The RF strategy differs from CART in the use of a limited number of variables to derive each node while creating hundreds to thousands of trees. This strategy has proved to be immune to the over fitting generated by CART. In RF, variables that appeared repeatedly as predictors in the trees were identified. The mis-classification rate was recorded for each approach. The TreeNet strategy was used as a complement to the CART and RF strategies because it reaches a level of accuracy that is usually not attainable by single models such as CART or by ensembles such as bagging. The TreeNet algorithm generates thousands of small decision trees built in a sequential error-correcting process converging on an accurate model. The number of variables considered to derive each node with RF was fip , where n is the number of independent variables . To derive honest assessments of the derived models and have a better view of their performance on future unseen data, we applied a cross-validation strategy where both training with all the data and then indirectly testing with all the data were performed. To do so, we randomly divided the data into separate partitions of different sizes. This strategy allowed us to review the stability of results across multiple replications.
We used a 10-fold cross-validation as implemented in the SPM software. A fixed-effects meta-analysis of the overall fraction of correctly classified individuals using the derived models from each of the four samples was applied to derive a general perspective of the SUD predictive capacity of this demographic-clinical-genetic framework.A series of predictive models were built on our data using combinations of the following criteria: the rules of splitting ; the priors; the size of the terminal nodes; the costs; the depth of branching; and the size of the folds for cross-validation, to maximize the accuracy of the derived classification tree while considering class assignment, tree pruning, testing and cross-validation. A parsimonious and informative reconstructed predictive tree derived from CART for the Paisa sample revealed demographic , clinical , and genetic variables . The importance of these variables was corroborated, and their potential over fitting discarded by the TreeNet analyses that revealed a set of predictors for SUD containing those derived by CART . This predictive model displays good sensitivity and specificity as shown by areas under the receiver-operating characteristic curve during TreeNet cross-validation using folding . The proportions of mis-classification for SUD cases in the cross-validation experiment for the learning and testing data were 0.124 and 0.177, respectively . In the case of the Spanish sample, a parsimonious and informative tree was reconstructed with CART revealing demographic , clinical , and genetic variables . This predictive model displayed good sensitivity and specificity as shown by areas under the ROC curve of 0.911 and 0.897 for learning and testing samples, respectively, during TreeNet cross-validation using folding . The proportions of misclassification for SUD cases obtained by TreeNet analysis for learning and testing data were 0.151 and 0.175, respectively . As in the previous cohorts, for the MTA sample we derived a parsimonious and informative predictive tree with CART depicting demographic , and genetic variables .
The TreeNet analyses revealed a set of predictors for SUD containing those derived by CART . This predictive model displays good sensitivity and specificity as showed by AUC of 0.808 and 0.643 for learning and testing samples, respectively, during TreeNet cross-validation using folding . The proportions of misclassification for SUD cases obtained by TreeNet analysis for learning and testing data were 0.314 and 0.358, respectively . Finally, for the Kentucky sample, we derived a parsimonious and informative predictive tree with CART involving demographic , clinical and schizophrenia diagnosis, and genetic variables . The TreeNet analyses revealed a set of predictors for SUD containing those derived by CART . This predictive model displays good sensitivity and specificity as showed by AUC of 0.811 and 0.744 for learning and testing samples, respectively, during TreeNet cross validation using folding . The proportions of misclassification for SUD cases obtained by TreeNet analysis for learning and testing data were 0.285 and 0.252, respectively . The results from the RF analysis were consistent with those produced by TreeNet cross-validation using folding. A fixed-effects meta-analysis for overall accuracy returned a value of 0.727 , suggesting potential eventual clinical utility of predictive values. Overall, ADGRL3 marker rs4860437 was the most important variant predicting susceptibility to SUD, a commonality suggesting that these networks may be accurate in predicting the development of SUD based on ADGRL3 genotypes. We conducted independent analyses for alcohol or nicotine dependence and compared these results with those of our composite SUD phenotype,commercial grow room setup as defined by the disjunctive presence of substance use phenotypes and explained by likely common neuropathophysiological mechanisms. In general, across cohorts, we found significant alcohol and nicotine risk variants, some of which have reasonably high odd ratios . For instance, in the Spain sample, marker rs2271339 conferred significant riskto nicotine use: the heterozygote genotype A/G confers 43% increased risk of being diagnosed with nicotine use . In the same vein, we found in the Paisa sample that the heterozygote A/T genotype for rs1456862 confers 83% increased risk to nicotine use than the A/A genotype. Regarding alcohol use, we found in the Paisas that the heterozygote C/T genotype for rs2159140 confers susceptibility, whereas the C/C genotype does not . Supplemental Fig. 1 shows the ROC curves of nicotine and alcohol use prediction in the Paisa sample. Note that the AUC is greater than 0.7 in both cases, which suggests a straight performance of markers rs1456862 and rs2159140 in predicting nicotine and alcohol use, respectively. To determine the significance of improvement of prediction when genetic markers are introduced in the ARPA-based predictive model for SUD, we compared the performance measures across all cohorts under two disjunctive scenarios: inclusion of genetic markers or not. We found that including genetic markers improved the performance measures of the resulting ARPA-based predictive model of SUD, regardless of cohort . For instance, the AUC for the Spain sample was 81.6% when genetic information was included, and 77.5 when it was excluded. A bootstrap-based test with 10,000 replicates revealed that the former AUC was statistically greater than the latter . Similar results were obtained for the Paisa sample: the AUC was 90% when genetic information was included versus 78.8% when it was not .
Improvements were also observed in the correct classification rate for the Spanish and Paisa samples, the sensitivity values in all samples, the specificity in the Spanish and Paisa samples, and the lift in the Paisa sample . Similar results were observed for the MTA and Kentucky samples, where including genetic information in the predictive model for SUD drastically improved these performance measures .SUD genetic epidemiological studies across multiple substances have been plagued with inconsistency in the replication of genetic association results. This may be due to reasons such as: small effect size of variants expected to influence the SUD phenotype, as with any complex disease;insufficient power to detect significant associations due to small sample size;phenotypic heterogeneity of SUD across samples that may reflect different disease stages or multiple sub-types ; genetic heterogeneity arising from distinct risk genes sets; ethnicity inconsistencies between discovery and replication samples; and comorbidity with other psychiatric conditions with shared genetic and environmental architecture. Consequently, additional studies are required to identify new SUD candidate genes and to help dissect genetic contributions in the context of complex interactions with co-morbid conditions. In this study, we present a demographic, clinical and genetic framework generated using ARPA that is able to predict the risk of developing SUD. Interestingly, marker rs4860437 showed a differential splitting pattern in the Paisa, Spain, and Kentucky cohorts. For instance, in Fig. 1a, rs4860437 splits into and T/T; in Fig. 2a, the same variable splits into and G/T; and in Fig. 4a, it splits into and G/G. The most parsimonious and plausible explanation of this splitting pattern is the presence of genomic variability surrounding this proxy marker, reflecting ancestral composition. Future studies of genomic regions surrounding rs4860437 might reveal a cryptic mechanism. It is particularly compelling that ADGRL3 marker rs4860437, which is a major predictor variable component in the trees for SUD, is in complete LD with ADHD susceptibility markers rs6551665 and rs1947274 in Caucasians, suggesting that the phenotype underpinning SUD is under the pleiotropic effect of ADGRL3 variants. Unfortunately, rs4860437 was not included in the exome chip used to genotype the MTA sample and, therefore, could not be included in the analyses for this sample. Given the limited overlap of markers across datasets and possible stratifi- cation differences among study populations, a generather than a marker-level approach has been advocated. Adopting such a perspective, our results suggest that genetic variants harbored in the ADGRL3 locus confer susceptibility to SUD in populations from disparate regions of the world. These populations are from three different countries and involve different investigators, diverse inclusion criteria, and different clinical assessments, which suggests that our results may replicate in other settings and are likely to be clinically relevant. Of particular interest is the generalization of our findings to a longitudinal study , where adding genetic information to baseline data predicted the development of SUD at later ages, as determined from information gathered over a period of more than 10 years. Additionally, our results generalized to a sample of patients with severe SUD from Kentucky that were not ascertained on the basis of ADHD diagnosis. The first genome-wide significant ADHD risk loci were published recently. Marker rs4860437 is not represented in this dataset; however, this study was not aimed at identifying loci shared between ADHD and SUD. In any case, while genome-wide association studies are a useful tool for discovering novel risk variants—as it involves a hypothesis-free interrogation of the entire genome—the lack of genetic association may be a reflection of the polygenic, multi-factorial nature of ADHD, with both common and rare variants likely contributing small effects to its etiology. In addition, an important factor may be the genetic heterogeneity of ADHD sub-types, which may have different underlying genetic mechanisms.