Evaluation of supervised machine-learning methods for predicting appearance traits from DNA

Walsh S.
et al.

IrisPlex: a sensitive DNA tool for accurate prediction of blue and brown eye colour in the absence of ancestry information.

,

18

Pospiech E.
et al.

The common occurrence of epistasis in the determination of human pigmentation and its impact on DNA-based pigmentation phenotype prediction.

,

Chaitanya L.
et al.

The HIrisPlex-S system for eye, hair and skin colour prediction from DNA: introduction and forensic developmental validation.

,

Walsh S.
et al.

The HIrisPlex system for simultaneous prediction of hair and eye colour from DNA.

]. Most widely used predictive marker sets, lab tools and statistical models include in the IrisPlex system [

13

Liu F.
et al.

Eye color and the prediction of complex phenotypes from genotypes.

,

Walsh S.
et al.

IrisPlex: a sensitive DNA tool for accurate prediction of blue and brown eye colour in the absence of ancestry information.

,

21

Walsh S.
et al.

DNA-based eye colour prediction across Europe with the IrisPlex system.

] for eye color prediction, the HIrisPlex system [

[20]

Walsh S.
et al.

The HIrisPlex system for simultaneous prediction of hair and eye colour from DNA.

] for hair (and eye) color prediction, and the HIrisPlex-S system [

Chaitanya L.
et al.

The HIrisPlex-S system for eye, hair and skin colour prediction from DNA: introduction and forensic developmental validation.

] for skin (and hair and eye) color prediction. The aforementioned statistical models are based on multinomial logistic regression (MLR) using established genetic marker panels, resulting in posterior probabilities for each trait category i.e., three eye color, four hair color, and five skin color categories [

Chaitanya L.
et al.

The HIrisPlex-S system for eye, hair and skin colour prediction from DNA: introduction and forensic developmental validation.

], and are publically available for use via https://hirisplex.erasmusmc.nl/. Almost all previously established pigmentation prediction models were based on MLR. Some exceptions include fuzzy logic, artificial neural networks and classification trees used by Liu et al. [

[13]

Liu F.
et al.

Eye color and the prediction of complex phenotypes from genotypes.

] for eye colour prediction modelling and Snipper [

[14]

Ruiz Y.
et al.

Further development of forensic eye color predictive tests.

], which is a Bayesian classifier that provides the prediction results as likelihood ratios. Further exceptions include the iterative naïve Bayesian approach from Maroñas and Söchtig [

22

Maronas O.
et al.

Development of a forensic skin colour predictive test.

Google Scholar

,

23

Söchtig J.
et al.

Exploration of SNP variants affecting hair colour prediction in Europeans.

] for skin and hair color respectively, and classification trees and partition modeling applied by Allwood et al. [

[24]

A.J S.
Harbison S.

SNP model development for the prediction of eye colour in New Zealand.

] (see [

[25]

Katsara M.A.
Nothnagel M.

True colors: a literature review on the spatial distribution of eye and hair pigmentation.

] for a further review).

Currently, machine learning (ML) has become a powerful and widely used method for solving classification and clustering problems. It is a field in data analytics that focuses on the development of mathematical models that have the ability to recognize patterns in the datasets and use this information to predict future events. In parts inspired by the human brain, these algorithms can be trained on the data (training data) [

[26]

Alpaydin E.

Introduction to Machine Learning.

Google Scholar

]. The training data is actually a set of examples which are used in order to fit, or estimate, the parameters of the model. The use of these algorithms is motivated by problems with large numbers of classes, linear and non-linear boundaries between them and can be implemented for different applications in versatile areas such as such as those observed in medicine, education, robotics and many others [

27

Kotsiantis S.B.

Use of machine learning techniques for educational proposes: a decision support system for forecasting students’ grades.

,

28

Sidey-Gibbons J.A.M.
Sidey-Gibbons C.J.

Machine learning in medicine: a practical introduction.

,

29

J. Kreuziger, Application of machine learning to robotics - an analysis. In Proceedings of the Second International Conference on Automation, Robotics, and Computer Vision (ICARCV '92), (1992).

Google Scholar

]. These boundaries refer to the decision boundaries, a hyper-surface that separates the vector space in mutually exclusive sets, one for each class. They can be either straight lines or non-linear curves. Some indicative examples of ML algorithms are linear and logistic regression [

[30]

Hosmer D.W.
Lemeshow S.

], decision trees, random forests (RF) [

[31]

Breiman L.

Random forests.

], k-nearest neighbors (k-NN) [

[32]

Mucherino A.
Papajorgji P.J.
Pardalos P.M.

k-nearest neighbor classification.

], support-vector machines (SVM) [

[33]

Vapnik V.N.

The Nature of Statistical Learning Theory.

] and artificial neural networks (ANN) [

[34]

Ripley B.D.

Neural networks and related methods for classification.

]. Despite the fact that these methods have huge potential in different fields, and an ability to handle various types of data, selecting a ML algorithm for specific data sets (problems) as well as their optimal hyperparameters to gain maximal performance can be challenging. A comparative analysis is often necessary in order to arrive at a method that provides the best prediction accuracy for the data set used.

In the context of forensic sciences, various classifiers have been used and compared for different purposes, such as the inference of biogeographic ancestry from DNA, file type detection - the identification of evidential files that criminals hide in order to mislead police authorities, glass identification etc. [

35

Goswami S.
Wegman E.J.

Comparison of different classification methods on glass identification for forensic research.

Google Scholar

,

36

Zadora G.

Glass analysis for forensic purposes—a comparison of classification methods.

Google Scholar

,

37

Cheung E.Y.Y.
Gahan M.E.
McNevin D.

Prediction of biogeographical ancestry from genotype: a comparison of classifiers.

,

38

Karampidis K.
Kavallieratou E.
Papadourakis G.

Comparison of classification algorithms for file type detection a digital forensics perspective.

Google Scholar

,

39

Hurtado O.M.
et al.

Comparing machine learning classifiers and linear/logistic regression to explore the relationship between hand dimensions and demographic characteristics.

Google Scholar

,

40

Toma T.T.
Dawson J.M.
Adjeroh D.A.

Human ancestry indentification under resource constraints -- what can one chromosome tell us about human biogeographical ancestry?.

]. To the best of our knowledge, a systematic quantitative comparative performance analysis of different classification methods for DNA-based prediction of appearance traits has not been conducted thus far, except for some Naïve Bayes approaches [

14

Ruiz Y.
et al.

Further development of forensic eye color predictive tests.

,

16

Walsh S.
et al.

Global skin colour prediction from DNA.

].

In this study, we focused on the evaluation and comparison of three different popular ML approaches, namely SVM, RF and ANN, and compared them with MLR, for the set of EVCs most widely used in FDP, namely categorical eye, hair and skin color and by using the previously established DNA predictors from the IrisPlex, HIrisPlex, and HIrisPlex-S systems. These ML methods have gained a lot of importance in many different application areas and, despite their higher computational cost, are well-known for their often very good prediction performances; however, within the context of FDP, they have barely been used. The main motivation of this work is to assess whether any of these ML approaches has a higher prediction performance compared with the standard MLR that is currently widely applied in the context of EVC prediction, as one may expect from the experience in other areas. In this study, all methods are applied to two different datasets, namely one containing samples from different continental ancestries and one including only the European samples thereof, and results are compared. For all four methods, we assess the standard performance for each trait category and overall, for each trait, with the aim to investigate whether ML methods are superior, or not, over conventional MLR for DNA-based appearance prediction using pigmentation traits as examples.

2. Materials and methods

2.1 Data sets

For the present study, part of the previously used datasets for the establishment of IrisPlex model for eye color [

[17]

Walsh S.
et al.

IrisPlex: a sensitive DNA tool for accurate prediction of blue and brown eye colour in the absence of ancestry information.

], the HIrisPlex model for hair color [

[20]

Walsh S.
et al.

The HIrisPlex system for simultaneous prediction of hair and eye colour from DNA.

], and the HIrisPlex-S model for skin color [

Chaitanya L.
et al.

The HIrisPlex-S system for eye, hair and skin colour prediction from DNA: introduction and forensic developmental validation.

] were applied for the prediction of those EVCs. More specifically, we used phenotype and genotype datasets from 1095 samples for eye, 1702 for hair, and 1318 for skin color prediction (complete dataset; CD), originating from Europeans, Americans, South and East Asians, African, Middle Eastern and few admixed samples. Furthermore, we used the European subset (ES) of this collection in order to restrict the analysis to a more homogenous population, comprising 821 samples for eye, 1429 for hair, and 980 for skin color prediction and originating from Ireland, Poland, Russia, Germany and Spain. These datasets were randomly split into 80% for model training and 20% for model evaluation (Table 1) for all four methods (see below).

Table 1EVC-specific data sets used for prediction model training and testing for all four classification methods.

Appearance trait	N Training set (80%)	N Test set (20%)	Data references
Eye color	876 (656)	219 (165)	17 Walsh S. et al. IrisPlex: a sensitive DNA tool for accurate prediction of blue and brown eye colour in the absence of ancestry information. Forensic Sci. Int. Genet. 2010; 5: 170-180 Abstract Full Text Full Text PDF PubMed Scopus (245) Google Scholar , 19 Chaitanya L. et al. The HIrisPlex-S system for eye, hair and skin colour prediction from DNA: introduction and forensic developmental validation. Forensic Sci. Int. Genet. 2018; 35: 123-135 Abstract Full Text Full Text PDF PubMed Scopus (136) Google Scholar , 20 Walsh S. et al. The HIrisPlex system for simultaneous prediction of hair and eye colour from DNA. Forensic Sci. Int. Genet. 2013; 7: 98-115 Abstract Full Text Full Text PDF PubMed Scopus (272) Google Scholar
Hair color	1361 (1143)	341 (286)
Skin color	1054 (784)	264 (196)

Given are the numbers for the complete dataset (CD) and, in paratheses, for the European subset (ES).

Open table in a new tab

Samples from which these data were previously obtained had been collected for the purpose of appearance genetic research under written informed consent, and sample collections were approved by the Ethics Committee of the Jagiellonian University (KBET/17/B/2005), the Commission on Bioethics of the Regional Board of Medical Doctors in Krakow (48 KBL/OIL/2008), the Clinical Research Ethics Committee of the Cork Teaching Hospitals (ref ECM 4 (dd) 11/01/11) and by the Indiana University Ethical Institutional Review Board (#1409306349).

For all available datasets considered here, we used the same eye, hair, and skin color categorization as previously applied and already established. These well-defined broad categories have been used in a number of studies before and could be considered to be close to a standard for trait categories for the time being. Furthermore, such broad categorization, with its clear distinction between a few trait categories, may better serve their application in police investigations rather than some sort of continuous scales that are more difficult to be distinguished. Furthermore, such categories are likely to be closer genetically, likely negatively affecting the respective prediction model’s performance due to a larger genetic overlap between categories, rendering the model less able to distinguish between categories. For these reasons and as previously described in detail [

Walsh S.
et al.

IrisPlex: a sensitive DNA tool for accurate prediction of blue and brown eye colour in the absence of ancestry information.

,

Chaitanya L.
et al.

The HIrisPlex-S system for eye, hair and skin colour prediction from DNA: introduction and forensic developmental validation.

,

Walsh S.
et al.

The HIrisPlex system for simultaneous prediction of hair and eye colour from DNA.

], eye colour was classified into three categories (blue, intermediate, brown) and hair colour into four categories (red, blond, brown, black), while skin colour was classified into five categories (very pale, pale, intermediate, dark, dark to black), following previously established categories. Since the European subset did not comprise samples with dark or dark to black skin colour, analyses in this subset were based on three categories only (very pale, pale, intermediate). The 41 HIrisPlex-S DNA markers were previously described by Chaitanya et al. [

Chaitanya L.
et al.

The HIrisPlex-S system for eye, hair and skin colour prediction from DNA: introduction and forensic developmental validation.

]. In brief, for eye colour, hair colour, and skin colour, we applied the 6 SNPs from the previously established IrisPlex model for eye color prediction [

[17]

Walsh S.
et al.

IrisPlex: a sensitive DNA tool for accurate prediction of blue and brown eye colour in the absence of ancestry information.

]; the 22 SNPs used for hair color prediction from the previously reported HIrisPlex model [

[20]

Walsh S.
et al.

The HIrisPlex system for simultaneous prediction of hair and eye colour from DNA.

], and the 36 SNPs applied for the skin color prediction from the previously described HIrisPlex-S model [

Chaitanya L.
et al.

The HIrisPlex-S system for eye, hair and skin colour prediction from DNA: introduction and forensic developmental validation.

], respectively.

2.2 Appearance trait categories

Trait categories were coded as categorical variables and ascendingly named as ‘1′, ‘2′, ‘3′ etc. up to the corresponding number of categories for each trait:

•
Eye color: Blue (1), Intermediate (2), Brown (3)
•
Hair color: Blond (1), Brown (2), Red (3), Black (4)
•
Skin color: Very Pale (1), Pale (2), Intermediate (3), Dark (4), Dark to Black (5); the latter two were considered only for the complete dataset

Total samples of each color category for each trait are described in detail in Supplementary Table S1. The genetic markers included in the model were converted from their initial form of the bases adenine (A), cytosine (C), guanine (G) and thymine (T) and coded numerically as 0, 1, 2 where 0 indicates homozygosity of the major allele, 1 heterozygosity and 2 homozygosity of the minor allele. For example, for an autosomal marker with major allele C and minor allele T, an individual’s genotype CC, CT and TT would be converted to 0, 1 and 2, respectively. In all models no interaction terms were taken into account, thus only the additive effects of the corresponding genetic markers were included, similar to the previously established models [

Walsh S.
et al.

IrisPlex: a sensitive DNA tool for accurate prediction of blue and brown eye colour in the absence of ancestry information.

,

Chaitanya L.
et al.

The HIrisPlex-S system for eye, hair and skin colour prediction from DNA: introduction and forensic developmental validation.

,

Walsh S.
et al.

The HIrisPlex system for simultaneous prediction of hair and eye colour from DNA.

]. Given the simple nature of our data and their final coding form as described above, we did not pursue feature engineering, such as considering squared variables or their products, since this would most likely not strongly affect our final outcomes. All data sets were previously quality controlled [

Walsh S.
et al.

IrisPlex: a sensitive DNA tool for accurate prediction of blue and brown eye colour in the absence of ancestry information.

,

Chaitanya L.
et al.

The HIrisPlex-S system for eye, hair and skin colour prediction from DNA: introduction and forensic developmental validation.

,

Walsh S.
et al.

The HIrisPlex system for simultaneous prediction of hair and eye colour from DNA.

], including deviations from Hardy-Weinberg equilibrium, excessive heterozygosity, low minor allele frequencies, genetic outlier detection using principal-components analysis etc., and could therefore be directly used for prediction modelling. Samples with missing genotype data were excluded from our analysis.

2.3 Statistical analysis

The analysis was conducted in R version 3.4.3 [

[41]

Team R.C.

R: a language and environment for statistical computing.

Google Scholar

] and ‘RStudio’ version 3.5.1 [

[42]

R. Team, RStudio: integrated development environment for R, (2016). Available from: 〈http://www.rstudio.com/〉.

Google Scholar

] using the packages ‘nnet’ [

[43]

Venables W.N.
Ripley B.D.

], ‘caret’ [

[44]

M. Kuhn, Caret: classification and regression training, (2020).

Google Scholar

], ‘e1071′ [

[45]

D. Meyer, et al., e1071: Misc Functions of the Department of Statistics, Probability Theory Group (Formerly: E1071), TU Wien, (2019).

Google Scholar

] and ‘randomForest’ [

[46]

Liaw A.
Wiener M.

Classification and regression by random forest.

Google Scholar

]. Samples with missing genotype information were excluded.

2.4 Classification algorithms and hyperparameter tuning

We conducted a comparative statistical analysis in order to obtain the efficacy and classification accuracy of four different classification methods, namely Multinomial Logistic Regression (MLR), Support Vector Machines (SVM), Random Forest (RF) and Artificial Neural Networks (ANN). Tuned hyperparameters play an important role in obtaining the optimal performance and accuracy results when using SVM, RF and ANN. Each classifier requires different tuning steps and hyperparameters that need tuning and tuned values depend each time on the training dataset. For each classifier, we tested a series of values for the tuning process with the optimal hyperparameters determined based on the lower out-of-bag (OOB) prediction error. OOB is an estimation that measures the prediction error of each method. The classified results based on the optimal set of hyperparameters were used afterwards for the comparison of all classifiers. In order to assess the accuracy of classification performances, we report metrics such as sensitivity, specificity, positive predictive value, negative predictive value, area under curve, confusion matrix and overall accuracy were reported.

2.5 Multinomial logistic regression (MLR)

The MLR approach is a ML classification method that is used to predict a nominal dependent variable based on multiple independent variables. The independent variables can be either continuous or dichotomous. It is a simple extension of the binary logistic regression that allows the dependent variable to have more than two categories. Like binary logistic regression, multinomial logistic regression uses maximum likelihood estimation in order to evaluate the probability of each category. The model can be defined as follows for the 3-class traits [

[30]

Hosmer D.W.
Lemeshow S.

]:

\ln (\frac{p_{2}}{p_{1}}) = α_{2} + \sum_{j = 1}^{k} β_{2} {(p_{2})}_{j} x_{j}

(1)

\ln (\frac{p_{3}}{p_{1}}) = α_{3} + \sum_{j = 1}^{k} β_{3} {(p_{3})}_{j} x_{j}

(2)

Where

α_{i}, β_{i} (i = 2,3)

are the regression coefficients and

p_{i} (i = 1,2,3)

are denoting the probabilities for each individual sample to belong to a certain category. The latter can be calculated as follows:

p_{2} = \frac{\exp (a_{2} + \sum_{j = 1}^{k} β_{2} {(p_{2})}_{j} x_{j})}{1 + \exp (a_{2} + \sum_{j = 1}^{k} β_{2} {(p_{2})}_{j} x_{j}) + \exp (a_{3} + \sum_{j = 1}^{k} β_{3} {(p_{3})}_{j} x_{j})}

(3)

p_{3} = \frac{\exp (a_{3} + \sum_{j = 1}^{k} β_{3} {(p_{3})}_{j} x_{j})}{1 + \exp (a_{3} + \sum_{j = 1}^{k} β_{3} {(p_{3})}_{j} x_{j}) + \exp (a_{2} + \sum_{j = 1}^{k} β_{2} {(p_{2})}_{j} x_{j})}

(4)

p_{1} = 1 - p_{2} - p_{3}

(5)

where

x_{j}

is the number of minor (less frequent) allele of the j^th SNP and j is an indicator for the number of the genetic markers included for trait prediction. For this method no parameter tuning was done. Individuals were classified to the colour category with the maximum probability

p_{i}

without any threshold values to be taken into account.

2.6 Support vector machines (SVM)

SVM [

[33]

Vapnik V.N.

The Nature of Statistical Learning Theory.

] is a machine learning approach which finds the optimal hyperplane that separates the different classes with the maximum margin, i.e. the maximum distance between the data points that belong to the different categories. It can solve linear or non-linear problems regarding the kernel function used each time [

[47]

Kecman V.

]. In our case, we applied the Gaussian radial basis function (RBF) which is a widely used kernel appropriate for non-linear classification. It can be defined as follows:

K (X_{1}, X_{2}) = \exp (- γ {‖X_{1} - X_{2}‖}^{2})

(6)

where

‖X_{1} - X_{2}‖

is the Euclidean distance between the data points

X_{1}, X_{2}

. There are two parameters that need to be tuned when using SVM classifier with RBF kernel: the parameters of cost (C) and the kernel width parameter (γ). The parameter C determines the influence of the misclassification on the objective function and γ the shape and the smoothing of the optimal hyperplane obtained. These two parameters can significantly affect the performance of an SVM model. More specifically, large C values may lead to over-fitting models while large γ could affect the shape of the hyperplane which, as a result, can affect the classification outcomes. In order to find the optimal parameters for both CD and ES, we applied the grid-searching process between ten values of γ (2⁻⁵, 2⁻⁴, 2⁻³, 2⁻², 2⁻¹, 2⁰, 2¹, 2², 2³, 2⁴) and ten values of C (2⁻², 2⁻¹, 2⁰, 2¹, 2², 2³, 2⁴, 2⁵, 2⁶, 2⁷). This procedure was applied for all three traits tested and the optimal values were chosen according to the lowest OOB error (Supplementary Figs. S1 and S4).

2.7 Random forest (RF)

The RF [

[31]

Breiman L.

Random forests.

] is a ML method for classification and regression tasks. It operates by constructing multiple decision trees during training and, in order to classify a new instance, each decision tree provides a classification for input data. The majority-vote classification is then chosen as the prediction. In its implementation we chose to tune two hyperparameters: the number of trees (ntree) and the number of features at each split (mtry). Several studies have already been published that focus on the appropriate number of trees for which one could obtain optimal results from the RF model. However, different opinions have been voiced during these studies. One typical example is the study of Liaw and Wiener [

[46]

Liaw A.
Wiener M.

Classification and regression by random forest.

Google Scholar

] which states that larger numbers of trees provide more stable results of variable importance. On the other hand, studies such as those by Latinne et al. [

[48]

Latinne P.
Olivier D.
Decaestecker C.

Limiting the number of trees in random forests.

], and Hernandez-Lobato [

[49]

Hernandez-Lobato D.
Martinez-Munoz G.
Suarez A.

How large should ensembles of classifiers be?.

], found that smaller numbers of trees can also be sufficient. The study of Oshiro et al. [

[50]

Oshiro T.M.
Perez P.S.
Baranauskas J.A.

How many trees in a random forest?.

] comprehensively addressed this question by applying the RF model to 29 different data sets and comparing their Area Under Curve (AUC) values. The main conclusion of this study was that the performance of an RF model does not necessarily improve when number of trees is increased, suggesting that a range between 64 and 128 trees can provide satisfactory results.

For optimal tree number (ntree), we checked and compared the OOB error rate for a range of 1–1000 trees and chose, separately for each trait, that number which resulted in the lowest OOB error rate. In Supplementary Figs. S2 and S5 the best values for each trait for both CD and ES are presented. For optimal mtry hyperparameter values, we used the default of the integer-rounded value of

\sqrt{p}

, where p denotes the number of variables in the model, i.e. the number of genetic markers. The corresponding mtry values for the two datasets for eye, hair and skin color therefore equaled 2, 4 and 6, respectively.

2.8 Artificial neural networks (ANN)

ANN [

34

Ripley B.D.

Neural networks and related methods for classification.

,

51

Daniel G.G.

] is a family of approaches for classification and clustering that was inspired by the human brain in order to recognize patterns in data sets. Its history starts from the early 1940s where McCulloch and Pitts [

[52]

McCulloch W.
Pitts W.

A logical calculus of ideas immanent in nervous activity.

] wrote a paper on the functionality of human brain neurons and modeled a simple neural network by using electrical circuits. Later on 1949 Donald Hebb [

[53]

Hebb D.O.

Google Scholar

] introduced the fundamental idea of learning by supporting that neural pathways are strengthened every time that are used (Hebbian learning). In the 1950s when computers became more advanced, many ANN approaches were developed and simulated. Some examples were the approach of Farley and Clark [

[54]

Farley B.
Clark W.

Simulation of self-organizing systems by digital computer.

], who simulated the aforementioned Hebbian Network and also the approach of Rosenblatt [

[55]

Rosenblatt F.

The perceptron: a probabilistic model for information storage and organization in the brain.

], who created the perceptron, an algorithm for pattern recognition. The interest of ANN continued also in the 1970 s where Werbos [

[56]

P.J. Werbos, Beyond regression: new tools for prediction and analysis in the behavioral sciences, (1975).

Google Scholar

] introduced the backpropagation algorithm that enabled the training of multi-layer networks. More recent approaches have already been established, and successfully addressed the previous challenges of deep neural networks [

57

Schmidhuber J.

Learning complex, extended sequences using the principle of history compression.

,

58

D. Scherer, A.C. Müller, S. Behnke, Evaluation of pooling operations in convolutional architectures for object recognition, in: Proceedings of the 20th International Conference Artificial Neural Networks (ICANN), (2010) p. 92–101.

Google Scholar

,

59

A.Y. Ng, et al., Building high-level features using large scale unsupervised learning, (2012).

Google Scholar

].

The ANN consists of connected units, or nodes, called artificial neurons and these connections, just as the functionality of the human brain, can transmit signals or activate other neurons [

[60]

D. Kriesel, A brief introduction to neural networks, (2007) p. 286. Available at 〈http://www.dkriesel.com〉.

Google Scholar

]. Most ANN are organized in layers and neurons, and the input data are “moving” through them only in the forward direction until some final output is obtained. Each node has its own weight which is continuously adjusted during the training procedure until data with same labels consistently yield similar output.

A number of parameters need to be tuned in order to obtain the maximum performance of the ANN model. Here, we started by tuning the number of hidden layers. At first, we looked at a range of values, starting from 1 till 10 for the hidden layers. We obtained no significant differences in the model performance for eye color prediction when we increased the number of layers. For hair and skin color prediction, we noticed some deterioration in the model performance as we increased the number of layers. Therefore, for all three traits considered here we trained our models using only one hidden layer and used the logistic function as the activation function. Other parameters that required tuning were the layer size, referring to the number of units in the hidden layer, and the decay value, acting as a regularization parameter to avoid over-fitting. Supplementary Figs. S3 and S6 give the optimal values for CD and ES respectively, according to the lowest OOB error, chosen for each of the traits.

2.9 Accuracy assessment and comparisons

In order to compare the performance of the different classifiers we presented the model measurements evaluated on the corresponding test datasets. More specifically, for each model we calculated the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), area under curve (AUC), confusion matrix and overall accuracy. Sensitivity (true positive rate) measures the proportion of the actual positive samples that are correctly identified by the model while specificity (true negative rate) refers to the proportion of the actual negative samples that were correctly identified. In addition, PPV denotes the proportion of the correct classifications among all predictions of the trait category tested each time, and NPV refers to the proportion of the correct classifications among all predictions other than the trait category of interest. AUC is a performance measure of a classification model across all possible classification thresholds while the confusion matrix describes the performance of a classification model on the test dataset for which the true values are known. Ultimately, the overall accuracy refers to the proportion of all samples that were classified correctly.

3. Results

3.1 Parameter tuning

For three out of the four methods applied, namely SVM, RF and ANN, we proceeded into parameter tuning for each of the two datasets and for the three traits (i.e. eye, hair and skin color) in order to obtain the optimal performance of the classifiers. The best parameters were chosen according to the lowest out-of-bag (OOB) error. For SVM, the parameters that needed to be tuned were γ and C. We found out that the optimal value for γ was 0.03125 for all three traits and for both CD and ES. The optimal C in the CD was equal to 2 for eye and skin color and equal to 16 for hair color (Supplementary Fig. S1). For the ES, optimal value of C was equal to 1 for eye and skin color and equal to 8 for hair color (Supplementary Fig. S4). For RF, we needed to tune the number of trees (ntree) and the optimal values for each of the traits tested. We obtained 141 trees for eye color, 713 for hair color, and 589 for skin color for CD, respectively (Supplementary Fig. S2). For the ES we obtained 349 trees for eye color, 319 for hair color and 572 for skin color (Supplementary Fig. S5). Regarding ANN, the parameters that needed to be tuned were the layer size and the regularization parameter of decay for avoiding over-fitting. For the size, we obtained optimal values of 2 for eye color, 6 for hair color, and 3 for skin color for the CD, while for the ES we obtained optimal values of 7 for eye and hair color and 1 for skin color (Supplementary Figs. S3 and S6). For the decay in the CD, the optimal values were equal to 0.5 for hair and skin color, while for eye color it was 0.4 (Supplementary Fig. S3). For the ES we obtained the optimal values for decay equal to 0.5 for eye and hair color and 0.1 for skin color (Supplementary Fig. S6).

3.2 Overall prediction accuracy

As shown in Table 2, in terms of overall accuracy, the four classification methods performed equally well in predicting each of the three considered EVCs. For eye color and the CD, we found that MLR and ANN were able to predict the trait with an overall accuracy of 0.79, while SVM and RF performed almost at the same level with 0.78. Similarly, for the ES the highest performance was obtained with MLR and ANN (0.69), followed by SVM and RF with overall accuracy values of 0.68 and 0.67, respectively. For hair and skin color, the discrepancies among the classifiers were higher compared to eye color for both datasets. More specifically, in the CD the highest overall accuracy for hair color was obtained with MLR (0.60), while SVM and ANN performed almost equally well with accuracies of 0.57 and 0.58, respectively. The RF classifier, however, appeared to have a slightly inferior performance compared to the other classifiers, reaching the lowest overall accuracy of all classifiers at 0.55 for hair color. Similarly, for the ES the MLR had the highest performance of 0.59, followed by ANN and SVM which accuracies were equal to 0.56 and 0.55, respectively. The RF classifier appeared to have a deteriorated performance compared to the other three classifiers. Similar behavior was observed also for skin color prediction in the CD, where the MLR classifier yielded the highest performance with an accuracy of 0.63 compared to the other methods. The SVM classifier yielded an overall accuracy equal to 0.60, while RF and ANN yielded the lowest performances of 0.59 and 0.56, respectively. For the ES both MLR and SVM raised the accuracy to 0.65 for skin color, while the ANN had the lowest accuracy performance of 0.57.

Table 2Overall accuracy of the EVC predictions by the four classifiers.

		MLR	SVM	RF	ANN
Eye Color	CD	0.79 (0.73–0.84)	0.78 (0.72–0.83)	0.78 (0.71–0.83)	0.79 (0.73–0.84)
Eye Color	ES	0.69 (0.61–0.76)	0.68 (0.60–0.75)	0.67 (0.59–0.74)	0.69 (0.61–0.76)
Hair Color	CD	0.60 (0.55–0.65)	0.57 (0.50–0.60)	0.55 (0.49–0.60)	0.58 (0.49–0.60)
Hair Color	ES	0.59 (0.54–0.65)	0.55 (0.49–0.61)	0.53 (0.47–0.59)	0.56 (0.50–0.61)
Skin Color	CD	0.63 (0.57–0.69)	0.60 (0.53–0.65)	0.59 (0.52–0.64)	0.56 (0.49–0.66)
Skin Color	ES	0.65 (0.58–0.72)	0.65 (0.58–0.71)	0.66 (0.59–0.72)	0.57 (0.50–0.64)

MLR: multinomial logistic regression; SVM: support-vector machine; RF: random forest; ANN: artificial neural network. CD: complete dataset; ES: European subset.

Open table in a new tab

3.3 Predictive measurements

Similar to the results of the overall accuracies, the prediction accuracy measurements for eye color presented very little to no differences between the four methods regarding blue and brown eye color, while a few deviations between the methods were seen for intermediate eye color (Table 3). For example, the sensitivity of the intermediate eye color prediction for the CD equaled 0.20 for ANN but dropped to 0.18, 0.13 and 0.15 for MLR, SVM and RF, respectively. Another example is the PPV of the intermediate eye color prediction, which obtained its highest value of 0.63 for SVM, while it dropped to 0.58 for MLR. For the ES the PPV value of intermediate eye color was raised to 0.59 for ANN while for RF it dropped to 0.42. The confusion matrices for eye color showed, for both CD and ES, small deviations among the four classifiers. Blue and brown eye colors appeared to be better predicted by the model in comparison with the intermediate eye color (Supplementary Tables S2 and S3). AUC values were at similar levels, especially for SVM, RF and ANN, while MLR slightly outperformed (Supplementary Tables S4 and S5).

Table 3Predictive measurements for eye color for the four classifiers.

		MLR			SVM			RF			ANN
Category		1	2	3	1	2	3	1	2	3	1	2	3
Sensitivity	CD	0.93 (0.87–0.97)	0.18 (0.09–0.32)	0.91 (0.82–0.95)	0.93 (0.87–0.97)	0.13 (0.05–0.26)	0.91 (0.82–0.95)	0.92 (0.86–0.96)	0.15 (0.05–0.26)	0.91 (0.82–0.95)	0.93 (0.86–0.96)	0.20 (0.12–0.38)	0.91 (0.82–0.95)
Sensitivity	ES	0.84 (0.75–0.91)	0.23 (0.13–0.38)	0.82 (0.67–0.90)	0.83 (0.73–0.90)	0.23 (0.13–0.38)	0.80 (0.66–0.89)	0.83 (0.73–0.90)	0.26 (0.15–0.41)	0.73 (0.60–0.84)	0.82 (0.72–0.89)	0.26 (0.15–0.41)	0.84 (0.71–0.91)
Specificity	CD	0.72 (0.63–0.80)	0.97 (0.94–0.99)	0.93 (0.88–0.96)	0.74 (0.64–0.80)	0.98 (0.95–0.99)	0.89 (0.84–0.94)	0.74 (0.64–0.80)	0.98 (0.94–0.99)	0.90 (0.84–0.94)	0.72 (0.65–0.81)	0.97 (0.94–0.99)	0.94 (0.87–0.96)
Specificity	ES	0.68 (0.58–0.77)	0.92 (0.86–0.96)	0.89 (0.82–0.93)	0.66 (0.56–0.75)	0.92 (0.86–0.96)	0.89 (0.82–0.93)	0.64 (0.53–0.73)	0.89 (0.82–0.93)	0.92 (0.86–0.96)	0.67 (0.57–0.76)	0.94 (0.89–0.97)	0.87 (0.80–0.92)
PPV	CD	0.75 (0.67–0.82)	0.58 (0.32–0.81)	0.87 (0.78–0.93)	0.76 (0.68–0.82)	0.63 (0.31–0.86)	0.81 (0.89–0.95)	0.76 (0.67–0.82)	0.60 (0.24–0.76)	0.82 (0.73–0.90)	0.75 (0.68–0.83)	0.62 (0.39–0.84)	0.88 (0.78–0.92)
PPV	ES	0.70 (0.60–0.78)	0.47 (0.27–0.68)	0.75 (0.62–0.85)	0.68 (0.58–0.77)	0.47 (0.27–0.68)	0.75 (0.62–0.85)	0.67 (0.57–0.75)	0.42 (0.24–0.61)	0.80 (0.66–0.89)	0.68 (0.58–0.77)	0.59 (0.36–0.78)	0.73 (0.60–0.83)
NPV	CD	0.92 (0.85–0.96)	0.84 (0.78–0.88)	0.95 (0.90–0.98)	0.92 (0.85–0.96)	0.83 (0.78–0.88)	0.95 (0.90–0.97)	0.91 (0.84–0.96)	0.84 (0.78–0.88)	0.95 (0.90–0.98)	0.92 (0.84–0.96)	0.84 (0.79–0.89)	0.95 (0.90–0.98)
NPV	ES	0.83 (0.73–0.90)	0.79 (0.72–0.85)	0.92 (0.85–0.96)	0.82 (0.71–0.89)	0.79 (0.72–0.85)	0.91 (0.84–0.95)	0.81 (0.70–0.89)	0.79 (0.72–0.85)	0.89 (0.82–0.93)	0.80 (0.70–0.88)	0.80 (0.73–0.86)	0.93 (0.86–0.96)

Eye color categories: 1: Blue; 2: Intermediate; 3: Brown. MLR: multinomial logistic regression; SVM: support-vector machine; RF: random forest; ANN: artificial neural network. PPV: Positive predictive value; NPV: negative predictive value. CD: complete dataset; ES: European subset.

Open table in a new tab

For hair color, we also observed rather similar prediction performances for all four methods, although more pronounced differences were seen for some trait categories (Table 4) compared to eye color (Table 3). In particular, the sensitivity of Red hair color prediction in the CD reached its highest value with MLR (0.66), followed by ANN (0.58), while its value was almost halved to 0.28 for RF, and for SVM it reached 0.21 (Table 4). The sensitivity of Black hair color prediction completely dropped to zero for SVM, while its highest value was equal to 0.31 for ANN. Another example was the PPV for Black hair color, where we obtained the highest values with MLR and RF (0.58 and 0.47, respectively), while it dropped to 0.34 for ANN. We observed a similar behavior to the CD in the ES for the sensitivity of red hair color prediction where its highest values were yielded by MLR and ANN (0.69 and 0.62, respectively), while for RF and SVM the value was halved to 0.31 and 0.23, respectively. Sensitivity of black hair color dropped to zero for SVM and RF, while its highest value was obtained with MLR (0.26). PPV for black hair color reached its highest value with MLR, while it dropped to zero for RF. The confusion matrices for hair color showed similar patterns for CD and ES where the categories with fewer samples in the datasets, such as red and black hair color categories, showed higher deviations compared to blond and brown hair color (Supplementary Tables S6 and S7). AUC values for MLR outperformed for most category comparisons compared to the other ML classifiers (Supplementary Tables S2 and S3).

Table 4Predictive measurements for hair color for the four classifiers.

		MLR				SVM				RF				ANN
Category		1	2	3	4	1	2	3	4	1	2	3	4	1	2	3	4
Sensitivity	CD	0.70 (0.62–0.77)	0.59 (0.50–0.67)	0.66 (0.47–0.80)	0.20 (0.11–0.38)	0.66 (0.58–0.73)	0.65 (0.57–0.73)	0.21 (0.10–0.38)	0	0.67 (0.59–0.74)	0.56 (0.45–0.62)	0.28 (0.15–0.46)	0.26 (0.14–0.42)	0.69 (0.59–0.74)	0.46 (0.38–0.55)	0.58 (0.41–0.74)	0.31 (0.19–0.48)
Sensitivity	ES	0.81 (0.73–0.87)	0.43 (0.35–0.52)	0.69 (0.50–0.83)	0.26 (0.13–0.47)	0.88 (0.81–0.93)	0.40 (0.32–0.49)	0.23 (0.11–0.42)	0	0.78 (0.70–0.85)	0.44 (0.35–0.53)	0.31 (0.17–0.50)	0	0.72 (0.63–0.79)	0.46 (0.38–0.55)	0.62 (0.43–0.78)	0.17 (0.07–0.37)
Specificity	CD	0.70 (0.63–0.76)	0.68 (0.63–0.76)	0.98 (0.96–0.99)	0.98 (0.96–0.99)	0.68 (0.60–0.73)	0.58 (0.51–0.64)	1	1	0.62 (0.55–0.69)	0.67 (0.60–0.73)	0.99 (0.98–0.99)	0.97 (0.94–0.98)	0.67 (0.60–0.73)	0.67 (0.61–0.74)	0.99 (0.97–0.99)	0.93 (0.90–0.95)
Specificity	ES	0.57 (0.50–0.64)	0.82 (0.76–0.87)	0.98 (0.95–0.99)	0.97 (0.94–0.98)	0.41 (0.34–0.49)	0.83 (0.77–0.88)	0.99 (0.98–0.99)	1	0.48 (0.41–0.56)	0.75 (0.68–0.81)	0.99 (0.97–0.99)	0.99	0.59 (0.52–0.67)	0.73 (0.65–0.79)	0.99 (0.97–0.99)	0.96 (0.93–0.98)
PPV	CD	0.63 (0.55–0.70)	0.54 (0.48–0.64)	0.79 (0.60–0.91)	0.58 (0.32–0.81)	0.60 (0.51–0.66)	0.50 (0.43–0.58)	1	NA	0.56 (0.49–0.63)	0.52 (0.43–0.59)	0.80 (0.49–0.94)	0.47 (0.27–0.68)	0.60 (0.52–0.67)	0.48 (0.40–0.57)	0.85 (0.64–0.95)	0.34 (0.20–0.52)
PPV	ES	0.56 (0.49–0.64)	0.64 (0.53–0.74)	0.75 (0.55–0.88)	0.43 (0.22–0.67)	0.50 (0.44–0.57)	0.64 (0.52–0.73)	0.86 (0.49–0.97)	NA	0.51 (0.44–0.58)	0.56 (0.46–0.66)	0.80 (0.49–0.94)	0	0.55 (0.47–0.62)	0.55 (0.46–0.65)	0.84 (0.62–0.94)	0.29 (0.12–0.55)
NPV	CD	0.76 (0.70–0.82)	0.72 (0.65–0.78)	0.96 (0.94–0.98)	0.91 (0.87–0.95)	0.74 (0.66–0.79)	0.72 (0.65–0.79)	0.93 (0.90–0.95)	0.90	0.72 (0.65–0.79)	0.70 (0.62–0.75)	0.94 (0.90–0.96)	0.92 (0.88–0.94)	0.74 (0.67–0.80)	0.66 (0.60–0.72)	0.96 (0.93–0.98)	0.92 (0.89–0.94)
NPV	ES	0.82 (0.74–0.88)	0.66 (0.60–0.72)	0.97 (0.94–0.98)	0.94 (0.90–0.96)	0.83 (0.74–0.90)	0.66 (0.59–0.72)	0.93 (0.89–0.95)	0.92	0.77 (0.68–0.84)	0.65 (0.58–0.71)	0.93 (0.90–0.96)	0.92	0.75 (0.67–0.82)	0.65 (0.58–0.71)	0.96 (0.93–0.97)	0.93 (0.93–0.95)

Hair color categories: 1: Blond; 2: Brown; 3: Red; 4: Black. MLR: multinomial logistic regression; SVM: support-vector machine; RF: random forest; ANN: artificial neural network. PPV: Positive predictive value; NPV: negative predictive value. CD: complete dataset; ES: European subset.

Open table in a new tab

For skin color, as with hair color, we also observed uneven differences between classifiers for some predictive measurements and trait categories (Table 5). For example, in the complete dataset the sensitivity of the Very Pale skin color category prediction was 0.11 for both MLR and SVM but zero when RF and ANN were applied. Similar diminution was also observed for the sensitivity and the PPV of RF in predicting Dark skin color. RF was the only classification method where these values equaled zero (Table 5). Higher discrepancies were also observed for the specificity of pale skin color where its highest values were obtained for both MLR and RF (0.60); with SVM was applied the value dropped to 0.40. Sensitivity of dark to black category dropped to 0.66 for ANN, while for SVM and RF it reached the highest value of 0.96. In the ES, the sensitivity of very pale skin color reached the highest value of 0.25 with MLR, while for the rest of the classifiers it was almost equal to zero. The specificity of pale skin color yielded its highest value of 0.65 with MLR but dropped to 0.40 for RF. For most of the other skin color categories and predictive measurements, the four classification methods performed almost equally (Table 5). In the confusion matrices for skin color, the categories with the highest number of samples, namely Pale and Intermediate categories, were better predicted in comparison to the other categories (Supplementary Tables S8 and S9). Also and similar to eye and hair color prediction, the AUC values for MLR mostly outperformed the other classifiers (Supplementary Tables S2 and S3).

Table 5Predictive measurements for skin color for the four classifiers.

		MLR					SVM					RF					ANN
Category		1	2	3	4	5	1	2	3	4	5	1	2	3	4	5	1	2	3	4	5
Sensitivity	CD	0.11 (0.02–0.44)	0.76 (0.68–0.83)	0.47 (0.38–0.57)	0.75 (0.30–0.95)	0.88 (0.69–0.96)	0.11	0.83 (0.75–0.89)	0.31 (0.23–0.41)	0.25 (0.05–0.70)	0.96 (0.80–0.99)	0.00	0.76 (0.68–0.83)	0.19 (0.12–0.27)	0.00	0.96 (0.80–0.99)	0.00	0.61 (0.52–0.70)	0.50 (0.42–0.60)	0.50 (0.15–0.85)	0.66 (0.47–0.82)
Sensitivity	ES	0.25 (0.09–0.53)	0.70 (0.61–0.78)	0.65 (0.54–0.75)			0	0.76 (0.68–0.83)	0.58 (0.47–0.69)			0	0.81 (0.73–0.87)	0.54 (0.43–0.65)			0.08 (0.01–0.35)	0.68 (0.59–0.76)	0.49 (0.38–0.60)
Specificity	CD	0.99 (0.97–0.99)	0.60 (0.52–0.68)	0.80 (0.73–0.86)	0.98 (0.96–0.99)	1.00 (0.98–1.00)	0.99	0.46 (0.38–0.54)	0.85 (0.78–0.89)	0.98 (0.96–0.99)	0.99 (0.97–0.99)	1.00	0.60 (0.52–0.68)	0.92 (0.87–0.96)	0.99	0.99 (0.96–0.99)	0.99	0.58 (0.50–0.66)	0.65 (0.58–0.72)	0.99 (0.97–0.99)	0.99 (0.97–0.99)
Specificity	ES	0.97 (0.93–0.98)	0.65 (0.55–0.74)	0.74 (0.65–0.81)			1	0.56 (0.45–0.66)	0.80 (0.73–0.85)			1	0.49 (0.39–0.59)	0.81 (0.73–0.87)			0.97 (0.94–0.99)	0.50 (0.40–0.60)	0.70 (0.62–0.78)
PPV	CD	0.25 (0.05–0.70)	0.61 (0.53–0.69)	0.62 (0.51–0.72)	0.38 (0.14–0.69)	1.00 (0.85–1.00)	0.25	0.56 (0.48–0.63)	0.59 (0.46–0.70)	0.25 (0.05–0.70)	0,95 (0.80–0.99)	NA	0.61 (0.53,0.69)	0.63 (0.45–0.77)	0.00	0.88 (0.71–0.96)	0.00	0.55 (0.46–0.63)	0.50 (0.41–0.60)	0.40 (0.12–0.77)	0.94 (0.73–0.99)
PPV	ES	0.33 (0.12–0.65)	0.72 (0.63–0.80)	0.60 (0.49–0.70)			NA	0.69 (0.60–0.76)	0.58 (0.47–0.69)			NA	0.67 (0.59–0.74)	0.63 (0.51–0.74)			0.17 (0.03–0.56)	0.64 (0.55–0.72)	0.50 (0.39–0.61)
NPV	CD	0.97 (0.94–0.98)	0.76 (0.67–0.83)	0.69 (0.62–0.75)	0.99 (0.97–0.99)	0.99 (0.96–0.99)	0.96	0.76 (0.67–0.84)	0.64 (0.57–0.70)	0.99 (0.97–0.99)	0.99 (0.97–0.99)	0.97	0.76 (0.67–0.83)	0.62 (0.56–0.68)	0.98	0.99 (0.97–0.99)	0.97	0.65 (0.56–0.73)	0.66 (0.58–0.73)	0.99 (0.97–0.99)	0.97 (0.94–0.98)
NPV	ES	0.95 (0.91–0.97)	0.63 (0.53–0.72)	0.78 (0.69–0.84)			0.94	0.65 (0.54–0.75)	0.80 (0.73–0.85)			0.94	0.67 (0.54–0.77)	0.74 (0.66–0.81)			0.94 (0.90–0.97)	0.55 (0.44–0.66)	0.69 (0.61–0.77)

Skin color categories: 1: Very pale; 2: Pale; 3: Intermediate; 4: Dark; 5: Dark to Black. MLR: multinomial logistic regression; SVM: support-vector machine; RF: random forest; ANN: artificial neural network. PPV: Positive predictive value; NPV: negative predictive value. CD: complete dataset; ES: European subset.

Open table in a new tab

4. Discussion

In the present study, we compared four different ML classification methods, namely MLR, as widely used for EVC prediction from DNA in general, and pigmentation prediction in particular, in addition to SVM, RF and ANN with respect to their ability to predict various eye, hair and skin color categories based on the previously established IrisPlex, HIrisPlex, and HIrisPlex-S DNA markers. Since these ML methods have been barely applied for EVC prediction so far and are well-known for their often very good prediction performance in other application fields, the basic motivation for this study was to investigate and to identify, for each of the tested EVCs, the optimal classifier yielding the highest performance and assess whether any of them outperforms the standard MLR approach. In order to obtain the maximum performance of the SVM, RF and ANN methods, we first needed to perform hyperparameter tuning. Parameters such as cost and gamma for SVM, ntree for RF and size and decay for ANN were tuned and their optimal values were chosen according to the lowest OOB error (Supplementary Figs. S1–S6).

Our results showed that when it comes to overall accuracy, all four classifiers performed almost equally well for all pigmentation traits tested, with almost no variation across the classifiers for eye color and slight variation for hair and skin color. Thus, none of the other ML methods outperformed the conventional method of MLR in predicting eye, hair and skin color based on the IrisPlex, HIrisPlex, and HIrisPlex-S DNA markers, respectively. When looking at the full suite of prediction measurements per each of the three pigmentation traits, we noted slight differences between some classifiers for several trait categories, somewhat more for hair and skin color than for eye color. However, these differences do not allow a conclusion that any of the three ML classifiers perform superior over MLR, which is supported by our conclusion derived from the overall accuracy results. This pattern was also observed when we compared the prediction performances between the two datasets, CD and ES, where highest deviations were observed for hair and skin color compared to eye color. This was to be expected since European samples represent the major part of the CD, implying that our model was trained mostly on European samples and therefore, when we compare the performance of the CD-derived model with the one trained on the ES, we do not expect to see high differences in the overall performance.

For eye color and for both datasets, we saw a small but noticeable deviation between the four classification methods for the intermediate eye color category, while for blue and brown eye color categories, all four methods performed almost identically. As obtained with all four methods, prediction accuracies were high for blue and brown eye color, but low for intermediate eye color. This finding is in line with previous results obtained mostly based on MLR [

13

Liu F.
et al.

Eye color and the prediction of complex phenotypes from genotypes.

,

Walsh S.
et al.

IrisPlex: a sensitive DNA tool for accurate prediction of blue and brown eye colour in the absence of ancestry information.

,

Chaitanya L.
et al.

The HIrisPlex-S system for eye, hair and skin colour prediction from DNA: introduction and forensic developmental validation.

,

Walsh S.
et al.

The HIrisPlex system for simultaneous prediction of hair and eye colour from DNA.

,

21

Walsh S.
et al.

DNA-based eye colour prediction across Europe with the IrisPlex system.

,

24

A.J S.
Harbison S.

SNP model development for the prediction of eye colour in New Zealand.

]. As emphasized in all previous IrisPlex publications [

Walsh S.
et al.

IrisPlex: a sensitive DNA tool for accurate prediction of blue and brown eye colour in the absence of ancestry information.

,

Chaitanya L.
et al.

The HIrisPlex-S system for eye, hair and skin colour prediction from DNA: introduction and forensic developmental validation.

,