Credit Scoring Based on the Set-Valued Identification Method

J Syst Sci Complex Credit Scoring Based on the Set-Valued Identification Method∗ WANG Ximei · HU Min · ZHAO Yanlong · DJEHICHE Boualem DOI: 10.1007/s11424-020-9101-4 Received: 22 March 2019 / Revised: 28 May 2019 c The Editorial Office of JSSC & Springer-Verlag GmbH Germany 2020 Abstract Credit scoring is one of the key problems in financial risk managements. This paper studies the credit scoring problem based on the set-valued identification method, which is used to explain the relation between the individual attribute vectors and classification for the credit worthy and credit worthless lenders. In particular, system parameters are estimated by the set-valued identification algorithm based on a given recognition criteria. In order to illustrate the efficiency of the proposed method, practical experiments are conducted for credit card applicants of Australia and credit card holders from Taiwan, respectively. The empirical results show that the set-valued model has a higher prediction accuracy on both small and large numbers of data set compared with logistic regression model. Furthermore, parameters estimated by the set-valued identification method are more stable, which provide a meaningful and logical explanation for extracting factors that influence the borrowers’ credit scorings. Keywords 1 Credit scoring, logistic regression model, prediction accuracy, set-valued model. Introduction Credit scoring problem is extensively studied by academics and practitioners and has been one of the key topics in financial risk management[1] . The purpose of the credit scoring is to WANG Ximei Department of Mathematics, KTH Royal Institute of Technology, Stockholm 10044, Sweden. Email: ximei@kth.se. HU Min · ZHAO Yanlong (Corresponding author) Key Laboratory of Systems and Control, Institute of Systems Science, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China; School of Mathematical Sciences, University of Chinese Academy of Sciences, Beijing 100190, China. Email: humin17@mails.ucas.edu.cn; ylzhao@amss.ac.cn. DJEHICHE Boualem Department of Mathematics, KTH Royal Institute of Technology, Stockholm 10044, Sweden. ∗ This research was supported by the National Key R&D Program of China under Grant No. 2018YFA0703800, the National Natural Science Foundation of China under Grant No. 61622309, and the Verg Foundation (Sweden). ⋄ This paper was recommended for publication by Editor LIU Yungang. 2 WANG XIMEI, et al. assess the ability and willingness of individuals in meeting their financial obligations on time. For example, for the lenders such as banks or companies, it is important to evaluate whether the borrowers have good credit scores since accurate estimation of the scores can help to decide whether to approve the loan request as well as the lending rate to each borrower[2] . The lenders classify the borrowers based on the profile information such as financial ratios, job attributes, purposes of loan, etc[3, 4] . Generally, there are two kinds of credit scoring methods: Model-based and statistical-based method. The so-called model-based method includes CreditMetrics model, Credit Risk+ model, KMV model, etc. CreditMetrics and Credit Risk+ model are proposed by JP Morgan and credit suisse financial products (CSFP), respectively[5, 6] . They are based on the estimation of the forward-looking distribution in value changes for a loan portfolio and bond type products during a given period. KMV model[7] which is studied by KMV company focuses on computing the default probabilities under the assumption that the returns of the assets follow a Black-Scholes dynamics[8] . In this model, default occurs when the total value of the firm’s assets falls below a certain value. For the statistic-based method, credit scoring is a binary classification problem to distinguish good customers from the bad ones[9, 10] . The credit scoring problem is modeled as a binary or multi valued classification which the relation between probability of default and the attributes of the lenders are learned from data. Many statistical algorithms such as logistic regression (LR) model, ordered probit model, artificial neural networks (ANN), support vector machines (SVM) are widely applied in predicting the probability of default for lenders[11–14] . Around 10 to 30 attributes which generally consist of financial key measures, and other quantitative or qualitative variables are considered in the statistical-based method. Set-valued (SV) model is a new class of systems emerging from network and information technology. Unlike traditional systems, the available information to SV system is not accurate output. Specifically, the SV system output is in some set, even binary, which brings great difficulties for parameter identification. In recent years, through the tireless efforts of some researchers, many innovative identification methods have been successfully applied to set-valued systems and some interesting results have been obtained[15, 16] . With the improvement of the theoretical work, the SV system has gradually been applied to various fields, such as complex diseases and genome-wide association analysis[17] . In this paper, the SV model is used for the credit scoring problem and the recognition criteria is given. Then we conduct experiments on Australian data set with 690 credit card applicants and Taiwan data set with 30,000 credit card holders. Comparing the experiment results between the LR and SV model, the latter has a higher prediction accuracy and furthermore, parameters estimated by the SV model are more stable than the LR model, which means that the SV model not only has a better prediction accuracy results, but also competes in interpretations and explanation of the attributes information. Thus, we can extract meaningful attributes that influence the credit scoring. The main contributions of this paper can be summarized as follows. 1) A novel system model — The SV model is used in the credit scoring problem to classify credit worthless lenders from the credit worthy ones. 2) Compared to the LR model, the SV model has a higher prediction accuracy on both large CREDIT SCORING BASED ON A SET-VALUED IDENTIFICATION METHOD 3 and small numbers of samples. In addition, parameters estimated with the SV model is more stable than the LR model. 3) The main factors which influence the default of the lenders are analyzed under the SV model, which can be viewed as the warning indicator for lenders. The rest of this paper is structured as follows: Section 2 gives a general review of literatures for credit scoring model; Section 3 proposes a specific method for classification prediction — SV model and parameter identification algorithm; In Section 4 we conduct experiments on two data sets, Australian credit card applicants data and Taiwan credit card holders data respectively to predict the credit worthy and credit worthless categories. Conclusions of this paper are presented in Section 5. 2 Literature Review Credit scoring is a typical classification problem. Various techniques are applied to distinguish the “bad” (credit worthless) lenders from the “good” (credit worthy) ones[18–20] . While according to the prudence principle of financial risk management, not only the prediction accuracy but also the logic behind is an important evaluation standard for the model. LR model is a generalized linear regression method to calculate the probability of default from the attributes[21] . It specifies that the logit function of the probability of default is a linear function based on the observed values of the available explanatory variables. LR model is widely used in credit scoring problems for lenders because of its simple implementation and good explanation[22] . The limitation of the LR model is the assumption of logit noise. In addition, non-linear relation between attributes and the probability of default cannot be revealed in the model[23] . Fuzzy classification model makes use of fuzzy rules to evaluate similarity between objects[24] . It is a popular classification method which can be used in credit scoring problem[25, 26] and has been widely studied in recent years. The main idea of this method is converting the target feature to fuzzy sets and membership functions, then determining the target category through the fuzzy relationship and fuzzy reasoning. Fuzzy classification method can deal with some of the challenges encountered in conventional recognition pattern. With the explosion of information in technology, data mining techniques which are effective in data processing have been applied in credit scoring problems[27] . For example, Bayesian method, K-nearest neighbor (KNN) classifier, random forest (RF) model, ANN, etc[4, 28–30] . While Bayesian method requires the knowledge of prior distributions. Then the minimum error rate or the minimum risk criteria can be given and the target can be recognized by the criteria; KNN classifier makes no assumptions about the specific distributions. However, usually it has a better prediction performance if “good” and “bad” samples have equal numbers[31] . The choice of “k” and the measure of distance also highly affect the prediction accuracy. ANN has the capability of adaptive, self-organizing, e-learning, which can handle problems with very complex environmental information or unclear background knowledge. By learning to build the sample memory, the unknown mode will be sentenced to the closest memories. While it is not 4 WANG XIMEI, et al. always the “best” model in credit scoring problem. [32] compares the credit scoring prediction accuracy of the ANN with five other traditional techniques including discriminant analysis, LR, KNN, kernel density estimation and decision trees methods, it turns out that the LR model is more precise than the ANN in average cases. More details about the comparison of the statistic methods can be found in [23, 33, 34]. According to a large numbers of studies for credit scoring modeling, there is no consistent conclusion on which method is the most accurate method to be used. Sometimes there are conflicts comparing the findings in different studies[35, 36] . Generally, lenders prefer choosing models with interpretability and the transparency. According to the literature [35], “two aspects of methods for credit scoring are very important: That is the predictive performance, as well as the insights or interpretations that are revealed by the model”. 3 Set-Valued Model This section describes how to build the SV model that shows the relation between the attribute vectors and the target category. Then we can give the classification criteria based on the model. 3.1 System Model It is common to use a linear model to approximate the situation that the model is unknown. So we assume that there exists an implicit index which is the linear combination of the different features and the target category is decided by the implicit index. Then, the relation between the feature vector and the target category can be described as the set-valued system model as follows: ⎧ m  ⎪ ⎪ ⎨ yk = φT φkj θj + dk , k θ + dk = (1) j=1 ⎪ ⎪ ⎩s = I , k = 1, 2, · · · , N, k {yk ≤C} where m, N are the number of the attributes and samples (i.e., the number of lenders), respectively. Vectors φk = (φk1 , φk2 , · · · , φkm )T , θ = (θ1 , θ2 , · · · , θm )T are the attributes of the kth lender and the parameters to be estimated, respectively. {sk , k = 1, 2, · · · , N } is the output observation, where sk = 1 (sk = 0) means that the kth lender is the credit worthy (credit worthless). dk is the system noise and I(·) stands for the indicator function. Inevitably there will be some errors existing in the acquisition and processing of the lenders’ attribute information. Therefore the noise dk is added as the supplements of other environment factors. According to the center limit theorem, we assume that dk is distributed normally with mean of 0 and variance of σ 2† . 3.2 Set-Valued Identification Algorithm For the SV identification algorithm, the feature vector {φk , k = 1, 2, · · · , N } and its target category {sk , k = 1, 2, · · · , N } are used to estimate the parameter θ. † Technically, the noise dk can be any distribution. CREDIT SCORING BASED ON A SET-VALUED IDENTIFICATION METHOD 5 Traditional SV identification method needs some requirements for the input variable {φk , k ≤ N }, such as periodic input forms, and so on. In recent years, this constraint has been overcome by using the maximum likelihood estimation (MLE), which leads SV identification to a wide range of applications in different fields. We use an iterative identification algorithm based on expectation maximum (EM) proposed in [37]. The specific form is as follows: i+1 θ i  N  −1 φk φT k =θ +  N k=1  i · φk σ 2 f (C − φT kθ ) × − k=1 I{sk =1} F (C − i φT kθ ) + I{sk =0} i 1 − F (C − φT kθ ) , (2) where θi is the estimated value of the parameter at the ith iteration, C is a known threshold, F (·), f (·) denote the probability distribution function of the standard normal distribution and probability density function, respectively. [37] has proved that if the corresponding MLE exist, the iterative algorithm can converge to the point of MLE and can achieve exponential convergence rate regardless of the initial value of the iteration. In the credit scoring problem, the threshold C is unknown due to the lack of prior information. So we should estimate both the unknown parameter θ and the threshold C. This can be shown as follows: system {(φk , sk ), k ≤ N } −−−−−−−−→ (θ, C). identification To estimate C, we change C into a component of the new parameters: φk  (φk1 , φk2 , · · · , φkm , −1)T , θ  (θ1 , θ2 , · · · , θm , C)T . Then the model (1) is equal to the following model: ⎧ m  ⎪ ⎪ ⎨ y = φT θ + dk = φkj θj + dk − C, k k ⎪ ⎪ ⎩s = I k {y j=1 k ≤0} , k = 1, 2, · · · , N. By Equation (2), the estimate of the new parameter  θ can be given as follows: ⎡ ⎤⎞ N −1  N   I I i+1 i i {sk =1} {sk =0}   ⎦⎠ , θ ) × ⎣− φk σ 2 f (−φT = θ + φk φT + θ k k Ti Ti F (−φk θ ) 1 − F (−φk θ ) k=1 k=1 i where  θ is the estimation of the new parameter at the ith iteration. The estimation of the new  parameter  θ can determine the original parameter θ = (θ1 , θ2 , · · · , θn )T and the threshold C. 3.3 Classification Criteria The SV identification algorithm can give the estimations of the parameter θ and the thresh According to the model (1), we can get the following criteria by replacing old C (i.e., θ and C). 6 WANG XIMEI, et al.  where φ is the feature vector of the unknown target and the noise is θ and C with θ and C, assumed to be zero. m  s = I{y≤C} φj θj , y = φT θ =  . j=1 If s = 1, the unknown state of lender is credit worthy (credit worthless). 4 Classification Results This section gives classification results for two data sets: The credit applicants data set of Australia and credit card holders data from Taiwan, respectively. Firstly, the attributes for the two datasets are analyzed before the specific modeling. Second, feature extraction and standardization techniques are used to eliminate the irrelevant or less important factors. Finally, the SV and LR model are used to classify the “bad” lenders from “good” ones. The results show that not only prediction accuracy of the SV model is higher than which of the LR model, but also the estimations of the parameters with the SV model are more stable than those of the LR model. Besides, important factors that affect the credit scorings are analyzed. 4.1 4.1.1 Credit Card Applicants Data of Australia Data Description We obtain the Australian credit card applicants data from the UCI repository of machine learning database[11, 38]‡ . This data set includes 690 samples in which 383 (55.5%) applicants are credit worthless, denoted as class “0”, 307 (44.5%) are credit worthy belonging to class “1”. There are 14 attributes for each applicant, 6 of the attributes (A2, A3, A7, A10, A13, A14) are continuous and 8 (A1, A4, A5, A6, A8, A9, A11, A12) are categorical. 4.1.2 Data Processing Since the data is a mixture of continuous and categorical attributes, it is preprocessed by the following three steps: Step 1 Extracting the continuous and categorical attributes by Mann-Whitney-Wilcoxon rank sum test and contingency table method, respectively; Step 2 Transforming the categorical data into dummy variables; Step 3 Standardizing the data with MinMaxScaler scale. Mann-Whitney-Wilcoxon rank sum test[39] in Step 1 is used to test whether the continuous attributes will statistically significantly influence the classification results. Contingency table method[40] is used for testing the significance of categorical attributes. In Step 2, a dummy variable is a dichotomous variable which is used to code the categorical data. The number of dichotomous variables equals to G − 1, where G is the number of the original categories[41] . Regression method with dummy variables are widely used and cheaply fitted by the mechanical procedure of “dropping out” one of the variables in the system[42] . MinMaxScaler in Step 3 is one of the feature preprocessing methods. It transforms features by scaling each feature to ‡ Australian credit approval data set is submitted by quinlan@cs.su.oz.au, all attribute names and values have been changed to meaningless symbols to protect confidentiality of the data. CREDIT SCORING BASED ON A SET-VALUED IDENTIFICATION METHOD 7 a given range§ . Then the prepared data set are used to classify the credit worthy and credit worthless applicants with the SV and LR model, respectively. In this paper, training set and testing set are divided randomly in order to avoid specificity. In particular, training set is used to estimate the parameters of the model and testing set is to predict the classifications. 4.1.3 Result Analysis In this part, we analyze classification results between the SV and LR model from two aspects — The prediction accuracy and the stability of coefficients. Table 1 presents the prediction accuracy results for SV and LR models, respectively. Noting that for both training and testing data, the prediction accuracy of the SV model is higher than the LR method. Table 1 Prediction accuracy (Australia) Model Training Accuracy Testing Accuracy LR 0.8900 0.8444 SV 0.8917 0.8556 Furthermore, the classification prediction accuracies for training and testing data over 10 experiments are demonstrated in Figure 1. It shows that the prediction accuracy of the SV and LR model have little difference under the training set. However, for the testing set, SV model performs better than the LR model, which means the former one has a stronger generalization than latter. 0.9 0.89 0.88 prediction accuracy 0.87 0.86 0.85 0.84 0.83 SV training accuracy SV testing accuracy LR training accuracy LR testing accuracy 0.82 0.81 0.8 Figure 1 § In 1 2 3 4 5 6 7 numbers of experiment 8 9 10 Prediction accuracy of the SV and LR model for training and testing set (Australia) this paper, for the given range [0,1], all the features are scaled by Xscaled = X−Xmin . Xmax −Xmin 8 WANG XIMEI, et al. Figure 2 shows the standard deviation (SD) of ten experiments for all the coefficients in the SV and LR model, respectively. It shows that the SD of the coefficients for the LR model is larger, which means the corresponding coefficients are different for each experiment estimated with the LR model. Compared to the LR model, the SV model has more stable coefficients among the ten experiments with tiny fluctuation. Generally, we prefer to infer an exact model with stable coefficients from samples rather than models with unstable ones. Therefore, the SV model is better than the LR model in the sense of coefficient stability. 60 SD of SV parameters SD of LR parameters 50 SD 40 30 20 10 0 0 Figure 2 5 10 15 20 parameters 25 30 35 SD of the coefficients for the SV and LR model (Australia) Since the SV model has stronger generalization and stable training results, we can use it to explain the mechanism of the Australian credit approval data set. Table 2 displays the first 6 important anonymous attributes and corresponding coefficients. We can get the most important attribute which strongly influences the credit scoring of the lenders. A14 and A10 stand for the 14th and 10th attribute, respectively. A6(3) stands for the 3rd dummy variable of attribute A6, and similar explanations with A5(5), A5(1), and A6(7). Noting that the attributes A14, A10, A6(3) are negatively related to the credit scoring classification. One can make an effort on these specific attributes to change the values larger or smaller than the threshold C if someone wants to improve credit scorings. Table 2 First eight important coefficients and corresponding variables Variables C (threshold) A14 A10 A6(3) A5(5) A5(1) A6(7) coefficients 4.2020 −19.0764 −3.9817 −3.5541 3.4034 3.3248 2.9971 CREDIT SCORING BASED ON A SET-VALUED IDENTIFICATION METHOD 4.2 9 Credit Card Holders Data of Taiwan The second data set is the history payment of credit card holders taken from an important bank in Taiwan. It contains 30,000 observations in total, in which 6366 observations (22.12%) are with default payment, see [23] for the details of the data. In this classification analysis, if one defaults, he will be classified to the set “1”, otherwise “0”. 4.2.1 Data Description There are 23 explanatory variables in this dataset to be used as attribute information. While other attributes are created based on the original data to explore more information. For example, average (logarithmic) amount of the past payment, relatively amount of the payment for each month. All the additional variables are listed in Table 3. Finally, there are 72 attributes to be used in this data set. There is no doubt that some attributes are linear correlated to each other. Similarly, both continuous and categorical variables exist in the data set. In addition to the data processing mentioned in Subsection 4.1.2, we need to select most related features from all of the 72 attributes. In this paper, we select features according to the highest scores based on univariate feature selection with the analysis of variance (ANOVA) F-test for the data[44] . Thus, the attributes are reduced into k = 30¶ . After the above transformation of the data, the data set is divided into training and testing data randomly. Table 3 Additional variables Logarithmic of the payment and bill statement for each month Average (logarithmic) and standard deviation payment status Average (logarithmic) and standard deviation of payment (bill statement) amount Additional Variables Relative values of the average payment (bill statement) values Sign of the bill statement Sum product of the payment and bill statement amount for each month Sign of the sum product of payment state and bill statement amount for each month 4.2.2 Result Analysis Classification accuracy of the LR and SV model is presented in Table 4 . We can see that the prediction accuracy of the SV model for both training and testing data set are higher than the LR model. Figure 3 shows all the prediction accuracies for 10 experiments. Noting that for each experiment, the SV model has a higher prediction accuracy. Figure 4 shows the SD for all the parameters of the selected attributes. It is shown that for the large data set, parameters of the LR model and the SV model are stable in a range of 0–0.5. ¶ In python, the classes in the sklearn.feature selection module can be used for feature selection/dimensionality reduction on sample sets. 1  The threshold of logistic regression algorithm are calculated as p threshold = 1+e−c , where c is the intercept of the logistic function. 10 WANG XIMEI, et al. Table 4 Classification accuracy Method Training accuracy Testing accuracy LR 0.8115 0.8066 SV 0.8286 0.8229 0.845 SV training accuracy SV testing accuracy LR training accuracy LR testing accuracy 0.84 0.835 prediction accuracy 0.83 0.825 0.82 0.815 0.81 0.805 0.8 2 3 4 5 6 7 numbers of experiment 8 9 10 Prediction accuracy of SV and LR model for training and testing set (Taiwan) 0.5 SD of SV parameters SD of LR parameters 0.45 0.4 0.35 0.3 SD Figure 3 1 0.25 0.2 0.15 0.1 0.05 0 0 Figure 4 5 10 15 parameters 20 25 30 SD of the coefficients for SV and LR models (Taiwan) CREDIT SCORING BASED ON A SET-VALUED IDENTIFICATION METHOD 11 Table 5 represents the most important 6 attributes among the 30 explanatory variables for the credit card holders. Noting that the bill amounts relative to the limit value of the credit card in August, 2005 (bill relamt 2) has the strongest negative influence for predicting the credit worthy or worthless lenders. It makes sense that the larger the bill amount is, the easier for the client to default. The other factors are the logarithmic of the average amount paid from April to September in 2005 (pay amt avg log), average value of the payment status (pay ave), the logarithmic of sum product for the bill statement amount and payment status in each month (pay bill sum log), bill amounts relative to the limit value of the credit card in May, 2005 (bill relamt 5), the second dummy variable for the payment status of September, 2005 (pay 1[T.2]). Table 5 First six important attributes and corresponding coefficients Variables C (threshold) bill relamt 2 pay amt avg log pay avg pay bill sum log bill relamt5 pay 1[T.2] coefficients −0.5900 −1.5788 1.0276 −0.9603 −0.9251 0.8005 −0.7421 5 Conclusion This paper studies the credit scoring classification problem based on the SV model. The relation between the observed attribute information and the default of lenders are calculated by the SV identification algorithm for the credit applicants and credit card holders data set. Noises between the attribute information and the classification of the lenders is assumed as normally distributed. Empirical results show that the SV model performs better than the LR model not only on the prediction accuracy, but also on the coefficients stability. Based on the results of the paper, we can study more meaningful problems. For example, how to use the SV identification method to solve multi-target recognition problem. Acknowledgment We thank the anonymous researcher and I-Cheng Yeh in the department of information management, Chung Hua University, Taiwan and department of civil engineering, Tamkang University, Taiwan for sharing the data in UCI machining learning repository. We are also grateful for the tutorial of Natalino Busa, the chief data officer of Teko Ventures for the data analysis and credit scoring prediction. References [1] [2] Beaver W H, Financial ratios as predictors of failure, Journal of Accounting Research, 1966, 4: 71–111. Boyes W J, Hoffman D L, and Low S A, An econometric analysis of the bank credit scoring problem, Journal of Econometrics, 1989, 40(1): 3–14. 12 [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] WANG XIMEI, et al. Mays E, Handbook of Credit Scoring, Glenlake Publishing Company, Ltd, Chicago, 2001. Abdelmoula A K, Bank credit risk analysis with k-nearest-neighbor classifier: Case of Tunisian banks, Accounting and Management Information Systems, 2015, 14(1): 79. Morgan J P, Creditmetrics-Technical Document, JP Morgan, New York, 1997. Suisse C, CreditRisk+: A credit risk management framework, Credit Suisse Financial Products, 1997, 18–53. Crosbie P and Bohn J, Modeling default risk, Technical Report, KMV, LLC, 2003. Wang X, He X, Bao Y, et al., Parameter estimates of Heston stochastic volatility model with MLE and consistent EKF algorithm, Science China Information Sciences, 2018, 61: 042202. Karaa A and Krichéne A, Credit risk assessment using support vectors machine and multilayer neural network models: A comparative study case of a Tunisian bank, Accounting and Management Information Systems, 2012, 11(4): 587–620. Louzada F, Ara A, and Fernandes G B, Classification methods applied to credit scoring: Systematic review and overall comparison, Surveys in Operations Research and Management Science, 2016, 21(2): 117–134. Xu X, Zhou C, and Wang Z, Credit scoring algorithm based on link analysis ranking with support vector machine, Expert Systems with Applications, 2009, 36(2): 2625–2632. Wang X, Bao Y, and Zhao Y, Arbitrage-free conditions for implied volatility surface by Delta, The North American Journal of Economics and Finance, https://doi.org/10.1016/j.najef.2018.08.011. Bennell J A, Crabbe D, Thomas S, et al., Modelling sovereign credit ratings: Neural networks versus ordered probit, Expert Systems with Applications, 2006, 30(3): 415–425. Nehrebecka N, Predicting the default risk of companies, comparison of credit scoring models: Logit vs support vector machines, Econometrics, 2018, 22(2): 54–73. Zhao Y, Zhang J F, and Guo J, System identification and adaptive control of set-valued systems, Journal of Systems Science and Mathematical Science, 2012, 32(10): 1257–1265. Guo J, Zhang J F, and Zhao Y, Adaptive tracking of a class of first-order systems with binaryvalued observations and fixed thresholds, Journal of Systems Science and Complexity, 2012, 25(6): 1041–1051. Bi W, Zhao Y, Liu C, et al., Set-valued analysis for genome-wide association studies of complex diseases, The 32nd Chinese Control Conference (CCC), 2013, 8262–8267. Han J, Pei J, and Kamber M, Data Mining: Concepts and Techniques, Morgan Kaufmann, San Fransisco, 2011. Henley W E and Hand D J, Statistical classification methods in consumer credit scoring: A review, Journal of the Royal Statistical Society, Series A (Statistics in Society), 1997, 160(3): 523–541. Marques A I, Garcı́a V, and Sánchez J S, A literature review on the application of evolutionary computing to credit scoring, Journal of the Operational Research Society, 2013, 64(9): 1384–1399. Hosmer Jr D W, Lemeshow S, and Sturdivant R X, Applied Logistic Regression, John Wiley & Sons, Inc., Hoboken, New Jersey, 2013. Bolton C, Logistic Regression and Its Application in Credit Scoring, University of Pretoria, Pretoria, 2009. Yeh I C and Lien C, The comparisons of data mining techniques for the predictive accuracy of probability of default of credit card clients, Expert Systems with Applications, 2009, 36(2): 2473– 2480. Roubos J A, Setnes M, and Abonyi J, Learning fuzzy classification rules from labeled data, Infor- CREDIT SCORING BASED ON A SET-VALUED IDENTIFICATION METHOD 13 mation Sciences, 2003, 150(1–2): 77–93. [25] Morales M H, Rodrı́guez J T, and Montero J, Credit rating using fuzzy algorithms, Actas de la XVI Conferencia CAEPIA, Albacete, 2015, 539–548. [26] Yazdani H and Kwasnicka H, Fuzzy classification method in credit risk, International Conference on Computational Collective Intelligence, Springer, Berlin, Heidelberg, 2012, 495–504. [27] Galindo J and Tamayo P, Credit risk assessment using statistical and machine learning: Basic methodology and risk modeling applications, Computational Economics, 2000, 15(1–2): 107–143. [28] Paolo G, Bayesian data mining, with application to benchmarking and credit scoring, Applied Stochastic Models in Business and Society, 2011, 17: 69–81. [29] Sharma D, Improving the art, craft and science of economic credit risk scorecards using random forests: Why credit scorers and economists should use random forests, Academy of Banking Studies Journal, 2012, 11(1): 93–116. [30] Pacelli V and Azzollini M, An artificial neural network approach for credit risk management, Journal of Intelligent Learning Systems and Applications, 2011, 3(2): 103. [31] Hand J and Henley W, Statistical classification methods in consumer credit scoring, Computer Journal of the Royal Statistical Society Series a Statistics in Society, 1997, 160(3): 523–541. [32] West D, Neural network credit scoring, Computer & Operations Research, 2000, 27(11): 1131– 1152. [33] Abdou H A and Pointon J, Credit scoring, statistical techniques and evaluation criteria: A review of the literature, Intelligent Systems in Accounting, Finance and Management, 2011, 18(2–3): 59–88. [34] Berry M and Linoff G, Mastering Data Mining: The Art and Science of Customer Relationship Management, John Wiley & Sons, Inc, New York, 2000. [35] Miguéis V L, Benoit D F, and Van den Poel D, Enhanced decision support in credit scoring using Bayesian binary quantile regression, Journal of the Operational Research Society, 2013, 64(9): 1374–1383. [36] Baesens B, Van Gestel T, Viaene S, et al., Benchmarking state-of-the-art classification algorithms for credit scoring, Journal of the Operational Research Society, 2003, 54(6): 627–635. [37] Bi W and Zhao Y, Iterative parameter estimate with batched binary-valued observations: Convergence with an exponential rate, The 19th World Congress of the International Federation of Automatic Control, 2014. [38] Murphy P M and Aha D W, UCI repository of machine learning databases, Department of Information and Computer Science, University of California, Irvine, CA, http://www.ics.uci.edu/mlearn/ LRepository.html, 2001. [39] Ruxton G D, The unequal variance t-test is an underused alternative to student’s t-test and the Mann-Whitney U test, Behavioral Ecology, 2006, 17(4): 688–690. [40] Everitt B S, The Analysis of Contingency Tables, Chapman and Hall/CRC, London, 1992. [41] Hardy M A, Regression with Dummy Variables, Sage, Newbury Park, California, 1993. [42] Suits D B, Dummy variables: Mechanics vs interpretation, The Review of Economics and Statistics, 1984, 177–180. [43] Wang X, Djehiche B, and Hu X, credit rating analysis based on the network of trading information, Journal of Network Theory in Finance, 2019, 5(1): 47–65. [44] St L and Wold S, Analysis of variance (ANOVA), Chemometrics and Intelligent Laboratory Systems, 1989, 6(4): 259–272.