Decision Support Systems xxx (xxxx) xxx

Contents lists available at ScienceDirect

Decision Support Systems

journal homepage: www.elsevier.com/locate/dss

On the platform but will they buy? Predicting customers’ purchase

behavior using deep learning
Neha Chaudhuri a, Gaurav Gupta b, Vallurupalli Vamsi c, Indranil Bose d, *
a
Toulouse Business School, 20 Boulevard Lascrosses, 31068 Toulouse, France
b
NEOMA Business School, 1 Rue du Maréchal Juin, 76130 Mont-Saint-Aignan, France
c
School of Management & Entrepreneurship, Shiv Nadar University, NH - 91, Tehsil Dadri, Gautam Buddha Nagar, Uttar Pradesh - 201314, India
d
NEOMA Business School, 59 rue Pierre Taittinger, Reims, 51100, France

A R T I C L E I N F O A B S T R A C T

Keywords: A thorough understanding of online customer’s purchase behavior will directly boost e-commerce business
E-commerce performance. Existing studies have overtly focused on purchase intention and used sales rank as a natural proxy,
Customer relationship which however has limited business application. Additionally, intention to purchase does not necessarily convert
Deep learning
to actual retail purchases. We aim to further our understanding of online customer’s purchase behavior for an e-
Machine learning
Online purchase behavior
commerce platform by predicting the same using deep learning techniques, on a large multidimensional data
sample of more than 50,000 unique web sessions. This study used two distinct sets of variables, i.e., platform
engagement and customer characteristics, as key predictors of online purchases by retail customers. We further
compared the predictive capability of our deep learning method with other widely used machine learning
techniques for prediction, including Decision Tree, Random Forest, Support Vector Machines, and Artificial
Neural Networks. We found that the deep learning technique outperformed the machine learning techniques
when applied to the same dataset. These analyses will help platform designers plan for more platform engage­
ments while simultaneously expanding the academic understanding of purchase prediction for online e-com­
merce platforms.

1. Introduction behavior of both existing and new customers significantly. Unlike

traditional businesses where customers visit a store with a strong
Globally online purchasing has been growing consistently in the last intention to purchase, customers of online platforms spend a lot of time
few years. The US Department of Commerce Statistics has reported that browsing, comparing products and their prices across websites, and
retail e-commerce sales have witnessed an 11% rise from 2018 to 2019 voicing their opinions on brands and their products. These interactions
while brick-and-mortar sales have seen a rise of only 2%. In 2020, the on online platforms serve as a rich source of insights for businesses as
share of retail sales through the e-commerce mode amounted to 4.28 well as platform designers to understand the customer mindset. Busi­
trillion USD which represented 18% of all such sales. It is projected to nesses consider it lucrative to understand the product purchase journey
reach 21.8% of all sales globally. The recent COVID-19 pandemic has in general and purchase behavior in particular, by identifying and un­
further improved those projections owing to the sudden movement of derstanding the role of the actors involved in this process, especially the
customers to digital sales channels. Easier delivery and return policies, customers’ traits and the online platform and how their interactions
fast and free shipping on most e-commerce websites, and the option to affect actual purchases. Additionally, e-commerce platforms like
purchase anything round the clock from the comfort of one’s home, have Amazon and Target extensively use collaborative filtering to recom­
further increased the desirability of online purchasing during this mend products and product categories based on insights from other
pandemic. This trend has necessitated the setup of online portals for customer’s purchase decision processes. The landing page is highly
businesses that were traditionally offline. While this has extended the personalized for each customer to convert casual browsing to immediate
working hours of these businesses to 24 × 7, it has changed the shopping purchase. Evidently, any advances in prediction of individual-level or

* Corresponding author.
E-mail addresses: neha1731992@gmail.com (N. Chaudhuri), gaurav.gupta@neoma-bs.fr (G. Gupta), vamsi@bitsindri.gmail.com (V. Vamsi), indranil_bose@
yahoo.com (I. Bose).

https://doi.org/10.1016/j.dss.2021.113622
Received 1 December 2020; Received in revised form 13 May 2021; Accepted 7 June 2021
Available online 15 June 2021
0167-9236/© 2021 Elsevier B.V. All rights reserved.

Please cite this article as: Neha Chaudhuri, Decision Support Systems, https://doi.org/10.1016/j.dss.2021.113622
N. Chaudhuri et al. Decision Support Systems xxx (xxxx) xxx

group-level purchase decision has huge business implications. 2. Related work

Extant research on customer’s interaction leading to purchase in
online platforms can be divided into two distinct streams. The first 2.1. Role of customer’s platform engagement in determining purchases
stream has examined customers’ browsing behavior through various
forms of their engagement with the online platforms [1,2]. Clickstream Customer’s platform engagement with the online platform marks the
data is considered an important source for understanding such in­ first step in the complex process of purchase decision-making [7,8]
teractions. Moe [2] has examined a customer’s in-store navigational involving the creation of awareness, followed by engagement leading to
behavior based on the content of pages viewed. In this paper, click­ consideration before an actual purchase decision is made. Customer’s
stream data is used to empirically categorize shopping strategies of platform engagement goes beyond satisfaction and loyalty and provides
customers into directed buying, search/ deliberation, hedonic buying, a real competitive advantage that drives success for businesses in the
and knowledge building. Moe and Fader [1] have further developed a long term [9]. From a business perspective, a successful interaction with
model to predict the conversion of store visits into purchases based on the platform is considered to be one which necessarily leads to a valid
the observed history of online sessions of customers and their purchases. purchase. Customers’ engagement with online shopping platforms is
Schlosser, White, and Lloyd [3] have argued that varying platform de­ considered to be an essential antecedent for the overall purchase pro­
signs have led to different levels of customer trust on the e-commerce cess, from the perspective of platform design, and is central to the cus­
platforms, and have ultimately impacted purchase decisions. They have tomers’ online purchase behavior [10]. To examine this complex process
investigated the impact of various design elements such as background of purchase decision making, extant literature has approached cus­
colour, fonts, zoom and product display, on users’ trust and purchase tomers’ engagement with online platforms from two different perspec­
related decisions. tives, platform engagement attributes and customer attributes.
The other stream of research has examined customers’ characteris­
tics in their interactions with the online platform. Kumar et al. [4] have 2.1.1. Platform engagement attributes
computed the customer lifetime value for each customer and have Online e-commerce platforms have designed and developed a wide
shown its importance in developing differential marketing initiatives array of artefacts like user-friendly design, swift product search, easy
targeted to each customer. Another study has investigated the role of checkout process, and convenient and ubiquitous access through
customer demographics such as gender, age, profession, and other fac­ different web and mobile platforms [11], to stimulate a positive pur­
tors related to historical purchases to predict if a customer is likely to chase decision at every instance of customer interaction. Various studies
purchase a product in a consecutive visit to the online platform [5]. have determined that the interactivity of the web interface of the
Although there have been studies in each stream, a concerted effort shopping platform plays a crucial role in affecting consumers’ online
to link and examine platform engagement activities related to the online shopping performance [12,13]. Additionally, the purchase process dif­
e-commerce platform and customer characteristics is scant. Addition­ fers online and so the platform plays a crucial role in decision making
ally, existing studies have primarily adopted customer purchase inten­ towards a purchase. In line with these findings, various scholars have
tion on e-commerce platforms as their dependent variable. Although examined platform engagement through persuasion in the online plat­
measures of purchase intention do possess predictive usefulness [6], form design [14]. For instance, interactivity or the lack of it molds
these do not always lead to actual purchases and are thus weaker proxies purchase intention through meaningful platform engagement and
for understanding purchasing behavior of customers. In this study, we further affects the general attitude towards the platform. Website
address the above gap by examining the actual purchase behavior of interactivity refers to the ease of reciprocal communication between the
customers on an online platform as a consequence of their engagement customer and the online shopping portal. It allows customers to browse
with the online platform. We predict whether a customer purchases an the platform and have meaningful interactions with shopping portal
order placed in a particular web session (a web session is a single visit to across multiple sessions. Such interactive features are useful for
the online platform) and identify factors that affect the customer’s enhancing the customer-platform relationship and reinforcing the cus­
purchase. Unlike conventional predictive studies, we believe advances tomers’ intention to return to the platform for future purchases [15]. Ye
in computing techniques like deep learning (DL) will provide useful et al. [16] have found that website interactivity positively influences
insights due to their capability to learn and improve prediction contin­ purchasing intentions even for a premium-priced product.
uously. Therefore, we answer the following research questions in this
study: 2.1.2. Customer attributes
How does customer engagement with the online e-commerce platform Although platform attributes play a vital role in customer’s
affect their actual purchases? Can DL techniques be used to predict retail sales engagement with the platform, however, demographic characteristics
in an e-commerce platform? Does DL provide better prediction than con­ like age and gender, along with other attributes like technology efficacy
ventional machine learning (ML) techniques in this context? and individual value systems, are also significant influencers of the scale
We compare the predictive powers of different techniques to predict and scope of platform engagement [17]. As a result, even with the same
actual purchases on an online shopping platform. To this end, we set of platform engagement artefacts, different customers will engage
analyze the actual purchases of online customers across more than differently. For example, significant generational differences exist in the
50,000 sessions on an online shopping platform and predict them use of the Internet [18]. Consequently, these different user categories
accurately with the help of DL and ML techniques. The results of our engage with online platforms in their own distinct ways. Personality
study have significant implications for both theory and practice. In traits have thus been considered to be very important to examine online
terms of academic contribution, we supplant the rich scholarship on customer’s platform engagement [19].
user’s online purchase behavior by providing empirical evidence about
the role of platform interactions on actual purchases. Although there 2.2. Online purchase prediction
have been attempts to explore this phenomenon [6], researchers have
examined this primarily using sales rank, which does not reflect actual Predicting customers’ decision to purchase has been considered to be
user behavior. Our consideration of actual purchase behavior makes this the holy grail of research in different management studies, including
research unique. marketing, information systems, neuroscience, and so on. While mar­
keting scholars have developed various models to determine purchase
decisions [20], neuroscience scholars have used techniques like analysis
of real-time EEG to predict purchases [21]. While each of these different
approaches has found some success, there still exists immense

2
N. Chaudhuri et al. Decision Support Systems xxx (xxxx) xxx

opportunity to further extend our knowledge in this area. complex datasets. For example, a recent study by Loureiro et al. [34] has
Existing studies have further examined website usability [22], adopted DL to forecast sales in fashion retailing. Also, Korpusik et al.
modelled the convergence of transaction convergence as task comple­ [35] have applied a feedback-based DNN model (i.e., Recurrent Neural
tion [23] or have predicted this convergence [5]. Close & Kukar-Kinney Network) to a large corpus of tweets of potential customers to predict
[24] have examined the capabilities of the online platform as a function their choice of products and final purchases.
of the overall purchase process. They have linked user motivations on In summary, a detailed review of literature in this area, as shown in
the platform with their actions with a focus on their usage of the online Table A1, highlights important research gaps. First, there is a lack of
platform features. Various studies including those from Brown et al. empirical evidence connecting platform engagement with the actual
[10], and Olbrich and Holsing [25] have analyzed customers’ online purchase decision. The focus has primarily been on purchase intention
activity to draw insights about their purchasing behavior. However, a which, as theory has shown, can be different from actual purchase
willingness to purchase does not necessarily translate to the same in an behavior. While purchase intention represents the will to purchase a
uncontrolled real-life setting [20]. Increasingly, studies have started product, purchase behavior refers to the actual purchasing process on an
adopting real world data (e.g., clickstreams or retail sales datasets) to online platform, which is the focus of our research. Second, existing
examine online purchase behavior [35,36]. studies have examined the impact of platform engagement attributes
and customer attributes on purchase intentions, but separately. There is
a lack of a concerted effort to link these two streams of research. It is
2.3. Analytical methods for prediction
necessary to examine these distinctions and their combined impact
when attempting to examine online customer engagement. Additionally,
In order to predict customers’ decision to purchase, analytical
behavioral data analysis from clickstream and retail sales has huge un­
methods such as regression and ML have been used by researchers over
tapped potential in understanding customers’ activities on online
the years. The most widely used methods include Stepwise Logistic
shopping platforms and the impact that these activities have on the
Regression (SLR) [26], Decision Tree (DT) [27], Random Forest (RF)
customers’ purchase behavior. In this study, we attempt to bridge these
[28], Support Vector Machines (SVM) [29], and Artificial Neural Net­
gaps in literature by examining retail sales data to draw insights about
works (ANN) [30]. DT and RF have widespread applications for pre­
purchase behavior of customers through meaningful engagement on an
diction related problems because of their ease of use and the high
e-commerce platform. Table A1 provides a summary of the extant
interpretability of their generated results. Moreover, unlike ANN, DT,
research in this area.
and RF are both capable of directly handling categorical variables
[27,28]. However, DT is less robust than RF and has been found to be
3. Data description
highly sensitive to even small variations in data [31]. Additionally, RF is
simpler to tune because it has a smaller number of hyperparameters as
The challenges associated with the application of big data analytics
compared to neural network-based models [28]. However, ANN has
to predict customer purchases stem from the lack of actual sales data. As
been found to outperform DT and RF in terms of resource utilization and
a result, past literature has widely used ‘intention to purchase’ [36] as
handling of multidimensional complex datasets [31,32]. SLR has been
well as sales rank of products [37] as proxies for actual purchases of
used in extant literature for predictions involving binary dependent
customers on e-commerce platforms. However, recent studies have
variables. However, it suffers from a major limitation that makes it unfit
shown that analyzing actual purchase data would yield more convincing
for rigorous empirical analysis. SLR adds or removes variables during
results [38]. The dataset used in this study addresses this concern. We
analysis in a specific order and studies have found that this order of
have collected anonymized web browsing data from an online e-com­
addition or removal of variables can affect the final outcome [33]. This
merce platform. This e-commerce platform is a multi-vendor general
has prompted scholars to suggest the use of SLR for exploratory research
purpose online marketplace based in Germany and Belgium. The dataset
only.
comprised historical purchase data and other variables relevant for
While all of these approaches have improved the ability to determine
addressing the research questions of this study. The data consisted of
customers’ purchases, we believe that recent advances in computing,
429,013 unique sessions. One or more products were purchased in
especially DL techniques, hold much promise, primarily due to their
290,030 sessions (67.60%) while no product was purchased in 138,083
capability to improve predictions through learning. Recently various
(32.40%) sessions. Each row of this dataset represented a single online
studies have started embracing this approach for analyzing large and

Table A1
Summary of related literature using ML and DL methods.
Topics studied Research approach Algorithm used Research contributions Research limitations

Role of customer Dataset related to frequency, time Logistic lasso regression, Innovative pairwise comparison of time No distinction between high
characteristics and lapse and values of earlier extreme learning machine lapse and value difference between two versus low involvement product
engagement with e- purchases [62] and gradient tree boosting consecutive purchases to predict future categories [62]
commerce platforms in methods [62] purchase [62]
purchase predictions Online sale data of consumer LDA [63] Inclusion of customer heterogeneity as a Limited scalability of method
goods [63] predictor [63] [63]
Clickstream dataset related to Maximum likelihood Comparison between role of focused Clickstream data did not allow
browsing of online forum by estimation followed by versus unfocused product search and in-depth analysis and
potential customers [64] binary regression [64] browsing behavior on purchase decision classification of different
[64] browsing behaviors [64]
Role of platform Naturally obtained dataset from Tobit model followed by Data linked to advertisement exposure Examination of a single product
characteristics in purchase 4000 customers over 2 years Variational Bayes ML revealed negative effects of e-mail category (i.e. clothing category)
predictions related to multiple sources of method [65] catalogues and positive effects of paid [65]
advertisement exposure [65] effects and competitor catalogues [65]
Data from 400 respondents about Ordinary Least Squares Identification of website quality factors Lack of external validity due to
website quality in an experimental (OLS) regression [66] such as sophistication, genuineness, and use of student samples [66]
setup [66] unpleasantness [66]
Dataset of tourism e-commerce Co-EM logistic regression Combined semi-supervised and multi- Lack of generalizability to other
products [67] [67] view learning procedures to exploit product domains [67]
unlabeled data [67]

3
N. Chaudhuri et al. Decision Support Systems xxx (xxxx) xxx

session of a prospective customer. The corresponding variables included Table 2

activities of prospective customers, such as browsing, clicking on Demographics of the data.
products, adding products to shopping carts, purchasing products or Demographic traits Statistic
abandoning the carts. Table 1 summarizes the platform engagement and
Gender Female 73.35%
purchasing behavior related variables for customers on the e-commerce Male 26.65%
platform as observed from the dataset. Mean 44.93
Age
Table 2 summarizes the demographics of customers who are repre­ Std. dev. 11.93
sented in the dataset. Table 3 contains a brief description of variables Mean 44.51
Age (Female)
Std. dev. 11.68
present in the dataset used for this study. The dataset has 18 variables Mean 46.09
that can be broadly classified into two sets of attributes: the first set Age (Male)
Std. dev. 12.52
includes the platform engagement attributes, and the second set includes
the customers’ attributes. The first set of attributes represents variables
that are primarily related to a single online session on the portal,
Table 3
whereas the second set of attributes represents variables about the Descriptive statistics of variables used in this research.
prospective customers. Variables about customers include de­
Variables Minimum Maximum Mean Std.
mographics, historical purchase patterns, and user profile data for the e-
Dev.
commerce platform. Table 3 also provides the descriptive statistics for
Platform engagement attributes
this dataset.
Time of session 1.23 23 14.48 4.37
The platform engagement related variables included data about the Day of week of session 1 7 – –
web sessions of customers (such as time and day of week of session, Duration of session (minutes) 0 8736.92 1415.71 1711.91
duration of a session, and number of times the customer logged in) as Number of log-ins of customer 1 6 2.37 0.81
well as click-stream data related to exploratory search behavior of the Number of products clicked 1 114.97 21.52 24.73
Lowest price of product
customer (such as price and number of products clicked, cart size, and 0 429.21 32.33 71.41
clicked
value). Highest price of product
0 882.78 96.13 149.85
clicked
4. Methodology Sum of prices of all products
0 8776.98 704.37 1122.17
clicked
Number of products in cart 0 15.67 3.65 3.25
This study was aimed at identifying significant platform engagement Lowest price of product in cart 0 516.32 40.43 86.31
related and customer related predictors of online purchases, while also Highest price of product in
0 643.37 68.17 108.73
focusing on examining the potential of DL approach in this context. This cart
Sum of prices of all products in
approach is relevant for this study as the DL technique uses its dense 0 1144.08 130.10 168.21
cart
network structure to recognize complex patterns in datasets. This study Customers’ attributes
has used different analytical methods, including some traditional ML Customer account score
89.53 638 486.38 126.18
techniques and the more advanced DL technique to examine the assigned by online retailer
research question. The adopted methodology is shown in Fig. 1 below. Customer account lifetime
0 602 135.56 108.32
(days)
Number of payments made by
0 93.12 11.36 14.44
customer
4.1. Pre-process
Age of customer 17 81.17 45.07 11.98
Gender of customer 1 2 – –
We adopted a four-stage approach (as shown in the data pre-process Days elapsed since last
3 738 79.88 97.61
block of Fig. 1) to prepare the dataset in order to improve the predictive purchase
accuracy and computational efficiency. Apart from the platform
engagement related variables as described in Table 2, the dataset also
included records of availability status of products that customers were missing values as a dependent variable and the remaining independent
interested in purchasing and records related to the stage of completion variables as predictors. The steps followed in this model-based missing
of the multi-stage purchase process. However, we discarded these var­ value imputation was similar to the one followed by Zolbanin et al. [41].
iables from the dataset as they suffered from scant variability of data The remaining variables were first ranked by their ability to predict the
values and also more than 45% of the data values were missing. This is in dependent variable with missing values. The variables which only
line with extant research which suggest that the presence of missing marginally contributed to the estimation were subsequently removed
values would reduce the predictive power of a technique [39]. We used from consideration. Finally, the best predictor from the model was used
the filtered dataset for subsequent analysis. to replace the missing values of the dependent variable. For the records
The second stage included the imputation of any remaining missing in the dataset where the best predictor also had missing values, the next
values in the variables. For this, we employed a DT. We chose this best predictor with no missing value was used. This whole process was
method over using the mean or median of the variable because a DT repeated for all variables with some missing values in the dataset.
results in more accurate estimation by using the information of the Following this, numerical values of the variables that were more than
remaining variables [40]. Moreover, the DT is also capable of handling five standard deviations away from the mean were defined as extremes,
categorical variables in the dataset [27]. We treated the variable with while those values that were three standard deviations but within five
standard deviations away from the mean were defined as outliers. Re­
Table 1 cords containing more than one outlier were eliminated whereas for the
Platform engagement and purchasing behavior of customers for the online remaining records, outlier variables were coerced to a value equivalent
portal. to mean ± three times its standard deviation.
Description Frequency
The widely used DL and ML techniques are incapable of handling
categorical inputs efficiently. This is true for specifically NN models as
Number of prospective unique customers 105,038
well as SVM. Hence, we applied one hot encoding to transform cate­
Number of sessions 429,013
Number of sessions with a purchase 290,030 gorical variables into numerical variables while successfully averting
Number of sessions without a purchase 138,983 wrongful interpretation of hierarchy in data values by the models [42].

4
N. Chaudhuri et al. Decision Support Systems xxx (xxxx) xxx

Fig. 1. The process diagram of the study.

For every n-level categorical variable, n variables were generated. After layer as well as multiple sparsely connected intermediate layers. The
this preprocessing, the dataset had 25 numerical variables that acted as greater depth of its structure, as compared to the ANN, allows it to learn
predictors of customers’ purchases. multiple levels of representations in a dataset with increasing
complexity. This enhanced representation learning leads to a better
4.2. Data analysis predictive performance of the DNN. DL has been used for various
research applications, including image classification, emotion detection,
The DL and ML techniques required initialization of hyper­ sales prediction, and has been applied to a variety of data types. As a
parameters during model building. These hyperparameters determined result, different types of DL architectures have been developed for
the training capability of the techniques [43] and needed to be tuned different use cases. For example, convolutional neural network (CNN)
during the learning process. Moreover, the traditional approach of using has been used for image processing and is well-suited for multi-
a training dataset to train a ML technique and then using a testing dimensional records. The recurrent neural network (RNN) works well
dataset to evaluate the performance of the technique often suffers from with sequential data with a temporal dynamic behavior.
overfitting and lack of robustness [44]. To overcome this problem, we For this study, we used the feed-forward DNN. We chose this DL
employed k-fold cross validation with k = 5 for each run. This procedure technique because the dataset used in this study consisted of one-
involved partitioning of the training dataset into five folds. For every dimensional inputs only. Our choice to apply the DNN in this context
iteration of the procedure, one fold was treated as a validation set while was further affirmed due to the data-driven self-adaptive approach of
the remaining four folds acted as training sets. In each iteration, we the DNN that was in contrast to the traditional methods that required
performed grid-search based hyperparameter tuning until the training specific assumptions about the functional form of the data. The network
and validation errors stabilized. This helped to overcome the more structure of the DNN is defined in a way such that the initial layers learn
subtle ‘hyperparameter overfitting’ [45]. Further, following Zolbanin et al. the simpler data features, while the deeper layers handle the more
[41], we calculated the accuracy-based importance score of variables for complex features [47]; thus, enabling it to capture nonlinear relation­
each run. This importance score compared the predictive ability of the ships in the dataset. Moreover, prior studies have shown that the DNN is
independent variable based on how much it contributed to the overall less vulnerable to the curse of dimensionality, as compared to the ML
accuracy of a given model. To determine the importance of a variable, regression-based models. This makes the DNN suitable for this study
we started with the full combination of variables and kept dropping because it is expected to handle the multiple multi-class categorical as
variables one by one, and trained the technique with the remaining well as numerical variables of the dataset efficiently.
variables, and calculated the accuracy of prediction. We observed that We used the stochastic gradient descent (SGD) method to train the
the absence of a relatively important variable resulted in a significant DNN and initialized it with three layers and an equal number of neurons
drop in the accuracy of the model. Finally, following Shen et al. [46], we in each layer. SGD is the most preferred cost-effective optimization al­
used sensitivity analysis-based feature selection to extract the optimal gorithm to train neural networks as it ensures randomness by intro­
subset of variables that most accurately predicted purchases. We also ducing a single training sample at each iteration [61]. We investigated
compared the training and validation errors for the predictive models different configurations of the network structure, including three, five,
and found that the validation error was within 0.048% of the training and seven layers with 64,128 and 256 neurons in each layer. The results
error. This confirmed insignificant overfitting. are reported in the following section. Finally, since the DNN is vulner­
able to overfitting, we used an early stopping and a dropout layer to
address this problem, as suggested by Srivastava et al. [48].
4.3. Deep learning (DL)
We used Keras 2.3.0, an open source Python library running on Py­
thon 3, to develop the DL and ML techniques. Keras is a programming
The DNN is a DL technique that is an evolved variant of ANN. Unlike
interface which works on top of TensorFlow version 2.1.0. TensorFlow is
the network structure of a traditional ANN, a DNN has a fully connected

5
N. Chaudhuri et al. Decision Support Systems xxx (xxxx) xxx

an open source software library developed by Google for ML. Table 5

Performance metrics of the predictive techniques.
4.4. Performance metrics Metrics ML techniques DL technique

DT RF SVM ANN DNN

We calculated nine evaluation metrics to compare the performance
Accuracy 0.83 0.81 0.83 0.84 0.89
of the predictive models. These metrics are Accuracy, Recall, F1-score, PPV 0.81 0.82 0.84 0.83 0.87
Positive Predictive Value (PPV), Negative Predictive Value (NPV), Recall 0.95 0.90 0.91 0.94 0.96
False Positive Rate (FPR), ROC-AUC, and Matthews Correlation Coeffi­ NPV 0.87 0.79 0.82 0.87 0.92
cient (MCC) [49,50]. PPV and NPV are proportions of positive and FPR 0.38 0.34 0.31 0.34 0.26
F1-score 0.88 0.86 0.87 0.88 0.92
negative results that are true positives and true negatives, respectively.
ROC-AUC 0.85 0.84 0.84 0.85 0.89
The value of MCC range between [− 1,1] and the metric produces a good MCC 0.63 0.59 0.63 0.65 0.75
score only if a model has exhibited satisfactory performance for all four Std. dev. of accuracy 0.0241 0.1015 0.0277 0.0209 0.0053
confusion matrix categories (i.e., true positives, true negatives, false
positives, and false negatives) [50]. Unlike the scalar metrics like ac­
curacy, recall, and F1-score, MCC is symmetric and not sensitive to class Table A2
imbalance [51]. It represents the correlation coefficients between the Hyperparameter tuning for the DNN in this study.
true and predicted values and works better than accuracy and F1-score
Network hyperparameters Search space Selected values
for prediction of binary classes. The final reported values for the per­
formance metrics are the mean of the values that are obtained from the Input units Fixed 25
Learning rate [10–6,10–1] 10–3
five folds of the cross-validation process. Batch size [10,1000] 100
Number of hidden layers 3,5,7 5
5. Results Number of neurons per hidden layer 64,128,256 128
Dropout [0,0.33] 0.1
Batch normalization Yes
We initialized the DNN model with three layers and with an equal
–
Activation function in hidden layer ReLU, tanh ReLU1
number of neurons in each layer. Following Loureiro et al. [34], we Loss function – Cross-entropy
investigated different configurations of the network structure, including Activation function in output layer – Softmax
three, five, and seven layers with 64,128 and 256 neurons in each layer. 1
ReLU activation function: With the addition of extra hidden layers to the
The 5-layer network with 128 neurons in each layer exhibited the best DNN, the loss function starts approaching zero. This is particularly true for tanh
performance. The performance of the nine configurations are shown in and sigmoid functions. This is termed as the vanishing gradient problem. This
Table 4. The investigation revealed that an increase in network depth problem becomes more prominent for the DNN with an increasing number of
and width upto 5 layers and 128 neurons, steadily increased the pre­ layers, making it hard to train. The use of the ReLU function and batch
dictive ability of the model. However, we observed a deterioration in the normalization can solve this problem for the DNN.
overall performance with a further increase in the network elements
beyond these values. Existing literature asserts that additional network made for more than 80% of the web sessions.
elements beyond the optimal number leads to overfitting of the model The results indicate that the DNN was able to outperform the others
for the training dataset and results in poorer performance [41,52]. on all explored metrics, followed closely by DT and ANN. To verify that
Hence, we discarded denser network architectures. the models indeed performed differently, we conducted the widely used
Cochran’s Q test for statistical comparison of performances. We found
5.1. 5.3 Comparison of performances of the predictive techniques that the Q value (approximating χ 2) was 12.794 which corresponded to
a p-value 0.012327 (i.e., p < 0.05) and implied that the predictive
We have summarized the performances of the DL (DNN configura­ models did not perform equally well. Additionally, the McNemar’s tests
tion with 5 layers and 128 neurons per layer) and ML techniques in for pairwise comparison revealed that the DNN was indeed the best
Table 5. All models used in the study were tuned to optimize the performing model as compared to the other ML models (p < 0.001). This
hyperparameters. The value of the hyperparameters for the DNN is significant improvement in the predictive performance of the DNN can
shown in Table A2. Due to limitations in space, the performance sta­ be attributed to its distinct network elements (e.g., weight functions,
tistics of only the models with the best performing hyperparameters number of layers in the network, and the number of neurons in each
have been reported in Table 5. Similar values for performance metrics layer). The multiple network layers of the DNN enhanced the process of
were obtained when the ML and DL techniques were used for the vali­ feature learning [53]. The initial layers are meant for learning simpler
dation dataset. This is not reported due to paucity of space. The results features, while the later layers are responsible for predicting the
indicate that all techniques have the capability to support the decision- outcome based on more complex combinations of features produced by
making process of firms because their reported ROC-AUC values are the previous layers. On the other hand, unlike the ML techniques, the
above the standard benchmark value of 0.5. As shown in Table 5, all network structure of the DNN makes it less vulnerable to the curse of
techniques are able to accurately predict whether a purchase has been dimensionality [54]. Therefore, the deep network structure of the DNN
empowers it to handle the large dataset used in this study.
Table 4
Performances of different configurations of the DNN. 6. Discussion
Number of layers Neurons per layer Accuracy F1-score MCC
6.1. Relative importance of predictor variables
3 64 0.83 0.88 0.63
128 0.85 0.89 0.68
256 0.87 0.91 0.72
To gain deeper insights about the individual predictive capabilities
5 64 0.88 0.91 0.74 of variables, we calculated and ranked the accuracy-based relative
128 0.89 0.92 0.75 variable importance [55]. Variable importance is measured by the
256 0.87 0.91 0.72 decrease in the overall predictive accuracy when a predictor variable is
7 64 0.86 0.90 0.70
dropped from the model. The impact on the overall accuracy for a
128 0.85 0.89 0.68
256 0.84 0.88 0.64 predictor variable was calculated by dropping only one variable at a

6
N. Chaudhuri et al. Decision Support Systems xxx (xxxx) xxx

time from the complete DNN model and then re-analyzing the data. The successful orders, the duration of their association with the platform
impact (i.e., reduction in accuracy) was greater for a variable which had impacted their propensity to purchase.
a higher contribution to accurately predict purchase in a web session While we are not suggesting the lack of importance of repeat cus­
and this translated into a higher importance score for the variable. We tomers for online purchases, we want to highlight the importance of
then normalized the scores in the form of ‘relative importance’ of a loyal customers for all types of purchases. Such customers experienced
variable with respect to the whole set of variables available for analysis. greater familiarity and comfort with the online platform and returned to
The value of relative importance ranges from 0 to 1, where 1 represents the platform repeatedly for additional purchases. The inclusion of these
the variable with the highest contribution to the accuracy of the model variables were a strong indicator of the need for ease of navigation as
and 0 represents the variable with the lowest contribution. Table 6 well as the customers’ familiarity and trust on the platform.
shows the relative importance of the top twelve variables for the DNN The variables sum of prices of all products clicked on, and lowest price of
model with 5 layers and 128 neurons per layer. However, each of the product added to cart, were also identified to be among the important
five runs of the DNN (corresponding to the five-fold cross-validation) predictors. They indicated the existence of a relationship between
accepted different combinations of predictor variables in order to ach­ platform engagement and purchase behavior. The first variable indi­
ieve the best performance. This difference can be attributed to the cated a casual platform engagement session involving prospection while
varying specificities of each model [34]. Therefore, the results in Table 6 the second variable indicated consideration to add some products to the
denote the average impact of each predictor variable on the accuracy of cart and preparation for successful checkout. The variable lowest price of
the model. Also, the relative importance indices across the other four ML product added to cart was found to be a better predictor as compared to
techniques exhibited similar trends. the variable lowest price of product clicked on. This result suggested that
Of the top twelve predictors, nine belonged to the platform spending time in browsing for products did not necessarily convert to
engagement category. This result suggests that platform engagement actual purchases. Browsing was a phase where a potential customer
with the online e-commerce platform heavily impacts their purchase often explored and compared products and their features and this did
decisions. Specifically, the variables time when the session began and day not necessarily indicate an intent to purchase.
of the week when the session began were highly predictive. These results Finally, customer score indicated the effectiveness of the retailer’s
suggested that time had a significant impact on customer purchases on process for assigning this score to each customer. However, since this
an online platform. At specific times of the day and on specific days, the variable was proprietary and highly dependent on the product and other
customers had a higher propensity to purchase. In other words, there related characteristics, it was unlikely to exhibit importance for pre­
existed peak shopping times similar to offline stores, even for online diction of purchases on other online retail platforms.
shopping platforms. Another interesting observation from the results shown in Table 5
Customer account lifetime and days elapsed since last purchase were also was that the duration of a session and the number of times the user logged in
found to be universally important for all techniques. These variables on an online e-commerce platform did not significantly impact pur­
captured the sensitivity of the type of customer’s association with the e- chase. This indicated that engagement did not necessarily represent
commerce platform and established the conventionally accepted belief intention to purchase. This observation reinforced the need to design
that loyal customers had a higher propensity to purchase than new better recommendations and enhance platform engagement to help
customers. Interestingly, although customer account lifetime was found to customers make quicker purchase decisions while limiting online re­
be a good predictor of actual sales, the number of payments made by the sources devoted to each customer. Online e-commerce platforms needed
customer did not significantly contribute to purchase decision. This to address the balance between higher customer loyalty through better
suggested that even though the customer might not have multiple (often longer) engagement and streamlined design to support quicker
purchases (also implying lower platform engagement).

Table 6
6.2. Sensitivity analysis-based feature selection
Relative importance of variables.
Variable category Variables DNN Feature selection is a core component of any machine learning
Impact on Relative application. Discarding redundant variables improves the predictive
accuracy (%) importance accuracy of ML and DL models, speeds up the training process, and re­
Day of the week when duces the overall cost of computation [34,56]. The two frequently used
0.92 0.44
session began feature selection methods include filter-based and wrapper-based
Time when session methods. Filter-based methods (e.g., correlation coefficient) depend
0.42 0.20
began
Sum of prices of all
on properties of data and are often carried out as part of the data pre-
0.25 0.12 processing stage. These methods suffer from instability due to inde­
products added to cart
Sum of prices of all pendence from underlying predictive models [57]. The wrapper-based
0.11 0.05
products clicked on methods have been found to perform better than filter-based methods
Platform
Lowest price of product
engagement 0.10 0.05 because they use the knowledge of the underlying learning algorithms
added to cart
attributes
Number of products [58]. Therefore, we adopted a wrapper-based sensitivity analysis-based
0.09 0.04
added to cart feature selection method that has been used in previous studies by Zhang
Highest price of product
0.07 0.04
[58] and Shen et al. [46], to generate an optimal subset of features
added to cart which can most accurately predict purchases.
Number of products
clicked on
0.07 0.03 For this, we re-trained the DL and ML techniques with the features
Highest price of product shown in Table 6, in a descending order. This meant that the first input
0.06 0.03
clicked on variable was the one with the highest relative importance and so on. The
Customer account
2.10 1.00 changes in average accuracies for the ML and Dl models over five runs
lifetime
(corresponding to the five-fold cross-validation process) are shown in
Days elapsed since last
Customers’
purchase
1.48 0.71 Fig. 2.
attributes Fig. 2 indicates that the accuracy of all models peaked when trained
Customer account score
assigned by the online 0.15 0.07 with the top twelve features and decreased thereafter. This could be
retailer attributed to the curse of dimensionality which meant that adding new
features during the training of a model, beyond an optimal number,

7
N. Chaudhuri et al. Decision Support Systems xxx (xxxx) xxx

Fig. 2. The change in average accuracies of ML and DL techniques with feature selection.

could lead to degradation of its predictive performance [59]. However, [37] have limited business application, and hence it is prudent to focus
unlike the ML techniques which exhibited sharper reduction in their on retail sales data to develop insights about purchase behavior. We use
accuracies after the optimal point, the relative degradation in accuracy actual purchase data from a platform and show that it is possible to
for the DNN was lower. For all techniques, the sharp decline in accuracy predict purchase with a high accuracy using DNN.
could be attributed to overfitting and the noise introduced due to the Secondly, this study compares the usage of multiple advanced
addition of redundant variables. The DNN was able to handle the noise analytical techniques for prediction of purchases on an online platform.
better which made it a better choice as a predictive technique. Previous studies have asserted the importance of advanced analytical
techniques for predictive purposes [38]. We adopted a comparative
approach to examine the efficacy of DL and ML for the dataset. This
6.3. DL based prediction of purchase behavior
examination further supports other studies that have demonstrated the
higher predictive power of DL for large datasets [65]. As a result, it
Comparing the results of our analysis, we found strong evidence of
provides further empirical evidence to support the superior predictive
better performance of DL techniques over conventional ML techniques.
capabilities of DL in such a context.
The analysis using the DNN improved the accuracy over the widely used
Finally, DL has been used in domains like healthcare [41], and sales
RF model by over 6% and ROC-AUC by around 4%. While these im­
prediction [34] but not in the area of online retailing. Our research has
provements might not be significantly large, but its implication for
shown that the DNN was the best performing analytical technique that
businesses is enormous, given that it would directly translate into
offered high predictive power for purchases on an online shopping
improved purchase prediction. Additionally, we found that the FPR
platform. The popular DT technique was found to be close to it in terms
decreased by close to 10% when comparing the DT technique with the
of accuracy. Future research can use these insights when using multiple
DNN. The corresponding decrease between the best ML technique (SVM)
techniques for research in this and other similar domains.
with DT was 7%. Improvement in false positives is very important for
Apart from these contributions, this study advances the extant debate
online businesses in this context as it would help them to improve their
on improving prediction for e-commerce sales. For example, some
understanding of the underlying factors affecting the purchase decision.
interesting results from this study like the significance of user account
Hence, DNN could be used as a potent tool for businesses to improve
age, rather than the number of past transactions, on consumers’ pur­
their bottom line through better prediction.
chase opens up new debates on customer loyalty and platform engage­
However, DNN is more resource-intensive than conventional ML
ment. This needs to be responded through intensive studies in the future
techniques. A slew of methods have been proposed, for example,
about the impact of customer loyalty (specifically relating to past pur­
network pruning and deep compression [60], to reduce the resource
chases) on future purchases.
overhead without comprising on their performance. These new tech­
niques involve encoding and removal of less important network weights
to generate faster and smaller NN. The use of these techniques makes the 7.2. Managerial implications
DNN suitable for a wide range of practical applications.
Based on the identification of significant predictors of online pur­
7. Implications chases, we observe that platform designers should choose to design the
online platform for quicker purchase when the competition from other
7.1. Academic implications channels and competitive options is high. In such cases, platform arte­
facts supporting higher engagement may not be a smart choice as it they
There are three major academic implications of this study. First, this not lead to a positive purchase decision. Our results also indicate that
study bridges the acknowledged need to predict actual purchase instead account age impacts the purchase decision. Hence, if the competition is
of purchase intentions [38]. As mentioned earlier, proxies like sales rank not very high, awareness of the product is low, and the cost of

8
N. Chaudhuri et al. Decision Support Systems xxx (xxxx) xxx

engagement is not very high, it will be beneficial for businesses to [2] W.W. Moe, Buying, searching, or browsing: differentiating between online
shoppers using in-store navigational clickstream, J. Consum. Psychol. 13 (2003)
develop long term relationship with all users on the platform.
29–39, https://doi.org/10.1207/S15327663JCP13-1&2_03.
Finally, our use of customers’ historical data to improve the pre­ [3] A.E. Schlosser, T.B. White, S.M. Lloyd, Converting web site visitors into buyers:
diction of purchases on online e-commerce platforms, also accentuates how web site investment increases consumer trusting beliefs and online purchase
the need to invest in appropriate infrastructure to control the quality and intentions, J. Mark. 70 (2006) 133–148, https://doi.org/10.1509/jmkg.70.2.133.
[4] V. Kumar, G. Ramani, T. Bohling, Customer lifetime value approaches and best
veracity of sales data. Currently, businesses capture data for a large practice applications, J. Interact. Mark. 18 (2004) 60–72, https://doi.org/
number of variables and this often raises various privacy concerns. Also, 10.1002/dir.20014.
such unorganized data collection strains the organisation’s computa­ [5] D. Van Den Poel, W. Buckinx, Predicting online-purchasing behaviour, Eur. J.
Oper. Res. 166 (2005) 557–575, https://doi.org/10.1016/j.ejor.2004.04.022.
tional resources. Businesses can use the results from this study to [6] L.F. Jamieson, F.M. Bass, Adjusting stated intention measures to predict trial
streamline their data collection practices and focus only on collecting purchase of new products: a comparison of models and methods, J. Mark. Res. 26
those data items that can directly predict and influence the customers’ (1989) 336, https://doi.org/10.2307/3172905.
[7] T.R. Rao, Consumer’s purchase decision process: stochastic models, J. Mark. Res. 6
purchases. (1969) 321, https://doi.org/10.2307/3150138.
[8] S. Karimi, K.N. Papamichail, C.P. Holland, The effect of prior knowledge and
8. Conclusion decision-making style on the online purchase decision-making process: a typology
of consumer shopping behaviour, Decis. Support. Syst. 77 (2015) 137–147, https://
doi.org/10.1016/j.dss.2015.06.004.
This study examined predicted the actual purchase behavior of cus­ [9] C.K. Prahalad, V. Ramaswamy, Co-creation experiences: the next practice in value
tomers on an online platform as a consequence of their interactions with creation, J. Interact. Mark. 18 (2004) 5–14, https://doi.org/10.1002/dir.20015.
[10] M. Brown, N. Pope, K. Voges, Buying or browsing? An exploration of shopping
the online platform. For this, it used a unique anonymized web browsing
orientations and online purchase intention, Eur. J. Mark. 37 (2003) 1666–1684,
dataset comprised of historical purchase data along with data related to https://doi.org/10.1108/03090560310495401.
customer engagement with the online e-commerce platform. The study [11] L. Muzellec, E. O’Raghallaigh, Mobile technology and its impact on the consumer
identified two distinct sets of variables, i.e., platform engagement and decision-making journey how brands can capture the mobile-driven “ubiquitous”
moment of truth, J. Advert. Res. 58 (2018) 12–15, https://doi.org/10.2501/JAR-
customer characteristics, as key predictors of purchases. Out of these 2017-058.
two categories, we identified four variables, namely the time and day of [12] J.W. Palmer, Web site usability, design, and performance metrics, Inf. Syst. Res. 13
the week when a session began, duration of a customer’s association (2002) 151–167, https://doi.org/10.1287/isre.13.2.151.88.
[13] H.-H. Teo, L.-B. Oh, C. Liu, K.-K. Wei, An empirical study of the effects of
with the platform, and days elapsed since last purchase, as the most interactivity on web user attitude, International Journal of Human-Computer
significant contributors to accurate prediction of a purchase decision. Studies. 58 (2003) 281–305, https://doi.org/10.1016/S1071-5819(03)00008-9.
Additionally, the results found that the DNN outperformed other ML [14] M.M. Alhammad, S.R. Gulliver, Persuasive technology and users acceptance of E-
commerce: users perceptions of website persuasiveness, J. Electron. Commer.
techniques when applied on the same dataset. Retailers and e-commerce Organ. 12 (2014) 1–13, https://doi.org/10.4018/jeco.2014040101.
platform designers can use these findings and integrate them into their [15] H.A.M. Voorveld, G. Van Noort, M. Duijn, Building brands with interactivity: the
existing recommendation engines to improve their predictability. One role of prior brand usage in the relation between perceived website interactivity
and brand responses, J. Brand Manag. 20 (2013) 608–622, https://doi.org/
way to use these findings in such retail IT systems would be to assign 10.1057/bm.2013.3.
greater weightage to these variables in the existing recommendation [16] B.H. Ye, A.A. Barreda, F. Okumus, K. Nusair, Website interactivity and brand
engines. These, often ignored, platform engagement and customer development of online travel agencies in China: the moderating role of age, J. Bus.
Res. 99 (2019) 382–389, https://doi.org/10.1016/j.jbusres.2017.09.046.
characteristic variables should improve the accuracy of predictions
[17] J. Marbach, C.R. Lages, D. Nunan, Who are you and what do you value?
without major disruption to their ongoing operations. These findings Investigating the role of personality traits and customer-perceived value in online
will help improve purchase prediction using DNN on similar platforms. customer engagement, J. Mark. Manag. 32 (2016) 502–525, https://doi.org/
Despite these insights, this study has some limitations. First, this 10.1080/0267257X.2015.1128472.
[18] J.S. Stewart, E.G. Oliver, K.S. Cravens, S. Oishi, Managing millennials: embracing
study is situated in the context of retail sales in a particular e-commerce generational differences, Business Horizons. 60 (2017) 45–54, https://doi.org/
platform. While the dataset is sufficiently large to allow usage of ML and 10.1016/j.bushor.2016.08.011.
DL techniques for predicting purchase behavior, a much larger dataset [19] L. Aksoy, A. van Riel, J. Kandampully, J. Wirtz, A. Den Ambtman, J. Bloemer,
C. Horváth, B. Ramaseshan, J. Van de Klundert, Z. Gurhan Canli, Managing brands
might require some tweaking of the model to further improve its pre­ and customer engagement in online brand communities, J. Serv. Manag. 24 (2013)
dictive capabilities. Second, the data represents the customer behavior 223–244, https://doi.org/10.1108/09564231311326978.
in a European online shopping context in a particular product space. [20] S. Kagan, R. Bekkerman, Predicting purchase behavior of website audiences, Int. J.
Electron. Commer. 22 (2018) 510–539, https://doi.org/10.1080/
Although representative of the context, the findings might not be 10864415.2018.1485084.
generalizable across different customer demographics and product [21] N. Ravaja, O. Somervuori, M. Salminen, Predicting purchase decision: the role of
types. Further studies across different purchase contexts would be hemispheric asymmetry over the frontal cortex, journal of neuroscience,
psychology, and, Economics. 6 (2013) 1–13, https://doi.org/10.1037/a0029949.
needed to improve the generalizability of the findings. Additionally, [22] V. Venkatesh, R. Agarwal, Turning visitors into customers: a usability-centric
such future studies could allow us to analyze the findings and improve perspective on purchase behavior in electronic channels, Manag. Sci. 52 (2006)
them by statistically comparing the performance of various predictive 367–382, https://doi.org/10.1287/mnsc.1050.0442.
[23] C. Sismeiro, R.E. Bucklin, Modeling purchase behavior at an e-commerce web site:
models. Third, this study was not able to compare the predictive accu­
a task-completion approach, J. Mark. Res. 41 (2004) 306–323, https://doi.org/
racy of the model in real-time when the data was generated. This would 10.1509/jmkr.41.3.306.35985.
have allowed us to provide just-in-time recommendations to improve [24] A.G. Close, M. Kukar-Kinney, Beyond buying: motivations behind consumers’
retail sales and would have far superior practical implications. We hope online shopping cart use, J. Bus. Res. 63 (2010) 986–992, https://doi.org/
10.1016/j.jbusres.2009.01.022.
that future studies can adopt such combined approaches that generate [25] R. Olbrich, C. Holsing, Modeling consumer purchasing behavior in social shopping
real-time recommendations that would have strong business implica­ communities with clickstream data, Int. J. Electron. Commer. 16 (2011) 15–40,
tions. Finally, future research can also explore the development of a https://doi.org/10.2753/JEC1086-4415160202.
[26] S.L. Gortmaker, D.W. Hosmer, S. Lemeshow, Applied logistic regression, Contemp.
deep learning-based rule extraction method with applications in related Sociol. 23 (1994) 159, https://doi.org/10.2307/2074954.
situations and compare its performance with the existing benchmarks. [27] J.R. Quinlan, Induction of decision trees, Mach. Learn. 1 (1986) 81–106, https://
Author Statement. doi.org/10.1023/A:1022643204877.
[28] L. Breiman, Random forests, Mach. Learn. 45 (2001) 5–32, https://doi.org/
The authors do not wish to include any author contribution state­ 10.1017/CBO9781107415324.004.
ment for the paper. [29] H. Drucker, C.J.C. Surges, L. Kaufman, A. Smola, V. Vapnik, Support vector
regression machines, in: Advances in Neural Information Processing Systems,
1997: pp. 155–161.
References [30] W.S. McCulloch, W. Pitts, A logical calculus of the ideas immanent in nervous
activity, The Bulletin of Mathematical Biophysics. 5 (1943) 115–133, https://doi.
[1] W.W. Moe, P.S. Fader, Dynamic conversion behavior at e-commerce sites, Manag. org/10.1007/BF02478259.
Sci. 50 (2004) 326–335, https://doi.org/10.1287/mnsc.1040.0153.

9
N. Chaudhuri et al. Decision Support Systems xxx (xxxx) xxx

[31] S. Lessmann, B. Baesens, H.V. Seow, L.C. Thomas, Benchmarking state-of-the-art [50] S. Boughorbel, F. Jarray, M. El-Anbari, Optimal classifier for imbalanced data using
classification algorithms for credit scoring: an update of research, Eur. J. Oper. Res. Matthews Correlation Coefficient metric, PLoS ONE. 12 (2017). doi:https://doi.org
247 (2015) 124–136, https://doi.org/10.1016/j.ejor.2015.05.030. /10.1371/journal.pone.0177678.
[32] M. Chau, H. Chen, A machine learning approach to web page filtering using [51] D. Chicco, G. Jurman, The advantages of the Matthews correlation coefficient
content and structure analysis, Decis. Support. Syst. 44 (2008) 482–494, https:// (MCC) over F1 score and accuracy in binary classification evaluation, BMC
doi.org/10.1016/j.dss.2007.06.002. Genomics 21 (2020) 6, https://doi.org/10.1186/s12864-019-6413-7.
[33] S. Dreiseitl, L. Ohno-Machado, Logistic regression and artificial neural network [52] H. Larochelle, Y. Bengio, J. Louradour, P. Lamblin, Exploring strategies for training
classification models: a methodology review, J. Biomed. Inform. 35 (2002) deep neural networks, J. Mach. Learn. Res. 10 (2009) 1–40, https://doi.org/
352–359, https://doi.org/10.1016/S1532-0464(03)00034-0. 10.1145/1577069.1577070.
[34] A.L.D. Loureiro, V.L. Miguéis, L.F.M. da Silva, Exploring the use of deep neural [53] Y. Lecun, Y. Bengio, G. Hinton, Deep learning, Nature. 521 (2015) 436–444,
networks for sales forecasting in fashion retail, Decis. Support. Syst. 114 (2018) https://doi.org/10.1038/nature14539.
81–93, https://doi.org/10.1016/j.dss.2018.08.010. [54] B. Kim, J. Park, J. Suh, Transparency and accountability in AI decision support:
[35] M. Korpusik, S. Sakaki, F. Chen, Y.Y. Chen, Recurrent neural networks for customer Explaining and visualizing convolutional neural networks for text information,
purchase prediction on Twitter, in: CEUR Workshop Proceedings, 2016: pp. 47–50. Decision Support Systems. 134 (2020). doi:https://doi.org/10.1016/j.dss.2020.11
[36] M. Mousavizadeh, D.J. Kim, R. Chen, Effects of assurance mechanisms and 3302.
consumer concerns on online purchase decisions: an empirical study, Decis. [55] C. Strobl, A.L. Boulesteix, T. Kneib, T. Augustin, A. Zeileis, Conditional variable
Support. Syst. 92 (2016) 79–90, https://doi.org/10.1016/j.dss.2016.09.011. importance for random forests, BMC Bioinformatics. 9 (2008) 307, https://doi.org/
[37] C. Koçaş, C. Akkan, A system for pricing the sales distribution from blockbusters to 10.1186/1471-2105-9-307.
the long tail, Decis. Support. Syst. 89 (2016) 56–65, https://doi.org/10.1016/j. [56] I. Iguyon, A. Elisseeff, An introduction to variable and feature selection, J. Mach.
dss.2016.06.008. Learn. Res. 3 (2003) 1157–1182, https://doi.org/10.1162/153244303322753616.
[38] X. Hu, Q. Huang, X. Zhong, R.M. Davison, D. Zhao, The influence of peer [57] J.B. Yang, K.Q. Shen, C.J. Ong, X.P. Li, Feature selection via sensitivity analysis of
characteristics and technical features of a social shopping website on a consumer’s MLP probabilistic outputs, in: Proceedings of the 2008 IEEE International
purchase intention, Int. J. Inf. Manag. 36 (2016) 1218–1230, https://doi.org/ Conference on Systems, Man and Cybernetics, 2008: pp. 774–779. doi:https://doi.
10.1016/j.ijinfomgt.2016.08.005. org/10.1109/ICSMC.2008.4811372.
[39] G.A. Morgan, K.C. Barrett, N.L. Leech, G.W. Gloeckner, SPSS for introductory [58] P. Zhang, A novel feature selection method based on global sensitivity analysis
statistics: Use and interpretation (2004), https://doi.org/10.4324/ with application in machine learning-based prediction model, Applied Soft
9780429287657. Computing Journal. 85 (2019) In Press. doi:https://doi.org/10.1016/j.asoc.20
[40] E. Zinovyeva, W.K. Härdle, S. Lessmann, Antisocial online behavior detection using 19.105859.
deep learning, Decision Support Systems. 137 (2020). doi:https://doi.org/10.101 [59] J. Nascimento, W. Powell, Dynamic programming models and algorithms for the
6/j.dss.2020.113362. mutual fund cash balance problem, Manag. Sci. 56 (2010) 801–815, https://doi.
[41] H.M. Zolbanin, B. Davazdahemami, D. Delen, A.H. Zadeh, Data analytics for the org/10.1287/mnsc.1100.1143.
sustainable use of resources in hospitals: predicting the length of stay for patients [60] Y. He, X. Zhang, J. Sun, Channel pruning for accelerating very Deep Neural
with chronic diseases, Information and Management. In Press (2020), https://doi. Networks, in: Proceedings of the IEEE International Conference on Computer
org/10.1016/j.im.2020.103282. Vision, 2017: pp. 1398–1406. doi:https://doi.org/10.1109/ICCV.2017.155.
[42] H. Ahady Dolatsara, Y.J. Chen, C. Evans, A. Gupta, F.M. Megahed, A two-stage [61] N. Chaudhuri, I. Bose, Exploring the role of deep neural networks for post-disaster
machine learning framework to predict heart transplantation survival probabilities decision support, Decision Support Systems. 130 (2020). doi:https://doi.
over time with a monotonic probability constraint, Decision Support Systems. 137 org/10.1016/j.dss.2019.113234.
(2020). doi:https://doi.org/10.1016/j.dss.2020.113363. [62] A. Martínez, C. Schmuck, S. Pereverzyev, C. Pirker, M. Haltmeier, A machine
[43] Y. Guan, Q. Wei, G. Chen, Deep learning based personalized recommendation with learning framework for customer purchase prediction in the non-contractual
multi-view information integration, Decis. Support. Syst. 118 (2019) 58–69, setting, Eur. J. Oper. Res. 281 (2020) 588–596, https://doi.org/10.1016/j.
https://doi.org/10.1016/j.dss.2019.01.003. ejor.2018.04.034.
[44] Y. Rao, H. Xie, J. Li, F. Jin, F.L. Wang, Q. Li, Social emotion classification of short [63] B.J.D. Jacobs, B. Donkers, D. Fok, Model-based purchase predictions for large
text via topic-level maximum entropy model, Inf. Manag. 53 (2016) 978–986, assortments, Mark. Sci. 35 (2016) 389–404, https://doi.org/10.1287/
https://doi.org/10.1016/j.im.2016.04.005. mksc.2016.0985.
[45] Y. Bengio, Gradient-based optimization of hyperparameters, Neural Comput. 12 [64] X. Lu, S. He, S. Lian, S. Ba, J. Wu, Is user-generated content always helpful? The
(2000) 1889–1900, https://doi.org/10.1162/089976600300015187. effects of online forum browsing on consumers’ travel purchase decisions, Decision
[46] K.Q. Shen, C.J. Ong, X.P. Li, E.P.V. Wilder-Smith, Feature selection via sensitivity Support Systems. 137 (2020). doi:https://doi.org/10.1016/j.dss.2020.113368.
analysis of SVM probabilistic outputs, Mach. Learn. 70 (2008) 1–20, https://doi. [65] P.J. Danaher, T.S. Danaher, M.S. Smith, R. Loaiza-Maya, Advertising effectiveness
org/10.1007/s10994-007-5025-7. for multiple retailer-brands in a multimedia and multichannel environment,
[47] S. Lee, J.Y. Choeh, Predicting the helpfulness of online reviews using multilayer J. Mark. Res. 57 (2020) 445–467, https://doi.org/10.1177/0022243720910104.
perceptron neural networks, Expert Syst. Appl. 41 (2014) 3041–3046, https://doi. [66] A. Poddar, N. Donthu, Y. Wei, Web site customer orientations, web site quality, and
org/10.1016/j.eswa.2013.10.034. purchase intentions: the role of web site personality, J. Bus. Res. 62 (2009)
[48] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdinov, Dropout: a 441–450, https://doi.org/10.1016/j.jbusres.2008.01.036.
simple way to prevent neural networks from overfitting, J. Mach. Learn. Res. 15 [67] G. Zhu, Z. Wu, Y. Wang, S. Cao, J. Cao, Online purchase decisions for tourism e-
(2014) 1929–1958. commerce, Electronic Commerce Research and Applications. 38 (2019). doi:https
[49] D.M.W.D. Powers, Evaluation: from precision, recall and f-factor to ROC, ://doi.org/10.1016/j.elerap.2019.100887.
informedness, markedness & correlation, Journal of Machine Learning
Technologies. 2 (2011) 37–63. doi:10.1.1.214.9232.

10