Sales Prediction of Market Using Machine Learning
Sales Prediction of Market Using Machine Learning
Abstract:- Connected devices, sensors, and mobile apps make the interests on particular products for one to one marketing,
retail sector a relevant testbed for big data tools and applications. purchasing patterns in a multi-store environment to improve
We investigate how big data is, and can be used in retail the sales. Market basket analysis has been intensively used in
operations. Based on our state-of-the-art literature review, we many companies as a means to discover product associations
identify four themes for big data applications in retail logistics: and base a retailer’s promotion strategy on them.
availability, assortment, pricing, and layout planning. Our semi- A retailer must know the needs of customers and adapt to them.
structured interviews with retailers and academics suggest that Market basket analysis is one achievable way to find out which
historical sales data and loyalty schemes can be used to obtain
items can be place together. Market basket analyses gives
customer insights for operational planning, but granular sales
data can also benefit availability and assortment decisions. retailer good information about related sales on group of goods
External data such as competitors’ prices and weather conditions basis Customers who buys bread often also buy several
can be used for demand forecasting and pricing. However, the products related to bread like milk, butter or jam. It makes
path to exploiting big data is not a bed of roses. Challenges include sense that these groups are placed side by side in a retail centre
shortages of people with the right set of skills, the lack of support so that consumers can access them promptly. Such related
from suppliers, issues in IT integration, managerial concerns groups of goods also must be located side-by-side in order to
including information sharing and process integration, and remind customers of related items and to lead them through the
physical capability of the supply chain to respond to real-time centre in a logical manner. Market basket analysis determines
changes captured by big data. We propose a data maturity profile
which products are bought together and to design the
for retail businesses and highlight future research directions.
Association Rules is one of the data mining techniques which is supermarket arrangement, and also to design promotional
used for identifying the relation between one item to another. campaigns. Therefore, the Market consumer behaviours need
Creating the rule to generate the new knowledge is a must to to be analysed which can be done during dissimilar data mining
determine the frequency of the appearance of the data on the item techniques. Well-versed decision can be made easily about
set so that it is easier to recognize the value of the percentage from product placement, pricing, endorsement, profitability and also
each of the datum by using certain algorithms, for example finds out if there are any successful products that have no
apriori. This research discussed the comparison between market significant related elements. Similar products can be found so
basket analysis by using apriori algorithm and market basket those can beplaced near each other or it can be cross- sold.
analysis without using algorithm in creating rule to generate the
new knowledge. The indicator of comparison included concept,
the process of creating the rule, and the achieved rule. The Association rules can be mined and this process of mining the
comparison revealed that both methods have the same concept, association rules is one of the most important and powerful
the different process of creating the rule, but the rule itself aspect of data mining. One of the main criteria of ARM is to
remains the same. find the relationship among various items in a database. An
Key Words: Big data; retail operations; maturity; association rule is of the form A→B where A is the antecedent
availability; assortment; replenishment; pricing; layout; logistics. and B is the Consequent. and here A and B are item sets and
the underlying rule says us purchased by the customers who
I. INTRODUCTION purchase A are likely to purchase B with a probability
One of the challenges for companies that have invested a lot in percentage factor as %C where C is known as confidence such
consumer data collection is how to mine important information a rule is as follows: “seventy per cent of people who purchase
from their vast customer databases and product feature beer will also like to purchase diapers” This helps the shop
databases, in order to gain economical advantage. Several managers to study the behaviour or buying habits of the
aspects of market basket analysis have been studied in customers to increase the sales . based on this study items that
academic literature, such as using customer interest profile and are regularly purchased by the customers are put under closed
proximity. For example, persons who purchase milk will also II. BIG DATA
likely to purchase Bread. Big data is an “imprecise description of a rich and complicated
The interestingness measures like support and confidence also set of characteristics, practices, techniques, ethical issues, and
plays a vital role in the association analysis. The support is outcomes all associated with data”. A more technical and
defined as percentage of transactions that contained in the rule complementary definition is “datasets that could not be
and is given by Support = (# of transactions involving A and perceived, acquired, managed, and processed by traditional
B) / (total number of transactions). [infor-mation technology] and software/hard-ware tools within
a tolerable time” For companies, big data is a cornucopia of
The other factor is confidence it is the percentage of digitalized content about consumers’ cognitions, emotions,
transactions that contain B if they contain A behaviors, and reactions critical to the ongoing data-driven
industrial revolution.
Confidence = Probability (B if A) = P(B/A) Confidence = (# ‘Big data’ has different definitions that focus on size, range,
of transactions involving A and B) / (total number of and speed. While some incorporate a dynamic definition as
transactions that have A) beyond the ability of typical software, some use a static number
as 100 terabytes or have a growth rate that is higher than 60%
Consider the following example: annually. Big data is also defined as using the population in the
analysis rather than the sample, as this can change the mind-set
TABLE I. EXAMPLE OF PREDICTIVE ANALYSIS of data analysis in three ways:
Customer Item Purchased Item Purchased
1 Burger Coke
2 Puff Mineral Water
Accept errors as they can be smoothed by sheer quantity, accept
3 Burger Mineral Water correlation rather than causality, and discover value of data
4 Puff Tea besides its original purpose. Big data is also defined based on
volume, variety, velocity, and veracity.
If A is “purchased Burger “ and B is “purchased mineral water” Volume represents to the amount of data, which can be affected
then by the scope, and is the primary attribute.
Variety refers to the data included in the analysis which comes
Support=P(A and B)=1/4 from various sources and are in many mediums. Velocity
includes the frequency of both data generation and deliveries.
Confidence=P(B/A)=1/2 Finally, veracity is to do with the uncertainty of the data. In this
paper, the definition used is: all population data that can be
Item sets that satisfy minimum support and minimum accessed timely. This definition implicitly involves volume,
confidence are called strong association rules. variety, velocity, and veracity features of big data.
Predictive analytics is composed of two words predict &
analysis, but it works in reverse viz. first analyse then predict. III. RETAIL CHALLENGES AND POTENTIAL BIG
It is human nature to want to know and DATA APPLICATIONS
predict what the future holds. Predictive analytics deals with Although retailers were distributing products passively in the
the prediction of future events based on previously observed past, with the information about the end customer demand, they
historical data by applying sophisticated methods like machine are becoming more proactive in the supply chain. The retail
learning. The historical data is collected and transformed by supply chain contains four major activities: assorting goods,
using various techniques like filtering, correlating the data, and breaking goods into smaller packages, holding inventory near
so on. Prediction process can be divided into four steps: the customer, and providing value-adding services such as gift
• Collect and pre-process raw data; wrapping and warranties. Agrawal and Smith mapped the
• Transform pre-processed data into a form that can be easily supply chain of two major home furnishing retailers in the US
handled by the selected machine learning method; and proposed a comprehensive set of planning processes:
• Create the learning model (training) using the transformed product design (private label merchandise) and assortment
data; planning, sourcing and vendor selection, logistics planning,
Report predictions to the user using the previously created distribution planning and inventory management, clearance
learning model. and markdown optimization, and cross-channel optimization.
An essential goal in data mining is to create and enhance the Major activities of retailing are to decide what products to carry
precision of predictive models, and a basic challenge toward (assortment, product design, procurement), how to sell
this end lies in the discovery of new features, inputs or products to customers (marketing, including pricing), and how
predictors. This paper illustrates how rules generated from to complete these efficiently (supportive functions, such as
Market Basket Analysis (MBA) may be utilized to improve logistic planning). Out of stock (OOS) situations and poor on
predictive models. The goal of data mining process is the shelf availability (OSA) lead to customer dissatisfaction and
extraction of information from large data sets, transform such consequently loss of market share . Assortment, pricing, and
information into some understandable structure for future store layout are challenging processes in retail operations that
application. It is a process of using different techniques to find are expected to benefit from big data analysis. Assortment and
useful patterns or models from data. This process is use to pricing involve a large amount of granular decisions and can be
select, explore and model large amount of data. affected by various factors, such as customer preferences, store
location, and demand elasticity. Without effective quantitative
analysis, making these decisions can be resource-consuming Step 1: Frequent itemset generation: Find all itemsets for which
and ineffective. Store layout is also worth exploring as it can the support is greater than the threshold support following the
affect customers’ purchasing decisions. process we have already seen earlier in this article.
IV. ASSORTMENT, PRICING AND STORE LAYOUT Step 2: Rule generation: Create rules from each frequent
Assortment is one of the most difficult tasks in the retail supply itemset using the binary partition of frequent itemsets and look
chain and has received high priority due to its impact on sales. for the ones with high confidence. These rules are called
Big data is used to micro-segment customers and optimize candidate rules.
assortment. Common practices include analysis of the
correlations between items purchased as well as time- and
location- dependent purchase patterns. The output helps
retailers understand customer preferences and compositions,
and improve forecast processes.
Pricing is one of the most difficult issues facing retailers due
to the large amounts of SKUs whose price can vary in different
locations and over time according to local demand and
competition. In the case of markdowns and promotions, pricing
is even more difficult because of the increased level
of uncertainty and the lack of historical data with the same
Fig.1. Proposed Model
promotion conditions. As the promotions are a function of what
else is also on the market, it is almost impossible to replicate a
Algorithm for sales growth using Apriori.
promotion, although some similarities can be captured for
forecasting purposes. With extra computing and analysis
Step1: Start.
capacity, big data analysis enables promotion decisions to be
taken more effectively and efficiently. Big data can also help
Step2: Select the database of the market.
retailers evaluate sources of sales lifts and plan future
promotions more effectively.
Step3: Apply minimum support to find all the frequent sets
Store layout is worth exploring in terms of how it can benefit
with k items in a database.
from big data analysis as it has an impact on purchasing
decisions. In the past, customer in-store behavior could be
Step4: Use the self-join rule to find the frequent sets with k+1
analyzed by observing sampled customers and this information
items with the help of frequent k-itemsets. Repeat this process
could help retailers optimize store layout and shelf design.
from k=1 to the point when we are unable to apply the self-join
However, this exercise requires a significant level of effort and
rule.
reports have suggested that using videos, mobile services, WI-
FI, and RFID tags attached to shopping carts to track customer
Step5: RawData P 7−−> D F 7−−> X d RFCn0 7−−>
movement in store can provide high volumes of information at
ˆy*TopN 7−−> ˆ y//
low cost. Although in-store traffic alone cannot provide
RawData: Exploratory Data Analysis
information on customer behavior, customers’ in-store location
P: Partition RawData into Dtrain, Dtest, ...
data still provide useful correlations with sales.
F: Feature Design builds design matrix X from d RFCn0:
Random Forest Classifier makes probabilistic prediction ˆ y*
V. MINING ASSOCIATION RULES
X n0: Hyperparameters found using OOB classifier
Till now, we have looked at the Apriori algorithm with respect
TopN: TopN Variants make binary predictions ˆ y from ˆ y* ˆ
to frequent itemset generation. There is another task for which
y: Prediction Explorer inspects binary predictions ˆy//
we can use this algorithm, i.e., finding association rules
efficiently. For finding association rules, we need to find all
Step6: RawData P −> DDSets
rules having support greater than the threshold support and
RawData 7→{Ds}s∈DSets //DSets = {train,test,kaggle},
confidence greater than the threshold confidence.
P is the unique partition defined by a partition of the set of
But, how do we find these? One possible way is brute force,
users, U, ordered by, say, user id, into {Us}s∈DSets Utrain:
i.e., to list all the possible association rules and calculate the
80% of 131,209 users with available ultimate orders. Utest:
support and confidence for each rule. Then eliminate the rules
20% of 131,209 users with available ultimate orders.
that fail the threshold support and confidence. But it is
Ukaggle: 75,000 users whose ultimate orders are withheld by
computationally very heavy and prohibitive as the number of
Kaggle.
all the possible association rules increase exponentially with
{Ds}s∈DSets is the image of RawData under P.//[5]
the number of items.
Given there are n items in the set I, the total number of possible
Step7: : F : D → Mn×m(R); D 7→ X //F = (fj)m 1 is an m-
association rules is 3n–2n+1+1.
tuple of functions fj : D →Rn a feature fj(D) is a column of X
We can also use another way, which is called the two-step
fj are compositions of aggregations, filtrations, arithmetic ...
approach, to find the efficient association rules. The two-step
compute some fj via an unsupervised learning technique,
approach is:
Latent Dirichlet Allocation (LDA), introduced in [BNJ03].
Prod(u) are all products purchased by u ∈ U 10
Rows of X are user-product pairs ((u,p) | u ∈ U & p ∈ Prod(u))// reengineering, must be conducted to enable the software to
serve the needs of its end users. In essence, preventive
Step8: RFCn0 : Mn×m(R) → [0,1]n; X → ˆ y* maintenance makes changes to computer programs so that they
can be more easily corrected, adapted, and enhanced.
Step9: TopN : [0,1]n →{0,1}n ˆ y* 7→ ˆy //TopN: maps make
binary predictions from probability Evaluation:
TopNthreshold: chooses a classification threshold p0; best for System evaluation provides framework for classification
optimizing threshold metrics (F1-score); not best for product scheme to identifying sets of similar systems. The framework
applications TopNu uses user’s mean basket size as N; set integrates previous studies on software evaluation, productivity
basket size by user reduces its variance; higher precision models, software quality factors, and total quality models. It
versions to autopopulate carts? also classifies information about software systems from the
TopNN: uses a constant value N for each user; worse metrics perspective of the project, the system, and the environment.
but best for displaying on fixed-width web page (zero basket
size variance).//
VII. RESULTS
Step10: Stop
Types of Maintenance:
• Corrective Maintenance
• Adaptive Maintenance
• Perfective maintenance
• Preventive maintenance
Corrective Maintenance :-
Even with the best quality assurance activities, it is likely that
the customer will uncover defects in the software. Corrective
maintenance changes the software to correct defects.
Adaptive Maintenance :-
Over time, the original environment (e.g., CPU, operating
system, business rules, external product characteristics) for
which the software was developed is likely to change. Adaptive
maintenance results in modification to the software to
accommodate changes to its external environment.
Perfective maintenance :-
As software is used, the customer/user will recognize
additional functions that will provide benefit. Perfective
maintenance extends the software beyond its original
functional requirements.
VIII. CONCLUSION
Market basket analysis generates the frequent item set i.e.
association rules can easily tell the customer buying behaviour
and the retailer with the help of these concepts can easily setup
his retail shop and can develop the business in future. The main
algorithm used in market basket analysis is the Apriori
algorithm. It can be a very powerful tool for analysing the
purchasing patterns of consumers. The three statistical
measures in market basket analysis are support, confidence.
Support measures the frequency an item appears in a given
transactional data set, confidence measures the algorithm’s
predictive power or accuracy. In our example, we examined the
transactional patterns of grocery purchases and discovered both
obvious and not-so-obvious patterns in certain transactions.
Fig.6. Graph of period of reorder since last order Association rules and the existing data mining algorithms
usage for market basket analysis , also it clearly mentioned
about the existing algorithm and its implementation clearly and
also about its problems and solutions. Predictive modelling
offers the potential for firms to be proactive instead of
receptive. Predictive modelling using transactional data create
particular challenges which need to be carefully addressed to
develop valuable models. With MBA, leading retailers can
drive more profitable advertising and promotions, attract more
customers, increases the value of market basket and much
more. Consumers, planners, merchandisers and store
administrators have started to recognize how this new era of
easy-to-use market basket analysis tools helps to work more
intelligent and compete more successfully. Our future work
would be to design and develop intelligent prediction models
to generate the association rules that can be adopted on
recommendation system to make the functionally more
operational. Better and effective rule mining techniques can be
used for better performance of the recommendation system. In
future the same algorithm can be modified and it can be
extended in the future work which also decreases the time
complexity.
REFERENCES