4 IoT Big Data Analytics

Internet of things (IoT) generate large amount of data which is typically stored on cloud. These
large chunks of data is of no value without analytics. Analytics add power and context to the data
so that useful inferences can be drawn and a deeper insight can be gained to drive actionable
outcomes for improving business.
The primary objective of running analytics on big data is to support and enable organizations to
have enriched knowledge of data for improved decision making. The traditional data analytic
tools have limited storing, processing and analytic capabilities and thus cannot handle large bulk
of data whereas big data analytics empower data scientists to analyze large volume and variety of
data which is generated at a high velocity [1]. Big data analytics require statistical methods, data
mining and machine learning technologies to extract meaningful information to make
predictions, recognize useful patterns and find correlation between various parameters for better
decision making [2-4].
Unlike traditional big data, IoT big data is generated at a high rate from several sources such as
different types of sensors and objects, which introduce diversity and heterogeneity in data. In this
context, the data produced by IoTs can be characterized by 3 Vs of big data i.e volume, velocity
and variety. It is estimated that the number of IoT sensors will reach to approximately 1 trillion
in 2030, thereby adding more variety and complexity to data [3]. The data received from
multiple data sources can be in unstructured, semi-structured and structured format. The data is
transformed into more comprehensible format prior to performing analytics [4]. The integration
of data collected from multiple sources in variable formats make big data analysis more
complex. The extraction of useful knowledge from big data analysis entails efficient and scalable
techniques and methods while existing techniques are incapable of handling large data sets. The
challenge here is to make data analytics fast, efficient and accurate by distributing and
processing the data in parallel.
Distributed and parallel computing approaches such as cloud and grid computing can meet big
data requirements to some extent but unable to provide a complete solution. Grid computing
provided the solution for handling large volume and velocity of data by distributing and reusing
the storage and computational resources but it does not support large variety of data. It also
require a shared software middleware, dedicated hardware and expensive grid deployment and

1
management. On the other hand cloud computing provides flexible consumption of resources
that are managed by cloud providers. Cloud computing is based on the idea that hardware and
middleware can be centralized while applications can be managed by the consumer. The
services provided to the clients are based on the agreement between the cloud service provider
and the consumer but it is not suitable to integrate and manage resources from distributed
organizations. To overcome the limitations of these approaches, hybrid data infrastructure
approach is introduced which is based on the concept of integrating multiple technologies such
as grid and cloud to provide efficient data usage and management [5].

4.1 Big Data Analytics and Internet of Things (IoT)

Big data analytics and internet of things (IoT) work in tandem. The IoT devices produce streams
of data constantly or on demand, which requires real time storage, processing, analysis and
decision making. The unceasing growth in data volume, variety and velocity is beyond the data
management capabilities of traditional and commonly used data analytic tools such as MS excel,
MySQL. However, nowadays the real time IoT analytics are possible owing to the development
and availability of sophisticated and open source distributed storage and processing frameworks
such as Hadoop and Map Reduce [7]. These open source extensible frameworks can efficiently
manage big data and process it in parallel to satisfy the real time analytical needs of critical
solution. Additionally, these frameworks provide business industry to gain deeper insights into
their data by applying various types of analytics to better comprehend the growth of their
business, predict profits or loss based on the sales data, identify relations between financial
performances supply chain and also identify patterns of non-festive sale relative to festive sale,
etc.
There are several types of analytics which provide analysis based on the type of application.
These are discussed below:

2
4.2 Types of Data Analytics:
There are four major types of analytical techniques: descriptive analytics, diagnostic analytics,
predictive analytics, and prescriptive analytics. All these analytical techniques are interrelated
and provide different levels of understanding of data based on the type and complexity of
application.

Figure 1: Four Types of Big data Analytics [8]

4.2.1 Descriptive analytics:

Descriptive analytics is the most basic form of analytics which describes raw data and
summarizes it in easily understandable form using basic math functions such as sum, average,
percentage, etc. It describes the past and primarily answers the question, “what happened?” This
category of analytics is useful in terms of providing insights into the historical behavior or trend
of data to better comprehend its influence on future outcomes. For instance, accessing the current
credit statement of a customer who wants to purchase a car. The analysis of previous credit
history of the customer and the expected credit profile in future would help the sales manager to
make decision for the approval of customer order request. Other examples may include finding
total inventory stock, average sale per week, month or year, average monthly temperature, etc.

3
The big data analytics typically used to mine business industry data belong to descriptive
analytics [10-11].
Descriptive analytics only provide valuable information about what happened in the past without
explaining why it happened. For this reason, organizations which are extremely data-driven do
not rely on descriptive analytics only, rather they prefer to perform other kinds of data analytics
too to get a deeper insight [11].

4.2.2 Diagnostic Analytics

Diagnostics analysis allow us to discover, “why something happened?” This category of
analytics is frequently combined with descriptive analytics to determine dependencies, find
unknown patterns and discover new information that can give more detailed data insights [12-
13]. Typically diagnostics tools are used to explore the reasons behind the observed outcome and
are characterized by techniques such as data mining, classification, principal component analysis
(PCA), regression and correlations [7-8, 14]. The well designed and interactive business
information (BI) dashboards integrating filters, displays and complete representation of time
series or temporal data allow for such analysis. Tools such as BI platform support tool and
Flexible Log Reader are examples of diagnostics tools [15-16]. Companies go for diagnostic
analytics, as it gives a deep insight into a particular problem.
For example, health care industry is heavily dependent on diagnostic analysis for accurate
diagnosis of disease. The patient data describing the trends of blood pressure, glucose, oxygen
levels, heart rate, and respiratory conditions can help us to visualize high level view of the
patient’s health. But in order to investigate the real causes of for example short of breath, a more
detailed analysis is required. There could be many overlapping symptoms related to several
diseases, therefore, a deeper insights into the patient record by incorporating diagnostic tools
such as correlations, classifications, and time series analysis would greatly help to rule out many
symptoms and focus on only those which are the real trouble makers.
Diagnostic analysis can also be applied in agriculture industry to investigate a decline in the crop
yield. For this purpose, agriculture data of several years should be examined to determine what
factors and events contributed to low crop yield. In this context, the correlation and regression
analysis of crop yield and weather parameters (such as temperature and rainfall) can greatly
contribute to drill down and isolate the main reasons behind the reduced agriculture output.

4
4.2.3 Predictive Analytics
Predictive analytics is an advanced analytics which analyzes past and current data to project the
future events, states and actionable outcomes. This category of analysis mainly addresses “What
is likely or going to happen?” and is characterized by techniques such as regression analysis,
classification models, Monte Carlo analysis, random forest models, and Bayesian analyses,
pattern matching, and predictive modeling [14,16]. Predictive analytics hold great important in
business industry and is one of the major factor that leads to the integration of analytics in
businesses [7]. The future behavior of a system can be predicted by identifying patterns and
analyzing data trends over time.
Predictive analysis is heavily used in industries for preventive maintenance of factory parts or
instruments. The temperature and vibration profiles of machine parts are recorded through IoT
sensors and data acquired is analyzed to forecast maintenance to avoid possible ceasing of
machine and interruption in manufacturing processes.
Predictive analysis is also used in agriculture sector to monitor crop health. For this purpose,
time series analysis of meteorological parameters are performed to schedule irrigation of crops
based on the weather forecast. This would greatly help in the success of crop and prevent any
possible damage to crop in case of delay in rainfall.

4.2.4 Prescriptive Analytics

Prescriptive analytics is a kind of advanced analytics which analyses collected data to address,
“What should be done?” and is described by techniques such as graph analysis, simulation,
neural networks, recommendation engines, heuristics, and machine learning. It works in
conjunction with predictive analytics just like descriptive analytics work in pair with diagnostic
analysis [7]. Prescriptive analytics integrate big data, association rules, and machine learning to
generate predictions and recommend choices to gain benefit from those forecasts. Prescriptive
analytics not only projects what and when the event will occur but also why it will take place. It
incorporates machine learning models to both predict results and recommend actions to minimize

5
the potential risks. As past data is used to calculate future results, prescriptive analytics can be
used to make better choices and take advantage of opportunities For example, Google’s self-
driving cars run prescriptive analytics to make endless driving decisions based on the historic
and real time sensors’ data. The cars make driving decision on the fly based on the traffic and
weather data acquired from cloud using IoT infrastructure. The vehicle’s on-board computers
uses machine learning models to predict future outcomes and suggest actions accordingly. For
example, the car may predict bad weather and heavy traffic and based on that prediction makes
optimal choices about the safest route to travel [7].
Prescriptive analytics can be supportive in the health care industry for effective management of
patient care. For instance, in order to determine the total number of patients who are suffering
from obesity, factors like diabetes and LDL cholesterol levels are measured to make decision
where to focus treatment and finding the right patients for clinical trials, etc.
Prescriptive and Predictive can be used hand in hand in health care industry to predict possible
diseases or health risks. The data is typically collected through various IoT sensors deployed in
wearables such as shoes, watches, wrist bands which track the activities of a person and provide
predictions related to possible health risks and recommendations or advices to improve health
care.

4.3 IoT and Big Data Analytics Systems

4.3.1 Agriculture
Field Connect system is an IoT and big data system developed by John Deere, which is one of
the leading name in manufacturing equipment. This system consists of various sensors to
monitor crop data and perform analytics to reach irrigation decisions. The data is sent wirelessly
to farmers so they can observe the levels of different agrometeorological parameters such as soil
moisture, temperature of air & soil, humidity and rainfall. The data collected over time also help
to discover trends in climate change that impact the retention of moisture in soil [18, 19].
Grain Management System is another system based on IoT and big data technology which is
made by TempuTech and is making waves in agriculture sector. This system consists of sensors
wirelessly connected to monitor agrarian facility such as grain elevators. These elevators are
used to deposit grains in silos or alternative storage facility. The system provides the optimal
storage conditions by measuring the temperature and moisture in grain bin and provide automatic

6
aeration and drying service if required. The temperature and moisture data is also shared with
farm managers so that that can understand the variation in temperature and moisture data with
respect to changes in meteorological conditions [18, 20].
4.3.2 Healthcare
The Clermont-Ferrand University Hospital in France has set up an IoT and big data resource in
collaboration with Microsoft and CapsuleTech. The hospital has implemented an intelligent
system in its ICU and general medical units that gathers vital signs from medical devices,
transform the data into standard format, and transmits it directly to an electronic medical records
system. The system also provides a mobile app which the medical staff use to validate
themselves and subsequently transmit patient’s data directly from medical devices and gadgets to
medical record system. The data is processed and analyzed to deduce actionable insights to
provide intelligent care, advance hospital processes and provides a single, secure interface for
monitoring patient records [18, 21, 22].
4.3.3 Aerospace Industry
Virgin Atlantic has adopted IoT and big data technology for connecting 787 Boeing planes and
linking cargo devices. The planes are loaded with multiple IoT devices which can generate huge
amount of data approaching to more than half a terabyte. This data could be a rich source of
information for performing predictive analytics and can proactively plan upkeeps and repairs
before a failure occurs. The onboard instruments’ downtime can be very expensive and it is
usually hard to run last minute services, repairs, and replacement of parts.
The big data program of Virgin Atlantic is not fully operational but the idea is that the collected
data could be used to gain operational insights for predicting repair schedules to enhance flight
and fuel efficiency. As tons of flights fly each day, so the volume of data produced by onboard
sensors will also grow constantly, which need a scalable cloud solution to securely store,
process, and analyzed data for improving business value and operational efficiency [18, 23].

4.3.4 Customer Activity Analytics

Disney World’s exclusive “Magic band” is another example of integrating IoT and Big data. The
magic band is a wearable wrist band embellished with sensors to record information about all the
activities performed by a park tourist ranging from hotels check-ins, taking rides, to eating lunch,
booking a spot for famous attractions and many more. The visitors’ information will be cardinal

7
for the park management and enable them to improve their customer care services and provide a
more personalized experience to visitors thus making it more exciting and thrilling. Additionally,
the data analytics will give an understanding on how to better accommodate and manage
growing number of tourists and efficiently regulate food supply at densely occupied restaurants
and shops [18, 24]
Alex and Ani, a jewelry store chain is another great example where IoT and big data is working
together. The company has implemented the entire system in collaboration with Swirl Company
which is based on beacon technology. The jewelry store is armed with Swirl Bluetooth beacons
to track the customers entering the store and send more personalized and exciting offers to their
mobile phones. The system also records customers’ activities within the store and generate heat
maps to visualize customer movement. This helps the company to orchestrate their product
display in a manner that can enhance their sales [18, 25].

4.4 Tools and Languages

4.4.1 Map reduce
4.4.2 Hadoop

4.5 References
[1].N. Golchha, ``Big data-the information revolution,'' Int. J. Adv. Res., vol. 1, no. 12, pp.
791_794, 2015.
[2].C.-W. Tsai, ``Big data analytics: A survey,'' J. Big Data, vol. 2, no. 1, pp. 1-32, 2015.
[3].M. Chen, Related Technologies in Big Data. Heidelberg, Germany: Springer, 2014, pp.
11-18.
[4].M. Marjani et al., ‘‘Big IoT data analytics: Architecture, opportunities, and open research
challenges,’’ IEEE Access, vol. 5, pp. 5247–5261, Mar. 2017
[5].L. Candela and D. P. C. Pagano, “Managing big data through hybrid data
infrastructures,” ERCIM News, vol. 89, pp. 37-38, Jun. 2012.
[6].I. A. T. Hashem et al., “The rise of `big data' on cloud computing: Review and open
research issues,” Inf. Syst., vol. 47, pp. 98115, Jan. 2015.

8
[7].Syed Zaeem Hussain, The Definitive Guide: The Internet of Things for Business, 2nd
Edition, September 2016, http://www.aeris.com/hs-action/introducing-definitive-guide-
iot-business-2nd-edition/
[8].Shafiq Marediya, The Different Types of Data Analytics, Feb,
2017,https://blog.k2datascience.com/the-different-types-of-data-analytics-72613e4d0130,
Accessed online: 19th January, 2018.
[9].Descriptive, Predictive, and Prescriptive Analytics Explained, 2018,
https://halobi.com/blog/descriptive-predictive-and-prescriptive-analytics-explained/,
Accessed online: 20th January, 2018.
[10]. Diagnostic Analytics, 2017,
https://www.cornerstoneondemand.com/glossary/diagnostic-analytics, Accessed online:
19th January, 2018.
[11]. Analytics, 2018, https://ebrary.net/8574/marketing/analytics, Accessed online: 19th
January, 2018.
[12]. Science Soft, 4 types of data analytics to improve decision-making, 2017,
https://www.scnsoft.com/blog/4-types-of-data-analytics, Accessed online: 20th January,
2018.
[13]. Four Types of Big Data Analytics and Examples of Their Use, 2018,
http://www.ingrammicroadvisor.com/data-center/four-types-of-big-data-analytics-and-
examples-of-their-use, Accessed online: 20th January, 2018.
[14]. Tim Vlamis, The Four Realms of Analytics, June 2015,
http://www.vlamis.com/blog/2015/6/4/the-four-realms-of-analytics.html, Accessed
online: 20th January, 2018
[15]. Thomas Maydon, The 4 Types of Data Analytics, Jan 2017,
https://insights.principa.co.za/4-types-of-data-analytics-descriptive-diagnostic-predictive-
prescriptive, Accessed online: 20th January, 2018.
[16]. Diagnostic Tools, 2018, https://support.sap.com/en/tools/diagnostic-tools.html,
Accessed online: 20th January, 2018
[17]. IoT & Sensor Data Analytics, 2018, https://www.elderresearch.com/analytics-
solutions/sensor-data-iot-analytics-solutions, Accessed online: 5th February, 2018

9
[18]. Ten examples of IoT and big data working well together,
http://www.zdnet.com/article/ten-examples-of-iot-and-big-data-working-well-together/,
Accessed online: 5th February, 2018
[19]. Johndeere, Agriculture, 2018 https://www.deere.com/en/agriculture/, Accessed online:
5th February, 2018
[20]. Temputech.com, 2018, http://www.temputech.com/, Accessed online: 5th February,
2018
[21]. CapsuleTech, 2018, http://www.capsuletech.com/, Accessed online: 5th February, 2018
[22]. Microsoft, Hospital Improves Care with Automated Data Collection and System
Integration, Dec 2014, https://customers.microsoft.com/en-US/story/hospital-improves-
care-with-automated-data-collection, Accessed online: 5th February, 2018
[23]. Mathew Finnegan, Boeing 787s to create half a terabyte of data per flight, says Virgin
Atlantic, Mar 2013, https://www.computerworlduk.com/data/boeing-787s-create-half-
terabyte-of-data-per-flight-says-virgin-atlantic-3433595/, Accessed online: 5th February,
2018
[24]. Walt Disney World, https://disneyworld.disney.go.com/faq/bands-cards/understanding-
magic-band/, Accessed online: 5th February, 2018
[25]. Claire Swedberg, Alex and Ani Rolls Out Swirl's Bluetooth Beacons at 40 Stores,
February 2014, http://www.rfidjournal.com/articles/view?11475, Accessed online: 5 th
February, 2018

10