AWS Apps For An AI Enabled World US44941919
AWS Apps For An AI Enabled World US44941919
AWS Apps For An AI Enabled World US44941919
Organizations are using deep learning models to recommend products, predict pricing, recognize
images, and improve decision making as well as for a host of other use cases. Until recently,
developing deep learning models took significant amounts of time, effort, and knowledge and required
expertise in this field. Recently vendors like Amazon Web Services (AWS) have developed services
and tools for deep learning, that allow data scientists and developers to develop and deploy deep
learning models more quickly and easily.
There are numerous machine/deep learning tools and frameworks such as TensorFlow, Apache MXNet,
and PyTorch — all have valuable attributes that make them useful in developing intelligent applications.
However, there are many factors involved that inhibit the development of machine learning applications:
▪ Choosing the right machine/deep learning framework for the job at hand
▪ Choosing the right machine/deep learning algorithm
▪ Adjusting and tuning the machine algorithm and data for the most accurate predictions
▪ Identifying, locating, and curating training data for machine learning models
▪ Having the right amount of compute resources for both model training and generating
predictions in production (inferences)
▪ Integrating machine/deep learning models into existing enterprise applications
▪ Operationalizing models to perform at scale in production
For organizations that prefer the ease and convenience of using pretrained deep learning models
via APIs, services such as Amazon Rekognition for images and video, Amazon Lex for chatbot
integration, Amazon Polly for text to speech, Amazon Translate for natural language translation,
Amazon Transcribe for speech recognition, Amazon Personalize for individualized recommendations
for customers, Amazon Forecast for accurate time-series forecasting, Amazon Textract to extract
text and data from virtually any document, Amazon Comprehend Medical for detecting useful
information in unstructured clinical text, and Amazon Comprehend to find relationships in text can
accelerate adding intelligent capabilities to applications. The advantage to developers is that they
can just use these APIs simply and easily without having to go through the entire process of creating
their own custom machine learning models. Most developers would be well served to check if a
pre-existing API can solve their problem before beginning the process of creating a custom
machine learning model.
SITUATION OVERVIEW
Introduction
The market for machine learning and deep learning–based AI applications has grown rapidly and
continues to surge. IDC estimates that spending on machine learning and deep learning solutions will
exceed $57 billion by 2021, and by 2026, IDC predicts 75% of all enterprise software will include some
aspect of machine/deep learning for predictions, recommendations, or advice. Organizations need to
consider the following reasons why these systems are important to their future:
▪ Augment human judgment. The best business cases are about extending human capabilities,
not replacing them, by positioning AI-enabled applications as an extension of human intention.
Power tools in the hands of a craftsperson is the best analogy. Pricing optimization models are
good examples of deep learning in this area. A second example would be an AI imaging
application that automatically identifies cancerous tumors by examining radiology images,
providing assistance to radiologists.
▪ Accelerate investigation and discovery. Even the very best human readers can't comprehend
millions of pages of documents in one day. Applications that understand natural language can
be applied to this task for both the spoken and the printed word. Deep learning–based natural
language systems provide better results than handcrafted, taxonomy-based systems.
▪ Recommend "next best actions" and predict outcomes. Deep learning–based applications build
models using relevant data for recommendations and predictions, which are some of the
typical use cases.
▪ Personalize outcomes and recommendations. Many organizations are beginning to use
deep learning models to "personalize" content, predictions, and recommendations to
specific customers or prospects. This is especially true with mobile applications where
users increasingly expect their devices and applications to "know" their likes, dislikes,
and expectations.
At the same time as AI-enabled applications are beginning to emerge, we're seeing a growing market
for deep learning tools and solutions based on open source. A powerful combination of motivated,
capable developers; a proven open source community development model; and the need and desire
for low-cost or free software products to provide learning abilities for computing systems has led to a
growing market segment producing machine/deep learning software libraries and tools. These tools
are part of a larger group of technologies that include speech recognition, natural language
processing, predictive analytics, advanced analytics, neural networks, and supervised and
unsupervised machine learning. The endgame is all about making applications smarter by using
special libraries containing self-learning algorithms, which when unleashed on a data set can learn and
self-program to solve various types of problems. These self-programmed computer algorithms are
fueling the emergence of what we at IDC call intelligent applications.
The emergence of tools, frameworks, and libraries that provide services for machine learning and deep
learning is setting the stage for a low-cost enabler of intelligent applications to be built by developers
today. Organizations are looking at these services to replace rule- or heuristics-based approaches that
have to be extensively programmed and maintained today. The combination of high-performance
compute resources, tremendous amounts of data, and the frameworks and libraries for machine/deep
learning is solving problems and challenges without the need to resort to programming. These
machine/deep learning libraries and technologies are being used for an ever-wider array of use cases,
from image recognition and disease diagnosis to pricing optimization and product recommendations.
Machine/deep learning is a key component of most AI applications and is also being added to
enterprise applications. Improvements in the variety, efficiency, and reliability of machine learning will
make these applications more usable and stable and help increase their popularity.
There are many types of frameworks and tools for deep learning. A few of these are:
▪ TensorFlow, originally developed by Google, is an open source software framework for
numerical computation.
▪ Apache MXNet is a scalable training and inference framework from the Distributed (Deep)
Machine Learning Community (DMLC) consortium and is an incubating project of the Apache
Software Foundation.
▪ PyTorch is a "GPU first" deep learning framework that supports N-dimensional arrays, with
library routines offered to execute a variety of mathematical functions on the array data.
PyTorch is a python package based on Torch, allowing development and machine learning for
more advanced and complex AI tasks.
In addition to these frameworks, there are higher-level tools such as Gluon and Keras. Gluon is a
deep learning API that allows developers of all skill levels to prototype, build, and train deep learning
models. Gluon currently works with the deep learning framework Apache MXNet and gives developers
access to a simplified programming framework that allows them to accelerate both the development
of neural network–based models and the time required to train them. In addition, Gluon enhances the
agility of developers by enabling them to more easily make changes to neural networks and to debug
training models faster by incrementally understanding the effects of model training. Keras is a
high-level neural networks API, written in Python and capable of running on top of TensorFlow and
Microsoft Cognitive Toolkit. It was developed with a focus on enabling faster model development.
FIGURE 1
In-house developed
AI-enabled solutions
and the use of
commercial off-the-
shelf AI-enabled
software (23.8%)
Customizing commercially
available software using
AI frameworks (33.5%)
n = 839
FIGURE 2
Microsoft CNTK
Apache MXNet
TensorFlow
Caffe
Torch/PyTorch
Apache Mahout
Deeplearning4j
Accord.NET
Spark ML
Theano
Scikit-learn
Other
0 10 20 30 40 50 60
(% of respondents)
n = 340
FIGURE 3
AI Applications
Q. For what use cases are you developing or have you developed AI applications?
IT automation
Smart networking
Supply and logistics
Quality management investigation and recommendation systems
Intelligent process automation (business process)
Automated customer service agents
Fraud analysis and investigation
Digital assistants for enterprise knowledge workers
Regulatory intelligence
Control software for robotics
Sales process recommendation and automation
Automated preventative maintenance
Automated claims processing
Adaptive learning (in education)
Asset/fleet management
Automated threat intelligence and prevention systems
Expert shopping advisors and product recommendations
Public safety and emergency response
Digital twin/advanced digital simulation (manufacturing)
Freight management
Control software for autonomous vehicles
Program advisors and recommendation systems
Diagnosis and treatment (healthcare)
Merchandising for omni-channel operations
Pharmaceutical research and discovery
0 5 10 15 20 25 30 35 40 45
(% of respondents)
n = 505
Developing AI-enabled applications that use machine learning and deep learning is increasingly
popular across both the enterprise software market and the application development market. However,
there are significant challenges faced by enterprises wanting to adopt these new technologies:
▪ Disparate tools and technologies are freely available, but knowing where to start and which
tools or technologies to use can be confusing to organizations and their developers.
AWS has made it much easier for machine/deep learning developers by offering a broad range of tools
and services to get deep learning models created and into production. These tools range from
infrastructure like easily usable GPU services through Amazon Elastic Compute Cloud (EC2) P3
instances, which take advantage of NVIDIA Volta architecture, to frameworks such as TensorFlow and
Apache MXNet, platforms like Spark and Amazon EMR, and the newly announced Amazon
SageMaker. They also include deep learning–based services such as Amazon Comprehend, Amazon
Transcribe, and Amazon Rekognition. The architecture for these services can be seen in Figure 4.
AWS is unique in its approach in supporting all deep learning frameworks, rather than offering a single
preferred framework as some of the other cloud service providers do.
FIGURE 4
First, Amazon SageMaker Ground Truth helps customers build highly accurate training data sets
quickly using machine learning and reduce data labeling costs by up to 70%. Successful machine
learning models are trained using data that has been labeled to teach the model how to make correct
decisions. This process can often take months and large teams of people to complete. SageMaker
Ground Truth provides an innovative solution to reduce cost and complexity while increasing the
accuracy of data labeling by bringing together machine learning and human labeling. This is possible
with a technique called active learning.
SageMaker requires no setup and provides hosted Jupyter notebooks so that developers can start
processing training data sets and developing machine learning and deep learning models immediately.
All it takes is a few clicks in the SageMaker console to create a fully managed notebook workspace.
The service takes care of establishing secure network connections to your organization's VPC and
launching an Amazon EC2 instance, preloaded with useful libraries for machine learning and deep
learning frameworks like TensorFlow and Apache MXNet. Developers can build or import their own
notebook or just bring data to one of many prebuilt notebooks designed for common use cases, such as
risk modeling, churn prediction, and OCR. To prepare the data from Amazon S3, Amazon Redshift,
Amazon DynamoDB, and Amazon RDS for model training, developers can use AWS Glue, Apache Spark
on Amazon EMR for data preprocessing, and Amazon EFS as optional storage for your workspace.
When the application is ready to train, developers simply indicate the type and quantity of Amazon
EC2 instances that they need and initiate training with a single click. SageMaker then sets up the
distributed compute cluster, performs the training, and tears down the cluster when complete, so
organizations only pay for the resources that they have used and don't have to worry about the
underlying infrastructure. SageMaker seamlessly scales to virtually unlimited nodes, so developers no
longer need to worry about all the complexity and lost time involved in making distributed training
architectures work.
SageMaker provides high-performance, scalable machine learning algorithms optimized for speed,
scale, and accuracy. You can choose from supervised algorithms where the correct answers are
known during training, and you can instruct the model where it made mistakes. For example,
SageMaker includes supervised algorithms such as XGBoost and linear/logistic regression or
classification to address recommendation and time series prediction problems. SageMaker also
includes support for unsupervised learning (i.e., the algorithms must discover the correct answers on
their own), such as with k-means clustering and principal component analysis (PCA), to solve
problems like identifying customer groupings based on purchasing behavior.
Amazon SageMaker RL enables the ability to teach machine learning models to make decisions when
there is no training data, with the support for reinforcement learning. With support for simulation
environments such as Simulink and MATLAB and frameworks including MXNET, TensorFlow, and
Intel RL Coach, SageMaker RL is suitable for a wide range of use cases.
Amazon SageMaker Neo allows machine learning models to train once and run anywhere in the cloud
and at the edge. Ordinarily, optimizing machine learning models to run on multiple platforms is extremely
difficult because developers need to hand tune models for the specific hardware and software
configuration of each platform. Neo eliminates the time and effort required to do this by automatically
optimizing TensorFlow, MXNet, PyTorch, ONNX, and XGBoost models for deployment on ARM, Intel,
NVIDIA, Cadence, Qualcomm, and Xilinx support. With Neo, a machine learning model optimized for a
cloud instance or an edge device can be produced with just a few clicks. Optimized models run up to two
times faster and consume less than one-tenth of the storage space of traditional models.
After training, SageMaker provides the model artifacts and scoring images to the developer for
deployment to EC2 or anywhere else. The developer then can specify the type and number of EC2
instances, and SageMaker takes care of launching the instances, deploying the model, and setting up
the HTTPS endpoint for your organization's application to achieve low-latency/high-throughput
inferences. Once in production, SageMaker manages the compute infrastructure to perform health
checks, apply security patches, and conduct other routine maintenance, all with built-in Amazon
CloudWatch monitoring and logging. Organizations pay for AWS compute and storage resources that
the model uses for hosting the Jupyter notebook, training the model, performing predictions, and
logging the outputs. Building, training, and hosting are billed by the second, with no minimum fees and
no up-front commitments.
SageMaker eliminates code updates to incorporate new models. Amazon SageMaker also includes
built-in A/B testing capabilities to help users test models and experiment with different versions to
achieve the best results.
Amazon SageMaker enables you to deploy Inference Pipelines so you can pass raw input data and
execute preprocessing, predictions, and post-processing on real-time and batch inference requests.
Inference Pipelines can be made up of any machine learning framework, built-in algorithm, or custom
containers usable on Amazon SageMaker. You can build feature data processing and feature
engineering pipelines with a suite of feature transformers available in the Spark ML and Scikit-learn
framework containers in Amazon SageMaker and deploy these as part of the Inference Pipelines to
reuse data processing code and for easier management of machine learning processes.
Finally, one of the best aspects of SageMaker is its modular architecture. Developers can use any
combination of its building, training, and hosting capabilities to fit the organization's workflow. With
SageMaker, developing and deploying a machine learning model can be as straightforward as
choosing a notebook template, selecting an algorithm, and then training, testing, and deploying the
model using the management service. The bottom line is that SageMaker provides an end-to-end
machine learning environment that can significantly accelerate and simplify the process of creating,
training, and deploying models into production applications.
There are three versions of the AWS Deep Learning AMIs. The first is a Conda-based AMI with
separate Python environments for deep learning frameworks created using Conda — a popular open
source package and environment management tool. The second is a Base AMI with GPU drivers and
libraries for developers to deploy their own customized deep learning models. These are preconfigured
environments that allow developers the freedom and flexibility to use the setup and tools they need to
accomplish their desired goals with less work and aggravation. Experienced machine learning
developers that are already familiar with machine learning frameworks and the tools necessary to build
machine learning applications can use these AMIs as a way to deploy applications more quickly.
Today, many organizations that weren't early adopters of machine learning are unsure as to where the
technology will deliver the best business benefits. They're also unsure about what skill sets are needed
to build and deploy AI-enabled applications. Finally, organizations are still trying to determine the right
tools, infrastructure, and environments that are needed to put these AI-enabled applications to use.
Organizations need guidance about what types of tools and technologies can help them develop
AI-enabled applications. They also need to understand when, why, and how these applications will
be most effective in their organizations. In addition, organizations need to measure the effectiveness
of these applications to determine return on investment (ROI) for future projects that will include
deep learning.
Finally, the AI platform-as-a-service market is already crowded and is becoming more competitive with
every passing day. The need and desire for better (and simpler) tools, quicker time to market, and
efficiency are key concerns in the market for AI-enabled applications. There are numerous established
and emerging vendors addressing and providing services and solutions within this space at a very wide
range of capabilities. As such, Amazon Web Services faces the challenge of continuing as a leader in
this market and will need to maintain an aggressive pace of engineering and innovation. Although AWS
is productizing machine/deep learning services as the foundation of its solutions, this approach is also
not new to this market. What is new is that managed services like SageMaker and the AWS Deep
Learning AMIs combine numerous deep learning tools, frameworks, and technologies into a single
integrated platform that provides significant productivity enhancements for organizations and developers.
AWS needs to keep providing this level of innovation and expertise in this emerging market.
Critical success factors related to machine/deep learning implementation are related to people,
process, and technologies. Traditionally, emerging technical solutions require sharp and motivated
developers that like to live on the cutting edge of technology. However, cloud vendors are finding ways
to democratize the development and use of AI and deep learning technologies to promote wider use
and deployment within enterprises. The key is to develop successful models and products based on
deep learning quickly. Some factors that can assist with this are:
▪ Quick start packages/development tools. Some vendors offer templates, sample data, and
sample code to help jump-start developer productivity. With managed services like Amazon
SageMaker, data scientists and developers (and even nondevelopers) can be even more
productive than they could be with just templates and sample code.
▪ Assistance with data. Some vendors are either providing curated third-party data sets or
evaluating doing so to assist developers with creating the kinds of cutting-edge predictive and
prescriptive machine learning models that customers are looking for. For those that have their
own data, many cloud vendors are now also offering data curation and integration services
that make creating well-formulated data sets easier. AWS also hosts a number of open public
data sets on a variety of topics (geospatial, environmental, genomics, life science, regulatory,
and statistical data) as well as general-purpose data for machine learning such as images,
web crawls, news, and web video.
▪ Education. Providing training and courses on how developers can best make use of these
tools allows developers to get up and running without having to do everything themselves.
With machine/deep learning services like Amazon SageMaker, a little education can make
nontraditional developers and data scientists productive and able to build and deploy their own
deep learning models. For example, AWS offers machine learning training at no cost, using
the same curriculum used to train Amazon's developers and data scientists. Developers can
also get hands-on learning using AWS DeepRacer, a fully autonomous 1/18th-scale race car
designed to help developers learn about reinforcement learning through autonomous driving.
▪ Consulting and advisory services. These services will help developers become productive
and will help them with challenges related to the kinds of data that they're consuming.
An example of this is the new AI and machine learning competency for the AWS Partner
Network. Amazon is certifying partners in machine/deep learning with this program today.
In addition, AWS has created the Amazon Machine Learning Solutions Lab to help
organizations develop AI applications more quickly and easily. The Solutions Lab pairs
enterprise teams with AWS machine learning experts to prepare data, build and train models,
and put models into production.
Great AI-enabled applications require both advanced technology and solid design judgment.
Organizations should make sure the AI-enabled solution that they're building will be able to help
achieve the desired business outcome and/or address the issues that it is planned to overcome
utilizing deep learning. Engage in-house subject matter experts, the right stakeholders, and consulting
partners to help develop the right use cases to align with the desired business outcome. Make sure to
include past project experiences in the organization's design thinking approach and, if available,
include predefined use cases that have been developed for peers within the organization's industry to
help develop the optimal use cases for the desired outcome. This process should involve continuous
innovation and prototyping until the right use cases have been developed.
Global Headquarters
5 Speen Street
Framingham, MA 01701
USA
508.872.8200
Twitter: @IDC
idc-community.com
www.idc.com
Copyright Notice
External Publication of IDC Information and Data — Any IDC information that is to be used in advertising, press
releases, or promotional materials requires prior written approval from the appropriate IDC Vice President or
Country Manager. A draft of the proposed document should accompany any such request. IDC reserves the right
to deny approval of external usage for any reason.