8 BOOKS IN 1

PYTHOh
PROGRAMMING
FOR BEGINNERS

CHAD KNOWLES
PYTHON PROGRAMMING FOR
BEGINNERS

8 in 1

The Ultimate Step-by-Step Guide to Create Your

Business Projects Immediately

by

CHAD KNOWLES
 © Copyright 2022 - All rights reserved.
The content contained within this book may not be reproduced, duplicated or
transmitted without direct written permission from the author or the
publisher. Under no circumstances will any blame or legal responsibility be
held against the publisher, or author, for any damages, reparation, or
monetary loss due to the information contained within this book. Either
directly or indirectly.
Legal Notice: This book is copyright protected. This book is only for
personal use. You cannot amend, distribute, sell, use, quote or paraphrase any
part, or the content within this book, without the consent of the author or
publisher.
Disclaimer Notice: Please note the information contained within this
document is for educational and entertainment purposes only. All effort has
been executed to present accurate, up to date, and reliable, complete
information. No warranties of any kind are declared or implied. Readers
acknowledge that the author is not engaging in the rendering of legal,
financial, medical or professional advice. The content within this book has
been derived from various sources. Please consult a licensed professional
before attempting any techniques outlined in this book.
By reading this document, the reader agrees that under no circumstances is
the author responsible for any losses, direct or indirect, which are incurred as
a result of the use of information contained within this document, including,
but not limited to, errors, omissions, or inaccuracies.
TABLE OF CONTENTS
BOOK 1 - PYTHON
PROGRAMMING
lIMVd
 Introduction

Although Python is more known as a programming language, it has become a

consistently popular tool for data analytics. In recent years, several libraries
have reached maturity thereby permitting Stata and R users to take advantage
of the flexibility, performance, and beauty of Python without having to
sacrifice the functionalities gathered by the older programs over the years.
In this book, we will take a look at introducing the social science and data
analysis applications of Python. This book is particularly tailored for those
users that have little or no programming experience of note. It will be
especially useful for these programmers who wish to get things done and
have a lot of experience in programs such as Stata and R.
The greatest reason for learning Python also happens to be the hardest to
explain to someone who is just beginning his work in Python. Python is
superbly designed in terms of structure and syntax; it is intuitive; however,
very powerful general-purpose programming language.
Python's main advantage lies in how easy it is. For these kinds of things, you
need an easy language. A harder one will generally take quite a large toll on
your ability to program and analyze data.
Because of this, taking a difficult language will show off your programming
skills, but not your data analytics skills. In the end, there's a reason many of
the top companies in the world use Python as their main programming
language. It lets the coders focus on what's important instead of looking up
syntax every 20 minutes.
The programming language was explicitly designed; therefore, the code
written in the language is simple for humans to read and reduces the amount
of time needed for writing the code. Its ease of use is the main reason why
most of the top CS programs in the US use Python to introduce computer
science in their classes according to a recent study.
Having said all that, Python is very real and is a general-purpose
programming language. You will find major companies such as Dropbox and
Google using Python for their core applications. This sets the programming
language apart from other domain-specific languages such as R which are
highly tuned to cater to a specific purpose such as statistics and they work for
specific audiences. R was created by John Chambers with the target of
making a language that even the non-programmers can learn to use quickly,
and it could also be utilized by the power users. He succeeded in his endeavor
to a large degree as can be seen from the uptake of R. However, in the
process of making the programming language more accessible to the non­
programmers, some compromises had to be made in the language. R just
serves one purpose, and that is statistical analysis, and its syntax contains all
kinds of peculiarities and moles that come with the original bargain.
Python, on the other hand, needs some training to get started, although not a
great deal more. However, there are no limits to what you can do by using
Python; when you are learning Python, you are learning a full programming
language. It means if you have to work in another programming language
such as C or Java for some reason or have to understand pieces of code
written by somebody else or in some cases have to deal with programming
problems, this learning background in programming will provide a solid
conceptual foundation for anything you will come across. This is the reason
why all the top CS programs teach Python.
There are many reasons for choosing Python as your tool, but we have not
touched on the most compelling reason for them all. Python will set you up
for understanding and operating in the broad programming world. In case you
are interested in performing computational social science and building
general programming skills, Python gives you more flexibility. If you are
looking to run just the regressions R is great or if you are doing things that fit
the mold perfectly because someone has created the molds by using R
functions. However, social scientists will find new data sources such as text
and find newer ways of analyzing it. So the better you are at a general
programming language, the more prepared you are for stealing the tools from
other disciplines and write newer tools by yourself.
Most experienced programmers will find the idea of using a single
programming language extremely appealing; it allows you to unify your
workflow. For everyone, one of the best things about Python is that you can
pretty much do anything you wish by using this programming language.
However, everyone doesn’t feel that way. There are a lot of people who use
Python with other tools such as R and move back and forth depending on the
application at hand. However, even in case you are planning to do this mix
and match, the great thing about Python is that due to its generality several
people have suggested that becoming better at Python has turned them into
better programmers. Not only in Python but also on Stata and R.
Performance is not a criterion that comes into play for the majority of social
science applications. Therefore, it is not the top reason for selecting Python.
But in case you find yourself in a situation where performance does matter,
Python has some significant advantages over all the other high-level
languages including R and Mat lab, both in terms of memory use and
computation speeds. R is notorious for being a memory hog. More
significantly, there are new tools available in Python which make it possible
for writing Python code which runs at the same speed as that of FORTRAN
or C. Sometimes even faster than native Python or R. Although this is a
lower-level consideration in most cases, it is an example of the advantages of
using Python giving you options which will hold no matter what the future
will bring.
 Chapter 1:

Understanding Python: A Detailed

Background

What Is Python
There are many different coding languages you can work with. The option
you choose often depends on the amount of experience you have, along with
what you are trying to do within the code you want to make good. Some
coding languages will be a bit more advanced, some will work best with
some websites, and some will work best with some operating system. Each of
these coding languages offers you some advantages, and choosing a
language for beginners seems like a big challenge for a beginner.
Even though there are a lot of options that come with coding languages, you
will find that Python is one of the best options whether you are a beginner or
more advanced in coding. You will find that the Python language is easy to
use, while you can still really work on some high-quality encodings, without
posing all the challenges for a beginner. In fact, this coding language is often
a language recommended to those who are new to coding and who have
never been able to work with any kind of coding in the past.
There are many things to love about this type of language. It is easy to work
and learn, even with all the power that comes with it. You can write codes in
no time and the wording is in English, unlike some of the other options you
can choose there, making it a little easier to work with in general. And the
other tools, like having some good libraries, help from many other people in
the coding world, free open source programming, and more make it the
perfect option when you first get started with this kind of language.
You will notice that the Python language offers many advantages. The first
benefit we'll look at is the supporting libraries. You will notice that just by
opening the Python language, there are many options available in the library.
And you can view third-party libraries and extensions that can be easily
added to this coding language.
There are many different benefits that will help you see real results with the
Python code. You will first enjoy that it becomes a great option because of all
the options that work in the library. The library will have many useful codes
and options that you can work with, making it easier for the beginner to get
started.
Python is technically an interpreted programming language that came into
existence almost 3 decades ago. This wasn’t even the first language to come
out, as there had been many others in existence and in use by a large number
of computer and software experts. There were C, C++, Java, and so many
other major names which are still considered as all-time greats. However,
writing codes using these languages often caused more problems, especially
for people like us who had no idea about programming language.
Imagine receiving the entire project to learn from and analyze. Coded by
another programmer, it would pose quite a lot of challenges for you to read,
let alone understand what the programmer had tried to achieve using this
gigantic code.
This was becoming more than your ordinary issue to deal with. More and
more computer enthusiasts would spend days, weeks, and even months just to
come to terms with what the programmer had done. Similarly, one person, a
Dutch programmer named Guido Van Rossum, felt the need to have another
language that could simplify things for everyone and make writing code a
little more efficient. Little did he know that he would soon go on to become a
living legend.
With code readability in mind, Mr. Rossum set out on a quest to come up
with a programming language that would act as an interpreter between other
popular languages in existence back then. With little tools available at his
disposal in the 80s, he conceived the idea of Python and started working on
it. After much trial and error, the first implementation of Python came into
existence in 1989 and thus Python was born. Two years later, the language
was released for the masses and continues to this day to be one of the leading
languages every young aspirant wishes to learn and master.
The efforts were made to make the reading coded language easier, and that is
exactly what Python achieved. No longer did programmers have to spend
such vast spans of time. Now the code seemed much easier to read. Python
also became the pinnacle of the “Clean Code” practice.
Mr. Rossum soon had a huge following for his work on Python and was
crowned by the community as the “Benevolent Dictator for Life (BDFL).” It
was just in 2018 that he took a “permanent vacation” from his position and
allowed other members to share the responsibility of leading Python into the
future. A “Steering Council” was formed, where he was one of the five
members to lead the project and continues to do so.
As of 2019, Python stands as one of the most sought after programming
languages on earth. It is well on its way to surpassing both Java and C to
become the most popular programming language of all time. There should be
no surprise if that happens anytime soon owing to the ease and the super­
efficient nature of the language.
Many young programmers are drawn towards Python for its user-friendliness,
which provides all beginners a perfect environment to learn and master.
Using a text editor, such as Py Charm by Jet brains, you can write codes with
ease and in ways that anyone can understand.

Features Of The Python

Python has the following features:
Large library: It works with other programming projects like searching texts,
connecting to the web servers, and exchanging files.
Interactive: Using the Python is very easy as you can easily test out codes to
determine if they are working.
It is a free software; thus you can download it from the internet at any time
using your computer.
Python programing language can be extended into other modules such as
C++ and C.
Has elegant syntax making it easy for the beginners to read and use.
Has a variety of basic data types to choose from.

Who Uses Python?

To quite a few, this list would surely sound surprising, as many of these
involve great sites and social media platforms we may be using in our daily
lives. This just goes to show the scope of Python and how much potential this
language has to offer for those who are willing to learn it. Let us look at some
of these applications and websites to get a good idea of the popularity of
Python.
• Quora - A widely famous social website where you can literally ask any
question and experts will help you out with answers from across the globe.
Quora is one of the finest examples on the Internet that relies heavily on
Python to deliver the kind of experience we all have grown accustomed to
and love.
• Reddit - Another fine example where millions of users from around the
world have signed up to create one of the most active online communities of
recent times. With hundreds of thousands of posts being published every day,
this site and its app are kept running with the help of Python. Initially, Reddit
was written in another language called the Common Lisp. Later, for more
development and accessibility, Python was used to rewrite and recreate what
we see and use today.
• Pinterest - This social platform needs no introduction. Simple, crisp, and
elegant, all thanks to Python.
• Mozilla - One of the most popular names when it comes to Internet
browsers. While they have further widened the use of Python for data
sciences, what we should understand here is that Python allowed Mozilla to
become what it is today.
• Instagram - Yes! Even Instagram uses Python. Imagine how lucrative it
would be for a programmer to learn Python and become a part of something
as big as Instagram or even create something unique from scratch. Python can
get that job done for you.
• YouTube - Arguably the busiest website on earth. Hosting hundreds of
millions of videos from billions of users throughout the world-wide-web, this
website uses Python to understand user browsing experience, gather valuable
data, and provide analysis to which it concerns. The rich experience we have
as users is all made possible by Python.
• Yelp - If you are someone from the US, you are likely to be familiar with
this name. This is where reviews can make or break a business. Keeping
everything in check and in order is Python, which is constantly working in
the background to enhance our experience as users on the website.
• Google - Yet another jewel in the crown. Google used Python to develop
its first ever search engine and wrote the entire stack with the help of Python.
If Google relies on something, it most definitely is important.
• Netflix - The popular streaming service that has taken the world by
storm. Netflix uses Python to deliver its iconic services to consumers.
• Spotify - This giant music streaming platform uses Python for data
analysis and other back end services.
• Dropbox - It is perhaps one of the most popular cloud storage services in
existence today. The engineers behind Dropbox use Python for their desktop
client. Even Mr. Rossum himself joined Dropbox as an engineer. That is
quite a statement.
Apart from these famous, larger than life companies, there are individuals
who are looking forward to using Python for their day to day work.
Python, apart from being a language that is easy to read and write, is also a
language that carves a path out for automation. This is where things get quite
interesting. Imagine you have to gather data from various sources and
compile them into a file, located on your desktop. You need to get the work
done fast and you only have a limited time to get this done. If you know
Python, you can write yourself some codes and let your program do all the
work for you. By the time you are back, the document will be ready for your
review. This is just the icing on the cake. The real world applications go far
beyond just data collection and compiling.
 Chapter 2:

Why the Name “Python”

Python was originally conceived by Van Rossum as a hobby language in

December 1989. Also, the major and backward-incompatible version of the
general purpose programming language was released on 3rd December 2008.
But Python is recently rated by a number of surveyors as the most popular
coding language of 2015. The massive popularity indicates Python’s
effectiveness as a modern programming language. At the same time, Python
3 is currently used by developers across the world for creating a variety of
desktop GUI, web and mobile applications. There are also a number of
reasons why the huge popularity and market share of Python will remain
intact over a longer period of time. Python is an awesome decision on
machine learning for a few reasons. Most importantly, it's a basic dialect at
first glance. Regardless of whether you're not acquainted with Python,
getting up to speed is snappy in the event that you at any point have utilized
some other dialect with C-like grammar.
Second, Python has an incredible network that results in great documentation
and inviting and extensive answers in Stack Overflow (central!).
Third, coming from the colossal network, there are a lot of valuable libraries
for Python (both as "batteries included" an outsider), which take
care of essentially any issue that you can have (counting machine learning).

History of Python
Python was invented in the later years of the 1980s. Guido van Rossum, the
founder, started using the language in December 1989. He is Python's only
known creator and his integral role in the growth and development of the
language has earned him the nickname "Benevolent Dictator for Life". It was
created to be the successor of the language known as ABC.
The next version that was released was Python 2.0, in October of the year
2000 and had significant upgrades and new highlights, including a cycle­
distinguishing junk jockey and back up support for Unicode. It was most
fortunate, that this particular version, made vast improvement procedures to
the language turned out to be more straightforward and network sponsored.
Python 3.0, which initially started its existence as Py3K. This version was
rolled out in December of 2008 after a rigorous testing period. This
particular version of Python was hard to roll back to previous compatible
versions which are the most unfortunate. Yet, a significant number of its real
highlights have been rolled back to versions 2.6 or 2.7 (Python), and
rollouts of Python 3 which utilizes the two to three utilities, that help to
automate the interpretation of the Python script.
Python 2.7's expiry date was originally supposed to be back in 2015,
but for unidentifiable reasons, it was put off until the year 2020. It was
known that there was a major concern about data being unable to roll back
but roll FORWARD into the new version, Python 3. In 2017, Google
declared that there would be work done on Python 2.7to enhance execution
under simultaneously running tasks.
Basic Features of Python
Python is an unmistakable and extremely robust programming language that
is object-oriented based almost identical to Ruby, Perl, and Java, A portion of
Python's remarkable highlights:
Python uses a rich structure, influencing, and composing projects that can be
analyzed simpler.
It accompanies a huge standard library that backs tons of simple
programming commands, for example, extremely seamless web server
connections, processing and handling files, and the ability to search through
text with commonly used expressions and commands.
Python's easy to use interactive interface makes it simple to test shorter
pieces of coding. It also comes with IDLE which is a "development
environment".
The Python programming language is one of many different types of coding
languages out there for you. Some are going to be suited the best to help out
with websites. There are those that help with gaming or with specific projects
that you want to handle. But when it comes to finding a great general-purpose
language, one that is able to handle a lot of different tasks all at once, then the
Python coding language is the one for you.
There are a lot of different benefits to working with the Python language.
You will find that Python is easy enough for a beginner to learn how to work
with. It has a lot of power behind it, and there is a community of
programmers and developers who are going to work with this language to
help you find the answers that you are looking for. These are just some of the
benefits that we get to enjoy with the Python language, and part of the reason
why we will want to get started with this language as soon as possible!
The Python programming language is a great general-purpose language that
is able to take care of all your computing and programming needs. It is also
freely available and can make solving some of the bigger computer programs
that you have as easy as writing out some of the thoughts that you have about
that solution. You are able to write out the code once, and then, it is able to
run on almost any kind of program that you would like without you needing
to change up the program at all.

Common Terms in the Python

Understanding the standard terms used in Python is essential to you. It makes
everything easy to know when getting started. The following are the most
common terms in the Python programming language;
v Function: Refers to a block of code, invoked when a programmer uses
a calling program. Its purpose is to provide free services and accurate
calculation too.
v Class: A template used for developing user-defined objects. It is
friendly and easy to use by anybody including beginners.
v Immutable: Refers to an object assigned a fixed value and is located
within the code. It can be numbers, strings, or tuples. Such an object cannot
be altered.
v Docstring: Refers to a string that appears inside the function, class
definition, and module. This object is always available in the documentation
tools.
v List: Refers to datatype built within the Python and contains values
that are sorted. Such values include strings and numbers.
v idle: Stands for an integrated development environment that allows
the users to type in the code as you interpret and edit it in the same window.
Most suitable for the beginners as it is an excellent example of code.
v Interactive: Python has become the most appropriate programming
language to beginners because of its interactive nature. As a beginner, you
can try out many things in the IDLE (interpreter) to see their reaction and
effects).
v Triple Quoted String: the string helps an individual to have single and
double quotes in the string, making it easy to pass through different lines of
code.
v Object: It refers to any data within a state like attitudes, methods,
defined behavior, or values.
v Type: Refers to a group of categories of data in the programming
language, and differ in properties, functions, and methods.
v Tuple: Refers to the datatype build into the Python and is an
immutable sequence of values, though it contains some mutable values.

How Is Python Used?

Python is one of the best programming languages that is a general-purpose
and is able to be used on any of the modern operating systems that you may
have on your system. You will find that Python has the capabilities of
processing images, numbers, text, scientific data, and a lot of other things that
you would like to save and use on your computer.
Python may seem like a simple coding language to work with, but it has a lot
of the power and more that you are looking for when it is time to start with
programming. In fact, many major businesses, including YouTube, Google,
and more, already use this coding language to help them get started on more
complex tasks.
Python is also known as a type of interpreted language. This means that it is
not going to be converted into code that is readable by the computer before
the program is run. Instead, this is only going to happen at runtime. Python
and other programming languages have changed the meaning of this kind of
coding and have ensured that it is an accepted and widely used coding
method for many of the projects that you would like to handle.
There are a lot of different tasks that the Python language is able to help you
complete. Some of the different options that you are able to work with
include:
 1. Programming any of the CGI that you need on your web
applications.
2. Learning how to build up your own RSS reader
3. Working with a variety of files.
4. Creating a calendar with the help of HTML
5. Being able to read from and write in MySQL
6. Being able to read from and write to PostgreSQL

The Benefits of Working with Python

When it comes to working with the Python language, you will find that there
are a lot of benefits with this kind of coding language. It is able to help you to
complete almost any kind of coding process that you would like and can still
have some of the ease of use that you are looking for. Let’s take a quick look
at some of the benefits that come with this kind of coding language below:
Beginners can learn it quickly. If you have always wanted to work with a
coding language, but you have been worried about how much work it is
going to take, or that it will be too hard for you to handle, then Python is the
best option. It is simple to use and has been designed with the beginner in
mind.
It has a lot of power to enjoy. Even though Python is easy enough for a
beginner to learn how to use, that doesn’t mean that you are going to be
limited to the power that you are able to get with some of your codings. You
will find that the Python language has the power and more that you need to
get so many projects done.
It can work with other coding languages. When we get to work on data
science and machine learning, you will find that this is really important.
There are some projects where you will need to combine Python with another
language, and it is easier to do than you may think!
It is perfect for simple projects all the way up to more complex options like
machine learning and data analysis. This will help you to complete any
project that you would like.
There are a lot of extensions and libraries that come with the Python
language, which makes it the best option for you to choose for all your
projects. There are a lot of libraries that you are able to add to Python to
make sure that it has the capabilities that you need.
There is a large community that comes with Python. This community can
answer your questions; show you some of the different codes that you can
work with, and more. As a beginner, it is always a great idea to work with
some of these community members to ensure that you are learning as much
as possible about Python.
When it comes to handling many of the codes and more that you would like
in your business or on other projects, nothing is going to be better than
working with the Python language. In this guidebook, we will spend some
time exploring the different aspects of the Python language, and some of the
different things that you are able to do with this coding language as well.
 Chapter 3:

Python Glossary

These are Python glossary of terms you must know:

1>>>
This is the default prompt of the Python interactive shell. We have seen this
lot in our examples.
2. ...
The default prompts of the Python interactive shell when entering code under
an intended block or within a pair of matching delimiters. Delimiters may be
parentheses, curly braces, or square brackets. This is also called the ellipsis
object.
3. 2to3
While most of the applications existing today have their base in Python 2.x,
the future belongs to Python 3.x. But 2.x code isn’t completely compatible
with 3.x.Interestingly; we have a tool available that will help us convert
Python 2.x code to Python 3.x. 2to3 handles the incompatibilities, detecting
them by parsing the source and traversing the parse tree. The standard library
has this as lib2to3.
4. Abstract Base Class
An abstract base class provides a way to define interfaces. This way, it
compliments duck typing. For this, we have the module ABC. It introduces
virtual subclasses (classes that are recognized by is instance () and is subclass
(), but do not inherit from another class. Python has several built-in ABCs for
data structures (use the collections.abc module), numbers (use the numbers
module), or streams (use the IO module). You can also import finders and
loaders (use the importlib.abc module). And to create our own ABCs, we use
the ABC module.
5. Python Argument
An argument is a value we pass to function or a method when calling it. In
Python, we have the following kinds of arguments:
A. Default arguments
When defining a function, we can provide default values for arguments. This
way, when we call it without any missing arguments, the default values will
fill in for them. Default arguments can only follow non-default ones.
B. Keyword arguments python
Keyword arguments pertain to calling a function. When we then call the
function, we can pass it arguments in any order.
C. Arbitrary arguments
When we don’t know how many arguments we’ll get, we use an asterisk to
denote an arbitrary argument.
D. Positional arguments python
These are regular arguments that aren’t keyword arguments. Python
positional argument Example.
7. Asynchronous Context Manager
ACM is an object that controls the environment observed in sync with a
statement.
8. Asynchronous python generator
We have seen about generators in python. They let us yield one object at a
time.
An asynchronous generator is a function that returns an asynchronous
generator iterator. We define it with ‘async def’, and it contains ‘yield’
expressions to produce a series of values. We can use these values in an
async for-loop. Such an asynchronous generator function may contain await
expressions, and async for and async with statements.
9. Asynchronous Generator Iterator
An asynchronous generator function creates an asynchronous generator
iterator. When we call this iterator using the __anext__ () method, it returns
an await able object. This object executes the function’s body until the next
yield expression.
Actually, each yield suspends processing temporarily. It remembers the
location execution state, and the local variable and pending try statements.
On resuming with another await able return __anext__(), the generator
iterator picks up where it left off.
10. Asynchronous iterable
It an object that we can use in an async statement, it must return an
asynchronous iterator from its__ aiter__ () method.
Any doubt yet in python glossary? Please comment.
11. Asynchronous Iterator
An asynchronous iterator is an object that implements __ aiter__ () and
__ anext__ () methods.__ anext__ () must return an await able object. Async
for resolve the await able return from the iterato’s __anext__() method until it
raises a stop async iteration exception.
12. Attribute
An attribute is a value an object holds. We can access an object’s attributes
using the dot operator (.). In our examples, we have done this as following:
13. Await able
Any object in Python that we can use in an await expression is an await able.
It can be co-routine or any object with an __await_() method.
14. BDFL
Who other than Guido Van Rossum, the creator of Python, deserves to be
called Benevolent Dictator for Life?
15. Binary File
A file object that is able to read and write bytes-like objects is a binary file.
When we open a file in a binary mode, we use the modes ‘rb’, ‘wb’, or ‘rb+’.
More on File I/O.
16. Bytes-Like object
Any object that supports the Buffer Protocol, and is able to extort a C-
contagious buffer, is a byte-types like objects. Examples include bytes, byte
array, and аггау.аггау objects. It also includes many common memory view
objects.
We can use such objects for operations that deal with inary data
(compression, saving to a binary file, sending over a socket, and more)
17. Bytecode
As you know, Python compiles its source code into bytecode. It is the internal
representation of a Python program in the C Python interpreter. When we
talked earlier of .рус files, we mentioned that bytecode is cached into them.
This lets the files execute faster the second time since they don’t need to
recompile. In essence, bytecode is like an intermediate language that runs on
a virtual machine. This virtual machine converts it into machine code for the
machine to actually execute it on. However, one bytecode will not run on a
different virtual machine. If you’re interested in finding out about bytecode
instructions, you can refer to the official documentation for this module.
18. Python Class
A class in Python is a template for creating user-defined objects. It is an
abstract data type, and acts as a blueprint for objects of a kind while having
no values itself. To learn how to create and use a class, refer to Classes in
Python.
19. Coercion
When we carry out operations like 2+3.7, the interpreter implicit converts one
data type to another. Here, it converts 2 to 2.0 (int to float), and then adds, to
it, 3.7. This is called coercion, and without it, we would have to explicitly do
it this way:
>>> float(2)+3.7
5.7
20. Complex Number
A complex number is made of real and imaginary parts. In python, we use ‘j’
to represent the imaginary part.
>>> type(2+3.7j)
<class ‘complex’>
An imaginary number is a real multiple of -1(the imaginary unit). To work
with complex equivalents of the math module, we use c-math. For more on
complex numbers, read up on Python Numbers. These Python glossary terms
are very important to know before you dive into learning Python.
21. Context Manager
The context manager is an object that controls the environment observed in a
with-statement. It does so with the __enter__() and __exit__() methods.
22. Coroutine
A subroutine enters at one point and exits at another. A coroutine is more
generalized, in that it can enter, exit, and resume at many different points. We
implement them with the async def statement.
23. Coroutine Function
A coroutine function is simply a function that returns a coroutine object. We
may define such a function with the async def statement and it may contain
the keywords await, async for, and async with.
24. CPython
СРуШоп is the canonical implementation of Python in C. It is the one
distributed on python.org.
25. Python Decorator
A decorator is a function that returns another function, or wraps it. It adds
functionality to it without modifying it. For a simple, detailed view on
decorators, refer to Python decorators.
26. Descriptor
If an object defines methods __get__(), __set__(), or __delete__(), we can
call it a descriptor. On looking up an attribute from a class, the descriptor
attributes special binding behavior activates. Using a.b looks up the object ‘b’
in the class dictionary for ‘a’. If ‘b’ is a descriptor, then the respective
descriptor methods are called.
27. Python dictionary
A dictionary is an associative array that holds key-value pairs. Think or a
real-life dictionary. Any object with __hash__() and __eq__() methods can be
a key.
28. Dictionary view
A dictionary view is an object returned from dict.keys(), dict.values(), or
dict.items(). This gives us a dynamic view of the dictionary’s entries. So,
when the dictionary changes, the view reflects those changes.
29. Docstring
A docstring is a string literal that we use to explain the functionality of a
class, function, or module. It is the first statement in any of these constructs,
and while the interpreter ignores them, it retains them at runtime. We can
access it using the__ doc__ attribute of such an object. You can find out more
about docstrings in Python Comments.
30. Duck-Typing
We keep saying that Python follows suck-typing. But what does this mean?
This means that Python does not look at an object’s type to determine if it has
the right interface. It simply calls or uses the method or attribute. “If it looks
and quacks like a duck, it must be a duck.”
This improves flexibility by allowing polymorphic substitution. With duck­
typing, you don’t need tests like type() or isinstance(); instead, you use
hasattr() tests or EAFP programming.
31. EAFP Programming
EAFP stands for easier to ask for forgiveness than permission.
This means that Python assumes the existence of valid keys or attributes, and
catches exceptions on the falsity of the assumption. When we have many try
and except statements in our code, we can observe this nature of Python.
Other languages like C follow LBYL (Look Before You Leap).
32. Python Expression
An expression is a piece of code that we can evaluate to a value. It is an
aggregation of expression elements like literals, names, attribute access,
operators, or function calls. All of these return a value. An if-statement is not
an expression, and neither is an assignment, because these do not return a
value.
33. Extension Module
An extension module is one written in C or C++, using Python’s C API to
interact with the core, and with user code.
34. f-string
An f-string is a formatted string literal. To write these, we precede a string
with the letter ‘f’ or ‘F’. This lets us put in values into a string.
>>> name,surname='Ayushi','Sharma'
>>> print(f"I am {name}, and I am a {surname}")
I am Ayushi, and I am a Sharma
For more on f-strings, read up on Python Strings.
35. File Object
A file object, in Python, is an object that exposes a file-oriented API to an
underlying resource. Such an API has methods such as read () and write ().
We also call them file-like objects or streams, and have three categories:
Raw binary files
Buffered binary files
Text files
The canonical way to create a file object is to use the open () function. For
help with reading and writing files, refer to Reading and Writing Files in
Python.
36. Loop
A collection of commands that the program will repeat a certain number of
times.
37. Module
A type of file that contains various functions, classes and variables that can
be used in any program once imported.
38. Parameter
A variable that is attached to a function within its definition.
39. Range
A collection of values found between a minimum and a maximum value.
40. Shell
A command line user interface that reads and executes your commands
directly. IDLE is an example of a shell.
41. String
A character sequence that can form words or sentences. They include letters,
symbols, numbers, as well as spaces.
42. Syntax
The programming structure or rules of a certain coding element. In a way, it
is the grammar of programming.
43. Variable
A value with a name, which can always change inside the program.
 Chapter 4:

Python Installation: How To Install

Python?

Installing Python
Since many aspiring data scientists never used Python before, we’re going to
discuss the installation process to familiarize you with various packages and
distributions that you will need later. Before we begin, it’s worth taking note
that there are two versions of Python, namely Python 2 and Python 3. You
can use either of them, however Python 3 is the future. Many data scientists
still use Python 2, but the shift to version 3 has been building up gradually.
What’s important to keep in mind is that there are various compatibility
issues between the two versions. This means that if you write a program
using Python 2 and then run it inside a Python 3 interpreter, the code might
not work. The developers behind Python have also stopped focusing on
Python 2, therefore version 3 is the one that is being constantly developed
and improved. With that being said, let’s go through the step by step
installation process.

Step By Step Setup

To set up our workstation, we need to download Python 3 and Visual Studio
Code. While Python 2 and 3 are quite similar in syntax, there are some
discrepancies in some code keywords and operations, such as the print
statement, division, unicode, range/xrange, and error handling. Additionally,
Python 2 has become obsolete and discontinued in 2020, and as such, it is
more logical to use the newer, more advanced Python 3 that encompasses a
richer yet similar syntax and modern libraries.
We will begin with the Python 3 installation. To start, in your browser, go to
https://www.python.org/downloads and click the yellow Download Python
3.x.y button. Python’s website should automatically detect your operating
system, but if it doesn’t then select your operating system below the yellow
button.

Save this installer where you would like and open it. You should arrive at the
beginning of the installer. Make sure to fill the box for Add Python 3.* to
PATH. Finally, click Install Now. The installer may look a little different in
mac OS, but the installation process should be very similar to this.

Next, we will install the Visual Studio Code. Navigate in your browser to
https://code.visualstudio.com and select the Download for Mac/Windows
button. Save the installer/package to your download. If you have mac OS,
extract the zip file and drag the Visual Studio Code application to the
Applications Folder.
If you have Windows, run the installer and keep clicking ‘Next’ until you
reach the following screen. When you reach this, select all of the options.
Complete the remaining portion of the installation.
Now, open VS Code. Select the fifth icon from the top on the left sidebar.
This is called the Extensions Window.
Type into the search bar, “Python”. Select the Python made by Microsoft.
Click the green “Install” button and wait for VS Code to finish installing
Python. You previously installed the Python support files from their website,
but this Python module that you are now installing is simply a support tool
for VS Code so you can edit Python files here.
Now, close out of the tab by clicking the “X” on the tab. On the menu bar,
click “File” and select “New file”. Call this program MyFirstProgram.py.
Make sure that the file extension is .py.
Now comes the tricky part. We have a Python file that we can edit, but we
need to make sure that VS Code is using the correct interpreter. It may have
automatically set the interpreter as none or Python 2, so we will need to
change this. Additionally, we need to set up a linter (a utility that
automatically highlights syntax errors) for debugging.
First, let’s define the interpreter. On Windows, type the keystroke CTRL +
SHIFT + P (On mac OS, this is CMD + SHIFT + P), and search for Python
interpreter in the search bar. Select the following option afterwards.

Select Python 3.8.0 32-bit (or 64-bit if that’s the version you installed). This
will make sure that VSCode uses a Python 3 interpreter when compiling and
executing your code.

Next, type the keystroke CTRL + SHIFT + P (CMD + SHIFT + P for

macOS) again, and search for Python: Select Linter. Select the option that
pops up below.

Select pylint at the bottom of the search results.

pylint is the specific linter we are going to use when coding. Once you select
this, a dialog box may pop up alerting you that the linter has not been
installed.
 (x) Linter pylint is not installed. 0 x

Source: Python (Extension) Install Select Linter Do not show again

If so, just select Install and wait for the linter to finish installing.
We are now ready to begin. From now on, when creating a new Python file,
all you need to do is create the new file in VS Code and all the options and
utilities we set up earlier will carry onto the new Python file. Now, let’s write
our first program!
PART 2
 Chapter 5:

Python Data Types:

Python Numbers, Python Strings

Python Labels
Before we dive into learning all the data types, let’s take a side step and
discuss labels. Writing code involves naming variables and objects
appropriately so that you can understand what you’re looking at immediately.
Labels, also known as identifiers, are words represent something in such a
way that it makes the code easier to read. For instance, if you’re talking about
a bottle of water in your code, you shouldn’t name the variable that
represents it as “var1”. That tells you nothing and it makes the entire code
confusing. You would have to waste a great deal of time until you figure out
which variable you’re talking about.
Whenever you name your variables, make sure they are well represented by
the label and that they are unique. Do not use the same name multiple times
or you will confuse yourself, and worse the program itself. Furthermore, you
should avoid similar words as well. However, you should take note that
certain words cannot be used as labels. These words are exceptions because
they are part of Python’s own library of keywords that are reserved for
various commands. You should read the language’s documentation in order
to learn which words are reserved, however, some of them are: global, while,
false, class, import and so on.
Keep in mind that using an IDE or even certain text editors can help you out
with writing proper labels. They can’t read your mind, but they will tell you
whenever you are trying to use a reserved keyword. This is another advantage
of using dedicated programming tools. You won’t have to keep a sticky note
with all the keywords attached to your monitor.
Finally, you should also consider the naming system you use. There are
several ways you can name your variables and they are all just as valid. For
instance, it is extremely common to begin the identifier with an underscore or
a letter. Just make sure you don’t start with a number. Furthermore, you need
to keep in mind the fact that Python is case sensitive. That means that if you
have a variable called “myVariable”, another one called “my_variable” and
“MYVARIABLE”, you have three different variables with nothing in
common.
Now that you know a bit more about making your code as readable as
possible, let’s finally get down to business and talk about variables and data
types.
Now that you have a basic understanding of Python’s behavior, let’s start
discussing the data types. In the next section we will focus on the most basic
one, which is the string.

Strings
The string is the most basic data type, along with numbers. You have actually
already used a string when you wrote your first program. The line of text you
printed is considered a string. Simply put, strings are sets of characters that
are defined between quotation marks. Keep in mind that text also includes
numbers and punctuation marks. Even though numbers are normally
classified under their own data types such as integers and floats, if you write
them between quotes, they are considered textual characters part of a string.
In your first program you had a single statement that was printed with the
print function. Keep in mind that you can also print any number of
statements, even in the same line, even if they are represented by several
variables. This is done with one of the most popular operations you will
perform on strings called concatenation. This concept is simple. All it
involves is linking multiple strings together. Here’s a simple example:
charRace = “human”
charGender = “male”
print (charRace, charGender)
The output will be “human male”.
As you can see, we have two variables and each one of them holds a string.
We can print both of them by separating the variables with commas when
writing the print statement. Keep in mind that there are multiple ways you
can do this. For instance, if you don’t want to use variables but you need to
concatenate the strings, you can get rid of the commas inside the print
statement. You will notice a little problem, though. Here’s the example:
print (“school” “teacher”)
The result is “schoolteacher”. What happened? We didn’t leave any
whitespace. Take note that whitespace can be part of a string just as numbers
and punctuation marks. If you don’t leave a space, words will be glued
together. The solution is to simply add one blank space before or after one of
the strings, inside the quotes.
Next, let’s see what happens if you try to combine the two methods and
concatenate a variable together with a simple string.
print (charRace “mage”)
This is what you will see:
File "<stdin>", line 1
print (characterGender “warrior”)
л SyntaxError: invalid syntax
Congratulations, you got your first syntax error. What’s the problem here?
We tried to perform the concatenation without using any kind of separator
between the two different items.
Let’s take a look at one more method frequently used to concatenate a set of
strings. Type the following:
x = “orc”
y = “ mage”
x+y
As you can see you can apply a mathematical operator when working with
string variables. In this case, we add x to y and achieve string concatenation.
This is a simple method and works just fine, however, while you should be
aware of it, you shouldn’t be using it. Mathematical operations require
processing power. Therefore, you are telling your Python program to use
some of your computer juice on an operation that could be written in such a
way as to not consume any resources. Whenever you work on a project, at
least a much more complex one, code optimization becomes one of your
priorities and that involves managing the system’s resource requirement
properly. Therefore, if you have to concatenate a large number of string
variables, use the other methods that don’t involve any math.

Numbers
Numbers, just like strings, are basic but frequently used no matter how
simple or complex your program is. Assigning a number to a variable is done
exactly the same as with any other data type. You simply declare it like so:
x = 10
y=1
Don’t forget that Python will automatically know which data type we’re
assigning. In this case, it identifies our values as integers. Integers are whole
numbers that can be either positive or negative. It cannot contain decimal
points.
Another numeric data type we mentioned earlier is the float. This is a float:
x = 10.234
Floats can be negative or positive numbers but they must have decimal
points, otherwise they’re just integers.
Finally, we have the third numeric data type, which is the boolean. This type
holds only two values. It can either be true or false, or in computer language
1 or 0. Booleans are normally used together with logical operators instead of
mathematical operators like integers and floats.
Basic Operators
Now that you know a few data types and got used to working with variables,
let’s start actually performing some handy operations. Variables that hold
integers or floats can be manipulated by using the most basic arithmetic
operators. For instance, you can subtract, add, multiply, and divide.
Whenever you work with these operators you will create an expression
instead of a statement. What does that mean? Expressions are essentially code
that has to be processed by the computer system in order to find the value.
Let’s take a look at some exercises you can play with:
apples = 10 + 2
bananas = 10 - 4
pears = 6 * 2
fruit = apples + bananas * pears
fruit
84
Did you perhaps forget your elementary math and expected to see 216 as the
result? That’s probably because you calculate the sum in your head and then
multiplied it. However, that’s not how this calculation works. Python
automatically knows which rules to follow and it processes each operation in
the order it is required.
As you can see, Python is capable of evaluating the expression and then
deciding which blocks need to be processed before other blocks. This order
that the programming language follows is called an operator precedence.
Always pay attention to basic mathematical and logical rules because if you
don’t, Python will. If you had something else in mind for your program for
instance, and you wanted the result to be 216, you need to write the operation
to reflect that. In order words, you need to calculate the sum first and then
multiply it.
In this example we only worked with integers in order to showcase the most
basic operators. However, if you would replace them with floats, the same
rules apply.
In addition, it’s worth mentioning that Python is capable of converting an
integer to a float or even to a string. Any number can be converted to an
integer by typing “int (n)”, or a float by typing “float (n)” or a string by
typing str (object name).
You’ll notice that these are functions because they follow the same structure
as the print function which you used earlier. Once we declare the function we
want to use, we need to place the value or variable or object in between the
parentheses in order to manipulate it. Here’s how these basic conversions
work:
float (13)
Result: 13.0
int (13.4)
Result: 13
Now that you know the basics, you should start practicing on your own.
Create a few different variables and think of all the operations you can
perform with them. Don’t forget that reading is not enough and you should
always take the extra time to practice.
 Chapter 6:

Python Variables

Python Variables
Another name for Python identifiers is variable. Variable is a term used by a
computer or system to mean a memory region. You don't have to determine
these kinds of factors in Python as Python is a kind of infers programming
language and is astute enough to figure out its variables.
Also, we can say that Python's variables are locations of memory, having
different types of data, such as Integer or character. Variables in Python can
be changed and manipulated simply because they use a set of various
operations.
In any case, variables need to get initialized by a letter or an underscore. The
use of lower-case letters as the names of the element is suggested. Both are
excellent elements at Python, sled, and Mallet.
In Python, the variable definition is handled in two steps. The first step is
called the initialization and it refers to determining the container which is
identified via a label. The second step involves the assignment, which means
you attach a value to your variable and therefore determine the type of data it
holds. These two steps are actually taken at the same time and the process is
more of a theoretical one that you don’t really notice. Here’s how all of this
can be formulated:
myVariable = thisValue
The two steps are taken through the equal operator. What we have here is
called a statement, in this case an assignment statement. When you write
code, you should always keep your statements in proper order. Keep in mind
that Python processes code by analyzing it from the top to bottom and then it
starts over. Furthermore, you could write the statements in the same line,
however, that would lead to a lot of chaos.
Now that you know what a variable is, let’s see how Python is able to
determine which data type is assigned to the variable.
Python has this feature called dynamic typing, which means that it is able to
automatically determine what kind of variable it is dealing with. So if you
assign an integer to a variable, Python automatically knows that the variable
has an integer data type. When working with other programming languages,
you have to declare what type of data your variable will contain. Therefore, if
it’s an integer you have to declare it first and then assign an integer value to
it. This is another advantage of working with Python. All you have to do is
write your code without worrying too much about the details. Let the
interpreter do the heavy lifting for you. Furthermore, if you make the mistake
of performing an operation on a data type that is not eligible for that
operation, Python will let you know. However, there is one disadvantage
when relying on this system. Because you don’t have to declare your
variable’s data type, as a beginner, you might accidentally create a variable
when you don’t need one, or assign the wrong data type to it.

Nominating Variables or Identifiers

Identifier situations are factors. The actual coefficients and the integers used
in the software are constructed using a variable. The standards for naming a
variable for Python are given below.
An identifier's essential character must be either a letter or an underscore "_."
A letter arranged by lower case "a-z," capitals "A-Z," underlines, or digits "0­
9" can be used for each of the characters other than the essential aspects.
The name of a variable shall not contain any empty or void zone or any
individual or extraordinary character, such as! "@, #, percent, л, and, *."
The name of a variable must not imitate any catchphrase represented in the
syntax of your Python software.
Variables are case sensitive in Python. For instance, I'm cold and not
proportionate is I'm cool.
Considerable Identification instances: n696, v, v 69, etc.
Instances of invalid ids: 5a, v 69 percent, x69 etc.

Multiple Tasks
In a single explanation, Python allows one to maintain an incentive to
different identifiers, usually called different assignments. This is done by
using two methods, either by announcing a single reward at the same time for
multiple ids or by relegating various values at different times to numerous
variables.
Example - 1:
Open the Python console or IDE and write the command to declare variables.
>>> a=v=m=19
>>> print
>>> print (a, v, m)
Output:
The output will be something like this when you type the command to print
the value of variables.
>>> 19, 19, 19
>>>
Example - 2:
>>> a, v, m = 19, 24, 46
>>> print
>>> print (n)
>>> print (v)
>>> print (w)
Output:
For output,
When you will type your command
>>> print (a)
Your console will print "19".
When you will type your command
>>> print (v)
Your console will print "24".
When you will type your command
>>> print (m)
Your console will print "46".

In Python, there are many different types that are used for variables. What are
variables?

Variables are things that store information. A variable is like a box that
contains a ball. The box is not the ball, and is simply what points to it.
Likewise, the variable is only a title that points to data.

Variable

You’ve already seen variables in mathematics. For example, you could say
that there exist variables x and y such that
x = 25,
у = 42:
In this sense, x and y are the variable names, and 25 and 42 are their
respective values. It’s just like slapping a sticker on an item. The sticker is
used to identify the value of the item.
Likewise, in math, when you are given that
a = 25,
b = 25,
a — b
Then you would say that. This works the same with the data types that we
have currently learned.
There are three key components to working with variables: creating a
variable, modifying the variable, and getting the value of a variable. Let’s see
how this is done.

Creating a Variable
In Python, to create a variable, you would enter in some name (that is not
predefined by Python), and assign it to a value using an equal sign. Notice
how we say that we assign a variable. The value assigned to it should either
be a literal or an expression. A literal is a fixed value - in other words, a
value that is written exactly as it is meant to be interpreted. A literal value is
the actual value a variable corresponds to. So as an integer, for instance,
literals could be 4, 9, 13, and -25. A string literal would resemble something
similar to “What a nice day!”, where you have plain text enclosed either in
double quotes (“) or single quotes (‘). A boolean, as stated earlier, would
simply be True or False, and a float would be a number that contains a
decimal point.
# String literal

"Hello world!"

# Integer literal

25

# Float literal

49.2

# Boolean literal

True
An expression is something that can be computed. Examples of these include
mathematical expressions, or even expressions with strings. Expressions
simplify down to literals.
# Integer expression

13

(
4

5
)

# this evaluates to 4, a literal

The format for creating a variable in Python is:

variable_name = literal

Here are some examples of creating variables:

greeting
"Hello world!"

# str

myAge

15

# int

dinosaursExist

False

# bool

For each of the above statements, we are declaring a variable name and
initializing it with a literal.
greeting —— “Hello world!” (a string literal)
myAge —— 15 (an integer literal)
dinosaursExist —— False (a boolean literal)

Exercise
How to Create a Variable and Use it
Open the Python shell and create the following variables:
Create a variable named “course” and assign a value of “python” to it
Now create another variable named “rating” and assign an integer value to it
(any integer you like)
Print both of these variables
 Chapter 7:

Basic Operator Of Python Language

The next thing that we need to take a look at through here is some of our
operators. This will help us to go through and make sure that we are able to
get the most out of our process and can ensure that we can really see some
good results with the codes that we want to write as well. we will spend a bit
of time in this looks at how to handle these as we go through the process.
Let’s start out by learning a bit about how to work on these operators, and
then we can move on to some of the exercises that we can do with these as
well.

The Different Types Of Operators

The first operator type we are going to look at is the arithmetic operator. If
you think back to your high school math class, you will know what most of
these operators are like. These are the ones that are responsible for helping
you complete mathematical equations. It can be as simple as adding together
two of your operands in the code or subtracting two or three numbers from
each other.
Then we can move on to the comparison operators, which will take a
different option that can handle our codes, but they are so useful in all of this
as well. You can also work with your comparison operator when you create
your codes. This is a helpful option to work with because it allows you to
take two (and sometimes more0, values, and statements in your code and
compare them to each other. You will often need to bring out the Boolean
expressions for this because the answer you will get is either true or false. So,
your statements will either be the same as each other or not.
Next on the list is the logical operators. These may not be used as much as
the other two options, but it is still important to spend some time on them.
These operators are the ones that you will use to help evaluate the input that
the user gives you with any of the conditions that you set in your code. When
we work with some kind of code that needs to evaluate what is going on
along the way, then we want to bring out the logical operators.
And the last operator we are going to take a look at is the assignment
operator. This is the one that you have seen quite a bit in this guidebook
because you simply need to use the equal sign to take one of your values and
assign it to a variable. So, if you are looking to take the value of 100 and then
assign them to the chosen variable, then you just need to put the equal sign
between them. This is done with a lot of different values and variables in
your code and you should be pretty used to doing it at this point.

It is also possible to take several values and assign them to the same variable
if you choose. As long as you continue to use the equal sign, you can add as
many values to the same variable as you would like.
Working with these operators can be really simple, but they do make a big
difference when it comes to working on your code. You can add together
variables or choose some of the other mathematical operators. You can assign
a value over to your chosen variable, or even a few values to the same
variable. And you can even use these operators to compare two parts of the
code together and see if they are the same or not. There is just so much that
you can do with these operators and it is important to be on the lookout for
each one.
Even though all of these operators are slightly different from one another,
they are going to all come in and bring us some good codes. They are also
really simple to add to any code that you need without a lot of hassle, and as
we go through some of the activities that we need to do in this guidebook,
you will find that it can actually be pretty easy to work with them as well.

Exercise
These operators are usually pretty simple to work with so we will walk
through the different types and see how they are done. First, we will look at
the arithmetic operators. These are similar to what we would see when using
other coding languages and you can divide, multiply, subtract, and addition.
Try writing out a few of these operators, and then look at the example below.
number = 1 + 2 * 3 / 4.0
print(number)
Can you predict what the answer will be? If you run this program and it is
different than your prediction, consider whether you used the order of
operations and see if that gives you the right answer. Keep in mind that there
is one thing that is a bit different from this. If you work with two
multiplication symbols right next to each other, such as 2**2, then this is
going to be the power relationship instead.
Then we can move on to working with operators inside of some of the strings
we have. This is pretty easy to do. We will first look at how we can make one
of these strings with the help of the addition operator, and how that will fit
into this as well:
helloworld = "hello" + " " + "world"
print(helloworld)
There are even more fun things that we can do as we work through this. It is
possible to use these operators to help us out with some of the lists that we
want to create. These lists are simple enough to work with, and we talked
about them a bit earlier so we won’t spend too much time on them right now.
However, we can use the operators to help us join together two or more lists
if that is what we need to get done with our code. This is really simple to do,
and we can use the examples below to help us:
even_numbers = [2,4,6,8]
odd_numbers = [1,3,5,7]
all_numbers = odd_numbers + even_numbers
print(all_numbers)
To finish this off, we need to take a look at an actual exercise that we can
work with to help us understand how all of this is supposed to work. The
target of working on this is so that we can come up with two new lists as we
go through the whole process. We will have a y_list through this and then an
x_list as well, and then have ten instances of the x in the first one ad ten of y
in the second one. You will then need to take this further and create another
list known as the big_list, which will have the variables of x and y, both
showing up ten times, by concatenating the two lists that we already created.
Take a moment to try that out.
There are a lot of times when we will need to bring out these operators and
see how they work. And there are so many different types that we can use
along the way as well make sure to try out a few of these exercises to see how
they can work for our needs and get more familiar with the process as well.
 Chapter 8

Learning About Function:

Understanding A Concept Of Function,
Using Various Function

Functions
Functions are prewritten blocks of code that can be invoked to carry out a
certain set of actions. You’ve been making use of a few different predefined
functions in Python throughout this book, such as the print() function, which
has Python take actions that result in the printing of a statement to the
terminal. The print() function is a pre-defined Python function, but you’ll
soon learn how to write and use your own functions.
You can call functions in multiple ways. The most intuitive way of calling a
function is to simply use the function name, followed by parentheses. Typing
out the function name will typically invoke the function, although there is
another way to call a function as well. You can also use “dot notation” to call
a function. Using dot notation means placing a period before the name of the
function, like this:
.function_name()
Dot notation is used to tell Python you want to call a function on a specific
variable of an object, with the name of the object coming just before the
period, like this:
target_object.function_name()
Sometimes when we call a function, we need to provide the function with
certain variables or data values. These values that we give to the function for
it to use are called “parameters” or “arguments.” The parameters/arguments
are passed to the function by putting them within a set of parentheses that
follows the function name. When passing multiple arguments to a function,
separate the individual arguments with commas.
By now, you should be familiar with the one example of passing arguments
to a function, just consider how you’ve used the print() function before.
print("This string value is being passed into the function call as an
argument")
If the function requires two or more arguments, you would simply pass in
multiple arguments separated by commas.
Creating Your Own Functions
Let’s learn how to create our own functions and make use of them within
other scripts and programs. In order to tell Python that you would like to
create a function, you use the def keyword. The def keyword tells Python you
want to define a function and that it needs to recognize a variety of keywords
and values to follow. After using the def keyword, you need to provide the
function name and any arguments/parameters your function will make use of.
You can then begin writing out the commands you’d like your function to
carry out. In other words, the syntax for creating a function looks something
like this:
def name(parameters):
Code to carry out desired actions
Your functions will often require another keyword, the return keyword. The
return keyword specifies an expression, variable, or value you’d like the
function to pass back out to the main program once the function has finished
running. Once the return statement is executed, the function will cease
running, and the expression you’ve designated will be passed back. Here’s
how to return an expression or value:
def name(parameters):
Code to carry out desired actions
return desiredExpression
If your function returns a value, you can assign that value to a variable just by
calling the function and assigning it to a variable. You can then manipulate
the return value that has been stored in the variable.
returned_value = function_used(list of parameters)
If your function does not need to return a value, you can either just not use
the return keyword or use return None.
Here’s an example of a function that you will be able to run in PyCharm:
def multiply_values(num_1, num_2):
num_3 = 2
print("Number 1 is : " + str(num_1))
print("Number 2 is : " + str(num_2))
print("Number 3 is: " + str(num_3))
mult_num = num_1 * num_2 * num_3
print("Product of multiplication is: " + str(mult_num))
return mult_num
The function takes in two different numbers as parameters and multiplies
these numbers, then multiplies the product of those numbers by two. The
function then returns the multiplied value. The function also prints out the
numerical values that have been passed in, as well as the product of the
multiplication.
We can now call the function, assign it to a variable, and then print the
variable to make sure that the returned value is what we expect it to be.
multiplied = multiply_values(8, 9)
print(multiplied)
You may notice that we have called the print function within our own
function. Python allows you to call functions within functions like this, even
your own custom functions.

Variable Scope
We’ve worked quite a bit with variables thus far, but now we need to take
some time to discuss an important aspect of variables - “variable scope.”
Variable scope refers to the visibility or accessibility of variables. When you
look through a telescope at a landscape, you can only see objects that the
telescope is directed at. Similarly, variables can only be used by Python if
they are in the correct scope, and if the program can see the variable at the
current point in time.
Variables declared inside a function are a part of that function’s scope, and
they are handled differently than variables created outside the scope of the
function. The difference is that variables created within a function can only
be used inside that function; they are not accessible by the program at large.
Meanwhile, variables declared outside the function are “global” in scope, and
they can be accessed by the entire program. Global variables stand in contrast
to local variables (those created within a function).
We can understand the scope better if we look at the function we defined
above. Trying to access and print out the variable num_3 will cause an error,
as it is local to the function and not accessible by the rest of the program. For
this reason, a NameError is thrown. However, if we declare a new variable
(num_4) outside the function, this variable is globaland can be printed. If you
wanted to, you could pass num_4 to the function when you call it, and the
function would be able to use it as it is a global variable.
def multiply_values(num_1, num_2):
num_3 = 2
print("Number 1 is : " + str(num_1))
print("Number 2 is : " + str(num_2))
print("Number 3 is: " + str(num_3))
mult_num = num_1 * num_2 * num_3
print("Product of multiplication is: " + str(mult_num))
return mult_num
multiplied = multiply_values(8, 9)
print(multiplied)
num_4 = 12
print(num_4)
# Attempting to run this will cause an error
print(num_3)
It’s important to understand that if you have a function variable/local variable
that matches the name of a global variable, the function will not reference the
global variable. In other words, if a local variable and global variable have
the same name, the function uses the local variable and not the global
variable. However, any code written outside the function will reference the
global variable.

Default Parameters
We’ve covered functions and variable scope, which were the basics of
functions. Now we can start getting into the more subtle uses and options for
functions in Python.
Python allows you to specify default values for variables that will be used in
a function. Default values are useful because they enable the program to be
run without needing to select values for all the parameters at run time. In
order to create default values in a function, you use the assignment operator
to assign a default value when initially creating the function.
def sample_function(variable_1, variable_2 = 9):
print("Variable 1 is: " + str(variable_1))
print("variable 2 is: " + str(variable_2))

sample_function(12)
Running the code above, you’ll discover that the value for variable_1 is 12,
as specified when the function is called. However, the value for variable_2 is
9, as there was no value passed in for it, and 9 was the default value. If a
function has a default value, the parameters with the default value must be at
the end of the list. You cannot write a function that has default values with
those parameters defined in the middle or at the beginning of the list of
function parameters.
Just because the function above has a default value does not mean we have to
keep using the default value. We can overwrite the default value simply by
passing in a value for the second parameter when it is called.
sample_function(12, 24)
As you can see, overwriting the default function value is very simple. Just
remember that the values you pass into the function are assigned to the
parameters in the order that the parameters are defined. So if you wanted
variable_2 to have a value of 19 and not 12, you need to make sure the 19 is
passed second.

Variable-Length Arguments - Lists And Keywords

(*Args And **Kwargs)
Thus far, we’ve created functions where we know, in advance, the number of
variables we will need within the scope of the function. However, what can
be done if we don’t know how many arguments a function will need? Python
allows us to handle situations like this by using variable-length argument lists
and keyword argument dictionaries.
You can think of using variable-length argument lists as passing an empty list
into the function, a list that will be filled in with values as the program is run.
(Variable-length arguments aren’t perfectly synonymous with lists, but for
illustrative purposes, this is good enough.)
Let’s assume you have a database full of sentences, and you need to print out
the sentences. However, you don’t know how many sentences there are in the
database. A function that allows you to do this, using variable-length
arguments, would look like this:
def print_sentences(*sentences):
For sentence in sentences:
print(sentence)
The advantage of using a function like this is that you can just add sentences
straight to the function call as parameters, without having to alter the way
your function works in any way. You simply use the asterisk symbol (*) to
tell Python that it should be prepared to work with an unknown number of
arguments, passed in as a list.
print_sentences(sentence_1, sentence_2, sentence_3)
Essentially, the asterisk declares that a list of items that will vary in length
will be passed to the function. We can also do the same with keywords
instead of non-keyword argument lists. In order to use a variable-length
keyword dictionary, we declare the argument dictionary with two asterisks
instead of one (**).
Assume that we have a large dictionary of contact information, containing
names and email addresses. We want to print out both the key (name) and the
value (email address). We could accomplish that by writing a function that
looks something like this:
def print_emails(**emails):
for name, email in emails.items():
print("Name = {}, email = {}".format(name, email)
We can then use our function by calling it and pass in the names and emails.
print_emails('Dictionary information here')
For the sake of clarity, the functions above were written with sentences and
emails, as the names of the variable-length argument lists and keyword
dictionaries. However, the names could potentially be anything. That said,
when you use these concepts in your own code, you are encouraged to stick
to Python convention and use *args (arguments) and **kwargs (keyword
arguments) as their names.
One more thing to note when making use of *args and **kwargs. If your
function utilizes any combination of regular arguments, *args, and **kwargs,
the parameters must be declared within the parentheses in a specific order.
The regular argument must come first, then the *args, and finally the
**kwargs.

Exercise
Write a program in Python to find the max of three numbers:
def max_of_two( x, y ):
if x > y:
return x
return y
def max_of_three( x, y, z ):
return max_of_two( x, max_of_two( y, z ) )
print(max_of_three(3, 6, -5))
PART 3
 Chapter 9:

Reading And Writing Files In Python

Working With Files

Programs are made with input and output in mind. You input data to the
program, the program processes the input and it ultimately provides you with
an output.
For example, a calculator will take in numbers and operations you want. It
will then process the operation you wanted. And then it will display the result
to you as its output.
There are multiple ways for a program to receive input and to produce output.
One of those ways is to read and write data on files.
To start learning how to work with files, you need to learn the open ()
function.
The open() function has one required parameter and two optional parameters.
The first and required parameter is the file name. The second parameter is the
access mode. And the third parameter is buffering or buffer size.
The file name parameter requires string data. The access mode requires string
data, but there is a set of string values that you can use and is defaulted to "r".
The buffer size parameter requires an integer and is defaulted to 0.
To practice using the open() function, create a file with the name
sampleFile.txt inside your Python directory.
Try this sample code:
> >> file1 = open("sampleFile.txt")
>>>
Note that the file function returns a file object. The statement in the example
assigns the file object to variable file1.
The file object has multiple attributes, and three of them are:
name: This contains the name of the file.
mode: This contains the access mode you used to access the file.
closed: This returns False if the file has been opened and True if the file is
closed. When you use the open() function, the file is set to open.
Now, access those attributes.
> >> file1 = open("sampleFile.txt")
> >> file1.name
'sampleFile.txt'
> >> file1.mode
'r'
> >> file1.closed
False
>>>
Whenever you are finished with a file, close them using the close() method.
> >> file1 = open("sampleFile.txt")
> >> file1.closed
False
> >> file1.close()
> >> file1.closed
True
>>>
Remember that closing the file does not delete the variable or object. To
reopen the file, just open and reassign the file object. For example:
> >> file1 = open("sampleFile.txt")
> >> file1.close()
> >> file1 = open(file1.name)
> >> file1.closed
False
>>>

Reading From A File

Before proceeding, open the sampleFile.txt in your text editor. Type "Hello
World" in it and save. Go back to Python.
To read the contents of the file, use the read() method. For example:
> >> file1 = open("sampleFile.txt")
> >> file1.read()
'Hello World'
>>>

File Pointer
Whenever you access a file, Python sets the file pointer. The file pointer is
like your word processor’s cursor. Any operation on the file starts at where
the file pointer is.
When you open a file and when it is set to the default access mode, which is
"r" (read-only), the file pointer is set at the beginning of the file. To know the
current position of the file pointer, you can use the tell() method. For
example:
> >> file1 = open("sampleFile.txt")
> >> file1.tell()
0
>>>
Most of the actions you perform on the file move the file pointer. For
example:
> >> file1 = open("sampleFile.txt")
> >> file1.tell()
0
> >> file1.read()
'Hello World'
> >> file1.tell()
11
> >> file1.read()
''
>>>
To move the file pointer to a position you desire, you can use the seek()
function. For example:
> >> file1 = open("sampleFile.txt")
> >> file1.tell()
0
> >> file1.read()
'Hello World'
> >> file1.tell()
11
> >> file1.seek(0)
0
> >> file1.read()
'Hello World'
> >> file1.seek(1)
1
> >> file1.read()
'ello World'

>>>
The seek() method has two parameters. The first is offset, which sets the
pointer’s position depending on the second parameter. Also, an argument for
this parameter is required.
The second parameter is optional. It is for whence, which dictates where the
“seek” will start. It is set to 0 by default.
If set to 0, Python will set the pointer’s position to the offset argument.
If set to 1, Python will set the pointer’s position relative or in addition to the
current position of the pointer.
If set to 2, Python will set the pointer’s position relative or in addition to the
file’s end.
Note that the last two options require the access mode to have binary access.
If the access mode does not have binary access, the last two options will be
useful to determine the current position of the pointer [seek(0, 1)] and the
position at the end of the file [seek(0, 2)]. For example:
> >> file1 = open("sampleFile.txt")
> >> file1.tell()
0
> >> file1.seek(1)
1
> >> file1.seek(0, 1)
0
> >> file1.seek(0, 2)
11
>>>
File Access Modes
To write to a file, you will need to know more about file access modes in
Python. There are three types of file operations: reading, writing, and
appending.
Reading allows you to access and copy any part of the file’s content. Writing
allows you to overwrite a file’s contents and create a new one. Appending
allows you to write on the file while keeping the other content intact.
There are two types of file access modes: string and binary. String access
allows you to access a file’s content as if you are opening a text file. Binary
access allows you to access a file on its rawest form: binary.
In your sample file, accessing it using string access allows you to read the
line “Hello World”. Accessing the file using binary access will let you read
“Hello World” in binary, which will be b'Hello World'. For example:
>>> x = open("sampleFile.txt", "rb")
>>> x.read()
b'Hello World'
>>>
String access is useful for editing text files. Binary access is useful for
anything else like pictures, compressed files, and executables. In this book,
you will only be taught how to handle text files.
There are multiple values that you can enter in the file access mode parameter
of the open() function. But you do not need to memorize the combination.
You just need to know the letter combinations.
Each letter and symbol stands for an access mode and operation. For
example:
r = read only—file pointer placed at the beginning
r+ = read and write
a = append—file pointer placed at the end
a+ = read and append
w = overwrite/create—file pointer set to 0 since you create the file
w+ = read and overwrite/create
b = binary
By default, file access mode is set to string. You need to add b to allow
binary access. For example: "rb".
Writing To A File
When writing to a file, you must always remember that Python overwrites
and not insert file. For example:
>>> x = open("sampleFile.txt", "r+")
>>> x.read()
'Hello World'
>>> x.tell(0)
0
>>> x.write("text")
4
>>> x.tell()
4
>>> x.read()
'o World'
>>> x.seek(0)
0
>>> x.read()
'texto World'
>>>

You might have expected that the resulting text will be “textHello World”.
The write method of the file object replaces each character one by one
starting from the current position of the pointer.

Exercise
For practice, you need to perform the following tasks:
Create a new file named test.txt.
Write the entire practice exercise instructions on the file.
Close the file and reopen it.
Read the file and set the cursor back to 0.
Close the file and open it using append access mode.
Add a rewritten version of these instructions at the end of the file.
Create a new file and put similar content to it by copying the contents of the
test.txt file.
Working with files in Python is easy to understand, but difficult to
implement. As you already saw, there are only a few things that you need to
remember. The hard part is when you are actually accessing the file.
Remember that the key things that you should master are the access modes
and the management of the file pointer.
It is easy to get lost in a file that contains a thousand characters.
Aside from being versed in the file operations, you should also supplement
your learning with the functions and methods of the str class in Python. Most
of the time, you will be dealing with strings if you need to work on a file.
Do not worry about binary yet. That is a different beast altogether and you
will only need to tame it when you are already adept at Python. As a
beginner, expect that you will not deal yet with binary files that often contain
media information.
 Chapter 10:

Python Programming 101: Interacting

With Python In Different Ways

We have seen how to get the environment set up, to be able to code in
Python. We’ll focus on writing our very own Python programs. Below is a
snapshot of the Python interpreter, where we write the following command:
print(“Hello World”)

The ‘print’ command is used to output text back to the console. In the
remainder of this book, we will focus on writing programs that can
encapsulate these code statements in order to build bigger and better
programs in Python.

Creating A Python File

We start by creating a new file. We can either use an Integrated Development
Environment, or simply create a file in Notepad. We will do the latter, seeing
as it’s much easier as a beginner.
Let’s look at the steps to create a simple Python program:
Create a new file in Notepad.
Add the following line of code inside the file.
 print(“Hello World”)

Save the file as ‘Demo.py’. Note that we are saving the file with the .py
extension. This helps ensure that the Python runtime will be able to interpret
the file and run it accordingly.

Running A Python File

Next, depending on the environment you are in, there are different ways to
run the program. In Windows we can carry out the following steps:
Open Windows PowerShell by using the Windows search feature.
In PowerShell, type the following command._________
& python ‘location of the file’

In the screenshot below, our Python code file is located on H drive.

When we hit enter, we get the relevant output ‘Hello World’. That’s it!
What happens in the above process is that PowerShell submits our program
to the Python interpreter, which then executes the program and returns the
output back to the PowerShell console. This simple flow is depicted below.
Now let’s add a couple more lines of code to see how to execute multiple
statements in Python.
The following program is used to explore our first Python program further.
In our current program, let’s add the following lines:
print("This is our first program")
print("We want to write to the console")
print("Hello World")

Note that unlike other programming languages, there is no main method or

extra code needed to define the entry point for the program. It’s just plain and
simple code that gets executed.
From the output above, we can see that all the lines of code are executed and
the relevant text is written to the console.
The Python interpreter does the basic error checking for our Python
programs. To illustrate how this works, let’s make a mistake in our next
program.
The following program is used to introduce an error in the program.
Let’s change our code to the following:______________
print{"This is our first program"}
print("We want to write to the console")
print("Hello World")

Note that we have changed the first line of code and used curly braces instead
of the standard brackets. When we execute the program, we will get the
following output:
PS H:\> & python H:/Demo.py
File "H:/Demo.py", line 1
print{"This is our first program"}

Syntaxerror: Invalid Syntax

Here we can see that the Python interpreter has checked the correctness of the
program and pointed out the error accordingly.
Defining Values
We can also define values in a Python program. For now let’s look at a
simple example of how this can be done.
Example 12: The following program shows how we work with values in a
program.________________________________________
a="5"
print("The value of a is "+a)

This program’s output will be as follows:

The value of a is 5
In the above program:
We first defined a variable called ‘a’.
We then assigned a value of ‘5’ to this variable.
We also use the ‘print’ statement to output the value of the variable.
Note that in the print statement, we added the text ‘The value of a is’ before
the actual value of the variable ‘a’.

Using Python Libraries

Python has a host of built-in libraries that provide additional functionality to
Python programs. For now let’s look at a simple example of how this works.
The following program shows how to work with Python libraries.
print(sum([1,2,3]))

This program’s output will be as follows:

6
In the above program:
We are making use of the math library.
This is a built-in library that is always available.
We then use the ‘sum’ method in that library to carry out the multiplication
of the numbers.
Lastly, we print the resultant value to the console.
We can also use other library modules that are not directly accessible. This is
done by importing them. Let’s look at an example of this.
The following program illustrates how to import Python libraries.
 import random
print(random.randint(1,5))

This program’s output will depend on what gets generated. In our case we
received the following:
3
In the above program:
We are first using the ‘import’ statement to get the functionality of the
‘random’ library.
We then use the ‘randint’ method to generate a random number between 1
and 5.
Lastly we display that value to the console.

Using Comments
Comments are extremely useful in any programming language to ensure
better readability and maintainability of the program. Their main purpose is
to describe the various sections of the program for future reference. Let’s
look at the use of comments in Python.

The following program shows how to use comments in .a program.

# This program generates a random number
import random
print(random.randint( 1,5))

In this sample program, the first line starts with a ‘#’ which indicates that this
is a comment line. As such, this line will not be executed and will be ignored
by the interpreter.
 Chapter 11:

Creating A While Loop In Python

Loops
Sometimes we need to tell the program to repeat a set of instructions every
time it meets a condition. To achieve this, we have two kinds of loops, known
as the “for” loop and the “while” loop. Here’s an example of a “for” loop:
for x in range(1, 10):
print(x)
In this example, we instruct our program to keep repeating until every value
of x between 1 and 10 is printed. When the printed value is 2, for instance,
the program checks if x is still within the (1, 10) range and if the condition is
true, it will print the next number, and the next and so on.
Here’s an example with a string:
for x in “programming”:
print (x)
The code will be executed repeatedly until all characters inside the word
“programming” are printed.
Here’s another example using a list of objects:
medievalWeapons = [“swords”, “bows”, “spears”, “throwing axes”]
for x in medievalWeapons:
print(x)
In this case, the program will repeat the set of instructions until every object
inside the list we declared is printed.
Next up we have the “while” loop that is used to repeat the code only as long
as a condition is true. When a statement no longer meets the condition we set,
the loop will break and the program will continue the next lines of code after
the loop. Here’s an example:
x=1
while x < 10:
print(x)
x += 1
First we declare that x is an integer with the value of 1. Next we instruct the
program that while x is smaller than 10 it should keep printing the result.
However, we can’t end the loop with just this amount of information. If we
leave it at that, we will create an infinite loop because x is set to always be 1
and that means that x will forever be smaller than. The “x+= 1” at the end
tells the program to increase x’s value by 1 every single time the loop is
executed. This means that at one point x will no longer be smaller than 10,
and therefore the statement will no longer be true. The loop will finish
executing, and the rest of the program will continue.
But what about that risk of running into infinite loops? Sometimes accidents
happen, and we create an endless loop. Luckily, this is preventable by using a
“break” statement at the end of the block of code. This is how it would look:
while True:
answer = input (“Type command:”)
if answer == “Yes”:
break
The loop will continue to repeat until the correct command is used. In this
example, you break out of the loop by typing “Yes”. The program will keep
running the code until you give it the correct instruction to stop.

Functions
Now that you know enough basic programming concepts, we can discuss
making your programs more efficient, better optimized, and easier to analyze.
Functions are used to reduce the number of lines of code that are actually
doing the same thing. It is generally considered best practice not to repeat the
same code more than twice. If you have to, you need to start using a function
instead. Let’s take a look at what a function looks like in code:
def myFunction():
print(“Hello, I’m your happy function!”)
We declare a function with the “def” keyword, which contains a simple string
that will be printed whenever the function is called. The defined functions are
called like this:
myFunction()
You type the name of the function followed by two parentheses. Now, these
parentheses don’t always have to stay empty. They can be used to pass
parameters to the function. What’s a parameter? It’s simply a variable that
becomes part of the function’s definition. Let’s take a look at an example to
make things clearer:
def myName(firstname):
print(firstname + “ Johnson”)
myName(“Andrew”)
myName(“Peter”)
myName(“Samuel”)
In this example we use the parameter “firstname” in the function’s definition.
We then instruct the function to always print the information inside the
parameter, plus the word “Johnson”. After defining the function, we call it
several times with different “firstname”. Keep in mind that this is an
extremely crude example. You can have as many parameters as you want. By
defining functions with all the parameters you need, you can significantly
reduce the amount of code you write.
Now let’s examine a function with a set default parameter. A default
parameter will be called when you don’t specify any other information in its
place. Let’s go through an example for a better explanation. Nothing beats
practice and visualization. Type the following code:
def myHobby(hobby = “leatherworking”):
print (“My hobby is “ + hobby)
myHobby (“archery”)
myHobby (“gaming”)
myHobby ()
myHobby (“fishing”)
These are the results you should receive when calling the function:
My hobby is archery
My hobby is gaming
My hobby is leatherworking
My hobby is fishing
Here you can see that the function without a parameter will use the default
value we set.
Finally, let’s discuss a function that returns values. So far our functions were
set to perform something, such as printing a string. We can’t do much with
these results. However, a returned value can be reassigned to a variable and
used in more complex operations. Here’s an example of a return function:
def square(x):
return x * x
print(square (5))
We defined the function and then we used the “return” command to return the
value of the function, which in this example is the square of 5.

Exercise
Look at the following code and determine what will be printed from it:
alist = [4,2,8,6,5]
blist = [num*2 for num in alist if num%2==1]
print(blist)
Choose from:
• [4,2,8,6,5]
• [8,4,16,12,10]
• 10
• [10]
 Chapter 12:

Practice Projects:
The Python Projects For Your Practice

The final section of this book will involve completing one project. One
project will be a simple project that only involves the manipulation of text.
All the code necessary to complete the project will be given. Still, much like
the coding exercises earlier in the book, you are highly encouraged to try
solving the problems yourself before looking at examples of how the projects
can be completed.
Project: Story Idea Generator
In this project, we’ll be creating a story idea generator. We’ll have a sentence
template that we fill in with phrases to create an idea for a story. Our program
will use lists of characters, objects, themes, etc. and select one item from the
list at random. We’ll then insert these random choices into the sentence
template to create a story prompt.
Let’s start by thinking about the steps we’d need to take to accomplish the
project’s goal. Our goal is to generate a story prompt/idea from several
different, pre-defined attributes. For instance, we can have a list of characters
and settings to choose from. This also means that we can start by creating a
sentence template and choosing the fields we’d want to insert into the
sentence. So, let’s begin by setting up our sentence template. It could look
something like this:
"In" + " " + setting + ", there is a" + " " + protagonist + " " + "who" + " " +
conflict + antagonist + "."
You can see that we want to insert an item from four different categories into
our template. Do you know how this could be accomplished with a single
function? Try writing your own generation function before you look at how it
could be created.
Here’s one way we could write a generation program.
We’d want to start by creating four lists that will hold items belonging to
these various categories. You can go with any theme or genre that you would
like, but in this instance, we’ll theme our categories around sci-fi concepts.
We’ll have the following categories: setting, protagonist, conflict,
Antagonist. The “protagonist” field will be comprised of two separate
categories: gender and occupation. To begin with, we’ll create lists of
possible items for all the categories.
setting = ["future Seattle", "future New York", "future Tokyo", "a dystopia",
"a virtual world", "a base stationed on the moon", "a utopia", "a space
station", "a city under the sea", "an artificial island", "an underground
complex"]

gender = ["man ", "woman ", "robot ", "third gender ", "animal ", "mutant "]
occupation = ["writer", "pilot", "detective", "cyborg", "doctor", "soldier",
"hacker", "engineer", "corporate employee", "actor", "scientist", "racer",
"street rat", "delivery person"]
antagonist = ["a rogue AI", "a gigantic corporation", "a secret society", "a
collection of robots", "groups of internet trolls", "a group of aliens", "a
devastating virus", "a corrupt government", "new bandits", "new pirates", "a
powerful street gang", "a disruptive technology", "a clone of the hero",
"genetically-engineered monsters"]
conflict = ["tries to stop ", "falls in love with ", "seeks revenge against ",
"runs away from ", "fights against ", "defends against ", "exceeds beyond ",
"explores with ", "attempts to befriend ", "is in competition with ", "must
infiltrate ", "tries to redeem "]
Now that we’ve created lists for our different categories, we can randomly
select an item from these lists and insert it into our sentence template.
Fortunately, we can use the built-in library random and the random.choice()
function to randomly select an item from our lists. Import the module and
then wrap the lists in the random.choice() function, like this:
import random
setting = random.choice(
["future Seattle", "future New York", "future Tokyo", "a dystopia",
"a virtual world", "a base stationed on the moon", "a utopia", "a space
station", "a city under the sea", "an artificial island", "an underground
complex"])
...
After wrapping the lists in the random.choice() function, we can set pass in
the variables into our template and print it out.
print("In" + " " + setting + ", there is a" + " " + protagonist + " " + "who" + "
" + conflict + antagonist + ".")
That’s all we needed to do to get our story generator working. Here’s a
sample of what it printed out:
“In a base stationed on the moon, there is a robot cyborg who explores with a
group of aliens.”
However, what if we wanted to generate a bunch of story prompts instead of
just one? If we just call the print statement a few more times, we’ll find out
that all the printed prompts are the same because Python preserves the
random choice across all the print statements, not to mention that printing the
statement multiple times is repeating ourselves unnecessarily. If we wrapped
the code in a function that took in the number of sentences we’d like to
generate as an argument that would be much simpler. See if you can figure
out how to write a function that generates as many story prompts as the user
specifies.
Once you’ve had a shot at composing the function, look below to see one
way it can be done.
To create a function that will print the desired number of story prompts and
ensure that the story prompts are different, you would need to use a while
loop and have the random.choice() calls be inside of that while loop. You
want to keep looping until the number of iterations is equal to the desired
number of prompts, so you’ll start by creating a variable to keep track of the
number of prompts and then increase the value before ending the current loop
(but after printing the prompt). Given this, the entire function should look
something like this.
def plot_gen(num_gen):
i=1
while i <= num_gen:
setting = random.choice(
["future Seattle", "future New York", "future Tokyo", "a dystopia",
"a virtual world", "a base stationed on the moon", "a utopia",
 "a space station", "a city under the sea", "an artificial island", "an
underground complex"])

i += 1

plot_gen(5)
 Conclusion

Now that we have come to the end of the book, I hope you have gathered a
basic understanding of what machine learning is and how you can build a
machine learning model in Python. One of the best ways to begin building a
machine learning model is to practice the code in the book, and also try to
write similar code to solve other problems. It is important to remember that
the more you practice, the better you will get. The best way to go about this is
to begin working on simple problem statements and solve them using the
different algorithms. You can also try to solve these problems by identifying
newer ways to solve the problem. Once you get the hang of the basic
problems, you can try using some advanced methods to solve those
problems.
Thanks for reading to the end!
Python Machine Learning may be the answer that you are looking for when it
comes to all of these needs and more. It is a simple process that can teach
your machine how to learn on its own, similar to what the human mind can
do, but much faster and more efficient. It has been a game-changer in many
industries, and this guidebook tried to show you the exact steps that you can
take to make this happen.
There is just so much that a programmer can do when it comes to using
Machine Learning in their coding, and when you add it together with the
Python coding language, you can take it even further, even as a beginner.
The next step is to start putting some of the knowledge that we discussed in
this guidebook to good use. There are a lot of great things that you can do
when it comes to Machine Learning, and when we can combine it with the
Python language, there is nothing that we can’t do when it comes to training
our machine or our computer.
This guidebook took some time to explore a lot of the different things that
you can do when it comes to Python Machine Learning. We looked at what
Machine Learning is all about, how to work with it, and even a crash course
on using the Python language for the first time. Once that was done, we
moved right into combining the two of these to work with a variety of Python
libraries to get the work done.
You should always work towards exploring different functions and features
in Python, and also try to learn more about the different libraries like SciPy,
NumPy, PyRobotics, and Graphical User Interface packages that you will be
using to build different models.
Python is a high-level language that is both interpreter based and object-
oriented. This makes it easy for anybody to understand how the language
works. You can also extend the programs that you build in Python onto other
platforms. Most of the inbuilt libraries in Python offer a variety of functions
that make it easier to work with large data sets. You will now have gathered
that machine learning is a complex concept that can easily be understood. It
is not a black box that has undecipherable terms, incomprehensible graphs, or
difficult concepts. Machine learning is easy to understand, and I hope the
book has helped you understand the basics of machine learning. You can now
begin working on programming and building models in Python. Ensure that
you diligently practice since that is the only way you can improve your skills
as a programmer.
If you have ever wanted to learn how to work with the Python coding
language, or you want to see what Machine Learning can do for you, then
this guidebook is the ultimate tool that you need! Take a chance to read
through it and see just how powerful Python Machine Learning can be for
you.
BOOK 2 - PYTHON CRASH
COURSE
PART 4
 Introduction

The Python programming language would be really a contemporary online

programming language that was originally conceived and made by Guido
Rossum in 1980s. Since that moment, Python has become a high heeled
programming language that is modular and adaptive. A variety of the biggest
sites in the world are using Python, such as YouTube, Disqus, and Reddit.
Python presents several attributes which make it an attractive programming
platform such as stability, portability, object-oriented improvement, a strong
standard library, and a wealth of third-party modules or bundles.
Stability Python was under active development since the late 1980s and is
now thought to be a programming language. The programmers of this Python
language conduct comprehensive functionality and regression testing to
ensure the language stays bug-free and steady with every new release.
Portability Python programming provides several features that make it an
attractive option for online software development. Python programs are
portable as Python interpreters are easily available for many modern
operating systems together with some embedded computing applications.
Object-oriented improvement the object-oriented nature of Python makes it
the greatest initial language for new developers and simple to learn for
programmers migrating to Python from additional object-oriented languages.
Python programming is instinctive and reinforces great application structure
and object-oriented approaches. Standard library the standard Python library
provides developers various attributes like more complex languages such as
c++ while maintaining pragmatic and simple language syntax.
Comprehensive file-based i/o, database interactivity, innovative exception
handling and a slew of built-in data types make Python appropriate for both
web programs and mimicked programming. This makes Python net
programming a simple endeavor for program developers hoping to transition
into net software development.
Third-party modules Python is famous to be an inclusive language utilizing
extensive functionality inside the library. On the other hand, the growing
prevalence of Python programming has caused a massive group of third-party
packages modules or modules therefore that expand Python's functionality
and permit the language to look after programming challenges which are
exceptional. For example, modules can be obtained for managing non­
standard database links and advanced cryptography functionality.
Furthermore, there are modules available for managing everyday tasks such
as reading record metadata, which include graphs, and compiling Python
applications to standardized executable applications.
Python web programming has been made accessible as a consequence of
accessibility to several web-centric modules to manage tasks like email,
preserving http country, interacting with all JavaScript, along with other
ordinary web development tasks.
The information evaluation procedure: 5 steps to enhance decision making
You need greater information analysis. With the ideal information analysis
procedure and resources, what was an overwhelming quantity of disparate
data becomes an easy, clear decision stage.
To boost your information evaluation skills and simplify your decisions,
implement these five measures on your data evaluation procedure:
Step 1: establish your queries
On your organizational or business information evaluation, you have to start
with the ideal query (s). Questions must be quantifiable, concise and clear.
Layout your queries to qualify or disqualify prospective answers to your
particular issue or opportunity.
As an example, begin with a clearly defined issue: a government contractor is
currently experiencing increasing prices and is no more able to publish
competitive contract tips. Among the several questions to figure out this
business problem would comprise: could the firm reduce its employees
without compromising quality?
Step 2: establish clear measurement priorities
This step divides to 2 sub-steps: a) pick what to quantify, and b) decide on
how to quantify it.
A) pick what to quantify
Employing the authority’s contractor instance, consider what type of
information you would want to answer your main question. In cases like this,
you'd want to understand the quantity and price of present employees and the
proportion of time that they spend on essential business purposes.
In answering this query, you probably will need to answer several sub­
questions (e.g.) are employees presently under-utilized? If this is so, what
procedure developments could help?). At length, on your choice about which
to measure, make certain to incorporate any sensible understanding any
stakeholders may possess (e.g., if employees are decreased, how do the firm
react to surges in demand?).
B) pick how to quantify it
Thinking about the way you quantify your information is equally as
important, particularly prior to the information collection period, as your
measuring procedure either backs up or discredits your investigation in the
future. Crucial questions to ask to this measure include:
• What's your timeframe? (e.g., yearly versus quarterly prices)
• What is your unit of measure? (e.g., USD vs euro)
• What variables must be included? (e.g., only annual salary
versus yearly salary and cost of personnel benefits)
Step 3: collect data
Together with your query clearly defined along with your measurement
priorities place, now it is time to gather your own data. As you gather and
organize your information, don't forget to keep these important points in
mind:
• Before you gather new information, determine what data can be
gathered from existing sources or databases available. Collect
this information.
• Decide on a document saving and naming system beforehand to
aid all tasked staff members collaborate. This procedure saves
time and prevents staff members out of collecting the identical
data twice.
• Should you have to assemble data via interviews or observation,
then create a meeting template beforehand to guarantee
consistency and conserve time.
• Maintain your gathered data organized within a log together
with set dates and include some other origin notes as you
proceed (like any information normalization done). This clinic
divides your decisions in the future.
Step 4: analyse data
After you have gathered the ideal information to reply your query from step
1, it is time for deeper information analysis. Start by manipulating your
information in several of unique ways, like hammering out it and discovering
correlations or simply by making a pivot table in excel. A vanity enables you
to filter and sort information from different factors and permits you to figure
out the mean, maximum, minimum and standard deviation of your
information -- only make sure you prevent those five dangers of statistical
information analysis.
As you control information, you might find you've got the precise
information you require, but more inclined, you may have to update your
initial query or collect additional information. In any event, this original
investigation of trends, correlations, variants along with outliers will help you
concentrate your information evaluation on better replying your query and
some other objections others may have.
Through this period, information analysis tools and applications are very
beneficial. Visio, both Minitab and stata are excellent software packages for
complex statistical data evaluation. But, generally, nothing really compares to
Microsoft excel concerning decision-making tools. Should you require a a
primer on each of the purposes excel accomplishes your information analysis,
we advise this Harvard business course.
Step 5: allergic effects
After assessing your information and maybe conducting additional research,
it is now time to translate your results. As you translate your investigation,
remember which you can't ever establish a theory correct: instead, it's
possible just to don't reject the hypothesis. Meaning no matter how much
information you collect; opportunity could always hinder your own results.
As you translate the outcome of your information, inquire these critical
questions:
c Can the information answer your initial question? How?
• Can the information enable you to defend against any
conscience? How?
• Can there be some limit in your decisions, any angles that you
have not considered?
If the interpretation of this information holds up under all these questions and
concerns, then you probably have come into a successful decision. The only
remaining step is to utilize the outcomes of your data evaluation procedure to
determine your very best strategy.
By following these five measures on your information analysis procedure,
you create better choices for your business enterprise or government service
as your decisions are backed by information that's been robustly accumulated
and examined. With training, your information analysis gets quicker and
more precise -- meaning that you create better, more educated decisions to
conduct your business effectively.
 Chapter 13:

Evolution of Python

The Origins and Evolution of Python Language

Since its inception, Python has been designed to make available to the most
significant number, a simple and intuitive development tool for creating
scripts.
The Meeting between Python and Data Science
The emergence of data science is recent, and these new uses of data have
often had difficulty finding suitable tools. Indeed, the data scientist must be a
good developer while remaining a good data analyst. He has had to opt for a
tool that would combine this demand with more and more strong
development and automation (all the more so with the arrival of artificial
intelligence and connected objects), with the need for a toolbox suitable for
data applications.
Many avenues have been explored, in particular with the software R which
continues to be a reference in data science but that could seem too oriented
towards statistics for more developmentally oriented data scientists.
History, Origins and Evolution: From Birth to Version 3.717
Many other tools for setting up data science processes have been developed
(most proprietors such as Matlab or SAS), but it turns out that Python (which
combines powerful language and extremely simple) has managed to draw its
pin.
The first real advance was the creation of NumPy (Numerical Python)
package, which is still today the cornerstone of the Python ecosystem for data
science. On the other hand, setting up data-driven Python environments with
Anaconda has also enabled a whole community to come together around
Python and data. Finally, IPython and its notebooks (now Jupyter) have
completed the ecosystem to provide data scientists with a straightforward
language but one which is extremely comprehensive for data science. This
global environment resulted in the development of many packages and APIs,
which today make Python the best tool to automate data science treatments.
The Current Evolution
In recent years, Python has taken an essential place in the world of data
processing. While in the early 2010s, it seemed clear that in the world of
open-source data processing tools, the software R was going to carve out the
lion's share, a significant change has taken place since a few years. The
emergence of Python as a language related to data science, machine learning,
deep learning, and artificial intelligence is speedy.
Grace, an extremely active community under the PyData banner and frequent
and numerous events (PyCon, PyData, JupyterCon, SciPyCon ...), language
development took an unexpected turn. While we could hear in 2015
developers say that from machine learning the development of Python was
modeled on that of R with a few months late. Today, it is R who begins to
model his developments in the field of machine learning, deep learning, and
big data, on packages developed in Python. In 2018, KDnuggets, an
influential blog in the world of data science, even surveyed thousands of data
scientists around the world. The latter, for the first time, and showed more
users of Python than of R.
The Python adventure in data science is therefore recent but only because it is
a language that adapts perfectly to the approach led by a data scientist, which
would be: "better in programming than a statistician and better in statistics
than a developer. "
The Future of Python
The near future of Python is above all the abandonment of version 2 and the
generic version 3. The other significant current development concerns the use
of interactive interfaces with Python as a communication language with more
and more advanced APIs. We'll talk a little bit further about Jupyter's widgets
that allow you to develop interactively and build interfaces in a few lines.
Python is increasingly used as a language to communicate with other
environments. So Python can communicate with Apache Spark through
PySpark medium, or with deep learning ecosystems such as TensorFlow.
Calculations are no longer done in the Python engine but much more in
engine devices using distributed infrastructures or GPU-based computations
(Graphical Process Units). This trend is only beginning with the massification
of data and requests for real-time treatments are ever more common.
Python can not answer these challenges alone but, combined with other tools,
it allows the data scientist to manage the entire ecosystem of data processing,
be it big, small, smart...
Python vs R vs. The Rest of the World
If you read this book, you must have heard about other tools in data
science. Today we find an extremely developed ecosystem in this domain
with languages such as R and Python but also more software like Dataiku
DSS, IBM-SPSS Modeler, Enterprise Miner SAS, Azure machine learning
from Microsoft ... As a data scientist, you can be brought to cross some of
these tools in your missions. It is therefore vital to understand where
everyone's strength lies.
We focus here on Python and its use by data scientists. So why is Python
gaining ground on the competition?
In recent years, the global trend has shifted towards more code in the
processing process and Python responds correctly to this request. Thus, the
data scientist tools are increasingly different from those of the BI analyst
(business intelligence) which are more and more intuitive. In this context,
two open-source tools take the lead: Python and R.
Regarding proprietary tools, a trend is becoming widespread. This is the use
of languages such as Python or R inside these tools as soon as we need to
perform complex operations. So Dataiku DSS, RapidMiner or KNIME
integrate modules to develop in Python or R. Your skills in Python will,
therefore, be valued as part of the use of these tools.
R
R and Python are today the indispensable bases of the data
scientist. Furthermore, the rapid evolution of both languages leads to a form
of healthy competition making them more and more complete for data
processing. However, some notable differences will guide your choices of the
language decision to use for a project.
R is based on a language created for statisticians by statisticians, it is first and
foremost on a descriptive and data modeling approach. The R outputs are
outputs worthy of "classic" statistical software with many details. Python is
not based on this approach, it's a language of programming to automate
processes by just looking to calculate the bare minimum. As part of the
current approaches to data application science, Python is more adapted to
other needs than R.
On the other hand, Python is an elementary programming language based on
a very readable syntax. Its understanding by many users is facilitated, which
allows for better interaction between the different professions related to
information systems (managers, developers, data scientists, data engineers
...).
To summarize, Python is the language of automation that integrates perfectly
within a broader IT service framework and which adapts to the contexts of
artificial intelligence (unstructured data, complex environments). It stays
nevertheless lower than R for a statistician looking for statistical software.
R and Python are complementary today and will be very necessary for you
frequently for your treatments.
Python's license is a "classic" free license, it allows you to reuse source code,
modify it, market it, and to make all use without obligation to open your
code. This is the type of license conventionally used for programming
languages. All the Python ecosystem for data is based on this license. On the
other hand, R is based on a more restrictive license, this is the GPL v3
license. This one gives responsibilities to the development community.
Indeed, if you use source code R and modify it to distribute it, you will be
forced to make this code accessible to users (open source). Going again
further, this license is "contaminating", that is, if you embed code licensed to
your code, your code is licensed and must be open source. This point can
frighten some developers who sometimes turn to Python. This difference
between the two languages that are R and Python translates a difference of
language development between the two communities. R developers are more
in an idea to "force" advanced users to contribute to the project and Python
developers are betting on an extensive use of the language that draws more
contributions for language development.
Flow Processing Tools
The other tools of data science are for the most part facilitators, so the
majority of these tools are based on Dataiku, RapidMiner's DSS flow creation
(KNIME, IBM-SPSS Modeler ...) using your mouse. They simplify the life of
the users and, depending on your needs, can save you time. However,
everything these tools can do can be done directly with Python and they all
incorporate ways to add Python code to streams.
These tools will allow you to create analyzes, from acquisition to analysis of
data in a few clicks. For example, you can go for data in different formats at a
first level, check and merge these data to the following level, transform the
data, cut them out and apply and validate predictive models on these data. All
this is included in a single stream.
Python will be very different from this visual treatment but will be able to
reproduce this type of stream as a code. Moreover, as soon as bricks of your
treatments become more complex, the use of Python inside each brick
becomes necessary.
SAS
We will dwell here on a specific point because it concerns many users in the
professional world. For some time now, many companies decided to migrate
their infrastructures from SAS history to new languages (R and Python). It
may be helpful to clarify a few points about this.
SAS is a proprietary software specialized in data processing. It accumulates
nearly forty years of experience in this field and can not be replaced easily.
Companies have often relied on this tool for the processing of their
data. Nevertheless, moving to a tool like Python can be justified for several
reasons:
From an economic point of view: this is often the first reason given. SAS
licenses are costly. But the change will also be expensive, it requires
changing the ways of working and more exceptional support for
infrastructure than before.
From a technological point of view: this is the most essential point. The
transition to Python will provide access to much more powerful machine
learning, deep learning, and unstructured data processing methods than with
SAS.
It should be kept in mind that this change will lead to several
advantages. Mainly the fact that Python is a language that will load the data
in memory in your machine while SAS was using an intermediate system of
tables stored on physical memory. You will need to change your study and go
through more sophisticated queries to your databases. Then, the processing
and analyzing process will be largely simplified.
Python will, therefore, provide you with significant flexibility. Still, you will
have to modify your approach to data management and code development
with Python.
Other Languages
We compared Python to data science tools but another exciting comparison
can be being compared to other programming languages. Today many
languages are available. We can cite so many from the data universe: Julia,
MatLab, C / C ++, Scala. These languages all have their specificities; we can
classify them into two categories:
Interpreted languages such as MatLab, Julia, and Scala are credible
alternatives to Python in a data science-oriented framework.
How to Develop In Python?
Compiled languages such as C, C ++, Java that fall into another category and
are aimed at more experienced developers. In some cases, they are more
efficient than Python. Still, they do not have a package and API environment
as developed as Python.
 Chapter 14:

Introduction to Python

What Is Python?
Python, created in 1990 by Guido van Rossu, is a general-purpose, high-level
programming language. It has become trendy over the past decade, thanks to
its intuitive nature, flexibility, and versatility. Python can be used on a wide
variety of operating systems. Its clean, readable code style makes it relatively
beginner-friendly, while not as fast as other languages, such as C++ or
JAVA, Python code is often much shorter and simpler than other languages.
Python also supports several packages and modules created by other
developers to make the development of Python applications quicker and
easier.

Why Learn Python?

There are hundreds of different programming languages out there in the
world, with Wikipedia listing over 700 notable languages. Given how many
languages you could potentially learn, why learn Python?
Python has seen an explosion in popularity in recent years, driven by several
aspects that make it an incredibly versatile and intuitive language. A huge
selling point of Python is the cleanliness and readability of its syntax and
structure. Commands in Python can often be carried out using simple English
keywords, which makes the language much more intuitive than many other
languages. Python is also quite versatile in the sense that it supports both
structured programming and object-oriented programming approaches.
Python even allows the use of certain aspects of functional programming.
Python is supported by many different operating systems, including
Windows, Mac, and Linux platforms. Since Python is an interpreted
programming language, Python programs can be run on multiple platforms
without being recompiled.
Python comes with a robust standard library of modules, functions, and tools.
Every module that comes with Python is a powerful tool you can use without
creating additional code. Python comes pre-packaged with modules that
assist in the creation of various web services, manipulating strings, and
working with the operating system’s interface. Python also makes it easy for
users to create their libraries and frameworks, meaning that there is a large,
open-source Python community continually creating a wide variety of
applications. These applications can significantly speed up/simplify the
development of your application.
Despite its simplicity, Python is also sturdy and robust enough to carry out
sophisticated scientific and mathematical tasks. Python has been designed
with features that drastically simplify the visualization and analysis of data,
and Python is also the go-to choice for the creation of machine learning
models and artificial intelligence.
For all of these reasons, Python is one of the fastest-growing and most in­
demand computer programming skills.

A Note on Python Versions

There are various versions of Python available for use. It is highly
recommended that you use version 3.7 or advanced when following along
with this book. While Python 2 remains popular in some communities,
support for Python 2 will end in 2020, meaning that security issues will not
be resolved, and additional improvements won’t be made to it. Once Python
2 is officially retired, only Python 3.5 and advanced will see continued
support. Python 2’s syntax is a little different from Python 3’s syntax, and
this will not teach Python 2 because its retirement is impending.

Definitions: Interpreter, Terminal, Shell, IDE

Early on in this book, and as you continue to program with Python, you will
see many references to concepts like “interpreter,” “terminal,” “shell,” and
“IDE.” These concepts can be somewhat confusing for a beginner, so to
make things simpler, let’s define these concepts here. If you are already
somewhat familiar with these programming concepts and just looking to
learn Python as another language, feel free to skip this section.
An “interpreter” in the computer science/programming sense is a computer
program that can execute code, carrying out the written instructions specified
by a programming or scripting language. An interpreter carries out code
immediately and directly. In contrast, a “compiler” is a program that
translates instructions into efficient machine code. Meanwhile, a “shell” is a
wrapper or environment whose primary function is to run other programs and
the word shell is often used to refer to the command-line of the OS. The
command line takes in commands centered on the name of applications the
user wishes to interact with. The interface you see above is an example of the
Python shell, and it is running an interpreter.
Python has its shell; an interactive interpreter specialized for running Python
commands. It lets the user immediately execute Python code and see the
result as soon as the user enters the command. The Python shell that can be
accessed through the command-line is an example of a “terminal,” which is
simply the environment that allows the user to input text and receives
outputs. For the purpose of this book, the terms “shell” and “terminal” may
be used interchangeably in reference to an instance of the Python interpreter
accessed through the command line.

The Python Interpreter

There are two main ways to work with Python: with the interpreter and
command line or with an Integrated Development Environment (IDE).
We will be doing the majority of our programming in an IDE, but first, let’s
make sure you understand how to work with Python in the terminal.
Let’s start by opening the terminal/command prompt and checking that
Python is installed correctly by just typing the command “python.” If Python
is properly installed, the command prompt should transition you to an
instance of the Python interpreter/shell. This interpreter allows you to create
and run Python code. For instance, if you copied this line of code into the
terminal, you’d get “Using the terminal interpreter!” printed back out:
print ("Using the terminal interpreter!")
The command print() is responsible for printing out to the terminal whatever
is specified inside the parentheses.
Most programming is done in an IDE, but it is still a good idea to learn how
the Python interpreter works because there may be occasions where you may
have to do some programming in it. With that in mind, let’s take a few
moments to familiarize ourselves with the Python interpreter.
As mentioned, the Python interpreter can typically be invoked from the
command line only by entering the command “Python,” or perhaps the
specific Python version you want to run:
python3.8
The interpreter can typically be exited with the quit command: exit() - or
depending on the version you are running - quit().
The help() command is an incredibly helpful command that you will always
want to remember because it shows you all the various commands and
functions that you can use in the interpreter.
When you enter a command by hitting the return key, the statement will be
evaluated for correct syntax. If there is a syntax error, the error will be
displayed.
Python is waiting for a command if you see the “primary prompt,” which is
often indicated by the presence of three greater-than signs (>>>). If you are
on the second line of an input, these greater than signs will instead be
replaced with three periods.

Using an IDE
I wanted to make you aware of the Python interpreter in the terminal’s
existence, but most of our programming will be done in an IDE. If you
experimented with the terminal a little bit, you’d quickly find a significant
disadvantage of using the terminal, and it is that you can’t preserve many
lines of code on the same screen. In addition, whenever you enter a line of
code, and it contains any errors, a syntax error will be thrown immediately.
IDEs make the process of learning a language simpler because they will often
highlight syntax errors for you. Other benefits of using an IDE include auto­
completion for specific key phrases and functions, more accessible
collaboration with other programmers, and the ability to make changes to a
script even while an instance of the programming is running.
You can try out the code examples found in this either the terminal or in an
IDE. However, most of the examples presented in this will be presented in an
IDE. One excellent IDE is PyCharm (https://www.j etbrains.com/pycharm),
an open-source IDE designed from the ground up for use with Python.
PyCharm highlights syntax errors enables easy refactoring/renaming of files
and comes with an integrated debugger. PyCharm also has an integrated
terminal, and when you run programs in PyCharm, the results of the
program’s execution will be displayed in the terminal at the bottom of the
IDE.

Using PyCharm
Let’s go over some of the functions in PyCharm in greater detail, so that you
are familiar with how to use it.
After installing PyCharm and setting it up for the first time, you may be
slightly intimidated by all the options, but don’t worry, you won’t be using
most of these options for the exercises in this book.
 3 Ей” V'c* Kavi^ate Code Befactor Ryn loots VCS Window Help Python Tutorial c ','jsrrs' Darnel PythcnTtr ■ al] PyCharm

to Python Tutorial

s И Project -Г о ■=-!>-
- ► ■ PythonTut
tJ File
_ ► External lit1
X Cut Ctri+X ta New Scratch File Ctrl» Alt »Shrft»lnsert
TQ Scratches a
to Directory
Iq Copy Cttl+£
О Python Package
Copy Path Ctri+Shift*C
Copy Relative Path Ctrt*Att»Shift*C *р*аюп№

n Paste Ctri+V HTML File

Ф EdrtorConfig File
Find Usages Alt+F7
Find in £ath„. Cttt+Shift+F ni Resource Bundle

Replace in Path.. Ctrl» Shift* R Search Everywhere Double Shift

Inspect Code-

Refactor ► Go to File Ctrl+Shift+N

Clean Python compiled Files

a
Recent Files Ctrl + E
Add to Favorites
Show Image Thumbnails Orl+Shift+T
Navigation Bar Alt+Home
Reformat Code Ctrl» Alt» L
Optimize Imports Ctrt»Alt»O Drop files here to open
Show in Explorer
Directory £ath Cfrt+Alt+F12
a Open m Terminal

Local History ►
О Synchronize PythonTutonal
i Compare With- Ctrl*D
i
Mark Directory as ►
Remove BOM
*
О Create Gist-

As you can see in the image above, when you open PyCharm and are
confronted with the interface, you can navigate up to the file option in the top
left corner. Opening the file drop-down menu will let you either open an
existing project or create a new project. Opening an existing project enables
you to reopen projects you’ve already started and saved or even open the
projects that other people have worked on and which you have
downloaded/cloned. For now, just create a new project for the exercises
through the “File” option in the top left.
The New Project dialog box may look slightly different depending on which
version of PyCharm you are using, but it should ask you to select a project
interpreter. The default virtual environment (virtualenv) is beautiful for now,
and it should automatically detect your base Python interpreter if it is
correctly installed on your computer.
After this, you can create a folder to hold the scripts you create by right­
clicking in the project frame and choosing the “New” option from the drop­
down menu. To create a new Python script, just right-click on the folder
you’ve created and navigate to “New” and then the “Python File” option.
Now just enter a name for your new file Python file.
After you create a new Python file, it should automatically open in the editor
panel to the right. You can now enter code into the editor. If, for some reason,
the editor didn’t automatically open the file, just double click on the file to
open it up in the editor.
PyCharm should automatically save changes to the file, which means you
don’t need to worry about manually saving them. If for some reason, the file
doesn’t auto-save, or you just want to be sure it has saved, you can right-click
on the file to be presented with a drop-down menu that should contain the
option to save the file. You can also press Ctrl + S to save all the files
currently open in PyCharm.
Once you’ve written some code and want to try running it, you can either
navigate up to the “Run” tab on the top toolbar and select “Run (Current file
name here),” or press Shift + F10. The image above shows a program has
finished its run in PyCharm’s compiler. Note that the results of the program
are printed to the built-in terminal.
 Chapter 15:

Variables and Constants in Python:

What Is A Variable in Python?

When writing complex codes, your program will demand data essential to
conduct changes when you proceed with your executions. Variables are,
therefore, sections used to store code values created after you assign a value
during program development. Python, unlike other related language
programming software, lacks the command to declare a variable as they
change after being set. Besides, Python values are undefined like in most
cases of programming in other computer languages.
Variation in Python is therefore described as memory reserves used for
storing data values. As such, Python variables act as storage units, which feed
the computer with the necessary data for processing. Each value comprises of
its database in Python programming, and every data is categorized as
Numbers, Tuple, Dictionary and List, among others. As a programmer, you
understand how variables work and how helpful they are in creating an
effective program using Python. As such, the tutorial will enable learners to
understand declare, re-declare, and concatenate, local and global variables as
well as how to delete a variable.
Variable vs. Constants
Variables and constants are two components used in Python programming
but perform separate functions. Variables, as well as constants, utilize values
used to create codes to execute during program creation. Variables act as
essential storage locations for data in the memory, while constants are
variables whose value remains unchanged. In comparison, variables store
reserves for data while constants are a type of variable files with consistent
values written in capital letters and separated by underscores.
Variables vs. Literals
Variables also are part of literals which are raw data fed on either variable or
constant with several literals used in Python programming. Some of the
common types of literals used include Numeric, String, and Boolean, Special
and Literal collections such as Tuple, Dict, List, and Set. The difference
between variables and literals arises where both deal with unprocessed data
but variables store the while laterals feeds the data to both constants and
variables.
Variables vs. Arrays
Python variables have a unique feature where they only name the values and
store them in the memory for quick retrieval and supplying the values when
needed. On the other hand, Python arrays or collections are data types used in
programming language and categorized into list, tuple, set, and dictionary,
which will be later. When compared to variables, the array tends to provide a
platform to include collectives functions when written while variables store
all kinds of data intended. When choosing your charming collection, ensure
you select the one that fits your requirements henceforth meaning retention of
meaning, enhancing data security and efficiency.

Classifications of Python Arrays Essential for

Variables
Lists
Python lists offer changeable and ordered data and written while
accompanying square brackets, for example, "an apple," "cherry." Accessing
an already existing list by referring to the index number while with the ability
to write negative indexes such as '-1' or '-2'. You can also maneuver within
your list and select a specific category of indexes by first determining your
starting and endpoints. The return value with therefore be the range of
specified items. You can also specify a scale of negative indexes, alter the
value of the current item, loop between items on the list, add or remove
items, and confirming if items are available.
Dictionaries
Python dictionaries comprise of indexed, changeable but unordered items
typically written while with curly brackets with keys and values. Some of the
activities involved include item access by use of a keyword inside the
parentheses; conduct value changes, loop, check critical availability, length of
the dictionary, and both adding and removing unwanted items. Besides,
Python allows you to copy the dictionary by writing 'dict2 = dict1'. 'dict2' will
become a representation to 'dict1' therefore makes any necessary changes
automatically. Another way of creating a copy is also by using a built-in
Dictionary technique, that is, 'copy.'
In other instances, Python dictionaries can also have other dictionaries within
it a process referred to as nested dictionaries. You can readily determine the
number of dictionaries present in the nest through creating of three already
available. You can also generate your dictionary through the 'dict()'
contractor function. The function enables the copying of the earlier dictionary
or the creation of a completely new one. Within the Python dictionary, there
exist several built-in techniques to implement and enjoy the efficiency of the
dictionaries present.

Naming Variables
The naming of variables remains straightforward, and both beginners and
experienced programmers can readily perform the process. However,
providing titles to these variables accompany specific rules to ensure the
provision of the right name. Consistency, style, and adhering to variable
naming rules ensure that you create an excellent and reliable name to use
both today and the future. The rules are:
• Names must have a single word, that is, with no spaces
• Names must only comprise of letters and numbers as well as
underscores such as (_)
• The first letter must never be a number
• Reserved words must never be used as variable names
When naming variables, you should bear in mind that the system is case­
sensitive, hence avoid creating the same names within a single program to
prevent confusion. Another important component when naming is
considering the style. It entails beginning the title with a lowercase letter
while using underscores as spaces between your words or phrases used.
Besides, the program customarily prevents starting the name with a capital
letter. Begin with a lowercase letter and either mix or use them consistently.
When creating variable names, it may seem so straightforward and easy, but
sometimes it may become verbose henceforth becoming a disaster to
beginners. However, the challenge of creating sophisticated names is quite
beneficial for learned as it prepares you for the following tutorials. Similarly,
Python enables you to write your desired name of any length consisting of
lower- and upper-case letters, numbers as well as underscores. Python also
offers the addition of complete Unicode support essential for Unicode
features in variables.
Specific rules are governing the procedure for naming variables; hence
adhere to them to create an exceptional name to your variables. Create more
readable names that have meaning to prevent instances of confusion to your
members, especially programmers. A more descriptive name is much
preferred compares to others. However, the technique of naming variables
remains illegible as different programmers decide on how they are going to
create their kind of names.
Methods of Creating a Multi-Name for Python Variables
Pascal case: this method entails the first, second, and subsequent words
in the name as capitalized to enhance readability, for example,
ConcentrationOfWhiteSmoke.
Camel case: the second and subsequent words of the name created
remains capitalized, for example, the ConcentrationOfWhiteSmoke.
Snake case: snake method of creating variable names entails separator
of words using an underscore as mentioned earlier, for example,
concentration_of_white_smoke.

Learning Python Strings, Numbers and Tuple

Python strings are part of Python variables and comprise of objects created
from enclosing characters or values in double-quotes. For example, 'var =
Hello World’. With Python not supporting character types in its functions,
they are however treated as strings of one more characters as well as
substrings. Within the Python program, there exist several string operators
making it essential for variables to be named and stored in different formats.
Some of the string operators commonly used in Python are [], [:], ‘in’, r/R, %,
+ and *.
There exist several methods of strings today. Some include replacing Python
string () to return a copy of the earlier value in a variable, changing the string
format, that is, upper and lower cases and using the 'join' function, especially
for concatenating variables. Other methods include the reverse function and
split strings using the command 'word.split'. What to note is that strings play
an important role, especially in naming and storage of values despite Python
strings being immutable.

Types of Data Variables

String
A text string is a type of data variable represented in either String data types
or creating a string from a range of type char. The syntax for string data
comprises multiple declarations including 'char Str1[15], 'char Str5[8] =
“ardiono”; among others. As to declare a string effectively, add null character
'Str3', declare arrays of chars without utilizing in the form of 'Str1' and
initialize a given array and leave space for a larger string such as Str6. Strings
are usually displayed with doubles quotes despite the several versions of
available to construct strings for varying data types.
Char
Char are data types primarily used in variables to store character values with
literal values written in single quotes, unlike strings. The values are stores in
numbers form, but the specific encoding remains visibly suitable for
performing arithmetic. For instance, you can see that it is saved as 'A' +, but it
has a value of 66 as the ASCII 'A' value represents 66. Char data types are
usually 8 bits, essential for character storage. Characters with larger volumes
are stored in bytes. The syntax for this type of variable is 'char var = val';
where 'var' indicates variable name while 'val’ represents the value assigned
to the variable.
Byte
A byte is a data type necessary for storing 8-bit unsigned numbers that are
between 0 to 255 and with a syntax of 'byte var = val;.' Like Char data type,
'var' represents variable name while 'val’ stands for the value to he assigned
that variable. The difference between char and byte is that char stores smaller
characters and with a low space volume while byte stores values which are
larger.
Int
Another type of data type variable is the int, which stores 16-bit value
yielding an array of between -32,768 and 32,767, which varies depending on
the different programming platforms. Besides, int stores 2’s complement
math, which is negative numbers, henceforth providing the capability for the
variable to store a wide range of values in one reserve. With Python, this type
of data variable storage enables transparency in arithmetic operations in an
intended manner.
Unsigned int
Unsigned int also referred to, as unsigned integers are data types for storing
up to 2 bytes of values but do not include negative numbers. The numbers are
all positive with a range of 0 to 65,535 with Duo stores of up to 4 bytes for
32-byte values, which range from 0 to 4,294,967,195. In comparison,
unsigned integers comprise positive values and have a much higher bit. The
syntax for unsigned int is ‘unsigned int var = val;’ while an example code
being ‘unsigned int ledPin = 13;’
Float
Float data types are values with point numbers, that is to say, a number with a
decimal point. Floating numbers usually indicate or estimate analog or
continuous numbers, as they possess a more advanced resolution compared to
integers. The numbers stored may range from the highest of 7.5162306E+38
and the lowest of -3.2095174E+38. Floating-point numbers remain stored in
the form of 32 bits taking about 4 bytes per information fed.
Unsigned Long
This is data types of variables with an extended size hence it stores values
with larger storages compare to other data types. It stores up to 32 bits for 4
bytes and does not include negative numbers henceforth has a range of 0 to
4,294,967,295. The syntax for the unsigned long data type is 'unsigned long
var = val;’ essential for storing characters with much larger sizes.
 Chapter 16:

How to Install Python

Installing Python (Windows)

Here we introduce the installation and operation of Python under the
Windows environment. Since there is no built-in Python environment in the
Windows operating system, it needs to be installed independently. The
installation package can be downloaded from Python's official website
(www.Python.org). After opening the official website search for the
navigation bar that has a "Downloads" button.
The website recommends a link by default because it can detect your
operating system and recommend the latest version of Python 3.x, 3.6.5.
After entering the download page of the corresponding version, there is a
basic introduction to the environment you are trying to download. Several
different versions are mainly aimed at different operating system platforms.
You can choose different files to download according to whether your system
is 32-bit or 64-bit.
In the new page that opens, we can find other versions, including the latest
test version and the required version 3.4. If you want to install a 64-bit
version of 3.6.5, click the framed link on the current page.
At the bottom of the newly opened page, we can find several other links. The
file that starts with the Windows entity represents the 64-bit version of
Windows, while the file that does not include 64 represents the 32-bit
version.
The website shows a compressed installation package (Windows x86-64
Embedded ZipFile), an executable installation file, and a Web-based
installation file (Windows x86-64). It is most convenient to download the
executable installation package.
Note: 64-bit version cannot be installed on a 32-bit system, but a 32-bit
version can be installed on a 32-bit system or 64-bit system.
Install Python
The Windows executable installation package is easier to install. Just like
installing other Windows applications, we just need to select the appropriate
option and click "Next" to complete the installation.
When the options appear in the installation, do not rush to the following step
(the system demonstrated here is 64-bit in itself).
It must be noted that after "Add Python3.6 to PATH" is checked and Python
3.6 is added to the environment variable, Python's interactive prompt or
Python's commands can be started directly and conveniently at the Windows
command prompt in the future.
After selecting "Add Python 3.6 to PATH", select custom installation. Of
course, it is also possible to select the first item for installation, which means
Python is installed in the user directory of C disk. But at this time, you'd
better know what the user directory is so that you can find the installed
Python.exe files when necessary in the future.
Proceed with the instructions and python will be installed successfully in the
system.

Installing Python (Mac)

If you’re using a Mac, you can download the installation package from this
link:
https://www.python.org/downloads/mac-osx/
The progression of learning is getting further into Python Programming
Language. In reality, Python is an adaptable yet powerful language that can
be used from multiple points of view. This just implies Python can be used
intelligently when code or a declaration is to be tried on a line-by-line
premise or when you're investigating its highlights. Incredibly, Python can be
used in content mode, most notably, when you want to decipher a whole
document of declarations or application program.
Working with Python, be that as it may, requires most extreme caution -
mainly when you are drawing in or connecting with it. This caution is valid
for each programming language as well. So as to draw in with Python
intelligently, the Command Line window or the IDLE Development
Environment can be used.
Since you are an apprentice of either programming by and large or using
Python, there will shift ventures on how you could connect with and
cooperate with Python programming language. The following are essential
highlights of activities for brisk cooperation with Python:

The Command Line Interaction

Associating with the order line is the least demanding approach to work, as a
novice, with Python. Python can simply be imagined by seeing how it
functions through its reaction to each finished direction entered on the >>>
brief. The Command Line probably won't be the most favored commitment
with Python, at the same time, throughout the years, it has demonstrated to be
the easiest method to investigate how Python functions for learners.
Launching Python using the Command Line
If you're using macOS, GNU/Linux, and UNIX frameworks, you should run
the Terminal tool to get to the command line. Then again, if you are using
Windows, you can get to the Python order line by right-clicking on the Start
menu and launching Windows PowerShell.
As directions on programming require a contribution of an order, when you
need Python to do something for you, you will train it by entering directions
that it knows about a similar yield. This is an adjustment in the order may
give the ideal yield; be cautious.
With this, Python will make an interpretation of these directions to guidelines
that your PC or gadget can comprehend and execute.
Let’s take a look at individual guides to perceive how Python functions. Note
that you can use the print order to print the all-inclusive program
"Heydays, Savants!"
1. Above all else, open Python's command line.
2. At that point, at the >>>prompt, type the accompanying (don't leave space
among print and the section): print ("Heydays, Savants!")
3. Now, you should press enter so as to disclose to Python that you're finished
with the direction. Promptly, the direction line window will show Heydays,
Savants! In the interim, Python has reacted similarly as it has been told in the
composed arrangement that it can relate with. Then again, to perceive how it
will react wrongly when you request that it print a similar string using a
wrong linguistic structure for the print order, type and enter the
accompanying direction on the Python order brief: Print("Heydays,
Savants!")
The outcome will be: Syntax error: invalid language structure
This is a case of what get when you use invalid or fragmented explanations.
Note that Python is a case-touchy programming language, so at whatever
point you misunderstand the message it could be that you composed print
with a capital letter. Obviously, there is a choice to print direction, you can
simply type your announcement inside statements like this: "Primes,
Savants!" Note that an announcement is the words you wish to show once the
order is given; the words that can fit in are not confined to the model given
here, however.
The most effective method to leave the Python order line
To exit from Python, you can type any of these commands: quit() or exit().
Subsequently, hold Control-Z and afterward press Enter; the Python should
exit.
Your commonality with Python Programming ought to get fascinating now;
there are still parts to learn, tolerance will satisfy.
The area of IDLE: Python's Integrated Development Environment (IDE)
A standout amongst the fascinating pieces of Python is the IDLE (Integrated
Development and Learning Environment) apparatus. Despite the fact that this
specific device is incorporated into Python's establishment bundle, you can
download increasingly refined outsider IDEs as well. The IDLE instrument
gives you access to an increasingly active stage to compose your code and
work engagingly with Python. To get to IDLE, you can experience a similar
organizer where you found the direction line symbol or on the begin menu (as
you've gained from order line collaboration). When you click on the IDLE
symbol, you will be coordinated to the Python Shell window. This will take
us to the segment on cooperation with the Python Shell Window.
Connecting with the Python Shell Window
When you're at the Python Shell Window, you will see a dropdown menu and
a >>>prompt that resembles what you've found in the direction line window
(the primary connection talked about). There is a specific IDLE's function of
altering for the drawing in past order. Now, you will use a similar IDLE's
altering menu to look back to your past directions, cut, copy, and glue past
statements and, taking all things together, make any type of editing. Clearly
the IDLE is increasingly similar to a jump from the direction line association.
Incredibly, in the menu dropdown of the Python Shell window are the
accompanying menu things: File, Windows, Help, Shell, Options, Edit, and
Debug. Every one of these menus has various functions. The Shell and
Debug menus are used while making more significant projects as they give
get highlights to the procedure. In any case, while the Shell menu gives you a
chance to restart the shell or look the shell's log for latest reset, Debug Menu
has loads of valuable things for following the source record of an exemption
and featuring the blundering line. With the Debugger option, you will most
likely introduce an intelligent debugger window that will enable you to step
and look through the running projects on the Python. The Options menu of
the window enables you to edit and set IDLE to suit your own Python
working inclinations.

Start Python
Python can be started in two ways.
1) Start Python's Own IDLE
If you want to run Python, you can click the "start" button on the Windows
desktop and enter "IDLE" in the search box that appears to launch a Python
desktop application to provide a prompt of "read-evaluate-print-loop
quickly".
IDLE is Python's own simple IDE (Integrated Development Environment),
which is Python's graphical interface editor. IDLE can be regarded as a
simple version of an integrated development environment. Its function looks
simple, but it is helpful for beginners to learn the Python language itself.
Here, a REPL environment is provided, that is, it reads the user's input
("Read"), returns to the car, evaluates and calculates ("Evaluate"), then prints
the result ("Print"), and then a prompt "Loop" appears, thus circulating.
2) Start Python at Windows Prompt
Another way to start Python is to run Python programs through the Windows
command prompt, and enter "cmd" in the Windows search box (or press
"Win+R" key to open the run prompt box, note the "Win" key on the
keyboard), or click the start button to enter "cmd" in the pop-up search box
and enter to start the Windows command line window.
Note: the flashing cursor after ">" seen here is the command prompt provided
by Windows.
When installing Python, since the "Add Python 3.6 to PATH" option is
checked and the installed Python is added to the environment variable of
Windows, Python can be successfully started by entering "python" after the
prompt ">".
The prompt "> > >" indicates that Python installation was successful and
Python has been started. The prompt "> > >" is Python-specific.
Next, "print("Hello Python!" is executed in either the first or second startup
mode. ) ".
If you want to return to the Windows command prompt, you can reach the
goal by pressing the shortcut key "Ctrl+Z".
The above two methods are both REPL forms, suitable for writing relatively
short programs or commands, and have the advantages of simplicity and
flexibility. If the program has more functions and more modules or packages
are called, the REPL form is not very convenient to maintain.
 Chapter 17:

Designing and Using Functions

Python Functions
Python functions are a right way of organizing the structure of our code. The
functions can be used for grouping sections of code that are related. The work
of functions in any programming language is to improve the modularity of
code and make it possible to reuse code.
Python comes with many in-built functions. An excellent example of such a
function is the “print()” function which we use for displaying the contents on
the screen. Despite this, we can create our functions in Python. Such
functions are referred to as the “user-defined functions”.
To define a function, we use the “def” keyword which is then followed by the
name of the function, and then the parenthesis (()).
The parameters or the input arguments have to be placed inside the
parenthesis. The parameters can also be defined within parenthesis. The
function has a body or the code block and this must begin with a colon (:) and
it has to be indented. It is suitable for you to note that the default setting is
that the arguments have a positional behavior. This means that they should be
passed while following the order in which you defined them.
Example:
#!/usr/bin/python3
def functionExample():
print('The function code to run')
bz = 10 + 23
print(bz)
We have defined a function named functionExample. The parameters of a
function are like the variables for the function. The parameters are usually
added inside the parenthesis, but our above function has no parameters. When
you run above code, nothing will happen since we simply defined the
function and specified what it should do. The function can be called as shown
below:
#!/usr/bin/python3
def functionExample():
print('The function code to run')
bz = 10 + 23
functionExample()
It will print this:

The function code to run

That is how we can have a basic Python function.

Function Parameters
You can dynamically define arguments for a function. Example:
#!/usr/bin/python3
def additionFunction(n1,n2):
result = n1 + n2
print('The first number is', n1)
print('The second number is', n2)
print("The sum is", result)
additionFunction(10,5)
The code returns the following result:

The first number is 10

The second number is 5
The sum is 15

We defined a function named addFunction. The function takes two

parameters namely n1 and n2. We have another variable named result which
is the sum of the two function parameters. In the last statement, we have
called the function and passed the values for the two parameters. The
function will calculate the value of variable result by adding the two
numbers. We finally get the result shown above.
Note that during our function definition, we specified two parameters, n1 and
n2. Try to call the function will either more than two parameters, or 1
parameter and see what happens. Example:
#!/usr/bin/python3
def additionFunction(n1,n2):
result = n1 + n2
print('The first number is', n1)
print('The second number is', n2)
print("The sum is", result)
additionFunction(5)
In the last statement in our code above, we have passed only one argument to
the function, that is, 5. The program gives an error when executed:
Traceback (most recent call last):
■
File "main.py", line 9, in
additionFunction(5)
TypeError: additionFunction() missing 1 required positional argument: ’n2‘

The error message simply tells us one argument is missing. What if we run it
with more than two arguments?
#!/usr/bin/python3
def additionFunction(n1,n2):
result = n1 + n2
print('The first number is', n1)
print('The second number is', n2)
print("The sum is", result)
additionFunction(5,10,9)
We also get an error message:
Traceback (most recent call last):
File "main.py", line 9, in
additionFunction(5,19,9)
TypeError: additionFunction() takes 2 positional arguments but 3 were given

The error message tells us the function expects two arguments but we have
passed 3 to it.
In most programming languages, parameters to a function can be passed
either by reference or by value. Python supports parameter passing only by
reference. This means if what the parameter refers to is changed in the
function; the same change will also be reflected in the calling function.
Example:
#!/usr/bin/python3
def referenceFunction(ls1):
print ("List values before change: ", ls1)
ls1[0]=800
print ("List values after change: ", ls1)
return
# Calling the function
ls1 = [940,1209,6734]
referenceFunction( ls1 )
print ("Values outside function: ", ls1)
The code gives this result:
List values before change: [940, 1209, 6734]
I

List values after change: [800, 1209, 6734]

Values outside function: [800, 1209, 6734]

What we have done in this example is that we have maintained the reference
of the objects which are being passed and then values have been appended to
the same function.
In following example below, we are passing by reference then the same
reference will be overwritten inside the same function which has been called:
#!/usr/bin/python3
def referenceFunction( ls1 ):
ls1 = [11,21,31,41]
print ("Values inside the function: ", ls1)
return
ls1 = [51,91,81]
referenceFunction( ls1 )
print ("Values outside function: ", ls1)
The code gives this result:
Values inside the function: [11, 21, 31, 41]
Vlaues outside function: [51, 91, 81]

Note that the “ls1” parameter will be local to the function

“referenceFunction”. Even if this is changed within the function, the “ls1”
will not be affected in any way. As the output shows above, the function
helps us achieve nothing.

Function Parameter Defaults

There are default parameters for functions, which the function creator can use
in his or her functions. This means that one has the choice of using the
default parameters, or even using the ones they need to use by specifying
them. To use the default parameters, the parameters having defaults are
expected to be last ones written in function parameters. Example:
#!/usr/bin/python3
def myFunction(n1, n2=6):
pass
In above example, the parameter n2 has been given a default value unlike
parameter n1. The parameter n2 has been written as the last one in the
function parameters. The values for such a function may be accessed as
follows:
#!/usr/bin/python3
def windowFunction(width,height,font='TNR'):
# printing everything
print(width,height,font)
windowFunction(245,278)
The code outputs the following:

The parameter font had been given a default value, that is, TNR. In the last
line of the above code, we have passed only two parameters to the function,
that is, the values for width and height parameters. However, after calling the
function, it returned the values for the three parameters. This means for a
parameter with default, we don’t need to specify its value or even mention it
when calling the function.
However, it’s still possible for you to specify the value for the parameter
during function call. You can specify a different value to what had been
specified as the default and you will get the new one as value of the
parameter. Example:
#!/usr/bin/python3
def windowFunction(width,height,font='TNR'):
# printing everything
print(width,height,font)
windowFunction(245,278,'GEO')
The program outputs this:

245 278 GEO

Above, the value for parameter was given the default value “TNR”. When
calling the function in the last line of the code, we specified a different value
for this parameter, which is “GEO”. The code returned the value as “GEO”.
The default value was overridden.
 Chapter 18

A Modular Approach to
Program Organization

Modules, also known as packages, are a set of names. This is usually a library
of functions and object classes that are made available to be used within
different programs. We used the notion of modules earlier in this part to use
some function from the math library. We are going to cover in-depth on how
to develop and define modules. To use modules in a Python program, the
following statements are used: import, from, reload. The first one imports the
whole module. The second allows import only a specific name or element
from the module. The third one, reload, allows reloading a code of a module
while Python is running and without stopping in it. Before digging into their
definition and development, let’s start first by the utility of modules or
packages within Python.

Modules Concept and Utility within Python

Modules are a very simple way to make a system component organized.
Modules allow reusing the same code over and over. So far, we were working
in a Python interactive session. Every code we have written and tested is lost
once we exit the interactive session. Modules are saved in files that make
them persistent, reusable, and sharable. You can consider modules as a set of
files where you can define functions, names, data objects, attributes, and so
on. Modules are a tool to group several components of a system in a single
place. In Python programming, modules are among the highest-level unit.
They point to the name of packages and tools. Besides, they allow the sharing
of the implemented data. You only need one copy of the module to be able to
use across a large program. If an object is to be used in different functions
and programs, coding it as a module allows share it with other
programmers.
To have a sense of the architecture of Python coding, we go through some
general structure explanation. We have been using so far in this very simple
code examples that do not have high-level structure. In large applications, a
program is a set of several Python files. By Python files, we mean files that
contain Python code and have a .py extension. There is one main high-level
program and the other files are the modules. The high-level file consists of
the main code that dictates the control flow and executes the application.
Module files define the tools that are needed to process elements and
components of the main program and maybe elsewhere. The main program
makes use of the tools that are specified in the modules.
In their turn, modules make use of tools that are specified in other modules.
When you import a module in Python, you have access to every tool that is
declared or defined in that specific module. Attributes are the variables or the
functions associated with the tools within a module. Hence, when a module is
imported, we have access to the attributes of the tools as well to process
them. For instance, let’s consider we have two Python files named file1.py
and file2.py where the file1.py is the main program and file2.py is the
module. In the file2.py, we have a code that defines the following function:
def Xfactorial (X):
P=1
for i in range (1, X + 1):
P *= i
return P
To use this function in the main program, we should define code statements
in the file1.py as follows:
Import file2
A = file2.Xfactorial (3)
The first line imports the module file2.py. This statement means to load the
file file2.py. This gives access to the file1.py to all tools and functions
defined in file2.py by the name file2. The function Xfactorial is called by the
second line. The module file2.py is where this function is defined using the
attributes’ syntax. The line file2.Xfactorial() means fetch any name value of
Xfactorial and lies within the code body of file2. In this example, it is a
function that is callable. So, we have provided an input argument and
assigned the output result to the variable A. If we add a third statement to
print the variable A and run the file file1.py, it would display 6 which is the
factorial of 3. Along Python, you will see the attribute syntax as
object.attribute. This allows calling the attributes that might be a function or
data object that provides properties of the object.
Note that some modules that you might import when programming with
Python are available in Python itself. As we have mentioned at the beginning
of this book, Python comes with a standard large library that has built-in
modules. These modules support all common tasks that might be needed in
programming from operating system interfaces to graphical user interface.
They are not part of the language. However, they can be imported and comes
with a software installation package. You can check the complete list of
available modules in a manual that comes with the installation or goes to the
official Python website: www.Python.org. This manual is kept updated every
time a new version of Python is released.

How to Import a Module

We have talked about importing a module without really explaining what
happens behind in Python. Imports are a very fundamental concept in Python
programming structure. In this section, we are going to cover in-depth how
really Python imports modules within a program. Python follows three steps
to import a file or a module within the work environment of a program. The
first step consists of finding the file that contains the module. The second step
consists of compiling the module to a byte-code if required. Finally, the third
step runs the code within the module file to build the objects that are defined.
These three steps are run only when the module is imported for the first time
during the execution of a program. This module and all its objects are loaded
in the memory. When the module is imported further in the program, it skips
all three steps and just fetch the objects defined by the module and are saved
in memory.
At the very first step of importing a module, Python has to find the module
file location. Note that, so far in the examples we presented, we used import
without providing the complete path of the module or extension .py. We just
used import math, or import file2.py (an example of the earlier section).
Python import statement omits the extension and the path. We just simply
import a module by its name. The reason for this is that Python has a module
that looks for paths called 'search path module'. This module is used
specifically to find the path of module files that are imported by the import
statements.
In some cases, you might need to configure the path search of modules to be
able to use new modules that are not part of the standard library. You need to
customize it to include these new modules. The search path is simply the
concatenation of the home directory, directories of PYTHONPATH,
directories of the standard library, and optionally if the content of files with
extension .pth when they exist. The home directory is set automatically by
the system to a directory of Python executable when launched from the
interactive session, or it can be modified to the working directory where your
program is saved. This directory is the first to be searched when import a
module is run without a path. Hence, if your home directory points to a
directory that includes your program along with the modules, importing these
modules does not require any path specification.
The directory of the standard library is also searched automatically. This
directory contains all default libraries that come with Python. The directories
of PYTHONPATH can be set to point toward the directory of new modules
that are developed. In fact, PTYHONPATH is an environment variable that
contains a list of directories that contains Python files. When
PTYHONPATH is set, all these paths are included in the Python environment
and the search path directory would search these directories too when
importing modules. Python also allows defining a file with .pth extension that
contains directories, one in each line. This file serves the same as
PTYHONPATH when included appropriately in a directory. You can check
the directories’ paths included when you run Python using sys.path. You
simply print sys.path to get the list of the directories that Python will be
searching for.
Remember, when importing a module, we just use the name of the module
without its extension. When Python is searching for a module in its
environment paths, it selects the first name that matches the module name
regardless of the extension. Because Python allows using packages that are
coded in other languages, it does not simply select a module with .py
extension but a file name or even a zip file name that matches the module
name being imported. Therefore, you should name your modules distinctly
and configure the search path in a manner that makes it obvious to choose a
module.
When Python finds the source code of the module file with a name that
corresponds to the name in the import statement, it will compile it into byte
code in case it is required. This step is skipped if Python finds an already byte
code file with no source code. If the source code has been modified, another
byte code file is automatically regenerated by Python while the program runs
in other further executions. Byte code files have typically .pyc extension.
When Python is searching and finds the module file name, it will load the
byte code file that corresponds to the latest version of the source code with
.py extension. If the source code is newer than the byte code file, it will
generate a new one by compiling the source code file. Note that only
imported files have corresponding files with .pyc extension. These files, the
byte code files, are stored on your machine to make the imports faster in
future use.
The third step of the import statement is running the module’s byte code.
Each statement and each assignment in the file are executed. This allows
generating any function, data objects, and so on defined in the module. The
functions and all attributes are accessed within the program via importers.
During this step, you will see print statements if they exist. The 'def '
statement will create a function object to be used in the main program.
To summarize the import statement, it involves searching for the file,
compiling it, and running the byte code file. All other import statements use
the module stored in memory and ignore all the three steps. When first
imported, Python will look in the search path module to select the module.
Hence, it is important to configure correctly the path environment variable to
point to the directory that contains new defined modules. Now that you have
the big picture and the concept of modules, let’s explore how we can define
and develop new modules.
PART 5
 Chapter 19

Using Methods

String literals are usually surrounded by single quotation marks and double
quotation marks. For example, the world expression ‘string’ is written in the
same way as “string.” You can print it in the shell with the print() function,
just like I did with the data types in Python shell. The first step is to assign a
string to some variable of your choice. You can write down the name of the
variable that you want to use, which can be followed by the equal sign and
then the string. Please note that you can use either a single alphabet or a full
name as the name of a variable. Use them wisely in a program so that when
you read the code, you know the job of each variable.
>>> myString = "I am learning deep learning with python."
>>> print(myString)
I am learning deep learning with python.
>>> myString = "I am studying deep learning with python."
>>> print(myString)
I am studying deep learning with python.
>>> myString = """I am studying deep learning,
with Python,
and I am really enjoying it,
and writing programs with it."""
>>> print(myString)
I am studying deep learning,
with Python,
and I am really enjoying it,
and writing programs with it.
>>> myString = '''I am studying deep learning,
with Python,
and I am really enjoying it,
and writing programs with it.'''
>>> print(myString)
I am studying deep learning,
with Python,
and I am really enjoying it,
and writing programs with it.
>>>
Like many programming languages, Python strings are like byte arrays,
which represent Unicode characters. There is no character data type in
Python. However, a single character is a string that has a length of 1. You can
use square brackets to access elements of the string.
>>> myString = """I am studying deep learning,
with Python,
and I am really enjoying it,
and writing programs with it."""
>>> print(myString[10])
i
>>> print(myString[1])
>>> print(myString[2])
a
>>> print(myString[0])
I
>>>
Do you love slicing? Slicing always has a satisfying effect on the human
brain. Python strings offer you the freedom to return a wide range of
characters with the help of using the slice syntax. The first step in this regard
is to specify the starting index and the ending index. Separate the two by a
colon to return a part of the string.
>>> myString = """I am studying deep learning,
with Python,
and I am really enjoying it,
and writing programs with it."""
>>> print(myString[20:35])
earning,
with P
>>> print(myString[10:35])
ing deep learning,
with P
>>> print(myString[0:50])
I am studying deep learning,
with Python,
and I am
>>>
Python allows you to slice a string by using negative indexing as well. Let’s
see how you can do that. The only difference is that you will use negative
numbers in the string.
>>> myString = """I am studying deep learning,
with Python,
and I am really enjoying it,
and writing programs with it."""
>>> print(myString[-50:-1])
really enjoying it,
and writing programs with it
>>> print(myString[-60:-1])
,
and I am really enjoying it,
and writing programs with it
>>> print(myString[-70:-10])
ith Python,
and I am really enjoying it,
and writing program
>>>
You can calculate the total length of a string by using the following code.
>>> myString = """I am studying deep learning,
with Python,
and I am really enjoying it,
and writing programs with it."""
>>> print(len(myString))
100
>>>
String Methods
The first method that will come under discussion is the strip() method that
does the job of removing whitespaces from your string at the start or the end.
>>> myString = """ I am studying deep learning,
with Python,
and I am really enjoying it,
and writing programs with it. """
>>> print(myString.strip())
I am studying deep learning,
with Python,
and I am really enjoying it,
and writing programs with it.
>>>
In the following example, I will try three different methods on the same
string. One method is to convert the text into a lower case; the second is to
convert it into the upper case, while the third is to convert the text into title
case. All of them are simple and very handy when you are composing
messages that you have to display for your users.
>>> myString = """I am studying deep learning,
with Python,
and I am really enjoying it,
and writing programs with it."""
>>> print(myString.lower())
i am studying deep learning,
with python,
and i am really enjoying it,
and writing programs with it.
>>> print(myString.upper())
I AM STUDYING DEEP LEARNING,
WITH PYTHON,
AND I AM REALLY ENJOYING IT,
AND WRITING PROGRAMS WITH IT.
>>> print(myString.title())
I Am Studying Deep Learning,
With Python,
And I Am Really Enjoying It,
And Writing Programs with It.
>>>
Let’s talk about some more string methods to learn how it operates.
>>> myString = """I am studying deep learning,
with Python,
and I am really enjoying it,
and writing programs with it."""
>>> print(myString.replace("studying", "reading"))
I am reading deep learning,
with Python,
and I am really enjoying it,
and writing programs with it.
>>> print(myString.replace("really", " "))
I am studying deep learning,
with Python,
and I am enjoying it,
and writing programs with it.
>>> print(myString.replace("really", ""))
I am studying deep learning,
with Python,
and I am enjoying it,
and writing programs with it.
>>> print(myString.replace("studying",""))
I am deep learning,
with Python,
and I am really enjoying it,
and writing programs with it.
>>>
In the above example, I attempted to replace a word with a new word. Then I
moved on to eliminating a word by replacing it with no word. I tried it thrice
to explain how you can manage extra whitespaces that are likely to happen if
you don’t fix them. The best method is to eliminate extra spaces from the
code. There is a special method known as the split() method that can split the
string into several substrings.
>>> myString = """I am studying deep learning,
with Python,
and I am really enjoying it,
and writing programs with it."""
>>> print(myString.split(","))
['I am studying deep learning,' '\nwith Python,' '\nand I am really enjoying it,'
'\nand writing programs with it.']
>>> myString = "I, am, studying, deep, learning."
>>> print(myString.split(","))
['I', ' am', ' studying', ' deep', ' learning.']
>>>
There is another interesting method using which you can check if a certain
phrase or a character exists in a particular string or not. There are two
keywords ‘in’ or ‘not in’ that you can use for this method.
>>> myString = "I am studying deep learning."
>>> a = "studying" not in myString
>>> print(a)
False
>>> a = "studying" in myString
>>> print(a)
True
>>>
If you can recall the data types that I have shared with you earlier on, you
will realize that Python is communicating with you in the Boolean data type.
You have got the answer in False and True to your query. This method is the
way to extract more information about a Python string through a specific
method.
If you have got two strings, you can combine them easily by using the Python
string concatenation method. The primary operator that you can use here is
the ‘+’ operator. Let’s see how to do that.
>>> myString = "I am studying deep learning."
>>> myString1 = "with Python."
>>> combstring = myString + myString1
>>> print(combstring)
I am studying deep learning with Python.
>>> myString = "I am studying deep learning."
>>> myString1 = " with Python."
>>> combstring = myString + myString1
>>> print(combstring)
I am studying deep learning with Python.
>>>
The above code snippet has two similar code examples. The first one has a
flaw. There is no whitespace in the first after the word learning. Two words
have been wrongly combined. I fixed the issue by adding whitespace at the
start of the second string. This formula will be helpful for you when you are
combining two strings for writing a program. This method is the easiest;
however, there is another method that you can use to add necessary space
between two strings. Both play the same role, so it is up to you which one
you like the most.
>>> myString = "I am studying deep learning."
> >> myString1 = "with Python."
> >> combString = myString + " " + myString1
>>> print(combString)
I am studying deep learning with Python.
>>>
You can format your string at will by the following method. Formatting a
string means that you can combine two data types when you are writing a
program for a user. If you try to concatenate them by using the same
technique that we used for two strings, this is unlikely to work for you. Let’s
see what happens when you try to do that.
> >> myString = 23
> >> myString1 = "I am John and I am " + myString
Traceback (most recent call last):
File "<pyshell#98>", line 1, in <module>
myString1 = "I am John and I am " + myString
TypeError: can only concatenate str (not "int") to str
Don’t worry. There is a special method for this purpose, the format() method.
This method picks up the passed arguments, formats them, and then adds
them to the string where you put in the placeholders {}. Let’s see how to
insert numbers.
> >> myString = 23
> >> myString1 = "I am John and I am {}"
> >> print(myString1.format(myString))
I am John and I am 23
 Chapter 20:

Processing Files in Python

In Python, files are considered as a data object. In fact, Python has a built-in
data object type assigned specifically for files, the file data object is
associated only with common methods used to process files. The built-in
method open is a function that allows creating a file data object to be
processed. In short, this method links the file data object to the file stored in
the hardware of your machine. Once you call open function, you have access
to the file in order to read it or write it using the read and write attributes of
the file data object.
The table presented below lists all common methods used to process files. In
order to read a file, the function open is called with file name along with the
mode to process the file which is 'r' in this case. To write a file, the process
mode is 'w'. This mode creates a new file. If a file exists with the same name
used to open a file in a writing mode, it will be overwritten. In order to write
or add content into the existing file, the file should be open with mode
append (i.e. 'a').
The file name can include or not the file path of the directory that contains or
will contain the file. If the path is not specified, then Python would assume
that the file is in the working directory which is the directory where the
current program is running.

List of Methods to Process File

Method Explanation
File2Read = open
(file_path/file_name, 'r') Defines a file data object to write
File2write = open
(file_path/file_name, 'w') Defines a file data object to read
 Reads the whole file and assign the
content = File2read.read()
content to a single string
content = File2read.read(X) Reads only X bytes
Line = File2read.readLine() Reads following line
Reads the whole file and stores the
Lines = File2read.readLins()
content in a line strings list
File2write(data) Writes data in the file
Writes in the file the strings line of the
File2write.writeLine( List_line)
list List_line
File2write.close() Closes the file manually

After you open a file, you have a file data object. Then, its methods can be
used in reading and writing with the methods presented in the table above. In
either case, the file data object methods take and return only strings in
Python. In other words, the read method returns as a data object type as a
string. The write method takes as data object type as string, too. Both
methods have different varieties.
The role of the close method is closing the connection between Python and
the external file residing in the hardware of your machine. Python also
liberates the space in memory that was occupied by an object after it is no
longer referenced in the script. Python would also close automatically the
close if required. Hence, in Python, it is not necessary to call the close
method in order to delete the file object reference manually. However, it is
good practice to call the close method after you finish reading or writing a
file.

Example File Processing in Python

In this section, we are going to present some examples of processing files in
Python using the methods presented in Table 10. The first example illustrates
how to write the ‘Hello World’ in a file. So, let’s go back to prompt shell and
launch Python for practice.
 > > > File = open (' MyFile.txt’, 'w')
> > > File.writef Hello World! \nr)
> > > File.doseO
In the first statement, we called the method open in write mode (i.e. 'w') to
create the file. The second statement writes the line ‘Hello World!’ with a
newline marker. The third statement closes the file object. In the following
code example, we are going to open the file in reading mode (i.e. 'r') and get
the line written in the file:

>>> File = open (' MyFile.txt', 'r')

>>> A = File.readline()
>>> print (' This is an example of reading a file with readline: \n', A)
This is an example of reading a file with readline:
Hello World!
>>> File.close()
Now, we are going to add a second line in our file. To do so, we are going to
open the file in mode append (i.e. 'a') and write the line 'My first file in
Python':
>>> File = open (' MyFile.txt', 'a')
>>> File.write(' This is my first file in Python')
>>> File.close()
If we open the file again and check its content with the read method, we get
the following output:
>>> File = open (' MyFile.txt', 'r')
>>> A = File.read()
>>> print (' This is an example of reading a file with read: \n', A)
This is an example of reading a file with read:
Hello World!
This is my first file in Python
>>> File.close()
As you can notice when opening a file with append mode, it adds whatever
you write in the file at the end. We can also change what is already in the file
by opening the file in mode 'r+'. When you open the file in this mode and
write in it, it will overwrite everything in it. For instance:
>>> File = open (' MyFile.txt', 'r+')
>>> File.write(' This is my first file Python opened in mode r+')
>>> File.close()
>>> File = open (' MyFile.txt', 'r')
>>> A = File.read()
>>> print (' Checking file after opening in mode r+: \n', A)
Checking file after opening in mode r+:
This is my first file Python opened in mode r+
>>>File.close()
Remember, loops are very handy when it comes to repeating the same tasks
for a specific number of times. In particular, loops are very useful in
processing the file data object. We have seen through the above examples
that we can read the content of the whole file in one single step using the read
method. In some cases, we need to read the file line by line. In this case, we
would use the readLine method. We might also need to write the file line by
line in the case of formatted files, in which case, the writelines is handy. Let’s
practice some examples. First, we are going to write using writelines method.
Then we are going to read the same file line by line. For both tasks, we will
use a while loop.
>>> List_string = [' This is an example of \n',
... 'writing a file \n',
... ' on multiple lines\n',
... ' using write Lines \n',
... ' inside a while loop']
>>>print (' List of strings is:\n ', List_string)
List of strings is:
[' This is an example of \n', 'writing a file \n', ' on multiple lines\n', ' using
write Lines', ' inside a while loop']
>>> File = open ('Test', 'w')
>>> File.writelines (List_string)
>>> File.close ()
Remember that all file object methods process only strings. Therefore, we
created a list of strings where each element end with /n for newline maker.
Now, we open the file for reading:
>>> File = open ('Test', 'r')
 >>> while 1:
... Line = File.readline ()
... if not Line: break
... print (Line)
...
This is an example of
writing a file
on multiple lines
using write Lines inside a while loop
>>> File.close ()
As you can notice, we have a break statement in the while loop. This ensures
that the while loop stops when there is no line to read. In other words, it
ensures that the loop exit when it reaches the end of the file. Because 1 is
always evaluated to true, the loop will continue running until it runs to the
break. Hence, this loop reads the file line by line until it reaches the end of
the file. If we open the file and use the read method, we get exactly the same
results:
>>> File = open ('Test', 'r')
>>> A = File.read ()
>>> print ('This is the output from reading the file with reading method: \n',
A)
This is the output from reading the file with reading method:
This is an example of
writing a file
on multiple lines
using write Lines inside a while loop
>>> File.close ()
Note in this last example, we did not specify the file extension. In fact,
Python allows handling and processing any type of files that the extension
does not matter. These methods work the same on any file.
 Chapter 21:

Python Algorithms

Some of the Best Python Algorithms for Data

Analysis
We have spent some time in this guide looking at some of the basics of the
Python language, as well as some of the basics of data analysis and what we
are able to do with this for some of our own needs. Now it is time for us to
take both of these ideas and put them together.
In order to complete a good data analysis in the first place, we need to be able
to run the data that we have collected through a number of algorithms,
usually, a few, though in some cases, one will be fine. These algorithms are
going to help us to learn what patterns and insights are found in that data, and
can give us some good predictions in the end.
There are a number of different Python data analysis algorithms that we are
able to work with along the way, and it is going to often depend on the what
we would like to do with that data to start with, and how we are able to
benefit from this as well. With this in mind, let’s dive right into some of the
basics that come with these algorithms, and learn a bit more about why
Python algorithms are the best for data analysis.

Neural Networks
The first kind of Python algorithm that we are able to use for data analysis is
going to be known as the neural network. This is a fairly sophisticated type of
algorithm to work with in the first place, but it is going to provide you with a
lot of the power and more that you need to get some of the work done.
The neural network is going to work in a manner similar to how the brain will
function. It is going to form connections to the things that it gets right, and it
can remember this for future use as well. This helps it to learn as time goes
on, and makes it easier and faster and more efficient at some of the work that
it needs to complete as well. Once it has been able to go through and learn
something new, you will find that it will remember it in the future.
In addition, the neural network is really good at helping with things like
identifying what is in an image. It does this by going through the different
layers of the pictures, one at a time, and then making a good prediction of
what is going to be found in that image. The more layers that it is able to go
through, the more accurate it will be at guessing what is in the picture as well.
This makes it an effective tool to use for many of the projects you are
working with.
However, we have to keep in mind that the computational costs are going to
be a bit higher with this one. Because we are working with an algorithm that
is able to methodically go through a bunch of layers and make accurate
predictions, we can see why the costs will be higher with this one. You have
to determine ahead of time if this is the right kind of process for you to work
with, or if one of the other algorithms out there would work better.

Clustering
There are a number of different clustering techniques that you are able to
work with when it comes to handling a data analysis. These are all going to
take the data, though and put them together in clusters so you can see more
about the distribution of where your points of data are going to fall. There are
different ways that you are able to work with clustering, and different ways
that you can cluster up your data, but it is a good way to put the information
into groups and see some good insights right away.
The number of clusters that you will have in your work is going to depend on
what you are focusing on. For example, if you are working to divide your
group into males and females, then you will only need to have two clusters.
But if you would like to divide things up by the age of your customers, or by
where they live in the country or the world, then you will probably need to
have more than two clusters in order to have the best idea of what is going
on.
With these clusters, you will be able to plot them on something like a
scatterplot, and then figure out where everything is going to fall. This is a
great way to figure out the best options to take, whether there is a new
demographic that you need to work with, and more. You can just put these
clusters on your scatterplot, and you can learn a lot of information in a short
amount of time.
The idea of these clusters is pretty simple. When you have some data points
that fall into the same cluster, you can safely make the assumption that the
points are going to have some similarities with one another. But if the data
points are not in the same cluster, then this is a sign that they are going to be
different from one another. If we are able to keep this in mind, it is a lot
easier for us to work with understanding this kind of algorithm.

Support Vector Machines

These are going to be the algorithms that you are able to use in order to figure
out the right course of action to take. When you see that the line and the
hyperplane come together, then this is a good idea that the information that is
there is going to lead you to the results that you are looking for. This can
make picking out the right decision easier, as long as you know the right
steps that are going to help you to get the best results as well.
The biggest trick that comes with this one sometimes is figuring out which
lines and hyperplanes we should pick and work with. Sometimes, depending
on the type of data that you are working with and what is all in it, you will
end up with more than one hyperplane. Sometimes this will not be a big deal
because you will be able to see which one is the best for your needs. Other
times it may seem like there is more than one answer that is going to work
the best for your needs as well.
There are a number of options that we are able to work in order to pick out
the right SVM for what we want to do. You will learn how to do this more
with experience, but you will be able to learn about your data and then figure
this out as well. Sometimes experimenting with a few of these to see which
one is the best is another good option that we are able to choose as well.

Naive Bayes
The Naive Bayes algorithm is a good choice to go with when you want to do
a bit of exploration with the data in the beginning. Maybe you want to see
what the best way is to split up the data that you have, or you are not yet
certain about what kind of algorithm is going to be the best one for you to
focus your attention on yet. In some cases, you may need to show some of the
data and some of the information that you have ahead of time, right after
collecting it, to those who want to see what is going on, but may not
understand all of the more technical aspects that come with it.
This is where the Naive Bayes algorithm is going to come into play and can
really help us. With this option, we are able to take a good exploration of the
data that we have, and then determine the best steps to take after. Sometimes
this helps us to choose which of the other algorithms are the best ones for us
to go with. And other times, it may be a good way to create a beginner's
model so that we can show this off before being able to finish all of the work
for the final project.
The Naive Bayes algorithm is usually not going to be the first choice that we
make when it is time to handle some of our data, and we will usually go
through and make a few other adjustments to the process as well and finish
off with another kind of algorithm. But it is definitely a good algorithm to go
with because it adds in a lot of the different parts that you need to get a good
idea about what the data contains, and what else we are able to do with it
along the way.

Decision Trees
Another option that we are able to go with here is the decision trees. These
trees are going to help us to make some good decisions and compare some of
the likely outcomes of the decisions that we do make. If you are uncertain
about which steps to take, especially if you have a few choices to pick from,
then the decision tree is going to be one of the best algorithms for you to
choose from.
To start, you would be able to ask your question and implement it in the
information or the data that you have. Of course, we first need to clean and
organize it, but we will assume that this part is already done. When you are
ready, the data can tell us which steps are the best, and the likely outcome of
each choice that you try to make along the way. This can help you to make
some better decisions because you can look at the data in a more clear and
organized manner, and you know that it actually makes sound sense, rather
than just using your intuition along the way.
In addition, you are able to take a bunch of these decision trees and combine
them together into a random forest. The random forest is simply going to be
when we combine together a bunch of those decision trees in order to
compare more than one option.
Maybe you want to see which is best for you. If you ever want to compare
more than one idea or thought for you to take, and then working with the
random forest, or even a simple decision tree, can get this done. You just add
in the information that you want to work with, and then compare the
decisions and their potential outcomes before going.
As we can see, there are a lot of different types of algorithms that we are able
to use when it comes to handling our data and making sure that it makes as
much sense as possible. A bit point of working with data analysis is to help
us to sort through all of that data. Otherwise, that data is just going to sit
around and be worthless to us and just sitting there taking up space.
Each of the algorithms that we went through above is meant to help us to
figure out different ways that we are able to move around our data, and
different ways that we will come to a better understanding of what is inside
of our data, and how we can use that.
Whether you want to be able to make smarter decisions for your business, or
you are looking for ways to find new customers or beat out the competition,
the algorithms above, with the help of the Python code, will be able to help
you get there.
 Chapter 22:

Databases

Data management is not a scientific discipline per se. However, increasingly,

it permeates the activities of basic scientific work. The increasing volume of
data and increasing complexity has long exceeded manageability through
simple spreadsheets.
Currently, the need to store quantitative, qualitative data and media of
different formats (images, videos, sounds) is very common in an integrated
platform from which they can be easily accessed for analysis, visualization or
simply consultation.
The Python language has simple solutions to solve this need at its most
different levels of sophistication. Following the Python included batteries, its
standard library introduces us to the Pickle and cPickle module and, starting
with Version 2.5, the SQLite3 relational database.

Specifying the Database

johnsmith= os.path.expanduser (' ~ /. johnsmith’ )
if not os.path.exists (at the dir):
os.mkdir (at the dir)
sqlhub.process Connection = connectionForURI (' sqlite://’+johnsmithr
+’/knowdb’)
In specifying the database, we create the directory (os.mkdir) where the
database will reside (if necessary) and we will natively connect to the
database. We use os.path.exists to check if the directory exists. Since we
want the directory in the user's folder, and we have no way of knowing
beforehand what this directory is, we use os.path.expanduser to replace /
home/user as it would normally on the Unix console.
On line 11 of Specifying the database, we see the command that creates the
connection to be used by all objects created in this module.
Next, we specify our database table as a class, in which its attributes are the
table columns.
Specifying the database ideatable.
class Idea (SQLObject): name= UnicodeCol() nlinks= IntCol()
links= Pickle Col() address = StringCol
The class that represents our table is inherited from the SQLObject class. In
this class, each attribute (table column) must be assigned an object that gives
the type of data to be stored. In this example, we see four distinct types, but
there are several others. UnicodeCol represents texts encoded as Unicode, i.e.
it can contain characters from any language. IntCol is integer numbers.
PickleCol is an exciting type as it allows you to store any type of Python
object.
The most interesting thing about this type of column is that it does not require
the user to invoke the pickle module to store or read this type of variable.
Variables are automatically converted/converted according to the operation.
Finally, we have StringCol which is a simpler version of UnicodeCol,
accepting only ASCII character strings. In SQL it is common to have terms
that specify different types according to the length of the text you want to
store in a variable. In sqlobject, there is no limit to the size of the text that can
be stored in either StringCol or UnicodeCol.
The functionality of our spider has been divided into two classes: Crawler,
which is the creeper itself, and the UrlFac class that builds URLs from the
word you want in Wikipedia.
Each page is pulled by the urllib2 module. The urlencode function of the
urllib module makes it easy to add data to our request so as not to show that it
comes from a digital spider. Without this disguise, Wikipedia refuses the
connection.
The pages are then parsed by the VerResp method, where BeautifulSoup has
a chance to do its work. Using the SoupStrainer function, we can find the rest
of the document, which doesn't interest us, by analyzing only the links (tags
'a') whose destination is URLs beginning with the string/wiki/. All Wikipedia
articles start this way. Thus, we avoid chasing external links. From the soup
produced we extract only the URLs, i.e. what comes after “href =”.

The Pickle Module

The pickle module and its fastest cPickle cousin implement algorithms that
allow you to store Python-implemented objects in a file.
Example of using the pickle module
import pickle
class hi:
def say_hi (self):
print " hi "
a= hi()
f= open ('pic test’,’w’)
pickle.dump(a, f)
f.close()
f= open ('pic test’,’r’)
b=pickle.load (f)
b.say_hi()
hi
As we see in the example of using the pickle module, with the pickle module
we can store objects in a file, and retrieve it without problems for advanced
use. However, an important feature of this module is not evident in example
8.1. When an object is stored using the pickle module, neither the class code
nor its data are included, only the instance data.
class hi:
def say_hi (self, name=‘ alex’):
print’hi %s !’%name
f= open ('pictest’,’r’)
b=pickle.load (f)
b.say_hi()
hi alex !
This way we can modify the class, and the stored instance will recognize the
new code as it is restored from the file, as we can see above. This feature
means that pickles do not become obsolete when the code they are based on
is updated (of course this is only for modifications that do not remove
attributes already included in the pickles).
The pickle module is not built for data storage, simply, but for complex
computational objects that may contain data themselves. Despite this
versatility, it is because it consists of a readable storage structure only by the
pickle module itself in a Python program.
The SQLite3 Module
This module becomes part of the standard Python library from Version 2.5.
Therefore, it becomes an excellent alternative for users who require the
functionality of an SQL1-compliant relational database.
SQLite was born from a C library that had an extremely lightweight database
and no concept client-server. In SQLite, the database is a file handled through
the SQLite library.
To use SQLite in a Python program, we need to import the SQLite3 module.
import sqlite3
The following step is the creation of a connection object, through which we
can execute SQL commands.
c= sqlite 3.connect (' /tmp/ example’)
We now have an empty database consisting of the example file located in the
/ tmp directory. SQLite also allows the creation of RAM databases. To do
this, simply replace the file name with the string: memory. To insert data into
this database, we must first create a table.
c.execute ('‘‘ create table specimens (name text, real height, real weight)’’’)
< sqlite 3.Cursor object at 0 x83fed10 >
Note that SQL commands are sent as strings through the Connection object,
execute method. The create table command creates a table; it must
necessarily be followed by the table name and a list of typed variables (in
parentheses), corresponding to the variables contained in this table. This
command creates only the table structure. Each specified variable will
correspond to one column of the table. Each subsequent entry will form a
table row.
c.execute ('‘‘ insert into specimens values (' tom’, 1 2.5, 2.3)’’’
The insert command is another useful SQL command for inserting records
into a table.
Although SQL commands are sent as strings over the connection, it is not
recommended, for security reasons, to use the string formatting methods ('...
values (% s,% s)'% (1,2)) of Python Instead, do the following:
t= (' tom’,)
c.execute ('select from specimens where name=?’, t)
c.fetch all()
[(' tom’, 1 2.5, 2.2 9 9 9 9 9 9 9 9 9 9 9 9 9 9 8)]
In the example above we use the fetchall method to retrieve the result of the
operation. If we wanted to get a single record, we would use fetchone.
Below is how to insert more than one record from existing data structures. In
this case, it is a matter of repeating the operation described in the earlier
example, with a sequence of tubes representing the sequence of records to be
inserted.
t= ((' j e r r y’, 5.1, 0.2), (' butch’, 4 2.4, 1 0.3))
for i in t:
c.execute (' insert into specimens value s (?, ?, ?)’, i)
The cursor object can also be used as an iterator to get the result of a query.
c.execute (' selectfrom specimens by weight’)
for reg in c:
print reg
(' jerry’, 5.1, 0.2)
(' tom’, 1 2.5, 2.2 9 9 9 9 9 9 9 9 9 9 9 9 9 9 8)
(' butch’, 4 2.4, 1 0.3)
The SQLite module is really versatile and useful, but it requires the user to
know at least the rudiments of the SQL language. The following solution
seeks to solve this problem in a more Pythonic way.

The SQLObject Package

The SQLObject2 package extends the solutions presented so far in two ways:
it offers an object-oriented interface to relational databases, and also allows
us to interact with multiple databases without having to change our code.
To exemplify sqlobject, we will continue to use SQLite because of its
practicality.

Building a Digital Spider

In this example, we will have the opportunity to build a digital spider that
will gather information from the web (Wikipedia3) and store it in an SQLite
bank via sqlobject.
For this example, we will need some tools that go beyond the database. Let's
explore the ability of the standard Python library to interact with the internet,
and let's use an external package to decode the pages obtained.
The BeautifulSoup4package is a webpage breaker. One of the most common
problems when dealing with Html pages is that many of them have minor
design flaws that our browsers ignore, but can hinder further scrutiny.
Hence the value of BeautifulSoup: it is capable of handling faulty pages,
returning a data structure with methods that allow quick and simple
extraction of the desired information. Also, if the page was created using
another encoding, BeautifulSoup, returns all Unicode content automatically
without user intervention.
From the standard library, we will use the sys, os, urllib, urllib2 and re
modules. The usefulness of each character becomes clear as we move
forward in the example.
The first step is to specify the database. SQLObject allows us to choose from
MySQL, PostgreSQL, SQLite, Firebird, MAXDB, Sybase, MSSQL, or
ADODBAPI. However, as we have already explained, we will restrict
ourselves to using the SQLite bank.
 Chapter 23:

Exception Handling

What exactly are exceptions? Why haven’t we come across this so far? Well,
we have, or at least you may have quite a lot of time but never noticed before.
Create any program to make it crash deliberately.
Here’s one that I created:
name = "Bruce Wayne"
age = 45
print(name + age)
Process finished with exit code 1
See the error? Not the one in the middle, I am referring to the very last line. It
says the program ended with an exit code followed by the number one. That
‘one’ is informing us that something went wrong, and the program ended in a
crash or abruptly ended. If the code were zero, it would have meant that our
code went through and got executed beautifully!
We, as Python programmers, are bound to know when such errors are about
to come. It’s called anticipating, and it is something you should have already
done when you say the first three lines of my latest code.
The problem is, we programmers would know what this code means. For any
ordinary user, they would have no clue what this means and would end up
searching YouTube libraries, just to find a video that explains what the error
code 1 means. There is a way we can address this situation, and that is called
exception handling.
Before we begin, just remember the bold text that says ‘TypeError’ on our
error that occurred just a moment ago. We will need to recall that a little later.
Exception handling is where we tell a program to try a block of code and see
if the same works fine. If not, anticipate the type of error you will get. Except
for showing a console that is lit up with gibberish, we then tell Python to
print out a user-friendly text that means something. Don’t worry about the
words ‘try’ and ‘except’ because that is exactly what I will be showing you
now.
try:
name = "Bruce Wayne:"
age = 45
print(name + age)
except TypeError:
print("Please use a formatted string or convert age to a string")
We have just asked Python first to try a situation out. If the code is executed
without returning any errors, it’s fine. If not, it will type in a friendly message
instead of letting the users know what to do to avoid this. This will prevent
the application from crashing and keep users informed of the errors they may
have made. Now, let’s try and run this through to see what we get.
Please use a formatted string or convert age to a string
Process finished with exit code 0
Since we knew we were encountering a ‘TypeError’, we have just rectified
the situation and now look; the program ended with an exit code of zero.
Yay!
Here’s something for you to know! The exception we created here only deals
with a specific type of error. If this error were ValueError, the code would not
execute, and the program will still crash.
Let us look at how that works as well, shall we?
try:
age = 45
age1 = 0
average = age / age1
print(average)
except TypeError:
print("Please use a formatted string or convert age to a string")
What do you think will happen here? Would the program go through? Would
this program be able to catch the exception that might be caused, in case the
program decides to crash? Let’s find out.
Traceback (most recent call last):
File "C:/Users/Programmer/PycharmProjects/PFB/exception.py", line 4, in
<module>
average = age / age1
ZeroDivisionError: division by zero
Process finished with exit code 1
That was expected. Since the exception caused here is different from the one
we have created earlier on, it just went through and crashed. Time to put our
thinking caps on and come up with an exception to handle this situation.
try:
age = 45
age1 = 0
average = age / age1
print(average)
except ZeroDivisionError:
print("For God's sake! Who divides a number with 0?")
except TypeError:
print("Please use a formatted string or convert age to a string")
What do you think now? Will this work, or will we still end up with an exit
code of 1?
For God's sake! Who divides a number with 0?
Process finished with exit code 0
And that’s how you do this! Now, everyone knows what went wrong and
how they can correct the error they may have unintentionally caused.
It is something of a trait to have to be able to anticipate errors coming your
way beforehand. It comes with practice, but it is certainly a trait to have for
any programmer from any corner of the world.
The better you are at handling exceptions, the easier your users will find the
program/application to use. They would know what needs to be done, how to
sort matters out, and how to continue having a great experience while using
your written programs.

Debugging, Exception Handling in Detail

Straight away, by classes, I do not mean the regular classes that you would
expect at schools, colleges, and universities, nor do I mean that there are
qualities of Python in any way; they are completely different things.
Classes are not exclusive to Python, but they are as important as anything
else for any programmers across the globe. These are found in almost all
known computer programming languages.
In the simplest definition, classes are what we use to define new types of data
that we use. I did say there were three in the start, strings, numbers, and
Booleans. Then we came across little more complex things called lists,
tuples, and dictionaries. But what if you are still unable to get the desired
outcome from the program you have been working on for such a long time?
What if you feel like there must be something else apart from these types
which can help you achieve greater results? Fortunately, classes are your
answer.
A class can hold various functions and methods within itself. It does not need
parentheses like functions and methods, nor do we create these by using the
‘def’ keyword. These are created using the word ‘class’, and they can be
super helpful, especially for programmers with a keen interest in object-
oriented programming.
Creating Our First Class
Before you even begin to create a class, or function, or any other kind of
component, always visualize what you want to get out of it. It makes things a
lot easier for you as a programmer.
At this point, you may be blank and might be struggling to come up with a
class to create. Let me help you out with one. Let us create a class to which
we want specific functions and methods attached to. We want this class to do
things other data types were unable to do so.
I will not be creating anything that may fall outside the scope of this book, so
what I have here is easily understandable. However, there are a few things
that might take you by surprise, but those are deliberate so that you have
every chance of understanding what they are.
class Instructor:
def __init__(self, name):
self.name = name
def talk(self):
print("talk")
me = Instructor("Keanu Reeves")
print(me.name)
me.talk()
The first thing to notice here is the naming convention I have used to name
the class. Always use an uppercase for the first letter for every word that you
may type when naming a class. You do not need to use underscores to
separate words either. If you were to name this class as your first class, it
would look like this:
class MyFirstClass
Next, we have the familiar ‘def’ keyword. But what about the double
underscores and init? You may have already noticed these when calling
methods. These are called constructors. For now, all you need to know is that
we call upon these to initialize something.
Then, we have a parameter that says ‘self’, and I did not put it there. It is
something that will come up automatically. It is referencing itself. We have
only added another parameter called ‘name’ to allow us to use strings as
names to display. Next, we gave the object an attribute called ‘name’, as seen
above. Attributes are required to provide your functions a greater detail.
The following function is rather simple. We just created a function called to
talk and asked Python to print out the same on the prompt.
Moving forward, or downwards by two spaces, we created a variable called
‘me’ with an assigned value of the class we just created. Notice how I have
used class as a function (with parentheses). You might be wondering that I
just said moments ago that classes do not need parentheses, and yet, I am
using them here. When you are defining classes, you do not need these;
however, when you are using them, you will need to rely on them to pass
additional information.
Now, with the print command ready, I used my newly created class to call
upon an attribute of ‘.name’, which I created within this class. This then
allows the prompt to print out the name, followed by the last function, which
was again another print statement as defined above.
Classes are generally created so that other objects can be created using these
classes. Imagine two characters, Tony and Steve. We want each of these to be
objects carrying different attributes like name, age, and suit color. To do that,
we will first need to create a class. Let us go ahead and do so.
class Heroes:
def __init__(self, name, age, color):
self.name = name
self.age = age
self.color = color
def introduce_self(self):
print("My name is " + self.name)
print(f"I am about {self.age} years old")
print("My costume color is " + self.color)

hero1 = Heroes("Steve", 40, "Blue")

hero2 = Heroes("Tony", 38, "Red")
hero1.introduce_self()
hero2.introduce_self()
Output:
My name is Steve
I am about 40 years old
My costume color is Blue
My name is Tony
I am about 38 years old
My costume color is Red
We began by naming our classes appropriately. We then created a constructor
to help us create various attributes that we can call upon later. Before doing
so, ensure that you pass those as parameters after ‘self’ so that they are
recognizable by the program interpreter. After defining attributes and
assigning them values, we created a function called ‘introduce_self’ where
we had three statements printed. Notice how the second one is a formatted
string. That is because the age is an integer, and it will not work if you try to
merge a string and an integer on their own.
Once sorted and happy, we moved on to create objects called ‘hero1’ and
‘hero2’ from the same class. Now, instead of typing this information
separately, we just passed the information as arguments in the ‘Heroes()’
class. Next, we just ran a function we created earlier on, and the rest was just
plain history.
I know this might be a little complicated at first. Classes are a subject that is
normally apt for advanced students of Python. Still, it is essential that you, as
a programmer, do not just stop thinking you know everything and miss out on
some of the more advanced learning opportunities. Mastering classes will
take quite a bit of time. Might not suffice to give you the perfect command of
classes and some other aspects of programming. I intended to introduce you
to this vast world of advanced topics. How you practice, and research on this
is your call.
 Chapter 24:

Panda Python Data Analysis Library

This library is one of the best when it comes to machine learning and data
science, and will stand for Python Data Library. According to many sources
on this library, Pandas is going to be the name because it is derived from the
term of panel data. This is basically an econometrics term that handles data
sets that are multidimensional in structure.
Pandas are going to be seen as a bit game changer when it is time to analyze
the data that you have using the Python language, and it is often going to be
the number one Python library to use when it is time to handle data munging
and wrangling. It can also handle all of the other aspects of data science that
you want as well, making it an all in one library for your needs. Pandas are
also going to be open-sourced, free for any programmer to use, and is one of
the best data science libraries to focus on.
There are a lot of cool things that come with the Pandas library, so taking
some time to look it over and figure out what it all entails will help you with
a lot of data science projects. One thing that is cool with Pandas is that it is
able to take almost any kind of data and will then create an object in Python
with rows and columns. These are going to be called the data frame and it is
going to look pretty much like what we see with Excel. If you have worked
with the R programming language before, then you will see some similarities
here as well.
However, compared to working with the dictionaries or lists that come with
Python, or through loops or list comprehensions, the Pandas library is going
to be so much easier overall. The different functions that come with Pandas
can make it a much easier library to work with, especially when it comes to
some of the complexities of working with data science.
Installing the Pandas Library
The following thing that we need to take a look at here is how to actually
install the Pandas library and get it all set up. To install this library, we need
to have a Python version that is at least 2.7 or higher. The Pandas library is
not designed to work with any of the older versions of Python, so if you have
one of the older versions, it may be time to upgrade. At this time, you need to
make sure that some other deep learning libraries are in place. Pandas are
going to be dependent on a few other libraries, based on what you would like
to accomplish. It really needs to have at least NumPy associated with it, and
if you want to do something like plotting with your information, then you
need to work with Matplotlib.
Because you need a few extras that go with this library, we may want to
consider installing a package to make sure that all the extras are there when
you need them. The Anaconda distribution is a good option to work with and
it can work on all of the major operating systems including Linux, OS X, and
Windows systems.
Pandas is able to work with the Python IDE, including options like Spyder or
the Jupyter Notebook. But to get these to work, the Pandas library has to be
installed and ready to go. The Anaconda extension will come with both of the
IDE or Integrated Development Environment, so that can make things easier
to handle.
Importing one of these libraries means that you need first to load it into your
memory, and once the installation is all done, you will be able to open up the
needed files and work with them at any time. to make sure that you can
import Pandas in the right manner, all that you need to do is run the following
code below:
To make the codings easier, you can add second part (as pd) because it allows
you to access Pandas with just the pd.command, rather than having to go
through the process of writing out pandas.command each time that you wish
to use it. As we can see with the code above as well, you need to import
NumPy at this time. NumPy is a useful library to work with any scientific
computing in Python, and often the Pandas library will need to pull out
functions and other parts from this to get things done. At this point, Pandas is
up and running and ready for us to use.
The Benefits of Using Pandas
With that work done, it is time to take a look at some of the many different
benefits that come with using the Pandas library. There are a lot of benefits to
this one, and it is one of the most popular options that come with this kind of
data science work. With this in mind, let’s take a look at some of the benefits
that we are able to see with the Pandas library.
The first benefit that comes with the Pandas library is the data representation.
Pandas are going to provide programmers with a streamlined form of data
representation. This is going to be vital as you analyze and work to
understand the data that you hold onto a bit better. When you can simplify
some of the data representation that you have, it is going to facilitate better
results for some of your projects in data science.
The second benefit of this library is that it provides us with a way to get more
work done, without having to do as much writing. This is actually one of the
most significant advantages that we are able to see with this library. With the
traditional form of Python, we may have taken many lines of code to get the
work done, without any support libraries, but with Pandas, we can get that
same work done in just one or two lines of code. This means that by using
Pandas, we are able to shorten up the procedure of handling the data that we
have. When we can save all of that time, we are able to focus more on the
algorithms that we need for the data analysis.
An extensive set of useful features is following on the list of Pandas benefits.
Pandas are going to be seen as really powerful in the coding world. They are
able to provide us with a big set of commands and features that are important,
and which can be used to look through the data and analyze it quickly. We
can use Pandas in order to perform various tasks including filtering out the
data based on conditions that we set, or segregating and segmenting the data
according to the preferences that we would like to meet.
The following benefit that comes with working with Pandas is that this
library is able to handle a large amount of data in an efficient manner. When
the Pandas library was initially been created, its goal was to handle large sets
of data in an efficient manner. Pandas can really help us to save a lot of time
and hassle because it is able to import large amounts of data quickly and
efficiently.
The Pandas library is also able to make data customizable and flexible. There
is a massive set of features in Pandas that can be applied to the data that you
have. This can be great for beginners because it helps us to customize, edit,
and pivot that data according to what we want to see happen. This is going to
ensure that we can get the most out of our data each time.
And finally, the last benefit that we will see with the Pandas library is that it
is made for Python. Python is one of the most prominent and most sought-
after programming languages in the whole world, and it has an extensive
amount of features that we can enjoy. And with just the amount of
productivity that is offered, it is no wonder that many people want to learn
how to code in this language.
Because of this, and all of the great features that come with Python, the fact
that we are able to code with the help of Python in Pandas is going to be a
great thing. It allows the programmer to tap into the power of many libraries
and features that work with Python, which adds in some of the strength and
power that we need with our coding.
Now, there are a few disadvantages that come with this library compared to
some of the others, but often there are ways to work around these. Some of
the disadvantages that can come with the Pandas library that programmers
need to be aware of include:
The learning curve is steeper: Pandas was thought to have mild learning slow
in the beginning. But the more that you explore the library, the steeper the
learning curve is going to become. Sometimes the functionality of this library
is going to get confusing, and for beginners, this is going to bring on some
challenges.
The syntax can be hard: While Pandas is going to work with the Python
language, sometimes it is going to add in some challenges when it comes to
the syntax that has to be used. Switching back and forth between Python and
Pandas codes can cause some problems
It doesn’t work well with 3D matrices: If this is something that you want to
work with, it can be a drawback of this library. If you are planning on just
creating a 2D matrix, then this will not be a problem at all.
Wrong documentation: Without a right amount of documentation to go along
with the project, it can be challenging to learn a new library. The
documentation that comes in Pandas isn’t going to do much to help us get the
harder functions of the library done. This is going to slow down our learning
procedure and can make coding difficult.
Viewing and Inspecting the Data
One thing that Pandas is able to do to help with our data science project is to
work with viewing and inspecting the data. In reality, Pandas is able to help
with all of the various processes that you may want to do with data science,
but right now we are just going to focus on this part. You can use a variety of
the functions that come with Pandas in order to take a look at what is in the
data, figure out if there are any missing or duplicate values, and then make
the changes as needed to work on your data analysis.
With this in mind, it is also possible for us to get some statistics on the entire
series, or an entire data frame. Some of the codes that you would need to use
to make this happen includes:
Df.mean(): This one is going to help us return the mean of all our presented
columns.
Df.corr(): This one is going to return the correlation between the columns that
are in your frame of data.
Df.std(): This one is going to help us see what the standard deviation ends up
being between each of the columns.
Df.median(): This one is going to help us see what the median of each
column is like.
Df.min(): This one is going to help us see the lowest value that is present in
each of our columns.
Df.max(): This one is going to help us see the highest value in each of our
columns.
Df.count(): This one is going to help us by returning the number of all of the
non-null values that show up in each column of the data frame we are using.
 Conclusion

Thank you for making it to the end of Machine Learning with Python. Let us
hope that it was informative and able to provide you with all of the tools you
need to achieve your goals whatever they may be. The objective of this is to
present an introduction for the absolute beginners to machine learning and
data science
This covers the dominant machine learning paradigms, namely supervised,
unsupervised, semi-supervised, and reinforcement. This explains how to
develop machine learning models in general and how to develop a neural
network which is a particular method of performing machine learning. It
teaches how to train and evaluate their accuracy.
Python is a widely used programming language for different applications and
in particular for machine learning. This covers the basic Python programming
as well as a guide to use Python libraries for machine learning.
This presents machine learning applications using real datasets to help you
enhance your Python programming skills as well as machine learning basics
acquired through the book. These applications provide examples of
developing a machine learning model for predictions using linear regression,
a classifier using logistic regression and artificial neural network. Through
these applications, examples of data exploration and visualization using
Python are presented.
Nowadays, machine learning is used in every domain, such as marketing,
health care systems, banking systems, stock market, gaming applications,
among others. This book’s objective is to provide a basic understanding of
the significant branches of machine learning as well as the philosophy behind
artificial neural networks. This also aims at providing Python programming
skills for machine learning to beginners with no earlier programming skills in
Python or any other programming language.
Remember that deep learning is relatively easy, contrary to collective
thinking among programmers. The industry is quickly moving toward the top
to take control of machines. There are lots of people who are firmly in favor
of machine learning, but I have some solid reasons that deep learning is more
profitable than machine learning. Practically speaking, deep learning is a
subset of machine learning. It secures power and flexibility by learning the
world on the basis of the database that it has stored in the backend.
It works on the back of its hidden learning architecture that consists of
multiple layers with some dense layers. The data is processed through these
layers where the neural networks get to work on matching the input data with
the data that is stored in the databases in the backend. Upon each match, they
return the output on the basis of some highly educated findings that are
efficient and very well managed.
Once you have acquired the skills and understood the reasoning behind
machine learning models presented in this book, you will be able to use these
skills to solve complex problems using machine learning. You will also be
able to easily acquire other skills and use more advanced machine learning
methods. In this guide, I explained to you the basics of Python language.
Learning to program is like learning another language. It takes a lot of
patience, study, application, method, passion and above all perseverance.
What I can suggest is to do as much practice as possible by starting to rewrite
the hundreds of examples you find in this guide. Try to memorize them and
when you write the code, say it to yourself, in your mind (open bracket, close
round brackets and so on). In the beginning, this helped me a lot to memorize
better the various steps needed to write a program even if simple.
It is important not to feel like heroes when a program works but above all you
should not be depressed when you cannot find a solution to your
programming problems. The network is full of sites and blogs where you can
always find a solution.
I hope that this guide has been useful for you in learning Python.
BOOK 3 - PYTHON FOR DATA
ANALYSIS
PART 6
 Introduction

Congratulations on purchasing Python for Data Analysis and thank you for
doing so.
The following chapters will discuss everything that we need to know when it
comes to deep learning and a good data analysis. Many companies are
shifting the way they do things. Collecting data has become the norm, and
now they have to learn what is inside all of that data, and what insights they
can learn in the process. And this is exactly what a data analysis can help
them to do. With a bit of the Python language, an understanding of deep
learning, and some of the best deep learning libraries available we can create
our own models and put that data to work.

To start this guidebook, we are going to take a look at some of the basics of
deep learning and what data analysis is all about. There are a lot of buzz
words out there when it comes to machine learning and artificial intelligence,
and understanding how these two topics can fit into the mix is so important
for ensuring that we will get things done.
From there, we will move a bit into Python and how it fits into this process.
We will start this with some information on the best libraries form Python
that help with deep learning before moving on to how we can work with
some of the different types of neural networks you can create. This section is
also a great one to explore to learn more about some of the Python deep
learning libraries you can use including TensorFlow, Keras, and PyTorch and
how these can help us create some of the models that we want within the field
of deep learning and data analysis.

To end this guidebook, we will take a look at machine learning and how this
process can fit in with the other topics we have discussed, especially when
we look at deep learning. And then we end with a discussion on how deep
learning can help businesses with their own predictive analysis. This makes it
easier for companies to make data-based business decisions to help them get
ahead on the market.
A data analysis is an important part of any business, especially those who
have spent a lot of time collecting big data. But doing this in the right manner
can be critical to ensure we get accurate insights and predictions. When you
are ready to learn more

There are plenty of books on this subject on the market, thanks again for
choosing this one! Every effort was made to ensure it is full of as much
useful information as possible, please enjoy!
 Chapter 25:

What Is Deep Learning?

The world of data science and the various terms and processes that go with it
has really taken off steam. Many companies have started to realize that they
can use this information and the fact that computers and systems are able to
train themselves, for their own advantage, and they are excited to learn more
about how to make this happen.

No matter what industry you are in, processes like data analysis, machine
learning, artificial intelligence, and deep learning can come into play and
provide you with some great results in the process. But with this in mind, we
are going to spend some time focusing on deep learning and what it is able to
do for your business.
Deep learning is a process that can carry out what we need with the world of
machine learning, often using an artificial neural net that is composed of a lot
of levels arranged in a type of hierarchy to make things easier. The network is
going to learn something simple when you enter into the first level, and then
it will work to send that information, and everything that it has learned in that
part, over to the next level.
From here, the next level is able to take some of this simple information and
will try to combine it together with something that is seen as a bit more
complex, before passing all of that over to the third level. This is a process
that will just continue from one level to the next, with each level building
something that is more complex from any input that it received with the
previous level.

This is an interesting process that shows us exactly how deep learning is

meant to work, and why it is so valuable. It is basically a process where the
computer is able to teach itself how to learn, based on a simple program that
a data scientist is able to add to the system. It is that easy! We will talk about
some of the best libraries and the best algorithms from machine learning and
deep learning to make this happen, but having a good understanding of how it
all works can really make a difference in how you are able to use it.

What Is Deep Learning

The first topic that we need to dive into here is what deep learning is all
about. Deep learning is considered a function that comes with artificial
intelligence, one that is able to imitate, as closely as possible, some of the
workings we see in the human brain when it comes to creating patterns and
processing complex data to use with decision making. Basically, we can use
the parts of deep learning to help us take our machine or our system and teach
it how to think through things the same way that a human can, although at a
faster and more efficient rate.

Deep learning is going to be considered a subset of machine learning, which

is also a subset of artificial intelligence. It also has a network that is capable
of learning a lot from data that is unsupervised, along with data that is
unlabeled or unstructured. There are other names for this kind of learning as
well including deep neural network and deep neural learning.

So, to get a better idea of how this is going to benefit us, we first need to take
a look at how we can work with deep learning. The process of deep learning
has really evolved a lot in the past few years, going hand in hand with a lot of
the things we have seen in the digital era. This time period has really brought
about so much data, data that comes in so many forms. In particular, this data
is known as big data, and we will be able to draw it out of a lot of different
areas such as e-commerce platforms, search engines, social media, and more.

If a company uses some of the algorithms that come with machine learning,
they will be able to actually use all of the information that they are collecting.
They can use it to recommend products for their customers, to really work on
making predictions and finding patterns with the information so they can
really run their business the way that they would like.
You will notice though that this unstructured data is going to be pretty large,
and for an employee to go through this information and get the relevant parts
from it, it would take so long the information would no longer be relevant
and useful. And by the time they did, the information would be old, and the
world would have already moved on and presented different information. But
many companies still realized the potential that they could learn from all of
this information, even if it is pretty large, and many are looking at the
different ways that various systems of AI can help them get through this
information and gain the insights that they need.

With this in mind, it is important that we take some time to look at how deep
learning is going to work. Deep learning has evolved at the same time and
often at the same pace as we see with the digital era. This is an era that has
seen a big explosion of data in all forms, and from every region of the world.
This is a lot of data, and it is there to help businesses make informed
decisions that weren’t possible in the past.

Think about all of the information that you already have at your fingertips,
and you may not even realize that it is there. Before you even decide to start
working with big data, you already know that if you need to look up
something, you can head to your favorite search engine and it will all be
there. Our digital era is bringing out a ton of new information and data, and
the smart companies, the ones who would like to get ahead, are the ones who
not only gather all of that data, but who learn how to use it.

This data, which is often called big data, is drawn from a variety of sources
depending on what the business is trying to accomplish. these can come from
places like e-commerce platforms, search engines, online cinemas, search
engines, and more. The enormous amount of data that fits into the category of
big data is going to be readily accessible to anyone who wants it, and it is
possible to share it through a lot of different applications like cloud
computing.

However, this data, which is normally going to come to us in a form that is

unstructured, is so vast that if a person manually went through all of it, it may
take decades to extract out the information that is relevant to what they need.
Companies realize this, and they now that there is a lot of potential that can
be found in all of that data that they have collected. And this is why creating
and adapting artificial intelligence systems with automated support is
something that many of them have started to focus their attention on.

How Is Deep Learning Different from Machine

Learning?
The next thing that we need to focus on in this chapter is the idea of deep
learning and machine learning. To someone who hasn’t had much of a chance
to explore these topics and some of the processes that come with them, these
two terms are going to seem like they are identical and that we can use them
in an interchangeable manner. But in this section, we are going to explore
these a bit more and see how they are similar and how they are different.

One of the most common techniques in artificial intelligence that can be used
to help us process some of that big data we have been collecting for a long
time is known as machine learning. Machine learning is going to be an
algorithm that is self-adaptive. This means that it is able to learn from what
has happened in the past and can work on making increasingly better analysis
and patterns with data that is newly added over time, and even with some of
its own experiences.

There are a lot of examples of how this would be able to work in the real
world, and there are a lot of companies who already work to make this
happen for their needs. Let’s say that we take a look at a company that
handles digital payments. One of their main concerns is to keep the levels of
potential and actual fraud from occurring in the system. Instances of fraud
can cost them millions of dollars a year, if not more, and being able to catch
these ahead of time and understanding how to prevent these before they
happen could be a lifesaver for most of these financial companies.
The good news is that these digital payment companies could employ some
tools of machine learning for this kind of purpose. The computational
algorithm that would be added into your computer model will work to
process all of the transactions that happen on our digital platform and can
make sure that we find the right patterns in any set of data. If it has been able
to learn in the right manner, the algorithm will be able to point out any
anomaly and more that is detected in this pattern.

Deep learning, which is a subset of machine learning, can work in a similar

manner, but it does this in a more specific way and includes specific machine
learning algorithms to get things done. When we talk about the deep learning
process, we are looking at a form of machine learning that will utilize a
hierarchical level of artificial neural networks in order to make sure that the
process of machine learning is carried out properly.
These artificial neural networks are going to be built up much like the human
brain, and there are nodes of neurons that will connect with one another
similar to a web. These nodes are able to send information to each other and
will communicate with one another to make this process work well.

Unlike some of the traditional programs that are going to work in a linear
way to build up a new analysis with the data at hand, the hierarchical function
that comes with a system of deep learning is going to enable our machines
and any systems that we use with this process to go through the data with an
approach that is more nonlinear.

Let’s go back a bit to that example that we did with fraud protection with an
online payment company. If this company worked with a more traditional
approach to detecting things like money laundering and fraud they would rely
on the system just picking up on the amount of the transaction. This means
that they would only catch the issues when there was a large amount of
money taken out, and maybe a few times if there was a really strange location
that didn’t make sense to where the person was located. We can see where
this can run into some troubles because not every instance of fraud or money
laundering is going to include big amounts, and most people who try to do
these tasks are going to stick with smaller amounts to stay under the radar.
But when we work with deep learning, we are able to work with a technique
that is seen as nonlinear, and this would include things like the time of the
transaction, the type of retailer that is being used, the IP address, the
geographic location of the user and when the transaction happens, and any
other features that the company would like to rely on, in order to point out
when a transaction is likely to be fraudulent.

The first layer that comes with this process of the neural network is that we
will take the raw data and input it. This could include something like the
amount of the transaction. Then this is turned over as the output of that first
layer. The second layer will use that output to work on itself, and may
include some additional information like the IP address, and will pass all of
this as its results or output as well.

Then we move on to the third layer. This layer is going to take all of the
information from the second layer, and includes some more raw data, like the
geographic location, and will make the pattern of the machine even better
from here. This will continue on through all of the layers that the programmer
set up with the neural network until the system can determine whether the
transaction is legitimate or fraudulent.

Remember here that deep learning is going to be considered a function of

artificial intelligence that helps us to mimic the workings of what happens
inside the human brain, at least when it comes to processing through a lot of
data to make important decisions. Deep learning with artificial intelligence
can be useful because it can learn from the data, whether we have data that is
labeled, or if we are working with data that is both unlabeled and
unstructured. Deep learning, which is a subset of the popular machine
learning, can be used to help out almost any industry that we can think of, but
in the example that we talked about above, it is especially helpful with
detecting things like fraud and the laundering of money.

One Example of How Deep Learning Works

The next thing that we need to focus on is how deep learning works. And the
best way to really see how this kind of technology is going to benefit us and
give us some of the results we want is by taking a look at some of the
examples of it in action.

If we stick with that system of fraud detection that we worked on before with
machine learning, it is possible to create an example of deep learning in no
time. if the system for machine learning was able to create a model with
parameters built around the number of dollars a user is able to receive or
send, the method of deep learning can start to build on some of the results
that are offered through machine learning.
Each layer that comes with this neural network is able to work because it
helps to build-up on the previous layer. And with each of these layers, we are
going to be able to add on some important data, like the IP address, credit
score, the retailer and the sender, a social media event, the user, and even a
credit score based on what needs to happen with the data and the machine.

The algorithms that come with deep learning are going to be trained not just
to create some patterns from the transactions, but they will also let us know
when a pattern is signaling that there is a need for someone to come in and
investigate activity that may seem fraudulent. The final layer of this is going
to relay a signal over to the analyst, who can then choose to freeze the
account in question until the investigation is completed and they determine
whether or not money laundering is happening.

It is possible to work with deep learning across all industries with a number
of tasks based on what those industries need to accomplish. some commercial
apps, for example, are able to use this technology to help out with things like
image recognition. There are open-source platforms that come with consumer
recommendation apps and even some medical research tools that explore the
possibility of reusing drugs for new ailments. These are just a few examples
of what we are able to see when we add in deep learning and some of the
algorithms that come with this type of learning.

Why Does Deep Learning Matter?

Another thing that we need to explore a bit here is why deep learning is going
to matter so much? And why is this form of machine learning able to help us
attain such impressive results where other methods may fail? To make it
simple, deep learning works due to accuracy. The algorithms with deep
learning are able to achieve recognition accuracy at levels that are much
higher than was possible before.
This is helpful because it helps any consumer electronics meet the
expectations of the user, and it is so crucial when it comes to applications that
rely on safety such as the recent development of driverless cars. Some of the
recent advances in deep learning have been able to step up and improved to
the point wherein many cases, the systems that rely on deep learning are
actually able to outperform what a human can do manually in several tasks,
such as classifying the objects found I images.

While the idea of deep learning has been around for a number of years and
was first theorized in the 1980s, there are two big reasons why we are just
hearing about it today, and why it wasn’t really seen as something useful to
work with in the past.

These two reasons include:

1. To get the most out of deep learning, we needed to have a large
amount of labeled data. Getting this data is expensive, especially
when it comes to the large amounts that were needed to get this to
run.
2. Another issue is that deep learning requires us to have quite a bit of
computing power. High-performance GPUs have a parallel
architecture that is the most efficient when it comes to deep learning.
When we combine them with things like cloud computing or clusters,
this is going to enable the teams to reduce their time for training this
network. They can get it done in a few hours, rather than in a few
weeks.

How Does This Deep Learning Work?

To really understand this deep learning, we need to take this further and see
how it works. Most of the methods that we want to use with deep learning are
going to rely on the architecture of a neural network, which is why we often
refer to these models as deep neural networks. The term deep is there because
it will refer back to the number of hidden layers that show up in the neural
network. Traditional neural networks that you can create with machine
learning are just going to work with two or three hidden layers in most cases,
but deep networks can go up to 150, and it is likely this number will grow
over time.

The models of deep learning are trained because they rely on large sets of
data that are labeled, along with architectures of neural networks. These are
able to come together to learn features directly without the need for extraction
of the features manually.

While there are a few different types of deep neural networks that a
programmer is able to focus on, one of the best options, and the one that data
scientists are the most likely to use, is the convolutional neural network, or
CNN. A CNN is going to convolve together the features that are learned with
the data you've add-in, and then uses the 2D convolutional layers, which will
make this architecture well suited when it is time to process any 2D data,
including options like images.

CNN's are nice for a number of reasons, but they also help with eliminating
the need for manual feature extraction. What this means is that we do not
need to go through and identify features that are used to help classify the
images. CNN is going to work by extracting features right out of the image.
The relevant features will not be trained ahead of time because they will be
learned while the network is training on a collection of images that you
provide as we go.
This may sound complicated, but when we add in the automated feature
extraction to the mix, it is going to make things easier. It will ensure that the
models you make with deep learning end up being really accurate for a
variety of computer vision tasks, including object classification.

CNN's are able to learn how to detect a variety of features that come within
an image, using what could be tens of thousands of hidden layers in the
process. Every hidden layer is going to increase the complexity that comes
with the learned image features. An example of this would be that the first
hidden layer is going to be responsible for detecting the edges of the image,
and then the last layer, however many that will be, will work on detecting
some of the more complex shapes that will help it to figure out what objects
are in that specific image.
Creating and Training Our Deep Learning Models
We will take more time to get in-depth about this in a bit, but we are going to
take a moment to look at some of the basics that come into play when we
want to create and then train some of our own deep learning models. There
are three methods that are pretty common to work with, especially when we
want to focus on object classification including:
Training from scratch. To work on training our own deep network from
scratch, we have to take on a few big steps that can take some time. First, we
need to gather up a large amount of data, specifically a large labeled data set.
Then we have to design the architecture that we need for the network,
ensuring that it is going to be able to learn the features that are found in that
labeled set of data, and that it can model it as well.

This is a very time-consuming option to work with, but it can be good when
we want to create a brand-new application, or if we are working with an
application that has a large number of output categories. Of course, because
of the amount of data that we need, especially considering that it is a large
amount of labeled data, and the rate of learning, this is the least common out
of all the approaches. Usually, they will take weeks or longer to properly
train, but they can still be a good option in some cases.

Then we can work with what is known as transfer learning. Most of the
applications that you see with deep learning will work on the transfer
learning approach. This is a process that will have us working on a pre­
trained model and then fine-tuning some of the parts.
To make this one work, we want to pick out a network that already exists,
such as GoogLeNet or AlexNet and then feed in some new data that contains
classes that were previously unknown. After we make the necessary changes
and tweaks to the network, we are now able to perform the new task, such as
categorizing just dogs or cats, instead of focusing on 1000 different objects at
a time.

This kind of model for deep learning is going to have a few advantages, but
one of these benefits is that it needs much less data to complete. We can limit
the processing to just a few thousand images, rather than focusing on millions
of images like the network may have done in the beginning. This allows the
computation costs and time to drop, often getting the model done in a few
minutes depending on the situation.

This kind of learning is going to require us to have an interface to the

internals of the pre-existing network, so it can be modified and enhanced in a
more surgical manner to handle the new tasks that you set out. MATLAB has
all of the functions and the tools that you need to make sure that you see the
best results with transfer learning.

And the third option that we can work with when it comes to creating a deep
learning model is feature extraction. A slightly less common method, mostly
because it is more of a specialized approach that can work with deep learning,
is to use our network as a feature extractor. Considering that all of our layers
are going to have the responsibility of learning certain features from an
image, we are able to pull these features out of the network at any time that
we want during our training process. These are the features that we can then
utilize in a variety of different models, including SVM or support vector
machines, and machine learning models of other kinds as well.

However, you may find that training one of these models for deep learning
can take up a lot of time, sometimes days and even weeks. This doesn’t mean
that they aren’t worth the time and the effort that you put in, but sometimes a
business wants to be able to take all of the data they have and create a good
model to work with right away, rather than having to wait days or weeks to
even get started.

When we work with something known as GPU acceleration, it is able to

provide us with a significant increase in this process. When we use the
MATLAB with GPU, it is able to reduce the amount of time that it takes to
train a network, and can cut the training time for an image classification
problem from days to just hours. Think about how much faster this can be!
We can turn on one of our deep learning models with these tools within one
working day, and get amazing results with deep learning if we use it the right
way.
Don’t worry if you are not completely familiar with the GPUs and how they
work. MATLAB is going to use the GPU, whenever it is available, without
requiring us to understand how to program with these GPUs explicitly. This
makes the whole process a bit easier to handle.

There is just so much that we are able to do when it comes to deep learning.
This is a process that comes with machine learning, so as you get started, it is
easy to get the two terms mixed up and a bit confused. But with some of the
lessons that we will take a look at in this guidebook, we can gather a better
understanding of deep learning and what it is able to do for us.
 Chapter 26:

What Is a Data Analysis?

While we are on the topic of deep learning, we need to take some time to dive
into another topic that is closely related. When we were talking about deep
learning a bit above, we mentioned quite a bit about data, and how this data is
meant to help train and test the models that we try to make with the use of
deep learning. In this chapter, we are going to talk about data analytics,
which is the process of not only gathering the data but sorting through that
data and finding what predictions, insights, and information is inside.

A data analysis can happen in many different ways. But since most of these
analyses are done on large sets of data, also known as big data, it is hard for a
human to manually go through all of that information and learn what is
inside, especially in a timely manner. This is where deep learning comes into
the mix and helps us to do a good analysis in no time. With that in mind, let’s
dive in a bit and take a closer look at what data analysis is all about.
What Is a Data Analysis?
There is a ton of data that is available in our world. Companies like to spend
time collecting information from their customers, data from their social
media accounts, and data from other sources in their industry. This data is
going to help them to really get a competitive edge if they use it the right way
and can ensure that they release the right products while providing good
customer service in the process.

The problem is that after you collect your data, you then have to be able to go
through all of that data. This is not an easy thing to do all of the time. There
is often a ton of data, and figuring out how to go through it, and what
information is found inside the data is going to be hard. Gathering all of the
data that you need and then not following through and analyzing it is such a
waste, and you might as well save time and not go through with this at all.
But if you have gathered the data, we need to be able to go through and
analyze the data in order to make smarter business decisions. And that is
what we are going to spend time on in this guidebook.

Data analysis is going to be a practice of a company where they take the raw
data and then order and organize it out in a manner that we are able to look
through all of the useful information that is in there is extracted out to be
used. The process of organizing and then be able to think about the data is
going to be the key to understanding what the data does, and what it doesn’t,
contain. There can be a lot of data that the company provides you, but if you
are not able to go through it and see what information is inside of it, then the
data is not going to help you, and you will have no idea what is inside the
data, and what is not inside of the data.
Now, when we are working with data analysis, there are going to be quite a
few methods that we can add in here in order to handle the analysis, and
make sure that we will see some results. However, no matter what method
you choose to go with, there has to be some caution with how we want to
manipulate the data. We don’t want to end up pushing our own agendas and
conclusions on the data. We want to see the actual insights and information
that are found inside of all that data.

Yes, you will go through this process with some questions that you want
answered, and maybe also a hypothesis about what you are going to find in
the data. But if you are careful, you will not open up your mind to all of the
information that is found in that data, and you will miss out on valuable
insights and predictions that your business needs. And how much worth or
value will we assign to the data if we end up analyzing it in the wrong
manner in the first place? Our whole goal here is to find out what is in the
information, and learn what the data can actually tell us, without our own
biases in the process.

If you are going to ignore this advice, and just dive in with your own
conclusions and without paying attention to what the data is actually trying to
tell you, then you may as well give up right now. You will find exactly what
you want, but that doesn’t mean that we looked at the information in the right
manner. And often this steers you, and your business, down the wrong path.

The first thing that we need to take a look at here is what data analysis is all
about. When we look at data analytics, we see that it is the science used to
analyze lots of raw data to help a company make smart and informed
conclusions and decisions about that information. Many of the techniques and
the processes that come with data analytics have been automated into
mechanical processes and algorithms that are able to work the raw data over
in a manner that makes it easier for humans to learn from.

This process is much easier to complete, and much faster, than having
someone manually go through and read that information. This makes it easier
for companies to use all of that big data they have been collecting. You can
have all of the big data that you want, and you can store it in any manner that
you would like, but if that data is never analyzed and read through, you will
find that it is basically useless.

The techniques that are available for us to use with data analysis are going to
be helpful in that they reveal trends and metrics that would otherwise get
really lost in the mass of information. Remember that companies have
learned the value of lots of data and this means that while they are collecting
all of that data they could easily lose out on some of the insights because it
gets hidden inside with all of the noise.

Once the company is able to use a model of deep learning to help with a good
data analysis, and they figure out what the metrics and trends for that
information entails, they can then use this information to optimize the
processes that will help increase the overall efficiency that we see in that
system, and maybe even for the whole company.

To make this a bit further, we have to understand that data analytics is a very
broad term that is able to encompass a lot of diverse types of data analysis at
some point. Any type of information that you have collected and perhaps
stored can be subjected to the techniques of data analytics if you want to see
what is in a group of information to use it to further your business and to beat
out the competition, then it doesn’t matter what kind of information you
have, the data analyst can still find the insights that will improve your
business.

For example, we may find that many manufacturing companies are going to
work with a data analysis to help them run and perform better. For example,
they may record the runtime, the amount of downtime, and then the work
queue, or how long it takes the system to do a specific item, for various
machines. They can then analyze all of this information and data to make it
easier to plan out the workloads, ensuring that each machine is operating
closer to the peak capacity that it can during working hours.

Data analytics can help us to make sure that there are no longer any
bottlenecks when it comes to production, but this is not the only place where
a good data analysis is able to shine. For example, we can see that gaming
companies are often going to work with a form of data analytics to ensure
that they can reward their players on a set schedule, in a manner that makes it
more likely that the majority of the users will remain active in the game,
rather than giving up and getting frustrated.

Another example of how this data analysis can work, especially with the right
deep learning model, is with the world of content companies. Many of these
are already relying on data analytics to keep their customers clicking, re­
organizing, or watching content. The more that they can get these clicks and
views, the more money they are able to generate from advertisements and
such on their website, so they come up with content, headings, and more that
keep readers there for as long as possible.

Now, the steps that come with doing a data analysis can be quite involved
and take some time to get through. But we are going to focus on some of the
basics that come with each step, so that we can get a good general idea of
how this process is meant to work, and why so many businesses are able to
see some great results when they use a data analysis on their own data:

1. The first thing that we need to do is make a determination of the

requirements we want to place on the data, or how we would like to
 group our data. Data may be separated out in any manner that you
would like including by gender, income, demographics, and age. And
the values of the data can be numerical, or we can divide them out by
category. Knowing your business problem and what you are hoping
to solve with this data analysis can make it easier to know exactly
what the requirements of the data should be.
2. The next step that we need to focus on here is to collect the data.
Without any data, we are not going to be able to create the deep
learning models that we want, and we will be stuck without any
analysis at all. We are able to collect the data from a variety of
sources, including from computers, online sources, cameras and
through personnel, and environmental sources.
3. Once we have been able to collect the data that we want to use, it is
time to go through a few steps in order to organize that data. This
makes it easier to analyze the data without mistakes or errors
showing up. Organization may take some time and the method that
you use will depend on the type of data and what you are trying to
learn.
4. While you are organizing the data, we need to also take some time to
clean it up. This means that we want the data scrubbed and then
checked to make sure that there are no duplications or errors that will
be there and that the data is not incomplete in any manner. This is a
good step to work with because it will fix up any of the errors in the
set of data before you try to move the data along and perform your
analysis on it.

The Data Analytics Process

The application of data analytics is going to involve a few different steps,
rather than just analyzing the data that you have gathered, particularly on
some of the more advanced projects of analysis, much of the required work is
going to take place up front, such as with collecting, integrating, and
preparing data. Then we can move on to the part where we develop, test, and
review the analytical models that we have to ensure that they are producing
results that are accurate.
The analytics process is going to start from the very beginning, where we
work on collecting data. This is where the data scientist and their team will
identify the information that they need to find and gather for a particular
analytics application, and then they will continue to work on their own, or
with some IT staffers and data engineers to assemble all of that gathered data
for human consumption.

Data from different sources can sometimes be combined, with the help of a
data integration routine, transformed into a common format, and then loaded
up into what is known as an analytics system. There are a number of these
systems available for you to choose from including a data warehouse,
NoSQL database, and a Hadoop cluster.

In some other situations, the collection process is going to be a bit different.

In this case, the collection process could consist of pulling the relevant subset
out of a stream of raw data that flows into your storage, and then moving it
over to a second, and separate, partition in the system. This can be done in
order to allow for an analysis of the information, without any of the work that
you do affect the set of data overall.

Once we have been able to gather up the data that we need and we have
gotten it into place, the next step that we need to work on is to find and then
fix any of the quality problems that are in the data. We want to clean up any
of the quality problems that could potentially affect the accuracy of our
applications as we go along. This can include a number of different processes
including data cleansing and data profiling to ensure that the information in
our set of data is as consistent as possible, and that the duplicate entries and
errors can be eliminated.

In addition to what we have been able to do so far, there is some additional

work for data preparation that we need to focus on. This work is important
because it is going to manipulate and organize the data that you plan to use in
the analysis. You should add in some policies of data governance in order to
help the data stay within the standards of your company, and that everything
is done according to industry standards.
When you reach this point, the process of data analytics is going to get started
in earnest. The data scientist is going to build up an analytical model working
with some tools of predictive modeling or analytics software. There are a
number of programming languages that we are able to focus on as well,
including SQL, R, Scala, and Python, to get the work done. The model is
initially going to be run against a partial set of data because this is one of the
best ways to check out the amount of accuracy that is present in that model.

Of course, the first test is not going to be as accurate as you would like,
which means that the data scientist has to revise the model as needed and test
again. This is a process that is known as training the model, and we continue
working with it until get can get all of the parts together, and the model
functions as we intended.

Finally, we are going to run the model on what is known as the production
model. This means that the model is going to be run against the full set of
data. This is going to be done once because it is going to help us address a
specific need in information. Then there are times when it is going to be done
on an ongoing basis, any time that we update the data.

In some cases, the applications of the analytics can be set up in a manner that
will trigger business actions automatically. For example, we may see this
happen with some of the stock trades that a financial services firm is going to
use. Otherwise, the last step of this process of data analytics is
communicating the results generated by the analytical models that you used
to business executives and other end-users to aid in how they make their
important decisions.

There are a few different methods that you can use to make this happen, but
the most common technique to work with here is data visualization. What this
means is that the data scientist and any team they are working with will take
the information they gathered out of the model, and then turn this into a chart
or another type of infographic. This is done to help make it easier to
understand the findings

One more thing that we need to take a look at here is that we have to consider
the variety of statistical methods that we have available with our data
analysis, and then decide how we can use each one. There are a few that are
really good at getting this done, but often it will depend on what we would
like to accomplish with all of this. But first, we need to take a look at them
and how they will be able to help us get the results that we want. Some of the
best statistical methods that you may want to consider for your project will
include:

1. The general linear model. This is going to be a generalization of the

linear regression to the case of having two or more of the dependent
variables that you need to rely on.
2. Generalized linear model. This one may sound like the other model,
but it is a bit different. It is going to be an extension and works best
when your dependent variables are more discrete.
3. Structural equation modeling. This particular type of modeling is
going to be usable when you would like to assess some of the latent
structures that were measured from your manifest variables.
4. Item response theory: With these types of models, they are going to
be used to help us asses just one of the variables that are latent from
the variables that are binary measured.

Depending on the kind of information that you have present, and what your
final goal in the process is, there are a few different approaches that you are
able to use in order to help get the data analysis done. Some of the most
common ones that you can choose from (and again, look to your data and
what you are trying to figure out from that information to help make the
decision), are going to include the following:

1. Cross-cultural analysis
2. Content analysis
3. Grounded theory analysis
4. Discourse analysis
5. Hermeneutic analysis
6. Constant comparative analysis
7. Phenomenological analysis
8. Narrative analysis
 9. Ethnographic analysis

Why Does Data Analytics Matter?

While we are here, we need to take a few minutes to discuss why data
analytics is so important, and why so many businesses are jumping on board
with this in the hopes of seeing some improvements along the way. the
reason that data analytics is so important is because it can help a business to
optimize their performances overall.

When the company is able to implement the data analysis into their business
model, it means that they are able to reduce the costs that they experience on
a day to day basis. This happens because the analysis will help them to
identify the best, and the most efficient, ways of doing business, and because
they are able to store up large amounts of data to help them get all of this
process done in a timely manner.

Another benefit of using this data analytics, and why it really does matter for
a lot of companies, is that the company can use this process in order to make
the best business decisions. These business decisions no longer need to rely
on what other companies are doing or on the intuition of key decision­
makers. Instead, they rely on the facts and insights provided in the collected
data.

Many companies also like to work with the process of data analytics because
it will help them learn more about and serve their customers better. Data
analytics can help us to analyze customer trends and the satisfaction levels of
our customers, which can help the company come up with new, and better,
services and products to offer.

The Types of Data Analytics Available

When we take a look at the process of data analytics, we will find that there
are a few different types that a company can choose from to get their work
done and to really learn what is hidden inside all of that data they have been
collecting. To keep it simple, we are going to focus on the four basic types of
data analytics that many companies are going to rely on.

First, we have what is known as descriptive analytics. This is the type of data
analytics that will describe to us what happens over a given period of time.
We may use this one when we want to see whether or not the sales have been
stronger this month compared to last month. Or we can use it on our social
media pages to figure out whether the number of views that we have received
on posts has gone up, down, or remained the same.

Then the second type of data analytics that we can work with is called the
diagnostic analytics. This one is a bit different because it will focus more on
the why, rather than the what, of something happening. This will involve
some data inputs that are a bit more diverse, and the data scientist has to
come in here with a bit of hypothesizing ready to go. For example, the data
scientist may focus on whether or not the last marketing campaign that was
sent out actually impacted sales in a positive or negative manner before
prolonging or canceling that marketing.

We can also work with what is known as predictive analytics. This one is
going to move us over to what is the most likely thing to happen in the near
term. We may ask questions with this one like What happened to sales the
last time we had a hot summer? How many of the various models on weather
that we have looked at predicting that this summer is going to be a hot one.

And finally, the fourth type of data analytics that we can focus on is going to
be known as the prescriptive analytics. This is the type of analytics that is
able to suggest a course of action. if we look at the example above, we would
take some time to check out how hot the summer is likely to be. When we see
that the likelihood of a hot summer is measured as an average of five models
of weather, and they predict that the hot summer is 58 percent likely to
happen, then you would make some changes to accommodate.

Let’s say that when the weather gets hot in the summer, you sell more of your
product. Since we have a good estimate from a few different sources, that the
weather is going to be hot, we would want to plan accordingly. Maybe you
will hire more people, add on some extra shifts, or stock up on your inventory
to make sure that you don’t run out.

At its core, data analytics is going to underpin some of the quality control
systems that show up in the world of finances, including the program known
as Six Sigma. If you are in business, it is likely that you have spent at least a
bit of time working with Six sigma and all that it entails.

The idea that comes with this Six Sigma program is that a company wants to
learn how to cut out waste in any manner possible. The choices they may to
do this will depend on their business model, what they hold most dear, and
what needs the most work. One company may need to work on reducing the
amount of returns that they get from customers, and another may need to find
a way to limit the number of steps that are taken to complete the process of
creating the product.

The goal with Six Sigma is to slowly but surely cut out the waste and help the
company reach near perfection as much as possible. And while these two
topics, that of deep learning and Six Sigma, are not topics that most people
are going to associate with one another, they can really work hand in hand to
make sure that the goals of each can be met.

The main idea that comes with both of these processes though is that if you
don’t take the time to measure something out properly, whether you are
looking at your own weight or another personal measurement, or the number
of defects, per million, that happen on the production line, how can you ever
hope to optimize the results? This is what deep learning can help us to get
done if we just learn how to use it.
Is Anyone Really Using Data Science?
Another question that a lot of people have when it comes to data science and
data analysis is whether there are other companies or other industries that are
using this kind of technology. Sometimes, with all of the work, the
algorithms, and the various programs, it seems like using data science and
data analysis is just too much work to handle. But you may be surprised that
there are a number of industries and companies out there who are already
working with data science, and a data analysis that comes with it, to help
them really gain a competitive advantage.

A good example of where this is being used is in the industry of traveling and
hospitality. Both of these rely on a quick turnaround to make as much profit
as possible, and to ensure that they are not turning guests away all of the
time. The good news for this industry is that they are able to collect a ton of
data on the customer in order to find any of the problems they need to worry
about, which makes it much easier to actually fix the issue.

Another place where we are going to see a lot of data analysis in the
healthcare industry. There are so many areas of this industry that can use a
good data analysis, especially when it is combined together with machine
learning and deep learning, to make doctors and other medical professionals
better at their jobs. It can help doctors to look through images and diagnose
patients faster, it can help to keep up with the patient and ensure that they are
getting the right treatment, it can be used on the floor to monitor the patient
and alert nursing staff when something is wrong, and even can help as
receptionists and other similar roles when no one else is available to take this
job.
And finally, the retail industry is able to benefit from the use of data science
in many ways. This is because the companies that fit into this industry are
going to use copious amounts of data in order to help them keep up with the
demands of the shopper, even though these demands can change over time.
the information that is collected and analyzed by retailers will come in use
when it is time to identify big trends, recommend products, and increase the
amount of profit the company makes.

As we can see here, there are a lot of different parts that come into play when
we want to work with a data analysis. It is a pretty easy concept, but one that
takes some time and effort in order to see the best results. But when we can
take all of the data that we collect over time, and then actually perform an
analysis on the information to gather some good insights and predictions to
make smarter business decisions, we will find that data analysis can be well
worth our time.
 Chapter 27:

The Python Libraries for Deep

Learning

So, at this point, we have talked a lot about deep learning and data analysis,
and now it is time to take some of that information, and put it to good use.
You are probably interested in deep learning, and maybe even in making
some of your own Convolutional Neural Networks, but are wondering where
you should start. The best step is to pick out the library that you want to use.
But this brings up another challenge because there are just so many coding
libraries out there that you can choose from, and all of them have some
amazing power and features behind them.

To start with, we are going to take a look at some of the best Python libraries
that can help with deep learning. There are other languages that can help with
things like machine learning and deep learning. But for most of the tasks that
you want to do, especially if you are a beginner in data analysis and all of the
processes that we have been talking about, then Python is going to be the
choice for you. Even within Python, there are a number of libraries that you
can choose from to get your deep learning work done. So, with that in mind,
let’s dive right in and see some of the best Python deep learning libraries that
you can use for your data analysis
Caffe
It is pretty hard to get started with a look at deep learning libraries through
Python without spending some time talking about the Caffe library. In fact, it
is likely that if you have done any research on deep learning at all, then you
have heard about Caffe and what it can do for some of the projects and
models that you want to create.

While Caffe is technically not going to be a Python library, it is going to

provide us with some bindings into the Python language. We are going to use
these bindings when it is time to deploy the network in the wild, rather than
just when we try to train the model. The reason that we are going to include it
in this chapter is that it is used pretty much everywhere and on all of the parts
of a deep learning model that you need to create.

Theano
The next kind of library that we are able to work with is known as Theano.
This one has helped to develop and work with a lot of the other deep learning
libraries that we have that work with Python. In the same way that a
programmer would not be able to have some options like scikit-image, scikit-
learn, and SciPy without NumPy, the same thing can be said when we talk
about Theano and some of the other higher-level abstractions and libraries
that come with deep learning.

When we take a look at the core of this, Theano is going to be one of the
Python libraries that not only helps out with deep learning, but can be used to
define, optimize, and evaluate a lot of mathematical expressions that will
involve multi-dimensional arrays. Theano is going to accomplish this because
it is tightly integrated with the NumPy library, and it keeps its use of GPU
pretty transparent overall.

While you are able to use the Theano library to help build up some deep
learning networks, this one is often seen as the building blocks of these
neural networks, just like how the NumPy library is going to serve as the
building blocks when we work on scientific computing. In fact, most of the
other libraries that we will talk about as we progress through all of this are
going to wrap around the Theano library, which really makes it more
accessible and convenient than some of the other options.

TensorFlow
Similar to what we are able to find with the Theano library, TensorFlow is
going to be an option that is open-sourced and can work with numerical
computation with the help of a data flow graph. This one was originally
developed to be used with research on the Google Brain Team within
Google’s Machine Intelligence organization. And this library, since that time,
has turned into an open-sourced option so that the general public can use it
for their deep learning and data science needs.

One of the biggest benefits that we are going to see with the TensorFlow
library, compared to what we see with Theano, is that it is able to work with
distributed computing. This is particularly true when we look at multiple-
GPUs for our project, though Theano is working on improving this one as
well.

Keras
Many programmers find that they love working with the Keras library when
it comes to performing models and other tasks with deep learning. Keras is
seen as a modular neural network library that is more minimalistic than some
of the others that we talk about. This one is able to use either TensorFlow or
Theano as the backend so you can choose the one that works the best for any
needs you have. The primary goal that comes with this library is that you
should be able to experiment on your models quickly and get from the idea
that you have over to the result as fast as possible.

Many programmers like this library because the networks that you architect
are going to feel almost natural and really easy, even as a beginner. It is going
to include some of the best algorithms out there for optimizers,
normalization, and even activation layers so this is a great one to use if your
process includes these.

In addition, if you want to spend some time developing your own CNNs, then
Keras is a great option to work with. Keras is set up to place a heavy focus on
these kinds of neural networks, which can be valuable when you are working
from the perspective of computer vision. Keras also allows us to construct
both sequence-based networks, which means that the input is going to be able
to flow linearly throughout that network and the graph-based network, which
is where the inputs are able to skip over some of the layers if needed, only to
be concatenated at a later point. This is going to make it easier for us to
implement network architectures that are more complex.

One thing to note about this Python library is that it is not going to support
some of the multi-GPU environments if you would like to train a network in
parallel. If this is something that you want to do, then you may need to
choose another library that you want to use. But for some of the work that
you want to do, this may not be a big issue.

If you want to get your network trained as fast as possible, working with a
library like MXNet may be a better choice. But if you are looking to tune
your hyperparameters, then you may want to work with the capability of
Keras to set up four independent experiments and then evaluate how the
results are similar or different between each of these.

Sklearn-Theano
There are going to be times when working with deep learning when you will
want to train a CNN end-to-end. And then there are times when this is not
needed. Instead, when this is not needed, you can treat your CNN as the
feature extractor. This is going to be the most useful with some situations you
may encounter where there is just not enough data to train the CNN from
scratch. So, with this one, just pass your input images through a popular pre­
trained architecture that can include some options like VGGNet, AlexNet,
and OverFeat. You can then use these pre-trained options and extract features
from the layer that you want, usually the FC layers.

To sum this up, this is exactly what you want to have happen when you bring
out this kind of library. You are not able to train a model from scratch with it
so if this is your goal, then you will need to look for some other library to
focus on. However, it is one of the best options to choose when you want a
library that can evaluate whether a particular problem is going to be suitable
for deep learning and other processes.
Nolearn
A good library for you to work with is the nolearn library. This is a good one
to help out with some initial GPU experiments, especially with a MacBook
Pro. It is also a good library to help out with performing some deep learning
on an Amazon EC2 GPU instance.

While Keras wraps TensorFlow and Theano into a more user-friendly API,
you will find that the nolearn library will be able to do the same, but it will do
this with the Lasagna library. In addition, all of the code that we find with
nolearn is going to be compatible with Scikit-Learn, which is a big bonus for
a lot of the projects that you want to work with.

Digits
The first thing to notice with this library is that it isn’t considered a true deep
learning library. Although it is written out in Python and it stands for Deep
Learning GPU Training System. The reason for this is because this library is
more of a web application that can be used for training some of the models of
deep learning that you create with the help of Caffe. You could work with the
source code a bit to work with a backend other than Caffe, but this is a lot of
extra work in the process. And since the Caffe library is pretty good at what it
does, and can help with a lot of the deep learning tasks that you want to
accomplish, it is really not worth your time.

If you have ever spent some time working with the Caffe library in the past,
you can already attest to the fact that it is tedious to define your .prototxt
files, generate the set of data for the image, run the network, and babysit the
network training with the terminal that you are provided. The good news here
is that the DIGITS library aims to fix all of this by allowing you to complete
a lot of these tasks, if not all of these tasks, just from your browser. So, it may
not be a deep learning library per se, but it does come into use when you
struggle with the Caffe library.

In addition to all of the benefits above, the interface that the user gets to
interact with is seen as excellent. This is due to the fact that it can provide us
with some valuable statistics and graphs to help you rain your model more
effectively. You can also easily visualize some of the activation layers of the
network to help with various inputs as needed.

And finally, another benefit that is possible with this library is that if you
come in with a specific image that you want to test, you have a few options
on how to get this done. The first choice is to upload the image over to the
DIGITS server, or you can enter in the URL that comes with the image, and
then the model you make with Caffe will automatically be able to classify the
image and display the results that you want in the browser.

Python is one of the best coding languages available for helping with tasks
like deep learning, machine learning, and even with the topic of artificial
intelligence, which encompasses both of the other two ideas. There are other
languages that can handle the deep learning that we have been talking about,
but none are going to be as effective, as powerful, have as many options, or
be designed for a beginner in the way that Python can.

And this is exactly why we have focused our attention on the Python
language and some of the best libraries that we are able to choose to help
with a variety of deep learning tasks. Each of these libraries can come on
board with your project and will provide us with a unique set of functions and
skills to get the job done. Take a look through some of these libraries and see
which one is going to be just right for your data analysis and for providing
you with great insights while completing deep learning.
 Chapter 28:

The Mathematics That Come with

Neural Networks

The next topic that we need to spend some time looking through is the idea of
neural networks. You will not get very far with your work in deep learning if
you are not able to work with these neural networks, and there are a few
different types that you can create and work with as well. So, let’s dive right
in and learn more about these great neural networks that can help us with our
deep learning models.

Neural Networks
The first type of network we are going to look at is the “normal” type of
neural network. These neural networks are going to fit into the category of
unsupervised machine learning because they are able to work on their own
and provide us with some great results in the process. Neural networks are a
great option to work within machine learning because they are set up to catch
onto any pattern or trend that is found in a set of data. This can be done
through a variety of levels, and in a way that is going to be much faster and
more effective than a human going through and doing the work manually.

When we work with a neural network, each of the layers that we will focus
on are responsible for spending time in that layer, seeing if they are able to
find a pattern or trend inside the image, or through the data, that it looks at.
Once it has found a trend or a pattern, it is going to start its process for
entering into the next layer. This process is going to continue, with the
network finding a new pattern or trend, and then going on to the next level,
until it reaches a place where there are no more trends or patterns to find.

This process can end up with a lot of different layers, one over the top of the
others again and again, until you have been able to see the whole thing that
comes in the image. When the algorithm is created, and the program can
make a good prediction based on what is in the image or in the data that you
present, then you know that it has all been set up properly.

Before we move on though, we have to remember that there are a few parts
that will start to occur at this point, based on how you set up the program to
work. If the algorithm was able to read through all of the layers and the steps
above, and it had success with reading through the different layers, then it is
able to make a good prediction for you. If the algorithm is accurate with the
prediction that it made, then the neurons that come with this algorithm will
strengthen and become faster and more efficient at their job overall.

The reason that this happens is because the program is relying on artificial
intelligence, and more specifically on deep learning, in order to create those
strong associations between the patterns it saw and the object. Keep in mind
that the more times that the algorithm is able to provide the right answer
during this process, the more efficient it will become when you try to use it
another time as well. The neurons get stronger, and you will see that the
answers come faster and are more accurate overall.

Now, if you haven’t been able to work with machine learning and deep
learning in the past, it may seem like these neural networks would be
impossible to actually see happen. But a closer examination of these
algorithms can help us to see better how they work and why they can be so
important to this process. For the example that we are going to work with,
let’s say that we have a goal to make a program that can take the image we
present, and then, by going through the different layers, the program is able
to recognize that the image in that picture is actually a car.

If we have created the neural network in the proper manner, then it is able to
take a look at the image that we use and make a good prediction that it sees a
car in the picture. The program will then be able to come up with this
prediction based on any features and parts that it already knows comes with a
car. This could include things like the color, the license plate, the door
placement, where the headlights are, and more.

When we take a look at coding with some of the traditional methods, whether
they are Python methods or not, this is something that you may be able to do,
but it takes way too long and is not the best option to work with. It can take a
lot of coding and really just confuse up the whole process. But with these
neural networks, you will be able to write out the codes to get this kind of
network done in no time.

To get the neural network algorithm to work the way that you want, you have
to provide the system with a good and clear image of a car. The network can
then take a look at that picture and start going through some of the layers that
it needs to work with to see the picture. So, the system will be able to go
through the first layer, which may include something like the outside edges of
the car. When it was done with this, the network would continue on from one
layer to the next, going through however many layers it takes to complete the
process and provide us with a good prediction. Sometimes this is just a few
layers, but the more layers this program can go through, the more likely it
will provide an accurate prediction in the end.

Depending on the situation or the project that you want to work with, there is
the potential for adding in many different layers. The good news with this
one is that the more details and the more layers that a neural network can
find, the more accurately it can predict what object is in front of it, and even
what kind of car it is looking at.

As the neural network goes through this process, and it shows a result that is
accurate when identifying the car model, it is actually able to learn from that
lesson, similar to what we see with the human brain. The neural network is
set up in a way to remember the patterns and the different characteristics that
it saw in the car model, and con store onto that information to use at another
time if it encounters another car that is the same again. So, if you present, at a
later time, another image with that same car model in it, then the neural
network can make a prediction on that image fairly quickly.
There are several options that you can choose to use this kind of system for,
but remember that each time you make a neural network, it is only able to
handle one task at a time. you can make a neural network that handles facial
recognition for example, and one that can find pictures that we need in a
search engine, but you can’t make one neural network do all of the tasks that
you want. You may have to split it up and make a few networks to see this
happen.

For example, there is often a lot of use for neural networks when it comes to
creating software that can recognize faces. All of the information that you
need to create this kind of network would not be available ahead of time, so
the neural network will be able to learn along the way and get better with
recognizing the faces that it sees in video or images. This is also a method
that can be effective when you would like to get it to recognize different
animals or recognize a specific item in other images or videos as well.

To help us out here, we need to take a look at some of the advantages that can
come with this kind of model with machine learning. One of the advantages
that a lot of programmers like with this one is that you can work with this
algorithm without having to be in total control over the statistics of the
algorithm. Even if you are not working with statistics all of the time, or you
are not really familiar with how to use them, you will see that these networks
can be used without those statistics, still that if there is any relationship, no
matter how complex it is, is inside the information, then it is going to show
up when you run the network.

The nice thing with this one is that the relationships inside your data can be
found, whether the variables are dependent or independent, and even if you
are working with variables that do not follow a linear path. This is great news
for those who are just getting started with machine learning because it
ensures that we can get a better understanding of how the data relates to each
other, and some of the insights that you want to work with, no matter what
variables you are working with.

With this in mind, we have to remember that there are still times when we
will not use a neural network, and it will not be the solution to every problem
that we want to handle in deep learning. One of the bigger issues that come
with these neural network algorithms, and why some programmers decide to
not use this is that the computing costs are going to be kind of high.

This is an algorithm that is pretty in-depth, and because of this, the

computing costs are going to be a bit higher than what we find with some of
the other options out there. and for some businesses, and even on some of the
projects that you want to do with deep learning, this computation cost will
just be too high. It will take on too much power, too much money, and often
too much time. For some of the projects that you want to take on, the neural
networks will be a great addition to your arsenal with deep learning, and
other times, you may want to go another route.

Neural networks are a great option to work with when it is time to expand out
your work and when you would like to create a program that can handle some
more complex activities. With the right steps here, and with some time to
train the neural network, you will find that the neural network is a great way
to handle your data and find the trends and predictions that you want.

Recurrent Neural Networks

Now that we have had some time to look at the regular neural networks, it is
time for us to dive a bit deeper and look at another option, known as the
recurrent neural networks. These are going to follow some of the same rules
that we can see with the discussion above, but they can also take some of
your projects to the next level. Let’s take a look at what these recurrent neural
networks are like, and how they can really benefit a project that you are
working with.

A good way to start looking at these recurrent neural networks, or RNN, is by

taking a look at the human brain. When we do this, we know that it is
reasonable to have the understanding that our thought processes or our
understanding doesn’t restart every second. We are able to retain the
information that we hear and learn and then build on it. This is something that
we do from childhood. We don’t just see the letter A and then forget about it
five seconds later. We use it as the start of the alphabet and build on that to B,
C and so on. We are always building on new knowledge, whether it is from
our childhood, or if it is something that we just learned.

As we go through some of the other parts that show up in this guidebook, you
will start to see how each of the words, based on how much understanding
you had of the words that we wrote on the page before. Your brain is not
going to see a word and then immediately threw it away, and then restart its
thinking process from the beginning. The point here is that our thoughts are
basically able to have some consistency and some persistence with them,
which is part of what makes them so powerful to work with.

With the traditional neural network that we discussed in the last section isn’t
capable of doing this kind of thing. And this can be a bigger shortcoming in
many cases. For example, if you are working on a project that needs to
classify the kind of event that is happening during all of the different parts of
a movie, it wouldn’t help you much with a traditional neural network because
it would not be able to reason with the events that occurred earlier on in this
film. There just isn’t that kind of communication or power showing up within
the program.

Even though the traditional neural networks may struggle with doing this
kind of task and others, the good news is that we can rely on the recurrent
neural networks to help us address these problems and projects in machine
learning for us. These are a type of network that comes with a loop, which is
going to allow the information that it learns to persist. In this method, the
loop will allow information to pass from one part of the network and then
move it over to the next part of the code. The recurrent neural network can be
a similar idea to having multiple copies of the same network, and with each
message being handed over to the successor in the process.

This chain-like nature that comes with this network is going to reveal how
these networks can be intimately related to sequences and lists as we go
through the process. These are going to be the natural architecture of a neural
network to use for all of this data. And there are quite a few times in deep
learning when these networks are going to be used.
In fact, over the past few years or so, there has already been a lot of success
when it comes to applying these recurrent neural networks to a variety of
projects, and it is likely that this kind of success is going to continue in the
future. Some of the current examples of how the recurrent neural network can
be used include speech recognition, language modeling, translation, and
image captioning projects.

One of the limitations that is pretty obvious when it comes to working on

these neural networks is that the API that comes with it is going to contain a
good deal of constraints along the way. The API is only able to take in a
vector that is a fixed size for the input, and then they can only produce a
vector of a fixed size for the output, which can be a hassle when you need to
take on larger forms of data to get some results.

Of course, this is just one of the issues that can come up with a system that is
as complicated as an RNN. These models, for example, are able to perform
the mapping that you need with a fixed number of computational steps, which
is basically going to equal out to the same number as the layers that you will
use to see the model.

Now, one thing to remember here is that the main reason that the RNN option
is able to make your coding more exciting and can add in more work to what
you can do is due to the fact that it allows the programmer to take their work
and operate it over a sequence of vectors. This can often include the
sequences in the input, the output, and often it will include a combination of
both.
Let’s take a look at the chart above. Each of the rectangles that are there is
going to be a vector and the arrows are going to show us the functions. The
input vectors are going to show up in red, and then the output vectors that we
need to know are going to be in blue. And then the green vectors will hold
onto the RNN state (which we are going to talk about in a minute). Going
from the leftover to the right, let’s take a look at how each of these work:

1. The first one is going to be the vanilla mode of processing, the one
that doesn’t use the RNN at all. This is going to include an input that
is fixed and an output that is fixed. This is also known as image
classification.
2. The sequence output is going to be the second part. This is going to
be image captioning that is able to take an image and then will
provide you with an output of a sentence of words.
3. Sequence input: This is going to be the third picture above. It is
going to be more of a sentiment analysis that shows us a given
sentence and makes sure that it is classified as either a negative or
positive sentiment.
4. Sequence output and sequence output. You can find this one in the
fourth box, and it is getting a bit closer to what we want. This one is
going to be similar to a machine translation. This is when the RNN is
able to read a sentence out in English, and then can take that
information and provide you with an output that reads the sentence in
French.
5. And finally, the last box is going to be the synced sequence input and
output. The video classification here is going to help us to label out
each of the frames that occur in a video if we decide to.

Notice that in each of these, there aren’t going to be any constraints put on
the lengths of the sequences that we have to specify ahead of time. this is
because the recurrent transformation, which is going to be shown in green, is
fixed, and we are able to apply it out as many times as we would like, or as
many times as work with our project.
These neural networks are a great addition to any of the codes that you would
like to write out in the Python language, and they are definitely good
examples of what we are able to work with when it comes to deep learning.
Deep learning can lead us to a lot of different projects, many of which we
may not think were possible in the past. But with some great neural networks,
including the recurrent networks and the traditional neural networks, we are
able to make these programs happen, search through a lot of information, and
get the predictions and insights that we are looking for out of that data.
PART 7
 Chapter 29:

The Basics of Using Our TensorFlow

Library for Deep Learning

As we talked about a bit before, there are a lot of different libraries and
options that you are able to work with when it comes to Python helping out
deep learning. We talked about a few of these before, but now it is time for us
to dive right into some of the best Python libraries that work with deep
learning, and see how they work and what they are able to offer to you.

The first library that we are going to take a look at here is known as
TensorFlow. This one needs some time because the complexities of what you
can do with this library can make it a bit intimidating for a lot of people to
work within the beginning. But it is a great library to go with to help with
things like linear algebra and vector calculus to name a few. The Tensors that
show up in this library are able to provide us with some multi-dimensional
data arrays, but some more introduction is most likely needed before we dive
in and really understand what these tensors are all about. So, let’s get started:
TensorFlow is a type of framework that is going to come to us from Google
and it is used when you are ready to create some of your deep learning
models. This TensorFlow is going to rely on data-flow graphs for numerical
computation. And it has been able to stop in and make machine learning
easier than ever before.

It makes the process of acquiring the data, training some of the models of
machine learning that you want to use, making predictions, and even
modifying some of the future results that you see easier. Since all of these are
important when it comes to machine learning, it is important to learn how to
use TensorFlow.

This is a library that was developed by Google’s Brain team to use on

machine learning when you are doing it on a large scale. TensorFlow is going
to bring together machine learning and deep learning algorithms and models
and it makes them much more useful via a common metaphor. TensorFlow is
going to use Python, just like what we say before, and it gives its users a
front-end API that can be used when you would like to building applications,
with the application being executed to a high-performance C++.

We will find that TensorFlow is often the library that is used when we want
to not only build-up but also train and run our deep neural networks. These
neural networks can be great when we want to work with image recognition,
handwritten digit classification, natural language processing, recurrent neural
networks, and so much more in the process. There is a lot that we are able to
do with the TensorFlow language, and when it is combined together with
some of the extensions, the deep learning that you can accomplish will be
amazing.

TensorFlow, along with some of the other deep learning libraries that are out
there, are a great option to get some of your work done. Whether you want to
work with deep neural networks or another form of deep learning to help get
through your data and help you find the insights and predictions that are
found inside, or you are looking to find some other uses of deep learning to
help your business out, TensorFlow is one of the best Python libraries out
there to help you get the work done.

Plane Vectors
To start off our discussion, we need to take a look at something called a
vector. The vectors in TensorFlow are going to be a special type of matrix,
one that can hold onto a rectangular array of numbers to help with deep
learning. Because these vectors are going to be ordered collections of
numbers, they will be seen in the view of column matrices. These vectors are
going to come with one column in most cases, along with the rows that you
need to get the process done. A good way to think about these vectors is a
kind of scalar magnitude that is given some direction on how to behave.

Remember a good example of a scalar is going to be something like 60 m/s or

5 meters, while the vector would be more like 5 meters north. The difference
that shows up between these two is that the vector will give us a direction on
where to go, not just the distance, but the scalar is not going to do this and
just focuses on the distance. Despite this happening, these examples are still
going to be a bit different than what you may see with some of the other
machine learning and deep learning projects we have focused on along the
way, and this is pretty normal for now.

But, keep in mind with this one that the direction we are given in the vector is
just going to be relative. This means that the direction has to be measured
relative to some reference point that you add to the whole thing. We will
show this direction in units of radians or in degrees to make this easier. For
this one to work in most situations, you have to assume that the direction is
going to be positive and that it is going to head in a rotation that is
counterclockwise from the direction that we used as the reference point.
When we look at this through a more visual lens though, we have to make
sure that the vectors are represented with arrows. This means that you have to
consider these vectors as arrows that are going to come with a length and a
direction at the same time. the direction you will need to follow can be
indicated by looking at the head of the arrow, but then the length or the
distance that we want to follow is going to be best indicated when we see
how long the arrow truly is.
So, this is going to bring us to a crossroads where we need to focus on what
these plane vectors are all about? Plane vectors are going to be a very easy
setup when it comes to the tensors. They can be similar to what we see with
the regular vectors that we talked about before, but the sole difference is that
they are going to be found in what is known as the vector space with these
ones.

This may be a bit confusing at first, but to give us a better understanding of

what this means, we need to bring out an example of how it all works. Let’s
say in this example that we are working with a vector that is 2 X 1. This
means that the vector is going to belong to a set of real numbers that are
paired two at a time. To say this in another manner that may be easier to
explain, they are going to both be a part of the two-space. When this does
happen, you can then represent the vectors on the coordinate, with the idea of
x and y like we are familiar with, plane, with rays or the arrows like we
discussed.

As we work form the coordinate plane that we just discussed, we have to start
out with the vectors on the standard position with an endpoint at the origin,
which is the point (0, 0), or you will derive the value of our x coordinate by
looking at the first row of the vector. It is also possible for us to go through at
this time and find they coordinate in our second row.

The thing that we have to keep in mind with this one is that the standard
position is not something that has to stay the same all of the time, and it is not
something that we have to maintain either. The vectors can move, often going
parallel to themselves in the same plane, and we won’t have to be concerned
about any of the changes that show up with this.

The Basics of the TensorFlow Library

With some of this background information ready to go and a better
understanding of the tensors and vectors that come with this library, it is now
time to go on another track a bit and learn some more of the basics that come
with TensorFlow. We are going to start by looking at some of the steps to set
up this library and get it ready to use, starting with some basics so you can
become more familiar with the basics.
When you write out some of the codes that you want to use with TensorFlow,
all of this is going to happen in the program this library provides, and we
need to make sure that it is run as a chunk. This can seem a bit contradictory
in the beginning, since we want to make sure that the programs we write out
are done in Python. However, if you are able to, or if you find that this
method is easier, working with the part known as the TensorFlow’s
Interaction Session is often the best choice. This is even a good one that helps
us to be more interactive with our code writing and works well with IPython.

For what we are looking at here with some of the basics, we want to put some
of our focus on the second option. This is important because it is going to
help us get a nice start with not only working in the TensorFlow library but
also some help with how to work with deep learning in the process. Before
we get into some of the cool things that we are able to do with TensorFlow
and deep learning, which we will discuss in the next section, we first need to
get into a few topics to help us out.

To do any of the basics, we need to first import the TensorFlow library under
the alias of “TF,” as we did before. We will then be able to initialize two
variables that are going to be constants. Then you need to pass an array of
four numbers using the function of “constant().”
It is possible that you could go through and pass in an integer if you would
like. But for the most part, working with and trying to pass an array is going
to be easier and will work the best for your needs. Tensors are often about
arrays so this is why it is better to use them.

Once you have put in the integer, or more likely the array, that you want to
use, it is time to use the function of “multiply()” to get the variables to
multiply together. Store the result as the “result” variable. And last, print out
the result using the function of print().

Keep in mind that if you have defined the constants in the DataCampLight
code. However, there are a few other value types that you can work within
this, namely known as the placeholders. These placeholders are important
because they are going to be values that are unassigned and that can be
initialized by the session where you choose to let it run. Like the name is
giving away, the placeholder for a particular tensor that you want to work
with will be fed when the session actually runs.

During this process, we can also bring in the variables. These are the options
that will have a value attached to them (you get to choose the value that you
would like to attach to each of the variables as you go along), and we can
change them as we go along. If you would like to go through this process and
double-check that no one is able to come in here later and make changes to
the values as you have them in the equation, then you need to switch over to
some constants. If you are not too worried about the fact that these values can
change occasionally, or you would like to have the ability to go in later and
make some changes to the value, then the variables are a great option to work
with.

The results of the code that you are able to write up here will end up with an
abstract tensor in your computational graph. Even though this may not seem
like it should work all of the time in the program, the results that come with
this one are not going to be calculated. It is just going to be pushed through
the algorithm and helps to define the model, but there wasn’t a process that
ran to help us calculate the results.

Deep Learning and TensorFlow

Now that we have had some time to look at the basics that come with
TensorFlow and seen some of the benefits that come with using this library, it
is time to actually take a look at how we can add TensorFlow in with some of
the deep learning that we have discussed in this guidebook. The way that we
are going to work with this, for now, is by constructing our own architecture
for a neural network. But this time, we want to make sure that we can
accomplish this with the package from TensorFlow.

Just like we can do when we bring out the Keras library (we will touch on
this a bit in the next chapter), we will need to create our own neural network
by going through it layer by layer. If you haven’t taken the chance to do this
yet, make sure that you go through and import the package for “TensorFlow”
into the workspace that you want to use and use the conventional alias of
“TF” to make it easier. You can also choose out another name if needed.

From this step, our goal is to initialize the graph using the simple function of
Graph() to help get this done. This is an important function to bring into the
group because it is the one that you will use to define the computation as you
go along. Remember as we are doing the work here and we create a graph,
that we do not need to worry about computing things at this time. This is due
to the fact that the graph, right now at least, is not holding onto any of the
values that we need. Right now, the graph is being used to define the
operations that we would like to get running later.
In this situation, we are going to work with a function called as_default() to
make sure that the default context is set up and ready to go. This one works
the best because it will return to us a context manager that can make the
specific graph that we want into the default graph. This is a good method to
learn because you are able to use it any time that your model needs to have
more than one graph in the same process.

When you use the function that we listed out above, you will then need to
come up with what is known as the global default graph. This is important
because it is the graph that will hold onto all of your operations. That is, until
you go through and create a new graph to help out with this.

Once we are to this point, we also need to take some time to add in the
various operations that we would like to see in the graph. The way that we do
this is by building up the model. Once this is done, we compile the model and
then define the metric, the optimizer, and the loss function. You can then
work with TensorFlow and this is the step where we bring this in. Some of
the steps that need to happen when we get to this point include:

1. Define the placeholders that you want to use for your labels and
inputs because we are not going to put in the real data at this time.
remember that the placeholders that you are using here are values
that are unassigned and that will be initialized as soon you as go
through and run it. so, when you are ready to run the session, these
placeholders are going to get the values from your set of data that
you pass through in the function for run().
2. Now we want to work to build up our network. You can start this by
flattening out the input, and this is done by working with the function
flatten(). This will give you an array of shape, rather than the shape
that is used with images that are grayscale.
3. After you have been able to flatten up the input, your construct needs
to become a fully connected layer that generates logits of size. Logits
is going to be the function that operates on the unscaled output of
previous layers. And then it is going to use the relative scale to make
sure that there is an understanding that the units are linear.
 4. After you have had some time to work with the perceptron that is
multi-layer, you will then be able to make sure that the loss function
is defined. The choice that you make with your loss function is going
to depend on what kind of task you are doing at the time.
a. Remember here that when you use regression, it is going to
be able to predict values that are continuous. When you work
with classification, you are going to predict the discrete
classes or values of data points.
b. From here, you can wrap up the reduce_mean) function,
which is going to compute out the mean of elements across
the whole tensor.
5. Now you want to take the training optimizer and define that. Some of
the best algorithms that you can use to optimize this include
RMSprop, ADAM, and Stochastic Gradient Descent. Depending on
the algorithm you use, you may need to set up some time parameters,
such as learning rate or momentum.
6. And to finish, you will need to initialize the operations in order to
execute the whole thing before you go and start on the training.

While we did spend some time looking at some of the amazing things that we
are able to do with the TensorFlow library, remember that these are just a
sampling of all the neat models and algorithms that come with this library.
This is really a great program to use because it helps you out with artificial
intelligence, deep learning, and machine learning all wrapped up in one. If
this sounds like a library that can help out with some of the projects that you
want to complete, take some time to download it to your computer, and
experiment a bit to see what deep learning models and features it is able to
help you out with.
 Chapter 30:

Can I Work with the Keras Library?

The next library on the list that we can focus on a bit is the Keras library.
This is going to be one of the best options that we can work with when it
comes to deep learning and handling all of that data you want to work with
while completing a data analysis. Keep in mind here that the two top
numerical platforms that we can find with Python, the ones that can provide
us with some of the basis that we need for research and development in Deep
learning are going to include TensorFlow and Theano.

Both of those two libraries are going to have a lot of power with them, but
there are some times when they can be difficult to use in a direct manner in
order to create the deep learning models that we want. Because of this, we
often need to bring in the Keras library to help handle some of these models.
It works on top of the TensorFlow and Theano library, so you still get the
power and functionality that comes with these, with some added power and
ease of use with Keras to work with deep learning.
What Is Keras?
To start, we need to take a look at some of the basics of the Keras library,
what it can do for us, and the importance of learning some of this library.
Keras is considered a minimalist Python library for deep learning that can run
along with TensorFlow and Theano. This was a library that was developed to
make implementing the models by deep learning as fast and as easy as
possible, ensuring that developers and researchers are able to get things done
without all of the hassle.

To use this library, the programmer needs to make sure that they have either
Python 2.7 or 3.5 available on their computer. The library is also a great one
because it can seamlessly execute on CPUs and GPUs given the frameworks
that are underlying it. And it has been released thanks to the permissive
license of MIT so that other programmers and data scientists are able to get
the full benefit of using this.

The Keras library was developed and continues to be maintained by Francois

Chollet, who was an engineer for Google. This library is run on the four
guiding principles that include:

1. Modularity: This one goes by the idea that a model can be

understood as either a graph alone or a sequence. All of the concerns
that come with the model of deep learning are going to be discrete
components that the programmer can come in and combine in
arbitrary ways.
2. Minimalism: This library is about cutting down on some of the
clutter and keeping things as simple as possible. We find that the
Keras library is going to provide us with just enough tools and
functions and features to achieve our outcome. The idea is on
minimal frills and maximizing the readability.
3. Extensibility: This one means that the library is set up on purpose so
that it is easy to add and then use some of the new components
within that framework. This was done so that researchers are able to
try out new things and explore some of the new ideas that come their
way without issues.
4. Python. Everything that we do in this language is going to rely on the
native Python language. There will be no separate model files with
custom file formats. These can be nice in some cases, but often they
add in a lot of hassle and confusion that is just not worth it. Having
everything work with the Python language can make a big difference.

There are so many benefits that come with working on the Keras library. It
was designed to be as simple as possible while making the creation of models
of deep learning a bit easier than what we can see with TensorFlow or
Theano. Adding it in when you want to create some of your models can really
make things easier.
We can also spend some time looking at the basics of installing this library
on your system. Without installing this in the proper manner, it is going to be
impossible to get all of the efficiency, and all of the model creation
functionality, that we want out of this whole library.

First, we need to make sure that the Python library is all set up on our system,
but we are going to assume that this is already done and move on to some of
the other steps that you need to take in order to make this library work for
your needs. Some of the steps that a data scientist or a programmer can use in
order to add the Keras library on their system, and then start using it for their
needs includes:

1. OpenCV Installation Guides. This is going to be a kind of launchpad

that can link to a variety of tutorials that help with the whole
installing process. This can work on any of the systems that you are
trying to install this library.
2. Install Keras with TensorFlow: You will be able to use a pip in order
to install both the TensorFlow and the Keras libraries in just a few
minutes.
3. You can install these with the Raspberry Pi program if you would
like, but this is often hard to do it on these because of the limited
space, and they are hard to train ahead of time.
4. Install the libraries of Matplotlib, Scikit-Learn, and imutils. You
want to make sure that these are installed on your chosen computer
first. If you can, try to get this installed onto the virtual environment
as well. It is easy to install these using the pip and the following
code:

2 $ workon <your_env_name> # optional

3 $ pip install --upgrade imutils
4 $ pip install --upgrade scikit-learn
$ pip install --upgrade matplotlib

With the code that we just wrote above, it is not possible to get the whole
program of Keras on your computer, including all of the files and everything
else that you need, in order to work with this library. It is now time to try out
this library a bit and see what is inside, and how you can benefit from using
this library for your own needs.

Keras and TensorFlow

While we are on this topic, we need to take a look at how Keras works with
the TensorFlow library. Given that the project for TensorFlow has adopted
Keras as a high-level API for the newest release it seems that this library is
going to be a winner. This is because so many people are going to work with
Keras along with the TensorFlow library, that the developers of TensorFlow
have seen it as one library that they should welcome.

Remember that Keras is not a library that we are able to work on all on its
own. It needs to have either Theano or TensorFlow behind it to finish some
of the different things that we want, and to ensure that we are able to create
the models of deep learning that are necessary.

While some developers like to work with Theano to get some of this work
done, others like to be able to pull up TensorFlow and this is often the choice
that people prefer out of all of them. All of the features that come with
TensorFlow encourages it to be one of the best out there, and when you can
combine it with Keras, there is nothing you can’t do with deep learning.

The Principles of Keras

The principles that come with Keras are really unique and are part of the
reason why this kind of Python library is going to be so great to work with,
even when it comes to neural networks and deep learning models. To start
with, Keras was created in order to be more user-friendly, easy to extend
when needed, works with Python, and modular.

One of the key benefits that comes with this library is that it is designed for
humans to use, rather than machines. Sometimes the API of some of the other
coding libraries that we have talked about are too difficult to work with. They
are designed without the user in mind, and learning how to bring out the
functionalities and all of the algorithms can be a pain. But, with the API that
is found in Keras, that is not as big of a problem. It was designed to be used
for humans, while also following some of the best practices in the industry
for reducing the cognitive load.

If you have taken a look at some of the other Python libraries for deep
learning models before, you may have worried that the program would be too
hard to work with. And this is just not the case. Keras is set up to be easy for
anyone to use, and just by snooping around inside of it for a bit of time, you
will notice that it is intuitive, and easy to use, no matter how complicated
your model will turn out to be.

There are also a lot of standalone modules that come with the Keras library.
You will be able to combine these together as needed to create the new
models that you need with Python behind them. Some of the examples of the
standalone modules that you can enjoy will include the regularization
schemes, activation functions, initialization schemes, optimizers, cost
functions, and neural layers to name a few.

In addition to combining some of these standalone modules to create some of

the new models that you want to work with, it is easy to add in new modules.
This is usually done as new functions or classes. The models that are done
here will be defined in the code of Python, rather than having their own
separate model configuration files to work with.

The biggest reasons that you and other programmers would want to work
with the Keras library stem from those guiding principles that we talked
about before. Primarily, the fact that Keras is so user-friendly is making big
waves in this industry. Beyond the ease of learning this, and how easy it
makes the process of building models, Keras is going to offer us the
advantages of broad adoption, a support for a lot of options of production
deployment, integration with a minimum of five engines for the backend, and
a strong support for a lot of GPUs. We even find that it can help out with
some of the distributed training that we need to complete.

If you are still uncertain that this is the right library for you, think about all of
the companies who are already working with this library. It is not just
programmers and a few researchers who are happy with this library and using
it for their own needs, but also some of the big names in companies out there
are using this library as well. These include Uber, NVIDIA, Apple, Amazon,
Microsoft, and Google.

Building a Deep Learning Model with Keras

The final thing that we need to take a look at here is how we are able to work
with the Keras library in order to help us build some of our own models of
deep learning. The focus on Keras is going to be the idea of a model. The
main model that we are able to work within this library is going to be known
as a Sequence, and this is a linear stack of layers.

As a programmer, you will start out with a basic sequence, and then you can
add in some layers to it, adding them in the order that you wish for them to be
computed during the execution. Once you have had some time to define
these, you can then compile the model you will use, which basically will
make sure of the underlying framework to optimize the computation that you
want to see performed by that model. During this time, the programmer or the
researcher is able to specify the loss function, and the optimizer, that they
want to use.

After we have had some time to go through and compile all of this
information, we need to then take the model and find a method to fit it to the
data. There are a few methods that we are able to use to get this done. First,
you can do the fitting one batch of data at a time, or you can do it by firing
off the entire regime for the model training. This is where we will see the
computing happen in the first place.

Now, the training process is going to take some time, and it may not be as
efficient as we may hope. But the more data that we can show to the model,
and the more time we take to get this done with some high-quality data, the
better the model is going to perform. Once the model is properly trained, you
can then use that model to look through new data presented and make some
good predictions for you.
While there are a number of steps that have to happen to turn the model into
something that you can use for insights and predictions, especially when it is
a deep learning model, we are able to summarize this to make things a bit
easier. The summary that is needed to help us explain the construction of
deep learning models in Keras will include the following steps:

1. Define the model. This means that we have to start out with creating
our own sequence and then adding in the number of layers that we
want to work with.
2. Compile the model. This is the step where the programmer or the
researcher is able to specify all of the optimizers, as well as the loss
functions.
3. Fit your model. This is the step where you are able to execute the
model with all of the data that has been collected. This is going to
include a lot of training to ensure that the model is going to work the
way that we want.
4. Make predictions. After the model has had the time to go through the
proper training, we are then able to use that model to help generate
some of the predictions that are needed on any new data that is
presented. The model will get faster and more efficient at the
predictions as time goes on and it receives more data and more
feedback.

There is so much to love when it comes to working with the Keras library.
Many people worry that it is going to be too hard to work with, or that they
shouldn’t waste their time because this library won’t add in any
functionalities or extras that we can’t get from Theano or TensorFlow.
However, this is a library that is able to stand on its own. While it does rely
on one or the other of those libraries above, it can make deep learning model
creation easier and will ensure that we are able to see the best results with
some of our models in no time.
 Chapter 31:

The PyTorch Library and How This

Method Can Help with Machine
Learning

The next library that we need to take a look at is known as PyTorch. This is
going to be a Python-based package that works for scientific computing that
is going to rely on the power that it can receive from graphics processing
units. This library is also going to be one of the most common, and the
preferred, deep learning platforms for research in order to provide us with
maximum flexibility and a lot of speed in the process.
There are a lot of benefits that come with this kind of library. And it is known
for providing two of the most high-level features out of all the other deep
learning libraries. These will include tensor computation with the support of a
strong GPU acceleration, along with being able to build up the deep neural
networks on an autograd-system that is tape-based.
There are a lot of different libraries through Python that can help us work
with a lot of artificial intelligence and deep learning projects that we want to
work with. And the PyTorch library is one of these. One of the key reasons
that this library is so successful is because it is completely Pythonic and one
that is able to take some of the models that you want to build with a neural
network almost effortlessly. This is a newer deep learning library to work
with, but there is a lot of momentum in this field as well.

The Beginnings of PyTorch

As we mentioned above, PyTorch is one of the newest libraries out there that
works with Python and can help with deep learning. Even though it was just
released in January 2016, it has become one of the go-to libraries that data
scientists like to work with, mainly because it can make it easy to build up
really complex neural networks. This is perfect for a lot of beginners who
haven’t been able to work with these neural networks at all in the past. They
can work with PyTorch and make their own network in no time at all, even
with a limited amount of coding experience.

The creators of this Python library envisioned that this library would be
imperative when they wanted to run a lot of numerical computations as
quickly as possible. This is one of the ideal methodologies that also fits in
perfectly with the programming style that we see with Python. This library,
along with the Python library, as allowed debuggers of neural networks,
machine learning developers, and deep learning scientists to not only run but
also to test, parts of their code in real-time. This is great news because it
means that these professionals no longer have to wait for the entire code to
complete and execute before they can check out whether this code works or if
they need to fix certain parts.
In addition to some of the functionality that comes with the PyTorch library,
remember that you are able to extend out some of the functionalities of this
library by adding in other Python packages. Python packages like Cython,
SciPy, and NumPy all work well with PyTorch as well.
Even with these benefits, we still may have some questions about why the
PyTorch library is so special, and why we may want to use this when it is
time to build up the needed models for deep learning. The answer with this is
simple, mainly that PyTorch is going to be seen as a dynamic library. This
means that the library is really flexible and you can use it with any
requirements and changes that you would like. It is so good at doing this job
that it is being used by developers in artificial intelligence, students, and
researchers in many industries. In fact, in a Kaggle competition, this library
was used by almost all of the individuals who finished in the top ten.

While there are a lot of benefits that can come with the PyTorch library, we
need to start off with some of the highlights of why professionals of all sorts
love this language so much. Some of these include:
 1. The interface is simple to use. The PyTorch interface is going to
offer us an API that is easy to use. This means that we will find it
simple to operate and run like we do with Python.
2. It is Pythonic in nature. This library, since it is considered Pythonic,
will smoothly integrate to work with the Python data science stack.
For those who do not want to work with other coding languages
along the way, and want to just stick with the basics, and some of the
power, of Python, will be able to do so with this library. You will get
the leverage of using all of the functionalities and services that are
offered through the environment of Python.
3. Computational graphs: Another highlight that comes with the
PyTorch library is that it is going to provide us with the platform
with some dynamic computational graphs. This means that you have
the ability to change these graphs up even during runtime. This is
going to be useful any time that you need to work on some graphs
and you are not sure how much memory you need to use while
creating this model for a neural network.

The Community for PyTorch

The next thing that we need to take a look at here is some of the community
that comes with the PyTorch library. Because of all the benefits that come
with PyTorch, we can see that the community of developers and other
professionals is growing on a daily basis. In just a few years, this library has
shown a lot of developments and has even led this library to be cited in many
research papers and groups. And when it comes to artificial intelligence and
models of deep learning, PyTorch is starting to become one of the main
libraries to work with.

One of the interesting things that come with PyTorch is that it is still in the
early-release beta. But because so many programmers are adopting the
framework for deep learning already, and at such a brisk pace, it already
shows the power and the potential that comes with it, and how the
community is likely to continue growing. For example, even though we are
still on the beta release with PyTorch, there are currently 741 contributors
just on the GitHub repository right now. This means that there are more than
700 people working to enhance and add some improvements to the
functionalities of PyTorch that are already there.
Think of how amazing this is! PyTorch is not technically released yet and is
still in the early stages. But there has been so much buzz around this deep
learning library, and so many programmers have been using it for deep
learning and artificial intelligence, that there are already a ton of contributors
who are working to add some more functionality and improvements to this
library for others to work with.

PyTorch is not going to limit the specific applications that we are working
with because of the modular design and the flexibility that comes with it. It
has seen a heavy amount of use by some of the leading tech giants, and you
may even recognize some of the names. Those who have already started to
work with PyTorch to improve their own deep learning models will include
Uber, NVIDIA, Twitter, and Facebook. This library has also been used in a
lot of domains for research including neural networks, image recognition,
translation, and NLP among other key areas.

Why Use PyTorch with the Data Analysis

Anyone who is working with the field of data science, data analysis, artificial
intelligence, or deep learning has likely spent some time working with the
TensorFlow library that we also talked about in this guidebook. TensorFlow
may be the most popular library from Google, but because of the PyTorch
framework for deep learning, we can find that this library is able to solve a
few new problems when it comes to research work that these professionals
want to fix.

It is often believed that PyTorch is now the biggest competitor out there to
TensorFlow when it comes to handling data, and it is going to be one of the
best and most favorited artificial intelligence and deep learning library when
it comes to the community of research. There are a number of reasons for this
happening, and we will talk about some of these below:

First, we will notice that the dynamic computational graphs are going to be
popular among researchers. This library is going to avoid some of the static
graphs that can be used in other frameworks from TensorFlow. This allows
researchers and developers to change up how the network is going to behave
at the last minute. Some of those who are adopting this library will like it
because these graphs are more intuitive to learn when we compare it to what
TensorFlow is able to do.

The second benefit is that this one comes with a different kind of back-end
support. PyTorch is going to use a different backend based on what you are
doing. The GPU, CPU, and other functional features will all come with a
different backend rather than focusing on just one back-end to handle all of
these. For example, we are going to see the THC for our GPU, and TH for
CPU. Being able to use separate backends can make it easier for us to deploy
this library through a variety of constrained systems.

The imperative style is another benefit of working with this kind of library.
This means that when we work with this library, it is easy to use and very
intuitive. When you execute a line of code, it is going to get executed just as
you want, and you are able to work with some tracking that is in real-time.
this allows the programmer to keep track of how the models for neural
networks are doing. Because of the excellent architecture that comes with
this, and the lean and fast approach, it has been able to increase some of the
overall adoption that we are seeing with this library throughout the
communities of programmers.

Another benefit that we are going to enjoy when it comes to working with
PyTorch is that it is easy to extend. This library, in particular, is integrated to
work well with the code for C++ and it is going to share a bit of the backend
with this language when we work on our own framework for deep learning.
This means that a programmer is going to be able to not just use Python for
the CPU and GPU, but it can also add in the extension of the API using the C
or the C++ languages. This means that we are able to extend out the usage of
PyTorch for some of the new and experimental cases, which can make it even
better when we want to do some research with it.

And finally, the last benefit that we are going to focus on here is that PyTorch
is going to be seen as a Python approach library. This is because the library is
a native Python package just by the way that it is designed. The
functionalities that come with this are built as classes in Python, which means
that all of the code that you write here can integrate in a seamless manner
with the modules and packages of Python.

Similar to what we see with NumPy, this Python-based library is going to

enable us to work on a tensor that is GPU-accelerated, with computations that
provide us with some rich options for APIs while applying a neural network.
PyTorch is going to provide us with the research framework that we need
from one end to another, which will come with most of the different building
blocks that we need to carry out the research of deep learning on a day to day
basis. And we also notice that this library is going to provide us with a high-
level neural network modules because it can work with an API that is similar
to the Keras library as well.

PyTorch 1.0 - How This Moves Us from Research

to Production
During this chapter, we have spent some time discussing a lot of the strengths
that we are able to see with the PyTorch library, and how these will help to
make lots of researchers and data scientists run to it as their go-to library.
However, there are a few downsides that come with this library and one of
these includes that it has been really lacking when it comes to support of
production. However, due to some of the improvements and changes that will
happen with PyTorch, it is expected that this is something that will change
soon.

The next release of PyTorch, which is known as PyTorch 10, is already

expected to be a big release that is meant to overcome some of the biggest
challenges that researchers, programmers, and developers face in production.
This is the newest iteration of the whole framework, and it is expected to
combine with the Python-based Caffe2, allowing for deep learning
researchers and machine learning developers to move from research to
production. And the point of doing this is to allow the process to happen in a
manner that is hassle-free and without the programmer needing to deal with
challenges that show up in migration.

The new version 1.0 is meant to help to unify the research and production
capabilities in one framework, rather than doing it in two parts, making
things easier and avoiding some of the missed values and complications that
happen when you try to merge together two parts. This allows us with more
performance optimization and the flexibility that we need to complete both
research and production.

This newer version is going to promise us a lot of help with handling the
tasks that come up. Many of these tasks are going to make it more efficient to
run your models of deep learning on a much larger scale. Along with the
support of production, remember that PyTorch 10 will have a lot of
improvements with optimization and usability.

With the help of the PyTorch 1.0 library, we are going to be able to take the
existing code and continue to work on it as-is. The existing API is not going
to change so that makes it easier for those who have already been able to
create some coding and programs with the old API. To help make sense of
the progress that is going to happen with the PyTorch library, you can look at
the PyTorch website to help out.

As we can see with all of the information that we explored in this chapter,
PyTorch is already seen as a compelling player when it comes to the various
processes that we can do with artificial intelligence and deep learning. Being
able to exploit all of the unique parts that come with this, and seeing that it is
going to work as a research first library can be an important part of our data
analysis overall.

The PyTorch library is able to overcome some of the challenges that it has
and can provide us with all of the power and the performance that is
necessary to get the job done. If you are a student, a researcher, or a
mathematician who is inclined to work with some of the models of deep
learning, then the PyTorch library is going to be a great option as a
framework for deep learning to help you get started.
PART 8
 Chapter 32:

Looking at Machine Learning and

How This Fits In

While we are on this topic, we need to also spend some time looking at
machine learning and how it is able to fit in with the topics of deep learning
and our data analysis. Machine learning is another topic that is getting a lot of
attention throughout the business world, no matter which industry you spend
your time in, and learning how to make this happen, and the importance of
machine learning and other parts of artificial intelligence in your project and
the models that you create.

When you start diving into all of the advancements that are present with
artificial intelligence it sometimes seems a bit overwhelming. But if you are
just interested in learning some of the basics for now, you can boil down a lot
of the innovations that come with artificial intelligence into two main
concepts that are equally as important. These two concepts are going to
include the deep learning that we have already spent some time on, and
machine learning as well.
These are two terms that have garnered a lot of attention over the years, and
because of the buzz that comes with them, it is likely that many business
owners assume that these words can be used interchangeably. But there are
some key differences that come between machine learning and deep learning,
and it is definitely important to know the differences and how these two
methods relate to each other

With that in mind, we are going to take some time to explore more about
machine learning and how it is able to fit into the model that we are creating.
There are a lot of examples of both deep learning and machine learning, and
we use both of these topics on a regular basis. So, let’s dive right in and see a
bit more about the differences and similarities between machine learning and
deep learning.

What Is Machine Learning?

The first thing that we need to take a look at here is the basics of machine
learning. This is going to include a lot of algorithms that are able to first
parse the data we have, learn from that data, and then apply what they were
able to learn from that data over to make a more informed decision.
Basically, it is a process we can use in order to teach our machines and
programs on how to learn and make important decisions on their own.

Let’s take a look at an example of how this is meant to work. A good

example of this process would be a streaming service for on-demand music.
For this service to stick with some decisions about which artists or songs to
recommend to one of their listeners, the algorithms of machine learning will
be hard at work. These algorithms are able to associate the preferences of the
user with other listeners who have a similar taste in music. This technique,
which is often given the generic name of artificial intelligence, is going to be
used in many of the other services that are able to offer us recommendations
in an automatic manner.

Machine learning is going to fuel all sorts of tasks that are automated and that
can span across many industries. This could start out with some firms for data
security, who will hunt down malware and turn it off before it infects a lot of
computers. And it can go to finance professionals who want to prevent fraud
and make sure they are getting the alerts when there are some good trades
they can rely on.

We are able to take some of the algorithms that come with artificial
intelligence and program them in a manner that makes them learn on a
constant basis. This is going to be done in a manner that stimulates the
actions of a virtual personal assistant, and you will find that the algorithms
are able to do these jobs very efficiently.
Machine learning is going to be a complex program to work with, and often it
takes the right coding language, such as Python, and some of the best
libraries out there to get things done. The algorithms that you can create will
involve a lot of complex coding and math that can serve as a mechanical
function. This function is similar to what we may see a screen on a computer,
a car or a flashlight do for us.

When we say that something such as a process or a machine, is able to do

“machine learning” this basically means that it’s something that is able to
perform a function with the data you provide over to it, and then it can also
get progressively better at doing that task as time goes on. Think of this as
having a flashlight that is able to turn on any time that you say the words “it
is dark,” so it could recognize the different phrases that have the word dark
inside of them, and then knows to continue on with the action at hand.

Now, the way that we can train these machines to do the tricks above, and so
much more, can be really interesting. And there is no better way to work with
this than to add in a bit of neural networking and deep learning to the process
to make these results even more prevalent overall.
Machine Learning Vs. Deep Learning
Now we need to take a look at how machine learning and deep learning are
going to be the same, and how they can be different. When we look at this in
practical terms, deep learning is simply going to be a subset that we see with
machine learning. In fact, one reality that we see with this is that deep
learning is technically going to be a type of machine learning, and it will
function in a manner that is similar. This is why so many people who haven’t
been able to work with either of these topics may assume that they are the
same thing. However, it is important to understand that the capabilities
between deep learning and machine learning are going to be different.

While the basic models that come with machine learning are going to become
steadily better what the function you are training them to work with, they are
still going to rely on some guidance from you as well. If the algorithm gives
you a prediction that is inaccurate, then the engineer has to step in and make
sure that the necessary adjustments are done early on. With a model that
relies on deep learning though, the algorithm can determine, without any
help, whether the prediction that it made is accurate. This is done with the
help of a neural network.

Let’s go back to the example that we did with the flashlight earlier. You
could program this to turn on any time that it can recognize the audible cue of
someone when they repeat the word “dark.” As it continues to learn, it might
then turn on with any phrase that has that word as well. This can be done with
just a simple model out of machine learning.

But if we decide to add in a model from deep learning to help us get this
done, the flashlight would then be able to turn on with some other cues.
Instead of just waiting for the word “dark” to show up, we would see it work
when someone said a phrase like “the light switch won’t work” or “I can’t
see” which shows that they are in need of a light right then. A deep learning
model is going to be able to learn through its own method of computing,
which is going to be a technique that helps the system act in a manner that
seems like it has its own brain.

Adding in the Deep Learning to the Process

With this in mind, a model of deep learning is going to be designed in a
manner that can continually analyze data with a logic structure, and this is
done in a manner that is similar to the way that a human would look at
problems and draw conclusions. To help make this happen, the application of
deep learning is going to work with an artificial neural network, which is
going to be basically a layered structure of algorithms that we can use for
learning.

The design of this kind of network can seem a bit confusing in the beginning,
but it is designed to work similar to the biological neural network that we see
in the human brain. This is a great thing for the machine learning and deep
learning that you want to do because it can lead us over to a process of
learning that will be more capable of hard and complex than what the
standard models with machine learning can do.
Of course, there are going to be times when it is tricky to ensure that the
model of deep learning isn’t going to draw some conclusions that are
incorrect. We want it to be able to work on its own to get results, but we have
to make sure that we are not getting the wrong answers out of the model. And
we need to catch these issues as quickly as possible. If the model is able to
continue on and learn the wrong outputs and information, then it is not going
to be incorrect the whole time and will not do the work that we want.

Just like with some of the other examples that we are able to use with
artificial intelligence, it is going to require a lot of training to make sure that
we can see the learning processes turn out the right way. but when this is able
to work the way that it should, the functional deep learning is going to be
seen ore as a scientific marvel that can be the backbone of true artificial
intelligence.

A good example that we can look at right now for deep learning is the
AlphaGo product from Google. Google worked on creating a computer
program that worked with a neural network. In this computer program, the
network was able to learn how to play the board game that is known as Go,
which is one of those games that needs a lot of intuition and intellect to
complete.

This program started out by playing against other professional players of Go,
the model was able to learn how to play the game and beat out some of these
professionals, beating a level of intelligence in a system that had never been
seen before. And all of this was done without the program being told at all
when it should make a specific move. A model that followed the standard
machine learning requirements would need this guidance. But this program is
going to do it all on its own.

The part that really shocked everyone and brought this kind of technology to
the forefront is the idea that it was able to defeat many world-renowned
“masters” of the game. This means that not only could the machine learn
about some of the abstract aspects and complex techniques of the game, but it
was also becoming one of the best players of the game as well.
To recap this information and to help us to remember some of the differences
that show up between machine learning and deep learning, we need to discuss
some of the following:

1. Machine learning is going to work with algorithms in order to parse

data, learn from the data it has, and then make some smart and
informed decisions based on what the algorithm has been able to
learn along the way.
2. The structures of deep learning algorithms are going to come in
layers. This helps us to end up with an artificial neural network, that
is going to learn and make some intelligent decisions all on its own.
3. On the other hand, deep learning is going to be a subfield of machine
learning. Both of them are going to fall under the category of
artificial intelligence. But with deep learning, we are going to see
more of what powers the artificial intelligence that resembles the
human way of thinking in a machine.
Now, this is all going to seem a bit complicated at times because there is just
so much that has to come into play to make this work. The easiest takeaway
for us to understand some of the differences between what we see with deep
learning and machine learning is to just realize that deep learning is a type of
machine learning.

To make this a bit further and to add in some specifics, deep learning is going
to be considered a kind of evolution of machine learning, and shows us how
this kind of technology has changed over the years. There are a lot of
techniques that can show up with this kind of learning, but it often workings
with a neural network that we can program, and that will enable a machine to
make decisions that are accurate, without getting any help from a
programmer.

Data Is the Fuel We Need for the Future

With all of the data that we see being produced in our current era, the era of
Big Data, it is no surprise that we are going to see some innovations that
marvel and surprise us all of the time. And there are likely that we will see a
lot of new innovations in the future that we can’t even fathom yet. According
to many experts who are in the field, we will likely see a lot of these
innovations show up in applications with deep learning.

This can be a great thing for our future. Think of how this kind of technology
is going to be able to take over the world and help us to do things and
experience things in ways that we never thought possible. And just think
about where this kind of technology is going to be able to take us in the
future, and how it is going to change up a lot of the way that we interact with
each other and with other businesses as well.

Many of the applications that we see with artificial intelligence show up in

customer service. These are going to be used to help drive the advent of self­
service, they can increase the productivity of the agent, and they ensure that
the workflows are more reliable.

The data that is fed into these algorithms will come from a constant flux of
incoming customer queries. This can include a lot of relevant context into
some of the issues that the customers of the business are facing. There is
already a lot of data that a business is able to collect when they work with
their customers, and they can add in some third-party surveys and even ask
customers for opinions in order to create some good machine learning
algorithms. These algorithms then can help us figure out what the customers
want, what products to release, and more.

Machine learning and deep learning are two topics that seem to be almost the
same, and for those who are just getting started with this kind of process, it is
hard to see how these two terms are going to be different. as we dive more
into the idea of a data analysis and what we are able to do with all the data we
have been collected, we will see that machine learning and deep learning are
two very different processes that can help us do some amazing.
 Chapter 33

How Deep Learning Can Help with

Completing Your Own Predictive
Analysis

Many times, we can all agree that machine learning has been able to step in
and readily improve the way that we are going to interact with all of the data
we have, and with the internet. For example, when we talk about machine
learning, we can see it work with the search engines that we use, in spam
filters that get rid of unwanted emails based on just snippets of the wording in
each one, and even facial recognition software.

However, it is intriguing to take this a bit further and learn more about how
all of this works. We are going to start out with our own model of machine
learning here to see how things will work, and how we can use this for some
of our own predictive analysis as we go. For this one, we want to be able to
separate out the peaches from the mangoes. For machine learning to work,
we have to be able to create our own digital image of the objects, and then we
can classify them based on the sets of their features.

This is something that a simple handwritten code is able to do. This code
needs to have some descriptions inside of it that will be able to add in inputs
about the fruits including color, shape, and some of the other metrics that are
involved in the segregation between the peach and the mango. Once the code
is written, and it has enough power behind it, the machine will be able to
extract the shape and the color out of each desired fruit.
Now, in this program, we will find that the images can be differentiated
depending on the spaces as both fruits are going to be put on their own points
inside that chart. This means that if at a later time you add in a new point
within the map, the machine can have some success with separating out the
peach and the mango, based on the characteristics that you provided for each
fruit in the morning.

This is basically what the algorithms for machine learning are all about. It is
there to use a lot of techniques, including deep learning, to help differentiate
the data from other parts of data. And this is exactly how machine learning,
and some of the techniques that come with it, can then move into the realms
of things like self-learning and sometimes even artificial intelligence.

Of course, even with all of that in mind, we have to remember that machine
learning, just like with any of the other options of artificial intelligence and
data analysis that we want to work with, can be prone to mistakes This is
especially true when you first start working with the model. Errors are often
going to be experienced when we have a new set of computed features when
they are closer to the existing classes that we work with.

An example of this would be if you have a mango that is not quite the right
shape, and seems to resemble a peach a bit more. Or if the mango images are
not in cohesion with one another when it comes to the shape. To help us
minimize the errors that we see we should add in an extra layer and more
classes to ensure that the algorithm is as accurate as possible.

What Is a Predictive Analysis?

One topic that we need to explore a bit while we are here, before diving into
how deep learning is able to help us out with it, is what predictive analytics is
all about. To keep it simple, predictive analytics is going to be the use of
techniques from machine learning, data, and statistical algorithms to help
identify the likelihood of future outcomes based on data that is more
historical. The goal is to go beyond what we know happened in the past to
provide the best predictions and guess what will happen in the future.
Though the idea of predictive analytics is something that has been around for
a long time, it is a technology that is starting to garner more attention than
ever. Many companies, throughout a variety of industries, are turning to
predictive analytics to increase their bottom line, and to add a competitive
advantage to everyone who uses it. Some of the reasons why this predictive
analysis is gaining so much popularity now includes:

1. The data that is available can help with this. The growing volume as
well as the types of data are a good place to start. And there is more
interest from many companies in using data to help produce some
great insights in the process.
2. The computers and systems that are needed to complete this
predictive analysis is cheaper and faster than ever before.
3. The software to finish the predictive analysis is easier to use.
4. There is a lot of competition out there for companies to work again.
Thanks to these tougher conditions in the economy, and with the
competition, businesses need to find their own way to differentiate
and become better than the competition. A predictive analysis can
help them to do this.

With all of the interactive software that is easier than ever to use predictive
analytics has grown so much. It is no longer just the domain of those who
study math and statistics. Business experts and even business analysts are
able to use this kind of technology as well.

With this in mind, it is time to take a look at how the predictive analysis.
Predictive models are going to work by using known results to help train or
develop a model that can be used to predict values for different or new data.
Modeling that comes with the predictive analysis can provide us with results,
often in the form of a prediction that can represent a probability of the target
variable. This variable is going to vary based on the results that you are
looking to find and could include something like revenue.

Keep in mind here that this is going to be a bit different compared to the
descriptive models that can help us understand what happened in the past, or
some of the diagnostic models that we can use that help us understand some
key relationships and determine why we say a certain situation happen in the
past. In fact, there are entire books that will be devoted to the various
techniques and methods that are more analytical than others. And there are
even complete college curriculums that will dive into this subject as well, but
we can take a look at some of the basics that come with this process and how
we can use this for our needs as well.

There are two types of predictive models that we can take a look at first.
These are going to include the classification models and regression models.
To start with are the classification models that work to predict the
membership of a class. For example, you may work on a project to try and
figure out whether an employee is likely to leave the company, whether a
person is going to respond to solicitation from your business, or whether the
person has good or bad credit with them before you loan out some money.

For this kind of model, we are going to stick with binary options which
means the results have to come in at a 0 or a 1. So, the model results will
have these numbers, and 1 tells us that the event that you are targeting is
likely to happen. This can be a great way to make sure that we see whether
something is likely to happen or not.

Then we have the regression models. These are going to be responsible for
predicting a number for us. A good example of this would be predicting how
much revenue a customer is going to generate over the next year, or how
many months we have before a piece of our equipment will start to fail on a
machine so you can replace it.

There are a lot of different techniques that we are able to work with when it
comes to predictive modeling. The three most common types of techniques
that fall into this category of predictive modeling will include regression,
decision trees, and neural networks. Let’s take a look at some of these to see
how these can all work together.

First on the list is a decision tree. These are an example of classification

models that we can take a look at. This one is going to partition the data we
want to work with and put it into subsets, based on categories of the variables
that we use as input. A decision tree is going to look like a tree that has each
branch representing one of the choices that we can make. when we set this up
properly it is able to help us see how each choice we want to make compares
to the alternatives. Each leaf out of this decision tree is going to represent a
decision or a classification of the problem.

This model is helpful to work with because it looks at the data presented to it
and then tries to find the one variable that is there that can split up the data.
We want to make sure that the data is split up into logical groups that are the
most different from each other.

The decision tree is going to be popular because they are easy to interpret and
understand. They are also going to do well when it is time to handle missing
values, and are useful when it comes to preliminary selection of your data.
So, if you are working with a set of data that is missing many values or you
would like a quick and easy answer that you can interpret in no time, then a
decision tree is a good one to work with.

Then we need to move on to the regression. We are going to take a look at

logistic and linear regression. The regression is going to be one of the most
popular models to work with The regression analysis is going to estimate the
relationship that is present among the variables. It is also intended for
continuous data that can be assumed to follow a normal distribution, it finds
any of the big patterns that are found in sets of data, and it is often going to
be used to determine some of the specific factors that answer our business
questions, such as the price that can influence the movement of an asset.

As we work through with the regression analysis, we want to make sure that
we can predict a number, which is called the response, or in this case the Y
variable. With some linear regressions, we are going to have one independent
variable that can be used to help explain, or else predict, the outcome of Y.
And then the multiple regression is going to work with two or more
independent variables to help us predict what the outcome will be.

Then we can move on to the logistic regression. With this one, we are going
to see that it is the unknown variable of a discrete variable that is predicted
based on the known value of some of the variables. The response variable is
going to be more categorical, which means that it can assume only a limited
number of values compared to the others.

And finally, we have the binary logistic regression. This one is going to be a
response variable that has only two values that go with it. All of the results
that happen will come out as either 0 or as a 1. If we see 0, this means that the
expected result is not going to happen. And if it shows up as a 1, then this
means that our expected result is going to happen.

And then we can end with the neural networks like we talked about before.
These are going to be a more sophisticated technique that we are able to work
with that have the ability to model really complex relationships. These are
popular for a lot of reasons, but one of the biggest reasons is that the neural
networks are so flexible and powerful.

The power that we are able to see with the neural network is going to come
with the ability that these have with handling nonlinear relationships in data,
which is going to become more and more common as we work to collect
some more data. Many times, a data scientist will choose to work with the
neural network to help confirm the findings that come with the other
techniques that you used, including the decision trees and regression.

The neural networks are going to be based on a lot of features including

pattern recognition and some of the processes of artificial intelligence that
can model our parameters in a graphical manner. These are going to work
well when no mathematical formula is known that relates to both the inputs
and the outputs that we are doing., when prediction is going to be more
important than working with the explanation, or when there is a ton of
training data that we can work with.

Another option that we have to look at is the artificial neural networks. These
were originally developed by researchers who were trying to mimic what we
can find in the brain of a human on a machine. And when they were
successful, we get a lot of the modern pieces of technology that we like to use
today.
A predictive analysis is going to do a lot of great things for your business. It
can ensure that you will be able to handle some of the more complex
problems that your business faces, and will make it easier to know what is
likely to happen in the future. Basing your big business decisions on data, and
the insights found with predictive analysis can make it easier to beat out the
competition and get ahead.
Looking into Deep Learning
Now, going apart from some of the categories that we manually code, a
machine is able to use some of the neural networks to help it learn the
features of the objects you are working with. Looking back at the example
above, we can use the neural network to help the machine learn some of the
features that happen with peaches and mangoes, and the machine can do it all
on itself.

Remember from before, remember that these networks are going to work in a
manner that is similar to what we see with the brain, and they are going to
comprise of artificial neurons to help us model the data. There is also going
to be a type of synaptic response as well with the network that uses different
values as the inputs. The values that come with the featured output can then
be used as the new input to the other artificial neurons.

These neural networks are going to be the very core of what we see with
machines that can learn on their own. And we can expect, after the right kind
of training and testing, that they will segregate peaches from mangoes by
learning the traits from the perceived images.
This is a great place for deep learning to come into play because it will help
us to define the number of layers that we need to use inside our neural
network. To make this easier, this is basically the term of the depth of a given
network. For the network to be considered exhaustive, we need to add in as
many layers as possible, so we need to make it as deep as we can.

Even when you have taken the time to create a neural network, and the
features are all set up, many aren’t going to be put in real practice. The main
reason that this happens is because there isn’t enough data to make the
training work. If you have not been able to collect a lot of big data to use to
train the model, then the model will never know how it should behave, and
there is no way to ensure accuracy.

The most important pitfall that comes with this one though is that the
machine, or the system, may not be able to generate enough computing
power to get the work done. This is again a place where big data can come in
and help. This big data is able to help deep learning and the networks in a
natural manner and can offer some computational power with the help of
business intelligence. If you are able to add in enough power behind what
you are doing here, and you use big data in the proper manner, you will find
that the issues with computational power will not be as big of an issue.

The Applications of Deep Learning

In many companies, the use of machine learning and deep learning
techniques are going to be widely used. They can be used in the medical field
to help us learn more about different diseases and to help a doctor provide an
accurate diagnosis in a faster manner. They can be used in the retail industry
to help figure out the best marketing techniques to reach customers and
where to place different products. Manufacturing firms can use deep learning
to help them find where waste is to eliminate it, and to make predictions on
when a piece of equipment is going to fail, so they can fix it ahead of time.

Even with all of these options, there are still some other targeted approaches
and uses of machine learning, especially the kind that have been assisted with
deep learning. Some of these are going to include:

1. Text analysis: This is going to be a form of predictive analytics that

can help us extract what the sentiments behind the text, based on the
intensity of the presses on the keys, and the typing speeds.
2. Artificial intelligence: This may be its own field, but it can take a lot
of its cues from deep learning. This is because many of the artificial
intelligence models that you work with are going to put the idea of
neural networks to use. Google DeepMind is a good example of this.
3. Predictive anomaly analysis: Deep learning can help us to find out
 when there are some anomalies and abnormal patterns in signals.
This can be useful for many companies who want to catch
catastrophes early on and can help us to avoid some failures on major
systems.
a. The deep neural networks are going to be preferred in many
cases when it is time to detect these anomalies because it is
able to take the input signal and reconstruct it as the output. If
there are any changes that happen with the journey, this will
be reported. If you have a network that goes really deep, it is
possible to work with this information in a manner that is
more refined.
4. Failure analysis: Neural networks are often able to detect failures that
will happen, ahead of time, even when they aren’t meant for the
failing system. The server overloads and some other behaviors that
may be erratic are also easily detected if we set up deep learning.
5. Disruptions in IT environments: Most of the organizational services
of seen some big changes over the years, including things like
microservices, delivery systems, and IoT’s. While most teams are
working on a variety of tools to help comprehend the nature of these
services, deep learning is going to be helpful when it is time to gauge
the patterns and then change up the IT stacks.

Deep learning, along with machine learning, is the key to the future of big
data, and even to what we can see with business intelligence. This is why it is
important for us to address the issues of neural networks, with as much
precision as we can. The more precision that comes with these neural
networks and running them, the better they will perform. And we can bet that
as time goes on, machine learning is going to be the part that helps us to
really handle a lot of menial and even up to complex tasks in no time.
 Conclusion
Thank you for making it through to the end of Python for Data Analysis, let’s
hope it was informative and able to provide you with all of the tools you need
to achieve your goals whatever they may be.

The next step is to start putting some of the models and the Python libraries
that we discussed in this guidebook to good use. Companies are already
gathering up large amounts of data. Now it is time for us to take this data and
actually learn what predictions and insights are found inside all of this data.

This guidebook took some time looking at deep learning and how it can make
our data analysis easier than ever before. This data analysis is going to make
a difference in how you can run your business, and it will ensure that you are
able to get the best results each time out of that data. We looked at the
different languages, libraries, and techniques that we can use when it comes
to using deep learning for our data analysis.

When you are ready to take all of that data you have collected and put it to
use with a good data analysis with deep learning, make sure to check out this
guidebook to help you get started.
Finally, if you found this book useful in any way, a review on Amazon is
always appreciated!