Search icon Close icon
Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Checkout

Change country

Modal Close icon
Country selected Country selected
country flag United States
Country selected
country flag Great Britain
Country selected
country flag India
Country selected
country flag Germany
Country selected
country flag France
Country selected
country flag Canada
Country selected
country flag Russia
Country selected
country flag Spain
Country selected
country flag Brazil
Country selected
country flag Australia
Country selected
country flag Singapore
Country selected
country flag Canary Islands
Country selected
country flag Hungary
Country selected
country flag Ukraine
Country selected
country flag Luxembourg
Country selected
country flag Estonia
Country selected
country flag Lithuania
Country selected
country flag South Korea
Country selected
country flag Turkey
Country selected
country flag Switzerland
Country selected
country flag Colombia
Country selected
country flag Taiwan
Country selected
country flag Chile
Country selected
country flag Norway
Country selected
country flag Ecuador
Country selected
country flag Indonesia
Country selected
country flag New Zealand
Country selected
country flag Cyprus
Country selected
country flag Denmark
Country selected
country flag Finland
Country selected
country flag Poland
Country selected
country flag Malta
Country selected
country flag Czechia
Country selected
country flag Austria
Country selected
country flag Sweden
Country selected
country flag Italy
Country selected
country flag Egypt
Country selected
country flag Belgium
Country selected
country flag Portugal
Country selected
country flag Slovenia
Country selected
country flag Ireland
Country selected
country flag Romania
Country selected
country flag Greece
Country selected
country flag Argentina
Country selected
country flag Netherlands
Country selected
country flag Bulgaria
Country selected
country flag Latvia
Country selected
country flag South Africa
Country selected
country flag Malaysia
Country selected
country flag Japan
Country selected
country flag Slovakia
Country selected
country flag Philippines
Country selected
country flag Mexico
Country selected
country flag Thailand
Country selected
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Newsletter Hub
Free Learning
Arrow right icon
timer SALE ENDS IN
0 Days
:
00 Hours
:
00 Minutes
:
00 Seconds
Home > Data > Data Analysis > Python Data Analysis Cookbook
Python Data Analysis Cookbook
Python Data Analysis Cookbook

Python Data Analysis Cookbook: Clean, scrape, analyze, and visualize data with the power of Python!

Arrow left icon
Profile Icon Ivan Idris
Arrow right icon
€41.99
Full star icon Full star icon Full star icon Empty star icon Empty star icon 3 (2 Ratings)
Paperback Jul 2016 462 pages 1st Edition
eBook
€28.99 €32.99
Paperback
€41.99
Subscription
Free Trial
Renews at €18.99p/m
Arrow left icon
Profile Icon Ivan Idris
Arrow right icon
€41.99
Full star icon Full star icon Full star icon Empty star icon Empty star icon 3 (2 Ratings)
Paperback Jul 2016 462 pages 1st Edition
eBook
€28.99 €32.99
Paperback
€41.99
Subscription
Free Trial
Renews at €18.99p/m
eBook
€28.99 €32.99
Paperback
€41.99
Subscription
Free Trial
Renews at €18.99p/m

What do you get with Print?

Product feature icon Instant access to your digital copy whilst your Print order is Shipped
Product feature icon Paperback book shipped to your preferred address
Product feature icon Redeem a companion digital copy on all Print orders
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
OR

Contact Details

Modal Close icon
Payment Processing...
tick Completed

Shipping Address

Billing Address

Shipping Methods
Table of content icon View table of contents Preview book icon Preview Book

Python Data Analysis Cookbook

Chapter 2. Creating Attractive Data Visualizations

In this chapter, we will cover:

  • Graphing Anscombe's quartet
  • Choosing seaborn color palettes
  • Choosing matplotlib color maps
  • Interacting with IPython notebook widgets
  • Viewing a matrix of scatterplots
  • Visualizing with d3.js via mpld3
  • Creating heatmaps
  • Combining box plots and kernel density plots with violin plots
  • Visualizing network graphs with hive plots
  • Displaying geographical maps
  • Using ggplot2-like plots
  • Highlighting data points with influence plots

Introduction

Data analysis is more of an art than a science. Creating attractive visualizations is an integral part of this art. Obviously, what one person finds attractive, other people may find completely unacceptable. Just as in art, in the rapidly evolving world of data analysis, opinions, and taste change over time; however, in principle, nobody is absolutely right or wrong. As data artists and Pythonistas, we can choose from among several libraries of which I will cover matplotlib, seaborn, Bokeh, and ggplot. Installation instructions for some of the packages we use in this chapter were already covered in Chapter 1, Laying the Foundation for Reproducible Data Analysis, so I will not repeat them. I will provide an installation script (which uses pip only) for this chapter; you can even use the Docker image I described in the previous chapter. I decided to not include the Proj cartography library and the R-related libraries in the image because of their size. So for the two recipes...

Graphing Anscombe's quartet

Anscombe's quartet is a classic example that illustrates why visualizing data is important. The quartet consists of four datasets with similar statistical properties. Each dataset has a series of x values and dependent y values. We will tabulate these metrics in an IPython notebook. However, if you plot the datasets, they look surprisingly different compared to each other.

How to do it...

For this recipe, you need to perform the following steps:

  1. Start with the following imports:
    import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import matplotlib as mpl
from dautil import report
from dautil import plotting
import numpy as np
from tabulate import tabulate
  2. Define the following function to compute the mean, variance, and correlation of x and y within a dataset, the slope, and the intercept of a linear fit for each of the datasets:
    df = sns.load_dataset("anscombe")

    agg = df.groupby('dataset')\
             .agg...

Choosing seaborn color palettes

Seaborn color palettes are similar to matplotlib colormaps. Color can help you discover patterns in data and is an important visualization component. Seaborn has a wide range of color palettes, which I will try to visualize in this recipe.

How to do it...

  1. The imports are as follows:
    import seaborn as sns
import matplotlib.pyplot as plt
import matplotlib as mpl
import numpy as np
from dautil import plotting
  2. Use the following function that helps plot the palettes:
    def plot_palette(ax, plotter, pal, i, label, ncol=1):
    n = len(pal)
    x = np.linspace(0.0, 1.0, n)
    y = np.arange(n) + i * n
    ax.scatter(x, y, c=x, 
                cmap=mpl.colors.ListedColormap(list(pal)), 
                s=200)
    plotter.plot(x,y, label=label)
    handles, labels = ax.get_legend_handles_labels()
    ax.legend(loc='best', ncol=ncol, fontsize=18)
  3. Categorical palettes are useful for categorical data, for instance, gender or blood type. The following function plots...

Choosing matplotlib color maps

The matplotlib color maps are getting a lot of criticism lately because they can be misleading; however, most colormaps are just fine in my opinion. The defaults are getting a makeover in matplotlib 2.0 as announced at http://matplotlib.org/style_changes.html (retrieved July 2015). Of course, there are some good arguments that do not support using certain matplotlib colormaps, such as jet. In art, as in data analysis, almost nothing is absolutely true, so I leave it up to you to decide. In practical terms, I think it is important to consider how to deal with print publications and the various types of color blindness. In this recipe, I visualize relatively safe colormaps with colorbars. This is a tiny selection of the many colormaps in matplotlib.

How to do it...

  1. The imports are as follows:
    import matplotlib.pyplot as plt
import matplotlib as mpl
from dautil import plotting
  2. Plot the datasets with the following code:
    fig, axes = plt.subplots(4, 4)
cmaps = [&apos...

Interacting with IPython Notebook widgets

Interactive IPython notebook widgets are, at the time of writing (July 2015), an experimental feature. I, and as far as I know, many other people, hope that this feature will remain. In a nutshell, the widgets let you select values as you would with HTML forms. This includes sliders, drop-down boxes, and check boxes. As you can read, these widgets are very convenient for visualizing the weather data I introduced in Chapter 1, Laying the Foundation for Reproducible Data Analysis.

How to do it...

  1. Import the following:
    import seaborn as sns
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from IPython.html.widgets import interact
from dautil import data
from dautil import ts
  2. Load the data and request inline plots:
    %matplotlib inline
df = data.Weather.load()
  3. Define the following function, which displays bubble plots:
    def plot_data(x='TEMP', y='RAIN', z='WIND_SPEED', f='A', size=10, cmap='Blues...

Viewing a matrix of scatterplots

If you don't have many variables in your dataset, it is a good idea to view all the possible scatterplots for your data. You can do this with one function call from either seaborn or pandas. These functions display a matrix of plots with kernel density estimation plots or histograms on the diagonal.

How to do it...

  1. Imports the following:
    import pandas as pd
from dautil import data
from dautil import ts
import matplotlib.pyplot as plt
import seaborn as sns
import matplotlib as mpl
  2. Load the weather data with the following lines:
    df = data.Weather.load()
df = ts.groupby_yday(df).mean()
df.columns = [data.Weather.get_header(c) for c in df.columns]
  3. Plot with the Seaborn pairplot() function, which plots histograms on the diagonal by default:
    %matplotlib inline

# Seaborn plotting, issues due to NaNs
sns.pairplot(df.fillna(0))

    The following plots are the result:

    How to do it...
  4. Plot similarly with the pandas scatter_matrix() function and request kernel density estimation plots on...

Visualizing with d3.js via mpld3

D3.js is a JavaScript data visualization library released in 2011, which we can also use in an IPython notebook. We will add hovering tooltips to a regular matplotlib plot. As a bridge, we need the mpld3 package. This recipe doesn't require any JavaScript coding whatsoever.

Getting ready

I installed mpld3 0.2 with the following command:

$ [sudo] pip install mpld3

How to do it...

  1. Start with the imports and enable mpld3:
    %matplotlib inline
import matplotlib.pyplot as plt
import mpld3
mpld3.enable_notebook()
from mpld3 import plugins
import seaborn as sns
from dautil import data
from dautil import ts
  2. Load the weather data and plot it as follows:
    df = data.Weather.load()
df = df[['TEMP', 'WIND_SPEED']]
df = ts.groupby_yday(df).mean()

fig, ax = plt.subplots()
ax.set_title('Averages Grouped by Day of Year')
points = ax.scatter(df['TEMP'], df['WIND_SPEED'],
                    s=30, alpha=0.3)
ax.set_xlabel(data...

Creating heatmaps

Heat maps visualize data in a matrix using a set of colors. Originally, heat maps were used to represent prices of financial assets, such as stocks. Bokeh is a Python package that can display heatmaps in an IPython notebook or produce a standalone HTML file.

Getting ready

I have Bokeh 0.9.1 via Anaconda. The Bokeh installation instructions are available at http://bokeh.pydata.org/en/latest/docs/installation.html (retrieved July 2015).

How to do it...

  1. The imports are as follows:
    from collections import OrderedDict
from dautil import data
from dautil import ts
from dautil import plotting
import numpy as np
import bokeh.plotting as bkh_plt
from bokeh.models import HoverTool
  2. The following function loads temperature data and groups it by year and month:
    def load():
    df = data.Weather.load()['TEMP']
    return ts.groupby_year_month(df)
  3. Define a function that rearranges data in a special Bokeh structure:
    def create_source():
    colors = plotting.sample_hex_cmap()

    month...

Combining box plots and kernel density plots with violin plots

Violin plots combine box plots and kernel density plots or histograms in one type of plot. Seaborn and matplotlib both offer violin plots. We will use Seaborn in this recipe on z-scores of weather data. The z-scoring is not essential, but without it, the violins will be more spread out.

How to do it...

  1. Import the required libraries as follows:
    import seaborn as sns
from dautil import data
import matplotlib.pyplot as plt
  2. Load the weather data and calculate z-scores:
    df = data.Weather.load()
zscores = (df - df.mean())/df.std()
  3. Plot a violin plot of the z-scores:
    %matplotlib inline
plt.figure()
plt.title('Weather Violin Plot')
sns.violinplot(zscores.resample('M'))
plt.ylabel('Z-scores')

    Refer to the following plot for the first violin plot:

    How to do it...
  4. Plot a violin plot of rainy and dry (the opposite of rainy) days against wind speed:
    plt.figure()
plt.title('Rainy Weather vs Wind Speed')
categorical = df
categorical...

Visualizing network graphs with hive plots

A hive plot is a visualization technique for plotting network graphs. In hive plots, we draw edges as curved lines. We group nodes by some property and display them on radial axes. NetworkX is one of the most famous Python network graph libraries; however, it doesn't support hive plots yet (July 2015). Luckily, several libraries exist that specialize in hive plots. Also, we will use an API to partition the graph of Facebook users available at https://snap.stanford.edu/data/egonets-Facebook.html (retrieved July 2015). The data belongs to the Stanford Network Analysis Project (SNAP), which also has a Python API. Unfortunately, the SNAP API doesn't support Python 3 yet.

Getting ready

I have NetworkX 1.9.1 via Anaconda. The instructions to install NetworkX are at https://networkx.github.io/documentation/latest/install.html (retrieved July 2015). We also need the community package at https://bitbucket.org/taynaud/python-louvain (retrieved July...

Introduction

Data analysis is more of an art than a science. Creating attractive visualizations is an integral part of this art. Obviously, what one person finds attractive, other people may find completely unacceptable. Just as in art, in the rapidly evolving world of data analysis, opinions, and taste change over time; however, in principle, nobody is absolutely right or wrong. As data artists and Pythonistas, we can choose from among several libraries of which I will cover matplotlib, seaborn, Bokeh, and ggplot. Installation instructions for some of the packages we use in this chapter were already covered in Chapter 1, Laying the Foundation for Reproducible Data Analysis, so I will not repeat them. I will provide an installation script (which uses pip only) for this chapter; you can even use the Docker image I described in the previous chapter. I decided to not include the Proj cartography library and the R-related libraries in the image because of their size. So for the two recipes involved...

Graphing Anscombe's quartet

Anscombe's quartet is a classic example that illustrates why visualizing data is important. The quartet consists of four datasets with similar statistical properties. Each dataset has a series of x values and dependent y values. We will tabulate these metrics in an IPython notebook. However, if you plot the datasets, they look surprisingly different compared to each other.

How to do it...

For this recipe, you need to perform the following steps:

  1. Start with the following imports:

    import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import matplotlib as mpl
from dautil import report
from dautil import plotting
import numpy as np
from tabulate import tabulate

  2. Define the following function to compute the mean, variance, and correlation of x and y within a dataset, the slope, and the intercept of a linear fit for each of the datasets:

    df = sns.load_dataset("anscombe")

    agg = df.groupby('dataset')\
             .agg([np.mean, np.var])\
         ...

Choosing seaborn color palettes

Seaborn color palettes are similar to matplotlib colormaps. Color can help you discover patterns in data and is an important visualization component. Seaborn has a wide range of color palettes, which I will try to visualize in this recipe.

How to do it...

  1. The imports are as follows:

    import seaborn as sns
import matplotlib.pyplot as plt
import matplotlib as mpl
import numpy as np
from dautil import plotting

  2. Use the following function that helps plot the palettes:

    def plot_palette(ax, plotter, pal, i, label, ncol=1):
    n = len(pal)
    x = np.linspace(0.0, 1.0, n)
    y = np.arange(n) + i * n
    ax.scatter(x, y, c=x, 
                cmap=mpl.colors.ListedColormap(list(pal)), 
                s=200)
    plotter.plot(x,y, label=label)
    handles, labels = ax.get_legend_handles_labels()
    ax.legend(loc='best', ncol=ncol, fontsize=18)

  3. Categorical palettes are useful for categorical data, for instance, gender or blood type. The following function plots some of...

Left arrow icon

Page 1 of 14

Right arrow icon

Key benefits

  • Analyze Big Data sets, create attractive visualizations, and manipulate and process various data types
  • Packed with rich recipes to help you learn and explore amazing algorithms for statistics and machine learning
  • Authored by Ivan Idris, expert in python programming and proud author of eight highly reviewed books

Description

Data analysis is a rapidly evolving field and Python is a multi-paradigm programming language suitable for object-oriented application development and functional design patterns. As Python offers a range of tools and libraries for all purposes, it has slowly evolved as the primary language for data science, including topics on: data analysis, visualization, and machine learning. Python Data Analysis Cookbook focuses on reproducibility and creating production-ready systems. You will start with recipes that set the foundation for data analysis with libraries such as matplotlib, NumPy, and pandas. You will learn to create visualizations by choosing color maps and palettes then dive into statistical data analysis using distribution algorithms and correlations. You’ll then help you find your way around different data and numerical problems, get to grips with Spark and HDFS, and then set up migration scripts for web mining. In this book, you will dive deeper into recipes on spectral analysis, smoothing, and bootstrapping methods. Moving on, you will learn to rank stocks and check market efficiency, then work with metrics and clusters. You will achieve parallelism to improve system performance by using multiple threads and speeding up your code. By the end of the book, you will be capable of handling various data analysis techniques in Python and devising solutions for problem scenarios.

Who is this book for?

This book teaches Python data analysis at an intermediate level with the goal of transforming you from journeyman to master. Basic Python and data analysis skills and affinity are assumed.

What you will learn

  • Set up reproducible data analysis
  • Clean and transform data
  • Apply advanced statistical analysis
  • Create attractive data visualizations
  • Web scrape and work with databases, Hadoop, and Spark
  • Analyze images and time series data
  • Mine text and analyze social networks
  • Use machine learning and evaluate the results
  • Take advantage of parallelism and concurrency
Estimated delivery fee Deliver to Switzerland

Standard delivery 10 - 13 business days

€11.95

Premium delivery 3 - 6 business days

€16.95
(Includes tracking information)

Product Details

Country selected
Publication date, Length, Edition, Language, ISBN-13
Publication date : Jul 22, 2016
Length: 462 pages
Edition : 1st
Language : English
ISBN-13 : 9781785282287
Category :
Data
Languages :
Python
Concepts :
Data Analysis

What do you get with Print?

Product feature icon Instant access to your digital copy whilst your Print order is Shipped
Product feature icon Paperback book shipped to your preferred address
Product feature icon Redeem a companion digital copy on all Print orders
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
OR

Contact Details

Modal Close icon
Payment Processing...
tick Completed

Shipping Address

Billing Address

Shipping Methods
Estimated delivery fee Deliver to Switzerland

Standard delivery 10 - 13 business days

€11.95

Premium delivery 3 - 6 business days

€16.95
(Includes tracking information)

Product Details

Publication date : Jul 22, 2016
Length: 462 pages
Edition : 1st
Language : English
ISBN-13 : 9781785282287
Category :
Data
Languages :
Python
Concepts :
Data Analysis

Packt Subscriptions

See our plans and pricing
Modal Close icon
€18.99 billed monthly
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Simple pricing, no contract
START FREE TRIAL
BUY NOW
€189.99 billed annually
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just €5 each
Feature tick icon Exclusive print discounts
START FREE TRIAL
BUY NOW
€264.99 billed in 18 months
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just €5 each
Feature tick icon Exclusive print discounts
START FREE TRIAL
BUY NOW

Frequently bought together

Python Machine Learning Cookbook
Python Machine Learning Cookbook
Read more
Jun 2016 304 pages
Full star icon 4.4 (5)
eBook
eBook
  • eBook eBook €35.98
  • Paperback Paperback €49.99
€35.98 €39.99
€49.99
Python Data Analysis Cookbook
Python Data Analysis Cookbook
Read more
Jul 2016 462 pages
Full star icon 3 (2)
eBook
eBook
  • eBook eBook €28.99
  • Paperback Paperback €41.99
€28.99 €32.99
€41.99
Advanced Machine Learning with Python
Advanced Machine Learning with Python
Read more
Jul 2016 278 pages
Full star icon 3.1 (7)
eBook
eBook
  • eBook eBook €26.98
  • Paperback Paperback €36.99
€26.98 €29.99
€36.99
Stars icon
Total € 128.97
Python Machine Learning Cookbook
€49.99
Python Data Analysis Cookbook
€41.99
Advanced Machine Learning with Python
€36.99
Total € 128.97 Stars icon

Table of Contents

17 Chapters
1. Laying the Foundation for Reproducible Data Analysis Chevron down icon Chevron up icon
1. Laying the Foundation for Reproducible Data Analysis
Introduction
Setting up Anaconda
Installing the Data Science Toolbox
Creating a virtual environment with virtualenv and virtualenvwrapper
Sandboxing Python applications with Docker images
Keeping track of package versions and history in IPython Notebook
Configuring IPython
Learning to log for robust error checking
Unit testing your code
Configuring pandas
Configuring matplotlib
Seeding random number generators and NumPy print options
Standardizing reports, code style, and data access
2. Creating Attractive Data Visualizations Chevron down icon Chevron up icon
2. Creating Attractive Data Visualizations
Introduction
Graphing Anscombe's quartet
Choosing seaborn color palettes
Choosing matplotlib color maps
Interacting with IPython Notebook widgets
Viewing a matrix of scatterplots
Visualizing with d3.js via mpld3
Creating heatmaps
Combining box plots and kernel density plots with violin plots
Visualizing network graphs with hive plots
Displaying geographical maps
Using ggplot2-like plots
Highlighting data points with influence plots
3. Statistical Data Analysis and Probability Chevron down icon Chevron up icon
3. Statistical Data Analysis and Probability
Introduction
Fitting data to the exponential distribution
Fitting aggregated data to the gamma distribution
Fitting aggregated counts to the Poisson distribution
Determining bias
Estimating kernel density
Determining confidence intervals for mean, variance, and standard deviation
Sampling with probability weights
Exploring extreme values
Correlating variables with Pearson's correlation
Correlating variables with the Spearman rank correlation
Correlating a binary and a continuous variable with the point biserial correlation
Evaluating relations between variables with ANOVA
4. Dealing with Data and Numerical Issues Chevron down icon Chevron up icon
4. Dealing with Data and Numerical Issues
Introduction
Clipping and filtering outliers
Winsorizing data
Measuring central tendency of noisy data
Normalizing with the Box-Cox transformation
Transforming data with the power ladder
Transforming data with logarithms
Rebinning data
Applying logit() to transform proportions
Fitting a robust linear model
Taking variance into account with weighted least squares
Using arbitrary precision for optimization
Using arbitrary precision for linear algebra
5. Web Mining, Databases, and Big Data Chevron down icon Chevron up icon
5. Web Mining, Databases, and Big Data
Introduction
Simulating web browsing
Scraping the Web
Dealing with non-ASCII text and HTML entities
Implementing association tables
Setting up database migration scripts
Adding a table column to an existing table
Adding indices after table creation
Setting up a test web server
Implementing a star schema with fact and dimension tables
Using HDFS
Setting up Spark
Clustering data with Spark
6. Signal Processing and Timeseries Chevron down icon Chevron up icon
6. Signal Processing and Timeseries
Introduction
Spectral analysis with periodograms
Estimating power spectral density with the Welch method
Analyzing peaks
Measuring phase synchronization
Exponential smoothing
Evaluating smoothing
Using the Lomb-Scargle periodogram
Analyzing the frequency spectrum of audio
Analyzing signals with the discrete cosine transform
Block bootstrapping time series data
Moving block bootstrapping time series data
Applying the discrete wavelet transform
7. Selecting Stocks with Financial Data Analysis Chevron down icon Chevron up icon
7. Selecting Stocks with Financial Data Analysis
Introduction
Computing simple and log returns
Ranking stocks with the Sharpe ratio and liquidity
Ranking stocks with the Calmar and Sortino ratios
Analyzing returns statistics
Correlating individual stocks with the broader market
Exploring risk and return
Examining the market with the non-parametric runs test
Testing for random walks
Determining market efficiency with autoregressive models
Creating tables for a stock prices database
Populating the stock prices database
Optimizing an equal weights two-asset portfolio
8. Text Mining and Social Network Analysis Chevron down icon Chevron up icon
8. Text Mining and Social Network Analysis
Introduction
Creating a categorized corpus
Tokenizing news articles in sentences and words
Stemming, lemmatizing, filtering, and TF-IDF scores
Recognizing named entities
Extracting topics with non-negative matrix factorization
Implementing a basic terms database
Computing social network density
Calculating social network closeness centrality
Determining the betweenness centrality
Estimating the average clustering coefficient
Calculating the assortativity coefficient of a graph
Getting the clique number of a graph
Creating a document graph with cosine similarity
9. Ensemble Learning and Dimensionality Reduction Chevron down icon Chevron up icon
9. Ensemble Learning and Dimensionality Reduction
Introduction
Recursively eliminating features
Applying principal component analysis for dimension reduction
Applying linear discriminant analysis for dimension reduction
Stacking and majority voting for multiple models
Learning with random forests
Fitting noisy data with the RANSAC algorithm
Bagging to improve results
Boosting for better learning
Nesting cross-validation
Reusing models with joblib
Hierarchically clustering data
Taking a Theano tour
10. Evaluating Classifiers, Regressors, and Clusters Chevron down icon Chevron up icon
10. Evaluating Classifiers, Regressors, and Clusters
Introduction
Getting classification straight with the confusion matrix
Computing precision, recall, and F1-score
Examining a receiver operating characteristic and the area under a curve
Visualizing the goodness of fit
Computing MSE and median absolute error
Evaluating clusters with the mean silhouette coefficient
Comparing results with a dummy classifier
Determining MAPE and MPE
Comparing with a dummy regressor
Calculating the mean absolute error and the residual sum of squares
Examining the kappa of classification
Taking a look at the Matthews correlation coefficient
11. Analyzing Images Chevron down icon Chevron up icon
11. Analyzing Images
Introduction
Setting up OpenCV
Applying Scale-Invariant Feature Transform (SIFT)
Detecting features with SURF
Quantizing colors
Denoising images
Extracting patches from an image
Detecting faces with Haar cascades
Searching for bright stars
Extracting metadata from images
Extracting texture features from images
Applying hierarchical clustering on images
Segmenting images with spectral clustering
12. Parallelism and Performance Chevron down icon Chevron up icon
12. Parallelism and Performance
Introduction
Just-in-time compiling with Numba
Speeding up numerical expressions with Numexpr
Running multiple threads with the threading module
Launching multiple tasks with the concurrent.futures module
Accessing resources asynchronously with the asyncio module
Distributed processing with execnet
Profiling memory usage
Calculating the mean, variance, skewness, and kurtosis on the fly
Caching with a least recently used cache
Caching HTTP requests
Streaming counting with the Count-min sketch
Harnessing the power of the GPU with OpenCL
A. Glossary Chevron down icon Chevron up icon
A. Glossary
B. Function Reference Chevron down icon Chevron up icon
B. Function Reference
IPython
Matplotlib
NumPy
pandas
Scikit-learn
SciPy
Seaborn
Statsmodels
C. Online Resources Chevron down icon Chevron up icon
C. Online Resources
IPython notebooks and open data
Mathematics and statistics
D. Tips and Tricks for Command-Line and Miscellaneous Tools Chevron down icon Chevron up icon
D. Tips and Tricks for Command-Line and Miscellaneous Tools
IPython notebooks
Command-line tools
The alias command
Command-line history
Reproducible sessions
Docker tips
Index Chevron down icon Chevron up icon
Index

Recommendations for you

Left arrow icon
LLM Engineer's Handbook
LLM Engineer's Handbook
Read more
Oct 2024 522 pages
Full star icon 4.9 (29)
eBook
eBook
  • eBook eBook €43.99
  • Paperback Paperback €54.99
€43.99
€54.99
Getting Started with Tableau 2018.x
Getting Started with Tableau 2018.x
Read more
Sep 2018 396 pages
Full star icon 4 (3)
eBook
eBook
  • eBook eBook €28.99
  • Paperback Paperback €41.99
€28.99 €32.99
€41.99
Semantic Kernel SDK for Intelligent Applications
Semantic Kernel SDK for Intelligent Applications
Read more
Dec 2024 3hrs 35mins
Video
Video
  • Video Video €90.99
€90.99
AI Ecosystem for the Absolute Beginners - Hands-On
AI Ecosystem for the Absolute Beginners - Hands-On
Read more
Dec 2024 5hrs 7mins
Full star icon 2 (1)
Video
Video
  • Video Video €89.99
€89.99
Python for Algorithmic Trading Cookbook
Python for Algorithmic Trading Cookbook
Read more
Aug 2024 406 pages
Full star icon 4.2 (21)
eBook
eBook
  • eBook eBook €31.99
  • Paperback Paperback €44.99
€31.99 €35.99
€44.99
RAG-Driven Generative AI
RAG-Driven Generative AI
Read more
Sep 2024 338 pages
Full star icon 4.3 (18)
eBook
eBook
  • eBook eBook €28.99
  • Paperback Paperback €40.99
€28.99 €32.99
€40.99
Machine Learning with PyTorch and Scikit-Learn
Machine Learning with PyTorch and Scikit-Learn
Read more
Feb 2022 774 pages
Full star icon 4.4 (88)
eBook
eBook
  • eBook eBook €28.99
  • Paperback Paperback €41.99
  • Hardcover Hardcover €59.99
€28.99 €32.99
€41.99
€59.99
Building LLM Powered Applications
Building LLM Powered Applications
Read more
May 2024 342 pages
Full star icon 4.2 (21)
eBook
eBook
  • eBook eBook €26.98
  • Paperback Paperback €37.99
€26.98 €29.99
€37.99
Python Machine Learning By Example
Python Machine Learning By Example
Read more
Jul 2024 526 pages
Full star icon 4.9 (9)
eBook
eBook
  • eBook eBook €24.99
  • Paperback Paperback €34.99
€24.99 €27.99
€34.99
AI Product Manager's Handbook
AI Product Manager's Handbook
Read more
Nov 2024 488 pages
eBook
eBook
  • eBook eBook €23.99
  • Paperback Paperback €33.99
€23.99 €26.99
€33.99
Right arrow icon

Customer reviews

Rating distribution
Full star icon Full star icon Full star icon Empty star icon Empty star icon 3
(2 Ratings)
5 star 0%
4 star 0%
3 star 100%
2 star 0%
1 star 0%
Dimitri Shvorob Oct 31, 2016
Full star icon Full star icon Full star icon Empty star icon Empty star icon 3
The author has kindly made the code in the book publicly available - I will ungratefully advise interested readers to get the code and skip the book. (Python beginners should skip both). "Python Data Analysis Cookbook" is a typical low-quality Packt "book product" - I don't want to call these things "books" - which packages, but does not add much value to, a ragtag but large collection of Python code. The considerable page count should be heavily discounted - first, because a Packt page, ahem, packs less text than a page in a book from a regular publisher; second, because the author supplies a self-contained code sample, with a 2-by-2 plot visualization, for most of his "recipes". (It might take a single line to calculate a correlation coefficient, but the code will take a page, and the large chart will take another). Even so, you get a whole lot of code, doing all sorts of things, from Fourier transforms to manipulating database tables. (And, consistently, plotting - but using the author's own "dalib" Python package. This means that you will have to stick with "dalib" as well). This scattershot approach is a double-edged sword: it is likely that you will find *something* useful, but it is also likely that *most* of the book will be a dead weight. And then you ask how well that useful something is explained, and how easy would it be to find something similar online... A "pass" from me; review the table of contents to make up your mind.
Amazon Verified review Amazon
Tseliso Nov 03, 2016
Full star icon Full star icon Full star icon Empty star icon Empty star icon 3
Not as good as one expected.
Amazon Verified review Amazon

People who bought this also bought

Left arrow icon
Causal Inference and Discovery in Python
Causal Inference and Discovery in Python
Read more
May 2023 466 pages
Full star icon 4.5 (47)
eBook
eBook
  • eBook eBook €28.99
  • Paperback Paperback €40.99
€28.99 €32.99
€40.99
Generative AI with LangChain
Generative AI with LangChain
Read more
Dec 2023 376 pages
Full star icon 4.1 (33)
eBook
eBook
  • eBook eBook €42.99
  • Paperback Paperback €59.99
€42.99 €47.99
€59.99
Modern Generative AI with ChatGPT and OpenAI Models
Modern Generative AI with ChatGPT and OpenAI Models
Read more
May 2023 286 pages
Full star icon 4.1 (30)
eBook
eBook
  • eBook eBook €26.98
  • Paperback Paperback €37.99
€26.98 €29.99
€37.99
Deep Learning with TensorFlow and Keras – 3rd edition
Deep Learning with TensorFlow and Keras – 3rd edition
Read more
Oct 2022 698 pages
Full star icon 4.5 (44)
eBook
eBook
  • eBook eBook €26.98
  • Paperback Paperback €37.99
€26.98 €29.99
€37.99
Machine Learning Engineering with Python
Machine Learning Engineering with Python
Read more
Aug 2023 462 pages
Full star icon 4.6 (37)
eBook
eBook
  • eBook eBook €26.98
  • Paperback Paperback €37.99
€26.98 €29.99
€37.99
Right arrow icon

About the author

Profile icon Ivan Idris
Ivan Idris
Ivan Idris has an MSc in experimental physics. His graduation thesis had a strong emphasis on applied computer science. After graduating, he worked for several companies as a Java developer, data warehouse developer, and QA analyst. His main professional interests are business intelligence, big data, and cloud computing. Ivan Idris enjoys writing clean, testable code and interesting technical articles. Ivan Idris is the author of NumPy 1.5. Beginner's Guide and NumPy Cookbook by Packt Publishing.
Read more
See other products by Ivan Idris
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

What is the digital copy I get with my Print order? Chevron down icon Chevron up icon

When you buy any Print edition of our Books, you can redeem (for free) the eBook edition of the Print Book you’ve purchased. This gives you instant access to your book when you make an order via PDF, EPUB or our online Reader experience.

What is the delivery time and cost of print book? Chevron down icon Chevron up icon

Shipping Details

USA:

'

Economy: Delivery to most addresses in the US within 10-15 business days

Premium: Trackable Delivery to most addresses in the US within 3-8 business days

UK:

Economy: Delivery to most addresses in the U.K. within 7-9 business days.
Shipments are not trackable

Premium: Trackable delivery to most addresses in the U.K. within 3-4 business days!
Add one extra business day for deliveries to Northern Ireland and Scottish Highlands and islands

EU:

Premium: Trackable delivery to most EU destinations within 4-9 business days.

Australia:

Economy: Can deliver to P. O. Boxes and private residences.
Trackable service with delivery to addresses in Australia only.
Delivery time ranges from 7-9 business days for VIC and 8-10 business days for Interstate metro
Delivery time is up to 15 business days for remote areas of WA, NT & QLD.

Premium: Delivery to addresses in Australia only
Trackable delivery to most P. O. Boxes and private residences in Australia within 4-5 days based on the distance to a destination following dispatch.

India:

Premium: Delivery to most Indian addresses within 5-6 business days

Rest of the World:

Premium: Countries in the American continent: Trackable delivery to most countries within 4-7 business days

Asia:

Premium: Delivery to most Asian addresses within 5-9 business days

Disclaimer:
All orders received before 5 PM U.K time would start printing from the next business day. So the estimated delivery times start from the next day as well. Orders received after 5 PM U.K time (in our internal systems) on a business day or anytime on the weekend will begin printing the second to next business day. For example, an order placed at 11 AM today will begin printing tomorrow, whereas an order placed at 9 PM tonight will begin printing the day after tomorrow.


Unfortunately, due to several restrictions, we are unable to ship to the following countries:

  1. Afghanistan
  2. American Samoa
  3. Belarus
  4. Brunei Darussalam
  5. Central African Republic
  6. The Democratic Republic of Congo
  7. Eritrea
  8. Guinea-bissau
  9. Iran
  10. Lebanon
  11. Libiya Arab Jamahriya
  12. Somalia
  13. Sudan
  14. Russian Federation
  15. Syrian Arab Republic
  16. Ukraine
  17. Venezuela
What is custom duty/charge? Chevron down icon Chevron up icon

Customs duty are charges levied on goods when they cross international borders. It is a tax that is imposed on imported goods. These duties are charged by special authorities and bodies created by local governments and are meant to protect local industries, economies, and businesses.

Do I have to pay customs charges for the print book order? Chevron down icon Chevron up icon

The orders shipped to the countries that are listed under EU27 will not bear custom charges. They are paid by Packt as part of the order.

List of EU27 countries: www.gov.uk/eu-eea:

A custom duty or localized taxes may be applicable on the shipment and would be charged by the recipient country outside of the EU27 which should be paid by the customer and these duties are not included in the shipping charges been charged on the order.

How do I know my custom duty charges? Chevron down icon Chevron up icon

The amount of duty payable varies greatly depending on the imported goods, the country of origin and several other factors like the total invoice amount or dimensions like weight, and other such criteria applicable in your country.

For example:

  • If you live in Mexico, and the declared value of your ordered items is over $ 50, for you to receive a package, you will have to pay additional import tax of 19% which will be $ 9.50 to the courier service.
  • Whereas if you live in Turkey, and the declared value of your ordered items is over € 22, for you to receive a package, you will have to pay additional import tax of 18% which will be € 3.96 to the courier service.
How can I cancel my order? Chevron down icon Chevron up icon

Cancellation Policy for Published Printed Books:

You can cancel any order within 1 hour of placing the order. Simply contact customercare@packt.com with your order details or payment transaction id. If your order has already started the shipment process, we will do our best to stop it. However, if it is already on the way to you then when you receive it, you can contact us at customercare@packt.com using the returns and refund process.

Please understand that Packt Publishing cannot provide refunds or cancel any order except for the cases described in our Return Policy (i.e. Packt Publishing agrees to replace your printed book because it arrives damaged or material defect in book), Packt Publishing will not accept returns.

What is your returns and refunds policy? Chevron down icon Chevron up icon

Return Policy:

We want you to be happy with your purchase from Packtpub.com. We will not hassle you with returning print books to us. If the print book you receive from us is incorrect, damaged, doesn't work or is unacceptably late, please contact Customer Relations Team on customercare@packt.com with the order number and issue details as explained below:

  1. If you ordered (eBook, Video or Print Book) incorrectly or accidentally, please contact Customer Relations Team on customercare@packt.com within one hour of placing the order and we will replace/refund you the item cost.
  2. Sadly, if your eBook or Video file is faulty or a fault occurs during the eBook or Video being made available to you, i.e. during download then you should contact Customer Relations Team within 14 days of purchase on customercare@packt.com who will be able to resolve this issue for you.
  3. You will have a choice of replacement or refund of the problem items.(damaged, defective or incorrect)
  4. Once Customer Care Team confirms that you will be refunded, you should receive the refund within 10 to 12 working days.
  5. If you are only requesting a refund of one book from a multiple order, then we will refund you the appropriate single item.
  6. Where the items were shipped under a free shipping offer, there will be no shipping costs to refund.

On the off chance your printed book arrives damaged, with book material defect, contact our Customer Relation Team on customercare@packt.com within 14 days of receipt of the book with appropriate evidence of damage and we will work with you to secure a replacement copy, if necessary. Please note that each printed book you order from us is individually made by Packt's professional book-printing partner which is on a print-on-demand basis.

What tax is charged? Chevron down icon Chevron up icon

Currently, no tax is charged on the purchase of any print book (subject to change based on the laws and regulations). A localized VAT fee is charged only to our European and UK customers on eBooks, Video and subscriptions that they buy. GST is charged to Indian customers for eBooks and video purchases.

What payment methods can I use? Chevron down icon Chevron up icon

You can pay with the following card types:

  1. Visa Debit
  2. Visa Credit
  3. MasterCard
  4. PayPal
What is the delivery time and cost of print books? Chevron down icon Chevron up icon

Shipping Details

USA:

'

Economy: Delivery to most addresses in the US within 10-15 business days

Premium: Trackable Delivery to most addresses in the US within 3-8 business days

UK:

Economy: Delivery to most addresses in the U.K. within 7-9 business days.
Shipments are not trackable

Premium: Trackable delivery to most addresses in the U.K. within 3-4 business days!
Add one extra business day for deliveries to Northern Ireland and Scottish Highlands and islands

EU:

Premium: Trackable delivery to most EU destinations within 4-9 business days.

Australia:

Economy: Can deliver to P. O. Boxes and private residences.
Trackable service with delivery to addresses in Australia only.
Delivery time ranges from 7-9 business days for VIC and 8-10 business days for Interstate metro
Delivery time is up to 15 business days for remote areas of WA, NT & QLD.

Premium: Delivery to addresses in Australia only
Trackable delivery to most P. O. Boxes and private residences in Australia within 4-5 days based on the distance to a destination following dispatch.

India:

Premium: Delivery to most Indian addresses within 5-6 business days

Rest of the World:

Premium: Countries in the American continent: Trackable delivery to most countries within 4-7 business days

Asia:

Premium: Delivery to most Asian addresses within 5-9 business days

Disclaimer:
All orders received before 5 PM U.K time would start printing from the next business day. So the estimated delivery times start from the next day as well. Orders received after 5 PM U.K time (in our internal systems) on a business day or anytime on the weekend will begin printing the second to next business day. For example, an order placed at 11 AM today will begin printing tomorrow, whereas an order placed at 9 PM tonight will begin printing the day after tomorrow.


Unfortunately, due to several restrictions, we are unable to ship to the following countries:

  1. Afghanistan
  2. American Samoa
  3. Belarus
  4. Brunei Darussalam
  5. Central African Republic
  6. The Democratic Republic of Congo
  7. Eritrea
  8. Guinea-bissau
  9. Iran
  10. Lebanon
  11. Libiya Arab Jamahriya
  12. Somalia
  13. Sudan
  14. Russian Federation
  15. Syrian Arab Republic
  16. Ukraine
  17. Venezuela
Modal Close icon
Modal Close icon