Ten Steps to Keras

Author: Valerio Maggio

PostDoc Data Scientist @ FBK/MPBA

Contacts:

@leriomaggio

vmaggio_at_fbk_dot_eu

Get the Materials

git clone https://github.com/leriomaggio/deep-learning-keras-tensorflow.git
git checkout tags/euroscipy2017

---

Outline (in ten-ish notebooks)

1. Multi-layer Fully Connected Networks (and the backends)
2. Hidden Layers features and Embeddings
3. Convolutional Networks
4. Hyperparameter Tuning
5. Cutsom Layers
6. Deep CNN and Residual Networks
7. Transfer Learning and Fine Tuning
8. Recurrent Neural Networks
9. AutoEncoders
10. Multi-Modal Networks

---

Requirements

This tutorial requires the following packages:

• Python version 3.5

  • Python 3.4 should be fine as well
  • likely Python 2.7 would be also fine, but who knows? :P

• numpy version >= 1.12: http://www.numpy.org/

• scipy version >= 0.19: http://www.scipy.org/

• matplotlib version >= 2.0: http://matplotlib.org/

• pandas version >= 0.19: http://pandas.pydata.org

• scikit-learn version >= 0.18: http://scikit-learn.org

• keras version >= 2.0: http://keras.io

• tensorflow version 1.2: https://www.tensorflow.org

• ipython/jupyter version >= 6.0, with notebook support

(Optional but recommended):

• hdf5 and h5py (required if you use model saving/loading functions in keras)
• NVIDIA cuDNN if you have NVIDIA GPUs on your machines. https://developer.nvidia.com/rdp/cudnn-download

The easiest way to get (most) these is to use an all-in-one installer such as Anaconda from Continuum. These are available for multiple architectures.

---

Python Version

I'm currently running this tutorial with Python 3 on Anaconda

!python --version

Python 3.5.4

---

Setting the Environment

The repository provides a keras-tutorial.yml file to simply re-create the Anaconda Python Environment, using conda [1].

To re-create the virtual environments:

conda env create -f keras-tutorial.yml

A new keras-tutorial conda environment will be created. To activate the environment:

source activate keras-tutorial

[1]: Note: The conda environment creation has been tested on both Linux and OSX platforms. Therefore, hopefully, it should also work on Windows !-)

Notes about Enabling GPU support for Theano and TensorFlow

Prerequisites:

To enable GPU support for theano and tensorflow, it is mandatorily required that NVIDIA Drivers and CuDNN are already installed and configured before hand (having GPU cards physically installed in your hardware configuration is assumed and took for granted!).

Please refer to the official NVIDIA cuDNN documentation for further details.

Theano Configuration

Preamble

• theano package is assumed to be already installed, as it is provided inside the Anaconda Virtual Environment.
• To date, Theano only supports cuDNN 5.1. No support for cuDNN 6 or 7 is still available. Therefore, be sure to download and install the proper version.

Configuring Theano

echo "[global]
device = cuda0
floatX = float32

[lib]
cnmem = 1.0" > ~/.theanorc

TensorFlow Configuration

To date, tensorflow is available in two different packages, namely tensorflow and tensorflow-gpu, whether you want to install the framework with CPU-only or GPU support, respectively.

For this reason, if you want to enable GPU support for tensorflow, please be sure that the keras-tutorial.yml file has been properly modified to include tensorflow-gpu==1.2.1 package (instead of the default tensorflow==1.2.1).

Configure Keras with TensorFlow

By default, Keras is configured with theano as backend.

If you want to use tensorflow instead, these are the simple steps to follow:

1. Create the keras.json (if it does not exist):

touch $HOME/.keras/keras.json

2. Copy the following content into the file:

{
    "epsilon": 1e-07,
    "backend": "tensorflow",
    "floatx": "float32",
    "image_data_format": "channels_last"
}

3. Verify it is properly configured:

>>> import keras
Using TensorFlow backend.

Test if everything is up&running

import numpy
print('numpy:', numpy.__version__)

import scipy
print('scipy:', scipy.__version__)

import matplotlib
print('matplotlib:', matplotlib.__version__)

import IPython
print('iPython:', IPython.__version__)

import sklearn
print('scikit-learn:', sklearn.__version__)

numpy: 1.12.1
scipy: 0.19.1
matplotlib: 2.0.2
iPython: 6.1.0
scikit-learn: 0.19.0

import keras
print('keras: ', keras.__version__)

# optional
import theano
print('Theano: ', theano.__version__)

import tensorflow as tf
print('TensorFlow: ', tf.__version__)

keras:  2.0.8
Theano:  0.9.0
TensorFlow:  1.2.1

If everything worked till down here, you're ready to start!