Skip to content

[MRG] Mlp finishing touches #3939

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 6 commits into from
Closed

[MRG] Mlp finishing touches #3939

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
4da167a
(WIP) Added Multi-layer perceptron (MLP)
IssamLaradji Jun 30, 2013
c846370
refactor mlp optimization methods into _fit_lbfgs and _fit_sgd, add C…
amueller Dec 4, 2014
8b7a733
minor fixes to bench_mnist.
amueller Dec 17, 2014
f0af3aa
FIX partial fit test for MLP
amueller Dec 17, 2014
7f70965
ENH better 'constant' learning rate schedule
ogrisel Dec 18, 2014
c93b7a8
iterate, improve. Nesterov's momentum.
amueller Jun 5, 2015
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 16 additions & 11 deletions benchmarks/bench_mnist.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,19 +9,19 @@
covertype dataset, the feature space is homogenous.

Example of output :

[..]

Classification performance:
===========================
Classifier train-time test-time error-rat
Classifier train-time test-time error-rate
------------------------------------------------------------
Nystroem-SVM 105.07s 0.91s 0.0227
ExtraTrees 48.20s 1.22s 0.0288
RandomForest 47.17s 1.21s 0.0304
SampledRBF-SVM 140.45s 0.84s 0.0486
CART 22.84s 0.16s 0.1214
dummy 0.01s 0.02s 0.8973

MultilayerPerceptron 475.76s 1.31s 0.0201
Nystroem-SVM 218.38s 17.86s 0.0229
ExtraTrees 45.54s 0.52s 0.0288
RandomForest 44.79s 0.32s 0.0304
SampledRBF-SVM 265.64s 19.78s 0.0488
CART 21.13s 0.01s 0.1214
dummy 0.01s 0.01s 0.8973
"""
from __future__ import division, print_function

Expand All @@ -47,6 +47,7 @@
from sklearn.svm import LinearSVC
from sklearn.tree import DecisionTreeClassifier
from sklearn.utils import check_array
from sklearn.neural_network import MLPClassifier

# Memoize the data extraction and memory map the resulting
# train / test splits in readonly mode
Expand Down Expand Up @@ -86,7 +87,11 @@ def load_data(dtype=np.float32, order='F'):
'Nystroem-SVM':
make_pipeline(Nystroem(gamma=0.015, n_components=1000), LinearSVC(C=100)),
'SampledRBF-SVM':
make_pipeline(RBFSampler(gamma=0.015, n_components=1000), LinearSVC(C=100))
make_pipeline(RBFSampler(gamma=0.015, n_components=1000), LinearSVC(C=100)),
'MultilayerPerceptron': MLPClassifier(
hidden_layer_sizes=(100, 100), max_iter=400, alpha=1e-4,
algorithm='sgd', learning_rate_init=0.5, momentum=0.9, verbose=1,
tol=1e-4, random_state=1)
}


Expand Down Expand Up @@ -120,7 +125,7 @@ def load_data(dtype=np.float32, order='F'):
print("%s %d (size=%dMB)" % ("number of train samples:".ljust(25),
X_train.shape[0], int(X_train.nbytes / 1e6)))
print("%s %d (size=%dMB)" % ("number of test samples:".ljust(25),
X_test.shape[0], int(X_test.nbytes / 1e6)))
X_test.shape[0], int(X_test.nbytes / 1e6)))

print()
print("Training Classifiers")
Expand Down
Binary file added doc/images/multilayerperceptron_network.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 2 additions & 0 deletions doc/modules/classes.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1007,6 +1007,8 @@ See the :ref:`metrics` section of the user guide for further details.
:template: class.rst

neural_network.BernoulliRBM
neural_network.MLPClassifier
neural_network.MLPRegressor


.. _calibration_ref:
Expand Down
Loading