Change references to SigmoidalLayer to HiddenLayer

zacstewart · zacstewart · commit a376f41de719 · 2013-12-30T13:52:08.000-05:00
The SigmoidalLayer and TahnLayer classes seem to have been merged into
the HiddenLayer with has an activation function attribute. Several
places in the tutorial still refer to the SigmoidalLayer, so I updated
them to HiddenLayer and took care to specify the T.nnet.sigmoid
activation function where appropriate.
diff --git a/code/mlp.py b/code/mlp.py
@@ -108,7 +108,7 @@ class MLP(object):
     A multilayer perceptron is a feedforward artificial neural network model
     that has one layer or more of hidden units and nonlinear activations.
     Intermediate layers usually have as activation function thanh or the
-    sigmoid function (defined here by a ``SigmoidalLayer`` class)  while the
+    sigmoid function (defined here by a ``HiddenLayer`` class)  while the
     top layer is a softamx layer (defined here by a ``LogisticRegression``
     class).
     """
@@ -136,10 +136,10 @@ def __init__(self, rng, input, n_in, n_hidden, n_out):
 
         """
 
-        # Since we are dealing with a one hidden layer MLP, this will
-        # translate into a TanhLayer connected to the LogisticRegression
-        # layer; this can be replaced by a SigmoidalLayer, or a layer
-        # implementing any other nonlinearity
+        # Since we are dealing with a one hidden layer MLP, this will translate
+        # into a HiddenLayer with a tanh activation function connected to the
+        # LogisticRegression layer; the activation function can be replaced by
+        # sigmoid or any other nonlinear function
         self.hiddenLayer = HiddenLayer(rng=rng, input=input,
                                        n_in=n_in, n_out=n_hidden,
                                        activation=T.tanh)
diff --git a/doc/DBN.txt b/doc/DBN.txt
@@ -189,7 +189,7 @@ the MLP, while ``self.rbm_layers`` will store the RBMs used to pretrain each
 layer of the MLP.
 
 Next step, we construct ``n_layers`` sigmoid layers (we use the
-``SigmoidalLayer`` class introduced in :ref:`mlp`, with the only modification
+``HiddenLayer`` class introduced in :ref:`mlp`, with the only modification
 that we replaced the non-linearity from ``tanh`` to the logistic function
 :math:`s(x) = \frac{1}{1+e^{-x}}`) and ``n_layers`` RBMs, where ``n_layers``
 is the depth of our model.  We link the sigmoid layers such that they form an
diff --git a/doc/SdA.txt b/doc/SdA.txt
@@ -126,7 +126,7 @@ representations of intermediate layers of the MLP.
 ``self.dA_layers`` will store  the denoising autoencoder associated with the layers of the MLP. 
 
 Next step, we construct ``n_layers`` sigmoid layers (we use the
-``SigmoidalLayer`` class introduced in :ref:`mlp`, with the only
+``HiddenLayer`` class introduced in :ref:`mlp`, with the only
 modification that we replaced the non-linearity from ``tanh`` to the
 logistic function :math:`s(x) = \frac{1}{1+e^{-x}}`) and ``n_layers``
 denoising autoencoders, where ``n_layers`` is the depth of our model.
@@ -154,10 +154,11 @@ bias of the encoding part with its corresponding sigmoid layer.
             else:
                 layer_input = self.sigmoid_layers[-1].output
 
-            sigmoid_layer = SigmoidalLayer(rng=rng,
+            sigmoid_layer = HiddenLayer(rng=rng,
                                            input=layer_input,
                                            n_in=input_size, 
-                                           n_out=hidden_layers_sizes[i])
+                                           n_out=hidden_layers_sizes[i],
+                                           activation=T.nnet.sigmoid)
             # add the layer to our list of layers 
             self.sigmoid_layers.append(sigmoid_layer)
 
diff --git a/doc/lenet.txt b/doc/lenet.txt
@@ -498,7 +498,7 @@ instantiate the network as follows.
             image_shape=(batch_size, 20, 12, 12),
             filter_shape=(50, 20, 5, 5), poolsize=(2, 2))
 
-    # the SigmoidalLayer being fully-connected, it operates on 2D matrices of
+    # the HiddenLayer being fully-connected, it operates on 2D matrices of
     # shape (batch_size,num_pixels) (i.e matrix of rasterized images).
     # This will generate a matrix of shape (20, 32 * 4 * 4) = (20, 512)
     layer2_input = layer1.output.flatten(2)
