Merge pull request lisa-lab#101 from lisa-lab/misc_fixes

carriepl · carriepl · commit c1a1155af3c1 · 2015-07-10T14:53:45.000-04:00
Misc fixes
diff --git a/doc/deep.txt b/doc/deep.txt
diff --git a/doc/lenet.txt b/doc/lenet.txt
@@ -401,10 +401,31 @@ to be compatible with our previous MLP implementation.
 
 
 .. note::
-    Note that the term convolution could corresponds to different mathematical operations.
-    1. theano.tensor.nnet.conv2d which is the most common one in almost all of the recent published convolutional models. In this op for each output feature map, all the input feature maps are summed together after being convolved with the filter.
-    2. Original LeNet model: In this work for each output feature map, only subset of input feature maps were selected.
-    3. The convolution used in signal processing: theano.tensor.signal.conv.conv2d which works only on single channel inputs.
+    Note that the term "convolution" could corresponds to different mathematical operations:
+
+    1. `theano.tensor.nnet.conv2d
+       <http://deeplearning.net/software/theano/library/tensor/nnet/conv.html#theano.tensor.nnet.conv.conv2d>`_,
+       which is the most common one in almost all of the recent published
+       convolutional models.
+       In this operation, each output feature map is connected to each
+       input feature map by a different 2D filter, and its value is the sum of
+       the individual convolution of all inputs through the corresponding filter.
+
+    2. The convolution used in the original LeNet model: In this work,
+       each output feature map is only connected to a subset of input
+       feature maps.
+
+    3. The convolution used in signal processing:
+       `theano.tensor.signal.conv.conv2d
+       <http://deeplearning.net/software/theano/library/tensor/signal/conv.html#theano.tensor.signal.conv.conv2d>`_,
+       which works only on single channel inputs.
+
+    Here, we use the first operation, so this models differ slightly
+    from the original LeNet paper. One reason to use 2. would be to
+    reduce the amount of computation needed, but modern hardware makes
+    it as fast to have the full connection pattern. Another reason would
+    be to slightly reduce the number of free parameters, but we have
+    other regularization techniques at our disposal.
 
 
 
diff --git a/doc/logreg.txt b/doc/logreg.txt
@@ -266,7 +266,7 @@ instance we used a batch size of 600.
 
 
 Prediction Using a Trained Model
-+++++++++++++++++++++++++++++++
+++++++++++++++++++++++++++++++++
 
 ``sgd_optimization_mnist`` serialize and pickle the model each time new
 lowest validation error is reached. We can reload this model and predict
diff --git a/doc/lstm.txt b/doc/lstm.txt
@@ -211,21 +211,21 @@ If you use this tutorial, please cite the following papers.
 
 Introduction of the LSTM model:
 
-* `[pdf] <http://deeplearning.cs.cmu.edu/pdfs/Hochreiter97_lstm.pdf>`_ Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), 1735-1780.
+* `[pdf] <http://deeplearning.cs.cmu.edu/pdfs/Hochreiter97_lstm.pdf>`__ Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), 1735-1780.
 
 Addition of the forget gate to the LSTM model:
 
-* `[pdf] <http://www.mitpressjournals.org/doi/pdf/10.1162/089976600300015015>`_ Gers, F. A., Schmidhuber, J., & Cummins, F. (2000). Learning to forget: Continual prediction with LSTM. Neural computation, 12(10), 2451-2471.
+* `[pdf] <http://www.mitpressjournals.org/doi/pdf/10.1162/089976600300015015>`__ Gers, F. A., Schmidhuber, J., & Cummins, F. (2000). Learning to forget: Continual prediction with LSTM. Neural computation, 12(10), 2451-2471.
 
 More recent LSTM paper:
 
-* `[pdf] <http://www.cs.toronto.edu/~graves/preprint.pdf>`_ Graves, Alex. Supervised sequence labelling with recurrent neural networks. Vol. 385. Springer, 2012.
+* `[pdf] <http://www.cs.toronto.edu/~graves/preprint.pdf>`__ Graves, Alex. Supervised sequence labelling with recurrent neural networks. Vol. 385. Springer, 2012.
 
 Papers related to Theano:
 
-* `[pdf] <http://www.iro.umontreal.ca/~lisa/pointeurs/nips2012_deep_workshop_theano_final.pdf>`_ Bastien, Frédéric, Lamblin, Pascal, Pascanu, Razvan, Bergstra, James, Goodfellow, Ian, Bergeron, Arnaud, Bouchard, Nicolas, and Bengio, Yoshua. Theano: new features and speed improvements. NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2012.
+* `[pdf] <http://www.iro.umontreal.ca/~lisa/pointeurs/nips2012_deep_workshop_theano_final.pdf>`__ Bastien, Frédéric, Lamblin, Pascal, Pascanu, Razvan, Bergstra, James, Goodfellow, Ian, Bergeron, Arnaud, Bouchard, Nicolas, and Bengio, Yoshua. Theano: new features and speed improvements. NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2012.
 
-* `[pdf] <http://www.iro.umontreal.ca/~lisa/pointeurs/theano_scipy2010.pdf>`_ Bergstra, James, Breuleux, Olivier, Bastien, Frédéric, Lamblin, Pascal, Pascanu, Razvan, Desjardins, Guillaume, Turian, Joseph, Warde-Farley, David, and Bengio, Yoshua. Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for Scientific Computing Conference (SciPy), June 2010.
+* `[pdf] <http://www.iro.umontreal.ca/~lisa/pointeurs/theano_scipy2010.pdf>`__ Bergstra, James, Breuleux, Olivier, Bastien, Frédéric, Lamblin, Pascal, Pascanu, Razvan, Desjardins, Guillaume, Turian, Joseph, Warde-Farley, David, and Bengio, Yoshua. Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for Scientific Computing Conference (SciPy), June 2010.
 
 Thank you!
 
diff --git a/doc/rnnslu.txt b/doc/rnnslu.txt
@@ -27,15 +27,15 @@ Papers
 
 If you use this tutorial, cite the following papers:
 
-* `[pdf] <http://www.iro.umontreal.ca/~lisa/pointeurs/RNNSpokenLanguage2013.pdf>`_ Grégoire Mesnil, Xiaodong He, Li Deng and Yoshua Bengio. Investigation of Recurrent-Neural-Network Architectures and Learning Methods for Spoken Language Understanding. Interspeech, 2013.
+* `[pdf] <http://www.iro.umontreal.ca/~lisa/pointeurs/RNNSpokenLanguage2013.pdf>`__ Grégoire Mesnil, Xiaodong He, Li Deng and Yoshua Bengio. Investigation of Recurrent-Neural-Network Architectures and Learning Methods for Spoken Language Understanding. Interspeech, 2013.
 
-* `[pdf] <http://research.microsoft.com/en-us/people/gokhant/0000019.pdf>`_ Gokhan Tur, Dilek Hakkani-Tur and Larry Heck. What is left to be understood in ATIS?
+* `[pdf] <http://research.microsoft.com/en-us/people/gokhant/0000019.pdf>`__ Gokhan Tur, Dilek Hakkani-Tur and Larry Heck. What is left to be understood in ATIS?
 
-* `[pdf] <http://lia.univ-avignon.fr/fileadmin/documents/Users/Intranet/fich_art/997-Interspeech2007.pdf>`_ Christian Raymond and Giuseppe Riccardi. Generative and discriminative algorithms for spoken language understanding. Interspeech, 2007.
+* `[pdf] <http://lia.univ-avignon.fr/fileadmin/documents/Users/Intranet/fich_art/997-Interspeech2007.pdf>`__ Christian Raymond and Giuseppe Riccardi. Generative and discriminative algorithms for spoken language understanding. Interspeech, 2007.
 
-* `[pdf] <http://www.iro.umontreal.ca/~lisa/pointeurs/nips2012_deep_workshop_theano_final.pdf>`_ Bastien, Frédéric, Lamblin, Pascal, Pascanu, Razvan, Bergstra, James, Goodfellow, Ian, Bergeron, Arnaud, Bouchard, Nicolas, and Bengio, Yoshua. Theano: new features and speed improvements. NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2012.
+* `[pdf] <http://www.iro.umontreal.ca/~lisa/pointeurs/nips2012_deep_workshop_theano_final.pdf>`__ Bastien, Frédéric, Lamblin, Pascal, Pascanu, Razvan, Bergstra, James, Goodfellow, Ian, Bergeron, Arnaud, Bouchard, Nicolas, and Bengio, Yoshua. Theano: new features and speed improvements. NIPS Workshop on Deep Learning and Unsupervised Feature Learning, 2012.
 
-* `[pdf] <http://www.iro.umontreal.ca/~lisa/pointeurs/theano_scipy2010.pdf>`_ Bergstra, James, Breuleux, Olivier, Bastien, Frédéric, Lamblin, Pascal, Pascanu, Razvan, Desjardins, Guillaume, Turian, Joseph, Warde-Farley, David, and Bengio, Yoshua. Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for Scientific Computing Conference (SciPy), June 2010.
+* `[pdf] <http://www.iro.umontreal.ca/~lisa/pointeurs/theano_scipy2010.pdf>`__ Bergstra, James, Breuleux, Olivier, Bastien, Frédéric, Lamblin, Pascal, Pascanu, Razvan, Desjardins, Guillaume, Turian, Joseph, Warde-Farley, David, and Bengio, Yoshua. Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for Scientific Computing Conference (SciPy), June 2010.
 
 Thank you!
 
