Sequence Learning and Generation Using a Spiking Neural

Network

Fuyuko Watanabe and Robert H. Fujii

Computer Systems Department
University of Aizu
Aizu-Wakamatsu City, Fukushima Pref., Japan 965-8580
Corresponding author: fujii@u-aizu.ac.jp

Abstract— A new neural network that uses the ReSuMe desired output sequence in response to a particular input
supervised learning algorithm for generating a desired spike sequence is a relatively new area in the field of neural
output sequence in response to a given input spike sequence networks. ReSuMe [4, 5] is a supervised learning method
is proposed. Possible advantages of the proposed new neural for spiking neural networks that can recognize a particular
network system compared to the Liquid State Machine based input sequence of inter-spike times and can generate a desired
ReSuMe network system include better learning convergence sequence of spike outputs at desired spike times in response
and a smaller neural network size. to an input sequence of spikes. ReSuMe optimizes the
synaptic weights of the learning neuron using the input and
desired output spike time sequence information .
Keywords: spiking neuron, sequence learning, sequence
generation, Liquid State Machine. In this paper the neural network used in the ReSuMe
algorithm [4,5] is modified in order to aid the convergence of
I. INTRODUCTION
the learning algorithm and to simplify the neural network.
An artificial neural network is a circuit made up of
interconnected neurons that mimics the behavior of a
biological neural network present in the brain. Each neuron in II. RESUME ALGORITHM
a neural network responds to the sum total of the inputs that
A. Algorithm Description
arrive at its weighted synapses; once a certain input sum
threshold is reached, the neuron produces an output spike. There are three types of neurons used in the ReSuMe
algorithm: input, learning, and teaching neurons. The set of
input, learning and teaching neurons are defined as follows,
In this paper, a neural network consisting of leaky integrate-
and-fire spiking neurons is considered. Input spikes that arrive Input neuron set: , ,…
at a leaky integrate-and-fire spiking neuron are summed and Learning neuron set: , ,…
the neuron membrane potential (charge) increases with a Teaching neuron set: , ,…
certain time constant; if no inputs occur, the accumulated in
membrane potential decreases (i.e. charge leaks out) with a i
certain time constant. When the membrane potential reaches a
certain threshold value, a pulse-like action potential output is
generated. This action potential is considered to be the output Siin t
Liquid
spike of the neuron. Immediately after an output spike occurs, State
the membrane potential is reset to a resting potential. There is . Machine .
an interval of time right after the neuron fires called the . (LSM) .
“refractory time” during which the neuron does not respond to . .
inputs. The leaky integrate-and-fire spiking neuron responds
to the input spike arrival time information (inter-spike time
coding) in order to produce an output spike at a certain time
[1]. The traditional artificial neuron model on the other hand
responds to the number of input spikes it receives at a certain
time (rate coding). Figure 1. ReSuMe Neural Network

The output spike firing times from these neurons is defined

Recognition of input sequences such as for example a by the following equation:
sequence of symbols “A,B,C,…J” has been researched in the
past using recurrent neural networks [7,11,12] and spiking ∑ , (1)
neural networks [8-10] among others . The generation of a

978-1-4673-2493-9/12/$31.00 ©2012 IEEE 500

where represents the sequence of spikes produced by d
dt
w t =
neuron , t represents the current time, is the firing time,
∞
δis the delta function, and f represents thee ith firing.
in Sd t ad + Wd sd Sin t-sd dsd +
Sin
i is the input spike sequence. S t reepresents the vector
i 0
set of spike sequences produced by the ouutputs of the LSM.
∞
Each LSM output can produce multiplee spikes. The So t ao + 0
Wo so Sin t-so dso (3)
teaching neuron provides the desired outpput spike sequence
times Sid t to the learning neuron. represents the
actual output spike time sequence generated by the learning
in
neuron. Si t , Sdi t , and are used during
d the learning
phase to adjust the synaptic weights of the learning
l neuron . = firing time of a spikee in the desired output spike
sequence. Provided by the teaching neuron.
It is assumed that an output spike produuces a short current = learning neuron output sppike firing time.
pulse defined by equation 2. = input spike arrival time at
a a synapse of a learning neuron.
Isyn e-s⁄τ , (2)
is constant valuue parameter. For excitatory
where τ is a decay constant and s reepresents the time synapses 0 0 , while 0 0 for
difference between the actual time annd the time the inhibitory synapses.
presynaptic spike occurs.
and are synaptic weight window
For the postsynaptic membrane potentiial, ReSuMe [4] functions defined by the folloowing equations,
adopts the leaky integrate-and-fire spikinng neuron model.
Thus, the postsynaptic membrane potentiaal takes some time -sd
to rise and to fall as shown in Figure 2. Such a postsynaptic d d Ad+ ·exp
+A τd+
if sd >0,
W s =
membrane potential model allows a postssynaptic neuron to
produce an output action potential (i.e. an (t) output spike) 0 if sd ≤0, (4)
after the input current pulse occurs; this cann be seen in Figure
2, where the output spike occurs after thee input spike at t3.
The overall postsynaptic membrane poteential response is -so
o o Ao+ ·exp
-A τo+
if so >0,
approximated by the linear sum of all a the individual W s =
responses to incoming presynaptic current pulses.
p 0 if so ≤0, (5)

action
potential and are positive valuues for excitatory synapses and
negative values for inhibitoryy synapses. and are time
Neuron threshold constants.

postsynaptic If it is assumed that | | and Ad+ = Ao+ = A+ ,

potential Eq. (3) can be simplified as follows:
d
w t = Sd t –So t ‧
resting dt
potential ∞
a + 0
W s Sin t-s ds (6)
(t) input
t1 t2 in Eq. (6) is defined ass:
(t) output
t3
-s
+A+ ·exp if s >0,
Figure 2. Leaky Integrate-and –Fire Sppiking Neuron W s = τ+ (7)
Postsynaptic Membrane Poteential 0 if s ≤0,

B. Limitation of the ReSuMe Algorithm

t input signal S in t is passed
As illustrated in Figure 1, the
During learning, the synaptic weightss of the learning
neuron noi are changed according to the folllowing equation: through a liquid state machinne (LSM) [2] where S i n t is
transformed into synaptic outpuut spike signals iin ; the iin
spikes (multiple spikes are typpically generated at each LSM

501
output) are then input into the learning neuron synapses. Each uniformly spaced in time from . Just like the neurons in
synapse of the learning neuron receives an input from only one described earlier, neurons in output a single spike at
of the outputs of the LSM. The synaptic weights of the times ( t s )’= t is delay. . In other words, neuron only
learning neuron are adjusted using the desired output firing outputs a spike in response to the ith input spike which occurs
times provided by the teaching neuron ndi , iin spike times, at time t s .
and the actual Sio t times so that a desired output spike S in t
sequence can be generated.
nd
According to [5], the ReSuMe learning algorithm has
difficulty in accurately generating the desired output sequence
times unless the number of LSM outputs is large (e.g. 300).
The fact that during learning, synaptic weights of the learning
neuron have to be adjusted simultaneously by considering a
large number of spike time values (all the spikes in ( iin ))
can contribute to convergence and output spike sequence
accuracy problems. Furthermore, in order to produce a large
number of outputs ( iin , the LSM must be built with a large
number of inter-connected spiking neurons.

III. MODIFICATIONS TO THE RESUME ALGORITHM

1) New Neural Network
The ReSuMe synapse weight learning algorithm remains
unchanged but it is applied to a new neural network in order
to produce a desired spike output sequence in response to a
given input spike sequence. Figure 3. New Neural Network
The function of the teaching neuron nd is to provide the
The new neural network that does not use a liquid state desired output spike sequence times S d t to the learning
machine is shown in Figure 3. Input spikes S in t are fed into neuron. S o t represents the actual output spike time sequence
, ,…, neurons. A special feature of the generated by the learning neuron. The spikes from the
neurons in is that each neuron only outputs a single spike. and neurons are input into the learning neuron synapses;
Neuron outputs a spike only at time ( t in )’= t iin delay; , , S d t , and S o t spike time data are used during the
delay here represents the response time of the neuron; thus, learning phase.
neuron only outputs a spike in response to the ith input
2) Synaptic Response Current
spike which occurs at time t in .
The synaptic response current is assumed to be of fixed
Having only one spike input per learning neuron synapse amplitude and to have a fixed duration of 3 ms.
from makes the synaptic weight assignment during
learning simpler because the adjustment becomes a localized 3) Spiking Neuron Model
(in time) task around a single input spike time. In the method A leaky integrate-and-fire neuron model was assumed. The
used in [4,5], multiple input spikes are sent out from the LSM equation of the postsynaptic membrane potential Mv was
to each of the learning neuron synapses; as a consequence, defined by the following equation:
adjusting the learning neuron synapse weight can become
difficult to achieve because a synapse weight adjustment s s-p
= 1-exp - H p-s + exp - U s-p , (8)
made for one input spike time can be changed by a τa τa
subsequent input spike time. H s is the Heaviside function and U s is the unit step
function. represents the time difference between the actual
In order to be able to generate any arbitrary desired output time and the time the presynaptic spike occurs. p represents
spike sequence, input spikes times should be distributed the time duration of the presynaptic current pulse. The
uniformly throughout the time window during which input postsynaptic membrane potential Mv rises and falls with a
spikes are received. However, in general the S in t spikes are time constant .
scattered in time with various time gaps and/or clustered; thus,
the S in t spikes alone are not sufficient for the generation of IV. NEW NETWORK PERFORMANCE
an arbitrary desired output sequence, additional input spikes
are needed. 1) Learning Convergence
In order to provide an additional set of input spikes, a Learning convergence of the new system described in
section III was investigated using various values and
supplementary input neurons , ,… are used.
combinations of the a, and parameters in Equations (6)
Neurons in , , … receive spikes that are

502
and (7). Simulations were done using MATLAB [3]. Desired
output spike sequences Sid t were selected for a given set of
input spike sequences Siin t . The learning neuron was trained α= 0 α= 0.01
to produce the desired sequence Sid t . Synaptic weights of
the learning neuron were adjusted using Equations (6) and (7). 6.0 8.0 10.0 6.0 8.0 10.0
Parameters values that were tried in the simulations are as
follows: a = {0, 0.01, 0.05, 0.1}, = {0.5, 1.0, 1.5, 2.0}, and 0.5 o 0.5 o o o
= {6.0, 8.0, 10.0}ms. The postsynaptic membrane 1.0 o o o 1.0 o o
potential was modeled using Equation (9). The input spike 1.5 1.5 o
sequence Siin t consisted of 20 manually selected spike 2.0 2.0
times. For the desired output spike sequence Sid t ,spike α= 0.05 α= 0.1
sequences of length 4, 6 and 8 were manually made. The
supplementary input consisted of uniformly distributed 6.0 8.0 10.0 6.0 8.0 10.0
spikes spaced 3.5 ms. apart from each other. The pulse width
0.5 0.5
of the postsynaptic current was set to 3ms. The refractory
time for a spiking neuron was set to 0.5ms. It was assumed 1.0 o 1.0 o
that all the input spikes in the input spike sequence 1.5 1.5
Siin t would enter the system within 100ms. A network 2.0 2.0
training session consisted of 1000 learning epochs. The
output firing time error was defined using the following Table 2. Learning Convergence, Output Sequence
equation: Length = 6
ef t = tfd - tfo (9)

When the absolute values of the for all output firing

times did not exceed 1ms, it was assumed that learning α= 0 α= 0.01
convergence had been achieved.
6.0 8.0 10.0 6.0 8.0 10.0
1.1) Learning Convergence Results
0.5 o o 0.5 o o o
In Tables 1-3, the combination of parameters that have 1.0 o 1.0 o o o
made learning convergence possible are indicated with a “ ” 1.5 o 1.5 o
mark, those that did not result in learning convergence are
2.0 o 2.0
indicated by a “ ” mark, and those that produced output
S o t spike oscillations, despite satisfying the condition of α= 0.05 α= 0.1
learning convergence several times, are marked with an “o”. 6.0 8.0 10.0 6.0 8.0 10.0
denotes number of spikes in the desired output 0.5 o o o 0.5 o o o
spike sequence. 1.0 o o 1.0 o
No 1.5 1.5
2.0 2.0
α= 0 α= 0.01
Table3. Learning Convergence, Output Sequence
6.0 8.0 10.0 6.0 8.0 10.0
Length = 8
0.5 0.5
1.0 o 1.0 1.2) Single Spike Vs. Multiple Spikes per Learning Neuron
1.5 1.5 Synapse
2.0 2.0
The above learning convergence results were obtained when
α= 0.05 α= 0.1
only one input spike per learning neuron synapse was used. To
6.0 8.0 10.0 6.0 8.0 10.0 investigate the learning convergence performance of the new
network when multiple spikes per learning neuron synapse were
0.5 0.5 allowed, the following experiments were carried out:
1.0 1.0
1) Distribute the spike input time values from one of the input
1.5 1.5
spike sequences used during learning into 7 groups. The
2.0 2.0 distribution of time spike values among the 7 groups was
done randomly. Since an input spike sequence used during
Table 1. Learning Convergence, Output Sequence learning had 20 spikes, the distribution of spikes into 7
Length = 4 groups resulted in 6 groups having 3 spikes and one group
having 2 spikes.

503
2) Use spike firing times of the 7 groups as inputs to 7 resulted in a successful discrimination of all five input
learning neuron synapses. sequences has a D.
3) The components of the neural network for the experiments
The network trained with = 0.01, = 1.5, and =
were: a) seven neurons shown in Figure 1; each 10.0ms could not recognize noisy input sequences. The
neuron received an input from one of the seven groups. network trained with = 0.1, = 1.5, = 6.0ms could not
Thus, the multiple spike inputs from each group was input discriminate the original input signal from the others. In
into 7 synapses of the learning neuron; b) the learning general, recognition and discrimination performance had a
neuron that receives input spikes from the 7 groups through 97 % success rate.
its seven synapses: c) the neurons shown in Figure 3
that provide uniformly spaced (in time) inputs to the
learning neuron; and d) the teaching neuron.
4) Train the learning neuron by using the combination of Parameters
parameters that resulted in learning convergence for an 4 6 8
output spike sequence of length 6, i.e. Table 2 entries that 1.5 10.0 R,D R,D R,D
have “ ” marks. For every such combination of 0 8.0 R,D R,D R,D
parameters, train the network using 1,000 epochs. 2.0
10.0 R,D R,D R,D
Results of the above experiments have shown that when 8.0 R,D R,D R,D
1.5
multiple input spikes per learning neuron synapse were used, 10.0 R,D R,D D
learning convergence could not be achieved regardless of the 0.01 6.0 R,D R,D R,D
parameter combinations used. These results clearly show that 2.0 8.0 R,D R,D R,D
having only one spike input per learning neuron synapse from 10.0 R,D R,D R,D
the neurons makes the synaptic weight adjustment during 1.0 10.0 R,D R,D R,D
learning simpler and as a consequence learning convergence 6.0 R,D R,D R,D
possible. 1.5 8.0 R,D R,D R,D
0.05 10.0 R,D R,D R,D
2) Testing 6.0 R,D R,D R,D
The successfully trained networks were tested for a) 2.0 8.0 R,D R,D R,D
recognition capability using sequences that had noise added to 10.0 R,D R,D R,D
the original input sequences; and b) for discrimination 8.0 R,D R,D R,D
capability using input spike sequences that were quite different 1.0
10.0 R,D R,D R,D
from the original input spike sequences. Noise for each input 6.0 R R,D R,D
spike sequence consisted of shifting five of the original input 1.5 8.0 R,D R,D R,D
0.1
spike times in the range of 0.5ms per spike. For all test cases, 10.0 R,D R,D R,D
the total number of spikes was kept at 20 (the same number of 6.0 R,D R,D R,D
spikes as in the original input spike sequence). All parameter 2.0 8.0 R,D R,D R,D
combinations that had resulted in learning convergence were 10.0 R,D R,D R,D
used for the tests.
Four noise-added input spike sequences were used to test the Table 4. Recognition and Discrimination Performance of
recognition capability of the network. When the absolute Trained Network
values of all the values were 1ms or less, it was assumed
that the network had successfully recognized the noisy input V. CONCLUSIONS
sequence as being similar to the original noise-free input spike
The learning convergence and testing results have shown
sequence.
that the new network can be made to recognize and
Five input spike sequences that were different from the discriminate input spike sequences consisting of 20 spikes
original input spike patterns were used to test the and can generate a desired sequence of spikes for a given
discrimination capability of the network. When the absolute input spike sequence.
values of one or more value(s) exceeded 1ms, it was
assumed that the network had correctly discriminated the input Having only one spike input per learning neuron synapse
spike sequence as being different from the original input spike made the synaptic weight assignment during learning simpler
sequence. because the adjustment became a localized (in time) task
around a single input spike time. Learning convergence
2.1) Testing Results experiments in section IV part 1.2) have shown that having
In the following Tables, the combination of parameters that only one spike input per learning neuron synapse resulted in
resulted in correct recognition of all four noisy input much better learning convergence when compared to a
sequences has an R; the combination of parameters that method that uses multiple input spikes per learning neuron
synapse.

504
 In order to be able to generate any arbitrary desired output REFERENCES
spike sequence, input spikes times should be distributed [1] W. Gerstner and W. Kistler, Spiking Neuron Models Single Neurons,
uniformly throughout the time window during which input Populations, plasticity”, Cambridge University Press, Cambridge,
spikes are received. However, in general the S in t spikes are 2002.
scattered in time with various time gaps and/or clustered. [2] W. Maass, T. Natschlaeger, and H. Markram, “Computational models
for generic cortical microcircuits”, In J. Feng, editor, Computational
Thus, in order to provide an additional set of input spikes, Neuroscience: A Comprehensive Approach, Chapman and Hall/CRC,
supplementary input neurons , , … were used. Boca Raton, 2004, pp. 575-605.
These neurons fired only one spike and provided uniformly [3] MATLAB, The Math Works, Inc, http://www.mathworks.com.
spaced (in time) input spikes to the learning neuron. The [4] F. Ponulak, “ReSuMe - new supervised learning method for Spiking
Neural Networks”, Technical Report, Institute of Control and
supplementary spike inputs were essential in achieving Information Engineering, Poznań University of Technology, 2005,
accurate output spike sequence times. http://d1.cie.put.poznan.pl/~fp/.
[5] F. Ponulak, “Supervised Learning in spiking neural networks with
The number of N in neurons is dependent on the number of ReSuMe method”, Ph. D. thesis, Institute of Control and Information
input spikes in S in t . The number of N s neurons is dependent Engineering, Poznań University of Technology, 2006,
http://d1.cie.put.poznan.pl/~fp/.
on the input spike window and the time spacing between [6] F. Ponulak and A. Kasiński, “Supervised Learning in Spiking Neural
spikes. Thus an overall network size comparison between the Networks with ReSuMe: Sequence Learning, Classification and
newly proposed neural network and the original LSM-based Spike-Shifting”, Institute of Control and Information Engineering,
ReSuMe network is dependent on the number of neurons Poznań University of Technology, 2009.
[7] Y. Konishi and R. H. Fujii, “Incremental learning of temporal
used for N in and in the new network and the size of the sequences using state memory and a resource allocating network”,
LSM. 2004 IEEE International Joint Conference on Neural Networks, vol. 4,
pp. 25-29.
The accuracy of the output spike sequence generated by the [8] H. H. Amin and R. H. Fujii., “Spiking neural network inter-spike time
new network is believed to be good. A direct accuracy based decoding scheme”, IEICE-Transactions on Information and
Systems, vol. E88-D, Issue 8, August 2005, pp. 1893-1902.
comparison with the LSM-based ReSuMe network will be [9] R. H. Fujii and T. Ichishita, “Effect of synaptic weight assignment on
made using the same test data and the same criteria for spiking neuron response”, 6th International Conference on Machine
gauging error. A detailed analysis of the effects of ReSuMe Learning and Applications (ICMLA), Cincinnati, Ohio, USA, 2007,
parameter value variations will also be investigated . pp. 217-222.
[10] R. H. Fujii and K. Oozeki, “Temporal data encoding and sequence
learning with spiking neural networks,” 16th International Conference
on Artificial Neural Networks (ICANN 2006), Athens, Greece,
pp.780-789.
[11] J. L. Elman, “Finding structure in time”, Cognitive Science, vol. 14,
1990, pp. 179-211.
[12] D. L. Wang and B. Yuwono, “Anticipation-based temporal pattern
generation,” IEEE Trans. Sys. Man Cybern., vol. 25, 1995, pp. 615-
628.

505