Neuromorphic systems comprised of self-assembled nanowires exhibit a range of neural-like dynamic... more Neuromorphic systems comprised of self-assembled nanowires exhibit a range of neural-like dynamics arising from the interplay of their synapse-like electrical junctions and their complex network topology. Additionally, various information processing tasks have been demonstrated with neuromorphic nanowire networks. Here, we investigate the dynamics of how these unique systems process information through information-theoretic metrics. In particular, Transfer Entropy (TE) and Active Information Storage (AIS) are employed to investigate dynamical information flow and short-term memory in nanowire networks. In addition to finding that the topologically central parts of networks contribute the most to the information flow, our results also reveal TE and AIS are maximized when the networks transitions from a quiescent to an active state. The performance of neuromorphic networks in memory and learning tasks is demonstrated to be dependent on their internal dynamical states as well as topological structure. Optimal performance is found when these networks are pre-initialised to the transition state where TE and AIS are maximal. Furthermore, an optimal range of information processing resources (i.e. connectivity density) is identified for performance. Overall, our results demonstrate information dynamics is a valuable tool to study and benchmark neuromorphic systems.
Nanowire networks (NWNs) represent a unique hardware platform for neuromorphic information proces... more Nanowire networks (NWNs) represent a unique hardware platform for neuromorphic information processing. In addition to exhibiting synapse-like resistive switching memory at their cross-point junctions, their self-assembly confers a neural network-like topology to their electrical circuitry, something that is impossible to achieve through conventional top-down fabrication approaches. In addition to their low power requirements, cost effectiveness and efficient interconnects, neuromorphic NWNs are also fault-tolerant and self-healing. These highly attractive properties can be largely attributed to their complex network connectivity, which enables a rich repertoire of adaptive nonlinear dynamics, including edge-of-chaos criticality. Here, we show how the adaptive dynamics intrinsic to neuromorphic NWNs can be harnessed to achieve transfer learning. We demonstrate this through simulations of a reservoir computing implementation in which NWNs perform the well-known benchmarking task of Mackey-Glass (MG) signal forecasting. First we show how NWNs can predict MG signals with arbitrary degrees of unpredictability (i.e. chaos). We then show that NWNs pre-exposed to a MG signal perform better in forecasting than NWNs without prior experience of an MG signal. This type of transfer learning is enabled by the network's collective memory of previous states. Overall, their adaptive signal processing capabilities make neuromorphic NWNs promising candidates for emerging realtime applications in IoT devices in particular, at the far edge.
Biological neural networks, unlike artificial neural networks (ANNs), can process information fro... more Biological neural networks, unlike artificial neural networks (ANNs), can process information from data that is inherently noisy, unstructured, sparse and dynamic. How is this possible? And how can we replicate this? A crucial clue comes from neuroscience: the brain is a complex physical system and the network topology of its neural circuitry is a determinant of its emergent collective properties. Indeed, as the brain's neural network is already entrained in its physical hardware, it clearly does not require an ANN software add-on to learn from natural data. Self-assembled nanowire networks with memristive junctions represent arguably the closest hardware architecture to real biological neural networks and are thus uniquely placed to demonstrate genuinely neuromorphic information processing. Here, we present preliminary results on polymer-coated silver nanowire networks. Their neuromorphic architecture (densely structured network topology, memristive switch junctions and efficient interconnect) gives rise to a rich repertoire of collective nonlinear dynamics manifested through adaptive current transport pathways. The potential for associative learning is demonstrated in a test protocol in which a nanowire network is stimulated by multiple electrodes mapped to different spatial patterns. The capacity to process information in the temporal domain is demonstrated via simulations of a reservoir computing implementation in which nanowire networks are shown to perform tasks such as time series prediction and handwritten digit recognition. Overall, their unique properties and neuromorphic information processing capabilities make nanowire networks promising candidates for emerging applications in cognitive devices in particular, at the edge.
We present simulations based on a model of selfassembled nanowire networks with memristive juncti... more We present simulations based on a model of selfassembled nanowire networks with memristive junctions and neural network-like topology. We analyze the dynamical voltage distribution in response to an applied bias and explain the network conductance fluctuations observed in our previous experimental studies. We then demonstrate the potential of neuromorphic nanowire networks as a physical reservoir by performing benchmark reservoir computing tasks. The tasks include sine wave nonlinear transformation, sine wave autogeneration and forecasting the Mackey-Glass chaotic time series.
The brain's efficient information processing is enabled by the interplay between its neurosynapti... more The brain's efficient information processing is enabled by the interplay between its neurosynaptic elements and complex network structure. This work reports on the neuromorphic dynamics of nanowire networks (NWNs), a unique brain-inspired system with synapse-like memristive junctions embedded within a recurrent neural network-like structure. Simulation and experiment elucidate how collective memristive switching gives rise to long-range transport pathways, drastically altering the network's global state via a discontinuous phase transition. The spatio-temporal properties of switching dynamics are found to be consistent with avalanches displaying power-law size and lifetime distributions, with exponents obeying the crackling noise relationship, thus satisfying criteria for criticality, as observed in cortical neuronal cultures. Furthermore, NWNs adaptively respond to time varying stimuli, exhibiting diverse dynamics tunable from order to chaos. Dynamical states at the edge-of-chaos are found to optimise information processing for increasingly complex learning tasks. Overall, these results reveal a rich repertoire of emergent, collective neural-like dynamics in NWNs, thus demonstrating the potential for a neuromorphic advantage in information processing.
Nanowire Networks (NWNs) belong to an emerging class of neuromorphic systems that exploit the uni... more Nanowire Networks (NWNs) belong to an emerging class of neuromorphic systems that exploit the unique physical properties of nanostructured materials. In addition to their neural network-like physical structure, NWNs also exhibit resistive memory switching in response to electrical inputs due to synapse-like changes in conductance at nanowire-nanowire cross-point junctions. Previous studies have demonstrated how the neuromorphic dynamics generated by NWNs can be harnessed for temporal learning tasks. This study extends these findings further by demonstrating online learning from spatiotemporal dynamical features using image classification and sequence memory recall tasks implemented on an NWN device. Applied to the MNIST handwritten digit classification task, online dynamical learning with the NWN device achieves an overall accuracy of 93.4%. Additionally, we find a correlation between the classification accuracy of individual digit classes and mutual information. The sequence memory...
Nanowire networks (NWNs) mimic the brain’s neurosynaptic connectivity and emergent dynamics. Cons... more Nanowire networks (NWNs) mimic the brain’s neurosynaptic connectivity and emergent dynamics. Consequently, NWNs may also emulate the synaptic processes that enable higher-order cognitive functions such as learning and memory. A quintessential cognitive task used to measure human working memory is the n -back task. In this study, task variations inspired by the n -back task are implemented in a NWN device, and external feedback is applied to emulate brain-like supervised and reinforcement learning. NWNs are found to retain information in working memory to at least n = 7 steps back, remarkably similar to the originally proposed “seven plus or minus two” rule for human subjects. Simulations elucidate how synapse-like NWN junction plasticity depends on previous synaptic modifications, analogous to “synaptic metaplasticity” in the brain, and how memory is consolidated via strengthening and pruning of synaptic conductance pathways.
2020 International Joint Conference on Neural Networks (IJCNN), 2020
We present simulations based on a model of self– assembled nanowire networks with memristive junc... more We present simulations based on a model of self– assembled nanowire networks with memristive junctions and neural network–like topology. We analyze the dynamical voltage distribution in response to an applied bias and explain the network conductance fluctuations observed in our previous experimental studies. We then demonstrate the potential of neuromorphic nanowire networks as a physical reservoir by performing benchmark reservoir computing tasks. The tasks include sine wave nonlinear transformation, sine wave auto– generation and forecasting the Mackey–Glass chaotic time series.
2020 IEEE International Symposium on Circuits and Systems (ISCAS), 2020
Biological neural networks, unlike artificial neural networks (ANNs), can process information fro... more Biological neural networks, unlike artificial neural networks (ANNs), can process information from data that is inherently noisy, unstructured, sparse and dynamic. How is this possible? And how can we replicate this? A crucial clue comes from neuroscience: the brain is a complex physical system and the network topology of its neural circuitry is a determinant of its emergent collective properties. Indeed, as the brain's neural network is already entrained in its physical hardware, it clearly does not require an ANN software add-on to learn from natural data. Self-assembled nanowire networks with memristive junctions represent arguably the closest hardware architecture to real biological neural networks and are thus uniquely placed to demonstrate genuinely neuromorphic information processing. Here, we present preliminary results on polymer-coated silver nanowire networks. Their neuromorphic architecture (densely structured network topology, memristive switch junctions and efficie...
Neuromorphic systems comprised of self-assembled nanowires exhibit a range of neural-like dynamic... more Neuromorphic systems comprised of self-assembled nanowires exhibit a range of neural-like dynamics arising from the interplay of their synapse-like electrical junctions and their complex network topology. Additionally, various information processing tasks have been demonstrated with neuromorphic nanowire networks. Here, we investigate the dynamics of how these unique systems process information through information-theoretic metrics. In particular, Transfer Entropy (TE) and Active Information Storage (AIS) are employed to investigate dynamical information flow and short-term memory in nanowire networks. In addition to finding that the topologically central parts of networks contribute the most to the information flow, our results also reveal TE and AIS are maximized when the networks transitions from a quiescent to an active state. The performance of neuromorphic networks in memory and learning tasks is demonstrated to be dependent on their internal dynamical states as well as topolo...
Neuromorphic Nanowire Networks (NWNs) are a novel class of information processing hardware device... more Neuromorphic Nanowire Networks (NWNs) are a novel class of information processing hardware devices, combining the advantage of memristive cross-point junctions and neural-like complex network topology. In addition to their low operating power, NWNs are also easy to fabricate with bottom-up self-assembly. Here, we implement the MNIST handwritten digit classification task on simulated NWNs to demonstrate their ability to perform complex learning tasks. Using a CNN-inspired kernel method and a reservoir computing framework, our simulation results attain an accuracy of nearly 98%. Moreover, this is achieved using only a fraction of the total MNIST training data and without requiring hardware accelerators. We also investigate the information theoretic metrics of mutual information (MI), transfer entropy (TE) and active information storage (AIS) to analyze the MNIST learning dynamics of NWNs. We find that MI with respect to classes is maximized after network feature extraction and that TE...
Numerous studies suggest critical dynamics may play a role in information processing and task per... more Numerous studies suggest critical dynamics may play a role in information processing and task performance in biological systems. However, studying critical dynamics in these systems can be challenging due to many confounding biological variables that limit access to the physical processes underpinning critical dynamics. Here we offer a perspective on the use of abiotic, neuromorphic nanowire networks as a means to investigate critical dynamics in complex adaptive systems. Neuromorphic nanowire networks are composed of metallic nanowires and possess metal-insulator-metal junctions. These networks self-assemble into a highly interconnected, variable-density structure and exhibit nonlinear electrical switching properties and information processing capabilities. We highlight key dynamical characteristics observed in neuromorphic nanowire networks, including persistent fluctuations in conductivity with power law distributions, hysteresis, chaotic attractor dynamics, and avalanche critica...
Neuromorphic systems comprised of self-assembled nanowires exhibit a range of neural-like dynamic... more Neuromorphic systems comprised of self-assembled nanowires exhibit a range of neural-like dynamics arising from the interplay of their synapse-like electrical junctions and their complex network topology. Additionally, various information processing tasks have been demonstrated with neuromorphic nanowire networks. Here, we investigate the dynamics of how these unique systems process information through information-theoretic metrics. In particular, Transfer Entropy (TE) and Active Information Storage (AIS) are employed to investigate dynamical information flow and short-term memory in nanowire networks. In addition to finding that the topologically central parts of networks contribute the most to the information flow, our results also reveal TE and AIS are maximized when the networks transitions from a quiescent to an active state. The performance of neuromorphic networks in memory and learning tasks is demonstrated to be dependent on their internal dynamical states as well as topological structure. Optimal performance is found when these networks are pre-initialised to the transition state where TE and AIS are maximal. Furthermore, an optimal range of information processing resources (i.e. connectivity density) is identified for performance. Overall, our results demonstrate information dynamics is a valuable tool to study and benchmark neuromorphic systems.
Nanowire networks (NWNs) represent a unique hardware platform for neuromorphic information proces... more Nanowire networks (NWNs) represent a unique hardware platform for neuromorphic information processing. In addition to exhibiting synapse-like resistive switching memory at their cross-point junctions, their self-assembly confers a neural network-like topology to their electrical circuitry, something that is impossible to achieve through conventional top-down fabrication approaches. In addition to their low power requirements, cost effectiveness and efficient interconnects, neuromorphic NWNs are also fault-tolerant and self-healing. These highly attractive properties can be largely attributed to their complex network connectivity, which enables a rich repertoire of adaptive nonlinear dynamics, including edge-of-chaos criticality. Here, we show how the adaptive dynamics intrinsic to neuromorphic NWNs can be harnessed to achieve transfer learning. We demonstrate this through simulations of a reservoir computing implementation in which NWNs perform the well-known benchmarking task of Mackey-Glass (MG) signal forecasting. First we show how NWNs can predict MG signals with arbitrary degrees of unpredictability (i.e. chaos). We then show that NWNs pre-exposed to a MG signal perform better in forecasting than NWNs without prior experience of an MG signal. This type of transfer learning is enabled by the network's collective memory of previous states. Overall, their adaptive signal processing capabilities make neuromorphic NWNs promising candidates for emerging realtime applications in IoT devices in particular, at the far edge.
Biological neural networks, unlike artificial neural networks (ANNs), can process information fro... more Biological neural networks, unlike artificial neural networks (ANNs), can process information from data that is inherently noisy, unstructured, sparse and dynamic. How is this possible? And how can we replicate this? A crucial clue comes from neuroscience: the brain is a complex physical system and the network topology of its neural circuitry is a determinant of its emergent collective properties. Indeed, as the brain's neural network is already entrained in its physical hardware, it clearly does not require an ANN software add-on to learn from natural data. Self-assembled nanowire networks with memristive junctions represent arguably the closest hardware architecture to real biological neural networks and are thus uniquely placed to demonstrate genuinely neuromorphic information processing. Here, we present preliminary results on polymer-coated silver nanowire networks. Their neuromorphic architecture (densely structured network topology, memristive switch junctions and efficient interconnect) gives rise to a rich repertoire of collective nonlinear dynamics manifested through adaptive current transport pathways. The potential for associative learning is demonstrated in a test protocol in which a nanowire network is stimulated by multiple electrodes mapped to different spatial patterns. The capacity to process information in the temporal domain is demonstrated via simulations of a reservoir computing implementation in which nanowire networks are shown to perform tasks such as time series prediction and handwritten digit recognition. Overall, their unique properties and neuromorphic information processing capabilities make nanowire networks promising candidates for emerging applications in cognitive devices in particular, at the edge.
We present simulations based on a model of selfassembled nanowire networks with memristive juncti... more We present simulations based on a model of selfassembled nanowire networks with memristive junctions and neural network-like topology. We analyze the dynamical voltage distribution in response to an applied bias and explain the network conductance fluctuations observed in our previous experimental studies. We then demonstrate the potential of neuromorphic nanowire networks as a physical reservoir by performing benchmark reservoir computing tasks. The tasks include sine wave nonlinear transformation, sine wave autogeneration and forecasting the Mackey-Glass chaotic time series.
The brain's efficient information processing is enabled by the interplay between its neurosynapti... more The brain's efficient information processing is enabled by the interplay between its neurosynaptic elements and complex network structure. This work reports on the neuromorphic dynamics of nanowire networks (NWNs), a unique brain-inspired system with synapse-like memristive junctions embedded within a recurrent neural network-like structure. Simulation and experiment elucidate how collective memristive switching gives rise to long-range transport pathways, drastically altering the network's global state via a discontinuous phase transition. The spatio-temporal properties of switching dynamics are found to be consistent with avalanches displaying power-law size and lifetime distributions, with exponents obeying the crackling noise relationship, thus satisfying criteria for criticality, as observed in cortical neuronal cultures. Furthermore, NWNs adaptively respond to time varying stimuli, exhibiting diverse dynamics tunable from order to chaos. Dynamical states at the edge-of-chaos are found to optimise information processing for increasingly complex learning tasks. Overall, these results reveal a rich repertoire of emergent, collective neural-like dynamics in NWNs, thus demonstrating the potential for a neuromorphic advantage in information processing.
Nanowire Networks (NWNs) belong to an emerging class of neuromorphic systems that exploit the uni... more Nanowire Networks (NWNs) belong to an emerging class of neuromorphic systems that exploit the unique physical properties of nanostructured materials. In addition to their neural network-like physical structure, NWNs also exhibit resistive memory switching in response to electrical inputs due to synapse-like changes in conductance at nanowire-nanowire cross-point junctions. Previous studies have demonstrated how the neuromorphic dynamics generated by NWNs can be harnessed for temporal learning tasks. This study extends these findings further by demonstrating online learning from spatiotemporal dynamical features using image classification and sequence memory recall tasks implemented on an NWN device. Applied to the MNIST handwritten digit classification task, online dynamical learning with the NWN device achieves an overall accuracy of 93.4%. Additionally, we find a correlation between the classification accuracy of individual digit classes and mutual information. The sequence memory...
Nanowire networks (NWNs) mimic the brain’s neurosynaptic connectivity and emergent dynamics. Cons... more Nanowire networks (NWNs) mimic the brain’s neurosynaptic connectivity and emergent dynamics. Consequently, NWNs may also emulate the synaptic processes that enable higher-order cognitive functions such as learning and memory. A quintessential cognitive task used to measure human working memory is the n -back task. In this study, task variations inspired by the n -back task are implemented in a NWN device, and external feedback is applied to emulate brain-like supervised and reinforcement learning. NWNs are found to retain information in working memory to at least n = 7 steps back, remarkably similar to the originally proposed “seven plus or minus two” rule for human subjects. Simulations elucidate how synapse-like NWN junction plasticity depends on previous synaptic modifications, analogous to “synaptic metaplasticity” in the brain, and how memory is consolidated via strengthening and pruning of synaptic conductance pathways.
2020 International Joint Conference on Neural Networks (IJCNN), 2020
We present simulations based on a model of self– assembled nanowire networks with memristive junc... more We present simulations based on a model of self– assembled nanowire networks with memristive junctions and neural network–like topology. We analyze the dynamical voltage distribution in response to an applied bias and explain the network conductance fluctuations observed in our previous experimental studies. We then demonstrate the potential of neuromorphic nanowire networks as a physical reservoir by performing benchmark reservoir computing tasks. The tasks include sine wave nonlinear transformation, sine wave auto– generation and forecasting the Mackey–Glass chaotic time series.
2020 IEEE International Symposium on Circuits and Systems (ISCAS), 2020
Biological neural networks, unlike artificial neural networks (ANNs), can process information fro... more Biological neural networks, unlike artificial neural networks (ANNs), can process information from data that is inherently noisy, unstructured, sparse and dynamic. How is this possible? And how can we replicate this? A crucial clue comes from neuroscience: the brain is a complex physical system and the network topology of its neural circuitry is a determinant of its emergent collective properties. Indeed, as the brain's neural network is already entrained in its physical hardware, it clearly does not require an ANN software add-on to learn from natural data. Self-assembled nanowire networks with memristive junctions represent arguably the closest hardware architecture to real biological neural networks and are thus uniquely placed to demonstrate genuinely neuromorphic information processing. Here, we present preliminary results on polymer-coated silver nanowire networks. Their neuromorphic architecture (densely structured network topology, memristive switch junctions and efficie...
Neuromorphic systems comprised of self-assembled nanowires exhibit a range of neural-like dynamic... more Neuromorphic systems comprised of self-assembled nanowires exhibit a range of neural-like dynamics arising from the interplay of their synapse-like electrical junctions and their complex network topology. Additionally, various information processing tasks have been demonstrated with neuromorphic nanowire networks. Here, we investigate the dynamics of how these unique systems process information through information-theoretic metrics. In particular, Transfer Entropy (TE) and Active Information Storage (AIS) are employed to investigate dynamical information flow and short-term memory in nanowire networks. In addition to finding that the topologically central parts of networks contribute the most to the information flow, our results also reveal TE and AIS are maximized when the networks transitions from a quiescent to an active state. The performance of neuromorphic networks in memory and learning tasks is demonstrated to be dependent on their internal dynamical states as well as topolo...
Neuromorphic Nanowire Networks (NWNs) are a novel class of information processing hardware device... more Neuromorphic Nanowire Networks (NWNs) are a novel class of information processing hardware devices, combining the advantage of memristive cross-point junctions and neural-like complex network topology. In addition to their low operating power, NWNs are also easy to fabricate with bottom-up self-assembly. Here, we implement the MNIST handwritten digit classification task on simulated NWNs to demonstrate their ability to perform complex learning tasks. Using a CNN-inspired kernel method and a reservoir computing framework, our simulation results attain an accuracy of nearly 98%. Moreover, this is achieved using only a fraction of the total MNIST training data and without requiring hardware accelerators. We also investigate the information theoretic metrics of mutual information (MI), transfer entropy (TE) and active information storage (AIS) to analyze the MNIST learning dynamics of NWNs. We find that MI with respect to classes is maximized after network feature extraction and that TE...
Numerous studies suggest critical dynamics may play a role in information processing and task per... more Numerous studies suggest critical dynamics may play a role in information processing and task performance in biological systems. However, studying critical dynamics in these systems can be challenging due to many confounding biological variables that limit access to the physical processes underpinning critical dynamics. Here we offer a perspective on the use of abiotic, neuromorphic nanowire networks as a means to investigate critical dynamics in complex adaptive systems. Neuromorphic nanowire networks are composed of metallic nanowires and possess metal-insulator-metal junctions. These networks self-assemble into a highly interconnected, variable-density structure and exhibit nonlinear electrical switching properties and information processing capabilities. We highlight key dynamical characteristics observed in neuromorphic nanowire networks, including persistent fluctuations in conductivity with power law distributions, hysteresis, chaotic attractor dynamics, and avalanche critica...
Uploads
Papers by Alon Loeffler