Use of a Hysteresis Loop Activation Function to Enable an Analog Perceptron to Gain Memory
Science Journal of Circuits, Systems and Signal Processing
Volume 7, Issue 2, June 2018, Pages: 68-73
Received: May 17, 2018;
Accepted: Jun. 14, 2018;
Published: Jul. 10, 2018
Views 493 Downloads 52
William Brickner, Department of Electrical and Computer Engineering, Florida Polytechnic University, Lakeland, USA
Muhammad Sana Ullah, Department of Electrical and Computer Engineering, Florida Polytechnic University, Lakeland, USA
With the advent of memristors, analog artificial neural networks are closer than ever. Neural computing is growing as a topic of research. In the context of analog artificial neural networks, the purpose of this research is to verify that a perceptron could gain a discrete memory from implementing a hysteresis loop in the activation function. The discrete memory is represented by the difference path of the hysteresis activation function that took from logic 1 to logic 0. To write to the memory, the input to the hysteresis loop would have to exceed threshold. To read the stored value, the input would have to be between the thresholds of the hysteresis function. In order to verify the perceptron’s memory, a network with manually chosen weights is selected which acts as a shift register. The components of this network are assembled in a circuit simulation program. Functionally, the network receives two inputs: a data signal and an enable signal. The output of the network is a time-shifted version of previous input signals. A system whose output is a time-shifted version of the previous inputs is considered to have memory.
Muhammad Sana Ullah,
Use of a Hysteresis Loop Activation Function to Enable an Analog Perceptron to Gain Memory, Science Journal of Circuits, Systems and Signal Processing.
Vol. 7, No. 2,
2018, pp. 68-73.
D. O. Hebb, The organization of Behavior: A Neuropsychological Theory. First Edition, New York, 2012.
K. Anjaneyulu, “Deep Blue beats Kasparov in a rematch,” Resonance, vol. 2, no. 7, pp. 89-90, 1997.
W. Dean, "Computational Complexity Theory (Stanford Encyclopedia of Philosophy)", Plato.stanford.edu, 2017.
D. Nassimi and S. Sahni, "Bitonic Sort on a Mesh-Connected Parallel Computer", IEEE Transactions on Computers, vol. C-27, no. 1, pp. 2-7, 1979.
L. Chua, "Memristor-The missing circuit element," IEEE Transactions on Circuit Theory, vol. 18, no. 5, pp. 507-519, 1971.
D. Strukov, G. Snider, D. Stewart, and R. Williams, “The missing memristor found,” Nature, vol. 453, pp. 80-83, 2008.
H. Kim, M. Sah, C. Yang, S. Cho and L. Chua, “Memristor Emulator for Memristor Circuit Applications,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 59, no. 10, pp. 2422-2431, 2012.
Q. Li, A. Serb, T. Prodromakis, and H. Xu, “A Memristor SPICE Model Accounting for Synaptic Activity Dependence,” PLOS ONE, vol. 10, no. 3, pp. 1-12, 2015.
M. Kumar, “Memristor - Why Do We Have to Know About It?” IETE Technical Review, vol. 26, no. 1, pp. 1-6, 2009.
R. Williams, "How We Found The Missing Memristor", IEEE Spectrum, vol. 45, no. 12, pp. 28-35, 2008.
C. Yakopcic, R. Hasan and T. Taha, “Hybrid crossbar architecture for a memristor based cache,” Microelectronics Journal, vol. 46, no. 11, pp. 1020-1032, 2015.
M. Sah, C. Yang, H. Kim and L. Chua, “A Voltage Mode Memristor Bridge Synaptic Circuit with Memristor Emulators,” Sensors, vol. 12, no. 12, pp. 3587-3604, 2012.
S. Adhikari, H. Kim, R. Budhathoki, C. Yang and L. Chua, “A Circuit-Based Learning Architecture for Multilayer Neural Networks With Memristor Bridge Synapses,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 62, no. 1, pp. 215-223, 2015.
J. Capulong, B. Briggs, S. Bishop, M. Hovish, R. Matyi and N. Cady, “Effect of Crystallinity on Endurance and Switching and Behavior on HfOx-based Resistive Memory Devices,” IEEE International Integrated Reliability Workshop Final Report (IRW), South Lake Tahoe, CA, USA, pp. 22-25, 14-18 October 2012.
Z. Chew and L. Li, “A discrete memristor made of ZnO nanowires synthesized on printed circuit board,” Materials Letters, vol. 91, pp. 298-300, 2013.
N. Duraisamy, N. Muhammad, H. Kim, J. Jo and K. Choi, “Fabrication of TiO2 thin film memristor device using electrohydrodynamic inkjet printing,” Thin Solid Films, vol. 520, no. 15, pp. 5070-5074, 2012.
N. Mou and M. Tabib-Azar, “Photoreduction of Ag+ in Ag/Ag2S/Au memristor,” Applied Surface Science, vol. 340, pp. 138-142, 2015.
M. Hu, H. Li, Y. Chen, Q. Wu, G. Rose and R. Linderman, “Memristor Crossbar-Based Neuromorphic Computing System: A Case Study,” IEEE Transactions on Neural Networks and Learning Systems, vol. 25, no. 10, pp. 1864-1878, 2014.
W. Wang, L. Li, H. Peng, J. Xiao and Y. Yang, “Synchronization control of memristor-based recurrent neural networks with perturbations,” Neural Networks, vol. 53, pp. 8-14, 2014.
S. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” Neural Computation, vol. 9, no. 8, pp. 1735-1780, 1997.
G. Zhou, J. Wu, C. Zhang and Z. Zhou, “Minimal gated unit for recurrent neural networks,” International Journal of Automation and Computing, vol. 13, no. 3, pp. 226-234, 2016.
J. Hopfield, “The effectiveness of analogue ‘neural network’ hardware,” Network: Computation in Neural Systems, vol. 1, no. 1, pp. 27-40, 1990.
O. Law and C. Salama, “GaAs Schmitt trigger memory cell design,” IEEE Journal of Solid-State Circuits, vol. 31, no. 8, pp. 1190-1192, 1996.
D. Dong and J. Hopfield, “Dynamic properties of neural networks with adapting synapses,” Network: Computation in Neural Systems, vol. 3, no. 3, pp. 267-283, 1992.
F. Beaufays, “The neural networks behind Google Voice transcription,” Google Research Blog, 11 August 2015.
W. Li and M. Nordahl, “Transient behavior of cellular automaton rule 110,” Physics Letters A, vol. 166, no. 5-6, pp. 335-339, 1992.
F. Berto and J. Tagliabue, “Cellular Automata-The Stanford Encyclopedia of Philosophy”, Plato.stanford.edu, 2017.
K. Swingler, “Lecture 2: Single Layer Perceptrons,” University of Stirling, Scotland, UK, 2017.