Information Processing in Neuron with Exponential Distributed Delay

Authors

  • Saket Kumar Choudhary Impel Lab Pvt. Ltd.
  • Sunil Kumar Bharti Central University of Haryana, Mahendragarh – 123031

Keywords:

Artificial Intelligence, Distributed Delay, Fokker-Planck Equation, LIF Model, LSTM, Spiking Activity, Recurrent Neural Network, Steady State Probability Distribution

Abstract

Artificial intelligence (AI) has been become the primary need in nearly all sectors namely engineering, services, banking, finance, defense, space etc [3], [33]. Artificial intelligence in these sectors can be implemented in two ways: (i) hardware level implementation (ii) software level implementation. Both kinds of AI implementation require neuron models which mimic the minimal set of real neuron functionality. To this end, Leaky Integrate-and-Fire (LIF) model is performing as the backbone for both kinds of AI implementation. At hardware level implementation, it’s a variant, called as neuristors, is used at chip level implementation, whereas a number of variants LIF model are used to implement AI at software level.  In this work, the extended LIF model in distributed delay kernel regime is analyzed.  The impact of exponentially distributed delay (EDD) memory kernel on spiking activity and steady state membrane potential distribution (SVD) of LIF neuron is investigated. Fokker-Planck equation associated with the considered model is solved to investigate SVD of the neuron in sub-threshold regime, which results Gaussian distribution. In order to study the information processing, spiking activity of the model is investigated, which is further extended to neuronal rate-code scheme. These finding have been compared with simple LIF model with stochastic input. It is evident that steady state membrane potential distribution of the LIF neuron is invariant due to the presence of EDD. Such kinds of neuron models are useful to implement artificial neural networks. To this end, the proposed model can used to implement recurrent neural networks (RNN) with comparatively more accuracy.  Similarly, this model can also be investigated in term of chip level implementation of AI.

References

Abbott, L. F. and Dayan, P., (2001), “Theoretical Neuroscience: Computational and mathematical modeling of neural systems”, The MIT press, [ISBN-10: 9780262541855].

Arcas, y A AB; Fairhall, A. L. and Bialek, W., (2003) “Computation in a single neuron: Hodgkin and Huxley Revisited,” Neural Computation, Vol. 15, pp. 1715-1749 https://doi.org/10.1162/08997660360675017

Belgrano, A., Malmgren, B. A. and Lindahl, O., (2001), “Application of artificial neural networks (ANN) to primary production time-series data”, Journal of Plankton Research, Vol. 23, Issue: 6, pp. 651-658 https://doi.org/10.1093/plankt/23.6.651

Borst, A. and Teunissen, F. E., (1999), “Information theory and neural coding,” Nature Neuroscience, Vol. 2, No. 11, pp. 947-957 https://doi.org//10.1038/14731

Burkitt, A. N., (2006), “A review of the integrate-and-fire neuron model: I. homogeneous synaptic input,” Biological Cybernatics, Vol. 95 , No.1, pp. 1-19 https://doi.org//10.1007/s00422-006-0068-6

Burkitt, A. N., (2006), “A review of the integrate-and-fire neuron model: II. inhomogeneous synaptic input and network properties, ”Biological Cybernatics, Vol. 95, No. 2, pp. 97-112 https://doi.org/10.1007/s00422-006-0082-8

Choudhary, S. K. and Singh, K., “Spiking activity of a LIF neuron in distributed delay framework,” International Journal of Interactive Multimedia and Artificial Intelligence, Vol. 3, No. 7, pp. 70-76, June 2016 https://doi.org/10.9781/ijimai.2016.3710

Choudhary, S. K., Singh, K and Bharti, S. K., “Variability in spiking pattern of leaky integrate-and-fire neuron due to excitatory and inhibitory potentials”, 2nd International Conference on “Computing for Sustainable Global Development”, pp. 2025-2030, March 2015 [INSPEC Accession Number: 15110012 ].

Choudhary, S. K. and Singh, K., “Temporal information processing and stability analysis of the MHSN neuron model in DDF”, International Journal of Interactive Multimedia and Artificial Intelligence, Vol. 4, No. 2, pp. 40-45, December 2016 https://doi.org/10.9781/ijimai.2016.427

Deco, G., Schumann, B., (1997), “Information transmission and temporal code in central spiking neuron,” Physical Review Letters, Vol. 79, No. 23, pp. 4697-4700 https://doi.org/10.1103/PhysRevLett.79.4697

Dilek, S., Cakir, H. and Aydin, M., (2015), “Applications of artificial intelligence techniques to combating cyber crimes: a review”, International Journal of Artificial Intelligence & Applications, Vol. 6, No. 1, pp. 21-39 https://doi.org/10.5121/ijaia.2015.6012

Ermentrout, G. B. and Terman, D. H., (2010), “Mathematical foundations of neuroscience,” Springer https://doi.org/10.1007/978-0-387-87708-2

Fairhall, A., Brown, E. C. and Barreiro, A. (2012), “Information theoretic approaches to understanding circuit function,” Neurobiology, Vol. 22, pp. 653-659 https://doi.org/10.1016/j.conb.2012.06.005

Frank, T. D., (2005), “Nonlinear fokker-planck equations: fundamentals and applications,” Springer https://doi.org/10.1007/b137680

Gerstner, W. and Naud, R., (2009), “How good are neuron models?” Science, Vol. 326, No. 5951, pp. 379-380 https://doi.org/10.1126/science.1181936

Gerstner, W., Kreiter, A. K., Markram, H. and Herz, A. V., (1997), “Neural code: firing rate and beyond,” Proceedings National Academy of Science, USA, Vol. 94, pp. 12740-12741 [PMCID: PMC34168].

Gerstner, W. and Kistler, W. M., (2002), “Spiking neuron models: single neurons, populations, plasticity,” Cambridge University Press [ISBN-10: 0262514206].

Haykin, S., Principe J. C., Sejnowski, T. J. and McWhirter, J., (2007), “New directions in statistical signal processing: from systems to brains,” The MIT Press [ISBN: 9780262083485].

Higham D. J., (2001), “An algorithmic introduction to numerical simulation of stochastic differential equations,” SIAM Review, Vol. 43, No. 3, pp. 525-546 https://doi.org/10.1137/S0036144500378302

Published

2018-06-18

How to Cite

Choudhary, S. K., & Bharti, S. K. (2018). Information Processing in Neuron with Exponential Distributed Delay. International Journal of Machine Learning and Networked Collaborative Engineering, 2(02), 58–66. Retrieved from https://mlnce.net/index.php/Home/article/view/33