Readings – Statistical Physics and Neuroscience

Readings – Statistical Physics and Neuroscience

Statistical Mechanics, Entropy, and Information Theoretic Methods Applied to Neuroscience Models

Significant and/or Most Interesting Papers (Personal Choice)

Selections from J. Stat. Mech. Special Issue on Statistical Physics and Neuroscience (Personal Choice)

  • Self-organized criticality in a network of interacting neurons, Cowan JD, Neuman J, van Drongelen W: Self-organized criticality in a network of interacting neurons. J. Stat. Mech. 2013, 04: P04030. (Note: This article was included in my listing of “significant and/or most interesting articles” section above.)
  • MJ Berry II, G Tkačik, J Dubuis, O Marre, and R Azeredo da Silveira, A simple method for estimating the entropy of neural activity. J. Stat. Mech. 2013, 04: P03015. (doi:10.1088/1742-5468/2013/03/P03015). pdf
  • Madhu Advani, Subhaneil Lahiri and Surya Ganguli, Statistical mechanics of complex neural systems and high dimensional data. J. Stat. Mech. 2013, 04: pages unknown. (doi:10.1088/1742-5468/2013/03/P03014). PDF
  • John Barton; and Simona Cocco, Ising models for neural activity inferred via selective cluster expansion: structural and coding properties. J. Stat. Mech. 2013, 04: pages unknown. (doi:10.1088/1742-5468/2013/03/P03002). PDF

Statistical Mechanics of Small Neural Ensembles – Experimental Results and Theory

  • MJ Berry II, G Tkačik, J Dubuis, O Marre, and R Azeredo da Silveira, A simple method for estimating the entropy of neural activity. J Stat Mech, 04: P03015 (2013). (doi:10.1088/1742-5468/2013/03/P03015). pdf
  • G Tkačik, O Marre, D Amodei, E Schneidman, W Bialek, MJ Berry II, Searching for Collective Behavior in a Large Network of Sensory Neurons. PLoS Comput Biol 10 (1): e1003408. doi:10.1371/journal.pcbi.1003408 (2014). (PDF)
  • G Tkačik, T Mora, Ol Marre, Da Amodei, MJ Berry II, and W Bialek, Thermodynamics for a network of neurons: Signatures of criticality. arXiv 1407.5946v1 [q-bio.NC] (22 July 2014). (PDF)
  • G Tkačik, O Marre, T Mora, D Amodei, MJ Berry II, and W Bialek, The simplest maximum entropy model for collective behavior in a neural network. J Stat Mech P03011 (2013)(doi:10.1088/1742-5468/2013/03/P03011); arXiv: 1207.6319v1 [q-bio.NC] (26 Jul 2012. PDF

Free Energy and Information Theory: Neuroscience

  • The free-energy principle: a unified brain theory?, Friston K: The free-energy principle: a unified brain theory? Nature Reviews Neuroscience (February 2010), 11:127-138 (published online 13 January 2010; doi:10.1038/nrn2787)… great reference list.
  • A Free Energy Principle for Biological Systems, Friston, K: A Free Energy Principle for Biological Systems. Entropy 2012, 14, 2100-2121 (doi:10.3390/e14112100)
  • Armin Fuchs, J.A. Scott Kelso, and Hermann Haken: Phase Transitions in the Human Brain: Spatial Mode Dynamics. Int. J. Bifurcation Chaos 02, 917 (1992). (DOI: 10.1142/S0218127492000537) (Citations of this article; no pdf available.)
  • Modeling Phase Transitions in the Brain, Steyn-Ross, Alistair, Steyn-Ross, Moira (Eds.); Modeling Phase Transitions in the Brain Springer Series in Computational Neuroscience, Vol. 4, (2010, XXV). Read Exerpts from Modeling Phase Transitions in the Brain; no full pdf available.
  • Sleigh, J.W., Wilson, M.T., Voss, L.J., Steyn-Ross, D.A., Steyn-Ross, M.L., and Li, X. (2010). A continuum model for the dynamics of the
    phase transition from slow-wave sleep to REM sleep. In D. A. Steyn-Ross & M. Steyn-Ross (Eds), Modeling Phase Transitions in the Brain, 203-221. (New York, USA: Springer). pdf

PPT Presentations & Tutorials

Also: see the list of references in T. Mora’s homepage – he’s looking at thermodynamics of groups of neurons. He’s one of the authors of: 14.G. Tkacik, O. Marre, T. Mora, D. Amodei, M.J. Berry II, W. Bialek, The simplest maximum entropy model for collective behavior in a neural network. J. Stat. Mech. P03011 (2013)

I particularly am reading: Thermodynamics for a network of neurons: Signatures of criticality, Gasper Tkacik, Thierry Mora, Olivier Marre, Dario Amodei, Michael J. Berry II, William Bialek, ArXiv: arXiv:1407.5946v1 [q-bio.NC]; a predecessor for the above article (and 17 pp., 71 refs as opposed to 5 pp., 34 refs).

Nunez, P.L., and Srinivasan, R. (2014). Neocortical dynamics due to axon propagation delays in cortico-cortical fibers: EEG traveling and standing waves with implications for top-down influences on local networks and white matter disease. Brain Research (2014 Jan 13), 1542:138-166. pdf

Nunez, P.L., and Srinivasan, R. (2006). A theoretical basis for standing and traveling brain waves measured with human EEG with implications for an integrated consciousness. Clin Neurophysiol. Nov 2006; 117(11): 2424–2435. pdf

Srinivasan, R., Thorpe, S., & Nunez, P.L. (2013). Top-down influences on local networks: basic theory with experimental implications. Front. Comput. Neurosci. (18 April 2013) | doi: 10.3389/fncom.2013.00029 pdf

Ingber, L., & Nunez, P. (1990). Multiple scales of statistical physics of the neocortex: application to electroencephalography. Mathl. Comput. Modelling, 13 (7), 83-95. abstract & references

Neural Dynamics

Jirsa, V.K. (2009). Neural field dynamics with local and global connectivity and time delay. Phil. Trans. R. Soc. A (28 March 2009) 367 (1891) 1131-1143. pdf (viktor.jirsa@univmed.fr)

… reading this paper, it’s going to take some time to get through the equations … “Our main interest is the study of the stability of the rest state of a cortical architecture with local and global connectivity.” … a really interesting reference list.

Pérez Velázquez, J.L.P., and Galán, R.F. (2013). Information gain in the brain’s resting state: A new perspective on autism. Front. Neuroinform., 24 December 2013 | doi: 10.3389/fninf.2013.00037. pdf

{Extracted from Abstract] Using a stochastic dynamical model of brain dynamics, we were able to resolve not only the deterministic interactions between brain regions, i.e., the brain’s functional connectivity, but also the stochastic inputs to the brain in the resting state; an important component of large-scale neural dynamics that no other method can resolve to date. We then computed the Kullback-Leibler (KLD) divergence, also known as information gain or relative entropy, between the stochastic inputs and the brain activity at different locations (outputs) in children with ASD compared to controls. The divergence between the input noise and the brain’s ongoing activity extracted from our stochastic model was significantly higher in autistic relative to non-autistic children. This suggests that brains of subjects with autism create more information at rest.

Information Theory and Brain

  • Information theory of adaptation, Sharpee, T.O., Calhoun, A.J., and Chalasani, S.H., Information theory of adaptation in neurons, behavior, and mood. Current Opinion in Neurobiology 2014, 25:47–53

Dynamic Criticality in Brain (Personal Choice)

  • Droste, F., Do, A.-L., & Gross, T. (2012). Self-organized criticality in a network of interacting neurons. arxiv: 1203:4942v2 [nlin.AO] 17 Sept. 2012 , author contact: felix.droste@bccn-berlin.de
  • Droste, F., Do, A.-L., & Gross, T. (2013). Analytical investigation of self-organized criticality in neural networks. J R Soc Interface (Jan 6, 2013) 10(78): 20120558. online. See also the interesting review articles linked to this online report. Very useful & interesting references; online links. Usefully shows evolution of thought (refs. 16-23) of self-organized criticality (SOC) in neural networks. Important precursor paper on theory-end for memristor devices.

    Dynamical criticality has been shown to enhance information processing in dynamical systems, and there is evidence for self-organized criticality in neural networks. A plausible mechanism for such self-organization is activity-dependent synaptic plasticity. Here, we model neurons as discrete-state nodes on an adaptive network following stochastic dynamics. At a threshold connectivity, this system undergoes a dynamical phase transition at which persistent activity sets in. In a low-dimensional representation of the macroscopic dynamics, this corresponds to a transcritical bifurcation. We show analytically that adding activity-dependent rewiring rules, inspired by homeostatic plasticity, leads to the emergence of an attractive steady state at criticality and present numerical evidence for the system’s evolution to such a state.

  • Moreau, L., Sontag, E. (2003). Balancing at the border of instability. Phys. Rev. E. 68, 020901. Author address: Moreau: Systems-EESA, Ghent University, Technologiepark 914, 9052 Zwijnaarde, Belgium. Sontag: Department of Mathematics, Rutgers, The State University of New York. pdf
  • Kello, C.T., Kerster, B., & Johnson, E. (date unknown). Critical Branching Neural Computation, Neural Avalanches, and 1/f Scaling.
    Author addresses & emails: Christopher T. Kello (ckello@ucmerced.edu); Bryan Kerster (bkerster@ucmerced.edu);
    Eric Johnson (ejohnson5@ucmerced.edu), Cognitive and Information Sciences, 5200 North Lake Rd., Merced, CA 95343 USA. pdf.
  • Rodny, J.J., & Kello, C.T. (2014). Learning and Variability in Spiking Neural Networks. Proc. 37th Annual Meeting of the Cognitive Science Society. Author emails: Jeffrey J. Rodny (jrodny@ucmerced.edu), Christopher T. Kello (ckello@ucmerced.edu)pdf.
  • Rybarsch M, Bornholdt S (2014). Avalanches in Self-Organized Critical Neural Networks: A Minimal Model for the Neural SOC Universality Class. PLoS ONE 9(4): e93090. (April 17, 2014) DOI: 10.1371/journal.pone.0093090 online

Additional papers

  • Thibeault, C.M. (2014). A role for neuromorphic processors in therapeutic nervous system stimulation, Front. Syst. Neurosci., 8 (187). (07 October 2014 | doi: 10.3389/fnsys.2014.00187) online
  • Srinivasa N, Jiang Q (2013). Stable learning of functional maps in self-organizing spiking neural networks with continuous synaptic plasticity. Front Comput Neurosci 7 (10). online
  • O’Brien, M.J., Thibeault, C.M., and Srinivasa, N. (2014). A novel analytical characterization for short-term plasticity parameters in spiking neural networks, Front. Comput. Neurosci. 8:148. (19 November 2014 | doi: 10.3389/fncom.2014.00148) online
  • Thibeault, C.M., Harris Jr., F.C., and Srinivasa, N. (2013). Embodied modeling with spiking neural networks For neuromorphic hardware: a simulation study. Proc. of 26th International Conference on Computer Applications in Industry and Engineering (CAINE), (Los Angeles, CA, September 2013), 3-10. Author email: cmthibeault@hrl.com pdf
  • Srinivasa, N. (2013). A Scalable Analog Neuromorphic Learning System. NeuComp2013 Workshop (March 22, 2013, Grenoble, France). PPT presentation
  • Grossberg, S., Pilly, P.K. (2012). How entorhinal grid cells may learn multiple spatial scales from a dorsoventral gradient of cell response rates in a self-organizing map. PLOS Computational Biology. (October 04, 2012 | DOI: 10.1371/journal.pcbi.1002648).
    online
  • Fuhs, M.C., & Touretzky, D.S. (2006). A spin glass model of path integration in rat medial entorhinal cortex. J. Neurosci. 26(16): 4266-4276; doi: 10.1523/JNEUROSCI.4353-05.2006 . Author email: dst@cs.cmu.edu online
  • Edvard I. Moser, E.I., May-Britt Moser, M.-B. (20xx). Grid cells and neural coding in high-end cortices. Neuron http://dx.doi.org/10.1016/j.neuron.2013.09.043. Author email: edvard.moser@ntnu.no online
  • Khosla, D. Chen, Y., and Kim, K. (2014). A neuromorphic system for video object recognition. Front. Comput. Neurosci. | doi: 10.3389/fncom.2014.00147 online
  • Matthew J. Aburn, C.A. Holmes, J.A. Roberts, T.W. Boonstra, and M. Breakspear (2012). Critical fluctuations in cortical models near instability, Front. Physiology 3 (Article 331), 1-17 (August, 2012).
  • Jochen Braun and Maurizio Mattia (September, 2010). Attractors and noise: Twin drivers of decisions and multistability, Neuroimage 52 (3), 740-51 (September, 2010). DOI: 10.1016/j.neuroimage.2009.12.126. Epub 2010 Jan 18.

    • Abstract: Perceptual decisions are made not only during goal-directed behavior such as choice tasks, but also occur spontaneously while multistable stimuli are being viewed. In both contexts, the formation of a perceptual decision is best captured by noisy attractor dynamics. Noise-driven attractor transitions can accommodate a wide range of timescales and a hierarchical arrangement with “nested attractors” harbors even more dynamical possibilities. The attractor framework seems particularly promising for understanding higher-level mental states that combine heterogeneous information from a distributed set of brain areas.
    • Good overview article, discusses: (1) spontaneous activity fluctuations in sensory cortices and across the brain, (2) perceptual decision making, and (3) multistable perception, in other words, the spontaneous reversals of perceptual experience that are often induced by ambiguous sensory situations.
    • No mathematics, lots of good descriptions in English, long and excellent set of references – some historical, mostly clustered between 2007 – 2009.
    • Identifies two landmark studies done by other groups showing equivalence or near-equivalence of (visual or somato-) sensory stimulation with direct neural stimulation of corresponding neural areas; relevant to BCI.

Simulation Software Packages

  • Thibeault, C.M., O’Brien, M.J., and Srinivasa, N. (2014). Analyzing large-scale spiking neural data with HRLAnalysis™. Front. Neuroinform., 8:17. (05 March 2014 | doi: 10.3389/fninf.2014.00017) online

Another Group of Readings – Sejnowski et al.

  • Saremi, S., and Sejnowski, T.J. (2014). On Criticality in High-Dimensional Data. Neural Computation 26, 1329–1339. doi:10.1162/NECO_a_006078:17. pdf

Brain, Chaos, Equilibrium Points

  • Huang, G., zhang, D.-G., Meng, J.-G., & Zhu, X.-Y. (2011). Interactions between two neural populations: A mechanism of chaos and oscillation in neural mass model. Neurocomputing 74, 1026-1034. DOI: 102083-201104-2 (Obtain pdf via googling on title, author, to download.) (Note: model finds two stable equilibrium states.)