Neuromorphic Computing: Statistical Mechanics & Criticality

 

Last week, I suggested that we were on the verge of something new, and referenced an article by von Bubnoff: A brain built from atomic switches [that] can learn, together with the follow-on article Brain Built on Switches.

The key innovation described in this article was a silver mesh, as shown in the following figure.

This mesh is a “network of microscopically thin intersecting silver wires,” grown via a combination of electrochemical and chemical processes, using a process invented by Adam Stieg, an associate director at the California NanoSystems Institute, and by Jim Gimzewski, professor of chemistry at UCLA. Stieg described the network’s chaotic structure as resembling “a highly interconnected plate of noodles.” They are testing a device built around a 2mm x 2mm mesh of this material.

This mesh has some very interesting properties, including:

1. High synaptic density – approximately 1 billion artificial synapses per square centimeter,
2. High energy efficiency – very low power consumption, and
3. Ability to perform simple learning and logic operations, together with noise reduction.

These abilities, on their own, are nice. They’re even attention-getting. However, they just set the stage for the interesting part of the story.

The device exhibits self-organized criticality.

As described by graduate student Audrius Avizienis, who with Henry Sillin (another graduate student) was analyzing the data,

“That was really jaw-dropping; [it was] the first [time] we pulled a power law out of this.”

Time constraints prevent me from writing more today.

We’ll pick up next week, with a discussion of what the power law is, what self-organized criticality is and what it means for computing, and – perhaps most important – how you can learn about these topics.

The hint is: this is more statistical mechanics. It’s fairly advanced, but still accessible.

One more reason to push on through: there’s empirical evidence that the brain has its own self-organized criticality; that the processes that we’d observe in this silver mesh system would be very analogous to brain-based computing.

This means that if we really want to develop brain-based computing, we’ll need to work with this kind of physics.

Naturally, we can’t cover these topics (in depth) within a single blog post. This is a book-length subject area. However, we can perhaps envision what it is that we’d need to learn if we wanted to work with this kind of system, particularly at a more-than-superficial level.

 

 

Live free or die, my friend –

AJ Maren

Live free or die: Death is not the worst of evils.
Attr. to Gen. John Stark, American Revolutionary War

 

 

References – Neuromorphic Computing

 

• von Bubnoff, A. Artificial synapses could lead to brainier super-efficient computers, Wired (Oct, 4, 2017). online article.
• von Bubnoff, A. A brain built from atomic switches can learn, Quanta (Sept. 20, 2017). online article.
• Gomes, L. Neuromorphic Chips Are Destined for Deep Learning—or Obscurity, IEEE Spectrum (29 May 2017), online article.
• Li, W.Y., Ovchinnikov, I.V., Chen, H.L., Wang, Z., Lee, A., Lee, H.C., Cepeda, C., Schwartz, R.N., Meier, K., and Wang, K.L., A neuronal dynamics study on a neuromorphic chip, arXiv 1703:03560 (2017). pdf.

 

 

Some Useful Background Reading on Statistical Mechanics

 

• Hermann, C. Statistical Physics – Including Applications to Condensed Matter, in Course Materials for Chemistry 480B – Physical Chemistry (New York: Springer Science+Business Media), 2005. pdf. Very well-written, however, for someone who is NOT a physicist or physical chemist, the approach may be too obscure.
• Maren, A.J. Statistical Thermodynamics: Basic Theory and Equations, THM TR2013-001(ajm) (Dec., 2013) Statistical Thermodynamics: Basic Theory and Equations.
• Salzman, R. Notes on Statistical Thermodynamics – Partition Functions, in Course Materials for Chemistry 480B – Physical Chemistry, 2004. Statistical Mechanics (chapter). Online book chapter. This is one of the best online resources for statistical mechanics; I’ve found it to be very useful and lucid.
• Tong, D. Chapter 1: Fundamentals of Statistical Mechanics, in Lectures on Statistical Physics (University of Cambridge Part II Mathematical Tripos), Preprint (2011). pdf.

 

 

Previous Related Posts

• Third Stage Boost: Statistical Mechanics and Neuromorphic Computing
• 2025 and Beyond
• Deep Learning: The Fast Evolution of Artificial Intelligence
• Neg-Log-Sum-Exponent-Neg-Energy – That’s the Easy Part!
• Seven Essential Machine Learning Equations: A Cribsheet (Really, the Précis)
• Statistical Mechanics of Machine Learning blogpost – the great “St. Crispin’s Day” introduction of statistical mechanics and machine learning.

 

 

Return to:

• Dr. A.J.’s Home Page
• Blog Page