Browsed by
Tag: Ising spin glass

Book Chapter: Draft Chapter 7 – The Boltzmann Machine

Book Chapter: Draft Chapter 7 – The Boltzmann Machine

Comments 3 comments

Chapter 7: Energy-Based Neural Networks This is the full chapter draft from the book-in-progress, Statistical Mechanics, Neural Networks, and Artificial Intelligence. This chapter draft covers not only the Hopfield neural network (released as an excerpt last week), but also the Boltzmann machine, in both general and restricted forms. It deals with that form-equals-function connection, based on the energy equation. (However, we postpone the full-fledged learning method to a later chapter.) Get the pdf using the pdf link in the citation…

Read More Read More

Book Excerpt: Chapter 7

Book Excerpt: Chapter 7

Comments 0 Comment

Chapter 7: Energy-Based Neural Networks This is the first time that I’m sharing an excerpt from the book-in-progress, Statistical Mechanics, Neural Networks, and Artificial Intelligence. This excerpt covers the Hopfield neural network only; I’m still revising / editing / adding-to the remaining sections on the (general and restricted) Boltzmann machine. Get the pdf using the pdf link in the citation below: Maren, A.J. (In progress). Chapter 7: Introduction to Energy-Based Neural Networks: The Hopfield Network and the (Restricted) Boltzmann Machine…

Read More Read More

An Interesting Little Thing about the CVM Entropy (with Code)

An Interesting Little Thing about the CVM Entropy (with Code)

Comments 1 comment

The 2-D CVM Entropy and Free Energy Minima when the Interaction Enthalpy Is Zero:   Today, we transition from deriving the equations for the Cluster Variation Method (CVM) entropies (both 1-D and 2-D) to looking at how these entropies fit into the overall context of a free energy equation. Let’s start with entropy. The truly important thing about entropy is that it gives shape and order to the universe. Now, this may seem odd to those of us who’ve grown…

Read More Read More

A “Hidden Layer” Guiding Principle – What We Minimally Need

A “Hidden Layer” Guiding Principle – What We Minimally Need

Comments 6 comments

Putting It Into Practice: If we’re going to move our neural network-type architectures into a new, more powerful realm of AI capability, we need to bust out of the “sausage-making” mentality that has governed them thus far, as we discussed last week. To do this, we need to give our hidden layer(s) something to do besides respond to input stimulus. It’s very realistic that this “something” should be free energy minimization, because that’s one of the strongest principles in the…

Read More Read More

Visualizing Variables with the 2-D Cluster Variation Method

Visualizing Variables with the 2-D Cluster Variation Method

Comments 2 comments

Cluster Variation Method – 2-D Case – Configuration Variables, Entropy and Free Energy Following the previous blog on the 1-D Cluster Variation Method, I illustrate here a micro-ensemble for the 2-D Cluster Variation Method, consisting of the original single zigzag chain of only ten units (see previous post), with three additional layers added, as shown in the following Figure 1. In Figure 1, we again have an equilibrium distribution of fraction variables z(i). Note that, as with the 1-D case,…

Read More Read More

Modeling Trends in Long-Term IT as a Phase Transition

Modeling Trends in Long-Term IT as a Phase Transition

Comments 0 Comment

The most reasonable model for our faster-than-exponential growth in long-term IT trends is that of a phase transition. At a second-order phase transition, the heat capacity becomes discontinuous. The heat capacity image is provided courtesy of a wikipedia site on heat capacity transition(s). L. Witthauer and M. Diertele present a number of excellent computations in graphical form in their paper The Phase Transition of the 2D-Ising Model. There is another interesting article by B. Derrida & D. Stauffer in Europhysics…

Read More Read More

Gibbs Free Energy, Belief Propagation, and Markov Random Fields

Gibbs Free Energy, Belief Propagation, and Markov Random Fields

Comments 0 Comment

Correspondence Between Free Energy, Belief Propagation, and Markov Random Field Models As a slight digression from previous posts – re-reading the paper by Yedidia et al. on this morning on Understanding Belief Propagation and its Generalizations – which explains the close connection between Belief Propagation (BP) methods and the Bethe approximation (a more generalized version of the simple bistate Ising model that I’ve been using) in statistical thermodynamics. The important point that Yedidia et al. make is that their work…

Read More Read More

What Makes a Metastable State Happen?

What Makes a Metastable State Happen?

Comments 0 Comment

Metastable States – the Meltdown Precursors I’ve just read a recent column by JL, one of the editors from Taipan Daily. He states, in his column “There Will Be Blood in Europe”: Stepping back a bit: What is so frightening right now, not just in Europe but China and America and Japan too, is the presence of fraud-fueled “Lehman 2.0” catalysts threatening to explode. One could say that the 2008 financial crisis was the mother of all wake-up calls. But…

Read More Read More

"What is X?" – Modeling the Meltdown

"What is X?" – Modeling the Meltdown

Comments 0 Comment

“What is X?” – Modeling the 2008-2009 Financial Systems Meltdown We’re about to start a detailed walkthrough of applying a “simple” statistical thermodynamic model to the Wall Street players in the 2007-2009 timeframe. The two kinds of information that I’ll be joining together for this will be a description of Wall Street dynamics, based largely on Chasing Goldman Sachs (see previous blogposts for link), and the two-state Ising thermodynamic model that I’ve been presenting over the past several posts. The…

Read More Read More

Modeling Nonlinear Phenomena

Modeling Nonlinear Phenomena

Comments 0 Comment

Modeling Nonlinear Phenomena – What is “X”? Many of us grew up hating word problems in algebra. (Some of us found them interesting, sometimes easy, and sometimes fun. We were the minority.) For most of us, even if we understood the mathematical formulas, there was a big “gap” in our understanding and intuition when it came to applying the formulas to some real-world situation. In the problem, we’d be given a set of statements, and then told to find “something.”…

Read More Read More