Browsed by
Category: Statistical Mechanics

Interpreting Karl Friston (Round Deux)

Interpreting Karl Friston (Round Deux)

He might be getting a Nobel prize some day. But – no one can understand him. You don’t believe me? Have a quick glance at Scott Alexander’s article, “God Help Us, Let’s Try To Understand Friston On Free Energy”. We’re referring, of course, to Karl Friston. I’ve spent the past three-and-a-half years studying Friston’s approach to free energy, which he treats as the guiding principle in the brain. He has extended the classic variational Bayes treatment (frontier-material in machine learning)…

Read More Read More

Start Here: Statistical Mechanics for Neural Networks and AI

Start Here: Statistical Mechanics for Neural Networks and AI

Your Pathway through the Blog-Maze: What to read, and what order to read things in, if you’re trying to teach yourself the rudiments of statistical mechanics – just enough to get a sense of what’s going on in the REAL deep learning papers. As we all know, there’s two basic realms of deep learning neural networks. There’s the kind that only requires (some, limited) knowledge of backpropagation. That’s first semester undergraduate calculus, and almost everyone coming into this field can…

Read More Read More

Entropy Trumps All (First Computational for the 2-D CVM)

Entropy Trumps All (First Computational for the 2-D CVM)

Computational vs. Analytic Results for the 2-D Cluster Variation Method:   Three lessons learned: first computational results for the 2-D Cluster Variation Method, or CVM. The first-results comparisons between analytic predictions and the actual computational results tell us three things: (1) the analytics are a suggestion, not an actual values-prediction, and the further that we go from zero-values for the two enthalpy parameters, the more that the two diverge, (2) topography is important (VERY important), and (3) entropy rules the…

Read More Read More

Filling Out the Phase Space Boundaries – 2-D CVM

Filling Out the Phase Space Boundaries – 2-D CVM

Configuration Variables Along the Phase Space Boundaries for a 2-D CVM   Last week’s blog showed how we could get x1 for a specific value of epsilon0, by taking the derivative of the free energy and setting it equal to zero. (This works for the special case where epsilon1 is zero, meaning that there is no interaction enthalpy.) Last week, we looked at one case, where epsilon0 = 1.0. This week, we take a range of epsilon0 values and find…

Read More Read More

Obvious, But Useful (Getting the Epsilon-0 Value when the Interaction Enthalpy Is Zero)

Obvious, But Useful (Getting the Epsilon-0 Value when the Interaction Enthalpy Is Zero)

  This Really Is Kind of Obvious, But …   There’s something very interesting that we can do to obtain values for the epsilon0 parameter. Let’s stay with the case where there is no interaction enthalpy. In that case, we want to find the epsilon0 value that corresponds to the x1 value at a given free energy minimum. Or conversely, given an epsilon0 value, can we identify the x1 where the free energy minimum occurs? Turns out that, for this…

Read More Read More

An Interesting Little Thing about the CVM Entropy (with Code)

An Interesting Little Thing about the CVM Entropy (with Code)

The 2-D CVM Entropy and Free Energy Minima when the Interaction Enthalpy Is Zero:   Today, we transition from deriving the equations for the Cluster Variation Method (CVM) entropies (both 1-D and 2-D) to looking at how these entropies fit into the overall context of a free energy equation. Let’s start with entropy. The truly important thing about entropy is that it gives shape and order to the universe. Now, this may seem odd to those of us who’ve grown…

Read More Read More

Object-Oriented for the CVM (Continued), and an Oops!

Object-Oriented for the CVM (Continued), and an Oops!

Why Shifting to the Object-Oriented Coding Approach REALLY IS Important:   Well, I hate having to admit it. Mud on my face; all that. But I made a pretty significant Whoops! back this last winter when I posted a “Verification and Validation” document (hah!) to arXiv.     The Sad Story of My Previous Ineptitude   Well, there’s nothing like hearing about someone else’s screw-up in order to make us feel better about our own life, so here goes. I’d…

Read More Read More

Expressing Total Microstates (Omega) for the 1-D and 2-D CVM – Part 2

Expressing Total Microstates (Omega) for the 1-D and 2-D CVM – Part 2

Showing How the Omega Equation is Obtained for the 1-D and 2-D CVM – Part 2:   The cluster variation method (CVM) lets us characterize a system in terms of local patterns, and not just the numbers of units in on (A) and off (B) states. It works with a more complex entropy term. The natural question is: How do obtain this more complex entropy? This post continues a discussion begun in the last post, on how we actually get…

Read More Read More

Expressing Total Microstates (Omega) for the 1-D Cluster Variation Method – Part 1

Expressing Total Microstates (Omega) for the 1-D Cluster Variation Method – Part 1

The 1-D CVM – A Single Zigzag Chain – Part 1:   The cluster variation method (CVM) lets us characterize a system in terms of local patterns, and not just the numbers of units in on/off states. This is likely to be useful for machine learning and AI applications. Up until now, we’ve not told the story of how we actually compute the CVM entropy from the microstates. We’ll do that starting with this post; it will be a handy…

Read More Read More

Looking at Friends and Neighbors: A Node’s Point-of-View (2-D Cluster Variation Method)

Looking at Friends and Neighbors: A Node’s Point-of-View (2-D Cluster Variation Method)

Looking around from a Node’s Point-of-View:   As we move from procedural code to object-oriented Python for a Cluster Variation Method (CVM) grid, our perspective shifts. We now need to look at the world from a node’s point-of-view. It’s a lot like updating one’s relationship status in Facebook – except that after updating our own (node) status, we need to not only update the values for all of our own configuration variables, but then we need to travel around the…

Read More Read More