Browsed by
Category: Free Energy

Interpreting Karl Friston (Round Deux)

Interpreting Karl Friston (Round Deux)

Comments 2 comments

He might be getting a Nobel prize some day. But – no one can understand him. You don’t believe me? Have a quick glance at Scott Alexander’s article, “God Help Us, Let’s Try To Understand Friston On Free Energy”. We’re referring, of course, to Karl Friston. I’ve spent the past three-and-a-half years studying Friston’s approach to free energy, which he treats as the guiding principle in the brain. He has extended the classic variational Bayes treatment (frontier-material in machine learning)…

Read More Read More

Entropy Trumps All (First Computational for the 2-D CVM)

Entropy Trumps All (First Computational for the 2-D CVM)

Comments 0 Comment

Computational vs. Analytic Results for the 2-D Cluster Variation Method:   Three lessons learned: first computational results for the 2-D Cluster Variation Method, or CVM. The first-results comparisons between analytic predictions and the actual computational results tell us three things: (1) the analytics are a suggestion, not an actual values-prediction, and the further that we go from zero-values for the two enthalpy parameters, the more that the two diverge, (2) topography is important (VERY important), and (3) entropy rules the…

Read More Read More

Filling Out the Phase Space Boundaries – 2-D CVM

Filling Out the Phase Space Boundaries – 2-D CVM

Comments 3 comments

Configuration Variables Along the Phase Space Boundaries for a 2-D CVM   Last week’s blog showed how we could get x1 for a specific value of epsilon0, by taking the derivative of the free energy and setting it equal to zero. (This works for the special case where epsilon1 is zero, meaning that there is no interaction enthalpy.) Last week, we looked at one case, where epsilon0 = 1.0. This week, we take a range of epsilon0 values and find…

Read More Read More

Obvious, But Useful (Getting the Epsilon-0 Value when the Interaction Enthalpy Is Zero)

Obvious, But Useful (Getting the Epsilon-0 Value when the Interaction Enthalpy Is Zero)

Comments 1 comment

  This Really Is Kind of Obvious, But …   There’s something very interesting that we can do to obtain values for the epsilon0 parameter. Let’s stay with the case where there is no interaction enthalpy. In that case, we want to find the epsilon0 value that corresponds to the x1 value at a given free energy minimum. Or conversely, given an epsilon0 value, can we identify the x1 where the free energy minimum occurs? Turns out that, for this…

Read More Read More

An Interesting Little Thing about the CVM Entropy (with Code)

An Interesting Little Thing about the CVM Entropy (with Code)

Comments 1 comment

The 2-D CVM Entropy and Free Energy Minima when the Interaction Enthalpy Is Zero:   Today, we transition from deriving the equations for the Cluster Variation Method (CVM) entropies (both 1-D and 2-D) to looking at how these entropies fit into the overall context of a free energy equation. Let’s start with entropy. The truly important thing about entropy is that it gives shape and order to the universe. Now, this may seem odd to those of us who’ve grown…

Read More Read More

What We Really Need to Know about Entropy

What We Really Need to Know about Entropy

Comments 3 comments

There’s This Funny Little “Gotcha” Secret about Entropy: Nobody mentions this secret. (At least in polite society.) But here’s the thing – entropy shows up in all sorts of information theory and machine learning algorithms. And it shows up ALONE, as though it sprung – pure and holy – from the head of the famed Ludwig Boltzmann. What’s wrong with this is that: entropy never lives alone, in isolation. In the real world, entropy exists – always – hand-in-hand with…

Read More Read More

Wrapping Our Heads Around Entropy

Wrapping Our Heads Around Entropy

Comments 5 comments

Entropy – the Most Powerful Force in the ‘Verse:   Actually, that’s not quite true. The most powerful force in the ‘verse is free energy minimization. However, entropy is half of the free energy equation, and it’s usually the more complex half. So, if we understand entropy, then we can understand free energy minimization. If we understand free energy minimization, then we understand all the energy-based machine learning models, including the (restricted) Boltzmann machine and one of its most commonly-used…

Read More Read More

Artificial General Intelligence: Getting There from Here

Artificial General Intelligence: Getting There from Here

Comments 3 comments

What We Need to Create Artificial General Intelligence (AGI):   A brief recap: We know that we want to have neural networks (including deep learning) do something besides being sausage factories. We’ve know that the key missing step – a first principles step – to making this happen is to give the network something to do when it is not responding to inputs. Also, we’ve introduced something that the neural network CAN do; it can do free energy minimization with…

Read More Read More

Figuring Out the Puzzle (in a 2-D CVM Grid)

Figuring Out the Puzzle (in a 2-D CVM Grid)

Comments 7 comments

The Conundrum – and How to Solve It: We left off last week with a bit of a cliff-hanger; a puzzle with the 2-D CVM. (CVM stands for Cluster Variation Method; it’s a more complex form of a free energy equation that I discussed two weeks ago in this blogpost on The Big, Bad, Scary Free Energy Equation (and New Experimental Results); while not entirely unknown, it’s still not very common yet.) We asked ourselves: which of the two grids…

Read More Read More

2-D Cluster Variation Method: Code V&V

2-D Cluster Variation Method: Code V&V

Comments 7 comments

New Code (Not Released Yet): V&V the Code Before We Play:   Well, my darling, as you gathered from last week’s post, the world has shifted. Up until now, when we were talking about having a new free energy function to use inside a neural network, we had to do “Gedankenexperiments” (German for “thought experiments”). Now, though, there’s working code – and I so LOVE seeing the numbers and graphs come out; teasing it, playing with it … stroking it…

Read More Read More