Configuration Variables Along the Phase Space Boundaries for a 2-D CVM Last week’s blog showed how we could get x1 for a specific value of epsilon0, by taking the derivative of the free energy and setting it equal to zero. (This works for the special case where epsilon1 is zero, meaning that there is no interaction enthalpy.) Last week, we looked at one case, where epsilon0 = 1.0. This week, we take a range of epsilon0 values and find…
This Really Is Kind of Obvious, But … There’s something very interesting that we can do to obtain values for the epsilon0 parameter. Let’s stay with the case where there is no interaction enthalpy. In that case, we want to find the epsilon0 value that corresponds to the x1 value at a given free energy minimum. Or conversely, given an epsilon0 value, can we identify the x1 where the free energy minimum occurs? Turns out that, for this…
The 2-D CVM Entropy and Free Energy Minima when the Interaction Enthalpy Is Zero: Today, we transition from deriving the equations for the Cluster Variation Method (CVM) entropies (both 1-D and 2-D) to looking at how these entropies fit into the overall context of a free energy equation. Let’s start with entropy. The truly important thing about entropy is that it gives shape and order to the universe. Now, this may seem odd to those of us who’ve grown…
Why Shifting to the Object-Oriented Coding Approach REALLY IS Important: Well, I hate having to admit it. Mud on my face; all that. But I made a pretty significant Whoops! back this last winter when I posted a “Verification and Validation” document (hah!) to arXiv. The Sad Story of My Previous Ineptitude Well, there’s nothing like hearing about someone else’s screw-up in order to make us feel better about our own life, so here goes. I’d…
Showing How the Omega Equation is Obtained for the 1-D and 2-D CVM – Part 2: The cluster variation method (CVM) lets us characterize a system in terms of local patterns, and not just the numbers of units in on (A) and off (B) states. It works with a more complex entropy term. The natural question is: How do obtain this more complex entropy? This post continues a discussion begun in the last post, on how we actually get…
The 1-D CVM – A Single Zigzag Chain – Part 1: The cluster variation method (CVM) lets us characterize a system in terms of local patterns, and not just the numbers of units in on/off states. This is likely to be useful for machine learning and AI applications. Up until now, we’ve not told the story of how we actually compute the CVM entropy from the microstates. We’ll do that starting with this post; it will be a handy…
Looking around from a Node’s Point-of-View: As we move from procedural code to object-oriented Python for a Cluster Variation Method (CVM) grid, our perspective shifts. We now need to look at the world from a node’s point-of-view. It’s a lot like updating one’s relationship status in Facebook – except that after updating our own (node) status, we need to not only update the values for all of our own configuration variables, but then we need to travel around the…
The Cluster Variation Method – A Topographic Approach: Object-oriented programming is essential for working with the Cluster Variation Method (CVM), especially if we’re going to insert a CVM layer into a neural network. The reason is that approaching free energy minima via changing node states requires dealing with node, net, and grid topographies. If we’re going to be at all strategic in moving towards free energy minima, then we can’t just pick nodes at random. We need to know…
Working Through Salakhutdinov and Hinton’s “An Efficient Learning Procedure for Deep Boltzmann Machines” We can accomplish a lot, using multiple layers trained with backpropagation. However (as we all know), there are limits to how many layers that we can train at once, if we’re relying strictly on backpropagation (or any other gradient-descent learning rule). This is what stalled out the neural networks community, from the mid-1990’s to the mid-2000’s. The breakthrough came from Hinton and his group, with a…
Roy, A. Park, Y.J. and Pan, S.(2017, Sept. 21). Domain-SpecificWord Embeddings from Sparse Cybersecurity Texts, arXiv 1709.07470v1 [cs.CL]. pdf, accessed Apr. 25, 2018 by A.J.M. Abstract: Classic word embedding methods such as Word2Vec and GloVe work well when they are given a large text corpus. When the input texts are sparse as in many specialized domains (e.g., cybersecurity), these methods often fail to produce high-quality vectors. In this paper, we describe a novel method to train domain-specific word embeddings from…