Browsed by
Category: A Resource

Ontologies, Knowledge Graphs, and AI: Getting from “Here” to “There” (Part 2)

Ontologies, Knowledge Graphs, and AI: Getting from “Here” to “There” (Part 2)

Comments 0 Comment

A Principled Approach to AI: Representations and Transitions:   In the last post, on “Moving Between Representation Levels: The Key to an AI System (Part 1),” I re-introduced one of the most important and fundamental AI topics: how we can effectively use multiple representation levels. If we’re going to build (or gauge the properties of) an AI system, we need a framework. The notion of representations, and of moving between representation levels, is as fundamental as we can get. In…

Read More Read More

Moving Between Representation Levels – the Key to Making an AI System Work (Part 1)

Moving Between Representation Levels – the Key to Making an AI System Work (Part 1)

Comments 2 comments

Representation Levels: The Key to Understanding AI   “No computation without representation” Jerry Fodor (1975). The Language of Thought, p.34. online access.   One of the key notions underlying artificial intelligence (AI) systems is not only that of knowledge representation, but that a good AI system will successively move disparate pieces of low-level, or signal-level information up the abstraction ladder. For example, an image understanding system will have a low-level component that extracts edges and regions from the image (or…

Read More Read More

We’ve Been Really and Truly **cked (Insert a consonant and vowel of your choice)

We’ve Been Really and Truly **cked (Insert a consonant and vowel of your choice)

Comments 2 comments

High-Precision Mind-**cking   You already know the main storyline: Cambridge Analytica, Brietbart, Facebook, and possible other players. Trump’s win of the electoral vote by about 40,000 votes through carefully targeting not only certain swing states, but micro-elements within those states. The questions now are (for those of us techie folks): (1) Technically, just how did this happen? (We want more than the few words in the mainstream news), and (2) (That which really interests us:) What are the countermeasures? One…

Read More Read More

2-D Cluster Variation Method: Code V&V

2-D Cluster Variation Method: Code V&V

Comments 7 comments

New Code (Not Released Yet): V&V the Code Before We Play:   Well, my darling, as you gathered from last week’s post, the world has shifted. Up until now, when we were talking about having a new free energy function to use inside a neural network, we had to do “Gedankenexperiments” (German for “thought experiments”). Now, though, there’s working code – and I so LOVE seeing the numbers and graphs come out; teasing it, playing with it … stroking it…

Read More Read More

Statistical Mechanics, the Future of AI, and Personal Stories

Statistical Mechanics, the Future of AI, and Personal Stories

Comments 0 Comment

Statistical Mechanics and Personal Stories (On the Same Page!)   Yikes! It’s Thursday morning already. I haven’t written to you for three weeks. That’s long enough that I have to pause and search my memory for my username to get into the website. Thanksgiving was lovely. The Thursday after that was grading, all day – and for several days before and after. By now, I (and most of you) have had a few days of recovery, from what has been…

Read More Read More

Third Stage Boost – Part 2: Implications of Neuromorphic Computing

Third Stage Boost – Part 2: Implications of Neuromorphic Computing

Comments 6 comments

Neuromorphic Computing: Statistical Mechanics & Criticality   Last week, I suggested that we were on the verge of something new, and referenced an article by von Bubnoff: A brain built from atomic switches [that] can learn, together with the follow-on article Brain Built on Switches. The key innovation described in this article was a silver mesh, as shown in the following figure. This mesh is a “network of microscopically thin intersecting silver wires,” grown via a combination of electrochemical and…

Read More Read More

Third Stage Boost: Statistical Mechanics and Neuromorphic Computing – Part 1

Third Stage Boost: Statistical Mechanics and Neuromorphic Computing – Part 1

Comments 8 comments

Next-Generation Neural Network Architectures: More Brain-Like   Three generations of artificial intelligence.. The third generation is emerging … right about … now. That’s what is shown in this figure, presented in log-time scale. Brief history of AI in log-time scale The first generation of AI, symbolic AI, began conceptually around 1954, and lasted until 1986; 32 years. On the log-time scale shown in the figure above, this entire era takes place under the first curve; the black bell-shaped curve on…

Read More Read More

Machine Learning: Multistage Boost Process

Machine Learning: Multistage Boost Process

Comments 0 Comment

Three Stages to Orbital Altitude in Machine Learning Several years ago, Regina Dugan (then Director of DARPA) gave a talk in which she showed a clip of epic NASA launch fails. Not just one, but many fails. The theme was that we had to risk failure in order to succeed with innovation. This YouTube vid of rocket launch failures isn’t the exact clip that she showed (the “action” doesn’t kick in for about a minute), but it’s pretty close. For…

Read More Read More

Neg-Log-Sum-Exponent-Neg-Energy – That’s the Easy Part!

Neg-Log-Sum-Exponent-Neg-Energy – That’s the Easy Part!

Comments 12 comments

The Surprising (Hidden) “Gotcha” in This Energy Equation: A couple of days ago, I was doing one of my regular weekly online “Synch” sessions with my Deep Learning students. In a sort of “Beware, here there be dragons!” moment, I showed them this energy equation from the Hinton et al. (2012) Nature review paper on acoustic speech modeling: One of my students pointed out, “That equation looks kind of simple.” Well, he’s right. And I kind of bungled the answer,…

Read More Read More

Neural Network Architectures: Determining the Number of Hidden Nodes

Neural Network Architectures: Determining the Number of Hidden Nodes

Comments 0 Comment

Figuring Out the Number of Hidden Nodes: Then and Now   One of the most demanding questions in developing neural networks (of any size or complexity) is determining the architecture: number of layers, nodes-per-layer, and other factors. This was an important question in the late 1980’s and early 1990’s, when neural networks first emerged. Deciding on the network architecture details is even more challenging today. In this post, we’re going to look at some strategies for deciding on the number…

Read More Read More