Browsed by
Category: Neural Networks

A Tale of Two Probabilities

A Tale of Two Probabilities

Probabilities: Statistical Mechanics and Bayesian:   Machine learning fuses several different lines of thought, including statistical mechanics, Bayesian probability theory, and neural networks. There are two different ways of thinking about probability in machine learning; one comes from statistical mechanics, and the other from Bayesian logic. Both are important. They are also very different. While these two different ways of thinking about probability are usually very separate, they come together in some of the more advanced machine learning topics, such…

Read More Read More

Labor Day Reading and Academic Year Kick-Off

Labor Day Reading and Academic Year Kick-Off

Deep Learning / Machine Learning Reading and Study Guide:   Several of you have been asking for guided reading lists. This makes sense.   Your Starting Point for Neural Networks, Deep Learning, and Machine Learning   Your study program (reading and code) depends on where you are. Starting out (High-grass country; St. Louis to Alcove Springs): Basic neural networks and deep learning; architecture for common networks, such as CNNs (convolutional neural networks); learning rules and architecture design. Well on the…

Read More Read More

The Statistical Mechanics Underpinnings of Machine Learning

The Statistical Mechanics Underpinnings of Machine Learning

Machine Learning Is Different Now:   Actually, machine learning is a continuation of what it always has been, which is deeply rooted in statistical physics (statistical mechanics). It’s just that there’s a culmination of insights that are now a very substantive body of work, with more theoretical rigor behind them than most of us know.     A Lesson from Mom: It takes a lot of time to learn a new discipline. This is something that I learned from my…

Read More Read More

Deep Learning: The Fast Evolution of Artificial Intelligence

Deep Learning: The Fast Evolution of Artificial Intelligence

Just one of the slides from a presentation that I’m working up for an upcoming online presentation at Northwestern University, but it tells the story. Just one more thought: here’s the rapid pace of evolution within just the image analysis realm of AI, largely due to multiple layers (sometimes, many, many, MANY multiple layers) of networks, a good fraction of which are Convolutional Neural Networks, or CNNs.

Neural Networks and Python Code: Be Careful with the Array Indices!

Neural Networks and Python Code: Be Careful with the Array Indices!

Our Special Topics class on Deep Learning (Northwestern University, Master of Science in Predictive Analytics program, Winter, 2017) starts off with very basic neural networks: the backpropagation learning method applied to the classic X-OR problem. I’m writing Python code to go with this class, and the result by the end of the quarter should be five-to-six solid pieces of code, involving either the backpropagation or Boltzmann machine learning algorithm, with various network configurations. The following figure shows the dependence of…

Read More Read More

Deep Learning: The First Layer

Deep Learning: The First Layer

It’s been something of a challenging week. We’ve kicked off our new PREDICT 490 Special Topics Course in Deep Learning at Northwestern University. I’ve got a full house; there’s been a waiting list since Thanksgiving, and everyone is acutely aware of the business climate surrounding deep learning. However (and I’m not terribly surprised here), most people who want to learn Deep Learning (DL) really don’t have a solid foundation in neural networks just yet. Thus, what we’re really doing is…

Read More Read More

Getting Started in Deep Learning

Getting Started in Deep Learning

It’s been a lovely Christmas day. More social than any that I can remember, for a very long time. (Wonderful morning visit with my next-door neighbor. Great mid-day party. A sweet restorative nap afterwards.) And now, the thoughts that have been buzzing through and around my head for the past 48 hours — how to get started with deep learning. Of course there are all sorts of entry points. Historical, functional, mathematical… But what came to me, over these past…

Read More Read More

Approximate Bayesian Inference

Approximate Bayesian Inference

Variational Free Energy I spent some time trying to figure out the derivation for the variational free energy, as expressed in some of Friston’s papers (see citations below). While I made an intuitive justification, I just found this derivation (Kokkinos; see the reference and link below): Other discussions about variational free energy: Whereas maximum a posteriori methods optimize a point estimate of the parameters, in ensemble learning an ensemble is optimized, so that it approximates the entire posterior probability distribution…

Read More Read More

Brain-Based Computing: Foundation for Deep Learning

Brain-Based Computing: Foundation for Deep Learning

Three Key Brain Strategies Used in Deep Learning for Artificial Intelligence   References for Brain-Based Computing (Methodologies for Deep Learning and Artificial Intelligence) Maren, A.J. (2015) How the Brain Solves Tough Problems. In Making Sense: Extracting Meaning from Text by Matching Entities and Terms to Ontologies and Concepts, Chapter 2 Draft. (Dec. 31, 2015). pdf Maren, A.J. (2015). Brain-Based Computing. (PPT Slidedeck) PPT  

Novelty Detection in Text Corpora

Novelty Detection in Text Corpora

Detecting Novelty Using Text Analytics Detecting novel events – new words, meaning new events – is one of the most important text analytics tasks, and is an important step towards predictive analytics using text mining. On July 24, 2015, The New York Times (and many other news sources) published an article identifying potential inclusion of classified information in the emails which Hillary Clinton had sent via private email and stored on her private email server. How would we use text…

Read More Read More