Browsed by
Category: A Resource

Notational Frenzy

Notational Frenzy

When the Subtle Art of Mathematical Notation Defeats You (and How to Fight Back)   A couple of years ago, I was teaching Time Series and Forecasting for the first time. I didn’t know the subject – at all – but that didn’t bother me. Hey, it was mathematics, right? Anything that’s mathematical, I can eat for lunch, and then want some dessert-equations afterwards. First week, introducing the subject. That went fine. Second week, Simple Exponential Smoothing (SES). That’s simple….

Read More Read More

Labor Day Reading and Academic Year Kick-Off

Labor Day Reading and Academic Year Kick-Off

Deep Learning / Machine Learning Reading and Study Guide:   Several of you have been asking for guided reading lists. This makes sense.   Your Starting Point for Neural Networks, Deep Learning, and Machine Learning   Your study program (reading and code) depends on where you are. Starting out (High-grass country; St. Louis to Alcove Springs): Basic neural networks and deep learning; architecture for common networks, such as CNNs (convolutional neural networks); learning rules and architecture design. Well on the…

Read More Read More

The Statistical Mechanics Underpinnings of Machine Learning

The Statistical Mechanics Underpinnings of Machine Learning

Machine Learning Is Different Now:   Actually, machine learning is a continuation of what it always has been, which is deeply rooted in statistical physics (statistical mechanics). It’s just that there’s a culmination of insights that are now a very substantive body of work, with more theoretical rigor behind them than most of us know.     A Lesson from Mom: It takes a lot of time to learn a new discipline. This is something that I learned from my…

Read More Read More

2025 and Beyond

2025 and Beyond

Artificial Intelligence and Jobs by the Year 2025: One of my biggest take-aways from the recent (May, 2017) NVIDIA GTC (GPU Technology Conference) was less about the technology, and more about the near-term jobs impact of artificial intelligence (AI) and robotics. Making smart education and career decisions is crucial, as the emerging combination of AI and robotics will have a huge impact on jobs. Those of you studying artificial intelligence, deep learning, and neural networks will have a stronger career…

Read More Read More

How to Read Karl Friston (in the Original Greek)

How to Read Karl Friston (in the Original Greek)

Karl Friston, whom we all admire, has written some lovely papers that are both enticing and obscure. Cutting to the chase, what we really want to understand is this equation: In a Research Digest article, Peter Freed writes: … And today, Karl Friston is not explaining [the free energy principle] in a way that makes it usable to your average psychiatrist/psychotherapist on the street – which is frustrating. I am not alone in my confusion, and if you read the…

Read More Read More

Deep Learning: The Fast Evolution of Artificial Intelligence

Deep Learning: The Fast Evolution of Artificial Intelligence

Just one of the slides from a presentation that I’m working up for an upcoming online presentation at Northwestern University, but it tells the story. Just one more thought: here’s the rapid pace of evolution within just the image analysis realm of AI, largely due to multiple layers (sometimes, many, many, MANY multiple layers) of networks, a good fraction of which are Convolutional Neural Networks, or CNNs.

Neural Networks and Python Code: Be Careful with the Array Indices!

Neural Networks and Python Code: Be Careful with the Array Indices!

Our Special Topics class on Deep Learning (Northwestern University, Master of Science in Predictive Analytics program, Winter, 2017) starts off with very basic neural networks: the backpropagation learning method applied to the classic X-OR problem. I’m writing Python code to go with this class, and the result by the end of the quarter should be five-to-six solid pieces of code, involving either the backpropagation or Boltzmann machine learning algorithm, with various network configurations. The following figure shows the dependence of…

Read More Read More

Deep Learning: The First Layer

Deep Learning: The First Layer

It’s been something of a challenging week. We’ve kicked off our new PREDICT 490 Special Topics Course in Deep Learning at Northwestern University. I’ve got a full house; there’s been a waiting list since Thanksgiving, and everyone is acutely aware of the business climate surrounding deep learning. However (and I’m not terribly surprised here), most people who want to learn Deep Learning (DL) really don’t have a solid foundation in neural networks just yet. Thus, what we’re really doing is…

Read More Read More

Approximate Bayesian Inference

Approximate Bayesian Inference

Variational Free Energy I spent some time trying to figure out the derivation for the variational free energy, as expressed in some of Friston’s papers (see citations below). While I made an intuitive justification, I just found this derivation (Kokkinos; see the reference and link below): Other discussions about variational free energy: Whereas maximum a posteriori methods optimize a point estimate of the parameters, in ensemble learning an ensemble is optimized, so that it approximates the entire posterior probability distribution…

Read More Read More

Brain-Based Computing: Foundation for Deep Learning

Brain-Based Computing: Foundation for Deep Learning

Three Key Brain Strategies Used in Deep Learning for Artificial Intelligence   References for Brain-Based Computing (Methodologies for Deep Learning and Artificial Intelligence) Maren, A.J. (2015) How the Brain Solves Tough Problems. In Making Sense: Extracting Meaning from Text by Matching Entities and Terms to Ontologies and Concepts, Chapter 2 Draft. (Dec. 31, 2015). pdf Maren, A.J. (2015). Brain-Based Computing. (PPT Slidedeck) PPT