Browsed by
Category: A Resource – Book

Book Progress

Book Progress

Comments 0 Comment

The Uber-Important Chapter 7 – Introduction to Energy-based Neural Networks:   I tell students that it’s like being on the Oregon Trail. All of the stochastic gradient-descent networks (up to and including Convolutional Neural Networks, or ConvNets, and Long Short-Term Memory networks, or LSTM networks) can be understood using backpropagation. This requires only that first-semester calculus background. Sure, grunting through the chain rule (many, many times) gets tedious. But it’s doable. In contrast, the energy-based networks are the heart and…

Read More Read More

Machine Learning: Multistage Boost Process

Machine Learning: Multistage Boost Process

Comments 0 Comment

Three Stages to Orbital Altitude in Machine Learning Several years ago, Regina Dugan (then Director of DARPA) gave a talk in which she showed a clip of epic NASA launch fails. Not just one, but many fails. The theme was that we had to risk failure in order to succeed with innovation. This YouTube vid of rocket launch failures isn’t the exact clip that she showed (the “action” doesn’t kick in for about a minute), but it’s pretty close. For…

Read More Read More

Labor Day Reading and Academic Year Kick-Off

Labor Day Reading and Academic Year Kick-Off

Comments 9 comments

Deep Learning / Machine Learning Reading and Study Guide:   Several of you have been asking for guided reading lists. This makes sense.   Your Starting Point for Neural Networks, Deep Learning, and Machine Learning   Your study program (reading and code) depends on where you are. Starting out (High-grass country; St. Louis to Alcove Springs): Basic neural networks and deep learning; architecture for common networks, such as CNNs (convolutional neural networks); learning rules and architecture design. Well on the…

Read More Read More

Deep Learning: The First Layer

Deep Learning: The First Layer

Comments 5 comments

It’s been something of a challenging week. We’ve kicked off our new PREDICT 490 Special Topics Course in Deep Learning at Northwestern University. I’ve got a full house; there’s been a waiting list since Thanksgiving, and everyone is acutely aware of the business climate surrounding deep learning. However (and I’m not terribly surprised here), most people who want to learn Deep Learning (DL) really don’t have a solid foundation in neural networks just yet. Thus, what we’re really doing is…

Read More Read More

Brain-Based Computing: Foundation for Deep Learning

Brain-Based Computing: Foundation for Deep Learning

Comments 3 comments

Three Key Brain Strategies Used in Deep Learning for Artificial Intelligence   References for Brain-Based Computing (Methodologies for Deep Learning and Artificial Intelligence) Maren, A.J. (2015) How the Brain Solves Tough Problems. In Making Sense: Extracting Meaning from Text by Matching Entities and Terms to Ontologies and Concepts, Chapter 2 Draft. (Dec. 31, 2015). pdf Maren, A.J. (2015). Brain-Based Computing. (PPT Slidedeck) PPT  

Making Sense: Extracting Meaning from Text

Making Sense: Extracting Meaning from Text

Making Sense: Extracting Meaning from Text by Matching Terms and Entities to Ontologies and Concepts Text analytics is the means by which computer algorithms can extract meaning and useful insights from raw text sources. This can have enormous impact in realms such as marketing, business intelligence, and political campaigns. However, text analytics is one of the toughest challenges in predictive analytics. The reason why this is so hard? It’s because – when done right – text analytics must effectively connect…

Read More Read More

Biologically-Based Multisensor Fusion for Brain-Computer Interfaces

Biologically-Based Multisensor Fusion for Brain-Computer Interfaces

Comments 0 Comment

Multisensor Fusion for Brain-Computer Interfaces (BCIs) More than 25 years ago, sensor fusion was identified as a militarily critical technology. (See blog post describing role of sensor fusion for Navy air traffic control.) Since that time, both our knowledge of – and the importance of – sensor fusion has grown substantially. Groundbreaking work by Barry Stein and M. Alex Meredith, at the Bowman Grey School of Medicine at Wake Forrest University, elucidated the specific mechanisms of biological sensor fusion in…

Read More Read More

Statistical Mechanics, Neural Domains, and Big Data

Statistical Mechanics, Neural Domains, and Big Data

Comments 0 Comment

How Neural Domain Activation and Statistical Mechanics Model Interactions in Large Data Corpora (Big Data) I was enthralled. I could read for only a few pages at a time, I was so overwhelmed with the insights that this book provided. And I was about twenty-five years old at the time. I had just discovered this book while browsing the stacks as a graduate student at Arizona State (ASU). The book was The Mindful Brain: Cortical Organization and the Group-Selective Theory…

Read More Read More

Nonextensive Statistical Mechanics – Good Read on Advanced Entropy Formulation

Nonextensive Statistical Mechanics – Good Read on Advanced Entropy Formulation

Comments 0 Comment

Advances in Thinking about Entropy Starting through Tsallis’s book on entropy; Introduction to Nonextensive Statistical Mechanics. This is a fascinating discussion – really, it’s the roots of philosophy; the real “what-is-so” about the world. Which minimally requires a good solid year or two of graduate-level statistical thermodynamics to even start the read. But worth it. There’s some potential applications of this approach to areas in which I’ve worked before; need to mull this over and jig some ideas about to…

Read More Read More

Good Read on Modeling Social Emergent Phenomena – But Still Not There Yet!

Good Read on Modeling Social Emergent Phenomena – But Still Not There Yet!

Comments 0 Comment

Philip Ball – Critical Mass The most important thing we can do right now – given the huge changes ahead of us – both in society, the world, and technology – is to get some sort of “handle” on what’s coming up. By that, I mean a good set of models. And as a result, I’m on a search for good models. Those that I know, those that are new. Those that make sense, and those that don’t. (We need…

Read More Read More