Browsed by
Category: Predictive Intelligence

Making Sense: Extracting Meaning from Text

Making Sense: Extracting Meaning from Text

Making Sense: Extracting Meaning from Text by Matching Terms and Entities to Ontologies and Concepts Text analytics is the means by which computer algorithms can extract meaning and useful insights from raw text sources. This can have enormous impact in realms such as marketing, business intelligence, and political campaigns. However, text analytics is one of the toughest challenges in predictive analytics. The reason why this is so hard? It’s because – when done right – text analytics must effectively connect…

Read More Read More

Novelty Detection in Text Corpora

Novelty Detection in Text Corpora

Detecting Novelty Using Text Analytics Detecting novel events – new words, meaning new events – is one of the most important text analytics tasks, and is an important step towards predictive analytics using text mining. On July 24, 2015, The New York Times (and many other news sources) published an article identifying potential inclusion of classified information in the emails which Hillary Clinton had sent via private email and stored on her private email server. How would we use text…

Read More Read More

Ground-Truthing – The First Step in Predictive Analytics

Ground-Truthing – The First Step in Predictive Analytics

Ground Truthing – Your First Day in Class You may be joining me for the Summer 2015 class in Text Analytics (PREDICT 453) that I’ll be teaching through Northwestern University’s Master of Science in Predictive Analytics (MSPA) program, starting June 22, 2015. Or you may not. Either way, you can follow this class virtually – here through this blog. I’ll be posting lots of information about the upcoming class – and referencing my permanent pages that I’m setting up prior…

Read More Read More

Why Nonadditive Entropy Is Important for Big Data Corpora Combinations

Why Nonadditive Entropy Is Important for Big Data Corpora Combinations

Non-Additive Entropy – A Crucial Predictive Analysis Measure for Data Mining in Multiple Large Data Corpora Statistical mechanics has an important role to play in big data analytics. Up until now, there has been almost no understanding of how statistical mechanics provides both practical value and a theoretic framework for data analysis and even predictive intelligence (sometimes called predictive analysis). In a separate White Paper (link to be provided), I identify – for the first time – how statistical mechanics,…

Read More Read More

"Nonadditive Entropy" – An Excellent Review Article

"Nonadditive Entropy" – An Excellent Review Article

New Advances in Entropy Formulation – “Nonadditive Entropy” Well, chalk it up to being newly returned to the fold – after years of work in knowledge discovery, predictive analysis, neural networks, and sensor fusion, I’m finally returning to my roots and re-invigorating some previous work that involves the Cluster Variation Method. In the course of this, I’ve just learned (as a Janie-come-lately) about the major evolution in thinking about entropy, largely led by Constantino Tsallis. He has an excellent review…

Read More Read More

What Makes a Metastable State Happen?

What Makes a Metastable State Happen?

Metastable States – the Meltdown Precursors I’ve just read a recent column by JL, one of the editors from Taipan Daily. He states, in his column “There Will Be Blood in Europe”: Stepping back a bit: What is so frightening right now, not just in Europe but China and America and Japan too, is the presence of fraud-fueled “Lehman 2.0” catalysts threatening to explode. One could say that the 2008 financial crisis was the mother of all wake-up calls. But…

Read More Read More

"What is X?" – Modeling the Meltdown

"What is X?" – Modeling the Meltdown

“What is X?” – Modeling the 2008-2009 Financial Systems Meltdown We’re about to start a detailed walkthrough of applying a “simple” statistical thermodynamic model to the Wall Street players in the 2007-2009 timeframe. The two kinds of information that I’ll be joining together for this will be a description of Wall Street dynamics, based largely on Chasing Goldman Sachs (see previous blogposts for link), and the two-state Ising thermodynamic model that I’ve been presenting over the past several posts. The…

Read More Read More

Equilibrium and Utility: Two Different Realms

Equilibrium and Utility: Two Different Realms

Continuing with Beinhocker’s Origin of Wealth, it is important to distinguish carefully between some of the ideas that Beinhocker is expounding. While overall, he is doing a good job of bringing in many related thoughts and ideas, there is a slight tendency towards “mushing.” In that note, I’d like to suggest that we discern carefully between ideas involving utility (Origins, hardcover; pp. 34 & 37), and equilibrium. On pg. 34, Beinhocker begins a discussion of how utility is an underlying…

Read More Read More