Thrilled that so many of you are showing an interest in my book-in-progress, Statistical Mechanics, Neural Networks, and Artificial Intelligence. Since I’m putting draft chapters up on this website (even if they’re still a bit rough – like not having all the parts in place yet), you’re getting access to the full book-in-progress. 

That said, some of you who are newer to this community are wondering where some of the early chapters are, as witnessed by this — 

Question from one of my colleagues:

I went to the website to get your book. I did not see any links for the first 4 chapters. How do I get the first 4 chapters?

Answer:

Not there yet. I’ve been focusing on the chapters most essential for students – the ones most equations-dense. 

 

Book Writing: More Than Linearly Complex

So, now that I’m back to book-writing on a regular basis, I’ve checked out the book-writing processes used by successful authors. One of these authors is Robert Greene, who’s written very interesting (and best-selling) books, including The 48 Laws of Power and many others. 

Paulo Ribeiro writes about Robert Greene’s approach to writing, and specifically describes the complexity of writing a book: 

 

… Writing is something extremely hard to do and with a non-linear complexity: it increases faster than the size of the project. In another words, writing a piece 2x as long is not just 2x as difficult, but, say, 3 or 4 times.

Being able to write a facebook post is sensibly different from writing a school essay; which is not the same as composing a blog post. Let’s not even put writing a book on the same page, because it’s orders of magnitude more difficult.

Lately, I’ve been spending time just on managing the whole book-complexity thing.

The Complexity Behind-the-Scenes

If you’ve been part of my community, you’ve received emails from me (back on the regular Thursday schedule) for the last month – but they’ve all pointed you to the Table of Contents page for the book, not to a blogpost. That’s because, for over the previous four weeks and more, I’ve been working on the book, but not writing blogs. 

These last several weeks have less been about writing, and more about managing complexity. 

One Complexity Example

One project has been to get all of the chapters into the same kind of LaTex document (chapter) format. Earlier, when I was writing sporadically, I was using various LaTex templates. Now, I’m looking at getting a cohesive whole – and having a single template structure has been important. 

Oddly, this has been more difficult than expected. For example, when I ported one chapter into the new, common template, the LaTex compiler balked at my using reference tags such as “fig” when referring to a figure. It worked fine with all other versions … so ???

In my eagerness to just get that chapter up on the web, I went through and removed the reference label tags … and then found that later chapters, with the same new common template, were handling reference label tags just fine … so I’m going to go back and redo the entire chapter again. Not the writing part, just the administrative formatting … all stuff behind the scenes, and if we’re lucky, you won’t know that it’s happening. It’s just that THIS is now the kind of thing that’s taking up time. 

Another Complexity Example

The bibliography. Oh my God, the bibliography. 

As those of you who write using LaTex already know,  the bibliography is typically a separate, stand-alone file. It’s essentially a database; I can access a full bibliographic citation by using a citation keyphrase specific to that citation. 

LaTex citations are delicate, nuanced things. They require a deft and subtle hand – and a monomaniacal attention to detail. 

I have a “masterBibFile.” But because the way my chapters are stored (one folder for each chapter, with separate subfolders for figures and the bibliography), I have multiple copies of the bibliography. And therefore, multiple updates. I’ve had to learn some file management practices over the past few weeks. Lost references (due to deleting the wrong files) meant that I had to go re-research some key citations, and re-enter them into the (newly revised and updated) masterBibFile. 

It’s this kind of thing that takes lots of hours outside of the regular writing process, but which are really good examples of what Paulo referenced as the increased complexity of book writing. 

The Next Complexity – Style Sheets

I was writing the last email to you folks, Saturday a week ago, and used the phrase “multilayer Perceptron.” And then I couldn’t remember … in the book, was I capitalizing it as “Multilayer Perceptron,” or was it what I was using right then – “multilayer Perceptron.” 

Style sheets. They’re the answer to conundrums like this. 

This is about the stage (I’m five solid chapters into the book, with various notes and jagged starts on other chapters), where the need for a style sheet becomes very obvious. This is where I (and any author) keeps track of such matters – the little details that need to be smooth and stable across the entire volume. So … my next project is not getting a chapter done; it’s getting style sheet and checking the rough draft chapters (to which you already have access) against this sheet. 

That said, I hope to at least fill in the blanks for this coming week’s post by putting in links to the one other YouTube that I’ve created and also to specific, chapter-relevant blogs. Nothing new, but organized access to what I’ve done so far. 

 

Making Sure You Get the Latest

For those of you, who for some reason, come across these blogs but are not getting updates on the book – the best way to get the (now hopefully regular) updates is to do the classic Opt-In. The form is on the Main Book Page. It’s at the end of the page. Do the usual things, such as check your email for a confirmation email from me, confirm, start looking for the follow-up email sequence. All the normal stuff that happens when you’ve opted-in with someone.  

There’s another Opt-In form; it’s the one that you see in the right-hand-sidebar on almost all pages on this blog; that’s a different list. Feel free to opt-in there also. When I send out book updates or other emails, right now I’m sending them to BOTH lists, but the AWeber email management software (which I’m using) ensures that you only get one email, so you won’t get duplicates. Later on, when I separate the book completely and it has its own website, the Opt-In that you’ll have found at the bottom of the book page will be your connection with the book and all related things. 

 

Question for You (Feedback Request)

Now that the book is obviously in progress, and you have chapters to read, I’m going to start asking for feedback. Not the detailed stuff, just yet. (I KNOW I’ve got to fill in certain blanks, etc.) 

The real question is: what do you need? 

Specifically, what do you need to fill in the gaps so you can advance your understanding of neural networks, deep learning, AI in general – that you’re not easily able to get from some other source? 

For example, I researched the heck out of statistical mechanics intro material – available on the web in any form – trying to see if there was something out there that was a good point-of-reference for someone who needed to learn about partition functions. 

It was all WAY too abstract, theoretical, and terse. 

But … if someone wants to read any of the works on the Boltzmann machine and related subjects, knowing the partition function is a crucial starting place. So I’m writing about that. (Some material already up, some still to be drafted.) 

So I’ve got ideas about what I think you need … but now is a REALLY GOOD time to tell me, while the development course for the book is still very fluid.

 

An Example of What We Might Need 

Lately, I’ve been thinking about probability and neural networks a lot. 

In my MSDS 458 “AI and Deep Learning” class (in Northwestern’s Master of Science in Data Science program), the Week 7 Discussion is where I ask people to describe the difference between a “discriminative” versus a “generative” neural network. A “discriminative” one being, for example, a classic Multilayer Perceptron trained using a stochastic gradient descent algorithm (e.g., backprop), and a “generative” one being, as a classic example, the (restricted) Boltzmann machine, trained using its own algorithm (typically, contrastive divergence). 

So people dutifully troll around the web and come up with the expected material, usually in the form of a probabilistic explanation. Giving that explanation requires a few equations, and I don’t have the time to write them into this post right now, so that will be deferred to the next post. The essence is that it’s the difference between “conditional” probability and “co-occurrence” probability, all expressed in the form of Bayesian probabilities. 

What’s been on my mind lately is that – just because any one of us can write out the probability equations (“the probability of y conditioned on the occurrence of x …”), doesn’t mean that any of us has the faintest clue as to what y and x actually mean. 

This is a word problem. 

Remember those, from high school? Most of you hated, loathed, dreaded them. That’s because it was so damn difficult to figure out what was y, and what was x.

So … I’m thinking that a chapter, probably two or three chapters (by the time the book is done) on probability – with walk-throughs based on a couple of neural network examples – will be just what we need. This will help us understand that crucial difference between “discriminative” vs. “generative” at a gut-level sense, not just some vague concepts floating at the top level of our heads. 

But before I invest the months in those chapters … what do YOU think that YOU need? 

Leave comments. 

And thank you! 

Live free or die, my friend –

AJ Maren

Live free or die: Death is not the worst of evils.
Attr. to Gen. John Stark, American Revolutionary War

 

Relevant Sources

• • Ribeiro, P. (Apr. 12, 2014). The ‘Robert Greene’ method of writing books, Medium.com.

 

 

 

Previous Related Posts

• Nothing yet. (At least, I don’t think so.)