Introduction to ML

mlbrainmap

You can’t swing a cat through a newsfeed these days without hitting a story about the extent of learning algorithms in everyday life. As many readers are no doubt aware, there’s a type of computation that’s brought new perspective to a wide swath of common and previously intractable problems, and unlike a more deterministic programmatic approach, this new approach to is based on learning. That’s right, instead of a linear sequence of instructions these algorithms figure out what to do for themselves.

There’s probably not a single aspect of our modern world that isn’t affected by or directly based on the new field of learning-based methods. But this isn’t just any type of learning: Meat Learning, or ML for short, affects almost every essential experience in the modern world. Choosing what book to read next? Meat learning. What’s for lunch? ML. Driving cars? ML conquered that task years ago (despite a high-frequency of catastrophic failure). Whether you’ve been aware of it or not, and for better or for worse, we’ve truly entered the age of the learning algorithm.

Many of the basic ideas and underlying structures of modern ML have been around for millennia, if not epochs, but it’s only been in the last few million years that ML has really puffed up its breast-feathers. Almost all ML algorithms in common usage today run on a variant of the same hardware: a fatty grey mass known to the professionals that work with them as a “meat computer.”

Rather than an improvement in the basic design or clever new architectures, the enabling breakthrough for modern ML has been largely based on the sheer mass of computational meat (archaically known as grey matter) incorporated in bleeding edge modern platforms. This new type of learner, of which a wide swath of variants all share the same designation of “human,” hasn’t collectively been around for long (geologically speaking), but has already had a huge impact. It is nearly impossible to go anywhere on the planet without running into some sign of human meat-learners’ work, play, or self-destructive irony.

Previous generations of meat-learners had about half the mass of computational meat allocated for each kg of body mass compared to the current state-of-the-art human models, at least after engaging regularization factors to make the human variants feel better about themselves. This vast increase in computational mass in a meat computer is almost entirely comprised of warm, squishy subunits intended to mimic the activity of artificial neural networks. Like an iceberg, the majority of these networks are hidden from view, which makes them very mysterious (some would say nonsensical) to work with. More on that later on.

hiddenlayers

The official definition of a meat learning platform, according to ML pioneer Sammy Arthur is “…something that can learn, at least a little bit, but it has to be made of meat. The soft squishy stuff.” Some practitioners would say that the modern definition has grown since Arthur’s definitive statements. Although ML was at first considered a subset of the field of organic intelligence (OI for short), it is now widely acknowledged that ML has outgrown its humble beginnings as a specialization of OI, becoming a fully-fledged field in its own right. Also, it is quite clear that many of the modern systems running the cutting-edge human variants don’t possess all the attributes for qualification as bona fide general organic intelligence.

Unlike traditional programming, wherein the outcome depends on a succession of conditional statements, meat learners adapt to a given problem by being “trained” on large sets of data, aka, learning. To train a meat learning platform on solving a problem requires a large amount of input data, and in some cases, acquiring the right data can be quite the challenge. After learning on a sufficient number of training sets, a ML program becomes capable of independently completing a wide variety of tasks, including both stuff and things.

This unique learning approach means that meat learners often find solutions to problems that are unusual and sometimes downright surprising (or stupid). Due to the nature of how ML systems are trained, completing a task may include steps that appear to make very little sense until, finally, the program somehow converges to an interesting solution. Sometimes. Despite numerous advantages, these capabilities come at a cost.

Training an ML algorithm requires a vast input of training data, often taking twenty years or so before an ML system is ready for even the simplest of tasks. Due to the speed issue, most developers train an ML platform for multiple tasks in parallel, which seems to work out more or less OK. One potential disadvantage is that in most cases the vast majority of the squishy neural network performing ML computations is made up of hidden layers, the submerged part of the iceberg alluded to above. As a consequence, nobody is quite sure how they do what they do. In practice this isn’t much of a problem when a given ML is working, but it makes problems almost impossible to debug. Unlike a proper artificial neural network, not even the architecture of the system is available for design and adjustment. The only way to adjust a given ML system is by modifying the training inputs, often with unpredictable results. To make matters worse, in most cases learners heavily weight early training inputs, assigning little to no weight to those encountered as a more mature ML system. In all cases, once a particular ML circuit becomes entrenched in a wider ML architecture, it becomes subject to a “learned inertia” that makes it difficult for a learner to adapt previously learned strategies and can lead to typing in all caps.

Over-reliance on hidden layers and learned inertia aren’t the only problems associated with ML. Although meat-learners tend to be quite flexible, there are certain types of computation that ML just isn’t suited for. If your task involves Bayesian inference, for instance, you can just about forget about training an ML human platform to suit. Additionally, ML performance can be unacceptably erratic. A meat learner that previously performed reasonably well on a given task might perform terribly the next time around just because they are tired, drunk, or bored, and they usually are at least one of those three things.

ML has long been plagued by hardware problems as well. ML platforms tend to run at speeds of just a few tens of Hertz, and the upkeep of such a system can be expensive. Both dynamic and static memory suffer from volatility and infidelity, and these problems only get worse as the system matures. ML is inherently crufty: many aspects of the hardware have been retained from the very early days of the first chordate models, in turn borrowing much of their fundamentals from even earlier systems. The basis for switching action potentials, the discrete building block of meat learning activity, in microalgae is so similar to that in state-of-the-art meat learners that they can be functionally swapped. This highlights the archaic mechanics at the foundation of meat learning, but sheds some insight by making meat neurons (even in hidden layers) amenable to switching on or off with light exposure.

Despite myriad problems, ML seems to be here to stay and is liable to play a large role for at least another ten years or so. ML is slow, but gets around the problem by utilizing massive parallelization. ML takes years to train, but once it has learned a fair amount from training inputs it can perform a wide variety of tasks. ML is erratic and highly prone to various emotional and drive-reduction states, but most ML systems get around this issue by pretending everything is fine. Hidden layers make it difficult to determine the inner workings of a modern ML platform, but many agree that’s for the best anyway.

Whether ML developers overcome the existing issues or not may ultimately be irrelevant: for the nonce, meat learning in its many forms is the best and only irrevocable tool we’ve got.

Update 2016/12/23: Comment on action potentials added to paragraph 11

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s