Smart Machines, Complexity, and 42

In Hitchhiker's Guide to the Galaxy, there is a poignant moment when a machine is asked the answer to the ultimate question of life, the universe, and everything.

It crunches numbers for millions of years, and returns the baffling answer "42".

The scenario that Douglas Adams concocted might be amusing, but given our increasing reliance on machines, it has ripples in today's world.

Computers can often provide answers, without providing insight. For meaning-seeking humans, this can be deeply unsatisfying. Witness the unease surrounding computer-assisted proofs.

Machine learning can help us deduce models to navigate complex systems. Neural network models might start with simple rules for learning. But the models they "learn" or end up with, are anything but simple.

To keep things specific, consider programming a self-driving car. The model may start simple ("keep between the lanes"), but get hellishly complicated as numerous edge cases ("dog jumps into the road", "many people don't check their blind spot", "the night is foggy") are subsumed.

Even if the car works reasonably well, how it responds to a "black swan" situation (one it has never seen before) might be anybody's guess.

Practical models for complex systems might be insanely complicated.

A recent "Rationally Speaking" podcast touched upon many of these issues. In particular, I found the discussion of "physics thinking", which emphasizes universal models by ignoring details, and "biological thinking", which celebrates the diversity of phenomenon by focusing on details, incredibly fascinating. From the transcript:

The physics approach, you see it embodied maybe in like an Isaac Newton. A simple set of equations explains a whole host of phenomena. So you write some equations to explain gravity, and it can explain everything from the orbits, the planets, the nature of the tides, to how a baseball arcs when you throw it. It has this incredibly explanatory power. It might not explain every detail, but it maybe it could explain the vast majority of what's going on within a system. That's the physics. The physics thinking approach, abstracting away details, deals with some very powerful insights.

On the other hand, you have biological thinking. Which is the recognition that oftentimes in other types of systems, in certain types of systems, the details not only are fun and enjoyable to focus on, but they're also extremely important. They might even actually make up the majority of the kinds of behavior that the system can exhibit. Therefore, if you sweep away the details and you try to create this abstracted notion of the system, you're actually missing the majority of what is going on. The biological approach should be that you recognize the details are actually very important. And therefore they need to be focused on. 

I think when we think about technologies both approaches are actually very powerful. But oftentimes I think people in their haste to understand technology, oftentimes because technologies are engineered things, we often think of them as perhaps being more the physics thinking side of the spectrum. When in fact, because they need to mirror the extreme messiness of the real world, or there's a lot of exceptions, or they've grown and evolved over time, often it's a very organic, almost biological fashion. They actually end up having a great deal of affinity with biological systems. And systems that are amenable to biological thinking and biology approaches.