Chapter Content
Okay, so, like, let's talk about progress, right? And how we, like, actually *do* make stuff better. It's all about abstraction, you know? Like, building things out of bigger and bigger pieces.
Think about it. We're trying to solve, like, *way* harder problems than we used to. And that means things get complicated, right? More factors to consider, more parts to put together. Like, a little bridge over a creek? Easy peasy. But a *massive* bridge spanning a huge river? Suddenly you need tons more support, different materials, joints that can handle, like, wind and earthquakes and all that jazz.
That's just how it goes with difficult problems. They're packed with details. Look at anything we build today – it's got *way* more components than stuff from the past. A modern jet has, like, over a *million* parts! Our power grids are these crazy webs of connections. Even a microchip has billions of transistors. The harder the problem, the more pieces, plain and simple.
Now, you'd *think* that means we need to know, like, a *lot* more stuff to build these things, right? Like, imagine how much more a bridge builder needs to know *now* compared to 200 years ago. It's kind of overwhelming to think about.
But here's the thing... how are we even *doing* this? How are humans making all this incredibly complex stuff in just a few generations? How can *one* person even keep track of all those pieces? Are we just getting smarter? Are we all suddenly, like, working harder?
Nope! The human brain hasn't changed in, like, 50,000 years, and let's be real, we're probably not working harder. So, what's the secret?
Well, the real deal is that technological progress just wouldn't *happen* if we had to completely understand every single intricate detail. The reason we can actually *advance* is because each generation starts off further ahead than the last one, you know? We only need to understand how to use what we have *now*.
It's like, we're bootstrapping ourselves. We build on top of what came before, constantly improving and refining. Progress isn't about being a genius or having some crazy new idea, it's about automatically, inevitably advancing by taking the previous work and using it as the foundation for the next step. We move forward by packaging up all those details into easier-to-use tools. That's it. That's the whole story of progress: abstraction.
Think about software, right? Over the last 50 years, it's just become easier and easier to do the *same* stuff with *less* code. It started with machine code – just strings of zeros and ones that the computer understands directly. It's the *most* basic level, the closest thing to the microchip. It takes forever to do anything because every instruction is, like, super simple. It's like trying to build a house from raw materials instead of pre-made parts.
Then came assembly language, which is basically a slightly-more-human-friendly version of machine code. It uses symbols we can actually understand, which makes it faster to tell the computer what to do. It's an interface on top of machine code. For every "lever" you pull in assembly, multiple levers get pulled in machine code.
Next up, procedural programming. This lets us use named procedures and functions. Think of these as, like, pre-packaged chunks of assembly language. Another level of abstraction! Now we can think of computer functionality as reusable modules. Like, our house-building materials are partially assembled, making it even easier to put our house together.
But as software got more complex, with bigger teams and more requirements, procedural programming just got too clunky. Enter object-oriented programming (OOP). This abstracts away even *more* of the low-level details and brings in concepts like objects, which, basically, are like, even higher-level, reusable components that contain data and behavior.
With OOP, our house-building pieces are even *more* pre-built. It's more aligned with how we think. Concepts like inheritance and encapsulation mean we can focus on, like, the strategic part of creating software, not getting bogged down in the details. At the industrial scale, it means more focus on optimizing efficiency, quality control, scalability, and sustainability.
But why stop there, right? People want all that interactive stuff – online shopping, social media, you name it. That's where scripting languages come in. They make it possible to create dynamic user interfaces. And things like asynchronous operations and browser compatibility demanded even *more* abstract languages. Suddenly things like memory management were abstracted away. With scripting, the gap between having an idea and actually having working software got even smaller.
And, at the *very* top, you've got visual and audio interfaces. Graphical User Interfaces (GUIs) abstract away the inner workings of, like, everything by giving us visual representations of the data. We've got drag-and-drop tools for building software, and with AI, we're even getting completely codeless development!
Today, you can prototype entire applications in *weeks*. And the way we work is completely different than it was 50 years ago. Even people who aren't super technical play a big part, because the tools are just so much more accessible. The line between technical and non-technical is blurring because of these high levels of abstraction.
This all means we can solve problems that would have been *impossible* before. We can analyze massive amounts of data. We can build machines that show some form of intelligence. We can create advanced climate models, study space, connect people all over the world.
Software started as this small thing, and now it's, like, *the* biggest player in the economy. It brings collaboration and connectivity on an unbelievable scale. And it's all thanks to abstraction. Each generation has gotten a means to piece together software more easily than the last, and that means we can solve harder problems and manage more pieces.
Abstraction doesn't get rid of details, it bundles them up into higher-level constructs so we can operate them with fewer actions. It makes progress inevitable, because it pushes us forward by packaging up the details and creating those interfaces. Those interfaces are what the *next* generation uses as their starting point.
When you look around at skyscrapers and satellites and smartphones, it's hard to believe humans can build these things. All the inner workings are super detailed, and everything needs to be coordinated just right. But when you realize that each generation only needs to use what the last one created, it makes sense. Builders today don't need to know *everything*. They just need to understand their current level of abstraction.
That means nobody today actually knows how *everything* works. That knowledge has been hidden by each new level of abstraction. And it's true for all areas of innovation. We're not smarter than our ancestors, we don't work harder, and we don't even know *that* much more. It's all about abstraction and bootstrapped progress.
Now, we usually think of abstraction as a *mental* thing, right? Creating high-level concepts, like putting all different kinds of dogs into the category "dog." That's abstraction in the mind. It helps us understand and navigate the world.
But abstraction can also be *physical*. Remember that house-building example? It wasn't just an analogy. If you're using bigger, pre-built pieces, you're using physical abstractions. Just like the mind puts specifics into categories, physical details can be combined to serve a function at a higher level.
Think about making something usable, you know? Like, nobody could use a stone axe if they had to hold the blade while swinging. The binding combines the blade and handle into one object, so the user doesn't have to worry about them coming apart. Things are only usable if it takes less effort to operate them than it would to coordinate each piece separately.
Take a car, right? If drivers had to manually control every part of the engine and the transmission, nobody could drive. The interfaces we use – the steering wheel, the pedals – hide all that complexity. Even the stick shift in a manual transmission is a physical abstraction. It reduces the number of actions needed to do something. You can select the gear you want by moving the lever and the clutch, without having to mess with the gears themselves. The gear positions are an interface that lets you choose without knowing all the internal details. Of course, automatic transmissions take it to the *next* level.
Just like we mentally combine things into categories, the stick shift represents a physical construct with a bunch of internal details. The automatic transmission takes it even *higher*. Physical abstraction is a huge part of progress.
The best practices in any industry are built around the physical abstractions we create. An MRI technician isn't controlling every detail of the machine. A barista isn't manually heating water and grinding beans. An asphalt paver isn't sourcing all the materials. They're operating *above* the level of previously worked-out details. Our knowledge and skills are aligned with those physical abstractions.
Basically, it's all about moving humanity forward without needing more knowledge or effort. But even though it's automatic and inevitable, creating interfaces takes effort. Physical abstractions are intentional acts that require a lot of thought. Historically, we've done that through design.
The easiest way to create physical abstractions is through design. We consciously decide which pieces to include, how to connect them, and how to show their functionality through an interface. In the software example, procedural programming was designed by bundling machine code into higher-level syntax. It takes calculated reasoning. You have to dig into the current level and decide how to connect things to create a higher level. Design creates physical abstractions by packaging the inner workings. That allows us to coordinate disparate functions more easily.
The mental and physical versions of abstraction are super connected. The stick shift works because it connects to a set of internal pieces that share a common purpose. We can create those physical constructs because we can notice patterns and group them into higher-level forms. Physical abstraction reflects the conscious abstractions we make with our minds.
So, we achieve progress through abstraction, and design is how we realize that abstraction. That's why design has been a core part of progress. But design has its limits. You can't design without understanding how all the internal pieces interact.
To design is to reason about the connections between lower and higher levels. That's why design depends on determinism. It only works if we can predict how the outputs of a system will be produced. We can't bundle the inner workings of a transmission into a stick shift unless we can see how the gears affect each other. We can't bundle the C language into C++ unless we can see how C works. If we couldn't reason about how each piece interacts, our designs would just be guesses.
Now, you might think design can be high-level, just guidelines that help steer things in the right direction. But that's not design, that's a methodology or framework. Design is about making intentional choices about how different elements interact. It depends on the premise of deterministic knowledge.
Determinism means a system's behavior is completely predictable based on its inputs and the rules that govern it. If you give the same inputs, you'll always get the same outputs. That repeatability and predictability is built into the idea of design. It assumes there are explicit steps that flow from input to output. If not, choosing specific pieces and connections would be useless.
A nondeterministic system is one whose behavior isn't entirely predictable. Even with the same inputs, you can get different results. Traffic, for example. The same cars and drivers, driving at the same speeds, arriving at the same intersection will not produce purely predictable traffic flow. There will be a great deal of variation in the sequence of movements and flow of vehicles. That kind of system can't be designed.
You might say we *do* design traffic systems, and they *do* add predictable behavior. But we're not designing the *traffic* itself, we're designing an *outer framework* that governs the nondeterministic system. Traffic systems don't control the interactions between vehicles, they just place limitations that apply to everyone. It's like government regulating markets. Trying to design the system itself would interfere with its functioning.
Nondeterminism is directly related to how difficult a problem is. Just like you can't design a nondeterministic system, you can't solve hard problems with determinism. Hard problems require trial-and-error and heuristics. It's not just a *more* challenging problem, it's a fundamentally different situation that needs a different kind of solution.
Nature's solutions solve the *hardest* problems. Natural environments present hard problems, and nature solves them with objects that don't just convert inputs into outputs using deterministically-connected pieces. Nature creates emergent structures. There aren't well-defined steps between inputs and outputs. Nature's situations are made of too many factors and interactions. Even if we made something with a rule for every possible outcome, it wouldn't work.
Thinking of hard problems as just harder versions of rules-based systems is like trying to describe quantum mechanics with classical mechanics. It's not even close, because the quantum realm is fundamentally different.
Nature's solutions have true complexity. Their outputs aren't just the sum of interactions or a sequence of procedures. Nature creates matter configurations that produce the needed outputs in a completely different way. They also *require* that nondeterminism to function.
Take facial recognition, right? Humans and advanced AI do it naturally. What makes it a hard problem is that "solve" means something different than our usual deterministic view. Under determinism, solve means finding a specific, definitive answer. But nondeterministic systems are more like probabilistic answers, with some degree of correctness.
But while that "softness" analogy is closer to the spirit of nondeterminism, it still frames nondeterminism as an approximate version of determinism. Computational approaches find a satisfactory outcome. But the idea that nondeterminism is a second-rate version of determinism is wrong.
The solving that happens under complexity isn't weaker, it's completely different. And it's far more powerful and realistic than anything a deterministic solution could be. Approximation is just another byproduct of thinking about complex things incorrectly. Genuine solutions to hard problems don't approximate, they compute the answer directly using a different notion of computing.
The lack of causal mechanisms at the heart of complexity means design can't produce those kinds of solutions. There are no bundles that can be created to expose deterministically connected interfaces to users. Design's dependence on determinism means it can only solve simple problems.
This brings us to a critical point: 1) We need physical abstractions for progress, 2) we've always created them through design, 3) design can't solve hard problems, and 4) we *have* to solve hard problems now.
The current way we create physical abstractions will soon end. Design can't create what we need. The complexity we face demands something else. We can't reason about the pieces and the connections, because nature's solutions work differently.
It's a misplaced concreteness to talk about nature as a deterministic machine. It's a naive view, and sometimes it plays out dangerously. The notion that nature is like rules-based machinery is hindering progress. We need to rethink how we build things.
The shift in problem class and the solutions needed to resolve them undergirds the philosophical shift required to operate effectively in the age of complexity. The challenge is not one of merely accepting the degradation of causal understanding, it is redefining our concepts of knowledge, skill, and our approach to human innovation. We have to build differently.
Just because we can't use design doesn't mean we don't need abstraction. We can't just connect more pieces to solve a harder problem. Pieces need to be bundled into higher-level constructs for the next generation. That's how we place a lot of pieces into intricate objects correctly. It's too hard to arrange thousands of pieces individually. Lower-level pieces need to find the informational cohesion necessary to solve the problem one level higher, and abstraction is how this cohesion is achieved.
That's true for any kind of progress. Think about how we create abstractions to understand the world. We can't navigate life without categorizing things. Threats versus opportunities, friends versus foes, food, danger, shelter, etc. By creating those mental abstractions, we lower the cognitive load needed to get through life.
That ability to spot connections extends to our innovations. Everything from new philosophies to new technologies comes from binding information and physical things into new entities. Abstraction makes difficult challenges easier because it creates new structures that can compute without needing a one-to-one match between the problem and the solution. It arranges things so the internal coordination of pieces can attend to the multitude of factors inside harder challenges. In short, there's no progress without abstraction.
That's as true for nature as it is for us. Nature progresses by changing and developing over time. It continually solves problems against shifting stressors. Humans are just organisms solving problems en-masse. But we create our abstractions by design, using causal information about how pieces interact. That's not how nature does it. Nature doesn't use conscious reasoning to bind pieces into higher-level constructs. And yet it solves problems that are *way* harder than anything we tackle.
So, how does nature create its abstractions?
Physical abstraction is still the bundling together of functionality into higher-level groups and exposing an interface, but the bundle in nature is not some deterministically designed aggregation, it is a statistically automatic manifestation of matter.
But what matters at this juncture is the admission that nature’s physical abstractions, while critical for progress, are not designed. There is no conscious effort by nature to choose which pieces and interactions constitute a bundle, and so abstraction in nature must have a fundamentally different mechanism.
How does nature "know" what pieces to combine if it can't reason about them? How can natural selection spot the similarities that connect one level of matter to the next?