And you're way, way into that regime where you're just fitting noise and the whole thing shouldn't work. It obviously shouldn't, and yet it does. Francis Crick said, look, there's no way that life got to this level of complexity in such a short period of time. It must have come from outer space. The genetic code is optimal in the sense of minimizing errors. If you wanted to know what is the purpose of life, the purpose of life is to help planets come into equilibrium. So a phase transition, you know, is like, for example, what happens to a piece of metal when I cool it below a certain temperature? And I want to know, can I use it to stick the pictures of my kids' holiday pictures on the door of my refrigerator? And the answer is yes. If it's magnetic, it'll stick with a piece of metal holding the picture up.
Something went wrong!
Hang in there while we get back on track
The INTO THE IMPOSSIBLE Podcast
The Physics Reason AI Works When It Shouldn’t | Nigel Goldenfeld
Speaker
Nigel Goldenfeld
Speaker
Brian Keating
Nigel Goldenfeld explores why artificial intelligence astonishingly works even when theoretically it should not, using physics concepts like phase transitions and the renormalization group to reveal the hidden laws behind complexity and AI’s surprising effectiveness.
✨ Magic Chat
Don't have time for the full episode?
Ask anything about this conversation — get answers in seconds, sourced from the transcript.
Try asking
Featured moments
Highlights
“The Mystery of Life’s Origins: "Francis Crick said, look, there's no way that life got to this level of complexity in such a short period of time. It must have come from outer space.”
“As I get closer and closer to the temperature where the magnetization disappears, how does the magnetization disappear? Does it— it disappears gradually, in fact. And if you ask how much magnetization there is, the answer is it goes like the square root of the difference between the temperature you're at and the critical temperature where the magnetization fully goes to zero.”
“More is Different": "more is different”
“So I'm the one that phased transitioned from the South Pole to San Diego, right?”
“Did Physics Really Lead to Modern Technology?" Quote: "But tell me, Nigel, did we look into the laws of physics to get the technology on the screen that you talk about in your course? Is that really what happens, or do we describe it later on? on post facto by these laws that we discovered.”
Timeline
How it unfolded
Read along
Full transcript
Mm-hmm.
And if it's not magnetic, it won't stick. And if you take that piece of metal that works as a magnet, fridge magnet, and you heat it up, it will stop becoming a magnet. And that's called a phase transition, as you know, and maybe some of your listeners know or viewers know. The interesting question you might ask is, as I get closer and closer to the temperature where the magnetization disappears, how does the magnetization disappear? Does it— it disappears gradually, in fact. And if you ask how much magnetization there is, the answer is it goes like the square root of the difference between the temperature you're at and the critical temperature where the magnetization fully goes to zero. At least that's what you would expect. And that's what very generic, very persuasive, simple theory, theoretical arguments that anybody can understand. I can explain it to my class in literally, you know, 20 seconds.
That's what you would predict. When you do the experiment, you find that it doesn't go like the square root of the critical temperature minus the temperature. It goes like the Tc minus T to a power like 0.3265136, some weird, weird number like that. And you might say, well, it's just a, you know, a more accurate number.
Yeah.
The problem is that there's no known way, or there was no known way to account for the fact that the number is not a half. It, I mean, it's like To prove it's a half, all I need to know is that magnets can be either magnetized north or magnetized south. And that's basically it. It's an argument that is so compelling, it can't possibly be wrong. And yet, in the decades from the late 1940s up to the middle 1960s, it was discovered that it was wrong. And it wasn't just only the magnetization. There were other thermodynamic properties like heat capacity and things like this, which I won't go into, which also have the similar unaccountable behavior. And it was the fact that these numbers, which are themselves not particularly important, it was the fact that you couldn't even explain in principle why they are not these simple numbers like 1/2 and so on.
And that was the reason for the puzzle.
Hmm.
And the explanation is a truly mind-boggling explanation. But just to tell you the outlines of the story, this phenomenon was addressed by Leo Kadanoff, Ben Widom, eventually Ken Wilson. Ken Wilson, yeah. And they invented this process of looking at a physical system on different scales of energy. So, you know, you could look at matter at the scale of this room. You could hear the sound waves, you can see light bouncing off the surfaces. On the other hand, if you want to go and see that there are atoms and see that there are quarks and things like this, you need to build a machine that's put in a tunnel 17 miles long under the Swiss Alps in order to be able to see things like that. So what you can see depends on what energy you look at it at and what timescale and what length scale you look at it at.
And the same thing turns out to be true of the laws of physics themselves.
Hmm.
And Leo Kadanoff was the first person who realized that, and Ken Wilson turned it into a mathematical tool, which was called the renormalization group. In fact, it's not even a group. You ask whether it's a group.
Mm-hmm.
It's actually a semi-group. And the idea was this: take a physical system and then just say, well, you know, I've got magnetic dipole moments. They're really spins of electrons, but we'll just call them magnetic dipole moments. They're in this bit of the sample. We'll just lump them together into one effective dipole moment because, you know, in this patch over here, you know, 80% of them are pointing up, 20% are pointing down. So we'll just say, okay, it's basically just a spin pointing up. And so you sort of block things up in that way. And then once you've done that once, you can do it again and again and again, and you keep on doing it and you keep on doing it.
And then you ask, what happens when you take that process to the infinite limit?
Instead.
And it's called a group. It's really a semi-group because you can lump the spins together and then make these bigger and bigger spins, this coarse-graining as physicists call it, but you can't go backwards. If you know the configuration of the very large scale and you say, well, what was the actual microscopic configuration of the electric dipole moment, the electron dipole moment? There's no unique answer to that. You know, the one that we had, 80% up, 20% down, if it had been 75% up, 25% down, it would've still ended up being regarded as a spin just pointing up. And so there's no unique answer. You can't go backwards.
Absolutely.
And that is why it's so profound. Because when you start looking at the laws of physics and you say, actually what we're doing when we're doing this, we're actually looking at the laws of physics. And I can tell you why it's the laws of physics. in a minute. Then if you know what the laws are microscopically and you start saying, if I know what happens at the scale of atoms, can I work out what happens to my crystal or fluid or something like that? Well, I can do this process and cause grain like that.
Right.
But I can't go backwards. If I can't go backwards, then you say to yourself, wait, I'm a physicist. All I can see around me is the natural world. There's nothing in this room that, at least if I live in the 19th century, would let me know that atoms exist and things inside the atoms and so on and so forth.
It drove Boltzmann crazy, right?
Well, it drove Boltzmann to suicide. It's not just that. There isn't a way in principle. So let's suppose you're trying to work out, you know, theories of the Standard Model, as people were doing in the '60s and '70s. Well, you can write down all sorts of non-Abelian gauge theories that you like, and they all sound very interesting. You have no way to know which one of them is right because they will all give you the same standard electricity and magnetism that we use in everyday life. And so the renormalization group, this non-uniqueness of going down in scale and going up in scale—
I understand.
That's the thing that makes high-energy physics, fundamental physics if you like, hard. And it's important in condensed matter physics because you say, well, I'm going to start at a level of description, which is I'm going to take you know, atoms for granted, and then work out what are the properties of matter. And so you can go in the way where there's only one way to do it. And that's why condensed matter physics is so successful. The other thing I would say is that this facet of the normalization group is what enables us to do physics in the first place. So think about it this way. Suppose you're a chemist.
Yeah.
Okay? And you're trying to understand organic chemistry, chemical reactions, biochemistry, or something. Let's suppose, you know, somebody knocks on the door and says, hey, we've got some really disturbing use. Somebody's just measured the radiative corrections to the mass of the top quark, and they're 20% different than what we thought. Does that mean all our chemistry is wrong? Well, the answer is no, because all of that was just lumped into one constant in the theory, just like we were taking the spins and lumping them together into one effective spin. And that constant is the mass of the proton.
Yeah.
The mass of the hydrogen atom, whatever. So a chemist is not worried about that. We know that the microphysics can be lumped into you effective descriptions at a larger scale. All the QCD and superstrings and whatever all ends up just giving you the mass of the proton or the spin of the proton or whatever. Because we don't have to worry about that, you can do chemistry. You can say, I don't worry about what the world is really made of. I just start with my level of description that I'm comfortable with, atoms, and then I proceed from there. And so, Without that, if that didn't happen, then we wouldn't have been able to do science.
I want to get back to reversibility and maybe even touch upon the origin of time, perhaps in the context of temperature. But before I get there, why didn't Ising realize this 40 years before Wilson and then Kadanoff, et cetera?
There's a model called the Ising-Lenz model. Lenz was Ising's advisor, and Lenz decided it would be nice to make a simple mathematical model of a magnet. So this was in like 1925, 1926, I think. So they didn't really know all the microphysics. So they said, well, let's just say that we have magnetic dipole moments, and we'll just say that in appropriate units, they can either point up or point down, spin up, spin down. And that model has just 2 variables. Spin can be plus or minus 1, and that's it. And then you take a lattice of all of those things, and then here's the extra ingredient.
So A magnetic dipole, of course, interacts with an external magnetic field. If you apply an external magnetic field to a dipole, it will orient with the external field. But there's another field, which is the fact that the electric dipoles exert a field on each other.
Yes.
On their neighbors. And so this was one of the first models where you explicitly had a cooperative phenomenon built in. In other words, whether this spin points up or not depends on not only whether there's an external magnetic field, but whether its neighbors are pointing up. If its neighbors are also pointing up, it's got a much higher likelihood of pointing up. If its neighbors are pointing down, it will likely point down. And so that's why this has been, you know, it's a famous model. You know, you can model social behavior with it. You can model the production of pistachio nuts in California orchards using it.
As it often is, yeah. So that's what the model is. And it really is the Drosophila of theoretical— It's the elegant— Quantitative matter physics. What did Ising do? This is a fantastic, fantastic story of failure and missed opportunity. So they said, well, let's just simplify the problem. Let's just, instead of having a 3-dimensional material, which is what matter really is, we'll just say, we'll just do these spins in 1 dimension. And then it's possible to, even if the problem is nonlinear in a complicated way, you can solve it exactly. And when Ising did that, He found that it didn't have a phase transition.
This phase transition of the magnet, the fridge magnet that below a certain temperature will stick to the refrigerator door and above that it won't, it didn't work in this calculation. He didn't see that at all. And so they gave up. They just said, well, this model is a useless model. Okay? Now, there's a reason. It's a very interesting thing. What they didn't know was that the behavior of matter strongly depends on dimension. And that's not obvious.
And it wasn't known at that time. Now we know it. In fact, now we use dimension as a variable, which we treat as a continuous variable and do perturbation theory in dimension and things like this. And then Lars Onsager figured out in 1944 how to solve the 2-dimensional Ising model exactly. And I think it's on a par with Einstein's theory of relativity, general theory of relativity, as one of the most fantastic examples of theoretical physics I know of. It is a masterpiece, and it inspired many things, including string theory and all sorts of things like that. There's so much to be said about that. Okay.
So then we knew that there was a phase transition, and it didn't behave the way that you would've guessed, the simple square root theory that we talked about.
Right.
And eventually, we discovered that Well, it's not— as a community, we discovered that being able to solve the collective behavior of matter exactly is a fool's errand. It's much better to have an approximation method that is guaranteed to work on any problem rather than, say, something like the 2-dimensional Ising model, which Onsager solved by absolutely brilliant mathematics. His solution worked. But if you apply an external magnetic field, it all goes away. It doesn't work. And if you try to do it in 3 dimensions, nobody knows how to do that. So these special cases are special for certain reasons, just like integral systems, soliton equations in differential equation theory. If you can solve it exactly, it means there's something special about it.
And there was, and it's now understood. And the renormalization group, why it's taught and why it's so important in graduate physics is, well, we need to know how to solve any Any problem in condensed matter physics, whatever, and I'd say many other fields of science, we now know we can solve them systematically to any order you like, any accuracy you like, using the Vandal-Moses theory. Before I came here to UCSD, I spent 36 years at the University of Illinois at Urbana-Champaign. And one day I discovered, I don't remember how, that Ising was teaching at a university, a college, a teaching college in Peoria about 40 minutes away from the University of Illinois. And I thought, holy cow. And so I wrote to him because apparently he didn't really understand that his name is a revolutionary—
Wow.
Is attached to a revolutionary model in physics. And so I wrote to him, and unfortunately I was a few weeks too late. I could have, you know, I'd been there already maybe 15 years when this happened. And I cursed myself why I didn't do it earlier. But I did talk to his son and exchanged some correspondence with his son and explained to him various things about his father's work and so on.
That's unbelievable. It's like Aharonov works at Chapman University, which is less than an hour from here, and he's still alive. And many, including me, consider him worthy of a Nobel Prize for his work and inspired my late, great mentor, Jim Simons and C.N.
Yang. Just incredible.
It's called Frontiers in Physics: Frontiers on Phase Transitions and the Renormalization Group, and it's written by you. And it's got this lovely cover, and it's got your description. Take us through this cover, the title Frontiers in Physics, and this beautiful cover art, Nigel.
Well, the COVID art is a deliberate British understatement. And when you see a cover like that, you think to yourself, oh, I'm about to walk through a garden full of myriads of beautiful flowers, strange butterflies, and wonderful unexpected sights. And that's actually true. Because one of my colleagues at the University of Illinois, after the book came out, he wasn't in this field, wrote to me and described the book in exactly those terms.
Wow.
And the reason is because this book is, it's still used, widely used as a graduate text in advanced statistical mechanics. And I wrote the book because I thought I had something unique and new to say about the renormalization group, which other people hadn't noticed. There's things in it that you won't find in any other textbook in this topic, including the fact that the renormalization group has nothing at all to do with statistical mechanics. And the first exercise in the book, as you may remember, I don't remember exactly which order. I think one of them might be, the first problem might be to prove Pythagoras' theorem using dimensional analysis. And the second one is to work out the yield of the Trinity test of the atomic bomb—
Right.
Based on just the data from the motion of the shockwave from the photographs that were published in Life magazine.
And famously, Fermi did the same thing. He just sprinkled some pieces of paper.
He sprinkled bits of paper, and Taylor, G.I. Taylor, actually did the calculation that's in the book. And he actually got into trouble because he did this calculation, reported the results in the newspapers, and it was classified information. And so, you know, it says lots of things in this book that are very unusual, and those things have stood the test of time and have actually grown since then in importance and significance. And another thing that's interesting about this book is that it's about a very obscure and arcane problem. And if you want, we can get into it.
Yeah, yeah.
Riemannization group, the problem of critical exponents in of second-order phase transitions. But it turned out that this problem completely upended our view of what physics is, what we're doing when we do physics, and the nature of scientific explanation.
I want to take one more detour before we get too deep into the weeds. And that's this thing that you mentioned before, which had to do with reversibility and the fact that there's no injection or bijection, I guess you'd say, between final state and the initial state. There are many initial states that can produce a given final state. So it's not invertible, essentially.
Right. So we're not talking about states, we're talking about the variable is not time, the variable is scale.
Scale.
And that was the thing that Kadanoff realized. Right. That the way the energy scale at which you look at a system is the important thing. And so I will often talk about the importance of levels of description. I mean, it's a very important problem. For example, if you're a biological physicist, as I am, you might say, well, what is the right level of description to describe a biological system? Should I describe every atom in the biomolecules that are inside a cell, and then inside the cells, and inside the tissues, and so on? Or should I try to make a more coarse-grained description?
Coarse-grained, yeah.
And this is not an easy question to answer, because it depends what is the question you're trying to understand. If you're trying to understand how does some particular molecule bind some particular protein or something like this, you definitely need to understand the atomic level description, the binding and things like this. If you're trying to understand why is it that inside a eukaryotic cell, we now know just in the last 15 years or so that in fact the biomolecules phase separate from the rest of the cell and form a membraneless compartment inside which God knows what happened. We're still trying to understand the function of these things. So you had these sort of phase separation processes. We understand those at a very different level of description. It has nothing to do specifically with the atoms and molecules and the specific sequences of the RNA and things like that. It's a general property.
So depending on what question you're trying to understand, different levels of description are important. And this tension is very prevalent in biology because it's not obvious what is the right level. It's not as simple as saying, well, I'm a chemist, so I'm just going to assume all the microscopic high-energy physics, standard model particle physics stuff is just absorbed into the mass of the proton. We don't know that you can, when and where you can do that in something as complex as biology. Interesting.
You start your lectures in UCSD statistical mechanics. I can't tell if they're graduate or undergraduate because I'm—
They're graduate.
They're graduate.
So— Have you seen them on YouTube?
I do watch them on YouTube. It's Chopin Lover. Is that your channel name?
Chopin Junkie.
But you start the lectures, you use this famous phrase, which I've always felt— I hate to say it, Nigel, and I know he's a hero of yours, but Philip Anderson's, yeah, more is different. I always felt that was kind of simplistic, but maybe I'm wrong. I'm just a dull-headed experimental cosmologist. So tell me, what is the significance? Is there anything really significant about— I mean, of course, like, Where does a sand grain start to become a sand pile? Where do protons— He's more than that. Yeah. So tell me, what does it mean to you? Why is he a hero? Why do you start with that in that lecture?
Why is Anderson a hero? Okay. Many things. He won the Nobel Prize for his work on assorted electrons, but there's no field of condensed matter physics which was left untouched by his intellect. So in condensed matter physics, he is a giant in the same way that Einstein Einstein, Hawking, others.
Bohr, right?
And, well, I'd say more than Bohr, actually. The more is different. The article was immensely influential. First of all, he really was the first person. I mean, there's technical ways in which more is different is important. For example, you can't have phase transitions unless you have, you take the sort of thermodynamic limit. But it's not just that when you have phase transitions, it's just that you can have previously unanticipated complex behaviors that you would never have otherwise expected based on looking at the thing that stuff is made of. So again, let's go back to our fridge magnet.
Okay? You've got your electric— your electrons in the material, and they have magnetic dipole moments. You would have, unless you did a particular calculation, you would never know that this thing could be used to stick your kids' drawings on the door of your refrigerator.
Mm-hmm.
Okay? It's a cooperative effect. It is a conspiracy of the atoms. And I actually do an experiment which we can do right here if you're willing to do it. I'll make a trip to visit. Why wouldn't I? Well, so the experiment is this. Okay? So there's the ceiling up there.
Yeah.
Okay? And we're gonna move the ceiling.
Yeah.
Okay? And we're gonna do it like this. Take a finger.
Yeah.
This finger. Yeah. Okay. Okay? And push.
Well, my ego made it do it.
Come on, come on, keep on throwing. Put some effort into it.
But this is a gas, not a solid.
It's a gas, not a solid, so we didn't move it. The thing is this: the Hamiltonian, the formula for the energy of the gas, is exactly the same as the formula for the Hamiltonian of the solid. They're no different. And yet, when I take this and I push my water bottle, you know, well, my fingers don't go through. All the atoms in this conspire—
Is that really true? I mean, at some level, just to be—
It is. It is.
Van der Waals versus Hooke's law.
No, no, no, it's really true. And this is why it's important, because when we're talking about emergence, remember I said we're talking about new laws of physics. When you have a solid, There are new laws of physics. The atoms have decided that they're not just going to sit at particular sites in a checkerboard lattice that somebody has conveniently laid out for them. They've actually conspired that they're going to keep their relative separations the same.
Yes.
And because of that, they minimize their free energy by doing that. This is a statistical mechanical description of what is happening. That means that you now have new excitations, Mm-hmm. Which are, first of all, you have the rigidity, the stiffness, the emergent rigidity of a solid as measured by the Young's modulus and things like this. And you have the ability to transmit sound waves and other waves as well, of course. So at the level of description of the material, you now have new laws of physics. And the only thing that's changed is the temperature. You haven't changed the interactions between the atoms.
You have changed the correlations. And that's the important thing. But it's a statistical property, and it's not one that you can see just looking at 2 atoms. Sorry, just looking at 2. You have to look at the whole ensemble. So that was a thing that Anderson was very interested in and understood the depth of its significance more than other people. And later, he wrote that essay in 1972, In 1985 or so, when I came to the University of Illinois, the first project I did was with my cousin, Paul Goldbart, who's also a very well-known theoretical physicist and will next year be the president of the American Physical Society.
Oh, wow.
And we worked on this question of why rubber is solid. So everybody knows that rubber is stretchy and can expand and stuff. But But that's not the right question. It's stretchy, but, and a piece of, I don't know, this cable here can bend and deform and stretch, but it's still solid. The first question you should be asking is not why is rubber elastic, but why is it solid in the first place? Because you have a bunch of polymer molecules, they are stapled together by cross-links. It's like having a bucket of worms.
Yeah.
Or a bucket of worms are all flat Flapping around in a thermal equilibrium, you go in and do an experiment that you'd never get an IRB to do, even though a worm doesn't have a backbone, which is staple the worms together in random. And then you find that the thing is not just connected like a fishing net, which would just be floppy, but it's actually rigid.
Yes.
And like a gel. And then you can tap it and it will wobble and it has sound waves and things like that. And those are all emergent phenomena. It's very complicated to calculate them because the whole thing, the polymer chains are going at random, the cross-links are at random, everything is random. It's a very hard technical problem.
Mm-hmm.
But it's an example of this more is different. And the point about more is different and the point about emergence that everybody misses is that when you talk about emergence, something emerges, what is it? And the answer is it is a particular type of rigidity Which is a generalized rigidity, as Anderson called it, which basically technically comes from a certain type of response function of how does the system respond to perturbations when you poke it. And that was the lesson of that. And it wasn't really understood in those terms. And once you understand in those terms, then you can— that lays the groundwork for applying it to more complicated materials and more complicated systems. systems. If I may, I'd like to give you another example of more. Okay? So the most stunning example of more is different is something that all of you know, all the viewers know and use, and that is AI.
Okay? When you ask how, as I do, how is it possible for AI to even work in the first place?
Right.
Okay? Let's think about this. So the first thing you would say is, let's suppose I've got a time series of data points you know, whatever it might be, stock prices, who knows. And you say, well, I want to make a model of that. So the first thing you do is you take your data and you say, I'm going to make a model. It's just a straight line, goes through some of those data points, but it doesn't go through all of them. The data points wiggle and twist and turn. And so the straight line, if you ask, does it fit all the data? Of course it doesn't. If you ask, does it make good predictions? The answer is, Well, you know, not really, because it's too simple a model.
So then you might say, look, I've got, you know, 50 data points here that I'm training my AI on. You know, why don't I just use, you know, a 100th-order polynomial, a much more complicated equation? That equation will fit every single data point that you want to train the AI on.
Every single point.
Every single point. 100th-order. I've got 50 data points. I've got a 100th-order polynomial.
No matter how you embed it, right?
Well, I can find a way I can find, you know, I have a lot of data points. I can find a way to make it fit through every single data point. So there'll be no error in the way that it fits the data. You say this thing is going to be really great at making predictions in the future, except it's not. And it's not because I've fitted the data, but I've also fitted the noise.
Hmm.
And so if you make predictions, they're basically contaminated by the noise. So then you would say, well, okay, so if I have a large error, well, I only have a a linear fit. That's not going to do very well fitting a complicated dataset. I've got a very complicated formula, but that's also not going to fit very well because it's fitted to the noise. Somewhere in between those extremes, there should be a sweet spot where the 2 things balance out, and that should be the place where you should try to make your model. That's what AI should do. And that was the conventional thinking. Okay? And so you say that's the answer, except that's not what happens.
In fact, we have, when we fit our data with modern AI, we are fitting it far more than 100 parameters. We're fitting with a trillion parameters.
Trillions of weights.
Yes. And you're way, way into that regime where you're just fitting noise and the whole thing shouldn't work. That's a mystery. How is it that having such a huge number of parameters can work in principle. Obviously, it obviously shouldn't, and yet it does. And that's a problem that we have solved, at least for a very simple, the simplest sort of non-trivial model of how AI works with a student here, Chan Li.
Oh.
But the answer is that there's a phase transition in the statistical physics of the learning process. And that phase transition has a Rigidity, the generalized rigidity, just like the rigidity of moving a solid, which nobody knew was there because they didn't do the calculation that we did. And so we could understand the transition. We could understand—
Does it have critical exponents? Does it have renormalization phenomena?
Yes, it has critical exponents. It has data collapse, all the phenomena that you'll find in my book. And the phase transition turns out to be very similar to the superconducting phase transition. Ah, I was going to say. And there's a lot that can be said about that. I don't know if you want to talk some more about that. But my point is, this is an example of more is different. Okay? It's not just that, well, you have more things and so you can fit more things.
It's different is the important thing, not the more.
But eventually—
And different means that there is a— it's qualitatively different, not just, well, there's a slightly different number. And it's that phase transition, the qualitative difference, That means that a material or a stochastic computer algorithm, say stochastic gradient descent that is used to train AI, has new behavior when you go beyond a certain point. And that's the thing that's why I think more is different is so important because it's not just Yeah, having more money is better than having no money. Yes, I could buy a slightly better car. There's a qualitative difference that comes when you have, and that's the message of Anderson's article.
But here's my pushback with respect to you and Anderson. Here's some ice. If I told you this is ice that I collected at the South Pole, Antarctica, you'd say, no, it's not, it's water. And if I had more, and actually this Chock-full. I filled it up to the very brim, and then it melted, and now it's this, right? So I'm the one that phased transitioned from the South Pole to San Diego, right? Now, if I keep putting more and more ice in there, more should be different, right? And actually, that's what Anderson's telling me. But if I keep doing it, it's just going to be more of the same. So there seems to be— yes, I agree, more is different. There's a water molecule.
It's not like this liquid in here. But if I doubled the amount of— if I added more and more and more, it doesn't behave quantitatively different from this. less, right? So at what point does the more start to be the same?
It does.
I mean, when it becomes a black hole? At what point?
No, no, it is different. So the amount of water and ice that you have in there, if you measure that, you've mentioned the critical exponents, like say you have the heat capacity divergence, it's a first-order phase transition, so you don't have critical exponents. But let's suppose we were talking about, say, the magnetic transition. So yes, you would find that there's a temperature where the, say, the divergence of the heat capacity, which you'll see in an infinite system, it'll literally go to infinity.
Yeah.
When it's a finite size system, it won't diverge. It will start going up and then eventually it'll smoothly go over. And that's important because you literally see that in, say, granular superconductors. And if you look at machine learning as a neural network, in the ideal case, where you have an infinite number of neurons and infinite numbers of datasets and some appropriately taken asymptotic limit, you can make a very sharp mathematical theory for that. And only in that case, mathematically, do you literally have the ability to say there's a phase transition and non-analytic behavior and so on. If you, on the other hand, make the system be finite, then the computer scientists would call this ridge regularity. ridge regression or regularization, then in fact this infinity goes away and the behavior is different. There isn't a transition.
Hmm.
You won't be able to see that if you put more and more ice in there. If you're doing this with a magnet or you're doing it with a superconductor, you can do the experiment and you literally can see that only when I go to infinity do I see the sharp phase transition. But, you know, I describe this in my book. If you get to within, you know, 10 to the minus 12 degrees of the critical temperature, then you'll start to see the fact that you don't have an infinite number of atoms in your water bottle. Interesting.
I want to talk about a man you mentioned in your course as well, and you mentioned with great glee that the only man to win 2 Nobel Prizes in physics was a condensed matter physicist.
John Bardeen.
John Bardeen. I often hear it said that if it wasn't for the laws of quantum mechanics, we wouldn't have had the transistor. And I always have a little bit of problem with that. Because if you look at the first transistor that they built, Shockley, it was—
Shockley didn't build it. He just posed in the photographs.
Right.
Yeah. In the famous photograph, he's sitting down as if he built the thing.
It looks like a piece of copper wire.
The whole team in Britain was strutting around like, why are we here? Why is he there?
Why is he there?
They were walking into— that actually was the reason why Bardeen Bardeen left and went to Illinois. He was so— he just couldn't get on with Shockley.
Yeah. Well, Shockley was a very, very troubling character, as I've talked about. But the question is, you look at it, it's a piece of chewing gum. There's a coat hanger stuck into it. There's a rock in the middle of it, right? It's very unlikely that you'd say, hmm, this is the solution of the Schrödinger equation with Fermi levels. And do you believe that, that we look into the laws? Because the reason I'm asking is people say, when we have a theory of everything, Nigel, They'll be able to look into it, and just like they did with quantum mechanics, instead of making transistors, we'll make warp drives and gravitational impellers and multiverse teleportation devices. Well, what do you make of this? First of all, is that historically accurate? And you've seen a lot of these people, you interacted with the Titans, you are one of the Titans. But tell me, Nigel, did we look into the laws of physics to get the technology on the screen that you talk about in your course? Is that really what happens, or do we describe it later on? on post facto by these laws that we discovered.
So just with the fact that the first transistor was a big lumpy thing. I mean, we said before that everything in this room is classical, but you knew that I didn't really mean that. I mean, look at the flowers there. They have color. The only reason that they have color is because of quantum mechanics.
Well, these are made of plastic. But anyway, if they were real—
Oh, in that case, you gave up. You blew my secret.
I'm not a real biologist, right?
I can't even tell a plastic flower from a real one. But you said it did. So yes, it was a macroscopic object just like your iPad is, but it's operating due to laws of quantum mechanics. And so yes, semiconductor electronics. It's not like before we understood semiconductor electronics, we could build iPads. And this thing didn't exist 15 years ago. In fact, we didn't even know enough about the liquid crystal displays, let alone the electronics to go into it and so on.
I want to give you a quote from a countryman of yours of some renown who said, I'm very poorly today and very stupid and I hate everybody and I hate everything. I'm going to write a little book for Murray on orchids and today I hate them worse than everything and I hate species as well. Oh my God, how do I hate species? Do you know who that was? that British gentleman of some renown whose father told me, or whose father has said about him, you care nothing except for shooting dogs and rat catching, and you will be a disgrace to yourself and to all your family. Who was that said about?
Darwin. Darwin.
So this man, you know, loved life. He created these ideas, and he was—
he was—
he's such a fascinating character. It's reputed, and okay, you're gonna disabuse me of this. Again, I'm a poor experimental cosmologist, Nigel. that you have seen and you were part of the group or team perhaps that is really working to maybe state the limitations of Darwinian or the restrictions on selection. So let's talk about why selection, why is biology— you talk about your paper with Woese, is it Woese?
Carl Woese.
Woese.
Life is Physics.
Life is Physics.
Yes.
Is that right? I mean, besides your blunder about this little plant.
Well, it could be worse. I mean, I could have said out of that spherical thing there was a cow.
What relevance does physics have in biology? You make the point in your course, again, everyone should watch your course because it's so enjoyable and easy. It's a graduate-level course, but let's be honest, you could take it as a freshman if you're energized and you're willing to do the work. You may not get the highest grade, but you talk about how easy physicists have it compared to sociologists and what you call it and what has been called the dismal science of economics. I just had Alvin Roth, who won the Nobel Prize in Economics, a few years back talking about repugnant markets. It sure seems easier to do that than to do cosmology and try to figure out what happened 10 to the minus 32nd seconds after the Big Bang. So tell me anyway, what does physics have to do with biology? And what role do you play in perhaps overthrowing this irascible kind of self-loathing man named Darwin?
So a lot of people did interpret our work as being against Darwin. But that's completely wrong.
Okay, say more. Okay.
The whole idea of, well, you know, I'd say it's Darwin and Alfred Russel Wallace. Wallace really was the first person who, you know, wrote the paper that was presented at the Linnean Society, and Darwin added his things to it and so on. And the correspondence between them is very interesting and revealing. But let me just say what people mistakenly are referring to. So the usual picture of evolution that people who are not necessarily deeply into biology think about is this. They say, well, you've got your genes, and then you transmit your genes to your children, they transmit their genes to your grandchildren, and so on and so forth, and the genes propagate like that. And that is indeed what happens. But there's a very fundamental problem.
We're now going to talk about what it is that we actually did, and then we'll talk about whether it's against Darwin or not.
Yeah.
Which is Just to be crystal clear about that.
Right.
So then you might ask the following question, as Francis Crick did, another one of my compatriots. So you might ask yourself the following question: could the genetic code evolve? All right? So let's think, what is the genetic code? So just to review some very simple biology, you have proteins that do lots of stuff in your body. The proteins are made out of amino acids. How do you know which amino acid to put into which protein. So then you read your genome and you read sequences of nucleotide bases, which we'll say U, C, A, and G. Those are the sort of abbreviations for their names.
Right.
And then you read those and then you read them in triplets. And then you take each of those triplets and if you get UUU, you get phenylalanine. And that's the amino acid that you then put at that position in the protein that the ribosome is building in every cell of your body. And the map that tells you, take triplets of nucleotides and convert them into one of the amino acids of life, the 20 amino acids of life, that's called the genetic code.
Yeah.
So it's not your genome. People always say the genome is your genetic code. That's not true. So the question is, well, where did that map come from? There's a very interesting feature about this map. It's called the genetic code. You can write it on a t-shirt. It's a many-to-one code because you've got your alphabet of 4 letters, words are 3 letters long.
Sounds, yeah.
So I've got 4 times 4 times 4, which is 64 possible amino acids I can get. But in fact, we only use 20. So you might say, well, why 20? Actually, Francis Crick had an answer to that, which I can tell you if you like. So there's obviously redundancy in this code. So then you ask, Well, when did this code— when was it developed? So you go back and you do what's called molecular phylogeny. There's ways that Carl Woese was the first person to develop to look at molecular sequences and then find what they were descended from, and therefore you can work out the evolutionary history of all life on the planet.
The LUCA, the Last Universal Common Ancestor.
That's right. You get to the Last Universal Common Ancestor where—
So he coined that or did he coin archaea?
He discovered archaea.
He discovered archaea.
So he started doing this thinking that there's prokaryotes and eukaryotes. And then one day he discovered that these things that are prokaryotes, they're not prokaryotes. There's something else in there. What the heck is this? Okay? And that was a methanogen that he'd— Woese was doing these experiments, they're very dangerous radioactive experiments. He was doing them virtually alone for 10 years. Everybody thought he was off his rocker. And his goal was to simply find a way to map out the evolution history of life on Earth. And he discovered a whole new domain of life that people just looked at under microscopes and say, oh, this is a round blobby thing.
It must be a bacterium. Turned out to have completely different evolutionary history from that. And in fact, we are descended from the archaea, we now know. He was doing this. And as you say, once you start building these trees, you eventually discover that you can build them all the way back to about 3.8 billion years ago. And that's the last the universal common ancestor of life on Earth. And there's various converging evidences that give you that number, 3.8 billion. And some people say it's even earlier, maybe 4 billion years ago.
Here's the interesting thing. How old is the Earth?
Well, I want to take a segue because I forgot to give you your gift. I'm talking about magnets. Here's a magnet.
Okay.
And here's a magnet with some gifts on it for you. So these are pre-Earth meteorites. These are discovered in Argentina. Those are yours to Thank you. On the Into the Impossible podcast. So the Earth is about 4.2, 4.3 billion years old. These are 4.35 billion years old, so they're quite a bit older, but they date from the pre-super— the supernova that blew up, which by the way was the mechanism by which was discovered by more countrymen of yours, one of whom occupied this office, Jeff Burbidge, and his wife Margaret. So I have Margaret's plates.
These are her photographic plates from Palomar. So we have a lot of things in common. But yeah, so the Earth is is older than that, but not by much. I mean, life began very early.
That's right. That's right. So whether it's 4.3 or some people say 4.5, something like that, as you say, it's very close. And so the thing is this, we know because we can do the molecular phylogeny back to that last universal common ancestor that essentially the architecture of the modern cell was already in place 3.8 billion years or so ago.
Yes.
So wait a minute, you're telling me that life went from nothing 4.5, whatever, billion, plus or minus billion years ago, of which half that time the Earth was completely uninhabitable, this Hadean, right? And then by 3.8 billion years, you've developed the machinery for replication and—
For our first ancestor.
Yeah. Yes. All of that. And somebody, you know, looking at the organisms around about that time, you would see very little, relatively little in the sense of the global architecture of the cell different from now. And so the question is, how is it possible for life to have evolved so quickly? So that's the first question. And Francis Crick was very perplexed by that. Second question. Second question.
Why is there only one genetic code?
Yeah.
Okay? We call it the canonical genetic code, and there's minor variations, mainly to do with stop codons, but it's basically the same genetic code. Then there's a third one, which you probably knew that there was only one canonical genetic code for all life on Earth. But the other thing you may not know is that the genetic code that we actually have is optimal in the sense that it minimizes errors of translation.
Hmm.
So let's suppose we were in the world of intelligent design and being particularly provocative here. So you say, okay, Brian, okay, you know, smart guy, you know lots of things, design for me a good genetic code. And you would say, well, if I'm gonna design a good genetic code, I know there can be lots of errors in reading and mutation.
Yeah, there's some redundancies. 64 minus 20.
So, well, not just the redundancy, but I should make a code so that if you get the wrong amino acid, I should make it so that the amino acid I do get is in some appropriate biochemical way, which has to be defined, is a decent approximation to the one that I should have got. So it doesn't do too much damage so that the protein has in it the wrong amino acid, but it can still fold and do the thing that the protein is supposed to do. And if you could create such a genetic code, you would say, well, that's going to be really, really good. That would be the one that I, as an intelligent designer, would choose. Okay. So then the genetic code, when you when you analyze it, you can do this calculation. There's many different ways you can do it. Every time you do it, you get the same qualitative answer.
The genetic code is optimal in the sense of minimizing errors. Okay? It's fantastic. Okay? So those are 3 facts.
Mm-hmm.
How on earth could all of those things have happened? Now, Francis Crick was very perplexed about this. Francis Crick said, look, there's no way that life could have It got to this level of complexity in such a short period of time, it must have come from outer space. So eventually he embraced the panspermia idea, which of course then just pushes the problem off to another wrang.
The origin of life on Earth is solved, but not the origin of life in general.
Exactly. Exactly. But in fact, there's more to the problem than that. There's these other 2 facts that I've talked about. Francis Crick was also very perturbed because, as he argued in 1968, it, the genetic code can't possibly be something that evolves, right? Because think about it this way. Suppose it does evolve. So think about this. Think about we're doing this experiment, okay? I'm communicating to you in code, okay? And I write down my coded message and you get the coded message.
You use a code book to translate the message. So that works fine. Let's suppose halfway along in us doing this and we're separate continents or something like this, I unilaterally decide I'm going to use a different code. Okay? Suddenly all my messages are going to stop making sense. You won't be able to interpret them.
Right.
So the code book is the genetic code. It literally is. It tells you how to translate from the message that is in the DNA and the mRNA into the protein that you're ultimately going to produce. And so obviously, if you evolve the code, which means change it midstream, then it won't— then there's a whole— then you'll start getting the wrong proteins and then everything will die out. Okay? So it can't evolve. What we did in this paper was we figured out how to solve all of these 3 problems. We figured out why the genetic code is unique, why it is optimal, and why it evolved so quickly. And in fact, that it really did evolve.
Obviously, the fact that it's optimal, which Francis Crick didn't know, the fact that it's optimal Either you think that it was intelligently designed or it evolved under selection.
So in what sense is this a canonical critique of Darwin? I mean, why do people even say that?
Well, I'll tell you why. Why do they say that? It isn't. I'll tell you why. Because what we discovered was that indeed these things would not have happened if you had just— were just using the vertical evolution that we talked about at the beginning. You give your genes to your children, they give their genes to your grandchildren, and so on and so forth. If that was the process operative at the dawn of life, it wouldn't have happened this way. But in fact, what happened was horizontal gene transfer, namely that genes can be transferred between organisms that are not related. For example, let's suppose we could do this.
Okay? So let's suppose you decide that you want to learn renormalization group theory from my book. So you could slog through my book and go to my classes and so on, but wouldn't it be easier if I could just pop out the gene that enables you to do Feynman diagrams in 4 minus epsilon dimensions? Let's suppose there were a gene for that, which of course there isn't. And I just give you the DNA, and you just take that DNA, put it into your DNA, and all of a sudden, bingo, great, I know how to solve the— I can solve Feynman diagrams in 4 minus epsilon dimensions. You will be I can compute pretty quickly. Okay. It doesn't happen like that for us, but it does happen in the world of microbes. That's how antibiotic resistance, for example, is transmitted so rapidly. And the reason it happens so rapidly is because when you're transmitting genes in this particular way, you're using a network effect.
I can distribute my genes not just to you and not just to my one or two children. I can distribute— I've got two children, Exactly. You can distribute them to hundreds of thousands. That's how libraries work. Libraries do this. It's a Lamarckian mode of evolution, but it's still evolution. In other words, only the books that are actually good end up in the library. Only the right physics books, the physics books that tell you the actual right physics, the storybooks that are actually really entertaining.
So you have a network process, a horizontal gene transfer process, which is different from your traditional view of how genes are transmitted vertically.
I have to interrupt. Sorry to interrupt, but I have to. It seems to me you're taking a PowerPoint file on a modern SSD drive and then putting it into a Windows 95 computer from 1995 and somehow it's working. How is that even possible? You just get gibberish. You'd get these glyphs. How is that even possible?
So you have to ask what happened at the dawn of life. At the dawn of life, the genotype-phenotype distinction had not yet really been clear. The organisms were very porous. They underwent endosymbiosis. That means that they would absorb one another and then the stuff inside that, hey, I can take all the stuff. And we know that that's where our mitochondria come from. That's where chloroplasts come from in the plastic flowers. And things like this.
It may well have been.
So life did that and life transmits, exchanged genes in that way. And today organisms do this. I mean, if you sequence the Drosophila genome, the fruit fly genome, okay? The eising model of biology, if you will.
It's like Harvard is the UCSD of the East Coast.
You'll find in it the genome of Wolbachia. It is a parasite, a microbe, a bacterial parasite of Drosophila, and it has inserted its whole genome, actually multiple times, into the Drosophila genome. And there's many other examples of horizontal gene transfer. If you look at the phylogeny of flowering plants, angiosperms, very, very complicated. It's not like a family tree, it's a network. And what we discovered was that the early life evolved through this through a network effect, which Rose called that state of life the progenote. I don't know why, but he did. And then there was a phase transition to an era of vertically dominated evolution.
And when we're talking about what is evolving, we're tracking the genes, specifically the genes that code for the architecture of the cell, the fundamental cellular processes such as translation, replication, and so on. So So that's how we define species today. And so today we build the tree of life, but there's nothing mandatory that says it should be a tree. And in fact, prior to the last universal common ancestor, it was a network. And as it was a network, it evolves faster, but it is still doing Darwinian evolution. Or what is Darwinian evolution? It is still survival of the fittest or all of that, however you interpret that.
Selection.
It's a complex argument in and of itself, but basically It's fundamentally, we're not saying anything different about that. We're just talking about what is called the mode of evolution.
One question about the network. Does it exhibit things like Rescham's Law, but the network law that the scaling goes geometric and the reason that it's so fecund is because of this network dynamics that lately we've learned about with social graphs, but in fact we can understand it maybe how successful it is via network theory rather than, you know, just pure genes.
Yes. I haven't personally done a network analysis of what kind of network you get from this specific process. I mean, I think the more interesting thing is that there is a network effect. And the thing that Crick had missed and other people had missed was that, I can explain by a kind of analogy. Let's suppose that I drive over to your house and the wheel comes off my Toyota Corolla while I'm there. I say, well, that's too bad. You say, well, you know, I've got a Tesla in my garage here. I don't know if you have, whatever you have, you know, why don't you just take the wheel of that? Well, obviously that's not gonna work.
Okay? But let's suppose we were doing that, say, 120 years ago, right? It was the dawn of the age of automobiles. So I drive over to your house with my jalopy whose top speed is like 20 miles an hour or something like this, and the wheel comes off and it's broken and so on. And you say, well, look, I've got a bicycle.
My Model T.
I've got a bicycle. So to take the wheel off my bicycle and stick it on, so I just get out, take the screwdriver out and screw it on, and I'm good to go. And you can do that because the very early primitive forms of an automobile are very, you know, you can just swap things in and out. The technology is not very advanced. It's not 6-sigma precision and things like this.
6-sigma.
In the early days of living systems, they were very simple. And so they could tolerate ambiguity in the proteins that they use. As they became more and more complex, then you really had to have just the right protein to fold in just the right way to be able to make the thing that goes into your neurons or something like this. And so what we realized was that you can build a dynamical systems model of the coevolution of the complexity of the organisms along with the evolution of the genetic code. And so you find that then through this network effect, it evolves very rapidly and eventually gets to the point where it shuts off the network effect and then transitions to the vertical era of evolution we're in right now.
So does that make you more or less sanguine, you know, getting back to Fermi's question to our late, great colleague, Herb York? where are they? Where are the aliens? Are you more— I mean, knowing this level of kind of punctuated equilibrium, you also quote Gould and the fact that we don't actually have that many more genes or anything productive compared to a worm.
No, that's pretty much the same. And if you really wanted to punch a hole in more is different, you would say, well, that can't be true because the same number of genes uses But it's, as Gould wrote in his year 2000, whatever it was, New York Times op-ed piece, there's many more interactions between the units than you have in C. elegans. C. elegans is so simple that we know every single— Right.
Neuron hitting another neuron.
Every neuron is mapped. It's not the case for us.
Yeah, exactly. Right. If we spray C. elegans throughout, you know, on the planet Mars, it's different than spraying koala bears on there. Where does this leave us?
Tardigrades would adapt.
Tardigrades. Well, they're already there. I mean, there's human poop on Mars right now. I guarantee it, because the astronauts are spraying out— they vent it out to space, and it eventually gets to— I have a piece of the moon here. This is a meteorite from the moon, so stuff is striking and we're sending—
There are tardigrades on the moon.
Yeah, exactly. So I'm sure they're on Mars. But tell me, Nigel, does this make you more or less sanguine about life elsewhere in the universe? Forgetting or pausing for now the concern, the origin of life generally, but just origin of life specifically on other solar systems, in other solar systems.
I tend to believe that life is the inevitable consequence of the laws of physics, which we understand imperfectly. And I say physics, not chemistry, because I don't think that life is restricted to particular chemistries.
It could be silicon-based or it could be Different genome?
Well, I have a question for you about that, which would you be the ideal person to answer. I do think that it is a physical process, and I can even say a little bit more about why I think that. I would say that if you wanted to know what is the purpose of life, what is the meaning of life, if you like, what is the purpose of life? The purpose of life is to help planets come into equilibrium.
How so?
So think about a planet. A planet after it's formed has a huge variety of chemical potential redox gradients in its environment. And those gradients will eventually relax and homogenize as they should through second law of thermodynamics and all sorts of other good reasons. And that happens. And what life does is life uses information to find new pathways to short-circuit, if you will, those chemical potential gradients and use the energy to power life. And that's how ecosystems work. Ecosystems compete with abiotic processes to literally take chemical potential differences and use the flow of energy in them to make living things. And those living things are powered by this chemical potential gradient.
Hmm.
So life uses the information just In the way that I was saying with the horizontal gene transfer, that's one very fast way of searching a space and finding new ways to solve the problems that emerge, the organizational problems that emerge. And we know that that happened. By the way, there's lots of supporting evidence for our horizontal gene transfer theory. And there's a recent paper that just came out in the journal Astrobiology, which is a sort of review is not one that I wrote it with other people, but it's looking back on that and there's even other data which supports this theory. But the point is, that's what living systems do and there's nothing special about doing it on Earth as opposed to Enceladus, which would be my favorite place. Well, I used to direct a NASA astrobiology institute.
Right.
And I tried very hard before NASA disbanded the whole NAI program, Sadly, to persuade anybody who would listen that the place we should go is not Europa. We should listen to 2001: A Space Odyssey and give Europa a miss. Go to Enceladus because there you've got a much, much better chance. And we already know from the Cassini mission that, you know, you can sample already the water that's there and it actually looks like alkaline hydrothermal vents. So there's all sorts of interesting astrobiology that could be done.
That sounds amazing because it seems to me It's closer to answering Schrödinger's question than Fermi's question, let alone that.
Yeah, it is. Carl Woese and I wrote a review article called Life is Physics. And the reason we wrote it like that is the following. First of all, Schrödinger wrote this book called What Is Life, which of course everybody's inspired by. And then the other reason is that when I go to astrobiology conferences, you know, on the first day somebody will stand up and say, well, life is chemistry.
Right.
But I don't agree any more than I think that a computer is— you know, if you asked me what a computer was in Victorian England, I'd say, well, it's Babbage's machine. It's built out of cogwheels and springs and levers, and you sort of turn things like this and it will compute.
And then Lovelace.
That's right, exactly. He's a Lovelace. And then you go and ask, you know, an engineer trying to figure out how to design hydrogen bombs at the Institute for Advanced Study in the 1950s. And he'll say, well, it's John von Neumann's building. It's in that shed over there. It's lots of fermionic valves and relays, and that's what a computer is. And now you ask—
NASA and the computer, Katherine Johnson.
You ask me, you know, 20 years ago, or anybody, you'd say, well, it's my iPad, it's my Windows, my MacBook, or my Windows computer. And you ask somebody today, what is a theorist? My phone, or my glasses, glasses or something. There's a difference between the substrate in which something is made and what it actually is. And so when we think about trying to understand the fundamentals of living systems, of course, if you want to know how to make somebody better because they're ill for some disease, well, you better understand something about biochemistry for sure.
Yeah.
But if you want to understand why is there life in the universe, why does the phenomenon of life even exist? That is a fantastically profound and interesting question. And we don't, truthfully speaking, I don't feel that we know the answer to it. I think we make steps towards the answer, but I think the answer is it is a physical process. It can be realized in certain types of atoms and so on. But here's a question for you. Could life exist 3 minutes after the Big Bang?
Depends on what you call life. I think the universe did transfer through a period of time when water was liquid. The CMB was once at 300 Kelvin, right? So there's no, you know, that wasn't, you know, very that soon after the Big Bang. I mean, I think in terms of atoms forming in 380,000 years.
Right.
So it's like very, very implausibly, but perhaps as Deutsch says, you know, if it doesn't violate the laws of physics, perhaps.
So let me tell you a science fiction story. Okay? It's not meant to be real. It's meant to be a thought experiment, which is meant to raise your consciousness. Okay? So I'm going to make the the following claim, which I emphasize, this is not a scientific statement, it's a thought experiment, okay? That you have at that early stage of the universe, we're way above the physics that applies to the standard model of particle physics as we know it now. And you've got some, I don't know, non-Abelian gauge fields or some God knows what gauge group or some strings or something like that.
Mm-hmm.
And they have non-Abelian flux tubes that go between whatever the quark-like excitations of this thing. And those things are non-Abelian, so they can wrap around, they store information, just like we want to use non-Abelian anyons to build quantum computers and store information. And so you could store information in this way, and then you could have, you know, the chemistry of these objects. And so you could imagine you could build self-organized objects that are built out of, you know, non-Abelian flux tubes. The thing is that they're on a scale of like 10^-20 meters. And an energy scale of, you know, 10^100 GeV or something like that. And they last for 10 to the minus—
Planck time.
Planck time or something like that. But you could imagine that. And you could imagine those beings sitting around in their non-Abelian gauge theory bar drinking beer and saying, you know what, do you think life could exist, you know, I don't know, for 13, 14 billion years after the Big Bang? And they say, oh, come on, don't be so stupid. I mean, they'd have to be absolutely enormous. The scales would be enormous. And the timescales, don't even ask me about the timescales, they'd be just huge. And the energy scales would be pathetic. You know, what would you do?
Ridiculous.
You know, it's a completely ridiculous suggestion. Of course not. You know, so I think it says that, you know, when you think about what is the physics of life, the processes that are involved in creating the phenomena of life. You know, they're on a sort of logarithmic scale of energy and time and space and complexity and so on and so forth. And when we talk about life, we usually mean life like us. But if you want to ask about life that's not like us, well, why not? And I certainly think that you're going to find microbial life long before you find—
Dolphins with iPhones.
Dolphins with iPhones. Yeah.
Yeah. the embodiment of the moor is different to me. I mean, I'm sure the listeners can determine for themselves the vast kind of depth and breadth that Nigel engages in. But you're also a citizen scientist in the kind of tradition of our late, great colleagues like Herb York and Roger Revelle and many others throughout history and throughout different continents that you've lived on and you've experienced this. But now I get sense, for someone as cheerful as you are, I consider you a very optimistic pessimist, and you're seeing things, and you and I have spoken offline about the kind of precariousness of the age that we live in. I want to ask you, first of all, is it our fault? Katalin Karikó, co-inventor of the COVID vaccine, sat in that chair, and she told me that we sort of have this inflated view of scientists, and actually we're quite egotistical. And she went through kind of the negative side of academia. How much of it our scientists to blame ourselves.
I don't want to make it sound like we are even a very large fraction of the blame, but you hear nowadays, and we're talking now, Fauci's going in front of Rand Paul and there's this big theatrics. I think it's all nothing, Burgum. I don't think anything's going to happen. But I do feel like we're living in an anti-science age, but not because of the reason that everyone has their truth. I don't care what you believe. I don't care what you do in your private life. But if we have different epistemologies, That's very dangerous, right? If you and I have different ways of discovering what is true and characterizing what is true, you may believe that 9/11 was an inside job and that fire doesn't melt steel and whatever. And I may believe that, no, no, no, actually it didn't have to melt.
So we have different ways, at least we have the same epistemology. We're using science in some way. I'm not saying that those people aren't crackpots. But Nigel, are we living in an age that's not post-truth? It's relative truth. renormalized truth where you can believe whatever you want, Nigel. I'm going to believe that, you know, that there's some nefarious conspiracy and I have different ways of getting to my truth. What do you make of this age and who's to blame and what can we do? Sorry to wrap 3 questions in one, but what's going on here?
I would say that the, you know, I've been very outspoken and active in trying to defend science in the United States over the last Year and a half or so.
I would say longer, just to give credit. I don't think people realize the role that you played in the 2020 COVID kind of pandemic and bring quantitative. I mean, you were really at the forefront of being data-driven and predictive using a lot of your models, which we didn't have time to talk about today, but we'll do a part 2. And I think that's remarkable. So I don't want to say the last year. I don't want to say it's just Trump. I do think that there are other factors at bay, but he's He certainly plays a role that's unique now.
Yes, but I think that's the biggest threat that I'd like to talk about. Most of my activity has been really to defend the public interest because I do feel that what is happening to science, and is happening to science, is not being done in good faith. As a member of the National Academy of Sciences, a fellow of the Royal Society and so on, I feel that it the job of scientists to speak out and to try to work with Congress, in the case of the United States and so on, in order to make sure that they have the best interests, the best information. And that's how the National Academy of Sciences was founded by Lincoln in 1863 during the Civil War. And so a lot of the work I've been doing, you know, has been work that the academy itself could not do on its own for various reasons, which I won't go into, but is now becoming actually in in the way that it possibly can, much more visible in the public eye. So much so that Donald Trump is literally issuing posts on social media about defunding the National Academy of Sciences.
Awfully, yeah.
Beliefly, just after the World Cup or during the World Cup. So I won't say too much about what I've been doing, but I want to talk about why I think it's important. And the reason is this. It's not that I feel, well, I want my money, I want my lab, I want the money coming in that supports the research. That's not the important thing. The reason we do science is because it is in the public interest. If people like me, I'm actually working on cancer at the moment, trying to understand how cancer works. I'm very excited about some work that we've done.
Wrote an $11 million grant proposal to NIH from here, which will probably never get funded. But the reason it's important—
Well, Jay Patachariya, who's the director, is a friend, and he sat in that chair too. So maybe we can talk to him.
Well, just maybe. Anyway, but the point is, it's not a question of, is my personal hobby funded? The question is, we are doing this for the public good. All the technology that we have, that we hold in our hands, our silicon and security blankets and the medicines that will help us live longer and all those things came from the scientific process. And I believe, as many do, that this is in peril at the present time in this country. And it is our duty and it's the duty of people who engage the public like yourself to make those things crystal clear to people so that People understand what is happening, who stands to benefit from this, and why it is not in the public interest. Is the scientific system perfect? No. No system is. Scientists are being attacked in the media.
If you poll people and ask who are the most trustworthy people, politicians are right at the bottom. Scientists and teachers are right at the top. You shouldn't be weaponizing the inevitable flaws in the system, like peer review. Is peer review broken? Is it really true that we've stopped innovating in science? Which I think is complete and utter nonsense. No, no, it's complete— But that's the rationale that's being used by Michael Kratsios in particular, the director of the Office of Science and Technology Policy in the White House, for saying we need to take science away from the universities, put it more into industry and so on towards—
Or target individual scientists, right?
Yeah. but none of which makes any sense.
Or benefit AI. AI is science now, according to all the David Sackses and the advisors to the president, and it's very troubling.
So what is alarming is that these are interesting discussions, but they're not being held in isolation by disinterested parties arguing in good faith. And I think that to me is a problem more than someone who difficulty getting tenure and then ends up winning a Nobel Prize and so on. Yes. We know that there's examples like that. And I was actually very lucky spending the first half of my career, the first 36 years, at the University of Illinois, where I was essentially working on the lunatic fringe end of condensed matter physics. But people like twice Nobel Prize winner in physics, John Bardeen, said, okay, I, you know, I support what you're doing. They gave you the encouragement and, you know, yes, go ahead and do this. You know, I was working on high-temperature superconductivity, and I had a view on it that was completely not shared by anybody else in the community for 5 or 6 years until eventually we could prove that it was correct.
We named it the D-wave, as a D-wave nature of the superconductivity. John Bardeen was the first person who says, that is wonderful. And he gave me moral support and told people we should follow this. So I do know what it's like to be an outsider.
Yeah.
But I've been very lucky that I've been able to do enough things that are sort of mainstream, as it were, that even though I don't stay in my lane, I've been supported. And it's not true of everybody. I think if I'd been at another university, We wouldn't be sitting here talking now. My career would've been very different.
I think you're absolutely right. And to use a phrase from the namesake generator of this podcast and the namesake generator of the word podcast in general, Arthur C. Clarke, he said, any sufficiently advanced technology is indistinguishable from magic. I want to ask you 2 questions kind of as we close that are prompted by him. And that's the first one. What is sort of the most magical— I mean, we talked about so many marvels. things today. And literally, we've scratched the surface.
I feel like you and I could talk for hours, and hopefully we'll get another chance.
Yeah.
You're the second condensed matter physicist from UCSD Physics Department after Jorge Hirsch to come on, and he's been a 2-time guest, so you have to be a multiple-time guest. Can't let Jorge have all the fun. But Nigel, tell me, what is the most magical? If you could put something on a monolith and launch it into space for 4 billion years, uh, what would it be? What would encapsulate, as Feynman said, the, the most information in the fewest lines of of text or code?
Well, Feynman's answer to that was that atoms exist. And I guess my answer would be more is different. Because it's not enough just to say that atoms exist.
Very good.
Everything that we've talked about are emergent properties of different levels of description and so on. And I would say, you describe me as a condensed matter physicist, and that's where my intellectual roots are, but I work in astrobiology and evolutionary and fluid mechanics and all sorts of other But I think it's the recognition that there are emergent phenomena, which I think is not a philosophically obvious thing. And if you don't know that, then many things in the universe are far more perplexing than they would seem to be. So I'll give you an example. Humans try to figure out how the world works. And so we came up with one answer, oh, there must be a God that makes everything do the things that it does. I'm a practicing Jewish atheist, okay? And I don't believe in God, but I do think that you see in, you know, what you see in society and in the world around you, phenomena that are seemingly inexplicable, the hidden hand as Adam Smith called it about—
Capitalism.
Economics. But you see the same thing in all aspects of human life. And you might say, well, that's really God. I'd say, well, it's an emergent aspect of things. And I think this sort of motif, really, the smallest difference, as you brought it up, that really does have many, many ramifications beyond the most trivial ones. So I think that would be my answer.
That would be that. Okay, last question.
But anybody who's sophisticated, you have to be able to read it, whether you know it. That's right.
And says he couldn't see this monolith. Put it in a little satellite and send it off into space. And they get 2 monoliths. That's even more exciting. Arthur C. Clarke said, the only way of knowing the limits of the possible is to go beyond them into the impossible. That's the namesake of this podcast, the name giver of the podcast. If you had 20 seconds with a 20-year-old Nigel Goldenfeld, what would you tell him? What would you tell him to give him the courage to do what you've done, which is to be a remarkable scientist, but a citizen scientist as well?
I would just say that you can do this. I don't think it's true that you have to be a genius to do good science. It might help, it may not help. It's not obvious that it does. It depends on how you approach things. Einstein, who I think could have won 7 Nobel Prizes, I can list them for you. I don't know that he was smarter than anybody else, but I think he had a better algorithm and a better approach. And so I think the question I always ask myself, and I ask other people this, I ask other scientists I meet, how do you choose problems you work on? What is the way you decide what to work on and what not to work on? And so I think that's what I would—
It's a matter of taste. Yeah.
It's not a matter of taste. I don't agree with that. It's a matter of how you can make the biggest impact and increase the likelihood of making discoveries. If I could, in more than 20 seconds, I would say it like this. The impact you make is the ratio of what you do divided by what everybody else does. And the usual algorithm that people have is, well, I'll try to maximize the numerator, but that's limited by things like how much funding you have, which university you're in, what facilities you have, how smart you are, your family circumstances, a million other things. But the better strategy is to minimize the denominator. Don't work on something.
I don't work on anything if I think that if I didn't do it, somebody else would do it 3 weeks later.
Right. Yeah. You only work on— that's my philosophy of writing books. I only write books that only I could write.
That's right. When I talk to students, I often try to give them— when they ask me for advice, I tell them this, do something different. It's not that more is different, it's that different is more. That's beautiful.
We just said the title of this episode. Nigel Goldenfeld, so proud and happy to have you as a colleague. And I have a question about the Ising model applied to cosmology, which I'm going to run by you on the blackboard outside.
But Nigel, thank you so much for Thank you very much for your interesting questions and for having me on your show.
Hopefully this will be part one of many, of more, many more. Thank you.
Also generated
More from this recording
🔖 Titles
The Physics Behind Why Artificial Intelligence Works When It Should Not
Unveiling the Science: Why AI Succeeds Against All Odds
The Phase Transition That Explains Surprising Power of AI
From Magnets to Machines: The Physics Making AI Possible
More Is Different: How Physics Reveals the Secrets of AI
Emergence, Phase Transitions, and the Mysteries of Artificial Intelligence
Exploring the Unexpected Physics That Makes AI Work
Critical Exponents to Critical Problems: Physics Lessons for AI
The Renormalization Group and the Unlikely Success of Artificial Intelligence
How Statistical Mechanics Explains the Wonders of Artificial Intelligence
💬 Keywords
phase transition, renormalization group, Ising model, condensed matter physics, critical exponents, magnetization, statistical mechanics, scale invariance, emergence, more is different, horizontal gene transfer, genetic code, molecular phylogeny, last universal common ancestor, optimality in genetic code, Francis Crick, Carl Woese, network effect, vertical evolution, evolution of life, panspermia, optimal error minimization, science communication, scientific explanation, AI phase transitions, neural networks, emergent phenomena, substrate vs. phenomenon, public interest in science, horizontal vs. vertical transmission
ℹ️ Introduction
Introduction
Welcome to The INTO THE IMPOSSIBLE Podcast. In this thought-provoking episode, we explore the surprising physics behind why artificial intelligence works—even when, by conventional wisdom, it absolutely shouldn’t. The conversation focuses on foundational concepts from statistical mechanics and condensed matter physics, discussing the mysteries of phase transitions and the renormalization group, and how these ideas unexpectedly illuminate both AI and the emergence of life itself.
One concept discussed is the idea that "more is different," highlighting how complexity and new properties emerge when systems reach certain thresholds—a theme that echoes across magnets, biological evolution, and even modern machine learning. The discussion explores the non-intuitive truth that adding more parameters or components to a system can fundamentally change its behavior in ways simple extrapolation cannot predict, explaining why systems as diverse as fridge magnets, life’s genetic code, and massive neural networks all reveal new laws at scale.
Several points were raised, including the universality and optimality of the genetic code, the role of horizontal gene transfer in early life, and why new phenomena—sometimes equivalent to new laws of physics—arise from collective behaviors. This episode will challenge what you think you know about prediction, emergence, and the very possibility of life and intelligence in the universe.
📚 Timestamped overview
00:00 The section discusses the historical challenge of justifying why certain thermodynamic properties, including magnetization and heat capacity, deviated from expected simple numerical values like 1/2, despite a seemingly convincing argument that was later proven incorrect from the late 1940s to the middle 1960s.
08:00 The passage discusses how chemists can focus on effective, larger-scale descriptions like atoms rather than the complex underlying microphysics, allowing the practice of chemistry and scientific progress to occur without needing to understand fundamental particles entirely.
13:08 The importance of the renormalization group in graduate physics is emphasized due to its ability to solve problems in condensed matter physics and other scientific fields systematically and accurately, and this realization was contrasted with a personal anecdote about discovering that Ising was teaching at a nearby college.
18:22 Understanding molecular binding at an atomic level is crucial for specific interactions, while studying biomolecular phase separation in eukaryotic cells requires a different approach that focuses on general properties rather than specific atomic and molecular details.
25:48 The section discusses the concept of "more is different" and emergence, emphasizing that emergence involves a particular type of rigidity related to the system's response to perturbations, which is crucial for understanding and applying concepts to complex materials and systems, with AI being a prime example.
26:57 The text discusses the process of creating a simple linear model for a time series data set, acknowledging that while a straight line might pass through some data points, it fails to fit all data and make accurate predictions due to its simplicity.
34:30 The section discusses whether technological innovations are directly derived from fundamental physics laws or if these laws are applied to describe inventions after their creation, questioning the historical accuracy of this view and whether future theories could lead to breakthroughs like warp drives.
37:33 The discussion explores the ease of physics compared to the challenges faced by sociologists and economists, and touches on the connection between physics and biology, hinting at a potential challenge to Darwin's theories.
45:18 The section discusses the idea of designing a genetic code where a substituted amino acid is biochemically similar enough to the intended one to minimize damage and maintain protein function, and how analysis consistently suggests the existing genetic code achieves this efficiently.
49:01 The section discusses the hypothetical idea of gaining complex mathematical skills instantly through genetic modification and compares it to how microbes rapidly acquire traits like antibiotic resistance through gene transmission and network effects.
54:45 The evolution from simple to complex living systems involved the coevolution of organism complexity and genetic code, leading to a rapid evolution through network effects before stabilizing into the vertical era of evolution.
58:37 The text discusses how life uses horizontal gene transfer as a rapid method to solve emerging organizational problems, supported by evidence including a recent review paper in Astrobiology, highlighting that this process is not unique to Earth and could occur on Enceladus.
01:07:26 The speaker expresses concern about threats to science from actions not in good faith, emphasizes the duty of scientists to inform policymakers like Congress, references historical roles of the National Academy of Sciences, and mentions recent increased visibility and criticism, including from Donald Trump.
01:13:43 The discussion focuses on the concept of emergent phenomena across various scientific fields and how understanding these phenomena can clarify aspects of the universe that might otherwise be attributed to divine intervention, as illustrated by the speaker's identity as a practicing Jewish atheist.
01:15:40 The discussion emphasizes that success in science does not require exceptional genius but rather a good approach and method for selecting problems, as illustrated by Einstein's achievements.
📚 Timestamped overview
00:00 Discovery of thermodynamic anomalies
08:00 Chemistry and effective descriptions
13:08 Discovering Ising's teaching location
18:22 Understanding cellular phase separation
25:48 Understanding Emergence and Rigidity
26:57 Introduction to time series modeling
34:30 Discussing technology and physics theories
37:33 Relevance of physics in biology
45:18 Designing a robust genetic code
49:01 Explaining gene transfer with microbes
54:45 Early evolution of genetic codes
58:37 Horizontal gene transfer in life systems
01:07:26 Defending Public Interest in Science
01:13:43 Discussing emergent phenomena
01:15:40 Choosing Research Problems
❇️ Key topics and bullets
Sequence of Topics Covered
1. The Puzzle of Why AI Works When It Shouldn’t
Overfitting and fitting noise in machine learning 00:00:00
The expectation that models with enormous parameter counts should fail, but they don't
Initial analogies to biological complexity and optimality in genetic codes
2. Phase Transitions and Magnetism
Explanation of phase transitions using magnets as an example 00:00:25
Theoretical vs. experimental outcomes in magnetization near critical points 00:01:04
The surprise in the non-integer value of the critical exponent 00:01:44
3. The Renormalization Group (RG) Theory
Historical development and contributors (Kadanoff, Wilson, Widom) 00:03:05
RG as a way of understanding systems at different scales 00:03:18
The irreversible, non-invertible process of coarse-graining 00:04:12
RG as a semi-group, not a true group
Application to the laws of physics and material properties 00:05:29
4. Levels of Description in Physics and Chemistry
Loss of information in moving to larger scales
Effectiveness and limits of different scientific disciplines due to RG 00:07:27
Example: why chemistry is stable despite changes at the particle physics level
5. The Ising Model and the Concept of Dimensionality
History of the Ising-Lenz model and early misconceptions 00:08:53
The significance of cooperative phenomena in magnets
Dependence of physical behavior on dimensionality 00:11:23
Onsager’s 2D Ising model solution and its influence
6. Emergence, More is Different, and Philip Anderson’s Legacy
The meaning of "more is different" 00:20:02
Examples of emergent properties: solids, rigidity, sound transmission, etc. 00:21:28
Experiment highlighting collective phenomena in solids vs. gases 00:22:00
New laws of physics as emergent from cooperative behavior
7. The Physics Reason AI Works: Phase Transitions in Learning
Over-parameterization in neural networks compared to polynomial fitting 00:26:46
The phenomenon where AI works despite excessive parameter counts 00:29:02
Explanation via statistical physics: phase transition and new emergent rigidity in learning 00:29:30
Concept of critical exponents and analogy to superconductivity 00:29:54
Experimental vs. theoretical understanding of transitions in AI as system size goes to infinity 00:32:23
8. Historical and Philosophical Aspects of Technology and Science
The roots of technological advances in theoretical understanding vs. post-hoc explanation 00:34:30
Real-world technological advances like the transistor and LCDs as emergent from collective knowledge 00:36:01
Discussion on quantum mechanics’ role in everyday technology
9. Biology Through the Lens of Physics
Critique and clarification of the relationship between physics and biology 00:37:33
Explanation of the genetic code's origin, uniqueness, and optimality 00:39:22
Molecular phylogeny and the Last Universal Common Ancestor (LUCA)
Contribution of Carl Woese: discovery of archaea and network-based evolution 00:41:32
The paradox of rapid evolution and optimally efficient genetic code 00:44:24
10. Horizontal Gene Transfer and Evolution
Distinction between vertical and horizontal gene transfer 00:48:23
Analogy to network effects and the speed of early life’s evolution
Phase transition from network-dominated evolution to vertical inheritance 00:52:19
Implications for microbial evolution, antibiotic resistance, and diversity
11. Networks, Complexity, and Evolutionary Dynamics
Network effects in evolution compared to early technological systems 00:54:06
Increasing complexity and specificity limiting further horizontal transfer
12. Likelihood and Meaning of Life in the Universe
Discussion on life’s inevitability given the laws of physics 00:57:05
Life as a physical process not restricted to earthly chemistry 00:57:23
Role of life in planetary equilibrium and energy gradients 00:57:41
Potential for alternate forms of life and implications from astrobiology 00:59:25
Conceptual thought experiment on possible "life" early in the universe 01:02:24
13. The Purpose and Philosophy of Science
Dangers of anti-science sentiment and relativistic epistemology 01:06:20
Defense of the scientific process in the public interest 01:08:46
The responsibility of scientists to communicate and protect science 01:09:46
Flaws and strengths of peer review and innovation in science policy debates
14. The Experience of Being an Outsider in Science
Emphasis on the importance of supporting innovative and unique research 01:11:45
Example: Development of the D-wave model of superconductivity
15. Reflections, Advice, and the Essence of Scientific Inquiry
"Any sufficiently advanced technology is indistinguishable from magic" — discussion of the most fundamental scientific concepts to preserve 01:13:11
Emergence as a core idea: more is different 01:13:43
Guidance to young scientists: maximize impact by minimizing competition; pursue problems only you can solve 01:16:32
Final emphasis: "Different is more" and the virtue of being different in scientific endeavor 01:17:05
👩💻 LinkedIn post
🚀 Just listened to a fascinating episode of the INTO THE IMPOSSIBLE Podcast: "The Physics Reason AI Works When It Shouldn’t." The conversation focused on deep insights from physics and biology to unravel why artificial intelligence is astonishingly effective—even when, in theory, it shouldn't be.
Here are 3 key takeaways:
Emergent Phenomena and "More is Different": One concept discussed was how increasing complexity leads to fundamentally new behaviors. This principle from condensed matter physics, famously stated as "more is different," helps explain why neural networks with billions (or trillions) of parameters outperform expectations—they undergo phase transitions that lead to new, robust forms of learning.
AI and Phase Transitions: A key theme that emerged was the analogy between learning in AI models and physical phase transitions (like magnetism). The discussion explored how, at certain thresholds, neural networks develop a "rigidity" that allows accurate generalization beyond simply memorizing data, revealing profound links between statistical physics and the success of machine learning.
The Power of Interdisciplinary Thinking: Several points were raised, including how physical concepts like the renormalization group and network theory are vital for breakthroughs not only in AI, but also in understanding the origins of life, the nature of scientific explanation, and even strategies for scientific discovery.
The full episode is a reminder that great leaps in one field often come from embracing ideas across disciplines. Highly recommended for anyone interested in the deep principles behind technological progress and scientific inquiry. #AI #Physics #Complexity #ScientificInnovation
🧵 Tweet thread
The conversation focused on the surprising connections between physics, AI, biology, and the origins of life. Here are some mind-bending takeaways 🧵👇
1. The Laws of Physics Are Not Fixed—They Emerge!
One concept discussed was the idea that what we consider “laws of physics” can change depending on the scale at which you observe them. This is thanks to something called the renormalization group—a mind-boggling way physicists explain why matter looks different at atomic scale vs. the everyday world around us 04:12.
2. More Is Different: Why “More of the Same” Can Create the Wildly Unexpected
A key theme that emerged was the phrase “more is different”—popularized by physicist Philip Anderson. When you add more and more identical parts (like atoms), new behaviors materialize that you'd never predict by studying just one part. Think: why a solid is rigid while a gas isn’t, even though both are made from the same basic stuff 21:02.
“The formula for the energy of the gas is exactly the same as the solid. And yet when I push my water bottle, my fingers don't go through. The atoms conspire!”
3. AI & Phase Transitions: Why Models With Billions of Parameters Actually Work
The discussion explored why AI with trillions of weights shouldn’t work in theory (it should just “fit the noise”), but it works anyway. The answer? There’s a phase transition—a sudden change in behavior, just like when water freezes or boils. This is the physics of emergence, happening inside machine learning 29:30.
4. The Origin of Life: Physics, Not Just Chemistry 🌱
Several points were raised, including the shock that the genetic code—the Rosetta Stone of life—is near “optimal” for error-minimization, and almost all life uses the same code. How did it emerge so quickly and uniformly? The answer involves network effects, horizontal gene transfer, and evolutionary phase transitions—ideas borrowed straight from physics 46:06.
“The purpose of life is to help planets come into equilibrium… Life uses information to short-circuit chemical potential gradients and power ecosystems”—one way to look at life is as a planetary process 57:41.
5. Emergent Laws: Why We Shouldn’t Expect to “Predict” iPhones or DNA from Quantum Mechanics
The conversation wasn’t just deep science; it tackled why many of humanity’s breakthroughs (transistors, computers, life itself) can’t be predicted only from basic physics. New laws and behaviors emerge as you go up in scale 14:44.
6. Don’t Just Maximize the Numerator—Minimize the Denominator!
When asked for advice to young scientists: “Impact is the ratio of what you do to what everyone else does. Don’t work on something unless, if you didn’t do it, no one else would.” In other words: find the empty spaces, be different, that's how you make real discoveries 01:16:19.
If you made it this far: physics isn’t just about the smallest particles or the biggest galaxies—it’s about how complexity, life, and even intelligence emerge from new laws at every scale.
Different is more.
🔗 Save this thread for when you need a burst of inspiration or a reminder that reality is stranger, deeper, and more hopeful than you imagined.
#physics #AI #biology #emergence #philosophy #science
🗞️ Newsletter
INTO THE IMPOSSIBLE Newsletter
Episode Highlight: The Physics Reason AI Works When It Shouldn’t
Dear INTO THE IMPOSSIBLE Podcast Community,
We’re excited to bring you a mind-expanding episode that dives into the surprising physics at the heart of artificial intelligence and so much more.
Featured Conversation:
The Physics Reason AI Works When It Shouldn’t
This week’s discussion focused on some of science’s biggest puzzles: Why does AI, which uses trillions of parameters to fit data—often far more than the number of actual data points—work at all when classic theory says it shouldn’t? The episode explored how phase transitions, a concept from physics, and the profound idea that “more is different,” help explain this paradox.
Key themes and insights:
Emergence & More is Different: The discussion explored how adding more components to a system doesn’t just make it bigger—it can transform the whole system, yielding entirely new behaviors and “laws” at larger scales. This principle, central to condensed matter physics, also underlies why AI systems with massive complexity can suddenly “work,” exhibiting critical phenomena similar to phase transitions in materials like magnets and superconductors.
Renormalization & Levels of Description: One concept discussed was the renormalization group, a revolution in physics that lets scientists understand how physical laws change with scale—fundamentally affecting everything from magnets on refrigerators to the workings of the universe. This also gives clues to AI’s counterintuitive success: the emergence of “rigidity” and generalization in over-parameterized systems.
Biology, Evolution, and Physics: The wide-ranging discussion touched on the origins of life, evolution, and why life’s rapid emergence—and the universal genetic code—remain such deep mysteries. The conversation focused on the importance of networks, horizontal gene transfer, and how life serves as a planetary equilibrium-seeking mechanism, driven by information and physics, not just chemistry.
Science, Society, and Truth: A key theme that emerged was the critical role of science in society and current challenges to its integrity and public standing. The episode emphasized why scientists must speak up for evidence-based thinking and the foundational principles of the scientific process.
Favorite Quote:
“It’s not that more is different, it’s that different is more.”
What You’ll Learn in This Episode:
Why AI defies “common sense” statistical limits
How profound concepts from physics explain not just matter, but learning and life itself
Why emergence and phase transitions are everywhere—from the magnets on your fridge to the origins of artificial and natural intelligence
The surprising evolutionary leaps that gave rise to all life—and why science always moves at the frontiers of the unknown
🎧 Listen to the full episode and let your curiosity roam the cosmos!
Questions, feedback, or wild theories of your own? Reply to this email—we love hearing from our INTO THE IMPOSSIBLE explorers.
With wonder,
The INTO THE IMPOSSIBLE Podcast Team
If you enjoy our show, please rate and review us on Apple Podcasts or Spotify. Your support helps us bring you more boundary-breaking conversations!
You’re receiving this because you signed up for updates from INTO THE IMPOSSIBLE. Want to leave orbit? Unsubscribe here.
❓ Questions
Discussion Questions
The conversation focused on the mystery of why artificial intelligence works so well, even when models are vastly over-parameterized. Why, according to the discussion, does AI succeed in a regime where it "shouldn't" work, and how does this relate to concepts in physics like phase transitions?
One concept discussed was the idea of "more is different." How does this principle, as advanced by Philip Anderson, relate to emergent phenomena in physics and AI?
The discussion explored the renormalization group and its significance in understanding physical systems at different scales. What is the importance of being unable to uniquely "go backwards" in scale, and how does this influence scientific modeling?
Several points were raised about the limitations of early models, such as the Ising model in one dimension. How did the understanding of dimensionality change approaches to phase transitions in physics?
The concept of horizontal gene transfer played a central role in explaining the rapid evolution and optimization of the genetic code. How does this mechanism challenge or complement traditional Darwinian evolution?
A key theme that emerged was the optimal nature of the genetic code in minimizing translation errors. Why is this property significant, and what challenges does it pose for theories about the origin of life?
The discussion compared the technological advances enabled by quantum mechanics with the way future physics might lead to entirely new capabilities. How directly did fundamental physics contribute to innovations like the transistor, according to the conversation?
The rigidity and phase transition behavior found in materials was analogized to the training of neural networks in AI. What parallels were drawn between the physics of solids and the statistical learning processes in AI systems?
The episode touched upon the role of science and scientists in society, especially regarding public trust and the risk of anti-science sentiment. What responsibilities do scientists have to the public, and how can the integrity of science be defended?
The conversation ventured into defining life as a physical, not just chemical, phenomenon. What arguments were presented for the inevitability of life emerging in the universe, and how might these ideas expand the search for life beyond Earth?
curiosity, value fast, hungry for more
✅ Think AI shouldn't work as well as it does? Physics has a surprising answer.
✅ Renowned physicist Nigel Goldenfeld sits down with Brian Keating on The INTO THE IMPOSSIBLE Podcast to unravel why complex emergent behavior — from AI to life itself — often defies our simple expectations.
✅ Discover how phase transitions, "more is different," and the hidden rigidities of nature connect magnets, microbes, and machine learning.
✅ The most astonishing truths in science aren't just counterintuitive—they open up entirely new ways of seeing the universe (and ourselves). Listen now and get ready to question everything you thought you knew!
Conversation Starters
Conversation Starters for "The Physics Reason AI Works When It Shouldn’t"
The episode describes how AI succeeds even when “fitting noise”—why do you think models with trillions of parameters can generalize so well, when traditional theory suggests they shouldn’t? 29:02
The concept of “more is different” was a major theme. How do you interpret this idea, and can you think of examples outside of physics where more really becomes different, not just more? 26:48
Phase transitions were compared to sudden shifts in behavior in physics and AI. What do you think are the most interesting “phase transitions” you’ve seen in technology, society, or your own experiences?
The discussion explored how the laws of physics themselves change when you change the scale you’re observing—from atoms to rooms to the universe. What perspectives or questions does this raise about how we understand reality? 05:29
The conversation focused on the origin of life and why life on Earth developed so rapidly after the planet cooled. What are your thoughts on whether life is a chemical inevitability, a fluke, or something else? 44:24
One concept discussed was the unique and optimal nature of the genetic code. Do you believe this points towards a principle beyond random chance, or is it fully explainable by selection alone? 46:06
Several points were raised about emergent properties—new laws that appear only when enough elements come together. Can you share examples from your work or life where “emergence” shaped the outcome in surprising ways? 23:48
The episode questions whether life is better described by chemistry or by physics. What’s your take on this: is life best understood as chemistry, physics, information, or something else entirely? 01:00:01
The renormalization group was called a cornerstone of modern physics and used as an analogy for “lumping together” details in both science and life. How do you decide which details to keep and which to ignore when solving problems? 04:12
A key theme that emerged was the interplay between horizontal and vertical gene transfer in evolution. How might this network-based view change our understanding of evolution and the uniqueness of life on Earth? 50:19
🐦 Business Lesson Tweet Thread
Why does AI work, even when it should be doomed to fail? Physics has a surprising answer. 🧵👇
When you fit a trillion parameters to noisy data, logic says it should fall apart. And yet, AI models learn—they even thrive.
Turns out, it's not just about "more data" or "bigger models." It's about emergence—new behaviors that appear only when you go big.
Physics knows this well. When atoms team up, you don't just get a bigger pile—you get weird, collective magic. Magnets, solids, phase transitions. None of those make sense if you look at just one atom.
AI, too, undergoes a "phase transition." There's a tipping point where piling on complexity, instead of chaos, creates order—a sudden leap in capability.
Why does this happen? Because different is more important than more. Beyond a threshold, the rules change. Entirely new properties emerge.
So next time "experts" say a system is too complicated to ever work, remember: nature rewards those bold enough to add more, and wait for something utterly new to appear.
AI isn’t just a bigger brain—it’s a reminder that “more is different.” That’s the real secret weapon.
#AI #Physics #Emergence
✏️ Custom Newsletter
🆕 New Episode Alert: The Physics Reason AI Works When It Shouldn’t 🎙️
Hey there, friends of the INTO THE IMPOSSIBLE Podcast!
We’re thrilled to announce a brand new episode that dives deep into the mysteries of AI, the universe, and why sometimes science just refuses to behave like we expect. Get ready to have your mind thoroughly exercised in "The Physics Reason AI Works When It Shouldn’t." Trust us—you’re in for a wild ride.
What’s Inside This Episode?
Introduction
Ever wonder why AI, with trillions of parameters, pulls off the impossible—making sense and even learning, when technically it should just be fitting noise? Or how physics and biology are more connected than you might think? We’re breaking it all down in this episode—the surprising overlaps of machine learning, phase transitions, emergent phenomena, and the puzzles at the heart of life itself.
5 Fascinating Keys You’ll Learn
Why Over-Parameterized AI Still Succeeds:
Discover the mind-boggling reason—rooted in physics and phase transitions—that AI works in practice, even when, by all expectations, it shouldn’t 29:02.The Power of Emergence:
Learn about "more is different"—why having more isn’t just quantitatively different, but qualitatively different, unlocking new laws and behaviors that can’t be seen from the parts alone 20:02.How Physics Explains Life’s Complexity:
Hear why the genetic code is both unique and optimal, and how physics may be the root behind the rapid emergence of life on Earth 44:40.The Surprising Connection Between Machine Learning and Phase Transitions:
Unpack how training AI has more in common with magnets and superconductors than you ever imagined—yes, really! 30:00.Why Horizontal Gene Transfer Changed Everything:
Find out how life may have evolved through a genetic "network effect," changing the tempo and mode of evolution itself 52:00.
Fun Fact of the Episode
Did you know the genetic code—the actual mapping from DNA to amino acids—is not just a happy accident, but mathematically optimal at minimizing errors? So optimal, in fact, that Francis Crick once speculated it could have arisen from outer space 00:00, 44:46!
Listen Now & Join the Journey
We’re so excited to share this mind-expanding conversation packed with stories of missed scientific opportunities, the magic of emergence, and why "different is more."
Ready to see the world—and your computer—in a brand new light?
Let us know what you think! Reply to this newsletter, leave a review, or share your favorite “aha!” moment on social media.
Thanks for listening and for always being curious with us on INTO THE IMPOSSIBLE.
Stay inspired,
The INTO THE IMPOSSIBLE Podcast Team 🚀
🎓 Lessons Learned
1. Why AI Shouldn’t Work
The conversation focused on why, despite overfitting and excessive parameters, AI models like neural networks work unexpectedly well in practice.
2. Phase Transitions Explained
One concept discussed was phase transitions—how matter behaves differently at certain points, like magnets losing magnetization upon heating.
3. Renormalization Group Insights
A key theme that emerged was the renormalization group, explaining how physical laws and behaviors change with different observational scales.
4. Levels of Description Matter
The discussion explored how choosing the right level of detail—atomic vs. coarse-grained—changes our understanding of complex systems.
5. More Is Different Principle
Several points were raised, including how collective behaviors and emergent properties arise when systems become large or more complex.
6. Emergence in Physics
Emergence shows how new laws and collective effects arise in matter, like rigidity in solids or sound propagation, not evident from components.
7. The Ising Model’s Significance
The limitations and breakthroughs of the Ising model highlighted the impact of dimensionality and cooperative effects in theoretical physics.
8. Evolution Beyond Darwin
Biology lessons included how horizontal gene transfer and network effects drove rapid, optimal development of life’s genetic code.
9. Life as a Physical Process
Life’s emergence was presented as a physical—not solely chemical—process, involving information flow and planetary energy gradients.
10. Science’s Public Responsibility
The conversation emphasized the need for scientists to defend objective truth, engage the public, and act for the common good.
10 Surprising and Useful Frameworks and Takeaways
Ten Most Surprising and Useful Frameworks and Takeaways
1. More Is Different: Emergence and Qualitative Change
A key theme that emerged was the counterintuitive idea that adding more components to a system doesn't just give "more of the same," but can fundamentally change its nature—a concept encapsulated in "more is different." New, emergent laws and behaviors arise that cannot be predicted from the properties of a system's parts. This is exemplified by the transition from a handful of atoms to a rigid solid, or the leap from simple neural nets to advanced AI systems 00:20:02, 00:26:36, 00:29:30.
2. The Renormalization Group: Physics Across Scales
The discussion explored the renormalization group, a framework that explains how physical systems can be described at different scales. Coarse-graining—a process where complex microscopic details are "lumped" into larger, effective units—shows why understanding the micro-details isn't always necessary at larger scales, enabling meaningful macroscopic laws and practical science 00:03:55, 00:04:52, 00:05:29.
3. Irreversibility of Coarse-Graining
One concept discussed was the irreversible nature of coarse-graining: while you can combine microscopic information into macroscopic variables, you can't uniquely reverse the process. This non-invertibility underlies why our world has emergent laws and why, for example, chemistry doesn't depend on the details of high-energy physics 00:05:29, 00:05:56, 00:18:22.
4. Levels of Description and the Choice of Model
The conversation focused on the importance of choosing the right level of description for a given question—atomic for chemistry, molecular for certain biology problems, more coarse-grained for understanding cellular systems. The "correct" model depends on the phenomenon of interest; more detail is not always more useful 00:18:01, 00:19:18.
5. Phase Transitions in Physics and AI
An especially surprising takeaway was the analogy between phase transitions in material systems and the operation of AI. AI models, when overloaded with parameters (more than data points), seem like they should fail due to overfitting, but they can still generalize. The explanation is a phase transition in the statistical physics of learning; beyond a certain point, new rigidities and behaviors emerge, analogous to the onset of superconductivity or magnetism 00:26:36, 00:29:30.
6. Horizontal Gene Transfer as an Evolutionary Network Effect
The discussion revealed that the traditional vertical inheritance picture of evolution (parent to child) is insufficient to explain the rapid emergence and optimization of the genetic code. Instead, horizontal gene transfer—where genes spread among unrelated organisms—created a network effect, accelerating evolution and allowing for the establishment of today's universal genetic code 00:49:01, 00:51:32.
7. Optimality of the Genetic Code and the Role of Physics in Biology
The genetic code is not just arbitrary; it's actually optimal in minimizing the damage from translation errors. This insight, grounded in statistical and information theory, blurs the line between biology and physics and suggests that life's organization is constrained and shaped by physical principles 00:44:32, 00:46:06.
8. Purpose of Life: Driving Planets Toward Equilibrium
A striking conceptual framework presented was that life’s purpose (from a physics standpoint) is to accelerate a planet's progression toward thermodynamic equilibrium by exploiting chemical gradients—essentially "short-circuiting" them to power living processes 00:57:05, 00:57:41.
9. Learning from Phase Transitions About Generalization in Machine Learning
Several points were raised, including that techniques from statistical mechanics can quantify when a model (like a neural network) enters a regime where it generalizes well despite overparameterization. The study of "critical exponents" and phase transitions provides a rigorous understanding of why AI works when, by conventional wisdom, it shouldn't 00:29:30.
10. Innovation Strategy: Minimize the Denominator
The discussion concluded with a powerful career and research heuristic: to maximize impact, focus on doing what only you can do. Don’t just try to maximize your output (the numerator), but minimize the denominator—the number of others who could do the same thing. This strategy increases the chances of making a unique contribution 01:16:32, 01:16:54.
These frameworks bridge physics, biology, and learning theory, and offer powerful, surprising tools for understanding complexity in both nature and technology.
Clip Able
Social Media Clip Suggestions
1. Title: "The Physics Behind Emergence: Why More is Different"
Timestamps: 00:20:02 – 00:26:46
Caption:
Explore the powerful concept of "more is different" and how phase transitions, emergent properties, and collective behavior give rise to entirely new laws in physics—plus why this fundamental insight is core to understanding everything from fridge magnets to the mysteries of AI.
2. Title: "Why Does AI Work When It Shouldn't?"
Timestamps: 00:26:33 – 00:31:13
Caption:
Dive into the paradox of modern artificial intelligence: how overfitting and massive parameter counts defy classical expectations, and the surprising physics—phase transitions and generalized rigidity—behind why today's AI models perform so well in practice.
3. Title: "How Physics Solves the Mystery of Life's Origins"
Timestamps: 00:38:18 – 00:44:04
Caption:
Unravel the puzzle of the genetic code, optimality, and the rapid rise of life on Earth. Discover why the interplay of physics and biology—plus a bit of molecular detective work—challenges assumptions about evolution, design, and the universality of code in life.
4. Title: "Networks, Gene Swapping & the Fast Lane of Evolution"
Timestamps: 00:48:47 – 00:55:30
Caption:
Journey into the evolutionary fast lane with the concept of horizontal gene transfer—how genes can leap between unrelated organisms, why evolution once worked as a network rather than a tree, and what this means for the pace and pattern of life's unfolding complexity.
5. Title: "Life, Planets, and the Purpose Written in Physics"
Timestamps: 00:57:05 – 01:04:42
Caption:
Expand your mind with the bold idea that life is the universe's method for helping planets reach equilibrium. Connect astrobiology, thermodynamics, and the information-processing power of living systems—all rooted not in chemistry, but in fundamental physics.
💡 Speaker bios
Nigel Goldenfeld has always been fascinated by nature’s deep mysteries, from the improbable order of life to the surprising laws of matter. Grappling with ideas once pondered by Francis Crick—like whether life’s dazzling complexity could have arisen so quickly, or if the genetic code is almost magically optimal—Nigel searched for principles uniting biology and physics. He sees life as a process by which planets reach equilibrium, and explains complex ideas, like phase transitions turning metals magnetic, through everyday wonders, such as magnets holding up cherished family photos. In Nigel’s world, the seemingly impossible becomes understandable, and the noise gives way to hidden patterns that quietly govern our universe.
💡 Speaker bios
Brian Keating describes himself as a “dull-headed experimental cosmologist,” yet his curiosity and humility suggest a deeper intellectual passion. Known for beginning his lectures with the famous phrase, “more is different”—coined by the physicist Philip Anderson, whom Keating both questions and admires—he explores big questions about the universe’s complexity and emergence. Using analogies such as when a sand grain becomes a sand pile or how protons combine, Keating invites his audience to consider profound scientific concepts and challenges simplistic explanations, highlighting both his skepticism and his respect for foundational thinkers in physics.
💡 Speaker bios
Nigel Goldenfeld’s scientific journey has always gravitated toward the seemingly impossible—seeking order in noise and meaning in chaos. Fascinated by the unlikely optimality and complexity of life, he pondered questions others dismissed as unanswerable. While thinkers like Francis Crick speculated that life’s sophistication might be alien in origin, Goldenfeld probed the genetic code’s uncanny ability to minimize errors. His curiosity stretched from the mysteries of life to the fundamental shifts in matter, likening planetary equilibrium and life’s purpose to phase transitions—moments when complexity emerges from simplicity, much like a piece of metal suddenly acquiring magnetism and sticking holiday pictures to a refrigerator. For Goldenfeld, science is the art of uncovering how improbable harmonies arise from the apparent noise of the universe.
Made with Castmagic
Turn any recording into a page like this.
Upload audio or video — interviews, podcasts, sales calls, lectures. Get a transcript, summary, key takeaways, and social-ready clips in minutes.
Or learn more about Castmagic first.
Magic Chat
Try asking
Google
Apple