What if the smartest thing AI could do is refuse to give you the answer?

We trained it to never give answers. It is Socrates. It only gives context and questions. But Amazingly, upwards of 20% of participants switched into cyborg mode and did this amazing. Not just superhuman, but super AI performance. What is the point of us if the best and brightest can't beat what an AI can do? But then in about 5 to 10% did something amazing. We called them the cyborgs. If we zombie walk our way into a future where it has all the answers and we don't, what's the point of us? Exploring the unknown is the one thing humans are uniquely well suited to do.

What is the best use of AI right now?

I have a paper coming up and the title of that paper is Human Human Capital, Not AI Benchmarks Predicts Hybrid Intelligence in Forecasting. So how do you measure how creative a human is or an AI is, or the two together, when maybe a massive large language model has just memorized every measure of it we would typically use in science, which by the way, it has, and so it distorts. And then you have a bunch of marketing professors or computer scientists who, who I love them, but they aren't scientists running these experiments that aren't really valid. What have we made predictions of the future? What will the price of oil be in six months? Which, as we record this, everybody has some sense. But I will tell you, when we ran the experiment several months ago, nobody, nobody knew what the price of oil was today, much less than six months from now. So the humans in this experiment did terribly. The AIs did great and pretty much how they did, tracked with the traditional AI benchmarks that they were used on. But then we paired them together and Human Capital absolutely dominated, which is to say the vast majority of people in the experiment, including a whole lot of UC Berkeley students, smart kids, essentially said, gemini GPT, what will the price of oil be in six months? And then they submitted that answer.

What do you expect from the second best UC school? Right?

Yes. Well, I mean, we can't all be in the sunshine all the time. In this particular case, you know, you could look at that and feel really terrible. What is the point of us if the best and brightest can't beat what an AI can do? That essentially they're just a pair of legs to walk it across the answer across the room. But then in about 10, 5 to 10%, not a huge percentage, but we saw it in there, about 5 to 10% did something amazing. We called them the Cyborgs the cool thing was you couldn't tell who made the prediction. Was it the machine? Was it the person? Because what they would do is they'd make a big prediction and then the AI and we used a variety of different models, would sort of say, oh, but wait a minute, what about the data? And then the humans would say, okay, you're right, that wasn't right, but what about this? And then they go back and forth several rounds. They had to make 10 predictions in an hour.

They didn't have time to cheat the system like they went in. They did better than the best humans, even if they themselves on their own were modest. They did better than the best AIs even when our cyborgs just had a small open source model available to them. And we took the questions off of polymarket, which again six months ago, very few people knew about. Now that's in the news and in many contexts they were comparable to polymarket in high volume questions. So this is a prediction market. People have actual money trying to figure out what that price of oil will be. And a human with no prior knowledge paired with even a modest AI, but with this right set of behavior.