The Impact of AI on Society: Opportunities and Challenges

I think a lot of people don't realize how massive the positive impact AI is going to have on their life. Well, I

would argue that the idea that this AI disruption doesn't lead us to human catastrophe is optimistic. For example,

people are going to be unemployed in huge numbers. You agree with that, don't you? Yes. If your job is as routine as

it comes, it's gone in the next couple years. But it's going to create new opportunities for wealth creation. Let

me put it to you this way. We have created a new species and nobody on earth can predict what's going to

happen. We are joined by three leading voices to debate the most disruptive shift in human history, the rise of AI.

And they're answering the questions you're most scared about. This technology is going to get so much more

powerful. And yes, we're going to go through a period of disruption. But at the other end, we're going to create a

fair world. It's enabling people to run their businesses, make a lot of money, and you can solve meaningful problems

such as the breakthroughs in global healthcare and education will be phenomenal. and you can live an

incredibly fulfilling existence. Well, I would just say on that front, this has always been the fantasy of technologists

to do marvelous things with our spare time, but we end up doom scrolling, loneliness epidemic, right? Falling

birth rates. So, the potential for good here is infinite and the potential for bad is 10 times. For example, there's

war, undetectable beat fakes and scams. So, people don't understand how many different ways they are going to be

robbed. Look, I don't think blaming technology for all of it is the right thing. all these issues, they're already

here. You're all fathers here. So, what are you saying to your children? Well, first of all, this has always blown my

mind a little bit. 53% of you that listen to this show regularly haven't yet subscribed to the show. So, could I

ask you for a favor before we start? If you like the show and you like what we do here and you want to support us, the

free simple way that you can do just that is by hitting the subscribe button. And my commitment to you is if you do

that, then I'll do everything in my power, me and my team, to make sure that this show is better for you every single

week. We'll listen to your feedback. We'll find the guest that you want me to speak to and we'll continue to do what

we do. Thank you so [Music] much. The reason why I wanted to have

this conversation with all of you is because the subject matter of AI, but more specifically AI agents, has

occupied my free time for several weeks in a row. And actually amad when I started using replet for me it was a

paradigm shift. There was two paradigm shifts in a row that happened about a week apart. Chat GPT released their

image generation model where you could create any image. It was incredibly detailed with text and all those things.

That was a huge paradigm shift. And then in the same week I finally gave in to try and figure out what this term AI

agent was that I was hearing all over the internet. I heard vibe coding. I heard AI agent. I was like I will give

it a shot. Mhm. And when I used Replet, 20 minutes in to using Replet, my mind was blown. And I think that night I

stayed up till 3 or 4 a.m. in the morning. For anyone that doesn't know, Replet is a piece of software that

allows you to create software. Mhm. And pretty much any software you you want. So someone like me with absolutely no

coding skills was able to build a website, build in Stripe, take payment, integrate AI into my website, add Google

login to the front of my website and do it within minutes. I then got the piece of software that I had built with no

coding skills, sent it to my friends, and one of my friends put his credit card in and paid. Amazing. So I just

launched a SAS company with no coding skills. to demonstrate an AI agent in a very

simple way. I used an online AI agent called operator to order us all some water from a CVS around the corner. The

AI agent did everything end to end and people will be watching on the screen. It put my credit card details and it

picked the water for me. It gave the person a tip. It put some delivery notes in at some point a guy is going to walk

in. He has not interacted with a human. He's interacted with my AI agent. And I just the reason I use this as an example

is again it was a paradigm shift moment for me when I heard about agents. Mhm. about a month ago and I went on and I

ordered a bottle of electrolytes and when it when my doorbell rang I freaked out. I freaked out. But am who are you

and what are you doing? So uh I started programming at a very young age. You know I I built my first business when I

was a teenager. I used to go to uh internet cafes and program there. And I realized that they don't have software

to manage the business. I was like oh why don't you create accounts? I don't have a server. It took me two years to

build that piece of software. And that's sort of embedded in my mind this idea that hey like you know there there's a

lot of people in the world with really amazing ideas especially in the context where they live in that allows them to

build uh businesses. However, the main source of uh friction between an idea and software or call it an idea and

wealth creation is u infrastructure is physical infrastructure is a meaning a computer in front of you. It is um an

internet connection. It is the set of tools and skills that you need to build that. If we make it so that anyone who

has ideas who who wants to solve problems will be able to do it. I mean imagine the kind of world that we we

could live in where no one can be anyone who has merit anyone who can think clearly anyone who can generate a lot of

ideas can generate wealth. I mean that's an amazing world to live in right anywhere in the world. So with with

Replet the company that I started in 2016 the the idea was like okay coding is difficult how do we solve coding and

um we built every part of the process the hosting the code editor the only missing thing was the you know the AI

agent and so over the past two years we've been working on this AI agent that you can just you know similar since chat

GPT you know this revolution with Genai and you can just uh speak your ideas into existence. I mean this starts you

know sounding religious like this is like the you know the gods you know that the myths that that um that humans have

created they used to imagine a world where you can you can be everywhere and anywhere at once that's sort of the

internet and you can also speak your ideas into into existence and um you know it's it's still early I think uh

replet agent is is a fantastic tool and I think this technology is going to get so much more powerful

specifically what is an AI agent. I've got this um graph actually here which I don't need to pass to any of you for you

to be able to see the growth of AI agents. But this graph is Google search trend data. This also resembles our

revenue too. Oh, okay. Right. The the water has arrived. Hello. Thank you. You can come

on in. Can I have a go, please? Yes. It's 3951. Great. Thank you so much. Thank

you. Thank you. I mean this is this is like a supernatural kind of power. You conjured water. I conjured water from my

mind. Yeah. And it's shown up here with us and it clearly thinks we need a lot. But but just to define the term AI agent

for someone that's never heard the term before. Yeah. Yeah. So uh I assume most of the audience now are familiar with

chat, right? You can go in and you can talk to an AI. It can search the web for you. It has a limited amount of tools.

Uh maybe it can call a calculator to do some addition subtraction for you, but that's about it. It's a request response

style. Agents are when you give it a request and they can work indefinitely until they achieve a goal or they run

into an error and they need your help. It's an AI bot that has access to tools. Those tools are access to the to a web

browser like operator, access to a programming environment say like replet, access to um you know credit cards. The

more tools you give the agent, the more powerful it is. Of course there's all these consideration around security and

safety and all of that stuff. But uh the the most important thing is the AI agent will determine when it finished

executing. Uh today AI agents can run for anywhere between you know 30 seconds to 30 minutes. Uh there's a recent paper

that came out that's showing that every 7 months the number of minutes that the agent can run for is doubling. So we're

at like 30 minutes now. In seven months we're going to be at an hour then you know 2 hours. Pretty soon we're going to

be at days. And at that point, you know, AI agent is doing labor is doing kind of humanlike labor and actually uh OpenAI's

new model 03 beat the expectation. So it it sort of doubled coherence over long horizon tasks in just three or four

months. So we're in this massive and I mean this looks this exponential graph, you know, that shows you the massive

trend we're on. Brett, give us a little bit of of your background, but also I saw you writing

some notes there. There was a couple of words used there that I thought were quite interesting, especially

considering what I know about you. The word God was used a few times. Well, uh, let me just say I'm an

evolutionary biologist, and probably for the purposes of this conversation, it would be best to think of me as a

complex systems theorist. One of the things that I believe is true about AI is that this is the first time that we

have built machines that have crossed the threshold from the highly complicated into the truly complex.

And I will say I'm listening to this conversation with a um a a mixture of profound hope and dread

because seems to me that it is obvious that the potential good that can come from this technology is effectively

infinite. But I would say that the harm is probably 10 times. It's a bigger infinity. And the question of how we are

going to get to a place where we can leverage the obvious power that is here to do good and dodge the worst harms. I

have no idea. I I know we're not prepared. So I hear you talking about agents and I think um that's marvelous.

We can all use such a thing right away and the more powerful it is, the better. The idea of something that can solve

problems on your behalf while you're doing something else is marvelous. But of course, that is the precondition for

absolute devastation to arise out of a miscommunication, right? to have something acting autonomously to

accomplish a goal, you damn well better understand what the goal really is and how to how to pull back the reins that

it starts accomplishing something that wasn't the goal. The potential for abuse is also utterly profound. You know, you

can imagine, just pick your your dark mirror uh fantasy dystopia where something has

been told to hunt you down until you're dead and it sees that as a, you know, a technical challenge.

So, I don't know quite how to balance a discussion about all of the things that can clearly come from this

that are utterly transcendent. I mean, I do think it is not inappropriate to be invoking God or biblical metaphors here.

You know, you're uh producing water seemingly from thin air. I believe that does have an exact biblical parallel.

Uh so, uh any case, the the power is here, but so so too is the need for cautionary tales, which we don't have.

That's the problem is that there's no body of myth that will warn us properly of this tool because we've just crossed

a threshold that is similar in its capacity to alter the world as the invention of writing. I really think

that's that's where we are. We're talking about something that is going to fundamentally alter what humans are with

no plan. You know, writing alters the world slowly because the number of people who can do it is tiny at first

and remain so for thousands of years. This is changing things weakly and that's an awful lot of power to just

simply have dumped on a system that wasn't well regulated to begin with. Dan, yeah. So, I'm an I'm an

entrepreneur. Um, I've been building businesses for the last 20 plus years. I'm completely well positioned between

the two of you here. the excitement of the opportunity and the terror uh of what could go on. There's this image

that I saw of New York City in 1900 and every single vehicle on the street is a horse and cart and then 13 years later

the same photo from the same vantage point and every single vehicle on the street is a car. And in 13 years all the

horses had been removed and cars had been put in place. And um if you had have interviewed the horses in 1900 and

said uh how do you feel about your level of confidence uh in in the world? The horses would have said well we've been

part of uh humanity for you know horse and hoof hand and hoof for for many many years for for thousands of years.

There's one horse for every three humans. Like how bad could it be? You know we'll always have a special place.

will always be part of society. Um, and little did the horses realize that that was not the case. That the horses were

going to be put out of of uh business very very rapidly. And to reason through analogy, you know, there's a lot of us

who are now sitting there going, "Hey, wait a second. Does this make me a horse in 1900?" I think a lot of people don't

realize how massive these kind of technologies are going to have as an impact.

You know, one minute we're ordering a water and that's cute and the next minute it can run for days and in your

words uh it doesn't stop until it achieves its goal and it comes up with as many different ways as it could

possibly come up with to achieve its goal and in your words it better know what that goal is. I'm thinking a lot as

Daniel's speaking about the vast application of AI agents and where are the bounds because if if this thing is

going to get incrementally smarter well incrementally might be an understatement it's going to get incredibly smart

incredibly quick and we're seeing this AI race where all of these large language models are competing for

intelligence with one another and if it's able to traverse the internet and click things and order things and write

things and create things and all of our lives run off the internet today. What can't it do? It's going to be smarter

than me. No doubt it already is. And it's going to be able to take actions across the

internet, which is pretty much where most of my professional life operates. It's like how I build my businesses.

Even this podcast is an internet product at the end of the day because you can create we've done experiments now and I

can show the graphs on my phone to make AI podcasts and they have we've just managed to get it to have the same

retention as the dire CEO. Now with the image generation model retention as in viewer retention the percentage of

people that get to one hour wow is the same now. So we can make the video, we can publish it, we can script it, you

can synthesize my voice sounds like me. So what what is it going to be able to do? Mhm. And can you give me the variety

of use cases that the average person might not have intuitively conceived? Yeah. So I I tend to be an an optimist

and and part of the reason is because I try to understand the limits of the of the technology. What can it do is

anything that we can any sort of set of human data that we can train it on. What can it not do is anything that uh humans

don't know what to do because we don't have the training data. Of course, it's super smart because it integrates

massive amount of knowledge that you wouldn't be able to read, right? It also much faster. It can run through massive

amount of computation that you know your brain you can't even comprehend because all of that they're smart. They can take

actions but we know the limits of what they can do because we trained them. They're able to simulate what a human

can do. So the reason you were able to order the water there is because it was trained NSF data that includes clicking

on Door Dash and ordering water. I applaud your optimism and I like the way you think about these puzzles, but I

think I see you making a mistake that we are about to discover is very common place. So we have several different

categories of systems. We have simple systems, we have complicated systems, we have complex systems and then we have

complex adaptive systems. And to most of us, a highly complicated system appears like a complex system. We

don't understand the distinction. Technologists often master highly complicated systems and they know, you

know, for example, a computer is a perfectly predictable system inside. There's it's deterministic. Mhm. But to

most of us, it functions, it's it is mysterious enough that it feels like a complex

system. And if you're in the position of having mastered highly complicated systems and you look at complex systems

and you think it's a natural extension, you fail to anticipate just how unpredictable they

are. So even if it is true that today there are limits to what these machines can do based on their training data, I

think the problem is to see what's going to happen. You really want to start thinking of this as

the evolution of a new species that will continue to evolve. It will partially be shaped by what we ask it to do, the

direction we lead it, and it will partially be shaped by things we don't understand. So, how does this computer

that that we have work? Well, one of the things that it does is we plug them into each other using language. It's almost

as if you've plugged an Ethernet cable in between human minds. And that means that the cognitive potential exceeds the

sum of the individual minds in question. Your AIS are going to do that. And that means that our ability to say what they

are capable of does not come down to well we didn't train it on that data. As they begin to interact that feedback is

going to take them to capabilities we don't anticipate and may not even recognize once they become present.

That's one of my fears. This is an evolving creature and it's not even an animal. If it were an animal, you could

say something about what the limits of that capability are. But this is a new type of biological creature and it will

become capable of things that we don't even have names for. Even if it didn't do that, even if it just stayed within

the boundaries that you're talking about, you mentioned about it having median level intelligence. Well, that by

definition means 50% of the people on the planet are less intelligent than uh AI. uh you know to a degree it's almost

as if we've just invented a new continent of remote workers. Um there's billions of them. They've all got a

masters or a PhD. They all speak all the languages. Anything that you could call someone or ask someone over the internet

to do, they're there 24/7 and they're 25 cents an hour. Uh if that um so like if that really happened like if we really

did just discover that there were a billion extra people on the planet who all had PhDs and were happy to work

almost for free that would have a massive disruptive impact on society. Like society would have to rethink how

everyone lives and works and gets meaning. Um so like and that's that's if it just stays at a median level of

intelligence. Like it's it's pretty profound. I still think it's it's a tool. this is power that is there to be

harnessed by entrepreneurs. You know, I I think that the world is gonna get disrupted, right? Um and the the you

know, the this you know post-war world that we created where you go through life, you go through 12 years of

education, you get to college and you just check the boxes, you get a job. We can already see the fractures of that

that it is, you know, this American dream is perhaps no longer there. And so I think the world has already changed.

So but but like what are the opportunities? Obviously there are downsides. The opportunities is for the

first time access to opportunity is is equal. And I I do think there's going to be more inequality. And the reason for

this inequality is because actually Steve Jobs uh you know made this analogy. It's like the the best taxi

driver in New York is like 20% better than the you know the you know average taxi driver. the best programmer uh can

be 10x better. You know, we call we say the 10x engineer. Now, the variance will be in the thousandx, right? Like the

best the best entrepreneur that can leverage those agents could be better uh could be a thousand times better than

than someone who doesn't have the grit, doesn't have the skill, doesn't have the ambition. Right? So, so that that will

create a world. Yes, there's massive access to opportunity, but there are people who will take seize it and then

they'll seize it and then there'll be people who don't. I imagine it almost like a um a marathon race and AI has two

superpowers. One superpower is to distract people um such as Tik Tok algorithm. That's right. And the other

superpower is to make you hyper creative. So you become a hyper consumer or a hyper creator. And in this marathon

race, the vast majority of people have got their shoes tied together cuz AI is distracting them. Some people are

running traditional race. Some people have got a bicycle and some people have got a Formula 1 vehicle. And it's going

to be very confronting when the results go on the scoreboard and you see, oh, wait a second. There's a few people who

finished this marathon in about 30 minutes. And there's a lot of us who finished in like 18 hours because we had

our shoes tied together. And I can't understand if we've got equal opportunity why there's so much

disparity between how fast it you know and do, you know, I'm using an analogy, but this idea that, you know, someone

like a lot of people are going to start earning a million dollars a month and a lot of people are going to say, "Hey, I

can't even get a job for $15 an hour." there's going to be this kind of interesting wedge. Well, but I I hear in

what both of you are saying a kind of assumption that this will all be done on the up and up. And I do want

to just I am not a doomer. I agree that the doomers are likely incorrect that their fears are misplaced. But I do

think we have a question of a related rates problem. You know I said the potential for good

here is infinite and the potential for bad is 10 times. Right? What I mean is there are lots of

ways in which this obviously empowers people to do things that they were going to be otherwise stuck in the mundane

process of learning to code and then figuring out how to make the code work and bring it to market and all of that.

And this solves a lot of those problems and that's obviously a good thing. really the what we should want is the

wealth creation object as quickly as we can get there. But the problem is you know as much as it that hyper creative

individual is empowered to make wealth the person who is interested in stealing may be even more empowered. And I'm

concerned about that at at a pretty high level. the the abuse cases may outnumber the use cases and we don't have a plan

for what to do about that. Um to can I can I give you a quick uh like introduction here like the optimistic

view? OpenAI invented uh GP the first version GPT came out in 2019. 2020 was GPT2 and so OpenAI you know now they get

a lot of criticism and lawsuit from IL Musk that they're no longer open source right they used to be. The reason is in

GPT2 they said we are we are no longer gonna uh open source this technology because it's going to create um

opportunities for abuse such as you know influencing elections um you know stealing you know grandma's credit card

and so on so forth. Wouldn't you say Brett that it is kind of surprising how little abuse we've seen so far?

I don't know how much abuse we've seen so far. I don't know how any of us do. And I also even the example that you

suggest where chat GPT is no longer open source to prevent abuse. I'm taking their word for it that that's the

motivation. Where as a systems theorist, I would say, well, if you had a technology that was excellent at

enhancing your capacity to wield power, then open sourcing it is a failure to capitalize on that. and that the most

remunerative use is to keep it private and then either sell the ability to manipulate elections to people who want

to do so or sell the ability to have it kept off the table for people who don't. And I would expect that that's probably

what's going on. There's no if you have a technology as transformative as this, giving it away for free is

counterintuitive, which leaves those of us in the public more or less at the mercy of the people who have it.

So I I don't see the reason for comfort there. We are at the dawn of this radical transformation of humans that by

its very nature as a truly complex and emergent innovation. Nobody on earth can predict what's going to happen. We can

we're on the event horizon of something. And the problem is you know we can talk about the obvious disruptions the job

disruption and that's going to be massive. And does that lead some group of elites to decide, oh well, suddenly

we have a lot of useless eaters and what are we going to do about that? Because that conversation tends to lead

somewhere very dark very quickly. Um, but I think that's just the beginning of the the various ways in which this could

go wrong without the doomer scenarios coming into play. This is an uncontrolled experiment in which all of

humanity is downstream. Yeah. So I I was trying to make the point that OpenAI has been sort of wrong about the uh the sort

of uh how big of a potential for harm it is like you know I think we would have heard about it in the news like uh the

sort of how much harm it's done and maybe you know some of it is working in the shadows but like the few incidents

that we've heard about where you know the cause of LLM's large language models the technology that's powering chat has

been a huge headliners uh like New York Times talked about this kid that was, you know, perhaps goated by some kind of

chat software that, you know, helps teenagers to be less lonely into into suicide, which is which is tragic. And

obviously, these are the kind of safety and and abuse uh issues that we want to we want to worry about. But these are

kind of these isolated uh incidents and we do have open- source large language models. Obviously, the thing that

everyone talks about is DeepSeek. Deepseek is uh coming from China. So what is Deepseek's incentive? You know

perhaps the incentive is to destroy the AI industry in the US. Uh you know when they released Deepseek, you know the

market tanked the market for Nvidia, the market for AI and all of that. But there is an incentive to open source. Meta is

open sourcing Llama. Llama is another AI similar to Chat GPT. The reason they're open sourcing Llama and Zuckerberg just

says that out loud is basically they don't want to be beholden to open AI. They don't sell AI as a service. They

use it to build products. And there's this concept in business called commoditize your compliment because you

need AI as technology to run your service. The best strategy to do is to open source it. So these market forces

are going to create conditions that I think are actually beneficial. So I'll give you a few few examples. One is

first of all the AI companies are motivated to create AI that is safe so that they can sell it. Second there are

security companies investing in AIs that allows them to protect from the sort of malicious uh acting of of AI. And so you

have the free market and we've always had that you know but generally as humanity we've been able to leverage uh

the same technology to protect against the abuse. So I I don't really understand this and maybe this is

actually this is the exact discussion that you would expect between somebody at the frontier of the highly

complicated staring at a complex system and a biologist who comes from the land of the complex and is looking back at

highly complicated systems. In game theory we have something called a colle collective

action problem. And in the market that you're describing, an individual company has no

capacity to hold back the abuses of AI. The most you can do is not participate in them. You can't stop other people

from programming LLMs in some dangerous way. And you can limit your own ability to earn based on your own limitations of

what you're willing to do. And then effectively what happens is the technology gets invented anyway. It just

that the dollars end up in somebody else's pocket. So the incentive is not to restrain yourself so that you can at

least compete and participate in the market that's going to be opened. And so the number of ways in which you can

abuse this technology. Let's take a couple. What is to stop somebody from training

LLMs on an individual's creative output and then

creating an LLM that can out compete that individual can effectively not only produce what they would naturally

produce over the course of a lifetime but can extrapolate from it and can even hybridize it with the insights of other

people so that effectively those who have the LLM can train it on the creativity ity of others, not cut them

in on the use of that insight. You can effectively end up putting yourself out of business by putting your creative

ideas in the world where they get sucked up as training data for future LLMs. That is unscrupulous, but it's

effectively guaranteed. In fact, it's already happening. So, that's a problem. And likewise, what what would stop

somebody from interacting with an individual and training an LLM to become like a personalized con artist?

Something that would play exactly to your blind spot. That does happen. That that is starting to happen. Um people

get phone calls and it sounds like their daughter and I've I've lost my phone and I'm borrowing a friend's phone and all

of that sort of stuff. What's what's interesting is that I I think you make a a really good point.

I worry about the impact on society. And yet when I look at every single individual who uses AI regularly, it

almost has nothing but profoundly positive impact on their life. I look at people like um I was just spending some

time with my parents-in-law um who are in their 70s and early 80s and they use AI regularly for all sorts of things

that they find incredibly valuable and that improves the quality of their life. I personally did a an M&A a mergers and

acquisitions deal where I bought a company last year and the AI was so powerful at helping that process. The

conversations were transcribed and they were turned into letters of intent and then press releases and uh legal

documents and we probably shaved $100,000 worth of uh costs and and we sped up the whole process and it was

pretty magical to see how how it could happen. With that said, you know, there's there's all of these like, well,

$100,000 worth of lawyers didn't get paid, right? So, well, what I want to know, yeah,

people upset about, but if we look back at the invention of the cell phone or the invention of the social media

platforms, there would be every reason to have the exactly the same perspective, right? I remember the

beginning of Facebook and I remember the idea that suddenly the process that used to afflict people where you would just

lose touch with most of the people who had been important to you, that was not something that needed to happen anymore.

You could just retain them permanently as a part of a diffuse uh social grouping that just simply grew

and value was added. There's no end to how much good that did, but what it did to us was profound and not evident in

the first chapter. Say the same thing about the cell phones and the dopamine traps and the way this has disconnected

us from each other, the way it has disconnected us from nature, the way it has altered the very patterns with which

we think. It has altered every classroom. Mhm. So, and those things I think are going to turn out to have been

sort of minor foreshadowings of the disruption that AI will produce. So, I agree with you today. The amount you can

do with AI, there's a tremendous amount of good. There's a little bit of harm. Maybe that's something we need to worry

about. But as this develops, as we get to, you know, to peer over the edge of this cliff that we're headed to, I think

we're going to discover that we can't yet detect the nature of the alteration that's coming. So, I just wanted to add

some context to that cuz Amjad, I I saw the interview you did in a newsletter in 2023 where you said, "I wouldn't prepare

for AGI in the same way that I wouldn't prepare for the end of days." It's effectively the end of days if the

vision of AGI that some of these companies have comes to bear because it's called the singularity moment

because you can't really predict what happens after that. And so like how would you even prepare for that and you

want to prepare for the more likely world and that world that you can actually predict is a world where yes

there's like a massive improvements of technology and there's like insane compounding effects of technology and

it's pretty hard to keep up. From that it appeared that in 2023 you were saying a similar thing to Brett in terms of we

can't see around the corner here because it is a singularity. Sorry you also used AGI artificial

general intelligence. It'd be interesting to know what your definition of AGI is. say that. Yeah. So, so what

what I was saying there is even if I'm wrong that you can actually create a unbounded seemingly conscious artificial

intelligence that can entirely replace humans and can act autonomously in a way that even humans can't act and can

coordinate across different AIs, different data centers to take over the world. Even if if that's so so the

definition of AGI is artificial general intelligence meaning that AI can acquire new skills in efficiently in the same

way that a humans can acquire skills. Right now AIs don't acquire skills efficiently you know they they require

massive amount of energy and compute entire data set of compute to acquire these these skills and I think there's a

again a limit on how general intelligence can get I think for most of the time we're lagging in terms of what

humans are are capable of doing the singularity is based on this concept of intellig intelligence explosion so once

Once you create an AGI, once you create an artificial general intelligence, that intelligence will be

able to modify its own source code and create the next version that is much more intelligent. And the next version

creates the next version and the next, you know, for infinity, right? Within a week, within a week, perhaps within

milliseconds at some point. Yeah. Right. Uh because it might invent new computing substrate and and and all of that.

perhaps they'll use quantum computing and and so then you have this intelligence

explosion in a way that it is impossible to predict how the world is going to be and what I'm saying is this is sort of

like an end of time story like how would you even prepare for that so if if that's coming like why would I spend my

time preparing for I think it's unlikely to happen can't see around the corner yeah but I'd rather prepare

what I was saying there. I'd rather prepare for the more likely world in which we have access to tremendous

power, but the world's not ending and humans are still uh important. I don't I don't know why you say more

likely. I mean, I I think the structure of your argument is is sound. You would prepare for the world that might happen

for which you can prepare. There's literally no point in trying to prepare for a world You can't predict it all.

The only thing you can do is just sort of upgrade your own skills and pay attention. But if I have one message for

the technologists, it's that your confidence about what this can and cannot do is misplaced because you have

without noticing stepped into the realm of the truly complex. In the truly complex, your confidence should drop to

near zero. that you know what's going on. Are these things conscious? I don't know. But will they be highly likely

they will become conscious and that we will not have a test to tell us whether that has happened. Elon Musk predicts

that by 2029 we will have AI with us AGI that surpasses the combined intelligence of

all humans. And Sam Alman actually wrote a blog three months ago that I read where

he said we are confident now Sam Alman being the founder of open arrow which um created chatbt we are confident now that

we know how to build AGI as we have traditionally understood it. When I put these things together, I go back to the

central question of what role do humans have in in this in the sort of professional output in GDP creation. If

it's smarter than all humans combined, if Elon Musk is correct there, and it's able to take actions across the internet

and continue to learn. This is like a central question that I'm hoping I can answer today, which is like where do we

go? Yeah. I I mean in my vision of the world we are in the creative seat. We're sitting there where um we we are

controlling swarms of intelligent being to do our job. You know the way you run your business for example you're sitting

at a computer you have an hour to work. Yeah. and you're going to launch like a thousand SDR you know sales uh you know

representative to go like grab as as many leads as possible and you're generating new update on replet for for

your website here and then uh on this side you you you're actually you you have an AI that's crunching data about

your uh your existing business to to figure out how to improve it and these AIs are kind of somehow all coordinating

together and I am trying to privilege the human like this is my my mission is to uh build tools for people. I'm not

building tools for agents and agents are a tool and so ultimately not only do I think that humans have a privileged

position in the in the world and in the universe. We don't know where conscious consciousness is coming from. We don't

really have the science to explain it. Um I think humans are special. That's one side is is my belief that humans are

are special in the world and another side which I understand that the technology today and I think for the

foreseeable future y is going to be a function of us training data. So there was this whole idea like what if chat

GPT generates uh pathogens. Well have you trained it on pathogens? They were doing that kind of stuff in Wuhan. know

I mean I mean a lot of the biotech companies are essentially using artificial intelligence like I can think

of Abselera I think it's Abselera in Canada their whole business is using AI to create new vaccines using artificial

intelligence and bigger data sets than we've never have had before and I know cuz I was I was very close to one of the

founders of people involved in Absela so that that work is going on anyway and if we think about Wuhan that's it's quite

probably well known now that it came out of a lab and people working in a lab and in that scenario that had a huge impact

and shut down the world. What I'm the central question I'd love to answer before I throw it back open to the the

room is what jobs because I know that you have this perspective. What jobs are going to be made redundant in a world

where I am sat here as a CEO with a thousand AI agents, right? I was thinking of all the names of my of the

people in my company who are currently doing those jobs. I was thinking about my CFO when you talked about processing

business data, my graphic designers, my video editors, etc. So what what jobs are going to be

impacted? Yeah, all of those. Uh so I I think and what do they do? You maybe this is useful for for the audience. I

think if your job is as routine as it comes, your job is gone in the next couple years. So meaning if you if in in

those jobs for example uh quality assurance jobs data entry jobs you're sitting in front of a computer and

you're supposed to click uh and and type things in a certain order operator and those technologies are coming on the

market really quickly and those are going to displace a lot of a lot of uh labor accountants accountants noise yes

I mean I've just pulled a ligament in my in my foot and they did an MRI scan and I had to wait a couple of days for

someone to look at the MRI scan and tell me what it meant. Yeah, I'm guessing that that's gone. Yeah, I think I think

the healthcare ecosystem is hard to predict because of regulation and and again there there's so many limiting

factors on how this technology can permeates the economy because of regulations and and people's willingness

to to take it. But you know things unregulated jobs uh that are purely text in text out. if your job,

you know, you get a you get a message and you produce some kind of artifact that's like probably text or images that

that job is is at risk. So, just to give you some stats here as well, about 50% of Americans who have a college degree

currently use AI. The stats are significantly lower for Americans without a college degree. So, you can

see how a splinter might emerge there and that crack will write widen because people like us at this table are all

messing around with it. But my mom and dad in Plymouth in the southwest, rural England, haven't got like they just

figured out iPhones. So like I got them an iPhone and now they're like texting me back. AI is a million miles away. And

if I start running off with my AGI, my agents, that gap is going to widen. Women are disproportionately affected by

automation, which is what you were talking about there. with about 80% of working women in an at risk job compared

to just over 50% of men according to the Harvard Business Review and jobs requiring only a high school diploma

have an automation risk of 80% while those requiring a bachelor's degree have an automation risk of just 20%. So we

can see again how how this will cause a sort of a it's also a huge risk um with business processor out outsourcing which

is essentially western countries sending jobs to India to the Philippines like at the moment millions of people have been

lifted out of poverty through the ability to do those kind of business process auto outsourcing jobs and those

are all going to go but these they're going to have a thousand employees but but but uh also uh these people are

actually already transitioning to training AI eyes. Mhm. You know, so so there's going to be a massive industry

around training AI until they're trained. Well, no, you you have to continuously acquire new skills and this

is what I'm talking about. I mean, this is again if AI is a function of its data, then you need increasingly more

data. And by the way, we ran out of internet data. I was actually thinking interestingly that this might not be

great for the United States or the UK, the Western world because it is going to be a leveler where now a kid in India

doesn't need a Silicon Valley office and $7 million in investment to throw up a a software company basically. My yeah my

belief is that so I have a I have a more broad definition of AGI and the singularity and for me AGI is do we have

artificial general intelligence in terms of generally speaking can AI just do stuff that humans used to be able to do

and we've already crossed that point we have this general intelligence that we can now all access and 800 million

people a week are now using uh chat GBT it's it's exploded in the last 3 months and then to me a

singularity Uh when the first tractor went out onto a farm, for me that was a singularity moment. Uh because everyone

who worked in farming, it used to take a 100 people to plow a field and now a tractor comes along and two guys with

the tractor can now plow the field in just as much time and now 98 people out of 100 are completely out of a job. We

also always underestimate a technology if it does go on to change history. When you look back through cars, horses,

planes, the Wright brothers just thought of a plane as being something that that army could use, we had no idea of the

application. So someone said to me recently, they said, "When it does change the world, we underestimate the

impact that it will change the world." And I see people now with their estimations of AI and AI agents already

incredibly optimistic. And so if history holds here, we're undershooting the impact it's going to have. And I think

this is the first time in my life where the industrial revolution analogies seem to fall a little bit short. Yeah.

Because we've never seen intelligent that's like you could I could think of this as a I'm not an intelligent person

on this, but I could see that as like the disruption of muscles, whereas this is the disruption of intelligence. That

that's that's exactly the thing is that what makes human beings special is our cognitive capacity and very specifically

our ability to plug our minds into each other. So that the sum is is uh or the whole is greater than the sum of the

parts. That's what makes human cognition special. And what we are doing is we are creating a something that can

technologically surpass it without any of the preconditions that make that a safe process. So yes, we've

revolutionized the world how many different times? It's innumerable. But you know, we've we've made farming

vastly more efficient. That's different than taking our core competency as a species and surpassing ourselves with

the product of our of our labor. I think your question is a good one. Then then what does become? We only have one thing

left. Um we have our muscles which we got rid of in the industrial revolution and then we have our intellect which is

this digital revolution. Now we're left with emotions and agency. So we essentially the the the agency idea I

think we used to judge people on IQ and now IQ is the big leveler and now going forward for the next 10 years we're

going to look at are you a high agency person or a low agency person. Do you have the ability to get things done and

coordinate agents? Do you have the ability to start businesses or give orders to digital armies? uh you know

and and essentially these high agency people are going to thrive in this new world because they have this other thing

that's been bubbling under the service surface which is really interesting when you said agency is going to remain as an

important thing we're sat here talking about AI agents and the crazy thing in a world of AI agents that have super

intelligence is I can just tell my agent listen I'm going on holiday please build me a SAS company that spots a market

opportunity throw up the website post it on my social media channel I'll be in Hawaii I and this new agentic world is

stealing that too cuz now it can take action in the same way that I can browse the internet. I can call Domino's Pizza,

speak to their agentic agent, organize my pizza to be there before I even wake up. And in fact, predictability, you

know, OpenAI now learns and Sam Alman said that they've expanded the memory feature. So, it knows every it's knowing

more and more and more and more about me. It'll almost be able to predict what I want when I want it. It'll know

Steve's calendar. He's arriving at the studio. Make sure his cadence is on the side. Make sure his iPad has the brief

on it. Do the brief. Do the research for me. And everything else say remember Brett's birthday so when I arrive

there'll be something. In fact, it's removing my need for any agency. Yes. And you know again I don't know how to

make this point so that it occurs to people what I'm really suggesting but today maybe it's not conscious but

well let me put it to you this way. If you're conscious, you started out as a child

that wasn't. And although this may not fully encapsulate it, you are effectively an LLM, right? You go from

an unconscious infant to a highly conscious adult. And the process by which you do that has a lot to do with

being trained effectively on words and other things in an environment in exactly the way that we now train these

AIs. So the idea that we can take consciousness off the table, it won't be there till we figure out how to program

it in and we're safe because we don't know how consciousness works. I take the opposite lesson. We've created the exact

thing that will produce that phenomenon and then we can have philosophers debate whether it's real consciousness or it

just behaves exactly as if it were. And the answer is those aren't different. Doesn't matter. And the same thing is

true for agency. you know, especially if you've created an environment in which these AIs are de facto competitors, what

you're effectively doing is creating an evolutionary environment in which they will evolve to fill whatever niches are

there. And we didn't spell out the niches. So, I have the sense we have um we have invited we have we have created

something that truly is going to function like a new kind of life and it's especially troubling because it

speaks our language. So that leads us to believe it's more like us than it is and it's actually potentially quite

different. So but by the way, he's the optimist here, right? Like he's so optimistic about LMS and how how they're

going to they're going to evolve. Yes. It's amazing. It's amazing technology. Like I think it raised global IQ, right?

Like 800 million people like 800 million people are that much more intelligent and emotionally intelligent as well.

Like I know people who previously were very coarse and they kind of robbed people the the the wrong way. They they

would say things in not so polite way and then suddenly they started putting their the you know what they're saying

through chat in order to kind of make it kinder and nicer and they're more liked now. And so not only is it uh making us

more intelligent but also it allows us to be the best version of ourselves. And the the scenario that you're talking

about, I don't think I don't know what's wrong with that. Like, you know, you know, the I would want less agency in

certain places. Like, I would want something to help me not, you know, open up a peanut butter jar at night, right?

You know, there are places in my life where I need more control and I would rather seed it to some kind of entity

that could help me make better choices. I mean unfortunately even if there is some small group of elites that are able

to go to Hawaii while something else does the mundane details of their business

building. We are rather soon going to be faced with a world that has billions of people who do not have the skills to

leverage AI. Some of them will be necessary for a time. you're going to need plumbers. But this is

also not a long-term solution because not only are there not enough of those jobs,

um, but of course we have humanoid robots that once imbued with AI capacity will also be able to take, you know,

they'll be able to crawl under your house into the crawl space and fix your plumbing.

So what typically happens when you have a massive economic contraction that arises from the fact that a huge number

of people are out of work is that the elites start looking at those people and thinking well we don't really need them

anyway. And so the idea that this AI disruption doesn't lead us to some very human catastrophe I think is overly

optimistic and that we need to start preparing right now. What are the rights of a person who has had whatever it is

that they've invested in completely erased from the list of needs? Is that person responsible for not having

anticipated AI coming? And is it their problem that that they are now starving and they're being eyed by others as you

know a useless eater? I don't think so. How is it different than uh when the uh uh what's it called the the looming

machine came and the textile workers you know that the result of the in the lad sort of revolution? uh h how is it how

is it different than any time in history when uh technology uh automated a a lot of people out of out of jobs? I would

say scale and speed that's how it's different and the scale and speed is going to result in a an unprecedented

catastrophe because the rate at which people are going to be simultaneously sidelined not just in one industry but

across every industry is just simply uh and it also did actually happen. There was a there was an uh for the first 50

years of industrialization from like late 1700s to early 1800s, you actually the Charles Dickens novels are

essentially people coming from the farms who are displaced arriving in cities, kids living on the streets. Uh the

British decided to pick everyone up and send them over to the over to Australia, which is where I came from. um and uh

you know there there were this there was this massive issue of displacement. I think we're going to go into a high

velocity economy where rather than this long arc of career that lasts 45 years, we're going to have these very fast

careers that last 10 months to 36 months. and you invent something, you take it to market, you put together a

team of five to 10 people who work together, you then get disrupted, you come Can I mention a story here? Uh

there's an entrepreneur that used Replet in a similar way. Uh his name is Billy Howell. You can find him on YouTube on

the internet. He would go to Upwork and he would find what people are asking for different requests for certain apps,

technologies. Then he would take the what they're asking for, put it into replet, make it an application, call

them, tell them, "I already have your application. Would you pay $10,000 for it?" And so, so that's sort of an

arbitrage opportunity that's that's there right now. That's not arbitrage. That's theft. How is no what is it? How

is that theft? You have somebody who has an idea that can be brought to market and somebody else is cryptically

detecting it and then selling back their own idea to them. Well, they're paying them to do that. They're saying, "I will

give you $500 if you if someone makes this for me." Right? But this is what I more or less think is going to happen

across the whole economy is that yes, from this perspective, we can see that everybody is suddenly empowered to build

a great business. Well, what do we think about the folks who are going to be displaced from the top? What are they

going to think about all these people building all of these highly competitive businesses? And are they going to find a

way to do, you know, what venture capital has done or what record producers have done? What they're going

to do is they're going to take their superior position at the top and they are going to take most of the wealth

that is produced by all of these people who have these ideas that in a proper market would actually create businesses

for them and they're going to parasitize them. I think that we with this in introduction of AI and AI

agents old value has moved and now it's not going to be the case that the idea itself is the moat and it's not going to

be the case that resources are the moat. So in such a scenario you still have to figure out distribution. You still have

to have for example like an audience. So if you're a podcaster now you have a million followers on Twitter. you're in

a prime position because you now have something that the the great guy with a great idea with no audience has you have

inbuilt distribution. So I now think actually much of the game might be moving to like yeah still about taste

and idea but also the the mo is distribution. Yeah. And speaking of adaptive systems um the one of the

adaptation that will happen is people will seek uh humans and will seek proof of humanity. Oh, I agree that uh

authenticity is going to become the coin of the realm and anything that can be faked or cheated is going to be devalued

and things you know spontaneous jazz or you know comedy that is interactive enough that it couldn't possibly have

been generated with the aid of AI those things are going to become prioritized you know spontaneous oratory rather than

speeches answers some of your questions no it answers it answers my question for the tiny number of people who are in a

position to do those things. Stephen, you use the mo word moat. Um, which I think is a really important word for

entrepreneurs. We like we like have to have a moat. We think a lot about moes and it's an industrial age. A lot of

people don't even know what a moat what you mean by moat. It's just I often think about this idea of what are the

moes that are left. So to define how I define a mo, you've got a castle and it's got a like a small circle of water

around it. And once upon a time that circle of water defended the castle from attack and you can pull up the

drawbridge so nobody could attack you very easily. It's a de defense from something. So it's your it's your

shield. It's your your defense. And once upon a time as an entrepreneur, you know, I've got a software company in San

Francisco called Third Webb and we raised almost $30 million. We have a team of 50 great developers. And much of

our moat was you can't compete with us if you don't have the 50 developers and the $30 million in the bank. How much of

that 30 million went to coding? The vast majority of it. I mean what else are we going to do? What else we do? So this is

a good thing. I think modes are a bad thing. Okay, let me make the argument there. Uh so everyone is looking for

modes you know for example like one of the more uh significant modes is network effects. Yeah, you know, so you can't

compete with Facebook or Twitter because to move people from Facebook or Twitter, you need to it's the collective

action problem. You need to move them all at once because if one of them moves, then it's the network is not

valuable, they'll go back. So you have this chicken and egg problem. Let's say that we have a more decentralized way of

doing social networks that will remove the power of Twitter to kind of censor and I I think you're at the other end of

of censorship, right? And so part of my optimism about humanity is that um generally there's self-correction.

Democracy is a self-correcting system. Uh free markets are largely self-correcting systems. There are

obvious problems with with free markets that that we can discuss. But take um health, you know, there is obesity uh

epidemic. This period of time when uh companies, you know, ran loose kind of making this sugary, salty, fatty kind of

snacks and everyone gorged on them and everyone got very uh you know, unhealthy. And now you have Whole Foods

everywhere. Today, people in Silicon Valley, they don't go to bars at all. They go to running clubs. That's how you

meet. That's how you go find a date. You go to running clubs. And so, there was a shift that happened because there was a

reaction. Obviously, cigarettes is another example. You know, you were talking about phones and our addiction

to phones. And I see a shift right now like in my uh friend circle like people who are constantly kind of on their

phones is already kind of frowned upon and they don't want to hang out with you because you're you're constantly staring

on on your phone. So there's always these reactions and and but the problem is you you reference selfcorrection and

I agree that there's actually an automatic feature of the universe in which the self-correction happens. You

can't have a positive feedback that isn't reigned in by some outer negative feedback. But the corrections, the list

of corrections involves things like you point to where people become enlightened and they realize that they're doing

themselves harm with either the sugar that they're consuming or the dopamine traps on their phone and they get

better. But also on the list of corrective patterns are genocide and war and you know parasetism. And the problem

is these things are destructive of wealth. And so you allude to the superior fact of an open market

without moes. Presumably the benefit of that is that more wealth gets created because people aren't kept from doing

things that are productive. I see that. But then what is the product of all of this new wealth that is going to be

generated by a world empowered by AI? Does it end up so highly concentrated that you have a tiny number of ultra

elites and a huge number of people who are utterly dependent on them? What becomes of those people? The learning

process, the self-correction pro process goes through harm in order to get to that more enlightened solution. There's

nothing that protects us from the harm phase being so apocalyptically terrible that, you know, we get to the other side

of it and we say, "Well, that was a hell of a correction." Or maybe there's nobody there to even say that. Those are

also on the table. It reminds me of a mouse trap where you see the cheese and we're going, "Oh my god, my

grandmother's going to be able to do some research and oh my god, my life's going to get easier." So you head closer

and closer to the cheese. And historically, if we look at all of the last 10,000 years, it's a very small

number of elites who own absolutely everything and a very large number of surfs and peasants who have a

subsistence living. You know, if the elites are too greedy and they freeze out the peasants at too high a level and

they try to use brutality or Yeah. eventually it comes back to haunt them. And so what you get is a recognition

that you you need a system that does balance these things and you know the west has the best system that we've ever

seen. It's one in which we agree on a level playing field. We never achieve it but we agree that it's a desirable thing

and the closer we get to it the more wealth we create. But again, if AI empowers those with ill

intent at a higher rate that it empowers those who are wealth creating and pro-social, we may be in for a massive

regression in how fair the the market that the West is. Is that your top concern versus economic displacement?

And I think they're the same thing. How are they the same thing? because the economic displacement is

going to start. I don't know how many million people are going to be displaced from their jobs in the US.

Suddenly, we're going to have a question about whether or not we have obligations to them. And you agree with that, don't

you? Yes. But but but again, it's it's the no pain, no gain. I mean, we're going to go through a period of of

disruption. And I think at the other end, the old, you know, sort of oppressive systems will be broken and

we're going to create perhaps a fair world, but it's going to have its own its own problems. And what's the scale

of that disruption in your estimation? It's hard to say because uh you know there's this concept of limiting factors

like you know there is um regulation there's the appetite of people to today for example the health care system is

very resistant to innovation because of regulation you know and that's a that's a bad thing on the regulation point it's