The Future of Technology: A Conversation with NVIDIA CEO Jensen Huang

at some point you have to believe something we've reinvented Computing as we know it what is the vision for what

you see coming next we asked ourselves if it can do this how far can it go how do we get from the robots that we have

now to the Future world that you see Cleo everything that moves will be robotic someday and it will be soon we

invested tens of billions of dollars before it really happened no that's very good you did some research but the big

breakthrough I would say uh is when we that's Jensen Wong and whether you know it or not his

decisions are shaping your future he's the CEO of Nvidia the company that skyrocketed over the past few years to

become one of the most valuable companies in the world because they led a fundamental shift in how computers

work unleashing this current explosion of what's possible with technology nvidia's done it again we found

ourselves being one of the most important technology companies in the world and potentially ever a huge amount

of the most futuristic Tech that you're hearing about in Ai and Robotics and gaming and self-driving cars and

Breakthrough Medical Research relies on new chips and software designed by him and his company during the dozens of

background interviews that I did to prepare for this what struck me most was how much Jensen Wong has already

influenced all of our Lives over the last 30 years and how many said it's just the beginning of something even

bigger we all need to know what he's building and why and most importantly what he's trying to build next welcome

to huge conversations [Music] thank you so much for doing so happy to

do it before we dive in I wanted to tell you how this interview is going to be a little bit different than other

interviews I've seen you do recently okay I'm not going to ask you any questions about you could ask company

finances thank you I'm not going to ask you questions about your management style or why you don't like one-on ones

I'm not going to ask you about regulations or politics I think all of those things are important but I think

that our audience can get them well covered elsewhere okay what we do on huge if true is we make optimistic

explainer videos and we've covered I'm the worst person to be an explainer video I think you might be the best and

I think that's what I'm really hoping that we can do together is make a joint explainer video about how can we

actually use technology to make the future better yeah and we do it because we believe that when people see those

better Futures they help build them so the people that you're going to be talking to are awesome they are

optimists who want to build those better Futures but because we cover so many different topics we've covered

supersonic planes and quantum computers and particle colliders it means that millions of people come into every

episode without any prior knowledge whatsoever you might be talking to an expert in their field who doesn't know

the difference between a CPU and a GPU or a 12-year-old who might grow up one day to be you but is just starting to

learn um for my part I've now been preparing for this interview for several months I've including doing background

conversations with many members of your team but I'm not an engineer so my goal is to help that audience see the future

that you see so I'm going to ask about three areas the first is how did we get here what were the key insights that led

to this big fundamental shift in Computing that we're in now the second is what's actually happening right now

how did those insights lead to the world that we're now living in that seems like so much is going on all at once and the

third is what is the vision for what you see coming next in order to talk about this big

moment we're in with AI I think we need to go back to video games in the

9s at the time I know game developers wanted to create more realistic looking Graphics but the hardware couldn't keep

up with all of that necessary math not enough man Nvidia came up with a solution that would change not just

games but Computing itself could you take us back there and explain what was happening and what were

the insights that led you and the Nvidia team to create the first modern GPU so in the early '90s when we first started

the company we observed that in a software program inside it there are just a few lines of code maybe 10% of

the code does 99% % of the processing and that 99% of the processing could be done in parallel however the other 90%

of the code has to be done sequentially it turns out that the proper computer the perfect computer is one that could

do sequential processing and parallel processing not just one or the other that was the big observation and we set

out to build a company to solve computer problems that normal computers can't and that's really the beginning of Nvidia

my favorite visual of why a CPU versus a GPU really matters so much is a 15-year-old video on the Nvidia YouTube

channel where the Mythbusters they use a little robot shooting paintballs one by one to show solving problems one at a

time or sequential processing on a CPU but then they roll out this huge robot that shoots all of the paintballs

at once doing smaller problems all at the same time or parallel processing on a GPU 3 2

1 so Nvidia unlocks all of this new power for video games why gaming first the video games uh requires parallel

processing for uh processing 3D graphics and we chose video games because one we loved the application it's a simulation

of virtual worlds and who doesn't want to go to Virtual Worlds and and we had the good observation that video games

has potential to be the largest market for for entertainment ever and it turned out to be true and having it being a

large Market is important because the technology is complicated and if we had a large Market our R&D budget could be

large we could create new technology and that flywheel between technology and market and greater technology was was

really the flywheel that got Nvidia to become one of the most important technology companies in the world it was

all because of video games I've heard you say that gpus were a time machine yeah could you tell me more about what

you me by that a GPU is like a time machine because it lets you see the future sooner one of the most amazing

things anybody's ever said to me was a uh quantum chemistry scientist he said Jensen because of invidious work I can

do my life's work in my lifetime that's time travel he was able to do something that was beyond his lifetime within his

lifetime and and this because we make applications run so much faster and you get to see the future and so

when you're doing weather prediction for example you're seeing the future when you're doing a a simulation a virtual

City with virtual traffic and we're um simulating our self-driving car through that Virtual City we're doing time

travel so parallel processing takes off in gaming and it's allowing us to create worlds in computers that we never could

have before and and gaming is sort of this this first incredible cas Cas of parallel

processing unlocking a lot more power and then as you said people begin to use that power across many different

Industries the case of the of the quantum chemistry researcher when I've heard you tell that

story it's that he was running molecular simulations in a way where it was much faster to run in parallel on Nvidia gpus

even then than it was to run them on the supercomputer with the CPU that he had been using before yeah that's true so oh

my God it's revolutionizing all of these other Industries as well MH it's beginning to change how we see what's

possible with computers and my understanding is that in the early 2000s you see this and you realize that

actually doing that is a little bit difficult because what that researcher had to do is he had to sort of trick the

gpus into thinking that his problem was a graphics problem that's exactly right no that's very good well you did some

research so you create a way to make that a lot easier that's right specifically it's a platform called cuda

which lets programmers tell the GPU what to do using programming languages that they already know like C and that's a

big deal because it gives way more people easier access to all of this computing power could you explain what

the vision was that led you to create Cuda partly uh researchers uh discovering it partly internal uh

inspiration and um uh and partly solving a problem and you know a lot of interesting interesting IDE ideas come

out of that soup you know some of it is is aspiration and inspiration some of it is just desperation you know and and so

in in the case of Cudo is very much this the same way and um probably the first external ideas of using our gpus for

parallel processing emerged out of some interesting work in medical imaging a couple of researchers at Mass General

were using it uh to do uh CT reconstruction they were using our Graphics process for that reason and it

inspired us meanwhile the problem that we're trying to solve inside our company has to do with the fact that when you're

trying to create these virtual worlds for video games you would like it to be beautiful but also

Dynamic water should flow like water and explosions should be like explosions so there's particle physics you want to do

fluid dynamics you want to do and uh that is much harder to do if your pipeline is only able to do computer

graphics and so we have a natural reason to want to do it in in the the the market that

we were serving so researchers were also um horsing around with using our gpus for general purpose uh acceleration and

and so there there are multiple multiple factors that were coming together in that soup uh we just when the time came

and uh uh we decided to to uh do something proper and create a Cuda as a result of that fundamentally the reason

why why I was certain that Cuda was going to be successful and we we uh put the whole company behind it was because

fundamentally uh our GPU was going to be the highest volume parallel processors built in the world because the market of

video games was so large and so this architecture has a good chance of reaching many people it has seemed to me

like creating Cuda was this incredibly optimistic huge if true thing to do where you were saying if we create a way

for many more people to use much more computing power they might create incredible things and then of course it

came true they did in 2012 a group of three researchers submits an entry to a famous competition where the goal is to

create computer systems that could recognize images and label them with categories and their entry just crushes

the competition it gets way fewer answers wrong it was incredible it blows everyone away it's called alexnet and

it's a kind of AI called the neural network my understanding is one reason it was so good is that they used a huge

amount of data to train that system and they did it on Nvidia gpus all of a sudden gpus weren't just a way to make

computers faster and more efficient they're becoming the engines of a whole new way of computing we're moving from

instructing computers with step-by-step directions to training computers to learn by showing them a huge number of

examples this moment in 2012 really kicked off this truly sizmic shift that we're all seeing with AI right now could

you describe what that moment was like from your perspective and what did you see it would mean for all of our futures

when you create something new like Cuda if you build it they might not come and and that's that's always the the cynic

perspective however The Optimist perspective would say but if you don't build it they can't come and that's

usually how we look at the world you know we we have to reason about intuitively why this would be very

useful and in fact uh in 2012 ilas Suk and Alis kvki and Jeff Hinton in the University of Toronto the lab that they

were at they reached out to a gForce GTX 580 because they learned about Cuda and that Cuda might be able to to be used as

a parallel processor for training alexnet and uh so our inspiration that GeForce could be the the vehicle to

bring out this parallel architecture into the world and that researchers would somehow find it someday was a good

was a good strategy it was a strategy based on Hope um but it was also reasoned hope the the thing that really

caught our attention was simultaneously we were trying to solve the computer vision problem inside the company and we

were trying to get Cuda to uh be a good computer vision uh processor and we were frustrated by by uh a whole bunch of

early developments internally with respect to our our um computer vision effort and getting Cuda to be able to do

it and all of a sudden we saw alexnet um this new algorithm that that is uh completely

different than computer vision algorithms before it uh take a giant leap in terms of capability for computer

vision and when we saw that it was partly out of interest but partly because we were struggling with

something ourselves and so we were we were highly interested to want to see it work and so when we when we looked at

alexnet we were inspired by that um but the big breakthrough I would say uh is when we when we saw alexnet we

asked ourselves you know how far can alexnet go if it can do this with computer vision how far can it go

and if it if it could go to the limits of what we think it could go um the type of problems it could solve what would it

mean for the computer industry and what would it mean for the computer architecture and we were we were um uh

we rightfully reasoned that if machine learning if the Deep learning architecture can scale uh

the vast majority of machine learning problems could be represented with deep neural networks and the type of problems

we could solve with machine learning is so vast that it has the potential of reshaping the computer industry

altogether and which prompted us to uh re-engineer the entire Computing stack which is where uh djx came from and this

little baby djx sitting here um uh all of this came from from that observation that we ought to reinvent the entire

Computing stack layer by layer by layer you know computers after 65 years since uh IBM system 360 introduced modern

general purpose Computing we've reinvented Computing as we know it to think about this as a whole story so

parallel processing reinvents modern gaming and revolutionizes an entire industry then that way of computing that

parallel processing begins to be used across different Industries you invest in that by building Cuda and then Cuda

and the use of gpus allows for a a step change in

neural networks and machine learning and begins a sort of Revolution that we're now seeing uh only increase in

importance today all of a sudden computer vision is solved all of a sudden speech

recognition is solved all of a sudden language understanding is solved these incredible problems associated with

intelligence one by one by one by one where we had no solutions for in past desperate desire to have solutions

for all of a sudden one after another get solved you know every couple of years it's incredible yeah so you're

seeing that in 2012 you're looking ahead and and believing that that's the future that you're going to be living in now

and you're making bets that get you there really big bets that have very high stakes and then my perception as a

lay person is that it takes a pretty long time to get there you make these best eight years eight uh 10 years so my

question is if Alex that happened in 2012 and this audience is probably seeing and hearing

so much more about Ai and Nvidia specifically 10 years later MH why did it take a decade and also because you

would Place those bets MH what did the middle of that decade feel like for you well that's a good question it probably

felt like today you know to me to me there's always some problem and then there's there's some reason to be to be

impatient there's always some reason to be uh happy about where you are and and there's always many reasons to carry on

and so so I I think as I was reflecting a second ago that sounds like this morning so but I would say that in all

things that we pursue first you have to have core beliefs you have to reason from from

your best principles um and ideally you're reasoning from it from princip of either physics or uh deep understanding

of of uh of the industry or deep understanding of the science wherever you're reasoning from um you reason from

first principles and at some point you have to believe something and if those principles don't change and the

assumptions don't change uh then you there's no reason to change your core beliefs and then along the way there's

always some uh evidence of of um you know of success and and that you're you're leading in the right direction

and sometimes you know you go a long time without evidence of success and you might have to course correct a little

but um the evidence comes and and if you feel like you're going in the right direction we just keep on going the

question of why did we stay so committed for so long the answer is actually the opposite there was no reason to not be

committed because we are we believed it and and um I've believed in Nvidia for 30 plus years and and I'm still here

working every single day and uh there's no fundamental reason for me to change my belief system and um uh I

fundamentally believe that that the work we're doing in revolutionizing Computing is as true today even more true today

than it was before and and um uh and so we'll we'll stick with it you know until until otherwise um there's of

course very difficult times along the way you know when you're investing in something and nobody else believes in it

and cost a lot of money and uh you know maybe investors or or others would rather you just keep the profit or you

know whatever it is improve the share price or whatever it is um but you have to believe in your future you have to

invest in yourself and and um uh we believ this so deeply uh that that we we invested you know tens of billions of

dollars uh before before it really happened and um uh yeah it it was it was 10 long years but it was it was it was

fun along the way how would you summarize those core beliefs what is it that you believe about the way

computers should work and what they can do for us that keeps you not only coming through that decade but also doing what

you're doing now making bets I'm sure you're making for the next few decades the first core believe uh was our first

discussion was about accelerated Computing parallel Computing versus versus general purpose Computing we

would add uh two of those processors together and we would do accelerated Computing and I continue to believe that

today the second was was the recognition that these deep learning networks these dnns that came to the public during 2012

these deep neural networks have the ability to learn patterns and relationships from a whole bunch of

different types of data and that it can learn more and more uh nuanced features if it could be larger and larger and

it's easier to make them larger and larger make them deeper and deeper um or wider and wider and so the scalability

of the architecture is is um uh empirically true uh the uh the fact that model size and the data size being

larger and larger can learn more knowledge uh is also true uh empirically true and so uh if that's the case uh you

could you know what what are the limits there not unless there's a physical limit or an architect ual limit or

mathematical limit and it was never found and so we believe that you could scale it then the question the only

other question is what can you learn from data what can you learn from experience data is basically digital

versions of human experience and so what can you learn uh you obviously can learn object recognition from images you can

learn speech from just listening to sound you can learn uh even languages and vocabulary and syntax and grammar

and all just by studying a whole bunch of letters and words so we've now demonstrated that AI or deep learning

has the ability to learn almost any modality of data and it can translate to any

modality of data and so what does that mean you can go from text to text right summarize a paragraph you can go from

text to text translate from language to language you can go from text to images that's image generation you can go from

images to text that's captioning you can even go from amino acid sequences to protein

structures in the future you'll go from protein to words what does this protein do or um give me an example of a protein

that has these properties you know uh identifying a drug Target um and so you could just see that all of these

problems are around the corner to be solved uh you can go from words to video why can't you go from words

to action tokens for a robot you know from the computer's perspective how is it any different and

so it it opened up this universe of opportunities and universe of problems that we can go solve and um that that

that gets us quite excited it feels like we are on the cusp of This truly enormous change when I

think about the next 10 years I unlike the last 10 years I know we've gone through a lot of change already but I

don't think I can predict anymore how I will be using the technology that is currently being developed that's exactly

right I think the last 10 the reason why you feel that way is the last 10 years was really about the science of AI the

next 10 years we're going to have plenty of Science of AI but the next 10 years is going to be the application science

of AI the fundamental science versus the application science and so the the applied research the application side of

AI now becomes how can I apply AI to digital biology how can I apply AI to climate technology how can I apply AI to

agriculture to fishery to robotics to Transportation um optimizing Logistics how can I apply AI to you know teaching

how do I apply AI to you know podcasting right and so I'd love to choose a couple of those to help people see how this

fundamental change in Computing that we've been talking about is actually going to change their experience of

their lives how they're actually going to use technology that is based on everything we just talked about one of

the things that I've now heard you talk a lot about and I'm have a particular interest in is physical AI or in other

words robots my friends meaning humanoid robots but also robots like self-driving cars and smart

buildings or autonomous warehouses or autonomous lawnmowers or more from what I understand we might be about to see a

huge leap in what all of these robots are capable of because we're changing how we train them up until recently

you've either had to train your robot in the real world where it could get damaged or wear down or you could get

data from Fairly limited sources like humans in motion capture suits but that means that robots aren't getting as many

examples as they'd need to learn more quickly but now we're starting to train Rob OTS in digital worlds which means

way more repetitions a day way more conditions learning way faster so we could be in a big bang moment for robots

right now and Nvidia is building tools to make that happen you have Omniverse and my understanding is this is 3D

worlds that help train robotic systems so that they don't need to train in the physical world that's exactly right you

just just announced Cosmos which is ways to make that 3D Universe much more realistic so you can get all kinds of

different um if we're training something on this table many different kinds of lighting on the table many different

times of day many different you know experiences for the robot to go through so that it can get even more out of

Omniverse as a kid who grew up loving data on Star Trek Isaac asmon's book and just dreaming about a future with robots

how do we get from the robots that we have now to the Future world that you see of Robotics yeah let me use um

language models maybe chat GPT as a reference for understanding um Omniverse and Cosmos and so so first of all when

chat GPT first came out it it was it was um extraordinary and it has the ability to do uh to basically uh from your promp

uh generate text however as amazing as it was it has um the tendency to hallucinate uh if it goes on too long or

if uh it pontificates about a topic it you know is not informed about it'll still do a good job generating plausible

answers um it just wasn't grounded in the truth and so so um uh people people uh called it

hallucination and so the Next Generation uh shortly it was it had the ability to be conditioned by um context so you

could upload your PDF and now it's grounded by the PDF the PDF becomes the ground truth it could be it could

actually look up search and then the search becomes uh its ground truth and between that it could reason about uh

what is how to produce the answer that you're asking for and so so the first part is a generative Ai and the second

part is ground truth okay and so now let's come into the the the the physical world uh the world model we need a

foundation model just like we need CH chat GPT had a core Foundation model that was the Breakthrough in order for

robotics to to be smart about the physical world it has to understand things like gravity friction

inertia um geometric and spatial awareness it has to uh understand that an object is sitting there even when I

looked away when I come back it's still sitting there object permanence um it has to understand cause and effect if I

tip it it'll fall over um and so so these kind of physical common sense if you will has to be captured or encoded

into a world Foundation model so that the AI has World Common Sense okay and so so we have to goes somebody has to go

create that and that's what we did with Cosmos we created a World Language model just like trpd was a language model this

is a world model the second thing we have to go do is we have to do the same thing that we did with PDFs and context

and um grounding it with ground truth and so the way we augment Cosmos with ground truth is with physical

simulations because Omniverse uses physics simulation which is based on principled solvers the the mathematics

is Newtonian physics is the right it's the math we know okay all of the the fundamental laws of physics uh We've

understood for a very long time and it's encoded into captured into Omniverse that's why Omniverse is a simulator and

using the simulator to ground or to condition Cosmos we can now generate an infinite number of Stories of the future

and they're grounded on physical truth just like between PDF or sege plus chat GPT we can generate an

infinite amount of interesting things answer a whole bunch of interesting questions the combination of Omniverse

plus Cosmos you could do that for the physical world so to illustrate this for the audience if you had a robot in a

factory and you wanted to make it learn every route that it could take instead of manually going through all of those

rots which could take days and could be a lot of wear and tear on the robot we're now able to simulate all of them

digitally in a fraction of the time and in many different situations that the robot might face it's dark it's blocked

it's Etc so the robot is now learning much much faster it seems to me like the future might look very different than

today if you play this out 10 years how do you see people actually interacting with this technology in the near

future Cleo everything that moves will be robotic someday and it will be soon you know the the idea that will'll be

pushing around a lawn mower is already kind of silly you know maybe people do it because because it's fun but but

there's no need to and and um uh every car is going to be robotic human robots uh the technology necessary to make it

possible uh is just around the corner and so everything that moves will be robotic and they'll they'll learn how to

be a robot in Omniverse Cosmos and will generate all these plausible physically plausible

Futures and the the robots will learn from them and then they'll come into the physical world and you know it's exactly

the same a future where um well you're just surrounded by robots is for certain and I'm just excited about having my own

R2-D2 and of course R2-D2 wouldn't be quite the can that it is and roll roll around it'll be you know

R2-D2 yeah it'll probably be a different physic iCal embodiment um but it's always R2 you know so my R2 is going to

go around with me sometimes it's in my smart glasses sometimes it's in my phone sometimes it's in my PC um it's in my

car so R2 is with me all the time including you know when I get home you know where I left a physical version of

R2 and you know whatever whatever that version happens to be you know we we'll interact with R2 and so I think the idea

that we'll have our own R2-D2 for our entire life and it grows up with us um that's a certainty now yeah I think a

lot of news media when they talk about Futures like this they focus on what

could go wrong and that makes sense there is a lot that could go wrong we should talk about what could go wrong so

we could keep it from from going wrong yeah that's the approach that we like to take on the show is what are the big

challenges so that we can overcome them yeah what buckets do you think about when you're worrying about this future

well there's there's a whole bunch of the stuff that everybody talks about bias or toxicity or or just

hallucination um you know speaking with great confidence about something it knows nothing about and as a result we

rely on that information um uh generating that's a version of generating uh fake information fake fake

news or fake images or whatever it is of course impersonation um it it does such a good job U pretending to be a human it

could be it could do an incredibly good job pretending to be a specific human and so so the the um uh the the spectrum

of of um areas we have to be concerned about uh is fairly clear and there's a lot of there's a lot of people who are

working on it there's there's a some of the stuff some of the stuff related to AI safety um requires deep research and

deep deep engineering and that's simply it wants to do the right thing it just didn't perform it right and as a result

hurt somebody you know for example uh self-driving car that wants to drive nicely and and drive properly and just

somehow the sensor broke down or or uh it didn't detect something or um you know made it to to aggressive turn or

whatever it is it did it poorly it did it wrong wrongly and so that's that's a whole bunch of engineering that has to

be done to to make sure that AI safety is upheld by making sure that the product function properly and then and

then lastly you know whatever what happens if the S the the AI wants to do a good job but the system failed meaning

the AI wanted to stop um stop stop something from happening and it turned out just when it wanted to do it um the

machine broke down and so this is no different than than a flight computer inside a plane having three versions of

them and then uh so there's there's triple redundancy inside the system inside autopilots and then you have two

pilots and then you have um uh air traffic control and then you have other Pilots watching out for these P

pilots and so so that the AI Safety Systems has to be architected as a community such that such that these AIS

uh one um uh work work function function properly when they don't function properly they don't put People In Harm's

Way and that they're sufficiently Safety and Security Systems all around them uh to make sure that that um uh we keep AI

safe and so there's this spectrum of conversation is gigantic and and um uh you know we have to take the parts take

the parts apart and and build them as Engineers one of the incredible things about this moment that we're in right

now is that we no longer have a lot of the technological limits that we had in a world of CPUs and sequential

processing and we've unlocked not only a new way to do Computing and

and but also a way to continue to improve parallel processing has a a different kind of physics to it than the

improvements that we were able to make on CPUs I'm curious what are the scientific or technological limitations

that we face now in the current world that you're thinking a lot about well everything in the end is about how much

work you can get done within the limitations of the energy that you have and so so that that's a that's a

physical limmit and uh the laws of physics uh about transporting s information and

um transporting bits flipping bits and transporting bits um at the end of the day the energy it takes to do that um

limits what we can get done and the amount of energy that we have limits what we can get done we're far from

having any fundamental limits that keep us from advancing in the meantime we seek to build better and more energy

efficient computers this this little computer uh the the big version of it was uh

$250,000 pick up yeah yeah that's little baby baby digits yeah this is an AI supercomputer the version that I

delivered this is just a prototype so it's a mockup and so the the the very first version was djx1 I delivered to

open AI in 2016 and that was $250,000 10,000 times more power more energy necessary uh than this version and this

version has six times more performance W I know it's incredible we're in a whole in the world and it's only since 2016

and so eight years later we've in increased the Energy Efficiency of computing by 10,000 times and imagine if

we became 10,000 times more energy efficient or if a car was 10,000 times more energy efficient or electric light

bulb was 10,000 times more energy efficient our light bulb would be right now instead of 100 Watts 10,000 times

less producing the same illumination yeah and so and so the the Energy Efficiency of computing particularly for

AI Computing that we've been working on has advanced incredibly and that's That's essential because we want to

create you know more intelligent systems and and we want to use U more computation to be smarter and and uh so

Energy Efficiency to do the work is our number one priority When I Was preparing for this interview I spoke to a lot of

my engineering friends and this is a question that they really wanted me to ask so you're really speaking to your

people here you've shown a value um of increasing accessibility and abstraction

with Cuda and allowing more people to use more computing power in all kinds of other ways as applications of technology

get more specific I'm thinking of Transformers in AI for example for the audience a Transformer is a very popular

more recent structure of AI That's now used in a huge number of the tools that you've seen the reason that they're

popular is because Transformers are structured in a way that helps them pay attention to key bits of information and

give much better results you could build chips that are perfectly suited for just one kind of AI model but if you do that

then you're making them less able to do other things so as these specific structures or architectures of AI get

more popular my understanding is there's a debate between how much you place these bets on burning them into the chip

or designing Hardware that is very specific to a certain task versus staying more General and so my question

is how do you make those bets how do you think about whether the solution is a car that could go anywhere or it's

really optimizing a train to go from A to B mhm you're making Bets with huge stakes and I'm curious how you think

about that yeah and and that that now comes back to um uh exactly your question what are your core

beliefs and and the question the the core belief either one that Transformer is the

last AI algorithm AI architecture that any researcher will ever discover again or that um Transformers is a stepping

stone towards uh evolutions of Transformers that are uh barely recognizable as a Transformer uh years

from now and we believe the latter and the reason for that is because um you just have to go back in history and ask

yourself in the world of of uh computer algorithms in the world of software in the world of of um uh uh engineering and

innovation has one idea stayed along that long and the answer is no and so that's the that's kind of the Bea that's

that's in fact the essential beauty of a computer that it's able to do something today that no one even imagined possible

10 years ago and if you would have if you would have turned that computer 10 years ago into a

microwave then why would the applications keep coming and so we believe we believe in the in the in the

richness of innovation and the richness of invention and we want to create an architecture that let inventors and

innovators and software programmers and AI researchers swim in the soup and come up with some Amazing Ideas look at

Transformers the the fundamental characteristic of a transformer is this idea called tension mechanism and it

basically says the Transformer is going to understand the meaning and the relevance of every single word with

every other word so if you had 10 words it has to figure out the relationship across 10 of them but if you have a

100,000 words or if you're Conta text is now as large as read a PDF and that read a whole bunch of PDFs and the context

window is now like a million tokens the processing all of it across all of it is just impossible and so the

way you solve that problem is there all kinds of new ideas flash attention or hierarchical attention or you know all

the wave attention I just WR read about the other day um the number of different types of attention mechanisms that have

been invented since the Transformer is quite extraordinary and so so I I think that

that's going to continue and um we believe it's going to continue and that that computer science hasn't ended and

that AI research have not all given up and we haven't given up anyhow and and uh and that that having a computer that

enables the the the flexibility of of research and Innovation and and um new ideas is fundamentally the most

important thing one of the things that I am just so curious about you design the chips

there are companies that assemble the chips there are companies that design Hardware to make it possible to work at

nanometer scale when you're designing tools like this how do you think about design in the context of what's

physically possible right now to make what are the things that you're thinking about with sort of pushing that limit

today um the way we do it is even though even though we have things made like for example our chips are made by

tsmc um even though we have them made by tsmc we assume that we need to have the Deep expertise that tsmc has and so we

have people in our company who are incredibly good at semiconductive physics so that we have a feeling for we

have an intuition for what are the limits of of of what today's semiconductor physics can do and then we

work very closely with them to discover the limits because we're trying to push the limits and so we discover the limits

together now we do the same thing in system engineering and cooling systems it turns out Plumbing is really

important to us because of liquid cooling and maybe fans are really important to us because of air Cooling

and we're trying to design these fans in a way almost like you know they're aerodynamically sound so that we could

pass the highest volume of air make the least amount of noise so we have aerodynamics engine in our in our

company and so even though even though we don't make them uh we design them and we have to ex deep expertise of knowing

how to have them made and and um and and from that we we uh we try to push the limits one of the themes of this

conversation is that you are a person who makes big bets on the future and time

and time again you've been right about those bets we've talked about gpus we've talked about Cuda we've talked about

bets you've made in AI self-driving cars and we're going to be right on Robotics and this is my question we're going to

be righte the latest bet of course we just described at the CES and I'm very very proud of it and and I'm very

excited about it is the fusion of Omniverse and Cosmos so that we have this new type of generative World

Generation system this Multiverse generation system I I think that's going to be profoundly important in the future

of of uh Robotics and physical systems of course the work that we're doing with human robots developing the

tooling systems and the training systems and um the human demonstration systems and all all of this stuff that that

you've already mentioned we're we're just seeing the beginnings of of that work and uh I

think the next 5 years are going to be very interesting in the world of human robotics of course the work that we're

doing in um digital biology so that we can understand the language of molecules and understand the language of cells and

just as we understand the language of physics and the physical world we' like to understand the language of the human

body and understand the language of biology and so if we can learn that uh and we can predict it then all of a

sudden uh our ability to have a digital twin of the human is plausible and so I'm very excited about that work I love

the work that we're doing in uh climate science and be able to from weather predictions understand and predict the

high resolution Regional climates the weather patterns uh within a kilometer above your head um that we can somehow

predict that with great accuracy uh its implications is really quite profound uh and so the number of things that we're

working on is is really cool you know we we're we're fortunate that uh We've created this this

instrument that that is a a time machine and we need time machines in all of these areas that we just talked about

so that so that we can see the future and if we could see the future and we can predict the future then we have a

better chance of making that future the best version of it and and that's the reason why scientists want to predict

the future that's the reason why that's the reason why we try to predict the future and everything that we try to

design so that we we can um optimize for the best version so if someone is watching this and maybe they came into

this video knowing that Nvidia is an incredibly important company but not fully understanding why or how it might

affect their life and they're now hopefully better understanding a big shift that we've gone through over the

last few decades in Computing this very exciting very sort of strange moment that we're in right now where we're sort

of on the precipice of so many different things if they would like to be able to look into the future a little bit how

would you advise them to prepare or to think about this moment that they're in personally with respect to how these

tools are actually going to affect them well there are several ways to reason about about the future that we're

creating um uh one way to reason about it is suppose the work that you do uh

continues to be important but but the effort by which you do it um went from you know being a week long

to almost instantaneous you know that that the effort of drudgery uh basically goes to zero what is the implication of

that uh this is this is very similar to what would change if all of a sudden we had highways in this

country and that kind of happened you know in the last Industrial Revolution all of a sudden we have interstate

highways and when you have interstate highways what happens well you know suburbs start to be created and and all

of a sudden um you know distribution of goods from east to west is is is no longer a concern and um

all of a sudden gas stations start cropping up on highways and uh and uh um fast food restaurants show up and you

know someone some motels show up because people you know traveling across the state across the country and just wanted

to stay somewhere for a few hours or overnight and and so all of a sudden new economies and new capabilities new

economies um what would happen if if a video conference made it possible for us to see each other uh without having to

travel anymore that all of a sudden it's actually okay to work far further away from home and uh from from uh from work

work and live further away uh and so so you ask yourself kind of these these questions you know what would happen if

if um uh I have a software programmer with me all the time and whatever it is I I Can Dream up the software programmer

could write for me you know what what would that do uh what would happen if if um uh if I just had a seed of an idea

and and I rough it out and all of sudden a you know a prototype of a a production was put in front of me and what how

would that change my life and how would that change my opportunity and um you know what what is it free me to be able

to do and and so on so forth and so so I think that the next the next decade intelligence um not for

everything uh but for for some things would basically become superhuman and and so so um uh

but I can tell you exactly what that feels like I'm surrounded by superhuman people super intelligence

from my perspective because they're the best in the world at what they do and they do what they do way better than I

can do it and and I'm surrounded by thousands of them and yet what it it never one day um

caused me to to think all of a son I'm no longer necessary it actually empowers me and gives me the confidence to go

tackle more and more ambitious things and so so suppose suppose now everybody is surrounded by these super

AIS that are very good at specific things or good at some of the things what would that make you feel well it's

going to empower you it's going to make you feel confident uh and and I I'm pretty sure you probably use cha GPT and

Ai and um I feel I feel more empowered today more confident to learn something today the knowledge of almost any

particular field the barriers to that understanding it has been reduced and I have a personal tutor with me all of the

time and and so I I think that that feeling should be Universal and and I if if there's one thing that I would

encourage everybody to do is to go get yourself an AI tutor right away and that AI tutor could of course just teach your

things uh anything you like um uh help you program uh help you write uh help you analyze help you think help you

reason uh you know all of those things uh is going to really make you feel empowered and and um I think that that's

that's going to be our future we're going to become we're going to become super humans not because we have super

we're going to become super humans because we have super AIS could you tell us a little bit about

each of these objects this is a new GeForce graphics card and yes and uh this is the RTX 50 Series it

is essentially a supercomputer that you put into your PC and we use it for gaming of course people today use it for

design and creative arts and it does amazing Ai and the the real breakthrough here and this is this is truly an

amazing amazing thing GeForce enable abled Ai and it enabled Jeff Hinton Alias es and Alex kushy to be able to

train alexnet we discovered Ai and we Advanced AI then AI came back to GeForce to help computer graphics and so here's

the amazing thing out of 8 million pixels or so in a 4K display we are Computing we're processing only 500,000

of them the rest of them we use AI to predict the AI guessed it and yet the image is perfect we inform it by the

500,000 pixels that we computed And We R traced every single one and it's all beautiful it's perfect and then we tell

the AI if these are the 500,000 perfect pixels in this screen what are the other 8 million and it goes it fills in the

rest of the screen and it's perfect and if you only have to do fewer pixels are you able to invest more in doing that

because you have fewer to do so then the quality is better so the extrapolation that the AI does exactly because

whatever Computing whatever attention you have whatever resources you have you can place it into 500,000 pixels now

this is a perfect example of why AI is going to make us all superhuman because all of the other things that it can do

it'll do for us allows us to take our time and energy and focus it on the really really valuable things that we do

and so we'll take our own resource which is you know energy intensive attention intensive and well dedicated to the few

100,000 pixels and use AI to Super reses it upres it you know to everything else and so this this graphics card is now

powered mostly by Ai and uh the computer Graphics technology inside is incredible as well and then this next one uh as I

mentioned earlier in 2016 I built the first one for uh AI researchers and we delivered the first one to open Ai and

Elon was there to receive it and this this version um uh I built a mini mini version and the reason for that is

because AI has now gone from AI researchers to every engineer every student every AI scientist and and AI is

going to be everywhere and so instead of these $250,000 versions we're going to make these $3,000 versions and schools

can have them you know students can have them and you set it next to your PC or Mac and um all of a sudden you have your

own AI supercomputer and you could you develop and build AIS build your own AI build

your own R2-D2 what do you feel like is important for this audience to know that I haven't asked one of the most

important things I would advise is for example if I were a student today the first thing I would do is to learn AI

how do I learn to interact with check GPT how do I learn to interact with Gemini Pro and how do I learn to

interact with Gro and and um I these learning how to interact with with AI is not unlike being someone who is really

good at asking questions you're incredibly good at asking questions and and prompting AI is very very similar

you you you you can't just randomly ask a bunch of questions and and so asking asking an AI to be assistant to you

requires requires some expertise in in Artistry and how to prompt it and so if I were if I were a student today

irrespective whether it's for uh for math or for science or chemistry or biology or doesn't matter what field of

science I'm going to go into or what profession I am I'm going to ask myself how can I use AI to do my job better if

I want to be a a lawyer how can I use AI to be a better lawyer if I want to be a better do doctor how can I use AI to be

a better doctor if I want to be a chemist how do I use AI to be a better chemist if I want to be a biologist I

how do I use AI to be a better biologist that question should be persistent across everybody and just as My

Generation Um grew up as the first generation that has to ask ourselves how can we use computers to do our jobs

better yeah the generation before us had no computers my generation was the first generation that had to ask the question

how do I use computers to do my job better remember I came into the industry before Windows

95 right 1984 there were no computers in offices and after that shortly after that computers started to emerge and so

we had to ask ourselves how do we use computers to do our jobs better the Next Generation doesn't have to ask that

question but it has to ask obviously next question how can I use AI to do my job better that is start and finish I

think for everybody it's a really exciting and scary and therefore worthwhile question I think for everyone

I think it's it's going to be incredibly fun AI is obviously a word that people are just learning now but it's just you

know what would it is it's made your computer so much more accessible it is easier to prompt chat GPT to ask it

anything you like than to go do the research yourself and so we've lowered a barrier of understanding we've lowered a

barrier of knowledge we've lowered a barrier of intelligence and and um uh everybody really had to just

go try it you know the thing that's really really crazy is if I put a computer in front of somebody and

they've never used a computer there is no chance they're going to learn that computer in a day there's just no chance

somebody really has to show it to you and uh yet with chat GPT if you don't know how to use it all you have to do is

type in I don't know how to use Chad GPT tell me and it would come back and give you some examples and so that's the

amazing thing you know this the the amazing thing about intelligence is uh it'll help you along the way and make

you uh superum you know along the way all right I have one more question if you have a second this is not something

that I plan to ask you but on the way here uh I'm a little bit afraid of planes

which is not my most reasonable quality and the flight here was a little bit bumpy mhm very bumpy and I'm sitting

there and it's moving and I'm thinking about what they're going to say at my funeral

and after she asked good questions that's that's that's what the tombstone's going to say I hope so yeah

and after I loved my husband and my friends and my family the thing that I hoped that they would talk about was

optimism I hope that they would recognize what I'm trying to do here and I'm very curious for you youve you've

been doing this a long time it feels like there's so much that you've described in this Vision

ahead what would the theme be that you would want people to say about what you're trying to

do very simply they made an extraordinary impact and I think that we're fortunate

because of some core beliefs a long time ago and sticking with those core beliefs and um uh building upon them uh we we

found ourselves today uh being one of the most one of the many most important uh and

consequential uh technology companies uh in the world and potentially ever and and and so we we take we take that

responsibility very seriously um we work hard to make sure that the capabilities that we've created are

available to uh large companies as well as individual researchers and developers uh across every field of

science no matter profitable or not uh big or small famous or otherwise um and and it's because of

this this understanding of the consequential work that we're doing and the potential impact it has on so many

people uh that we want to make make this capability uh as as uh as pervasively as possible and um I do think that that

when we look back in a few years uh and I do hope that what what uh the Next Generation

realized uh is as they they well first of all they're going to know us because of all the you know gaming technology we

create I I do think that we'll look back and the whole field of digital biology and

Life Sciences has been transformed our whole understanding of of um Material Sciences uh has complet been

revolutionized uh that robots are helping us do dangerous and mundane things all over the place uh that if we

wanted to drive we can drive but otherwise you know take a nap or um enjoy your car like it's a home theater

of yours uh you know read from work to home and and at that point you're hoping that you're you live far away and so you

could be in a car for longer and you know and and and you look back and you realize that there's this company almost

at the epicenter of all of that and and uh happens to be the company that you grew up playing games with and I I hope

I hope that that to be uh what what the Next Generation learn thank you so much for your time I enjoyed it thank you I'm

glad