Understanding Reality: The Brain's Role in Perception

[Music] the most complex thing we've discovered in the

universe is the human brain for the past 20 years i've been trying

to understand how what happens in three pounds of jell-o like material

somehow becomes us what we feel what matters to us our beliefs and our hopes everything we

are happens in here and for me there's one mystery that's

absolutely fundamental what is reality what if i told you that this world

around us this richly textured world we're all just an illusion constructed in your head

what if i said that the real world has no smell or taste no

sound what if i said there's no color if you could perceive reality as it

really is out there you wouldn't recognize it at all i want to show you how the brain takes

in information sifts through it to find patterns and uses it to build the multi-sensory

technicolor show that is your reality

[Music] when i'm in the world my senses are flooded with sights and sounds and

smells and it seems obvious that reality is just out there there's a person there's a cab

all i have to do is show up and my senses let me experience it all but there's a twist to this story

[Music] let me show you something so take a look at this middle square

here does that look more similar to the light square or the dark well it looks like a light square yeah

you might be surprised if i move it now it looks like a dark square oh my god it is the same oh

it's surprising right it is oh my goodness wow you're red seriously

do you have a guess as to why there's an illusion here well it seems like there's a shadow so it makes this darker

that's exactly right your brain is trying to understand the colors of things

irrespective of the lighting in the shadows so somehow it's not about what's hitting

your eyes it's about your brain's that's really trippy you just mess up my whole day

now this is about more than just a visual illusion it's about a fact that's central to our

lives our perception of reality has less to do with what's happening out there

and more to do with what's happening in here to understand what's going on we first

need to know how information from the world around us gets into the brain

it feels as if sights and sounds just stream in through our eyes and our ears but imagine if you could climb inside

a human skull when you step into the skull you'll find there's no way for light or

sound or smells to get directly in here this is a sealed chamber

so the brain sits in darkness and in silence it's in total isolation

your brain's never seen the outside world but somehow you experience it now this might seem

straightforward because we have portals to the outside world like your eyes and ears

but these aren't just piping in sights and sounds instead photons of light or air

compression waves these are getting converted into the common currency of the brain

electrochemical signals these signals travel through dense networks of brain cells called

neurons [Music] there are 100 billion neurons in the

human brain and in every second of your life each one of these

is sending tens or hundreds of electrical pulses to thousands of other neurons

and somehow all of this activity produces your sense of reality so whether it's the bark of a

dog or the smell of coffee or a view of a beautiful sunset it's all made of the same stuff in here

and this is the stuff of reality but how does the brain turn it into something meaningful

well it does it by sifting through the non-stop stream of incoming data to find patterns which it then assembles

into a reality it's an operation which is the product of millions of years of evolution

so efficient so powerful that its work seems effortless and instantaneous

take as an example sight the of seeing feels so natural that it's hard to appreciate the vast

sophisticated machinery running under the hood for us to see clearly many different

systems need to be operating in concert it's about more than just the eyes [Music]

the best way to understand this is to look at the extraordinary case of a man who lost his sight

and then was given the chance to get it back i lost my sight when i was three and a

half years old as the result of a chemical explosion and oddly it doesn't seem like it was a

big deal i guess as a three and a half year old my world

according to vision was not as well established as it would be for somebody who lost their vision later in life

after over 40 years of blindness mike may had pioneering stem cell treatment that

would repair the physical damage that the explosion caused to his eyes cameras were there to witness the moment

when for the first time the bandages came off dr goodman does the cornea transplant

[Music] and he peels back the bandages he gets them all the way off and there's

this whoosh of light and bombarding of images onto my eye holy smoke

in surgical terms the operation was a total success what's across the room over here but to

mike it wasn't there was something wrong all of a sudden you turn on this flood

of visual information it's overwhelming my brain is just going oh my gosh

so that's how the world proceeded one image at a time seeing cars as they whizzed by

and then i would see a sign ahead of us it looked like we're gonna smack right into it

and in fact it's a sign over the freeway and we're not gonna run into it we're going under him

and that was only the first hour it was going to get worse when mike got home

if you put four blonde boys together all roughly the same height and i looked at them i couldn't tell you

which two were mine don't go away i'm not finished looking at you

mike's new eyes were functioning perfectly and they were sending signals to the brain just like yours or

mine do but he couldn't see his sons in any meaningful way

i had no face recognition whatsoever none when he'd been totally blind mike was a

paralympic skier but his first sighted attempt at skiing was a complete failure when i

skied for the first time because of my depth perception difficulty i had no time to figure out

the difference between four dark things on the white snow a person a tree

shadow or a hole ten years on mike still needs his guide dog to get around

he can detect light and motion and identify colors but he struggles to gauge how far away

things are he still can't read the expressions on his son's faces

he still can't read words on a page what mike's story gives us is a glimpse of all the elements that have to be in

place for the brain to construct a visual reality

many regions of the brain are involved in vision they specialize in different aspects

such as motion edges colors face recognition somehow the brain weaves all of this

together unifies it to form what we experience as an image in mike's case decades of blindness

caused these regions of his brain to be taken over for other tasks like hearing and touch they just weren't

available for him to use even when he was given a pair of new [Music]

eyes we often get our best view of how the brain operates

when that operation is disrupted [Music] hey brian hey that's why neuroscientists

sometimes disrupt things deliberately [Music] brian is part of an experiment being

conducted by elissa brewer at the university of california volunteers wear these goggles for weeks

at a time their brains are forced to cope with a new view of the world that's

dramatically altered what these have inside are two prisms that take the whole visual world and

flip it so whatever you see normally on the left side of the world will now be on the

right side of the world so as you move for the world you're going to have a problem figuring out

where things are as you see them in one side but reach for them in the opposite side

what the world looks like is this but what i'm seeing is this it's a straightforward change

but it's also a massive mind mash the visual data streaming in through my eyes no longer makes any intuitive sense

and i'm struggling [Music] so yeah because the world is left right

flipped um i i know cognitively i'm supposed to reach out to the other direction but of

course i've had a lifetime of training telling me to reach out in in a particular direction so i feel like

this is going to take a little getting used to can you see my hand in your visual field

yep so it looks like if i reach out this way and this way

even though i'm consciously trying to get it right over here okay i can't help but respond in a

certain way and over here there you go very good welcome to the

prison world yes of course this is all new to me but brian's been wearing his goggles for

a week so how well has his brain adapted it's very difficult to figure out which

way to go and so his motor system and feeling of touch is sending him one direction while his

visual system sending him the other direction [Music]

brian's doing well me i have to consciously reconstruct my

reality this morning my brain could rely on automated interactions

but now it can't interestingly i've broken out in a sweat and i'm hot and i'm

super dizzy and nauseated oh you know i gotta take a break i'm so sorry i gotta take this off for a second

is that okay boy that is really nauseating we're going to go down to the maze down

here and see how you guys do in navigating your way through a spatial map that's there you're going

to start out going forward this way okay i'm just trying to get my head started

so how do i get as good as brian well it happens intuitively [Music]

just look at my hands i cross reference what i see with what i can touch in fact all my senses come into play

this is what brian's been doing for the last seven days [Music]

and the result is that his brain is now starting to decode that new visual input automatically

[Music] brian's not simply getting better at making conscious adjustments

his whole reality is changing [Music] now if you take those subjects and put

goggles on them for two weeks we find that it takes about a week to start behaving normally

they start being able to figure out how to interact with the world or constructing a new reality around them a

new way of dealing with these incoming perceptions and they say that initially they can

tell there's a new left and an old left and a new right and an old right by about a weekend they even lose the

concept of which right and left where the old ones are the new ones so it's like the whole spatial map of

the world is altering and by two weeks in they will write well read without a problem

do all of our walking tasks and reaching tasks and then when they remove the goggles it

actually takes about a day to go back to normal behavior what this exposes for me is how much

effort the brain goes through to construct our world because normally you're walking through the world and it

feels like there's reality out there but in fact there's so much work happening behind the scenes

to allow that reality to happen seeing requires an intensive training program

but new recruits come on board every day we call them babies when babies reach out to touch what's in

front of them they're not just learning what an object feels like they're learning how to see

they're establishing pathways in the brain that'll be used for the rest of their lives

because vision is a whole body experience the data coming in from our eyes only

means something if we can cross-reference it [Music]

if from birth you weren't able to interact with the world if you couldn't work out through

feedback what the sensory information meant in theory you'd never be able to see

[Music] this cross-referencing doesn't stop when we're fully grown

it continues throughout our lives [Music] what we touch influences how we

see [Music] taste is affected by our sense of smell

our sight informs how we hear our senses depend on each other and our reality

is built by comparing these streams of data when they're woven together we get our

perception of this moment [Music]

it's an astonishing feat to pull off but there's one factor which really adds complication

timing all those streams of sensory data are processed by the brain at different

speeds for our reality to be constructed they have to be synchronized what do i mean by this well the easiest

way for me to show you is right here at a racetrack that's it

when there's a loud sound it feels as though you react to it instantly but you don't

[Music] watching sprinters in slow motion we can see

that there's a gap between the gun going off and their start

they may train to make this gap as small as possible but their biology imposes limits

processing that sound then sending out signals to the muscles to move will take around two tenths of a second

and that time really can't be improved on in a sport where thousandths of a second

can be the difference between winning and losing it seems surprisingly slow

so why do we use a pistol to start sprinters everyone knows that light travels faster than sound

so why not use a light we set up a test to show you in the top screen we're triggered by a

light in the bottom screen we're triggered by the gun

you can see that when our start is triggered by a flash of light we respond more slowly

[Music] it takes 40 milliseconds longer to process

why because the visual system is more complex it's bigger it involves almost a third

of the brain so while all of the electrical signals inside the brain

travel at the same speed the ones related to sight go through more complex processing and that takes time

and this isn't just about hearing and seeing every type of sensory information takes

a different amount of time to process you'll react slower to a touch on the foot

than one on the hand the astonishing thing is that our brains hide all this

when i clap my hands everything seems synchronized why well your brain is pulling off fancy

editing tricks what it takes to be reality is actually a delayed

version it collects up all the information from the senses before it decides on a story of what

happened and that means you live in the past by the time you think the moment now

occurs it's already long gone to conjure a reality from all that sensory

information your brain needs around half a second that's the unbridgeable gap between an

event occurring and your conscious experience of it in that half a second a lot of things

need to happen sometimes it's easy to assume that there's a single spot in the brain that

takes care of this or that function like an area for memory or generosity or empathy

but in fact the vast networks of the brain are so much more complex than that think of the brain like a city

if you were to look out over a city and ask where is the economy located you'd see that there's

no single answer to that instead the economy emerges as an interaction of all the elements

and so it is with reality the raw materials of perception are gathered by our sensory receptors

turned into electrical signals and transported around our brains along superhighways of neurons

processed they become our reality some parts of grain city specialize in vision

other districts care about hearing some about touch and so on

and even within a sense like vision you have streets that specialize in colors or

edges or motion but just like in a city no neighborhood operates in isolation

instead the life of a city depends on the interaction between residents at all different scales and

somehow out of all of this interaction emerges your

personal reality [Music] reality is the brain's ultimate

construction it's based on all the streams of data from our senses

but it's not dependent on them how do we know because when you take it all away

reality doesn't stop it just gets stranger this is alcatraz

a jail built on the principle of isolation between its inmates and the rest of

society stood not only stone walls but the cold dangerous waters of the san francisco bay

prisoners were completely and deliberately cut off and there was one place inside the

prison where that seclusion went even further this is the hole prisoners who were sent

here were completely isolated from the outside world they had no interactions with people

there was no sound and there was no light robert luke was sent to alcatraz in 1954

for armed robbery he was known by the nickname cold blue luke

[Music] everybody knew about the dark hole the dark hole is a bad place

some guys couldn't take that i mean they were in there and within a couple of days they were

banging her head on the wall as punishment for smashing up his cell he was sent to the hole

for 29 straight days [Music] you didn't know how you would act when

you got in there didn't want to find out when they closed that door

there's just nothing there it's pitch black but it didn't stay that way for long

[Music] starved of input luke's brain started to produce its own reality

i remember i go on these trips one i used to remember was flying a kite

but it got pretty real [Music] they were all in my head

what luke felt was something that's also been reported by other prisoners kept in the same conditions

[Music] deprived of new sensory information they said they went beyond dreaming or

daydreaming they didn't just imagine pictures they saw

this testimony goes to the heart of the relationship between the outside world the brain and what we call reality

to understand it we need to look more deeply into the visual system

this is the thalamus one of the brain's major junctions most sensory information connects

through here on its way to the outer surface of the brain the cortex

so data collected from the eyes stops here before going to the visual cortex now you'd expect a

heavy flow of information from the thalamus to the visual cortex and there is

but there's six times as much traffic flowing in the opposite direction and that dwarfs the amount

coming in from the eyes and that suggests that in any one moment what we experience

as seeing relies less on the light streaming into our eyes and more on what's already inside

our heads even when brains are unanchored from external data they continue to generate

their own imagery in other words remove the world and the show still goes on

we all have this internally generated reality incredible as it may sound this world

lives inside your brain it's constantly updated by information from our senses

but moment to moment what we experience isn't what's really out there instead it's a beautifully

rendered simulation this is a surprising way to understand how you see the world

it's called the internal model and it's vital to our ability to function

as i walk down this city street i seem to automatically know what things are without having to work out

the details for example i don't have to work out the detail of what this rectangular metallic

thing is or this giant green fluffy thing behind me or

this huge object with reflective pains on it or this thing with four appendages

my brain makes assumptions about what i'm seeing based on my internal model and that's been built up from

years of experience of walking city streets just like this one instead of using my senses to rebuild my

reality from scratch every moment i'm comparing sensory information with a model that i've already constructed

[Music] updating it refining it correcting it our brains are so good at

doing this that we're normally unaware of it but sometimes under certain conditions we can see the

process at work look at this hollow mask of einstein's face

your brain tells you it's coming out at you and even when you know it's an illusion

you can't help but fall for it what you're seeing is the internal model

not the raw information that's coming in from your eyes your internal model is built on a

lifetime of experience with faces that stick out when you're confronted with one that's

hollow your model simply sees what it expects to see

the visual cortex sends its internal expectations to the thalamus and the thalamus compares those to

what's coming in through the eyes the difference between the two

is what the thalamus sends back so the cortex can update its model

thanks to the internal model the world out there remains stable even when i'm moving

let me show you what i mean so imagine that i really love this scene behind me and i want to go ahead and

capture it so i can view it later so i'm going to go ahead and videotape the scene

and i'm checking out all the buildings okay and now i'm going to play this back not surprisingly the resulting video is

nauseating so why does this video look so terrible given that when i look at the buildings

my eyes are making the same jerky movements although you're not generally aware of

it your eyes move about four times a second but your internal model operates under the

assumption that the world outside is stable so my eyes aren't taking a video they're

simply gathering bits of data to update the city that's already inside my head

having an internal model helps me make sense of my environment and that's its primary function to

navigate the world the brain doesn't bother picking up every detail

just enough to get us through but it plays the trick of making us feel as though we've seen

it all as another famous experiment shows in the 1960s the russian psychologist

paul yarbus used this painting called the unexpected visitor

in an experiment he devised a way to track the eye movements of volunteers who were seeing it for the first time

hi jennifer hello i'm going to ask you to put these glasses on we're gonna re-run what he did

my volunteers have a few seconds to take in the image tina look at this painting i want you to

gather what's going on in the scene we can watch in real time exactly where each person's eyes go

tell me what you think is going on in this painting i think the man in the brown is the unexpected visitor

one brief look is enough for the brain to model the picture but just how detailed is that model

how many children are there are two okay so go ahead and look back at the painting and ask that question again

oh quite different how many children are there i consider it three everyone who'd seen

the painting thought they knew what was in it but my specific

questions highlighted blanks that the brain had never filled in because the details weren't needed

how many paintings are on the wall in their house maybe two or three okay go ahead and look back at the

painting and answer that question there's a million yeah a map and then another and then

there's seven on the other wall and then one small one in the map okay

there's a ton this is not a failure of the brain it doesn't try to produce a perfect

simulation of the world the internal model is a hastily drawn approximation

and more details are added on a need to know basis when you looked at the painting

the first time you saw sort of a rough draft of what was going on

and when i asked you specific questions you had to go and answer those by looking by turning your attention onto

specific parts of the painting and only then did you actually see it so placing your eyes on an

object is no guarantee of seeing it but there's something else we're unaware of happening

every time we look at any picture or person or thing anytime we look at all

we might think of color as a fundamental defining quality of the world around us after all it's

everywhere but here's the startling thing in the outside world

color doesn't actually exist when electromagnetic radiation hits an object

some of it bounces off and is captured by our eyes we can distinguish between millions of

combinations of wavelengths but it's only inside our heads that any of this

becomes color add to that the fact that the wavelengths we can detect

are only a small part of what's out there you experience reality as it's presented

by your senses and it doesn't typically strike you that things could be very different

what we've been talking about so far is what we call the visible spectrum of light

which is a spectrum of wavelengths that runs from what we call red to violet

but it turns out that this only constitutes a tiny fraction of the electromagnetic

spectrum in fact less than 1 10 trillionth of it so all the rest of the spectrum

including radio waves and microwaves and x-rays and gamma rays all of this stuff is flowing through our bodies right now

and we're completely unaware of it because we don't have any specialized biological receptors to pick up on it

so what this means is that the part of reality that we can see is totally limited by our biology

and this isn't just about sight all our senses are only picking up a small part of the

information that's out there so for a dog he's tuned into a whole world of scent molecules that i'm not

his experience of smell is as rich as my experience of vision in the blind and deaf world of the tick

the important signals are temperature and body odor for cave dwelling bats it's all about

air compression waves that allow them to echolocate [Music]

but no one's having an experience of objective reality of the world that really truly exists

instead each creature perceives only what it has evolved to perceive

and this isn't just about variation between species if we're each experiencing a personal

reality constructed inside our brains how do i know that my reality is at all like

yours most of the time it seems as if we operate along the same lines as if you and i agree what a blue sky is

as if the sound of a dog bark provokes the same sort of response in both of us [Music]

but there's a small group of people whose perception is measurably different from ours

for me anytime i see a letter or a number or think of a word or say someone's name

there is a lot of color associated with that hannah is one of 6 000 people i've

studied who have synesthesia i study synesthesia because it's one of

the few conditions in which it's clear that someone else's reality is different from mine

and it makes it obvious that how we perceive the world is not one size fits all

in my mind i associate each letter with its own color so for example the letter a is always

red b is always blue c is always orange every time

so they never change but what's interesting is when they're formed into words in different

orders the configuration of the colors changes and that can be sort of interesting

so in the word hannah my name it looks like a sunset it's yellow fading into red fading into

kind of a clear like clouds almost and then goes back to red into yellow

these experiences come about because of the simple fact that inside the brain all

sensory information is made from the same stuff electrochemical signals

is the result of crosstalk between sensory areas of the brain think of the blurred borders between

city districts synesthesia shows us that even minute changes in brain wiring

can lead to different realities there are different kinds of synesthesia some people perceive weekdays to have

locations in space some taste words others see music and every time i meet someone who

has this kind of experience it's a reminder that from person to person

brain to brain our experiences of reality can be quite different for a small section of the

population that difference can be extreme and terrifying

we all know what it's like to have dreams at night to have bizarre unbidden thoughts that take us on

journeys sometimes journeys that we suffer through but when we wake up we're lucky

enough to be able to compartmentalize that to say okay that was a dream and this is my

waking life [Music] but just imagine what it would be like

if these were more and more intertwined and it was more more difficult to tell them apart from one another

i felt like the houses were communicating with me you are special you are especially bad

repent stop go you know kind of i did not hear these as words but i heard

them as thoughts put in my head but i knew they were the house's thoughts and not my thoughts

i think that explosions are being set off in my brain and i'm afraid that it's going to hurt other people

not just me i once had a fantasy that uh my brains were going to leak out of my ears and drown people

what is that you know but ellen sachs is a professor of law at the university of southern california

she's been experiencing schizophrenic episodes since she was 16 years old

it's scary it's unpredictable it's sort of interesting because there are different theories about psychotic

symptoms for some people they're just random firings of neurons i do think they tell the truth about

your psychic reality so when i say i've killed hundreds of thousands of people with my thoughts

that's just an archaic and extreme way of saying i feel like i'm a bad person schizophrenia is still not fully

understood but it involves chemical imbalances in the brain

which cause problems in the sending and receiving of signals thanks to medication and therapy ellen

has been able to lecture and teach for over 25 years so when you were at the bottom in one of

your worst psychotic episodes you took that to be reality i really believe what's hap i think is happening

is happening and it's terrifying it's like a waking nightmare confusion bizarre images violence terrors

i wouldn't wish it on anyone that said um everybody's reality is constructed right

you filter it through your beliefs and values

and issues and this is true for people who have mental illness and for people who don't have mental illness it's all a

spectrum [Music] reality differs from person to person

and more than that it changes from moment to moment there are times in all our lives when it

can seem enhanced intensified even the one great constant which we all

think we share and which should never change somehow becomes

stretched and distorted i'm talking about time time is something that we rarely stop to

consider but our brain's experience of time is often quite strange

it doesn't always seem in certain situations that time is running at an even pace

sometimes it runs more slowly or more quickly when i was eight years old i fell off of

the roof of the house about this height and the fall seemed to me to take a very long time

but when i got to high school i learned physics and i calculated how long did the fall actually take and

it turns out it was only eight tenths of a second so that set me off on a quest to understand

why did it seem to take so long and what did this tell me about our perception of reality

many people have reported the sensation during moments of terror professional wingsuit flyer jeb

corliss experienced it in an extreme way and because he falls for a living the

event he describes was captured on multiple cameras on this day i decided to aim for a

target like set up balloons and come in and hit balloons and i was flying towards the balloons

and as i was coming in to hit the black balloon i misjudged

i impacted flat solid granite at 120 miles an hour six seconds elapsed between the moment

that jeb hit the rock and the moment he pulled his ripcord he broke his leg and both ankles in the

[Music] fall jeb's perspective those six seconds

seem to last a long time you've got two options one is you can not pull and you know just be

dead right now it's like really quick semi painless you know over fast or

you can pull you know get a parachute over your head impact a second time and then bleed to

death while you're waiting for rescue these two separate thought processes felt like minutes of time

it feels like you're operating so fast that your perception of everything else seems to slow down

and everything just gets stretched but what was really happening in jeb's brain

i designed an experiment to find out it depended on inducing extreme fear in people

by dropping them from 150 feet in the air they fell with a digital display

strapped to their wrist its numbers were changing at a rate faster than human vision can normally

handle if perceptual time did slow then they would be able to read the numbers

but no one could so why did jeb recall his accident as happening

in slow motion it was a time distortion on a level i've never experienced before

i learned later that the rescue took about two and a half hours but at the time it felt like weeks i

mean it didn't feel like minutes or hours or even days it felt like it felt like little

eternities it felt like forever the answer seems to lie with how our memories are made

in a critical situation an area of the brain called the amygdala kicks into high gear it commandeers the

resources of the rest of the brain forcing everything to attend to the situation at hand

when the amygdala is in play memories are laid down with far more detail than under normal circumstances

[Music] these memories are richer and more vivid if you're ever in a similar situation

you have more information at your disposal to work out how to stay alive but there's a

fascinating consequence when the events are replayed in your memory they appear to have taken a longer time

jeb's time distortion is something that happened in retrospect

[Music] a trick of the memory that wrote the story of his reality

the brain is the universe's ultimate storyteller we believe whatever our brains serve up

to us the reality we take for granted requires intensive training

to interpret the world it takes time to process sensory information so we live in the past

[Music] and because all that information is ultimately just electrochemical signals

to be sorted matched rendered and packaged reality is something created inside

our head our brain sculpts our reality using the narrow

trickle of data that it can gather through the senses and from that trickle it tells a story

about our world it's possible that every brain tells a different narrative

with seven billion human brains wandering the planet trillions of animal brains

no one is tapped into the full picture [Applause] each brain carries its own unique model

of the world around us that is what we experience [Music]

we have no choice [Music] so what is reality it's whatever your

brain tells you it is next time on the brain i'm going to explore a fundamental

question about our lives what makes you you [Music]

i've spent many years of my life trying to decipher the mysteries of the brain and yet i'm still in awe every time i

hold one and that's because although this marvel of biology seems so

alien to us somehow it is us this three pound organ is made up of hundreds of billions of

cells with a quadrillion connections between them these cells fire trillions of

electrochemical signals every second of your life somehow all this wet biological stuff

results in the experience of being you [Music] what shapes who you become

i'm going to explore how your life shapes your brain and how your brain shapes your life

so you