Fundamentals of Scientific Method and Experimental Design in Cognitive Psychology

Hello and welcome to the course basics of experimental design for cognitive psychology. I am Arwarma from the

department of cognitive science at IT Kpur. uh in this week we are basically having a brief historical background of

the scientific method and then we'll eventually go on to the nitty-g gritties of experimental method in cognitive

psychology. Now in the last lecture we talked about a little bit about the logical positivism movement as one of

the fundamental movements that uh wonders about the generation of scientific knowledge. uh in this and we

stopped that lecture at some place around when we were talking about the principles of uh you know scientific uh

principles of generational scientific knowledge as proposed by the logical positivists. One of the interesting

aspects uh that we discovered therein was this idea of uh verification, observation and induction. So uh uh

observation, induction and verification as the three pillars of scientific knowledge. Now uh we looked at that and

we uh concluded with this idea that verification because it is uh physically tenable. It is logically impossible to

verify all the knowledge that we have generated. Uh verification probably cannot be the fundamental uh basis of uh

you know uh scientific knowledge in in that sense. So we'll move on. we'll look at other ways in which scientific

knowledge can be generated and interpreted and so on. Now an alternative to the uh shortcomings of

the positivist stance which basically healed uh observation induction and verification was observed by this

gentleman called Carl Puper. uh Carl Pauper uh uh basically uh you know published one of his most important

works uh logic therefores uh the logic of scientific discovery in 1934 and he provided us some of the most seinal and

the most uh you know fundamental ideas about the generation of scientific knowledge which probably guide the

scientific discipline even now. Now one of the central ideas that uh you know was proposed by Popper was this

idea that falsification instead of verification will be the more appropriate method for generation of

scientific inquiry for generation of scientific knowledge. Popper emphasized the importance of having theories uh you

know of scientific thinking uh because theories allow this weaving of you know observable and verifiable facts into a

framework and it also allow you know a means to question the presented explanation of these facts. Remember we

were talking about the fact of uh you know the fact that both inductive and deductive reasoning are based on some

observations. Now uh unless you have a theory unless you have a framework to interpret that observation within uh

these all all of these observations will seem as isolated disconnected unorganized facts. If you have only

these uh it is difficult to build knowledge. It is these are all isolated observations that will stand by

themselves and we will have no means to connect them. we will have no means. We will be no you know no closer to

understanding any given phenomena because we don't have a theory of how that particular phenomena functions or

how that particular how these particular findings can be stitched together. So in that sense, PAR actually says it is very

important for scientists to have theories and these theories will allow us this framework to you know stitch

together particular observations uh accept some of them because they sort of fall within the ambit of the predictions

that a theory generates and leave out some of them which basically do not uh you know fall within this ambit. For

instance, Papa dwelt on the difference between Freudian psychoanalysis and uh physics. More specifically around that

time, both of these uh you know were dominant uh you know schools of thought and both aimed to be part of scientific

knowledge. Studies in physics typically were aimed at the suspected weaknesses of these theories at identifying the

suspected weaknesses of these theories. Say for example Newton's laws of motions or Einstein's you know laws of general

relativity things like that most of the experiments were being carried out to test and to sort of you know falsify

these theories by aiming at the weaknesses. On the other hand, what was interestingly happening in psych

psychology or broadly psychoanalysis was that people were engaged more in finding confirming evidence rather than working

at refuting uh you know these theories. So what both of these disciplines found was that while the theories in physics

got stronger and stronger because they could not be let's say falsified or some were falsified and new theories emerge

in their in their place. uh in psychoanalysis what was happening in in psychology what was happening was that

uh uh psychologists in the psychoanalytic tradition were finding more and more confirming and

corroborating evidence that supported psychoanalysis as a preferred method. It's only later uh that people you know

reviewed these findings and said that they cannot be really treated as bonafide scientific uh theories. So

Puper pointed out that this latter approach of uh targeting weaknesses and testing whether a particular uh theory

can be falsified uh is is more important. He said that this uh approach of finding corroborating evidence uh is

basically the one that also is followed by religions and sex and so on and is basically more characterized by

knowledge based on belief and faith and so on and not scientific knowledge. So popper points out that what

discriminates scientific from non-scientific theories were the fact that uh the scientific theories are

indeed falsifiable. What does what do we mean by falsifiability? Falsifiability basically indicates that the statements

can be falsified. Statements within a given theory can be falsified as they offer precise clear predictions that can

be tested and found to be true or untrue. For example, uh if I have a statement like this, say for example, if

I say that it is colder in January than in March. Now, it's very easy to falsify this statement. You basically measure

the temperatures in the month of January, measure the temperatures in the month of uh March. And if you find out

uh the temperatures in January are not colder than in March, you can falsify it. On the other hand, you can actually

find uh you know supporting evidence for this as well. But that is not found to be that is not deemed to be the best way

of uh you know establishing this as a scientific theory or a scientific fact. So proper emphasized uh you know on the

need of the that proponents of scientific theory should put their ideas to test effectively by actually trying

to find instances where these theories will not work. Okay. So if you have a theory about uh something that okay this

is how things work you have to find out and be able to specify situations A b and c where this thing will not work

within the same tradition uh popper puts forward the hypothetical deductive model. Uh what is this model? According

to this model scientific progress involves a combination of both inductive and deductive reasoning. Uh we start

from observations. For example, one follows inductive reasoning. uh making these observations and then builds up

interpretation or a theory uh for a given phenomena. So you observe uh you know uh say for example you observe that

uh people are wearing uh you know uh more uh winter clothes in January, people are wearing uh you know they they

are switching on heaters in January, this and that and so you you observe a bunch of things that basically say okay

this is probably what is happening. On the basis of these observations, you say, "Okay, it seems that January is

colder than March." Something like that. You observe less of people switching on heaters and wearing winter clothes in in

March. You make these observations. You basically come up with an interpretation that okay, it seems that uh you know,

January is much colder than March, something like that. It's probably not the best example, but let's work with

this. Now, to check the correctness of this interpretation that let's say January is colder than March, the

researcher would employ deductive reasoning to generate testable predictions. So how do I go about this?

Now we generate testable predictions that okay uh you know the average temperature in the week first week of

January or the uh first week of March will be uh so average temperature in first week of January will be lower than

that of first week of March by so many degrees something like that. Okay. Now this is something that can be verified

by experiments or by data collection. You just have to uh you know take measurements maybe at different places

maybe at the same place and so on. You have this data and this data will tell you whether whatever I am interpreting

is correct or not correct. So the conducted experiments then provide uh new observations for further theorizing

and generating newer predictions and so on. Now uh an example of this probably a

better one than I was using is is presented by Brisbane and Brazil uh which is about say for example when Luis

Pasture discovered the source or the reason of why uh you know wine, milk and beer uh you know were getting soured

over time. So using inductive reasoning pasture concluded that this might be happening due to the microorganisms

present or introduced in these liquids by the environment. So he said once these liquids say for example wine, beer

or even milk are introduced to the environment they are kept in uncovered containers the air introduces

microorganisms in these environment which causes the souring of these uh you know liquids. So he hypothesizes uh that

if his interpretation is correct that there are microorganisms in these liquids introduced by the air then he

should be able to prevent this souring by putting a filter between the liquid and the air in which these liquids are

you know and the air in which these liquids are kept. Uh you basically create a filter. The filter prevents

microorganisms from the air entering the liquid. Indeed, Pashure was able to demonstrate

that an air filter actually prevented the souring of wine and beer which basically uh you know verifies the

fact which basically tells us that yes probably the microorganisms were the actual cause of this souring. This could

not be obviously done for milk because milk already contains some of these microorganisms. So later he discovers

that milk actually needs to be boiled to kill those microorgan organisms that are already present therein and hence that

will be the way to prevent uh you know souring of milk. Popper uh stressed on the fact that

hypothesis testing therefore should be directed at falsifying a theory rather than finding confirming evidence because

if the researchers fail at several attempts at rejecting a theory uh that constitutes a much stronger evidence

rather than any number of confirming evidence. Remember we were saying in the in the previous lecture that if you have

a theory that all swans are white even if one swan that is not white basically uh you know rejects this theory. So you

can find one swan 10 swan thousand swans million swans that are all white and use that as corroborating evidence for your

theory that all swans are white. But even if if somebody is able to present one black swan it actually you know

refuts this theory completely. So instead of finding confirming evidence, uh Papa said that science would progress

better if we focus on finding that black swan. If we are not able to find that black swan after repeated attempts, then

we are we have better grounds to assume that okay all swans are white is an acceptable theory.

So Popper proposed following the same thing that scientists should generate bold theories in form of speculative uh

and tentative conjectures and then go ahead with rigorously testing them on the basis of experimentation and

observation. So you come up with a particular theory and you work with in what conditions this will not work.

Now as theories uh you know are not generated as scientific theories typically are not generated with any

guarantee of being correct. It is perfectly plausible that theories typically will turn out to be wrong

every once in a while uh even when they initially seemed to be in line with the collected evidence. So every once in a

while it pro it basically allows scope for theories to be modified, corrected or even be junked.

Now this approach highlights an important aspect about theories. For instance, the scientists are never

completely sure about the correctness of their theories. You've seen in in physical sciences for example

several well accepted theories are after after a number of years being rejected or found to be untrue. In psychology for

example uh Pia's theory of cognitive development was found to be not true by more recent researchers who said that

say for example the timeline of objective permanence is not within as within the you know ambit of what PRG

had initially suggested. In fact, the confidence of scientists in their theories is only as much is

basically directly proportional to the degree to which a particular theory survives the attempts at falsification.

So the number of tests of falsification a theory survives, the more confident a scientist can be about his or her

theory. So in a sense, what can we conclude from this? Scientific theories can actually

be proven to be false. uh but they are true only to the extent to which they have survived these attempts at

falsification. Now following from this stance uh progress in sign can actually be seen as

a process of trial and error. You come up with something you keep on trying to falsify it. You falsify it sometimes

you're not able to falsify it sometimes. So it's almost like a uh you know a process of trial and error where several

possible explanations of a phenomena may be offered but only those explanations will survive that will survive this test

of falsification. The strongest and the fittest explanations will only continue and will basically form the you know

body of scientific knowledge as opposed to explanations that are weak conjectures that are proven to be false

by the you know initial tests or initial attempts at falsification. Also, fositigation ensures that

scientists are less likely to stick to wrong opinions uh more than common people. For example, common people or

commonplace knowledge is typically governed by belief and faith and opinions which are not extremely uh well

scrutinized which are not tested all the time. And because they are not tested because we are not looking for see there

are aspects like and we are aware of things like cognitive bias maybe or confirmation bias actually uh

confirmation bias is a tendency of people to look for confirming evidence rather than look for refuting evidence

because our typical common sense reasoning is afflicted by things like confirmation bias. uh scientists

knowledge is considered stronger uh because they are uh you know establishing and building on scientific

knowledge on the basis of falsification whereas we as common people are building upon uh you know our knowledge on the

basis of confirming evidence. So science therefore is more prone to learning from its mistakes. However,

again it must be noted that not all scientific advice or all scientific knowledge can be considered uh with a

lot of confidence unless it has been tested unless it has passed the test of falsification.

Now uh is falsification a binary? is something falsified and then junked at the same time or say for example there

can be degrees of falsifi falsifiability uh you know in these scientific theories. Now an interesting caveat that

also you know popper discusses is that scientific different scientific statements may lend themselves to uh you

know different degrees of falsifiability. For example, uh at least as what papa says that the degree to

which a scientific theory or a statement exposes itself to testing and falsification that determines how

scientific that theory can be deemed to be. For instance, look at these uh two statements. Wine sour because of

microorganisms. Wine sour because of bacteria coming from the air. Now here both are very

similar statements but in the second one uh the statement is a little bit more specific. We have also specified the

type of microorganism that may be responsible for souring of the wine. Now the second statement because of being

more specific is more prone to the test of falsification and it lends itself to uh you know this test of falsification

is therefore going to be deemed to be more scientific. The first theory we are just saying microorganisms. Now if you

don't find bacteria you can say oh it were viruses. If you don't find viruses you can say oh it is another class of

microorganisms. and so on and so on. All right. So the more specific a theory is, the more testable statements it will

generate and the more testable statements is generate, it exposes itself to falsifiability and in that

process becomes more scientifically valid theory. All right. So also there is uh an aspect

about the scope of a given scientific theory. Now according to popper the more facts that uh are covered within a

scientific theory make a given theory more scientific. So any theory that we are talking about should not explain

facts or phenomena selectively it should aim to explain more and more and as many of the facts observed facts that are

explained within this particular framework. So case in point can be say for example Einstein's you know a test

of Einstein's theory of relativity by this gentleman called Edington. Now Einstein had predicted that rays of

light would bend as they passed close to a massive uh you know object. Now Edington wanted to test this. So he

decided to make use of a solar eclipse in the year 1919. uh and he basically uh sent uh you know

expeditions to two different places where the observations uh you know the chances were to get best observations.

One was sent uh one expedition was sent to the island of Principi along the west coast and one was sent to northern

Brazil. Now each expedition was asked to take pictures of the sky during the eclipse and also on a night

where the stars were at the same position relative to the earth so that the observations are comparable. Now uh

the results obtained from this were obviously was actually less clear-cut than what Edington had hoped for but

they still were able to convince nearby all the research nearly all the researchers uh involved in this uh that

the data typically agreed with the with Einstein's predictions and was significantly different from what was

present what was basically predicted by Newtonian physics. So on the basis of collecting these observations, Edington

was able to uh you know test and confirm the theories offered by Einstein relative to the theory offered by

Newton. For example, just to sort of give you a a more detailed peak, there was a

significant difference between the perceived location of stars uh during the eclipse and the perceived location

uh you know without the interfering sun. uh also the magnitude of the deviation was in line with the measurement error

that was predicted by Einstein's theory and not by Newton's theory. So Paer basically says this could be you know

touted as one of the best examples of science at work. An extremely false survival theory because it gives very

specific and precise uh you know you know predictions uh was being put to test. There were hundreds of see if you

do not if you did not have a theory in the beginning if uh these observations were being randomly made

uh then there could be hundreds of plausible reasons to expect no difference at all in the perceived

locations of star during the eclipse and uh you know uh at a time when the eclipse was not happening there was only

one reason that was predicted by Einstein's theory uh that would basically predict this deviation and

because it was found these predictions fitted very well and basically were used to confirm the uh theory of offered by

Einstein and the you know uh calculations. Now uh degree of falsification uh scope

of falsification but uh again this idea of how falsifiable is something or do we throw or do we junk the theory at the

first test of falsification it fails. Now while puffer popper offers this uh you know very sound method of the

progress of science researchers have wondered about the fact that whether the theory should be rejected immediately

and completely upon failing the first test of falsification. Researchers needed to investigate

whether the data were sound because see researchers typically are sort of you know clingy towards the theory that they

come up with and they don't want to reject the the broad theory in the first place. Okay. So what do they do? They

start with uh you know asking questions about the veracity of the data, veracity of the observation, veracity of the

method of data collection and basically uh because they don't want to reject a given theory based on flawed data. Also

uh if they were convinced of the soundness of evidence, they will typically try and find ways in which

this new data can be accommodated by not you know uh by within the same theory without changing it so much. So it seems

therefore a more pragmatic approach that one can adapt or you know that we modify the theory a little bit to see if the

findings can be accommodated within the same theory rather than throwing away the theory altogether. So an example uh

basically that is cited by Brisbane and Russell is the is that modification of the existing theory to accommodate the

position of Uranus in the planetary system. For example, uh as per extent observations in the 19th century, uh it

gradually became undeniable that Uranus paths actually deviated too much from the Newton's law from Newton's laws to

be acceptable. So either you throw away the Newton's laws or you basically say that there is probably some issue that

is uh you know coming up in the calculation of the you know the path of the Uranus around the sun. Going by

falsification alone, scientists could have given up their theory and looked for an alternate explanation. However,

Learer in France and Adams in UK actually proposed that Newton's laws could be saved if we found somehow

that there is another planet in the neighborhood of Uranus that in that is influencing its orbit. On the basis of

these predictions, astronomers started to look for this probable uh you know planet probable uh you know heavenly

body and they started to calculate the whereabouts of this new planet and its approximate size and so on. These

efforts eventually led to the discovery of the planet Neptune and basically allowed for a way to to reconcile

Newton's laws with the orbiting path of Uranus. So you can basically see it's not probably the best idea to first uh

to throw away junk away a particular theory on the first test of falsification. The grounds of

falsification must be verified uh you know uh uh in in great detail. So this evidence basically suggests that a given

theory can be modified so that it is not any longer contradicted by the available empirical evidence. Even Popper

eventually agreed to this that scientists should not give up theories too easily and first test the tenacity

of the given theories through successive and iterative test. So multiple tests of falsification.

However, interestingly he points out that the modifications to a given theory should not in any condition make the

theory less falsifiable. So he was actually against ad hoc modifications. you know modifications. Let's say for

example you come across the first test of falsification that your theory does not well do well against you say okay

maybe we modify this particular aspect here and it'll work then another modification then another modification

if we are making random ad hoc modifications that are making the theory less and less falsifiable that is

bringing in vagueness in the theory then these modifications are what proper guards us against

now so this is this is so far now how do we move from here is basically something that uh you know is offered by Thomas

Kon. Uh Thomas was an American physicist who eventually became interested in the history and philosophy

of science and he takes the discussion forward about forming scientific theories and verifying and scrutinizing

these scientific theories uh through uh one of his famous works uh the structure of scientific revolutions. All right. So

Kun emphasized that all observations and theoretical uh concepts basically are some you know to a certain extent

influenced by the language of this adopted theory or the adopted conjecture and that typically scientific

disciplines are driven by prevailing uh traditions and conventions within their disciplines and in some sense if you

deviate from these conventions it invites isolation and ostracization and so on. So uh kun basically provides a a

framework that basically uh you know uh allows us to uh observe uh this falsification uh uh test and this

exercise of falsification in more context. Let us look at that. Now according to Thomas Kon uh scientific

progress has a structure uh according to which uh according to this structure scientific knowledge progresses through

a series of iterative steps. So it it basically says that that once at at some point people are in a stage of

pre-science uh then goes to normal science then there is a crisis or a revolution something major happens here

and then uh eventually people come to pre-science normal science and so on again and again. So this is a cycle that

repeats every uh you know once in several years. What is this pre-science that we're

talking about? uh at the beginning of each scientific discipline or each scientific inquiry researchers are

typically trying to understand the phenomena through a series of ill organized isolated facts. This happened

here, this happened here, this happened there and all of these facts are basically there and people are trying to

connect them, people are trying to make sense of them. Uh but because we do not have a wider framework to understand

this, uh there might be uh you know competing or contradicting explanations. There might even be uh disagreements

about the correct methods to be followed to understand a given phenomena to basically validate or situate the uh you

know investigation. So what KUN offers is that there should be a general framework uh which allows us to organize

the seemingly disconnected findings, disconnected observations uh even say for example theory. So it's probably a

superructure to these theories. So this framework uh will inform the researchers about the intercor intercorrelations of

these various uh ideas and also about uh some guidance about the methods that must be used to properly investigate

these facts and this framework is what uh you know con referred to as a paradigm. What is a paradigm? Simply

defined it is a set of common views about the discipline and uh you know about how the specific problems in the

discipline may be approached. what kind of methods have to be followed and what is considered a valid sort of bonafide

uh you know uh aspects of uh inquiry within that particular discipline. So according to kun paradigm basically

specifies a bunch of things. So for example it specifies what are the phenomena that must be studied what are

the things that must be observed and scrutinized. What questions do we ask about these phenomena? What are the more

acceptable questions to be asked? How do we structure these questions? How do we structure our scientific inquiry? And

also whatever we are finding what how do these how do the results of these scientific inquiries be interpreted. So

this is all of this falls within this uh you know superructure called a paradigm. Now once a paradigm is established

within a given discipline uh that is where con says that we are in the phase what is called normal science. So at any

point in uh time a discipline is functioning within this uh you know phase of normal science where there are

theories offered theories basically test particular phenomena uh uh you know predict particular phenomena test

particular phenomena and there are attempts at falsification it all is happening within a particular wide broad

framework. So within this framework what will happen is there will be attempts at proposing theories uh attempts to

falsify the proposed theories and test the overall strength of these theories. If however a systematic deviation is

discovered between uh between the predictions of a given theory and the observed line of findings uh you can

modify these theories within a given degrees of freedom and without making uh you know fundamental changes to the core

idea of a paradigm. For example, uh in psychology, we can talk about the behaviorist paradigm of psychology. The

idea that you know there are stimulus response associations that are the basis of uh you know understanding human

behavior. There can be any number of findings. There can be say for example conditions of uh classical conditioning,

operant conditioning, this and that and all of those findings are being viewed under the broad behaviorist idea.

Now every now and then there will be crisis uh uh in in any given scientific discipline. What is crisis? Basically

when you start having anomalies uh initially uh you know findings will start appearing that cannot be

accommodated within the existing framework or within the existing paradigm. So typically what happens is

in the beginning these findings will be seen as anomalies uh which cannot be uh you know uh accounted for or reconciled

within the given paradigm. Uh and when this happens uh basically what will happen is researchers will start

questioning the validity of the findings. They'll start questioning the methods that have been used to obtain

those findings and they'll find eventually if nothing is working they'll find ways of accommodating those

findings without changing the paradigm. However, in some cases, these anomalies will keep

on mounting up. They'll keep on accumulating and they'll become more and more severe. And at some point, it will

become inevitable that these anomalies that these new findings cannot be reconciled with the existing paradigm of

investigation. And when this happens, you can say that yes, the discipline is in a crisis at this point. uh there are

so many new findings that cannot be explained within the existing paradigm of let's say looking at human behavior.

Now when such a situation transpires there will emerge a need for the replacement of the existing paradigm and

make new predictions uh that can repeatedly stand the test of falsification. this particular uh point,

this particular emerging point has been termed as a scientific revolution. And a scientific revolution typically leads to

what is called a paradigm shift. So a a shift in the broad framework of how a discipline approaches a given phenomena,

what are the acceptable methods, what are the acceptable ways of inquiry etc sort of changes. So paradigm shifts are

characterized by intense scientific progress. Uh clarifications of of hitherto ill understood facts start

appearing and new ground is broken. New knowledge sort of appears and you basically move from one broad framework

to another new framework. Kun says that this could be the process of immense scientific excitement unlike

the everyday uh you know run-of-the-mill findings that are reported during the phase of normal science. So for example

during the behaviorist era or let's say during the psychoanalytic era or now during the cognitive era and within

cognitive era you have the information processing approach and the embodied approach and so on. Now uh within these

and they might be slightly loose definitions. Uh eventually what happens is that say for example once uh you know

uh there was this discovery of uh latent uh you know maps by these uh rats. uh it started becoming less and less uh

compatible, less and less reconcileable reconcilable with the idea that behavior is just stimulous response associations

that there is nothing called mind and there is no uh you know uh uh merit in talking about unobservable mental

phenomena. Now uh let's take some examples of scientific revolutions. uh you know for

example the Copernican uh revolution was uh something that was uh very significant which basically replaced

this idea of geocentrism that earth is the center of the planetary system with this idea of helioentrism which

basically said that not earth but sun is the center of this planetary system which was basically then called the

solar system. Now when scientific revolutions do take place, it could seem as if the entire backbone of a

discipline has been shifted or replaced, newer findings start coming in, newer topics basically become more relevant

and older topics lose their sheen and people lose interest in older investigations because they seem

misguided and there seems to be little value in pursuing those uh lines of inquiry.

Now eventually then what will happen is that once a new paradigm is set people start offering theories within this new

framework uh and the same process of offering a theory uh testing its predictions testing the tenacity of the

theory and so on progresses. So once the paradigm shift has gone through this new paradigm would take over and it'll form

the foundation against which new research will be conducted. The situation will return to normal and

things will you know proceed within the ambit of this new paradigm until the point that this same paradigm again

needs to be replaced by the emergence of newer and more and more irreconcilable findings. All right. So this is broadly

uh you know what I wanted to talk about. We basically charting the progress of uh scientific thought from uh you know the

positivist movement a little bit about how theories are generated what are broadly paradigms and then we'll move on

to more specific questions about is psychology a science how does psychology relate to theories and paradigms what

kind of inquiries you know do we undertake in psychology and so on and eventually then we'll start talking more

specifically about the experimental method in uh psychology. Thank you.