Foundations of Quantitative Experimental Design in Cognitive Psychology

Hello and welcome to the course basics of experimental design for cognitive psychology. I am Ark Warma from the

department of cognitive science at IIT Kpur. This is the first week of the course and I'm basically trying to uh

you know give us a brief historical background and the context of uh you know where the uh experimental method

actually derives from what are the underlying assumptions and so on. In today's lecture I will briefly survey

the quantitative methods in psychology. uh see in this course we are going deeper into experimental design which is

one of the uh most uh you know followed quantitative methods in psychology. Uh but it's important for us also to have

an overview of uh you know what are the other uh methods that are available and we'll try and uh build some

understanding an overview about uh you know quantitative methods and the underlying assumptions in today's

lecture. Now in the previous uh lecture we discovered that psychology could take at

least two paths. All right. Uh one was following the scientific method. uh that say for example uh derives uh broadly

from the logical positivist movement uh things that were uh let's say uh advocated for by uh Harmonow by uh you

know John Devi or William Wund uh or psychology to take could take a slightly different path say for example the one

advocated for by William Dily uh when he talks about the differences between mental sciences and natural sciences

All right. So there are two parts and there are two kinds of methods that have evolved through these traditions. From

the first one from the scientific tradition or the scientific method tradition, I'm not saying that the other

one is not scientific. That's that's not the idea. But from the scientific close to natural sciences tradition, uh the

methods that have evolved are basically uh termed under or clubbed under what is called quantitative research methods in

psychology. And from this other side uh the Dthian uh you know mental sciences approach the kind of methods that have

evolved uh are broadly referred to as qualitative research methods in psychology. Both are probably equally

scientific. Uh but we'll try and uh understand the underlying assumptions. So in today's lecture I'll talk mainly

about the quantitative methods and the assumptions that drive uh quantitative research methods. And in the next

lecture, the last one of this week, we will talk in some more detail about qualitative methods and the assumptions

therein. So let's let's begin. What are quantitative methods? All right. So quantitative research methods

are essentially those that work with quantifiable data that can be represented through numerical findings.

So whichever variables you talk about uh if you want to let's say talk about the uh the feeling of anger. Now what is it

about anger that I'm concerned with? When I'm studying anger through a quantitative experimental approach for

example or even a correlational study or even a descriptive study for that matter, I will need to provide this with

an experimental definition uh with an operational definition. Sorry. So what is the operational definition? I can say

okay the number of times a person uh you know uh say uses a cuss word or the number of times the heart rate goes up

or the number of times the uh breathing pattern changes the number of time the person something say for example maybe

skin conductance response. So uh when you're trying to study these uh things or these broad phenomena using the

quantitative methods the uh approach is to work with quantifiable data which can be presented as numbers. On the other

hand, if I were to uh do this with qualitative method, I'll probably be looking more towards the experience of

anger, uh where does anger come from? Why are some people more angry than others? And those kinds of questions.

For now, we're talking about quantitative research methods. And these methods typically because they are

working with quantifiable data because they are working with numerical representation of data. Uh they

typically work with statistical analysis preferred by researchers uh that uh you know and this is this method is as I was

saying earlier preferred by researchers in psychology that are working within or close to the natural science tradition

close to physics chemistry and those kinds of fields. Now let's uh very briefly try and understand the u

assumptions that drive the quantitative research methods. You will see that these are mostly in line with the

characteristics of the scientific method or the logical positivist uh ideas that we have talked about. Now uh one of the

fundamental assumptions that uh you know drive quantitative research methods in their totality or their overall

philosophy is this belief that there is an objective reality outside a person's mind. Okay. And this objective reality

is shared by different people. So when we try and measure this objective reality when we try and uh understand

this objective reality because this reality is outside an individual's mind the individual's context will not matter

and because it is outside and is shared by several other individuals the knowledge that we gain about this

objective reality can be extrapolated and can be generalized. All right. So this is something very very uh important

and fundamental to uh you know uh these quantitative methods. this idea of realism, this idea of empirically

verifiable objective reality that exists outside of an individual's mind. One, the second is uh within this tradition

and how do you approach this reality? Uh one of the best ways to approach this reality by the scientific method was uh

if you remember was uh you know proposed by Carl Pauper when he proposes the hypothetical deductive model. Remember

the hypothetical deductive model. You start with uh you know a theory. You come up with testable hypothesis. You

collect data and the data tells you whether to accept or reject this hypothesis. Whatever you learn at the

end of it adds to scientific knowledge. So broadly uh the way we deal with this objective reality, the way we approach

this objective reality is governed by what is called the hypothetical deductive model. We'll talk a lot more

about the hypothetical derived model in the next weeks or the uh you know uh future lectures.

Moving on, what do you want to do under the quantitative uh research you know methods regime? Uh one of the uh aims of

uh all of these quantitative methods is to try and decipher causal relationships between variables of interest. All

right. So for example uh if you are uh interested in understanding uh a relationship between two variables

say for example uh whether uh a person's attitude towards uh work uh leads to uh better or worse performance. So attitude

and performance. If you want to understand these things, uh the idea is that we will follow the methods that

will allow us to make causal judgments about how does attitude lead to uh you know job performance. Again, these two

variables probably might not be the best choice and there are other uh other examples that you can take. For example,

you can take uh you know whether adequate rest causes better memory or not. You know, this is much more

tractable and you can sort of have these uh you know operational definitions. you can control for a bunch of variables and

come up with causal relationships. But the broad aim within the quantitative uh you know uh research program is to be

able to understand systematic and causal relationships among phenomena among variables of interest. All right. So the

idea is that in this objective reality that we are trying to explore there are several variables and these variables

interact with each other in systematic manner. We carry out experiments. We carry out correlational studies. We

carry out descriptive research in order to get a snapshot of the of how these variables are interacting. And the

manner of our approaching these uh you know uh variables will uh equip us to make causal judgments about how these

variables are related to each other. Also because as I said you've specified one or two or three variables of

interest. There are other variables also in this reality that may or may not have an influence on these two variables. So

these variables that are irrelevant to your question that you're asking within a particular experiment for example

within a particular correlational study for example uh how do you work with these uh experimental methods basically

and in a more precise manner work very hard to control for and to avoid influence from these irrelevant

unplanned variables that can have uh an influence on the relationships of interest the independent and the

dependent variable. We'll talk about that as we go forward. So since the aim here is to find out true causal effect

cause effect mechanisms any outside variables that can have a potential effect on these interactions must be

avoided at all cost. And uh you will see researchers within the quantitative traditions try their best for uh

controlling these confound variables you know uh taking away these effects or sometimes if these variables are uh you

know randomly uh distributed in the environment you try to match for these things say for example that is one of

the reasons why uh in in a lot of studies participants are divided into experimental and control conditions by

random assignment. So you assume that there are let's say variables X, Y, and Z. They will each get canceled out in

the experimental and the control group. What do you want to do? You want to avoid confounds. You want to avoid

influence from all of these variables that uh you're not interested in at the moment. Maybe later, maybe once you

discover that okay, there is this third variable which is also important, you'll design another experiment to test for

the effect of that specific variable. All right. Uh yeah. So objective reality uh cause effect relationships avoiding

confounds and finally uh there is there's a very uh you know clear acknowledgment among the co

quantitative researchers that one of the major sources of confound could be the researcher himself or herself. Okay. So

an important factor that can color the results that you get that can color the procedures that you follow that can

color the data that you have collected is are basically the expectations the perceptions and the opinions of the

researcher. All right that is why you'll see in a lot of uh you know cases you have these double blind in in medicine

for example there are these double blind trials. Why are the double blind trials there? because uh if the researcher

himself or herself wants the medicine to work that should not color the uh results that you're actually getting uh

when you're testing these medicines. All right. So, uh the expectations, perceptions and opinions of the

researcher can actually so there is this uh idea can actually influence both the data the method of data collection and

the final results and uh quantitative researchers will try their best to keep uh you know this effect out. How do they

do this? They do this by following and uh you know adopting standardized measurements, standardized instruments

so that there is no difference uh between say for example the instrument that I am using for making this

investigation and somebody else uh in another university in India or another institute in the world is following. We

conduct the research entirely independent of both the researcher and the research. At this point we're

talking about the researcher mainly. Finally uh and something that is extremely important as uh you know

demonstrated uh by Carl Puper and we've discussed this in the past is this idea of falsification as the preferred method

and not verification. We've seen in the first lecture that verification is something that is almost logically

impossible to achieve. You cannot go out and sample the data from the entire population. There are you know more than

six billion people in this world. But what you can do is you can test for your theory or you can test for your

prediction by uh various uh attempts at falsifying the theory. The more attempts at falsification a theory survives, the

more tenacious, the more solid the theory is going to be considered. Okay. So, uh falsification is is very

important. uh the most important tool for quantitative researchers as is because what they're doing is they're

constantly trying to test the truth of their conclusions, their theories, their hypothesis, their instruments by trying

to replicate the experiment in various conditions and by trying to see what are the boundary conditions for this theory

where all this will not work. You remember in chemistry when we were doing when we do experiments we give the

temperature, pressure, volume, temperature is so and so, pressure so and so volume so and so. In some sense

we are defining the boundary conditions for a particular reaction to occur in these temperature, pressure and volume

conditions. This is what the this is the reaction that was observed. This is automatically the boundary condition and

this is similar to what the psychologists are trying to do when they're trying to uh you know carry out

these falsification tests to uh establish theories. Now uh what are the types of uh uh

quantitative research that exists? As I said, we'll very briefly survey and we'll have an overview of these kinds of

research methods. So the first one that is considered very important and as the first step in the quantitative tradition

is descriptive research. All right. Uh what what is what do we do in descriptive research here? The focus is

on making observations on generating the snapshot of a situation. Say for example, if you're interested in say for

uh things like how many people buy a particular detergent for washing their clothes, how many people like wearing uh

blue shirts when they're going to office, anything for that matter, how many subscriptions do uh you know social

media creators in the food blogging category have. You just go out and you measure and that measurement gives you a

broad snapshot of how the variable is distributed in the population. Let's say you discover that 9 out of 10 people

like to wear blue shirts when they go to office. Something like that. You get something. It does not tell you much

more. It does not tell you why do they wear blue shirts. It does not tell you what does the blue shirt achieve for

them. Uh it does not tell you uh anything more than that. It basically just tells you that so and so uh amount

of people buy this particular detergent or have so many subscribers or wear this particular color of shirt when they want

to go to office. why it is considered formal. So uh these data typically are gathered in a numerical form by

collecting measurements uh you know uh simply by uh you know frequency counts over large samples of the population. So

for example uh typically descriptive research uh you know there are surveys for example they are carried out on a

large number of people thousand people 10,000 people 100,000 people and these surveys basically give you the

distribution of that variable in the uh you know in the overall population. However, something that uh must be kept

in mind is uh the way the uh you know they uh the measurement of a variable is carried out is still tied to the

operational definition. Say for example uh if you want to measure angry outbursts on social media, you may

either define it say for example by the number of hateful comment a post receives or by the number of dislikes a

post receives let's say in YouTube or you know on some of these other platforms. So you have to still define

that okay this is the broad variable of interest but within that broad variable of interest this is the operational

definition that we are going with all right and again another interesting aspect of descriptive research involves

a collection of limited so again it's not a a lot of it's not in-depth data collection it's basically covering one

or two or three aspects of this overall data so it's almost superficial if you want to use that term uh But this is

data from a large group of participants. Okay. So you typically will uh uh you know ask people one or two or three or

four questions. Uh but you will ask these questions from a lot of people from a a large number of people so that

you get a better sense of how things are in the overall population. There are benefits to this also. For

example, collecting data from large groups allows the data to be more representative of the general

population. So it gives you the picture of the overall population. uh and the findings therefore whatever you generate

uh can be uh you know generalized across the population. Whatever you say, you can say oh this is true not for just my

sample but for the overall population because I've collected data across you know you you've seen these exit polls

and so on you know they sample a number of uh people from different stat of society from different geographies

within a particular uh state or district and so on and on the basis of that they say that this seems to be the you know

the uh broad trend of whether a particular candidate is winning or losing the election.

Also collecting data from a large number of people allows for more precise statistics. It allows for slightly more

powerful prediction uh because u as as I was just saying the mean of the sample in your uh data set is will tend to be

closer to the mean of the overall population. So in that sense whatever uh you know trends you observe in this uh

sample mean will be very similar to the uh overall population's characteristics. All right. So uh there are therefore you

can see there are benefits to carrying out descriptive research as well. But again uh this is the starting point of

the quantitative research program. It allows for a holistic overview but it does not tell us about how two variables

of interest are related. It does not tell us about the causal relationship between the two variables which we were

just saying moments ago that is the basic enterprise of this qualitative research paradigm. So what are the other

other ways? Let's let's move on and let's find uh another one uh or uh uh you know a helping arm to this is

correlational research or relational research. Now what do correlations do? Correlations allow psychologists to

measure two variables of interest and test for whether they influence of influence each other in meaningful ways.

And there are a number of statistical techniques available that would allow the researchers to calculate a

coefficient say r. Uh and this coefficient will tell you how uh you know closely the two chosen variables

covary. Say for example if you want to talk about job performance and job satisfaction how does job performance

and job satisfaction vary? If one increases as the other one also increase or if one increases the other one

decreases something like that. Okay. uh these statistical techniques provide us information about both the direction of

the co uh direction of the covariance. Say for example uh is it if it is positive both will increase or decrease

in the same manner. If it is negative one will increase and the other will decrease uh or let's say if it is not

related and both are varying independently. Also it tells us this r tells us about the strength of the

correlation as well. Say for example what is the magnitude of r? Is it.20 20 or is it 80?20

will suggest a weak correlation and 0.80 will suggest a very strong correlation. Again, how much these variables are

influencing each other is the question that we are asking. But there are a couple of issues in

carrying out correlational studies as well. For example, uh sometimes it is difficult to detect these correlations

in the first place. Sometimes you have so much data and you're carrying out these statistical tests uh tests. It is

possible that uh one is not able to detect uh correlations where they are you know when they actually exist. For

example, negative correlations uh are slightly more difficult to detect. Okay. For example, the correlation between

smoking and longevity. If you smoke a lot, obviously it has an has an effect on the longevity of of an individual's

life. But researchers have found that you know sometimes these negative correlations are much harder to detect

as opposed to uh you know correlations that are positive. Another issue is that a lot of times it is possible that there

is a third there is no correlation between uh you know the two variables but you tend to perceive that

correlation. So these are called illusory correlations. You tend to conclude that they there might be some

effect but there is actually no effect. For example, one of the examples that Brisbet and Rasel take is this uh

between the handwriting of individuals and their personality. It's it's common opinion uh that people with impulsive

personalities, you know, they write very fast, their writings writing uh is poor and things like that. Whereas people who

are cautious, consensious, more calm will write very slowly uh and uh their personality will in some ways interact

with their uh you know handwriting as well. Again uh objective a lot of research basically suggests that there

is nothing like that. Uh but it is possible in some studies uh because of various other factors to uh get these

correlations and you know try to make some sense out of them. Uh also uh when one carries out uh

correlational studies a lot of times it is not just the two variables or one or two variables that you're interested in.

there are other variables that also play a part. Okay. So it is possible that for a given uh you know uh phenomena that

you are interested in studying there are more factors that are having an influence. There are more factors that

are influencing each other than just the one or two that you were considering. So uh there is this technique called factor

analysis that allows us to study the inter relationship or the correlations between various uh variables. For

example, uh you know salary that one is getting, experience that one has at uh one has at the job, the expertise, how

good one is at the job and job performance are four variables that you can intuitively think will have an

influence on each other. How much does variable one influence variable two or three or how much does variable four

influence variables 1 2 and three is something that can be best determined through factor analysis. All right. So

it is a statistical technique. We'll talk about this uh later when we are doing uh you know when we are going in

more depth but it is a statistical technique that allows uh us to calculate how many factors are required to account

for correlations between uh the variables being measured and how these variables relate to these factors. What

are the contributing uh you know mechanisms? For example uh look at this figure.

There are three variables that are there education, family situation and income. and then uh the both all of these

variables are affected by a common factor that is uh you know the socioeconomic status. So what are the

the what are the categories what are the you know uh factors that are contributing uh to all three of these

variables. Now the final uh and the most u respected or preferred uh you know

branch within quantitative traditions is experimental research. Okay. It basically addresses one interesting

lacuna of correlational research is that uh in correlational research it is very difficult almost impossible uh to uh

infer causal relationships amongst variables because uh you know a lot of times you don't even have the best idea

about how the variables are co-aring. Uh sometimes there can be spurious correlations. For example, there's a

third factor that you have not considered which is causing uh you know uh the variation between both of these

variables of interest. So uh because you're not able to control uh one v all the other variables and manipulate one

which is the independent variable and measure it effect on the second one which is the dependent variable. The

correlational studies do not allow us to infer causal relationships. Experiments on the other hand actually provide us

with this uh uh you know with this uh luxury of being able to uh decipher how if the two variables of interest are

causally related to each other. Okay. So for example several uh you know studies and there's this very interesting

example that has been taken uh in in this chapter uh is that several studies may indicate that there is a negative

correlation between let's say uh having children and marital satisfaction. uh studies suggest that couples with

children show less marital satisfaction than couple with couples without children. Now uh there is a correlation

but is having children causing uh you know people uh causing couples to have less marital satisfaction? How will you

uh you know uh investigate this? So if you want to investigate something like this, you will need to follow the

experimental uh research paradigm. We we'll talk about this here. Now when you look at this example for example, it is

it is possible that there are many contributing factors that are uh you know uh contributing uh to people having

less or more marital satisfaction having or not having children is just one of them. So if you really want to determine

if there is a causal relationship you will need to control for several other things. You'll need to control for say

for example soio economic status overall family income where are where is the family living what kind of uh you know

compatibility for example is there between the couple and so on. So we'll need to control for several variables

and vary only whether the couples have children or not have children and then maybe make an uh you know uh an estimate

about whether it is having children that is causing uh people to have or not have marital satisfaction.

So experiments allow us this experiment allow experiments uh allow researchers to draw these cause effect uh you know

relationships because they allow as I was saying uh one to uh they allow the manipulation of one variable while

controlling all others and determining the effect of this manipulated variables manipulated variable which we call the

independent variable on the interest variable or which we call the dependent variable. Okay. So, as I was saying in

in the previous example, researchers, if they're interested in deciphering a causal relationship here, will try to

control for or match a number of possible causes that may contribute to lowering marital satisfaction. Uh, and

in that sense, they will probably uh have one group that has children and another group that does not have

children and then matching them on all other possible confounding variables. So, uh, and this is what is called

establishing initial equivalence. same ages, same backgrounds, uh compatibility is similar, uh family income is similar,

uh geography is similar, bunch of these things will need to be controlled. All right. Now, uh and also one of the

things is uh the time of the marriage and so on. So, both of these groups will need to be followed for several years to

see the evaluation uh evolution sorry of marital satisfaction. If the presence of children is indeed the cause of drop in

marital satisfaction, such a drop should be observed in the samples that have children only. Okay, so this is how you

set up an experimental uh you know uh research paradigm uh to try and work out the causal factors.

Now uh there's there's something uh I wanted to say before I move to the next slide is that a lot of times carrying

out experimental research is not feasible. Say for example uh you cannot have the same couple uh and observe them

when they don't have children and observe them when they have children and and you know things like this. So for

that you will need a a much larger time frame and so on. So a lot of times one would discover that it is difficult to

uh experimentally control uh confounding variables. It is difficult to match uh you know all the other possible

intervening variables. That is why uh this is one uh drawback uh in some sense or limitations of the experimental

method. All right. Now within the quantitative tradition a very interesting uh you know

demonstration of the strength of evidence provided by these various method descriptive, correlational and

experimental is uh basically uh you know best seen in in uh the hierarchy of evidence uh that is uh you know uh there

in the medical sciences. For example, you can see this is the hierarchy of evidence in this figure uh of uh in in

medical sciences. You will see at the bottom are case reports. Why are case reports at the bottom? They are part of

the descriptive research paradigm. Uh they allow us to collect a lot of data. Let's say for a smaller sample, one

individual and so on. But the interesting thing is case studies are less generalizable. Say for example, you

want to study a person uh with brain injury. Uh it will be difficult to have the same injury in in several people.

Okay. So for example, somebody comes with a particular lesion in uh in a given vauel of the brain. Uh it'll be

difficult to find other people which have injury in that same particular voxel in the brain and in that sense the

generalizability of the finding will become uh limited. All right. So case reports are interesting. They are

important. They give us a lot of information but they are least generalizable uh form of data. Another

thing is cross-sectional survey. Say for example again cross-sectional surveys you want to let's say study the

distribution of uh vitamin D for example at different age groups you can do that but what does it tell you does it tell

you a lot does it tell you why is vitamin D at whatever level that you found uh in this group versus that group

versus that group not really it gives you broad snapshot of how the vitamin D uh you know distribution is in the

general population then there are case control studies then there are follow-up studies. At the highest thing you will

see that these are randomized control trials which are uh you know closer to the experimental paradigm closer to the

double blind method and so on which control for all other possible intervening variables and allow you to

decipher a causal relationship between let us say a particular medication or a particular drug and its effect on a

given disease. At the top you can see there are meta analysis which are basically uh uh you know uh combinations

they are basically conclusions drawn from similar comparable but a large number of experimental studies. Okay. So

this is just you know it's it's a parallel to what we are trying to do here in psychology.

Now uh do quantitative methods have limitations? Yes, they do have limitations. For example, as I've said

earlier, uh there's no interest expressed in the person behind the participant. Okay? So, you are broadly

uh you know polishing over all the idiosyncratic qualities of the individual. You are polishing over the

context. You're polishing over uh agency, choice, this that motivations, goal, you know, the emotion uh

motivation kind of uh idea that uh Dilly was talking about. There's a lot of stress probably too much stress on

quantification and numerical representation and things that cannot be expressed numerically are typically kept

outside the scope of quantitative research methods and quantitative inquiry

also uh is falsification uh you know is is is the best tool that we can have. A lot of researchers opine that

falsification may not be pragmatic because while it you know allows us to test uh and bring down theories. It it

is a good tool to reject if a theory is uh not solid and so on. It does not generate a lot of new ideas. It's it

does not to a certain extent and at least it's it's what some people feel does not take the science forward so

much. It does not generate a lot of new ideas. So these are some of these uh you know limitations of the quantitative

methods and uh let's let's we at this point we're just having a holistic survey. We'll go uh in much detail about

some of these topics as we go forward. Thank you. Thank you.