Understanding Internal Validity in Cognitive Psychology Experiments

[music] [music] >> Hello and welcome to the course basics

of experimental design for cognitive psychology. I am Dr. Eric Verma from the Department of Cognitive Science at IIT

Kanpur. We are now entering the sixth week of the course and I will like to continue

the discussion on reliability and validity of experimental designs a little bit more. In this lecture we'll

basically talk about we'll we'll you know extend our discussion on validity of experiments a lot more.

If you remember in the previous lecture we talked about construct validity and we've talked about you know how

can research be reliable and how can our measurements be treated as reliable. In this one we want to talk about internal

validity. Now valid research basically refers to the one that leads to legitimate

conclusions. So whatever experimental protocols one has followed, whatever experimental manipulations one has

followed, whether they are yielding the correct and dependable results or not is basically referred to as by internal

validity. Now [snorts] researchers actually need to be sure that what that the entire research enterprise as

evaluated at various stages is free from any mistakes that could lead to errors in the procedure as well as conclusions.

So for example remember we start with say for example forming operational definitions of conceptual

variables. So we convert those conceptual variables to measured variables. In the previous lectures

we've talked about whether the measured variable is actually encompassing the conceptual variable or not. So we talked

about that as a result you know as a measure of construct validity. There is also for example you know a possibility

that the experimental manipulation has not worked. So if the experimental manipulation has not worked then how do

we sort of uh you know conclude correctly from our experiment results. So that basically can be a measure can

indicate that our experiment is not internally valid. It is the manipulations have not worked. The you

know the changes that we are observing on the dependent variable are not because of the

independent variable manipulation but because of something else. So these are some of the things that as experimenters

we have to be guarding against. Let's take an example. If a researcher claims that you know there is a particular drug

let's say drug A it works against a specific disease let's say such diabetes or any XYZ disease. Such [snorts] a

claim would be valid only if the new drug really works on the diabetic patients. It will not be valid say for

example if there is if the drug is not really causing the reduction in symptoms but something else entirely is actually

causing the reduction in symptoms in diabetic patients. So there are you know several possible

threats to the validity of an experiment which basically will make it difficult for researchers to make proper

conclusions proper interpretations about the research enterprise. And it it happens a lot of times because

despite you know sometimes the best intentions of the experimenters best intentions of the researchers it is

possible that several research conclusions that are coming out are actually invalid because of some mistake

at some step during the procedure. Maybe the experimental manipulation did not work. Maybe the measured variable is not

the correct measured variable you know for representing the conceptual variable. So

construct validity is not there. And in that sense validity also we cannot treat it as an all or none phenomena.

Sometimes most of the things in the experimental protocol are fine but because of a small mistake something

that is basically happening is not allowing you know is basically is not allowing the conclusions to be perfectly

valid and legitimate. So it's not an all or none phenomena. Sometimes it can be treated as a degree to which a

particular results are valid the degree to which the particular you know results are correct.

So only if researchers you know understand the potential threats to validity they [snorts] can ensure the

that the research being conducted by them is actually valid. Now there are four major types of you

know threats to validity of research and we've talked about some of them. We will continue our discussion in this week

talking about the others. Now we've talked about construct validity in the previous lectures as well. What is

construct validity? It is basically threatened when the measured variables that you are using in the research. Say

for example remember in the last class I was talking about if you want to measure >> [snorts]

>> aggression as a conceptual variable and you're taking the measured variable as the number of smiles per hour or per 20

minutes. That is obviously you know on the face of it not a valid indicator of whether somebody is going to be

physically aggressive or not. So in that sense you know that the construct validity of the measured variables is is

not there. And in that sense if you know an experiment is carried out using those definitions then the experimental

enterprise per se is not valid because it does not adequately assess the conceptual variables that it is designed

to measure. So that is the threat to construct validity. We can talk about statistical conclusion validity. So for

example once the results are there and you're trying to carry out analysis and on the basis of the

analysis of the results you're trying to make certain interpretations. At this point there are two kinds of

errors that are possible. For example it is possible that you incorrectly reject a null hypothesis. The difference is

actually not there but because of some reason or the other the experimenter or the researcher you

know ourselves we get an impression that no there is actually a significant difference between the experimental and

the control condition and we reject the null hypothesis mistakenly. Such a case is called type one error and type one

errors are very common within research because sometimes we do not you know analyze the results very well

and we basically mis-conclude the interpretations from a given experiment. So that is called type one error.

It is also possible that there is actually a significant difference and we basically mistakenly fail to reject the

null hypothesis. So if we are not rejecting the null hypothesis even in cases that you know the null hypothesis

is untrue then it is called an instant of the type two error. So there are you know type one errors and type two

errors. Whenever either of the two is there we basically say that there is you know a lack of what is called

statistical conclusion validity. Basically what has happened is that the researcher has drawn incorrect

conclusions about the research hypothesis based on the data. All right. So construct validity, statistical

conclusion validity. In today's lecture and you know coming lectures we will talk about internal validity and

external validity. What is internal validity? Internal validity specifically defined is under threat when the

researcher cannot trust the conclusions that have been drawn about the causal relationship between the independent and

the dependent variable. For example there is an extraneous variable or there's a confound variable which is

also causing a change in the dependent variable. If that is the case then you cannot be very sure that the causal

relationship that one is inferring about the independent and the dependent variable is correct or incorrect. In

such cases and in all such cases we would say that the internal validity of the experiment is questionable and the

experiment is not internally valid. We cannot draw legitimate definitive conclusions about our experiment.

Finally there is external validity which refers to the extent to which the results from

a given research enterprise can be generalized beyond the specific settings beyond the specific lab beyond the

specific protocols that are followed beyond the specific participants and can be generalized across people across

different settings and can actually be used to inform public policy and can be used to inform

you know our descriptions of the world. So these are the four main types of validity that basically

can affect our experimental enterprise and then experimental results. And it is therefore very important as

researchers to understand where these are coming from and how can we sort of you know work around them? How can we

increase the internal and external validity? How can we increase the construct validity or the statistical

conclusion validity of our experiments in order to be able to draw perfect conclusions about our experimental

enterprise. Now a very important aspect so we'll talk about you know aspects of internal

validity in this lecture. An important aspect of the experiments that ensures their validity is if it has or if the

experimental protocol has good experimental control. All right. And good experimental control will be there

to the extent to which that the experimenter has been able to eliminate the effects of the dependent effects of

other possible variables on the dependent variable. See why are we designing the experiment and

what are we trying to achieve out of it? We are trying to actually control for all other possible variables and

basically manipulate one selected independent variable or maybe two if you are doing a factorial design kind of a

thing. But one or two independent variables we are varying and we are measuring the effect of this variation

that we are doing on our dependent variable. If however by you know some case there

are other variables in play as well which have not been controlled for which have not been matched which are

violating the initial equivalence of the participants then what will happen is that the

effects on the dependent variable that we are measuring we will not be sure whether they are actually happening out

because of the independent variable manipulation that we are performing or because of any other variables. So the

experimental control is the degree to which a experimenter has been successful in controlling for all other possible

conditions. If the experimental If the experiment is well controlled, if there is a high degree of experimental control

in any experimental enterprise, then it is quite possible it is you know easy to follow that the

experiment will have higher internal validity. So the better the experimental control, the more confident a researcher

will be about that it is the independent variable that is actually causing the changes in the dependent variable which

we are measuring. Now there are two kinds of variables that can actually flout the you know

experimental control conditions and let us talk about them. The first kind of variables that can be

problematic are extraneous variables. These extraneous variables are variables other than the independent variable that

can potentially cause changes in the independent variable. For example, these would include in for example, these

would include the initial differences among the research participants. Remember

when we are going through the you know the aggression study, two groups of children watching violent

versus non-violent cartoons and then their aggression is being measured. We said in several previous lectures that a

starting point of that lecture will be to ensure that there is initial equivalence between the two groups of

children. They are matched say for example on age, socioeconomic status, educational background, parents

disciplinary style and so on and so forth. So the degree to which we are ensuring this initial equivalence, the

degree to which we are able to ensure this initial equivalence will basically suggest that [snorts] we have controlled

for possible extraneous variables. All right. Extraneous variables basically they obviously include both initial

differences among the research participants in the characteristics such as ability, mood, motivation and also in

the differences in how the experimenter you know treats the participants or how they react to or how the participants

react to the experimental setting. So while the participants are coming to your experiment and they have been

matched for and their broader characteristics, demographics, etc. have been matched sometimes what happens is

unwittingly the experimenters treatment of the participants becomes different and in those cases it is possible that

that treatment of the participants is causing the changes in the dependent variable. So that is also something that

we have to be very careful about. We have to be very sure that that is not causing our participants to respond

differently in two conditions, the experimental condition and the control condition. Remember suppose if we are

doing a within subjects kind of a design. Now as these variables are typically not measured by the

experimenter, their presence increases the within groups variability in an experimental research design making it

more difficult to find the differences among experimental conditions on the dependent measure. So remember if there

is more within group variability, we will never be able to conclude that the differences in the dependent variable

are actually due to differences in the participants characteristics or their different differences due to

experimental manipulations. That is what reduces the confidence in the experimental findings. That is what

reduces the validity, internal validity of our experimental protocol. These variables would typically cause

random error or noise in our experimental measurements and thus will increase the likelihood of what we call

type two errors. Okay. So we will basically a lot of times not be able to conclude you know properly that

measurements that we are taking on the dependent variables are actually due to IV manipulations, extraneous variable

manipulations or due to some other random factors which are not controlled for.

The other class of variables that are also equally important, that are also you know sources of

or threat to our internal validity are variables other than the independent variables wherein you know participants

in one experimental condition can differ systematically or on average from the other condition. You can imagine doing a

within subjects design as well or you can imagine doing a between subjects design, but if there is a systematic

variation between participants or systematic variation between experimental and control settings which

is causing the changes in the dependent variable, but which is not due to your independent variable manipulation. Okay.

So that basically is referred to as confounding variables. Let's take some examples. Now broadly when we start

remember what we were doing was we were basically going ahead with random assignment to conditions. So we are

saying we created an initial equivalence, even number participants go to watch the violent cartoons and then

their aggression will be measured. Odd number participants go to watch non-violent cartoons and then their

aggression will be measured and we are basically interested in knowing that okay, what kind of watching what kind of

cartoons is creating differences in their aggressive play. Let's say that's the framework of the example. Now when

we start with random assignment to prevent we basically hope that it will prevent the systematic differences among

the participants. Whatever confounding variables or possible confounding variables they might be

will basically be the ones that are created by the experiment itself. They will be created within the experiment

itself and they are sometimes unintentionally created by the experimental manipulation. For example,

it could be that something in the conditions in between the experimental and the control conditions is

systematically different. Okay. So the example that the book takes is say is say there is a researcher who is

interested in you know designing an experiment to study whether people in you know working in groups rather than

alone perform better on mathematics problems. Okay. Now the experimenter is very you know cautious. They have

designed a very good experimental manipulations. They are basically going to collect data.

But in their zeal and in their enthusiasm, what they did is that when participants were supposed to work on

these mathematics problems alone, they were made to work in the basement of a building with no windows you know dim

light and so on basically assisting their relative isolation. Now the participants are

working here in relative isolation and they are solving mathematical problems. And in

the experimental condition they are working in groups. Now in this time what is happening is they are working in

large groups, but in bigger rooms, larger windows, good lighting and so on. Now

while the critical manipulation is whether the participant is working alone or whether the participant is working in

a group, but there is also a systematic variation that has taken place here. Okay. You can pause and think about it,

but the idea is that if in the isolated condition they are working in a room which is systematically overall

different from when they were working in the group condition, you cannot be sure whether it is the group or isolated

manipulation that is causing the differences in the in the mathematical performance or it is

the nature of the room. So basically you can see what has happened here is that there is a systematic variation in the

conditions that has crept in. The researchers did not plan about it, but they sort of mistakenly missed this

particular manipulation. They missed that you know the conditions should be identical in both

the measurements. So if the participants were to work in isolation, they would work in the same room. If the

participants had to work in groups, they should have gone to work in the same basement kind of room. Or if they were

to work alone, they were to they were should have been made to work in the same large rooms with large windows and

if they were to work again and the next measurement is in groups, that should have been taken in the same large room

with large windows. In this case, there are two variations that are happening. One that is

accounted for which is whether the participant is working in isolation or in group and the other is the condition

of the room. The condition of the room becomes the confounding variable which may or may not have an effect. We can

sort of look at the results which may or may not have an effect on their you know mathematical performance and in that

sense has the potential to be a confounding variable, has the potential to have had some kind of mixed effect,

some kind of issue in the overall measurement. A lot of times you will see it can happen,

something like this can happen which will cloud the results of your experiment and you will not have planned

for it. In those cases we can say that the internal validity of your experiment is threatened.

Now confounding and internal validity basically you know that you can sort of work with it, you can try and sort of

you know get around this as well and that is something you know that as experimenters

we have to be very careful about. So as I was just talking about in the previous example, confounding refers to the fact

that the other variable gets mixed up with the independent variable making it impossible to determine that which of

the variables has produced the difference in the observed you know measurement of the dependent variable.

It could be just the fact going back to the previous example that given that these other rooms were larger you know

had better light, had better windows and air, they caused the improvement in performance rather than the fact that

the participants were working in groups. Okay. Now another way in which you know internal validity can be threatened or

you know confounding can happen is that if there are alternative explanations possible. Now when you are manipulating

your independent variable and you're measuring the effect of the manipulation of the independent variable on the

dependent variable, you are assuming and that is the fundamental assumption that it is only the IV manipulation that is

causing the change. And you go with that explanation when you are writing down the results, when you are describing

your results, that is what you are working with. But a lot of times and this will happen to all of you when you

start writing your papers and so on after conducting your research, a lot of times as experimenters, it is possible

that we miss that oh, there could have been another variable within our experimental manipulation that could

have caused this change. And this is something that a lot of reviewers point out that yes, there could be alternative

explanations to whatever results that you have found. Okay? The presence of a confounding variable does not

necessarily mean that the independent variable did not cause the changes in the dependent variable, but there can be

sometimes an alternative explanations that can explain these obtained results. Now, the

degree to which you can rule out any other alternative explanations is basically the control that you have had.

The degree to which alternative explanations can also play a part in explaining your results is the lack of

internal validity, is the lack of experimental control in your overall experimental paradigm.

Okay? Now, let us talk about ways in which we can control these things. One of the

very interesting ways to control the effect of extraneous variables experimental enterprise is follow what

is called limited population designs. Okay? So, this is one of the reasons why a lot of people use within subjects

experimental designs as well. Now, as the possible source of extraneous variables typically involves the initial

differences among the research participants to the extent that these differences are playing a part and they

constitute random error, we can choose for ways to undermine this. So, within subjects design if you're using, it is

automatically, you know, offering you some reprieve with respect to a lot of differences between participants will

have been taken care of. It is the same participant which is going through the experimental and the control condition.

Okay? And we can talk about order and other things we've talked about in the last lecture.

Another way you can actually do this is that you know, you select the participants from a limited and

therefore relatively homogeneous group. This is something that is followed a lot in experimental research because

typically what we do is across universities in, you know, around the world, we are typically working with

undergraduate college students who basically form a relatively homogeneous base. You know, most of them are within

the same age group, 18 to 30, 18 to 25, drawn relatively from a similar socioeconomic status given the kinds of

universities that there are. But a lot of times, you know, you'll see while sampling you take care of the fact that

they are drawn from a, you know, same level of socioeconomic status as well, same strata of socioeconomic

status as well. So, [snorts] if you're drawing your population from, which is, you know, coming from a,

drawing your sample from a relatively homogeneous population, it basically provides you a very good chance that

extraneous variables at least to the extent that, you know, we're talking about extraneous variables due to

initial differences between participants are controlled for and they're sort of, you know,

in check. So, that is one. There is also other, you know, kind of design when you do a

pre-post kind of a thing or you do a before-after research design. Now, what is a before-after research design? A

before-after research design is that you basically measure all your participants on your dependent variable prior to the

experimental manipulation once and then after you carry out the experimental manipulation, you measure them again. So

that what you have is now you'll have a baseline measure. Let's say your participants come and

they are, you know, performing on a test of memory. So, you first measure them on the memory of whatever you want to test

them and you get a, you know, an estimate of their overall memory. Then you perform your experimental

manipulation, then you again get the measurement and now you can compare the post-experimental manipulation

measurements to the pre ones. So, you can control this one with the baseline and that provides you a very good

estimate of, you know, the fact that the variations that are happening are basically only because of the

manipulations. All right? So, this is an example of a before kind of a design. The baseline measure, all the

participants study and recall a particular list A and then there's a list B which is equivalent to the list

A. So, what you do is you randomly assign participants to the experimental and control condition, then both groups

are studying the list B, one with using instructions. So, in this one they have been asked to remember each word from

the list by making a sentence and here they're asked to measure the each one, you know, each word in the experimental

list without any instructions. After that you basically have them have them recall the list B and whatever whatever

differences in these two conditions that you will get are probably because of the difference in instructions and not

because of the initial difference. You can actually compare the recalling of list B with recalling of list A to

basically get an estimate of the fact that okay, this was the baseline, you know, memory for a list with this

participant and this is the post-experimental manipulation. Did the instructions actually create a

difference? That will also give you a very good estimate. You will compare these two as well and you will come you

can compare the performance of these groups with their prior or baseline performance as

well. Okay. So, just revisiting this experiment so that it is clear. So, the

idea is there is an experimental design that we're doing where college students are given a list of words to remember.

Half of the students are randomly assigned to a condition in which they are instructed you know, to

which they basically construct sentences using each of the words and the other half are just told to remember the words

as best as they can. After a brief delay, the participants are asked to remember the words. Now,

obviously there can be many differences even without the manipulation. So, what we do is that you basically, you know,

have them give the, you know, recall on a prior list, list A, which is similar in most respects to the list B. Okay?

This will take care of prior differences such as verbal skills, current mood, motivation to take the experiment

seriously. All of that you already get when you're doing your list A recall. So that you know, okay, this is the general

pattern of performance here. After that you assign the participant randomly to the two conditions. Half of them

basically will make a sentence and then try to remember, half of them will just try to remember and memorize that and

then you give them the recall test again. So, basically what will happen is in such kind of a before-after design,

the dependent measure is assessed is basically measured both before and after the experimental manipulation. So, hence

in this design what will happen is that the students will memorize and they are

tested on one set of list of words, which is list A. Then they're randomly assigned to one of the two memory

conditions. So, with making sentences, without making sentences and then are tested again. The first memory test is

known as a baseline measure which can be compared with the post-experimental or post-manipulation test as well, which is

known as the, you know, critical measure. Now, what are the advantages of this

kind of design? Now, the idea in, you know, behind this before-after design is that any differences that you observe

among the participants will basically have influenced the baseline measures as well. That was also serving as a

dependent variable. So, if there are any differences between the participants, let's say half of them and other half of

them, those differences will come up already in the baseline measure and once you use the baseline as a reference to

mark up the post-experimental manipulation measure, you will know that okay, these differences are due to

experimental manipulation and not due to baseline differences or initial differences between your participants.

So, for example, in this particular experiment that we've taken as an example, if a student with a

particularly good memory would, you know, has scored better on, you know, list A, you know that this same student

will have scored better on list B as well. And now when you subtract, you will basically get actual difference

based on the instruction of, you know, the memory exam. It is notable, however, that both the before-after research

designs share some similarities with the repeated measures between subjects design, sorry, repeated measures within

subjects design that we were talking about before. In both of these cases, you know, the participant is measured

more than once. In the within subjects, the same participant goes through both conditions. Here, the same participant

gives a baseline measure as well as the critical measure.

Both of these designs, the repeated measures and the before-after design, increase the statistical power of the

experiment by basically controlling for variability among the research participants. All right?

The difference, however, there's a small difference here. The difference being that in repeated measures design

each individual is in more than one condition of the experiment, but in before-after experiment design, each

person is only in one condition. But basically what we're doing is we're measuring the dependent variable more

than once, as a baseline as well as the critical measure. There are also disadvantages of

before-after designs. For example, they can, while they can help reduce random error in measurements,

they also create, you know, some kind of a possibility of testing-retesting effects because it is possible that

there there is a fatigue happening, there is carryover happening or the participant has, you know, judged the

hypothesis and starts performing in a reactive manner. So, all of those possibilities are obviously there.

Other ways to reduce, you know, these kinds of, you know, problems with validity is trying to use matched group

designs. Now, what will happen here? An [snorts] alternative approach to measuring the dependent variable more

than once is to basically collect either before or after the experiment a different measure that that expected to

influence the dependent measure. So, what we are basically saying, let's say in this same memory experiment, if we

have concern about similar memory measures being taken twice, the experimenter might choose to measure

participants intelligence based on an IQ test with the assumption that the IQ will be correlated with memory. And

then, when they basically control for IQ, they'll they'll have reduced the between

person variance. A very uh important sort of a or another uh real-life example of this could be, say, for

example, if you're planning to test your participant on uh some kind of uh

inhibitory control kind of study or an executive function task switching kind of study. Now, in these kind of studies,

working memory working memory span is an important factor. Now, if you want to be sure that uh you know,

uh the results that you're getting from your experiment are not because of working memory, it's a very good idea it

it is often done is that you measure the participant's working memory span and you select people within the same band

of working memory span to participate in the experiment so that you're sure that your group is matched on that criteria.

In that sense, there is nothing that is going to affect your experimental results other than your IV manipulation.

Now, this kind of a design is known as the matched group design because participants here are measured on the

variable of interest, for example, uh IQ or working memory span before even the experiment begins. And after that, they

are assigned randomly on the basis of their scores on that variable. So, you can basically say, "Okay, now I have a

group of, let's say, 60 participants, all which are matched on this critical variable which I was thinking could have

had an effect. Now, I can sort of randomly assign them to all the conditions and there is uh you know,

uh practical initial equivalence created in my experiment. Okay? So, uh during assignment of participants to conditions

in this memory experiment, two individuals with the highest IQs would be randomly assigned to the

sentence creation condition and the no instruction condition, respectively. Then, the two participants on next on

the IQ scale can be awarded can be again awarded to condition uh sentence creation condition and no

that IQ in this case or working memory in our uh you know, task switching case are not

actually having an effect on the uh you know, independent variable manipulation because we have matched for the same

levels of IQ in both of our samples. We have matched for the same level of uh working memory span in both of our

samples. And in this sense, participants can be matched on a range of variables. You

know, they can you can match them on, say, for example, you can match them on demographic variables such as age and

SES and etc. You can match them on, say, for example, uh their performance on another memory task, uh IQ task,

uh their uh you know, other characteristics and so on. But, what will happen is it'll be difficult to

find participants because when you start matching, the number of participants that you can actually use

potentially from the population will start reducing. So, that is why you will see a lot of times matched group designs

are are you know, difficult to use and they are uh you know, not very popular within experimental research.

So, you can see here is an example of a matched group design. So, a matching variable is

IQ here. Uh so, highest IQ pair and the next highest IQ pair and the lowest IQ pair,

both are occurring in both conditions. Okay? So, in that sense, IQ is controlled for and uh it cannot be uh it

cannot cause uh or it cannot be a source of systematic uh you know, variance in the independent variable along with the

independent variable and it will not cause the change in the dependent variable and will not basically create a

threat to internal validity in the conclusions that one has to make. So, that's what I was saying. It must be

noted that uh you know, uh use of matched group designs is not normally necessary in an experimental in

experimental research as in most cases we assume that random assignment will be sufficient to ensure that there are no

differences between the experimental conditions. All right. Now, matching is and it is

typically used and it is used only in those cases when the experimenter feels that it is absolutely necessary to

attempt to reduce variability among participants on certain specific criteria. Say, for example, if you are

doing an a task switching kind of study, you might want to go out and actually control for uh working memory capacity.

You might want to go out and control for exposure to video games and some other factors which you will have theoretical

reasons for assuming that they can have an impact on your uh you know, IV manipulation and consequently your

dependent variable measurement. Now, another way, so you have the before-after design, you have the uh you

know, matched group design. You can also look for uh you know, controlling the conditions uh you know, most uh

typically by standardizing the conditions. Say, for example, you can have that uh the entire experimental

protocol is controlled, it is followed to the T. It is basically uh for each participant matched in in all respects.

So, the same room is used, so the same instructions are given. The independent variables are treated in the same way uh

so that the only thing that is varying and even the way in which the IV manipulation is done is exactly

identical for all participants. Now, this is this will also reduce the any chances of

uh you know, extraneous or confound variables creeping into your experiment and reducing the internal validity of

your uh results. All right. So, the idea basically is to hold constant every possible variable that could potentially

influence the dependent measure. And [snorts] for this, a lot of times what happens is the researcher will contact

all the participants in all of the experimental conditions in the same manner, provides the exact same consent

form, exact same instructions, ensures interaction happens in the same way. It happens uh exactly identically across

all participants. Say, for example, uh sometimes uh you know, when we are using verbal instructions, you uh describe

them much in detail to one participant but very superficially to other, that can also cause a difference in your

results. So, typically what has happened is the set of instructions is printed and it is handed over to the participant

uh so that there is no variability in the delivery of the instructions which can cause changes in the dependent

variable measurement. So, as the experiment proceeds, the activities of the groups and how the

experiment is treating the two groups remains exactly the same and this can also reduce uh to a large extent any

variability that can creep in in the experiment. So, that is all for uh this lecture. I

will continue this discussion on uh validity in the next lecture again. Thank you.