Designing Reaction Time Experiments in Cognitive Psychology

Hello and welcome to the course basics of experimental design for cognitive psychology. I am Dr. Ark Whmer from the

department of cognitive science at ID Kpur. This is the week seven of the course and we are discussing reaction

time and we are discussing reaction time studies. Now let's talk about the process of

creating a reaction time experiment. Now uh I have talked to you about experimental design in much detail. What

I will do in this lecture is we will try and apply whatever we have learned so far into creating a reaction time

experiment. All right. So designing and conducting reaction time research requires knowledge of some of the very

basic concepts and general principles things that we have already talked about so far in this course. Some of these

concepts and principles are related to research design in general such as the process of research types of variables,

manipulations, control and so on. Others are slightly more specific to creating reaction time experiments such as

counterbalancing, stimulus onset, asynchrony and so on. So we will sort of uh revise some of the things we've

learned earlier and we will add some of the things that are pertinent to reaction times experiments.

First and the for most important thing here is to identify a narrow enough research question. All right. So all

research as you know all research begins with a problem that can be formulated as a research question. It may come from

your own uh experience or it may come from existing literature. For example, let's say one has a question that is

second language learning affected by the first language. Say for example whether I am or how successful I am in learning

a second language let us say English is it dependent on what my first language is. If my first language is Dutch or

French or German will I be more successful or if my first language is Hindi or Tamil or Telugu I will be more

so based on what my first language is does it determine my success and the rate of learning of my second language?

This could be one question also. There could be another question. For example, will differences in

semantic structures in L1 and L2 influence the learning of L2 vocabulary? So, the way the uh you know the words

are rec are organized semantically for my first language because those are the ones that I have learned first. Uh is

that different to how I learned the words of my second language? So uh is the difference in this semantic

structure how the network is uh arranged uh in my first language versus in my second language does that have an effect

on my success and my time and overall uh effectiveness of learning the second language vocabulary. You can have

another question again we're basically talking about the same uh paradigm here is uh when a semantic distinction is

made in L2 but not in L1 can adult L2 learners develop these new L2 specific semantic distinctions for example if

there is there are words which are having very similar meaning in L1 but have very different meanings in L2 can

we sort of learn these rules specific to my L2 vocabulary again uh these are just Some questions I'm sort of uh drawing

from uh you know bilingualism research but you can actually have any number of questions. You can basically say uh how

much time does it take to recognize a face? How much time does it take to recognize a happy face versus a sad

face? How much time does it uh uh you know uh take to recognize an emotional expression on a female face or a male

face. any number of questions you can ask and you have to create your experiment such that your experiment uh

your calculation of reaction times using either additive or subtractive methods that we've discussed in the previous two

lectures will allow you to isolate the step that you are most interested in. For example, you can give two neutral

male faces and you can give two neutral uh emotional male faces. And then if you compare the reaction times between these

two, you may be able to isolate the time taken in recognizing emotional uh expressions because otherwise the both

the faces were generic male faces. Now the examples that we were just talking about each subsequent question

is more specific. It is more sharper and narrower than the previous one. Now if your question is very general, you will

most likely feel that you do not know where to begin designing the experiment. So you will not know how do I start? You

know when when you have a very generic question is my L2 vocabulary going to be better than my L1 vocabulary. Now that's

a broad question because you don't know what aspect of my L2 vocabulary I'm going to ask this question about. In

contrast, if you look at the third question, the third one already knows what structure you want to focus on. You

want to focus on semantic distinctions, meaning differences in made in L2, but you're not interested in meaning

differences made in L1 or your first language. This points to the direction for an adequate method that is the one

that can be used to assess the development of new semantic distinctions in the second language as compared to in

the first language. Similar example that I took, say for example, you have a uh you know, you're uh presenting all male

faces. uh half of them are uh emotional, half of them are not emotional. When you compare the reaction times, you should

broadly get the time taken to isolate an emotional expression in a generic male face.

Now, a good research question, you know, the one that is most suited to reaction time research would concern a problem

that you can relate to and feel passionate about. This is a general comment about how do we create our

research studies. Now when you're working on such a topic you bring all your own experiences and perspectives to

the problem and you're all obviously more likely to work with a purpose personal connection. So for example a

lot of times I see students who are uh you know conducting research they're designing their own projects but they're

not really in that topic. They're not it's it's not their topic. It's probably the supervisor's topic. It's probably

the instructor's topic and they're just executing that like zombies. Now that is not the ideal situation to work in. If

you are working on a topic, it should that topic should incite questions in your head. The topic should make you

curious. The topic should make you sort of get up in the get up get out of the bed uh every morning and work towards

answering those questions. Okay. If you have those questions in mind, if you sort of uh you know are uh impassioned

by that particular topic, only then your research is going to be of top quality. If you are just working like clerk and

executing uh the research ideas of somebody else, it's not a great place to be. Obviously and it depends on what

stage of your career you are. For example, when you are doing your undergraduate or your postgraduate

research uh you know master's research basically then obviously you you still have to learn you still have to learn

the methods you still have to learn how to attack a question and in that sense obviously working with whatever the

supervisor is assigning you how is he taking you from the different stages of development of research design is

important but at some point in time let's say when you are doing your own PhD or when you do pursuing research

after that when you have a sense of the process is it is equally important to have your own questions. It is equally

important to be uh curious yourself about the topic of research that you are selecting. All right. So a good research

question would be something that deals with a basic and fundamental issue of the field. These issues will be at core

of the field and are considered more important. Basically you want to talk about how the theory is going to be

developed. How are things basically going to move on in a given field. Say for example, if you consider concerned

about litality or language switching or visual word recognition or say for example perception of something so on

individual students may have different research interests and perspectives and they may be passionate about different

aspects of say for example second language learning. Uh but consider uh find out a way there is to link your

passion the question that uh you know incites your attention to a basic and thus important issue of the field. So

typically what you have to do is that's why conducting a very good literature search is important because uh you read

enough on a part on a particular topic and then the gaps in in research on that particular topic become apparent to you.

Some of those gaps might feel interesting to you. They might make you curious and the a good choice of a topic

is something that is making you curious, something that is inciting that uh you know aspect of curiosity within you and

then you're using that curiosity to basically work out a research question that you want to uh you know take

further. Now even before you sort of create your reaction time study and you uh you know

run away with it, it's always a good idea to conduct some kind of a preliminary research, some kind of a

pilot. So once you have a research question in mind, you obviously do not dive right into designing the study.

Instead would like to give yourself some time to read about and think about the topic. Understand the various aspects of

the topic. Understand the kind of work that is already done. The kind of questions uh you know people before you

have asked and what kind of answers they found, what kind of methods they have used, which methods have been successful

in answering some questions, which methods have not yielded very good results. So knowing that conducting this

preliminary inquiry is extremely important. Whatever the topic uh is it is most likely that there will be some

published work already. So uh you know I am not a big fan of novelty because a lot of uh you know at any point in time

you come up with a question you'll see that something similar if not the exact same thing has been done earlier. And it

is important that we do not reinvent the wheel every time every time we create a project. It is important that we sort of

know what is done in that area and then we know exactly what our unique addition is going to be. Sometimes people uh you

know conduct researches such that that oh I'm the first one pursuing this topic. But mostly that is not the case.

Mostly whatever topic you are interested in whatever uh you know topic or research gap that you have come across

it is probably done earlier as well. people have worked maybe not on that specific aspect that encou that excites

you but probably some other uh uh aspect of that same topic. So what you want to do is uh you want to go back and

thoroughly revisit the literature and basically understand what aspects of this topic have already been studied

what have been the methods used what is the extent result and finding from that particular topic. So familiarizing

oneself with these studies is important in several ways. It gives you a sense of what has been done so far in that topic

so that you make well-informed decisions regarding whether to pursue the topic. What is there to you know what is new to

find out what are the gaps? What kind of methods have used uh fruitful have been proven fruitful? what kind of methods

have proven futile those kinds of decisions and how would you like to approach a given topic say for example a

lot of times we think okay we'll use method A to uh investigate topic uh B but uh literature has shown that uh you

know in the past decade two decades three decades of research this method has not been able to uncover specific

aspects of that topic so you want to sort of make that connection you want to find out the best method to investigate

the uh right kind of topic Also getting to know the literature will help you develop a broader perspective

on the issue. It will basically help you understand what is going on, understand the theoretical significance underlying

the topic uh in a better sense. It'll give you a sense of the theoretical approaches from which the topic can be

studied. What is the question? Say for example, in second language uh learning or in language switching, what are the

questions that are important? Is proficiency an actually important variable? Is immersion a variable? uh is

uh you know say for example the kind of task that you're doing have they they have some specific effects. So unless

you've read uh extensively on a given topic, you will not be able to figure out which aspect of the topic uh you

want to pick up and you want you are most curious about. uh a lot of times if you're not aware of the literature and

you write uh you do your own research you write a paper and you send it to a journal journal uh you know says that

you know the reviewers send it back and they say oh you have not read this study this study this study uh in this study

the exact same thing that you've done now has been done and uh what is the new contribution that you are making to this

research so to avoid getting into that kind of a scenario it is always considered uh very important that you

have done your preliminary research very well. You are well aware of the theoretical background in a given topic

and you know which questions are going to be able to uh make new contributions. The topic may not be new. The method may

not be new. But because uh you have studied well, you know exactly what is the new aspect, what is the new finding

that your particular uh experimental uh study is going to reveal and in that sense how important is what you're

doing. So a good reaction uh time study is not only about producing valid data and discovering a new finding. It uses a

concrete reaction time finding uh to make a point about abstract theoretical issues related to mental processes. So

remember we were talking about the coast and study uh earlier they were trying to sort of uh re-examine the uh you know

the explanations offered by David Green that okay whether it is separation of language uh first language that is

leading to asymmetric switch cost responses and so on. So they basically created the experiment in such a way

that the experiments offered a way to uh answer the theoretical questions that were being uh you know raised about why

does it take more time to name in an L2 or name in an L1 after you have just named in a different language.

Third and uh equally important thing is that by reading others work the reading the extent work or literature in in a

given topic one becomes familiar with the methods that have been used in examining the particular issue and uh

the methods that might be potentially useful. See in science everything is as we've seen so many uh times is well

documented well recorded and it is described in the journal articles. It is there for two purposes. First is that

you basically you know allow for replication but you also allow others to learn from your mistakes. If you've sort

of made a particular mistake and you've documented it honestly and it is uh you know out there in publication somebody

else who's designing an experiment on a similar topic with a similar sort of a question will know what not to do. Okay.

or we'll know that okay with this specific manipulation these are the results I'm expected to get and if I do

something else uh if I create a different manipulation I might be able to uncover or I might be able to

illuminate a different aspect of this particular problem. Now what are the steps in designing a

reaction time experiment? This is something that is extremely important. Now there are a lot of decisions one has

to take in what kind of uh you know uh reaction time experiment you have to decide. Say for example, who are your uh

participants? I've talked about this earlier, but this is more in terms of reaction times. Who are you going to

test? Whether it is children, whether it is adults, whether it is older population. What kind of task you're

going to use? Are you going to use a naming task or lexical decision task, animation judgment task or go no-go

task? What kind of task you're using? And how is that task going to answer the research question that you have raised?

What kind of stimula you are going to use? So in language studies the most important thing and I'm talking about

language studies because that's uh you know partly my research area in language studies v visual word recognition and so

on the uh kind of stimula that you're using the characteristics of the stimula are extremely important words the way

people react to words is affected a lot by the frequencies of those words. So if you are creating a stimulus set and you

are interested in something let's say manipulating graphimic complexity you would want that the words vary only in

complexity but not in word frequencies. So you want to match for frequencies and vary for graphimic complexity and that

is what you want to reflect in your reaction times. How to present the stimulator? You want

to present a similarly at the center of the screen. If you're interested in literalized presentation, whether you

want to present it uh you know 3° to the left or 3° to the right or the center and so on. And then how do you collect

your data? How do you analyze your data? What statistics do you want to use? What kind of say for example a lot of people

nowadays are moving away from the frequentist analysis to the bayan analysis. What is it that you want to

do? All of these things have to be thought upon, have to be pondered, have to be uh evaluated in terms of past

work. And basically you decide all of all of these things before even you start with your reaction time

experiments. Now the process of designing a reaction time study leads not only to an action

plan based on whatever we just discussed and materials needed for data collection uh but it also leads to the formulation

of a new set of research questions new hypothesis or predictions in the context of a specific task and specific stimula.

So let's say if I'm using high frequency word or low frequency word in a naming task versus in a lexical decision task

or say for example I want to uh you know manipulate uh a graphic complexity between words. Am I using a same

different task or am I using a lexical decision task or am I using a naming task and how is my task going to

elucidate uh the aspect of uh you know manipulation that I have done with my stimuli. So all of this really needs to

be uh etched out very clearly. So one may consider uh the process of designing a reaction time experiment as

one when a topic oriented research question becomes a design specific research question. You start from

broadly a topic but you basically see how does that topic or how does a specific aspect of that topic get

reflected in your research design how is your research design going to enable collecting data about that specific

issue. Let's say you want to uh look at the effect of frequency in naming and uh you basically so what you'll do is

you'll basically have words which are varying on frequency and then you'll compare uh the naming times uh for high

frequency words versus low frequency words on all other aspects length etc graphic complexity etc. you've mashed

the stimuli. You're only interested in the effects of frequency and that is how your reaction time study should be

designed that it should lead you to uh you know getting to the effects of frequency and not any other variable. So

the former basically states the issue under investigation. So the topic and the latter the design specific question

basically indicates what to expect is going to happen in the specific experimental design that you have

chosen. So if you've chosen a naming task, you can expect uh differences in naming times. If you've chosen lexical

decision task, you will expect a decision in le a difference in lexical decisions made.

Now uh just to sort of put this idea into practice, let us consider a specific topic. Let's talk about the

representation and uh processing of formulaic expressions, broad expressions by non-native speakers if a language.

Now formalic expressions basically just to sort of give you a hint are phrases that represent a single semantic unit

and they occur with a high frequency such as as soon as as long as things like that by the way on the other hand

this happen. So these are formulaic expression that are con basically conveying a single idea. Now a research

question uh related to this topic would be are formulas represented holistically as a single unit in the mind of L2

learners or so this is a broad topic related question. This question can be explored in a grammaticality judgment

task. So for example in this task what we can do is we present a set of grammatical and ungrammatical phrases on

the other hand on the below idea something like that to participants and ask them to decide whether these are

grammatical or ungrammatical. Now see topic is how do we sort of store formulaic expressions but in the

research design given the task choice we will know how this formulaic expression manifests in the specific task that we

are using. So we have grammatical phrases we divide the grammatical phrases into two

categories formulaic expressions and nonformmulaic expressions. So as soon as on the other hand and non-formmulatic

expressions will be as slow as on the other page which are not uh you know canonical formulaic places they just

resemble them. The two sets have to be mashed for other characteristics. So you match them for length you match them for

lexical frequency. So that the only thing that is changing is one is a formulaic expression and the other is

not and one is grammatical and the other is not. The assumption underlining assumption in creating this task would

be that in performing grammaticality judgment participants will have to analyze the phrases in order to decide

if they are more or less grammatical. However, if the formalic expressions are treated are stored holistically as a

single unit more or less like lexical units then a positive response can be generated once it is located in the lex.

Just as you search a word, you can search for a single unit which says as soon as or a single unit which says on

the other hand by the way. So grammatical analysis can thus be bypassed and will basically not manifest

in the time taken. This should lead to faster reaction times for formulaic expression because

it is just checking of one entity as opposed to the combination of entities which will have to happen in the

nonformalike case. Let's say you have topic oriented research question. Just sort of

revisiting this. Are formulas represented holistically among non native speakers. You have design

specific questions. So do non-native speakers respond to formulas faster than non-formmalic expressions in a

grammaticality judgment task. And then you have a prediction just as we said. If formulaic expressions are represented

holistically, individuals will respond faster to formulic expressions than nonformmlike expressions if they are

matched for other lexical properties like frequency, length, etc. Now that is basically how you go from a

topic oriented question to a basic design oriented question and that is how you will be able to uh interpret your

reaction times better. All right. Now uh again just revisiting what are the variables uh here. So types of variables

you know that there is an independent variable and the dependent variable and independent variable is a property of

the qu property or quality that has the potential effect to affect the other variable which is your dependent

variable which you are measuring and the independent variable is the one that you are manipulating. Now independent

variable is as I said the one that you usually interesting in knowing more about and thus you design the experiment

to actually study the effect of the independent variable. effect of frequency or effect of whether the

expression is formulaic or not or effect of say for example whether an item has one feature or multiple features in a

visual search kind of setup or not. Now for example we might want to know uh how second language proficiency may affect

sentence processing strategies. So we can divide people into three proficiency groups uh low, medium and high sentence

processing strategies. Okay. How L2 proficiency. So what basically what we want to do is we want to vary

proficiency and we want to see its effect on sentence processing strategy. What how do we do it? We basically get a

proficiency test and then we observe how these participants perform in processing sentences.

Just taking this further in bilingualism research with reaction times three kinds of independent variables are most

important. First is participant related variables. So native speakers versus non-native speakers can be compared on a

participant's age the L1 background the age of onset so age of acquisition of the second language sequence of language

learning did he learn first langu did he learn uh you know say for example three languages L1 first L2 second L3 so in in

that order length of residence say for example in a given place how long that the person has been an immigrant for and

so on then there are stimulus materials for example what kind of words you are using. So uh word frequency is is very

important. Word length is important. The familiarity, concretess. So the characteristics of the word this is what

I was telling you earlier that in language studies stimulus characteristics are extremely important.

They have to be controlled for and matched and so on. And that is why it is sometimes hardest to create stimula in a

language study. Also procedure variables. For example, uh if you're presenting the stimulus in audiary

modality or in visual modality, that also has a bearing on how the participant is going to process that

interval between stimulate time. So for example, remember we've uh talked about in a particular uh in a previous

lectures that when in a picture word interference task when the picture is presented versus when the word is

presented uh if they're presented early or if they're presented late will basically govern the interaction between

the picture and the word whether it is a semantic interaction or is a phological interaction. So participant may be asked

to perform a task in L1 or L2 and things like that. Then there are dependent variables. Uh

uh dependent variable is what you are actually measuring. So whose variation is going to get potentially affected by

the independent variable manipulation and that is measured in order to affect or the lack of the effect of the

independent variable. We've talked about extraneous variables earlier. So extraneous variable is a variable that

may or may not affect the dependent variable and that is not manipulated as an independent variable in a study. So

something that may potentially affect your dependent variable but you're not manipulating it. So you're not sort of

uh expecting its effects to show. For example, a set of concrete and abstract words were selected for a study

to compare the concreteness effect among native speakers and non-native speakers. But uh if these words were not

controlled for frequency, frequency may play a spoil sport and it may basically you know show up in your results even

though you did not intend frequency to play a part. So you can have an extraneous variable. Uh it will become a

confounding variable when it is not adequately controlled or when it is not adequately matched. And an extraneous

variable can become a control variable if you have already thought about it. If you have foreseen that frequency can

have an effect on uh you know concreteness judgments and you have created a stimulus list that is matched

for frequency then it becomes a control variable. So it is very important to know foresee your extraneous variables

and find a way to control them. Then there are moderator variables also that play a part. Say for example, it is

one that modifies and changes the relationship between the IV and DV. It will not affect the DV directly but it

affects when an IV it sort of interplays uh you know there. Say for example if non-native speakers are found to respond

to concrete words faster than matched abstract words but this would happen only in high proficiency uh L2 learners

then proficiency would become a moderator. Say for example if you're finding a particular phenomena but it is

only in one particular kind of uh you know participants who are systematically different from others then you know that

oh that variable is also playing a part. Now obviously there are between subjects and within subjects variables we've

talked about this. So this distinction applies to independent variables only and it is very important in uh creating

reaction time experiments. So for example in any reaction time experiment in independent variable has let's say

two or more uh levels. So for example the concreteness level variable has two levels uh concrete and abstract. Now

when participants are divided into three levels of on the basis of proficiency low medium and high then there is this

another independent variable which has three levels. So now the design will basically become 2 + 3. The distinction

between within and part between participant variables also is important. Now remember concreteness how it is

varying among the stimulus is within subjects variable because all participants will look at both abstract

and concrete words. But you can decide and you can basically have your low proficiency, medium proficiency and high

proficiency participants in different groups and that therefore becomes the between participants variable here.

So that's basically what I was saying. So in the concretness effect example, the same group of participants respond

to both concrete and abstract words. So concretness is within participant variable. But if two separate group of

participants are asked to perform a task, one responding concrete words, other responding to abstract words, then

it will become a between participants variable. But here proficiency is a better choice for a between participants

variable. Now types of design also again something that we've learned earlier. So if you

want to follow a between participants design, so you keep proficiency as a between subjects variable and you have

three groups low proficiency participants doing the concreteness task, medium proficiency participants

and high proficiency participants. You can have them compare. Uh you can have a within participant design. Say for

example, you have the same participants basically uh you know uh doing the all three tasks. You can have a mix design

one between participant one between you know within participant variable can be there. So the current one the way we are

looking at it has one within subjects variable which is concreteness and one between subjects variable which is the

proficiency. So you can basically create your task according to those lines. Now returning to this uh if the purpose is

to find out whether the effect can be observed among non-native speakers uh then the study will have a single IV of

concreteness which has two levels concrete abstract. If the same group of non-native speakers are asked to respond

to both concrete and abstract words, then the study has a within participant uh within subject or within participant

independent variable and thus it is following a within participants design. However, if the purpose were to study

the role of language proficiency in concreteness by you know in concretness effects by comparing the performance of

high and low proficiency L2 speakers then the study will have two independent variables. One is within subjects which

is concreteness. The other is proficiency which can be a between subjects variable. So this is basically

how the design goes. Now how do you not notify a design? This one will basically become say for

example a level of independent variable it will be called a factor. You know that uh one variable is concreteness.

Two uh levels concrete abstract. So two the other level is proficiency. Suppose it has three three levels low, medium

and high. So it becomes three. So the notation becomes 2 + 3 design. Now the design of study you can be make

it more complex when more independent variables are used. For example, some research has shown that native speakers

of English recognize words with a regular spelling sound relationship faster than those with a irregular

spelling sound relationship. So regular is PC packed. Irregular is P I N T. You're not calling it pint. It is

actually referred as pint. So uh when there is a irregular spelling to sound relationship people take more time to uh

recognize this. If there is a regular spelling sound relationship people do it a bit faster. So this is often consider

taken as evidence for the involvement of phenology. Now we can conduct a study to find out whether English as second

language learners with an alphabetic and non-alphabetic L1 should be shown different sensitivity to spelling sound

regularity. So you can have people who have an alphabetic first language and you can have people uh non-alphabetic

first language and you basically look at how they are going to affect get affected by this spelling sound uh

regularity. L2 proficiency is also very important how well they are in their how good they are in their second language

for this kind of sensitivity and thus you can manipulate the L2 proficiency low medium high uh from the learners

from each background. So you can have people let's say who have a non-alphabetic script let's say Hindi

English speakers and you have a uh group uh with alphabetic first language. So you have a Dutch English speaker. Now

within Dutch English speaker uh you can have different levels of English proficiency. Within Hindi English

speaker you can have different levels of uh English proficiency. So L2 proficiency is changed. The background

first language is alphabetic and non-alphabetic in those cases. So this study will have a 2 + 2 + 3 mix

design where have three variables. First is the regularity, regular words, non-regular words. Second is L1

background, alphabetic, non-alphabetic. So these two have two levels. And the third is proficiency, high, medium, low.

So it has three. So basically now you have a design which has 2 + 2 + 3. You have 2 4 3 12 conditions in these

designs. So that is basically how complicated this design has become. This is basically how uh you know the

factorial will work. You have alphabetic high intermediate low proficiency. You have logoraphic let's say these are

Chinese uh speakers high intermediate uh logoraphic and you have uh basically regular words and irregular words

presented. So this is basically the design. Now different kinds of designs obviously

they have their own pros and cons. In principles when uh both within subjects and between subjects designs are

possible we prefer the within subjects design. I uh hope you remember the uh previous discussions because within

subjects design provide better control. They provide uh you know us with lower intra interubject variability and

therefore if it is possible to have a variable or if it is possible to have a design which is within subjects we go

and we prefer with the within subjects design more than we prefer the between subjects design.

Now parts of the reaction time experiment. Uh it's basically very simple to set up which task you're going

to use, what materials you will use, how will you present the materials and how will you measure the artist. So this is

basically the four decisions that you've made. Choosing a task, what kind of task you're going to choose. In many

situations, you will find that an adequate task can be identified by looking into studies on the same topic.

For example, if you find that lexical decision is used in many studies exploring the processing of complex

words or that self-paced reading is used in many studies of sentence processing, it will be usually safe to assume that

you will you can choose one of the two task if you are working on that same topic. Under other scenarios where there

is no precedent to follow, you can deise your own task with uh you know the principles of control etc that we've

talked about in mind. This is when knowledge, creativity and judgment always start uh to play a role. For

example, look at this experiment created by Zan and colleagues. Zan and colleagues uh you know they had this

purpose to explore whether mental imageries are viable forms of mental representations of meaning in language

processing. So the question they have is how do we know whether mental imagery actually uh has been created as a result

of reading a sentence or not. So that is basically what they want to check. Now to study this issu is issue they use a

sentence to picture matching task. So there will be a sentence and there'll be a picture and they have to say whether

it ma whether the sentence matches the picture or not. They use sentences like John hammered the nail into the floor or

John hammered the nail into the ceiling in combination with pictures of a nail in its vertical position and nail in its

uh horizontal position. So if you are hammering the nail into the wall, the nail should be horizontal. If hammering

the nail into the floor or the ceiling, the nail would be vertical. Now, if mental imagery is actually created as a

result of reading such a sentence, a nail in its vertical orientation would be created in a participant's mind after

reading the sentence wall uh which would lead to a faster response to a picture of nail in its vert and uh you know to a

picture of nail in its vertical orientation than its horizontal orientation. Depending on what kind of

mental imagery is created and whether it is congruent to the picture that is presented, the participants can be

expected to be faster or slower. What did really happen? He hammered the nail into the wall uh with the

horizontal representation of the nail was faster as compared to he hammered the nail into the floor uh with the

vertical orientation. So when there is a match in mental imagery and the orientation of the nail, participants

were faster. When there was a match between the orient mismatch between mental imagery and the orientation of

the nail, the participants were slower. A similar version was created. The ranger saw the eagle in the sky with

wings uh uh you know uh open as compared to the ranger saw the eagle in its nest with wings closed. So in its nest with

wings closed is faster. In its nest with wings open is slower. In the sky with wings open is faster. In the sky with

wings closed is slower. So when the mental imagery is matching the presented picture, participants will be faster.

When the mental imagery is not matching with the picture presented, then it will be slower.

So consistent with their hypothesis, the participants responded to pictures faster when they depicted an object

orientation implied by their mental image. So here very easily and very creatively you can actually see what

kind of mental imagery the people were uh actually getting. This is basically the result of the

orientation and the shape experiment. The one I was just discussing also understanding what your task involves.

It is also important what is it what are the mental processes that my task will uh you know lead to. Remember we were

talking about Sternberg and we were talking about uh da they were actually wondering they were working to what kind

of task what kind of mental operations each task will uh invoke. So there are two levels of understanding a task. One

is the technical level and the other is the theoretical level. At the technical level, basically one needs to know the

methodological conventions that go in a task. For example, many RT tasks have been around for a while and have been

used in many times in research. As a result, some methodological conventions have been established for their

implementation. Now, you know, say for example, when you're doing a lexical decision task, how do you present the

stimulus? How do you uh record the responses? Go task or stroke task, they are established very well and people

know how to run them. the Simon task for example or the flanker task for example. Such conventions may apply to all

aspects of a task ranging from the kind of test materials usually used to when and where to display a stimulus and from

when to start the timer and so on. You know if it is a well- established task, you know the entire details of the

protocol. At a theoretical level, understanding a task basically means to have a theory of

the mental function. Theory of what is implied in that task. So for example, this theory will include at least two

things. An assumption of what mental processes are going in. For example, I described that in a lexical decision

task, you are measuring the time to perform lexical access and the time to perform lexical access will get affected

by frequency. It will get affected by other uh you know variables the properties of the words for which you

have to perform lexical access. Second is in that particular experimental design that you are using the particular

test materials and and basically how the task is being done a statement of how the results obtained within a given

method should be interpreted. So for example when you're doing uh you know a stroke task how is uh you know what kind

of timing is there how is word reading manifesting in a stroke task as compared to a non-stroke reading of words

also identifying participants is is very important whom to test how many participants there's no typical rigid

rule but general number seems to be around 20 to 40 again there are a lot of power calculations and things like that

that nowadays people use to decide how how many participants are actually required. So you can basically calculate

your effect size after a pilot and use that effect size into softwares like GP power which will tell you exactly how

many participants you would need to sort of uh uncover the effect that you are after.

The actual number of participants basically will depend on two factors. One is whether there is a participant

related independent variable and how many levels does that variable have. The second is how many times you are

presenting the list. So for example multiple lists are typically involved due to the need for counterbalancing you

know in the factorial design and so on. So for example if the data for one such list requires 10 participants and there

are more list then you'll have those multiples of 10 to be used as participants. So the number of

participants required also uh it depends on the effect under investigation. A robust effect size will require fewer

participants. A weak effect size will require more participants. Again things that we have discovered we have uh done

earlier but we're sort of revisiting them. Two advantages of having a larger participant pool it will give you

greater power. It will give you better possibilities of post talk analysis that because each comparison will be

adequately powered. Other questions what kind of information that you want to say for example if you

think that your task is going to get affected by participant level variables then you might want to sort of have uh

you know participants data. Say for example, you might want to know their proficiency. You might want to know

their age of acquisition. You might want to know a bunch of these things. Test materials. How do you develop? What

kind of stimula you use? What are the uh you know factors on which your stimula would vary? Let's say they vary on

frequency. Let's vary on length. How would you sort of go on and develop that? Say for example, a simple lexical

decision experiment has these kinds of instructions. Say for example, welcome to the lexical decision experiment. In

this experiment, you will see a letter string such as flower or printy. You have to it will be presented on the

computer monitor. Your task is to decide whether the letter string is a valid English word or not. Press the yes

button if you think it is a word. Press the no button if it is not a word and other things. Say for example, you know,

calming the participant when they enter the room and so on. So this is again something that you will develop.

Stimulus. A reaction time study usually consists of three types of items. practice items, critical items on which

the manipulation is done and filler items. Okay, so practice items are typically needed to familiarize the

participant with the task. It's just let's say around 8 to 16 practice trials are typically used. Basically around 20%

of your overall experimental block. Uh more complicated the task is you might need more practice trials. Critical

items are where you are actually manipulating. So the ones that are actually differing in let's say if you

are interested in frequency filler items which are not differing in frequency but are sort of just there to you know make

the overall experiment longer. Finally you have to debrief and once you've done the thing you have to talk

to your participants you have to debrief them what happened in the experiment and basically uh it's a very good idea by

the way to take exit interviews. What did the participant go through? what are the strategies he or she used to answer

your questions or to basically make it responsive and that will basically give you a very good idea of how things uh

went. Okay, so this is basically a broad uh you know outline of the experiment uh how do you create an uh you know uh uh

reaction time experiment consent form instruction script of the experiment questionnaire you can have tests or

interviews or you cannot uh instructions then you have practice trial you have critical trial in this and then you have

the experimental item. You basically this is the method that you'll broadly follow in uh you know creating a

reaction time experiment. Stimulus presentations for example uh how many uh you know for how long at what time the

stimula will get presented. Most RT experiments are conducted nowadays with computers to ensure very accurate and a

very precisely timed display of the stimulus and a very precise measurement of reaction times elements of a trial.

what kind of uh you know say for example when does the blank chain come when does the fixation come when does the mass

come and so on. So all of that basically is there. Typically there are two uh uh two very important elements fixation

crosses or say for example sometimes you want to sort of uh you know uh have your participants focus you can present a

fixation cross or you can basically give them a two ready to be ready tone a beep sound or something so that they can

start focusing. Some common sort of procedural things that we have to consider is modality stimulus can be

presented only visually only auditory or in both modalities. uh how long you're presenting the target uh uh you know uh

for say 200 millconds, 50 millconds, 500 millconds and so on. Uh then you have how how well you have timed the

responses whether the participant has to so generally the instruction is to respond as quickly as possible. But you

can basically take say for example within 1200 milliseconds within 1500 milliseconds within 2,500 millconds you

can have that you may or may not want to give feedback. You may may not want to have a self-faced or continued mode of

presentation. Say for example in self-paced reading task the participant has to uh press a space bar read the

word then press the space bar and in other presentation it could be just the words are coming staying there for some

time and then other words are coming. Single list presentation blog representation interstimulus interval

stimulus on is incre these are all decisions that you'll have to make when you're deciding the whole chronology of

your task. So inter stimulus interval if there are two or more stimulus stimulus that you're using what is the timing

between them stimulus onset asynchrony is the interval between the onset of one stimulus and the onset of the other

stimulus. So both of these are sometimes varied in order to uh study this. For example here you can see you know the

elements of a trial a trial with a simple uh stimulus uh there's a fixation cross and there's a trial with a

compound stimulus. So you have a prime and you have a uh target here. So what is the diff difference of time between

these two? That is also what you would want to vary. Counterbalancing as I said it's typical Latin square design. List a

has these sentences and then these and these. And you can see the sentences have been varied. So uh for list D

sentences 1 to 10 come last. For list A, sentences 1 to 10 come first. Again how do you counterbalance? How do you

present the list and these kinds of things. Randomization as I said is very

important. two broad methods are used. Full randomization by a computer program or sometimes you have a pseudo

randomization where you want that each participant goes through at least one condition uh one version of that uh you

know condition and that's why sometimes you sort of create a pseudo randomization by uh arranging trials

just by your own. So that is called pseudo randomization. Timing of participants reaction time is usually

measured from the onset of the stimulus to the time when the response is proided. I've already talked about this.

And then how do you uh you know treat your data? How do you go ahead with analysis? Uh incorrect responses.

Typically the general practice is to exclude in uh you know reaction times for incorrect trials. Also sometimes the

experimentter has to take call is to exclude or include a participant with a particular error rate. So sometimes say

for example in lexical decision studies typically we exclude participants who have more than 10 or 15% error rate. But

again depending on the kind of research question you're asking you can choose to use or not use that uh you know um that

participant excluding outliers say for example typical reaction time studies faster than 300 millconds slower than

3,000 mconds you exclude uh or you can basically create mean standard deviation and plus -2 SD or plus - 2.5 SD is what

you take out statistical method do you want to use ANOV do you want to use uh you know

other kinds of methods or is basian statistics are used. So what is it that you want to basically use in your

reaction time analysis interpreting your results how uh say for example uh let's say the result is when

a prime is masked bilingual respond to an L2 word faster in a lexical decision task when it follows its L1 translation

than when it follows an unrelated L1. you get this finding. How will you interpret it? For example, here the uh

you know uh interpretation is that mass translation priming is asymmetrical in lexical decision. It occurs only from L1

to L2 but does not happen from L2 to L1. So whatever reaction time uh results you're getting you have to interpret it

based on the theory of the task based on an understanding of the situations that you have created in a given experiment.

Interpretation of the result would represent what we think the results means to us. For example, in the

following example, the second statement represents how the result is interpreted or understood. For example, bilingual

translate faster from L2 to L1 than from L1 to L2. That is the result. How do we interpreted? The lexical links between

L2 to L1 are stronger as opposed to L1 to L2 direction. So you have some finding your reaction times are telling

you something. How do you rep how do you interpret that depend on your understanding of the theory or your

understanding of the mental processes that are involved. interpretation typically is made with

certain assumptions. That's what I was saying in the earlier example. The interpretation is based on the

assumption that the strength of lexical connections is the only factor here. But you can also sort of have an

interpretation that say proficiency plays a part. So an interpretation can more or less sometimes be directly

available from the result. Sometimes you have to work around it. So for example here uh advanced Chinese

English second language learners showed no difference in reaction times while reading grammatical and unrammatical

sentences involving English plural markings. The interpretation first could be the advanced ESL uh Chinese ESL

speakers are not sensitive to English plural errors. This is one interpretation. You can have further

interpretations to that too based on your understanding of the theory. You can say they have not developed highly

integrated linguistic knowledge in involving English plural marking or plural plural marking is not acquirable

in adult alu. So it is not they're not going to learn it at all. They have not yet learned it or they're not going to

be able to learn it at all. And again based on the kind so based on these interpretations you can decide follow-up

experiments. You can sort of narrow down and you can basically come up to your uh unique conclusions.

Explanation of a result can mean the same as interpretation though sometimes it can be very different. For example,

bilingual translate faster from L2 to L1 than from L1 to L2. That is the result. The interpretation could be the lexical

links in the L2 to L1 direction are stronger than those in the L1 to L2 direction. That is your uh

interpretation. You can explain it and in uh sort of expand on it. Say for example L2 to L1 links are uh slightly

stronger because lexical links between the two languages are established while bilingual uh learn the second language

and it is more likely for them to associate a second language word to the first language word uh as opposed to a

first language word to the second language word because the first language is the one that is learned earlier. So

whenever they are learning something later that is what they're linking to the first language. When they were

learning the first language there was no L2 to link it to. So therefore L2 to L1 connections are stronger as opposed to

L1 to L2 connections which makes this pattern of translations possible. Finally obviously whatever experimental

uh design you make unexpected results are possible alternative interpretations are possible but one basically has to

perform many task uh work across uh several methods several follow-up experiments to obtain what is called

converging evidence. I'll stop here. That is uh overall my description of reaction time studies. We

will move in the next lecture to other methods, eyetracking studies and so on. Thank you.