this week i'm going to be going over measurement concepts in this video i'll be going over

reliability and the next one i'll be covering validity you may be wondering why on earth

psychologists and social scientists are scientists are interested in validity and reliability

the best explanation i can give you is that if you want to be a good scientist and you're you're developing a new

questionnaire you just don't write down questions and then hand it out to people you have to systematically look at and

see how stable over time your questionnaire is

how consistent the questions are and

do the questions really tap into something that you're interested in that's what reliability

and validity is getting at when i was studying at the university of colorado

this this man frederick coolidge was my advisor and he wanted to develop a new

questionnaire to to assess personality disorders and there is a

existing one called the milan axis ii inventory and it was expensive to administer

so he wanted to make one that was free to people and that was based on the diagnostic and

statistical manual for mental disorders the dsm so he he went about and he was looking

at the reliability and validity of the items in his questionnaire okay

so reliability reliability is the degree to which a measure is consistent

so most people could understand this and the best example is with like a bathroom scale

let's say you go on in the morning and you weigh yourself on your scale and it says 150 pounds

and if you step off and again you get back right on you want the scale to say 150 pounds

again right that's consistency that's at the heart of what reliability means

in the textbook they cover three major types of reliability test retest internal consistency and inter rater

so i'm going to be covering those test retest reliability is a reliability coefficient determined by the

correlation between scores on a measure given at one time with scores on the same measure given at a later time

usually it's going to be a week or two so for example using the coolidge access to

inventory on personalities push it for sound disorders dr coolidge gave it out to all his

psychology 100 students and then a week later he asked the same students to answer the same questions

in theory personalities should be stable over a week and also in theory

then the the the scales the the items

that are added up and form a scale for example anti-social personality disorder

those the results on those scales should be highly correlated from time one to time two

that's test retest reliability looking again at dr coolidge's published research

he found that the 13 personality disorder scales of the caddy had a mean test retest

reliability of 0.9 which is very good i'm sure he was proud of that the next type of reliability is internal

consistency reliability reliability assessed with data collected at one point in time with multiple

measures of a psychological construct a measure is reliable when the multiple measures provide similar results

and there's two types of internal consistency reliability that we're going to be covering

the first is chromebox alpha it's an indicator of the internal consistency reliability assessed by

examining the average correlation of each item in a measure with every other question

so what does that really mean let's say you were looking at preferences for ice cream flavor

okay and you had 10 items in your questionnaire

so one would be i like chocolate ice cream

another would be i like vanilla the third one would be i hate rocky road and then the fourth

item would be teslas are cool cars okay so i can go on but you can see there

that that item teslas are cool cars are it's not really related to the other items

okay and it probably shouldn't be in that questionnaire and chromebox alpha

is is looking at how consistent the items are in a questionnaire basically so you want if you're looking at

flavors of ice cream preferences you want all your items to be involved with

ice cream right and not teslas okay so basically what this is doing is that it's looking at the correlation of every

item in a questionnaire with every other item in this example here

there's six items so item number one is is is is 100 correlated with item

number one and then item number one has a 0.89 correlation with item number two

okay so as it says here chromax alpha is looking at the correlation of every item with every

other item that's what this grid is okay and then you look at that you you get the average

correlation so the average correlation of all these items together is 0.85 and here's a little cheat sheet here

in regards to how big a correlation you need in order to have good internal consistency

and look at looking again at dr coolidge's research

he found that chromebook's alpha was .76 so that was acceptable another type of internal consistency

reliability is split half reliability this is a reliability coefficient determined by the correlation between

scores on half of the items on a measure with scores on the other half of a measure

the two halves are created by randomly dividing the items into two parts the total score on one half of the test

is compared to the total score on the other half the spearman ground split half reliability coefficient is used to

calculate the reliability coefficient okay so basically in this example here there are six items

and they're randomly assigned into two groups and then the scores

of these two groups are totaled and then you look at the correlation between

those two totals and in this example we have 0.87 finally i want to talk about inter-rater

reliability and this is something again that probably a lot of people could understand

it's an indication of reliability that examines the agreement of observations made by two or more raiders or judges

so for example this might be like dancing with the stars or the olympics there they have a

they have usually in the olympics and dancing with the stars um i'm not an expert on dancing stars

but there's there's a panel and each member of the panel

rates the the the participants or the competitors

right and what you want is the the raters to be very correlated you don't want them

to be disagreeing with each other that much and interrelate rate or reliability

can is looking at how correlated the different raters are with each other and there's a statistic to to

look at that so that this is my presentation for reliability