Continuous Random Variables: PDF, CDF, and Key Concepts Explained

Overview: Transitioning from Discrete to Continuous Random Variables

This video marks the start of a new section on continuous random variables. While many concepts (like expectation and variance) have parallels to discrete random variables, continuous variables introduce unique subtleties, particularly due to their uncountable nature. The speaker strongly recommends comparing and contrasting each new concept with its discrete counterpart. For a comprehensive look at the discrete side, see our Comprehensive Review of Discrete Probability Distributions and Expected Values.

Foundational Concepts: Sample Spaces and Sigma Algebras

Sample Space (Ω): For continuous random variables, the sample space must be continuous and uncountable (e.g., the height of a person, a time interval). This is a crucial difference from discrete cases where Ω is finite or countably infinite.
Sigma Algebra (F): When Ω is continuous, you cannot simply use the power set (all subsets) as your collection of events. Instead, you must use a sigma algebra – a specific subset of the power set that satisfies three key axioms:
1. Ω itself must be in F.
2. If a set A is in F, its complement (A^c) must also be in F.
3. The countable union of sets in F must also be in F.
Probability Measure (P): This remains the same as before, a function from F to [0,1] with the standard Kolmogorov axioms (non-negativity, P(Ω)=1, and countable additivity for disjoint events).

Defining a Continuous Random Variable

A random variable is still a function X: Ω → R. The key difference is that for continuous variables, both the domain (Ω) and the range (the real numbers) are uncountably infinite. A discrete random variable, in contrast, has a countable range. For a refresher on core probability and statistical terminology like random variables, check out the Introduction to Probability and Statistics: Key Concepts and Terminology.

Example: Waiting for a Bus

Experiment: Waiting for a bus.
Source of Randomness: The time you arrive at the bus stop (e.g., uniformly between 7:10 and 7:30).
Random Variable (X): The amount of time you wait for the bus.
Key Insight: The probability of waiting for an exact time (e.g., exactly 4 minutes and 29 seconds) is zero. The meaningful probability is that the waiting time falls within an interval (e.g., between 4 and 5 minutes).

The Probability Density Function (PDF)

Since probabilities for exact values are zero (P(X = x) = 0), we cannot use a probability mass function (PMF). Instead, we use the probability density function (PDF), denoted as f_X(x).

Definition: It represents the probability of X being in a very small interval [x, x+dx] divided by the width (dx) of that interval. In essence, f_X(x) = P(x ≤ X ≤ x+dx) / dx.
Units: Unlike probability (which is unitless), the PDF has units (e.g., "per minute" for waiting time or "per meter" for height).
Properties:
1. Non-Negative: f_X(x) ≥ 0 for all x.
2. Total Area is 1: ∫_{-∞}^{∞} f_X(x) dx = 1. This is the continuous analog of ∑ P(X=x) = 1 for discrete variables.
3. Not Bounded by 1: The value of the PDF can be greater than 1, unlike a probability.

The Cumulative Distribution Function (CDF)

The CDF, denoted as F_X(x), remains a fully meaningful and identical concept for continuous variables. F_X(x) = P(X ≤ x).

Relationship to PDF: The CDF is the integral of the PDF from negative infinity to x: F_X(x) = ∫_{-∞}^{x} f_X(t) dt.
Properties:
1. F_X(-∞) = 0 and F_X(∞) = 1.
2. It is a non-decreasing function (since P(X ≤ x1) ≤ P(X ≤ x2) when x1 ≤ x2).

Worked Examples

Example 1: Uniform Waiting Time

Scenario: Arrival time is uniform between 7:15 and 7:30. Bus arrives every 15 minutes.
Random Variable X: Waiting time, between 0 and 15 minutes.
PDF: The PDF is uniform (constant) over the interval [0, 15]. To find the constant, solve ∫015 c dx = 1, so c = 1/15 per minute.
CDF: F_X(x) = 0 for x < 0, x/15 for 0 ≤ x ≤ 15, and 1 for x > 15. The CDF increases linearly from 0 to 1.

Example 2: Linear Density Function

Scenario: PDF is f_X(x) = x/2 for x ∈ [0, 2] and 0 otherwise.
Validation: The area under f_X(x) is a triangle with base 2 and height 1, so the area (1/2 * 2 * 1) is 1. It is a valid PDF.
CDF: For 0 ≤ x ≤ 2, F_X(x) = ∫0x (t/2) dt = x2/4. As expected, F_X(0)=0 and F_X(2)=1.
Conclusion: A linear PDF results in a quadratic CDF, which is smooth and increasing. Tree diagrams are another great tool for visualizing probability; see Calculating Conditional Probabilities Using Tree Diagrams and Dice Rolls.

[Music] hello everyone welcome to the second

subsection of the probability chapter in which we will look at continuous random variables we have seen some examples and

concepts related to discrete events and discrete random variables will a lot of those ideas will extend to continuous

random variables as well but there are some extra subtleties and important things which we will

see here right here is a brief overview of what we will

cover in the continuous random variables first we will just define continuous random variables and

the basic ideas which some of which you would have seen already in the discrete random

variables but how do you extend those ideas to continuous random variables and some of these are very similar to the

discrete random variable case you can see that there is expectation and variance multiple random variables and

so on these were all ideas that were available in discrete random variables as well so i would highly recommend that

each concept that you see in continuous random variables compare and contrast with discrete random variables how are

they similar how are they different and so on so ah the first thing is sample space we have already seen that an

experiment is made of a probability an experiment in probability is made of three things a sample space omega

and collection of events f and the probability measure p

in the discrete random variables case we have almost always been using omega's finite set or at most a

discrete infinite set that's what we have always worked with but

in general there is nothing restricting an experiment to have only a finite or discrete amount of outcomes right for

example let's say you pick a random person in the world and measure their height

right the output of this experiment is not discrete right so it can be i mean the height of a person can be six point

one three one one meters and so on right so uh this is an example of an experiment where the sample space is

continuous of course you could argue that uh

everything is discrete at a quantum level and so on but let's ignore those technical issues and imagine a case

where you can have a really continuous set of outcomes so that is the

that's your omega can have infinitely uncountable outcomes also that's an example that's how that's the

first requirement for studying continuous random variables if omega happens to be

discrete or finite then there is no such thing all all variables associated with such an experiment are going to have to

be discrete but even with experiments where omega the

sample space is continuous and uncountable you can have discrete random variables we will see but

if you you must have omega to be uncountable and continuous

for you to even consider i mean for you to even have continuous random variables so

in this particular module you will always have our sample space omega

to be a continuous or infinitely uncountable setup so that brings us to another an

important complication which is our f previously this is what we had used

right when when our omega sample space omega was either finite or

just discrete we just used our set of events as simply the set of all events which is 0 comma 1 power omega right

that's what we have been using but that is no longer possible if your omega is for example let's say all reals

right or let's say you are measuring someone's height it potentially any real num any positive real number is

a possible output of the someone's height so your omega is essentially the entire real

line if that is the case uh you can't have 0 comma 1 power omega all subsets of the real line is not a valid

there are issues with choosing that to be the set of all events so technically what people do is you choose a subset of

that right so that's why your collection of events f is a subset of 0 comma 1 power omega

so it must satisfy certain properties right what are the properties well the properties are pretty simple first is

that omega must be in that right so this is uh

all of these properties are going to hold your omega was discrete and f was simply 0 comma 1 power omega that is the

power set of omega but we don't we can't do that for continuous omega what we'll do is we'll actually

work with a subset of all subset of the power set of omega and what all are valid collection of events

well it must satisfy these properties the first is that your sample space must belong to the collection of events and

then if there is some set a which belongs to the collection of events then a complement must also belong to the

collection of events and similarly you have a 1 a 2 a n belongs to the collection of events then

the union of that must also belong to the collection of events right so this these are

these are mostly not these are technical considerations and mostly you don't need

to worry about it at this stage because your first probability course you don't need to worry about this but you must

understand that only events collection of events which satisfy these

axioms are allowed these are called the sigma algebra axioms then we have the probability measure

probability measure is once again the same as before is a mapping from

the omega sample space omega to r plus right all right it's

it's the same as before and as it has the same axioms as before these axioms are not new it's the

exactly the same as before that is you have probability of any set a should be

greater than equal to 0 and probability of omega should be equal to 1 and if you have a 1 to a n or disjoint sets then

the probability of the union of this joint sets is simply equal to the sum of the probabilities

right so these are the same axioms that we had for discrete random variables but still i am

going to emphasize them just for the sake of completeness

once again we have our omega f p as the tuple which represents the probability space or the experiment

right so all random variables are always in the context of an experiment and experiment

is defined by omega f and p right so i am not really going to go into great detail here now we will move

on to the main player here which is continuous random variable

the continuous random variable a random variable is defined in exactly the same way that is is a function which maps

your sample space omega to the reals r right that's exactly the same but

the difference is that your omega is going to be infinitely uncountable and your r is

also this which means that your domain and range

are domain and codomain are both infinite unlike the previous case where in the discrete random variables that

our domain and drain and your codomain were finite and domain and range were also finite but here we have that x

as omega so mapping omega to r we have both domain and range

to be uncountably infinite so if you don't understand what uncountably infinite means just take it to mean the

set of all real numbers right that's an example of an uncountable infinite the set of integers is a countable infinity

the set of real numbers is an uncountable infinity right that's the definition of random variable

is the same just that what's the difference between a continuous and a discrete random variable a discrete

random variable has the range of a discrete random variable is uh countable but the range of a

continuous random variable is not there are some extra properties also but that's one of the main differences

right so that's a random variable let's see some simple examples right for example let us take the

experiment that we had let let's say the experiment is that you are uh going uh to catch a bus right

so that's the experiment and the time at which you go to the bus stop is the random is the is the source of

randomness any time you have a an experiment you should always ask what is the source of randomness for example

let's say you toss the coin what is the source of the randomness the source of the randomness is that uh is simply

chaos right so you talk you task the coin you don't really know what's going to happen similarly this case for ah

let's say throwing the dice so any experiment what is the source of randomness the source of randomness can

mean can be mostly let's say if it's physics it's going to be chaos or something like

that or it can just be things that are beyond your control are things that you don't know how to model for example let

us say you are modeling stock prices right whether

that is your experiment your experiment you can ask the question your experiment is just simply let's say

observing the stocks stock prices over the day right even though technically speaking there is no randomness if

everyone uh it's simply a deterministic function uh

as far as you are concerned it's random right because it's uh the modeling the

the stock prices as a deterministic function is impossible so that's the reason why you model it as a random

variable so let's say in this example your randomness

is the time at which you reach the bus term so that's the randomness and there is let's say there is a bus every

15 minutes right so you can have the random variable x

i am not going to write exclusively write what the experiment is omega the experiment is

ok the experiment is simply

waiting for the ah waiting for bus lets call this waiting for bus

right that is the experiment and the random variable now x is the amount of time that you wait for

the bus lets say okay of course we need more details for this ah lets say that there is a

there is a buzz every 15 minutes right so there is a bus at 7 o'clock 7 15 720 and so on right

and let's say you reach the bus stop anytime at random

between 7 10 and 7 30 right so this is a pretty standard experiment so you have bus coming at let's say 7 o'clock 7 15

7 30 and so on right and you reach

the bus stop anytime at randomly between 7 us say 7 10 and 7 20.

right this is the experiment and the experiment is simply a manifestation of the real world and

the random variable in this experiment let's say you come to the bus stop and you wait and you catch the bus and you

go right so one experiment is this the random variable of interest let's say in this case is

the amount of time you wait for the bus point so what would this take it would take the results of this experiment

omega would be after this experiment is done ah your x is a random variable which takes in the result of this

experiment which would essentially take into account when you arrived when you caught the bus and so on and it would

give you how much time you waited for a bus this is an example of a random variable

okay so that's a pretty simple example and that's sufficient right so the same way that you have

the properties and attributes of a discrete random variable you can have properties and attributes of continuous

random variables also and the clearest we'll see several analogs the first analog is what's called the

density function so let's say you have a random variable x

the density function is f x of x what is the meaning of this of let's say

you have some x in reals what is f x of x

f x of x is essentially the probability of x being

very close to x let us say x to

x plus d x where d x is some arbitrary small let us say time interval in the case of

the bus experiment dx is arbitrarily small time interval divided by dx that's why f x of x is called a density

function it's the probability density function and this is really important because you can't use a probability mass

function why is that because let us say probability of x is equal to x is equal to 0 for any x

why is that because for any given time instant probability of u arriving exactly at the time

instant is zero right and then let's say in the bus experiment uh someone can ask the question what is the probability

that you exactly wait for four minutes and 29 seconds what is the probability you wait for exactly that well the

answer is zero because there are so many uh options why would you exactly wait for 4.99

seconds right the probability of that event is zero but someone can ask the question what is the probability that

you wait between 4 minutes 29 seconds and 4 minutes 35 seconds you can ask that question and that is

that might be small but still not be zero so that's the reason why you have probability of any if you have any

continuous random variable the probability of the continuous random variable taking an exact value x is

always going to be zero but despite that you can have density around that of that's why you have you

are dividing that by dx so even though this numerator is going to be zero the denominator is also going to be zero so

that's the idea behind a density function so even though probabilities don't have

units density functions in a way you could you can say they have units because they are dividing by x right for

example in the bus waiting experiment where x is the amount of time that you wait for the bus

the unit of the density is going to be let's say per hour or per minute right so based on what you want to use can use

per hour or per minute for if x was the height of the person that you are measuring let us say in height measuring

experiment x represented the height of the person that you are measuring the density would be let's say per meter

the unit of that would be dense because you are dividing by dx probability might does not have any unit but

when you divide by dx the the unit is automatically created

right that's uh okay so that's the reason why you can't have

probability mass functions you can you need to have densities and that's the reason why you will have

for continuous random variables probability mass functions don't make sense you only have density functions so

this is called the probability density function or the pdf so typically denoted by small f subscript x and you will see

that in almost all the definitions for which

there is a discrete random variable analog essentially probability mass functions

and probability density functions behave similarly so you will in a lot of places where you have in

discrete random variables where you have sum over x f x of x you will see integral f x of

x dx so the sum will be replaced by an integral but other than that you will

see very similar behavior throughout so but however you should understand the difference between a density function

and a probability mass function ah but the cumulative distribution function is still a completely

meaningful object here what is that well it's exactly the same as what we had for the discrete random

variable what is f x of x that is simply equal to probability of x less than or equal to x

this is the probability density function and this is the cumulative distribution function

okay right so let's see what are the properties of ok

let us give some important properties of the pdf and cdf right

first property is that the density function is greater than or equal to zero this is

obvious because probability is always going to be better than equal to 0 and d x is also

positive so probability of x in x belonging to x plus d x by d x is going to be positive so f x of x is greater

than or equal to 0 but not however i'm not saying f x of x is less than or equal to 1. why is that that's because

of the dx here even though this number can be this number can never exceed the numerator can never exceed one because

probabilities are going to be between zero and one ah there there's a denominator here which means that the

density can exceed one so that's the first probability the second

property is similar to the analog to the probability mass function summing to one right you

know the probability mass function sums to 1. if you sum over all the non-zero

arguments of a probability mass function it sums to 1. and why is that that's because the probability of

the sample space in the entire sample space is equal to 1 and that's simple axioms of probability by a similar logic

what you can have is f x of x times d x is simply probability of x belonging to x plus d x right which

means that to get the entire range you just integrated so that means that integral over minus

infinity to infinity f x of

x d x is equal to one right this is the analog

of this property is the analog of the probability mass function uh summing to one

so these two are the main properties of the probability density function are there any interesting properties of

the cdf well it's pretty much the same that of the discrete random variable that is

cdf would be f x of minus infinity is equal to 0

and f x of plus infinity is equal to one right at the left end the cdf is zero the

right end it is one and uh the other property is also the same as the discrete random variable that is

cdf is monotonically increasing it can never decrease for example if you have f x of 3 is 0.6 f x

of 5 can definitely has to be greater than or equal to 0.6 why is that because

probability of x less than or equal to 5 is definitely greater than or equal to probability of x less than or equal to 3

that's the reason why you have cdf fx is increasing

it's an increasing function i have to strictly say that it's non-decreasing because it can stay the

same right but i am going to say just increasing to make life easier for us so these are the main properties of the

probability density function and the cumulative distribution function let us see some simple examples

x is amount of time waiting time for

bus the smallest value is 0 and the largest value is clearly 15

right so let's make a simple simple

simplification here instead of you arriving uniformly between 7 10 and 7 30 let's say you arrive with uniform

between 7 15 and 7 30. right so if you are uniformly between 7 15 and 7 30 well the amount of time you wait it is

quite equally likely that you will wait for zero minutes which means that you will occur exactly at 7 30 or you will

wait for 15 minutes which means that you occur exactly at 750 right so this is a slightly simplified version in which

case you will have the same number here

which means that fx of x is equal to

some constant if x

is between 0 and 15 and 0 otherwise right so this is a simple example

before let's take the more complicated example later on where you are uniformly between 7 10 and 7 30. let's consider a

simple example where you arrive uniformly between 7 15 and 7 30 that's the

thing and how much time do you wait for the bus well it is quite likely that you wait any number between 0 and 15 minutes

equally likely so that's a constant but what is this constant and second question what is the unit in this case

because i am using minutes here the units here is f x of x is c per minute right so you

can take minute power minus 1 is the unit for the density function here but what is the value of c

here is where you can use the fact that when you integrate it over the entire real line you are going to

get have to get 1 so which means that c is equal to 1 by 15. so that is the seal 1 by 15 per minute

so if instead if you had used let's say hour instead instead of minute test the unit you would see it's 4 per hour

right so it's 4 per hour for x between 0 to point five sorry zero to zero point two five so you will wait anytime

between zero to zero point two five hours uh the density is value is four y is four

because if you integrate this function from the entire or the entire real line it has to

it has to equal one which means that the area under this rectangle has to be equal to one ah so the height of this

rectangle has to be simply equal to one by the width of this angle so that's one by fifteen per minute or

4 per hour so this is an example simple example of a density function

what is the cumulative distribution function for this

example what is capital f x of x well the answer for that is going to be is

going to be 0 if x is less than 0 so there is no way you can

wait for a negative amount of time so that's f x of x is equal to 0 and then you have

x by 15 if x belongs to

0 to 15 and

1 if x is greater than 50. this is an example of the cdf of this

random variable where it rises linearly from 0 to the probability of x less than or equal

to 0 is 0 probability of x n equal to 15 is 1 right so beyond 15 you are definitely you're never going to wait

for more than 15 minutes so that's why the probability of x less than or equal to 16 is the same as probability of x

less than equal to 17 is equal to and so on right if you plotted this it would look like this

so this is 0 this is 15 the value here this value here would be 0 the value

here would be 1 right so this is the cdf of this random variable

let us take one more example let us say you have your f x

of x is equal to x if

x belongs to 0 2 and

0 otherwise so this is a density function first of all let us try to plot this

this is 0 2. so this is the density function first of

you should check whether this is a valid density function how would you do that well if you integrate

it you should get one so if what is the integration of this function well it's simply the area under

the curve and this is a nice triangle right so what is this so it's half into base into height which seems to say that

half into two into two which suggests that this is not a density function so you should

immediately see that this is a not a valid density function you have to have x by 2 instead

this is a valid density function why is that because the area under the curve or this triangle is

simply equal to one right and what is the cumulative distribution function of this

is going to be is equal to probability of x less than equal to x and here is where you can use

the ah calculus part of this which is f x of x is probability of

probability of x less than or equal to x which is simply equal to integral minus infinity to

infinity problem sorry minus infinity to x probability of x

belongs to x 2 x plus d x

and this is exactly equal to integral minus infinity to x f x of x

d x and in this case it would be x by so minus infinity can be replaced by zero

here because anything below zeros anyway zero the density of

x belows below zero is zero anyway so again just replace by zero to x and what is this value this is simply equal to

let us just consider x in the range of 0 to 2 right when x is less than 0 you know that the cdf is 0

and x is greater than 2 it is going to be equal to 1 the only interesting range for cdf is between 0 to 2 and within

that range it's simply the integral over 0 to x f x of x dx and what is that integral

half x squared by 2 within the limit 0 to x

right and this is exactly equal to x squared by four you can check that when you substitute f

x of two you get two by four which is equal to one right so you have a cdf which is

quadratic for a linear linear density linear density function you have a cdf which is quadratic it is

0 at 0 and at 1 it is and it is at 2 it is 1 and after 2 it is constant at one right so that is how the cdf looks like

for this simple example

Keep this summary

Save it to LunaNotes and it becomes a real note in your library — editable, searchable, and ready to turn into flashcards or a diagram. Free to start.

Save to LunaNotes

Or summarise for another video.

This summary and transcript were automatically generated using AI with the Free YouTube Transcript Summary Tool by LunaNotes.

Related summaries

Comprehensive Review of Discrete Probability Distributions and Expected Values

This video review explains discrete probability distributions, including how to calculate probabilities, expected values, and conditional probabilities using practical examples. It covers machine breakdown probabilities and dice game scoring to illustrate key concepts in probability theory.

Introduction to Probability and Statistics: Key Concepts and Terminology

In this video, Dr. Gajendra Purohit introduces the fundamentals of probability and statistics, covering essential terminology, types of events, and key concepts such as random experiments, sample space, and probability calculations. The session aims to provide a solid foundation for students preparing for advanced mathematics exams.

Introduction to Probability: Foundations for AI & Bayes' Nets

Learn the core concepts of probability theory essential for AI, including probability models, random variables, joint & conditional distributions, and Bayes' rule. This lecture explains why probabilistic reasoning is crucial for handling real-world uncertainty in machine learning and decision-making, with intuitive examples and games.

Probability & Statistics: The Ultimate Guide to Modeling Uncertainty (Course Intro)

Professor Steve Brunton launches an exciting new short course on probability and statistics. This introductory overview explains why probability is a foundational tool for data science and machine learning, provides real-world examples from thermodynamics to weather, and outlines what you will learn in the first 10 hours dedicated to probability.

Partial Derivatives Explained: The Intuitive Connection to Ordinary Derivatives

This video demystifies partial derivatives by revealing their deep connection to ordinary derivatives. Using the function f(x,y) = x²y + sin(y) as an example, you'll learn how to compute a partial derivative by treating one variable as constant, and understand the geometric intuition of a

Start taking better notes today with LunaNotes

Start for free