ProbabilityContinuous Distributions

Not every random phenomenon is ideally modeled using a discrete probability space. For example, we will see that the study of discrete distributions leads us to the Gaussian distribution, which smooths its probability mass out across the whole real number line, with most of the mass near the origin and less as you move out toward or .

The Gaussian distribution spreads its probability mass out across the real number line. There is no single point where a positive amount of probability mass is concentrated.

We won't be able to work with such distributions using probability mass functions, since the function which maps each point to the amount of probability mass at that point is the zero function. However, calculus provides us with a smooth way of specifying where stuff is on the number line and how to total it up: integration. We can define a function which is larger where there's more probability mass and smaller where there's less, and we can calculate probabilities by integrating .

The probability measure associated with a density assigns the measure XEQUATIONX1787XEQUATIONX to each interval XEQUATIONX1788XEQUATIONX.

The simplest possible choice for is the function which is on and 0 elsewhere. In this case, the probability mass associated with a set is the total length of . In higher dimensions, with the probability measure gives us a probability space, as does with the probability measure .

Solution. We have

by the area formula for a rectangle. Since and , we conclude that and are independent.

The probability density function

Just as a function we integrate to find total mass is called a mass density function, the function we integrate to find total probability is called a probability density function. We refer to as a density because its value at a point may be interpreted as limit as of the probability mass in the ball of radius around divided by the volume (or area/length) of that ball.

Solution. We calculate .

If is constant on , then we call the uniform measure on . Note that this requires that have finite volume.

All of the tools we developed for discrete probability spaces have analogues for continuous probability spaces. The main idea is to replace sums with integrals, and many of the definitions transfer over with no change. Let's briefly summarize and follow up with some exercises.

• The distribution of a continuous random variable is the measure on .
• The cumulative distribution function of a continuous random variable is defined by for all .
• The joint distribution of two continuous random variables and is the measure on .
• If is a continuous pair of random variables with joint density , then the conditional distribution of given the event has density defined by , where is the pdf of
• Two continuous random variables and are independent if for all and . This is true if and only if has density , where and are the densities of and , respectively.
• The expectation of a continuous random variable defined on a probability space is , where is 's density. The expectation is also given by , where is the density of the distribution of .

Continue

Solution. Then the conditional density of given is the uniform distribution on the segment , since that interval is the intersection of the triangle and the horizontal line at height .

Continue

Solution. We calculate

Solution. The CDF is increasing since whenever .

To see that is continuous, we note that the difference between and is the integral of the density over a width- interval. Thus we can use the supplied note to conclude that as for all .

Continue

Solution. The equation follows immediately from the fundamental theorem of calculus. We have

at any point where is continuous.

Let be the CDF of . Since for , we have for . Differentiating, we find that the density is on .

Continue

Solution.

It can be shown that, as result of monotonicity and additivity,

whenever the maximum exists. Now, because a CDF is monotonic, if has a jump from to at it must be the case that and Therefore, Since

by additivity, it follows that as required.

The CDF of a uniform random variable is

Therefore we have

as required.

A random variable with this density function has a CDF defined by

Therefore the generalized inverse of is for all This leads to the following Julia code for sampling from this distribution.

np.sqrt(np.random.random_sample())

using Distributions
sqrt(rand(Uniform(0, 1)))

General probability spaces

So far we have discussed probability spaces which are specified with the help of either a probability mass function or a probability density function. These are not the only possibilities. For example, if we produce an infinite sequence of independent bits , then the distribution of has CDF as shown in the figure below. This function doesn't have jumps, so it does not arise from cumulatively summing a mass function. But it does all of its increasing on a set of total length zero (in other words, there is a set of total length 1 on which the derivative of this function is zero), so it also does not arise from cumulatively integrating a density function.

In general, a person may propose a probability space by specifying any set , a collection of subsets of which supports taking countable unions, intersections, and complements, and a function defined on that collection of subsets. We require that certain properties are satisfied:

The CDF of the uniform measure on the Cantor set.