Probability theory and mathematical statistics 3 discrete random variables and probability distributions (Part 2)

3.4 The Binomial Probability Distribution( The binomial distribution )

There are many experiments that conform either exactly or approximately to the following list of requirements( Fully or approximately meet the following requirements ):

1. The experiment consists of a sequence of n smaller experiments called trials( test ), where n is fixed in advance of the experiment.
2. Each trial can result in one of the same two possible outcomes (dichotomous trials, Dichotomy test ), which we generically denote by success (S) and failure (F).
3. The trials are independent, so that the outcome on any particular trial does not influence the outcome on any other trial( The results of any particular test will not affect the results of any other test ).
4. The probability of success P(S) is constant from trial to trial; we denote this probability by p.

DEFINITION:

An experiment for which Conditions 1–4 are satisfied is called a binomial experiment

We will use the following rule of thumb( Rule of thumb ) in deciding whether a "without replacement( Take it out and don't put it back )"experiment can be treated as a binomial experiment.

RULE：

Consider sampling without replacement from a dichotomous population of size N. If the sample size (number of trials) n is at most 5% of the population size, the experiment can be analyzed as though it were exactly a binomial experiment.

The Binomial Random Variable and Distribution

DEFINITION：

The binomial random variable X associated with a binomial experiment consisting of n trials is defined as

$$X=thenumberof{S}^{\prime }samongthentrials$$

NOTATION:

Because the pmf of a binomial rv X depends on the two parameters n and p, we denote the pmf by b(x; n, p).

THEOREM:

$$b(x;n,p)=\{\begin{array}{l}\left(\genfrac{}{}{0ex}{}{n}{x}\right)p^x(1-p{)}^{n-x},x=0,1,2,...,n\\ 0,otherwise\end{array}$$

Using Binomial Tables*

NOTATION:

For X~Bin(n,p), the cdf will be denoted by

$$B(x;n,p)=P(X\le x)=\sum _{y=0}^{x}b(y;n,p)x=0,1,...,n$$

The Mean and Variance of X

PROPOSITION:

If X~Bin(n,p), then E(X)=np, V(X)=np(1-p)=npq, and ${\sigma }_X$=$\sqrt{npq}$ (where q = 1 - p).

3.5 Hypergeometric and Negative Binomial Distributions( Hypergeometric distribution and negative binomial distribution )

The Hypergeometric Distribution

The assumptions leading to the hypergeometric distribution are as follows:

1. The population or set to be sampled consists of N individuals, objects, or elements (a finite population).
2. Each individual can be characterized as a success (S) or a failure (F), and there are M successes in the population.
3. A sample of n individuals is selected without replacement in such a way that each subset of size n is equally likely to be chosen.

PROPOSITION:

If X is the number of S’s in a completely random sample of size n drawn from a population consisting of M S’s and (N-M)F’s, then the probability distribution of X, called the hypergeometric distribution, is given by

$$P(X=x)=h(x;n,M,N)=\frac{\left(\genfrac{}{}{0ex}{}{M}{x}\right)\left(\genfrac{}{}{0ex}{}{N-M}{n-x}\right)}{\left(\genfrac{}{}{0ex}{}{N}{n}\right)}$$

for x, an integer, satisfying max(0,n-N+M) $\le$ x $\le$ min(n,M).

PROPOSITION:

The mean and variance of the hypergeometric rv X having pmf h(x; n, M, N) are

$$E(X)=n\cdot \frac{M}{N}V(X)=(\frac{N-n}{N-1})\cdot n\cdot \frac{M}{N}\cdot (1-\frac{M}{N})$$

The means of the binomial and hypergeometric rv’s are equal, whereas the variances of the two rv’s differ by the factor $\frac{(N-n)}{(N-1)}$, often called the finite population correction factor( Finite population correction factor ). This factor is less than 1, so the hypergeometric variable has smaller variance than does the binomial rv. The correction factor can be written $\frac{(1-\frac{n}{N})}{(1-\frac{1}{N})}$, which is approximately 1 when n is small relative to N.

The Negative Binomial Distribution

The negative binomial rv and distribution are based on an experiment satisfying the following conditions:

• The experiment consists of a sequence of independent trials.
• Each trial can result in either a success (S) or a failure (F).
• The probability of success is constant from trial to trial, so for i = 1,2,3,…
• The experiment continues (trials are performed) until a total of r successes have been observed, where r is a specified positive integer.

The random variable of interest is X = the number of failures that precede the rth success; X is called a negative binomial random variable because, in contrast to the binomial rv, the number of successes is fixed and the number of trials is random.

PROPOSITION:

The pmf of the negative binomial rv X with parameters r=number of S’s and p=P(S) is

$$nb(x;r,p)=\left(\genfrac{}{}{0ex}{}{x+r-1}{r-1}\right)p^r(1-p{)}^{x}x=0,1,2,...$$

In some sources, the negative binomial rv is taken to be the number of trials X+r rather than the number of failures.

In the special case r=1, the pmf is

$$nb(x;1,p)=(1-p{)}^{x}px=0,1,2,...$$

Both X=number of F’s and Y=number of trials (=1+X) are referred to in the literature as geometric random variables( Geometric random variable ), and the pmf above is called the geometric distribution( Geometric distribution ).

PROPOSITION:

If X is a negative binomial rv with pmf nb(x; r, p), then

$$E(X)=\frac{r(1-p)}{p}V(X)=\frac{r(1-p)}{p^2}$$

3.6 The poisson Probability Distribution( Poisson distribution )

DEFINITION:

A discrete random variable X is said to have a Poisson distribution with parameter $\mu (\mu >0)$ if the pmf of X is

$$p(x;\mu )=\frac{e^{-\mu }\cdot {\mu }^x}{x!}x=0,1,2,3,...$$

$\mu$ is in fact the expected value of X. The letter e in the pmf represents the base of the natural logarithm system; its numerical value is approximately 2.71828. In contrast to the binomial and hypergeometric distributions, the Poisson distribution spreads probability over all non-negative integers, an infinite number of possibilities.

$$e^{\mu }=1+\mu +\frac{{\mu }^2}{2!}+\frac{{\mu }^3}{3!}+...=\sum _{x=0}^{\infty }\frac{{\mu }^x}{x!}$$

If the two extreme terms are multiplied by and then this quantity is moved inside the summation on the far right, the result is

$$1=\sum _{x=0}^{\infty }\frac{e^{-\mu }\cdot {\mu }^x}{x!}$$

The Poisson Distribution as a Limit

PROPOSITION:

Suppose that in the binomial pmf b(x; n, p), we let $n\to \infty$ and $p\to 0$ in such a way that np approaches a value $\mu >0$ . Then b(x; n, p) $\to$ p(x;$\mu$).

According to this proposition, in any binomial experiment in which n is large and p is small, $b(x;n,p)\approx p(x;\mu )$ , where $\mu =np$. As a rule of thumb, this approximation can safely be applied if n>50 and np<5.

The Mean and Variance of X

Since $b(x;n,p)\to p(x;\mu )$ as $n\to \infty$ , $p\to 0$ , $np\to \mu$, the mean and variance of a
binomial variable should approach those of a Poisson variable. These limits are
​$np\to \mu$ and $np(1-p)\to \mu$.

PROPOSITION:

If X has a Poisson distribution with parameter $\mu$, then $E(X)=V(X)=\mu$.

The Poisson Process

Assumption:

1. There exists a parameter $\alpha >0$ such that for any short time interval of length $\Delta$t, the probability that exactly one event occurs is $\alpha \cdot \Delta t+o(\Delta t{)}^{\ast }$.
2. The probability of more than one event occurring during $\Delta t$ is $o(\Delta t)$ [which, along with Assumption 1, implies that the probability of no events during $\Delta t$ is $1-\alpha \cdot \Delta t-o(\Delta t)$.
3. The number of events occurring during the time interval $\Delta t$ is independent of the number that occur prior to this time interval.

Informally, Assumption 1 says that for a short interval of time, the probability of a single event occurring is approximately proportional to the length of the time interval, where a is the constant of proportionality. Now let $P_k(t)$ denote the probability that k events will be observed during any particular time interval of length t.

PROPOSITION:

$P_k(t)=e-\alpha t\cdot (\alpha t{)}^k/k!$ ,so that the number of events during a time interval of length t is a Poisson rv with parameter $\mu =\alpha t$. The expected number of events during any such time interval is then $\alpha t$, so the expected number during a unit interval of time is $\alpha$.

The occurrence of events over time as described is called a Poisson process; the parameter $\alpha$ specifies the rate for the process.