Basic rules of probability: independence and exclusion

There are two very important rules for calculating with probabilities. The first is related to the fundamental principle of counting we described earlier, and relates to the situation in which we consider two different events. We would like to know the probability of outcome A happening in the first event and outcome B happening in the second event. Each outcome has its own probability of occurring, say, P_A and P_B. We say the two events are independent of one another if the probability of both outcomes happening simultaneously is

$$P_A \times P_B$$

This is analogous to making two choices, and noting that the total number of possible choices is the product of the number of choices for each individual choice.

The second rule has to do with two outcomes that cannot occur simultaneously. For instance, a letter in a text could be an A, or it could be a B, but it cannot be both an A and a B. We say those two events are disjoint or mutually exclusive. Then the probability of one of two mutually exclusive outcomes A or B occurring is the sum of their individual probabilities P_A+P_B.

These are largely the only two rules necessary in the calculation of elementary probabilities. We will give an example of their use in the calculation of the index of coincidence of English. We consider whether or not two English letters match. The probability of the first letter being A is p_A, as is the probability of the second. Now assuming the two letters are ``independent,'' the probability of them both being A is the product p_A². Similarly, the probability of them both being B is p_B², and so on. Since these events (both being A versus both being B) cannot happen at the same time, they are mutually exclusive. Thus, the probability that both letters match, period, is the sum

$$p_A^2 + p_B^2 + \dotsb + p_Z^2$$

That is the index of coincidence of English, and when we use the values in Table 1.2 of [Beutelspacher, 1994] we find the index is about 0.065.