Discrete Structures for Computing Notes 9 ------------------------------------------------------------------------ Chapter 5: Counting ------------------------------------------------------------------------ This chapter studies how to COUNT objects with certain properties. Examples: * how many different telephone numbers can there be? * how many different Internet addresses? An important thing to count is the number of different PERMUTATIONS of a set of objects -- order of the objects matters. Ex: How many different ways are there to seat the guests at a banquet? Another important thing to count is the number of different COMBINATIONS of a set of objects -- order of the objects does not matter. Ex: How many different groups of people can sit at the head table? ------------------------------------------------------------------------ 5.1 The Basics of Counting ------------------------------------------------------------------------ PRODUCT RULE: Suppose there are x ways to do one task and y ways to do another task. Then there are x*y ways to do the first task followed by the second task. Generalizes to more than two tasks. Examples: * Suppose there are 12 available offices and two students to be assigned to office. How many ways are there to assign different offices to the two students? First, assign an office to the first student. There are 12 ways to do this. Second, assign an unoccupied office to the second student. There are 11 ways to do this (since one office is already occupied). So there are 12*11 = 132 ways to assign different offices to the two students. * How many bit strings of length 7 are there? First, assign a bit (0 or 1) to the first bit. 2 ways to do this. Second, assign a bit (0 or 1) to the second bit. 2 ways to do this. ... Last, assign a bit (0 or 1) to the seventh bit. 2 ways to do this. So there are 2*2*2*2*2*2*2 = 2^7 = 128 different bit strings of length 7. * How many different license plates when the pattern is 3 letters followed by 3 digits? (26^3)*(10^3) = 17,576,000 * How many functions are there from a set with m elements to a set with n elements? For each of the m elements in the domain, choose one of the n elements in the codomain to be its image. So number of functions is n*n*...*n = n^m. For instance, there 5*5*5 = 125 functions from {a,b,c} to {1,2,3,4,5}. * How many one-to-one functions are there from a set with m elements to a set with n elements? Assume the domain is {x_1,x_2,...,x_m}. Consider x_1 and choose one of the n elements in the codomain to be its image. Consider x_2 and choose one of the remaining n-1 elements in the codomain to be its image. Consider x_3 and choose one of the remaining n-2 elements in the codomain to be its image. ... Consider x_m and choose one of the remaining n - (m-1) elements in the codomain to be its image. So the number of such functions is n*(n-1)*(n-2)*...*(n-(m-1)). For instance, there are 5*4*3 = 60 one-to-one functions from {a,b,c} to {1,2,3,4,5}. * How many different subsets are there of a set with n elements? Use product rule to compute it. Suppose the set is {x_1, x_2, ..., x_n}. For each element of x, decide whether or not it is to be chosen for the subset. There are 2 choices for x_1 (in the subset or not in the subset). There are 2 choices for x_2. ... There are 2 choices for x_n. So the total number of subsets is 2^n. Product rule in terms of sets: If A_1, A_2, ..., A_m are finite sets, then |A_1 x A_2 x ... x A_m| = |A_1| * |A_2| * ... * |A_m|. ------ SUM RULE: If a task can be done either in x ways or in y *different* ways, then there are altogether x + y different ways to do the task. Extends to more than two classes of ways to solve the task. Examples: * Suppose there are 400 computer science majors and 300 computer engineering majors at a university and the president of the computer game club must be either a CS or a CE major. Then there are 400 + 300 = 700 different choices for the president. In terms of sets: If A_1, A_2, ..., A_m are *disjoint* finite sets, then |A_1 U A_2 U ... U A_m| = |A_1| + |A_2| + .. + |A_m|. When the sets are not disjoint, more complicated. (More later.) Sometimes you might need to use both the product rule and the sum rule together. Examples: * How many passwords can there be in a system with these rules? - 6 to 8 characters long - each character is an uppercase letter or a digit - must contain at least one digit Total number is P6 + P7 + P8, where Pi is the number of passwords with i characters. (Sum rule) First, let's forget about the rule that you have to have at least one digit. How many 6-character passwords are there? Each of the 6 characters is either a letter or a digit, i.e., 26+10 = 36 choices (sum rule). So the number of choices for the whole password is 36^6 (product rule). Now let's worry about the rule that you have to have at least one digit. total number of 6-character passwords = number of 6-character passwords with at least one digit + number of 6-character passwords with no digits How many 6-character passwords are there with no digits? 26^6. So the number of 6-character passwords with at least one digit is 36^6 - 26^6, which is about 1.8 billion. Similarly, the number of 7-character passwords with at least one digit is 36^7 - 26^7, which is about 70 billion. And the number of 8-character passwords with at least one digit is 36^8 - 26^8, which is about 2.6 trillion. So altogether, we have about 2.7 trillion. ------- INCLUSION-EXCLUSION PRINCIPLE: Suppose a task can be done in x ways or in y ways, but some of the x ways are the same as some of the y ways. Sum rule doesn't work as it overcounts. Instead: Let A_1 and A_2 be two sets (might not be disjoint). Then |A_1 U A_2| = |A_1| + |A_2| - |A_1 /\ A_2|. Example: * How many bit strings of length 8 either start with 1 or end with 00? Number that start with 1 is 1*2*2*2*2*2*2*2 = 2^7 = 128 (one choice for first bit, 2 choices for the other bits). Number that end with 00 is 2*2*2*2*2*2*1*1 = 2^6 = 64 (one choice for each of the last 2 bits, 2 choices for the other bits). We can't deduce that the number is 128 + 64, because we have double counted those strings that start with 1 AND end with 00. So let's find out how many they are and subtract off one copy of each. Number that start with 1 and end with 00 is 1*2*2*2*2*2*1*1 = 2^5 = 32. So final answer is 128 + 64 - 32 = 160. Principle of inclusion-exclusion can be generalized to more than 2 sets. ------ Sometimes a tree diagram might be helpful to solve relatively small problems, or to get some intuition about a problem. << draw figures from pp. 343-344 >> ------------------------------------------------------------------------ 5.2 The Pigeonhole Principle ------------------------------------------------------------------------ A seemingly obvious, yet very useful, observation is the PIGEONHOLE PRINCIPLE: if k+1 or more objects are placed into k boxes, then at least one box contains more than one object. PROOF: If every box has at most one object, then the total number of objects would be at most k, instead of at least k+1. QED COROLLARY: A function from a set with m elements to a set with n elements is not one-to-one if m > n. PROOF: Think of each element y of the codomain as a box that holds all elements of the domain that are mapped to y. So we have m objects mapped to n < m boxes. By the pigeonhole principle, at least one box (element of the codomain) has more than one object (more than one element of the domain mapped to it). QED More examples of using the pigeonhole principle: * Among any set of 367 people, at least 2 have the same birthday (since there are only 366 different possible birthdays). * In any group of 27 words, at least two start with the same letter (since there are only 26 letters in the alphabet) * There is a multiple of 6 that has only 0's and 1's in its decimal expansion. Proof: Look at numbers of the form 1, 11, 111, 1111, etc. until finding two that have the same remainder when divided by 6. 1 divided by 6 has remainder 1 11 divided by 6 has remainder 5 111 divided by 6 has remainder 3 1,111 divided by 6 has remainder 1 Now look at 1,111 - 1 = 1,110. I claim that 1,110 is a multiple of 6. Why? 1,111 = 185*6 + 1 and 1 = 0*6 + 1. So 1,111 - 1 = (185*6 + 1) - (0*6 + 1) = (185-0)*6. QED There is nothing special about 6. We can generalize this argument, using the pigeonhole principle: For every integer n, there is a multiple of n that has only 0's and 1's in its decimal representation. Proof: consider the n+1 integers in this list: x_1 = 1 x_2 = 11 x_3 = 111 x_4 = 1,111 ... x_{n+1} = (number consisting of a string of n+1 ones) BY THE PIGEONHOLE PRINCIPLE, there are at least two numbers in the list that have the same remainder when divided by n (since there are only n possible different remainders). Call these two numbers x_i and x_j (j > i) and let r be the common remainder. So x_i = a*n + r for some integer a and x_j = b*n + r for some integer b. So x_j - x_i = (a*n + r) - (b*n + r) = (a-b)*n, which is a multiple of n. -------- GENERALIZED PIGEONHOLE PRINCIPLE: If N objects are placed in k boxes, then at least one box has at least ceil(N/k) objects. Why? Proof by contradiction (notice how we work with ceiling). Suppose N objects are placed in k boxes, but every box has less than ceil(N/k) objects in it. Then every box has at most ceil(N/k) - 1 objects in it. The total number of objects is at most k*(ceil(N/k) - 1) < k*((N/k + 1) - 1) since ceil(N/k) < N/k + 1 = k*(N/k) = N. I.e., the total number of objects is less than N. But this is a contradiction to the hypothesis. QED Example uses of the generalized pigeonhole principle: * Among 100 people, at least ceil(100/12) = 9 were born in the same month. * How many cards must be drawn from a standard deck of 52 cards to ensure at least 3 of the same suit? One box for each suit, so k = 4. Each card drawn is put in the appropriate box. Need to find N such that ceil(N/4) >= 3. Can think of worst case as when the cards drawn are distributed as evenly as possible among the suits. Once we have drawn 2 of each suit (i.e., 2*4 = 8), the next card drawn will cause there to be 3 of some suit. So we have to draw 9 in the worst case. * Show every sequence of n^2 + 1 distinct real numbers contains a subsequence of length n+1 that is either strictly increasing or strictly decreasing. NOTE: subsequence does not have to be consecutive! For instance, let n = 3. Consider this sequence of 3^2 + 1 = 10 terms: 8, 11, 9, 1, 4, 6, 12, 10, 5, 7 == = = = Let's find a subsequence of length n+1 = 4 that is either strictly increasing or strictly decreasing: 11, 9, 6, 5 is one. Proof: Consider the sequence a_1, a_2, ..., a_{n^2+1}. Let i_k be the length of the longest increasing subsequence starting at a_k. let d_k be the length of the longest decreasing subsequence starting at a_k. Example: a_1 = 8 a_2 = 11 a_3 = 9 a_4 = 1 a_5 = 4 Then i_1 = 2: The increasing subsequences starting at a_1 = 8 are 8 8, 11 8, 9 And d_1 = 2: The decreasing subsequences starting at a_1 = 8 are 8 8, 1 8, 4 i_2 = 1: The only increasing subsequence starting at a_2 = 11 is 11 d_2 = 3: The decreasing subsequences starting at a_2 = 11 are 11 11, 9 11, 1 11, 4 11, 9, 1 11, 9, 4 i_3 = 1: The only increasing subsequence starting at a_3 = 9 is 9 d_3 = 2: The decreasing subsequences starting at a_3 = 9 are 9 9, 1 9, 4 etc. Suppose in contradiction there is no increasing or decreasing subsequence of length n+1. Then each i_k and d_k is a number between 1 and n. By the product rule, there are n^2 different possibilities for the combinations of i_k and d_k. Since we have n^2 + 1 numbers in the sequence, by the pigeonhole principle, there are two numbers in the sequence, say a_s and a_t, that have the same value for i_s and i_t (call it x), and have the same value for d_s and d_t (call it y). Case 1: a_s < a_t. Then we can build an increasing sequence starting at a_s with length x+1: start with a_s, then continue with an increasing sequence of length x starting with a_t. This is a contradiction to the assumption that i_s = x. Case 2: a_s > a_t. Then we can build a decreasing sequence starting at a_s with length y+1: start with a_s, then continue with a decreasing sequence of length y starting with a_t. This is a contradiction to the assumption that d_s = y. QED * Show that in any group of 6 people in which every 2 people are either friends or enemies, there are 3 people who are either all friends or all enemies. Proof: Pick one of the people in the group, say A. Use the generalized pigeonhole principle: there are 2 boxes, friends of A, and enemies of A. Put the other 5 people in the appropriate boxes. At least one box has at least ceil(5/2) = 3 people in it. Case 1: The box with at least 3 people is the "friends of A" box. Case 1.1: There are two people in the "friends of A" box who are friends with each other, say B and C. Then A, B and C are all friends. Case 1.2: All the people in the "friends of A" box are enemies with each other. Since there are at least 3 people in this box, we have a set of 3 people who are all enemies. Case 2: The box with at least 3 people is the "enemies of A" box. This case is argued analogously to Case 1. QED ------------------------------------------------------------------------ 5.3 Permutations and Combinations ------------------------------------------------------------------------ PERMUTATION of a set is an **ordered** list of the elements of the set. Ex: S = {1,2,3}, permutations of S are 1,2,3 3,2,1 2,1,3 1,3,2 3,1,2 2,3,1 r-PERMUTATION of a set is an ordered list of r elements of the set. Ex: The 2-permutations of S (from above) are 1,2 2,3 1,3 2,1 3,2 3,1 NOTATION: P(n,r) is the number of r-permutations of a set of n elements. THEOREM: P(n,r) = n*(n-1)*(n-2)*...*(n-r+1). PROOF: n ways to choose first element, n-1 ways to choose second element, etc. Use the product rule. QED NOTE: P(n,0) = 1. NOTE: P(n,r) = n!/(n-r)! NOTE: P(n,n) = n! EXAMPLE: How many ways are there to select first, second and third prizes from 100 entries in a contest? P(100,3) = 100*99*98 = 970,200 EXAMPLE: How many permutations of the letters A through H contain A,B,C (consecutively)? Think of having 6 objects to permute, ABC as a block and also D, E, F, G, and H. They can occur in any order, so we have P(6,6) = 6! = 720. r-COMBINATION: **unordered** collection (subset) of r elements from a set. EXAMPLE: S = {1,2,3,4}. Then {1,3,4} is a 3-combination from S. NOTATION: C(n,r) is number of r-combinations from a set of n elements. Alternative notation is ( n ). ( r ) Called BINOMIAL COEFFICIENT. EXAMPLE: S = {a, b, c, d}. Subsets of size 2 are {a,b,}, {a,c}, {a,d}, {b,c}, {b,d}, {c,d}. So C(4,2) = 6. Order doesn't matter! THEOREM: C(n,r) = n!/(r!(n-r)!). PROOF: Think about relationship between C(n,r) and P(n,r). P(n,r) (number of permutations, or orderings, of size r) equals C(n,r) (number of combinations, or subsets, of size r) times P(r,r) (number of ways to order each subst of size r). So n!/(n-r)! = C(n,r)*r!. Solve for C(n,r) to get the answer. QED EXMPLE: How many piker hands are there? C(52,5) = 52!/(5!47!) = 52*51*50*49*49/(5*4*3*2*1) = 2,598,960 Note that C(52,47) = C(52,5). FACT: C(n,r) = C(n,n-r). Why? n!/(r!(n-r)!) = n!/((n-r)!(n-(n-r))!) EXAMPLE: How many bit strings of length n contain exactly r ones? Order doesn't matter, so position of the 1's can be considered an r-combination. C(n,r) is the answer. BINOMIAL THEOREM: If n is a nonnegative integer, then (x+y)^n = Sum_{j=0}^n C(n,j)*x^{n-j}*y^j = C(n,0)*x^n + C(n,1)*x^{n-1}*y + C(n,2)*x^{n-2}*y^2 + ... ... + C(n,n-1)*x*y^{n-1} + C(n,n)*y^n PROOF: When you multiply out (x+y)^n, the coefficient on x^{n-j}*y^j is what you get from adding up all the terms x^{n-j}*y^j. We have n-j x's from the n sums and thus j y's. QED EXAMPLE: n = 2. (x+y)^2 = x^2 + 2xy + y^2 = C(2,0)*x^2 + C(2,1)*x*y + C(2,2)*y^2. EXAMPLE: n = 4. (x+y)^4 = C(4,0)*x^4 + C(4,1)*x^3*y + C(4,2)*x^2*y^2 + C(4,3)*x*y^3 + C(4,4)*y^4 = x^4 + 4x^3*y + 6x^2*y^2 + 4xy^3 + y^4 EXAMPLE: What is the coefficient of x^12*y*13 in the expansion of (x+y)^25? C(25,13) = 25!/(13!12!) = 5,200,300 FACT: Sum_{k=0}^n C(n,k) = 2^n. PROOF: 2^n = (1+1)^n = Sum_{k=0}^n C(n,k)*1^k*1^{n-k} by Binomial Theorem = Sum_{k=0}^n C(n,k). An alternative proof: A set with n elements has 2^n subsets (we showed this was true earlier in the semester). There are C(n,0) = 1 subset of size 0, C(n,1) = n subsets of size 1, C(n,2) = n(n-1)/2 subsets of size 2, ..., and C(n,n) = 1 subset of size n. QED FACT: Sum_{k=0}^n C(n,k)*(-1)^k = 0. PROOF: Sum_{k=0}^n C(n,k)*(-1)^k = Sum_{k=0}^n C(n,k)*(-1)^k*1^{n-k} = ((-1)+1)^n by the Binomial Theorem = 0 QED FACT: Sum_{k=0}^n C(n,k)*2^k = 3^n. PROOF: Sum_{k=0}^n C(n,k)*2^k = Sum_{k=0}^n C(n,k)*2^k*1^{n-k} = (2+1)^n by the Binomial Theorem = 3^n. QED PASCAL'S IDENTITY: C(n+1,k) = C(n,k-1) + (n,k). PROOF: Suppose T is a set of n+1 elements, i.e. T = {a} U S, where S has n elements. The number of k-subsets of T is C(n+1,k). Some of the subsets of T contain element a and some don't. The number that don't is C(n,k) (choose k elements from the n elements in S) . The number that do is C(n,k-1) (choose the remaining k-1 elements from S). Add them together to get all the subsets of T. QED Pascals triangle: << draw Fig 1 on p. 367 >> VANDERMONDE'S IDENTITY: C(m+n,r) = Sum_{k=0}^r C(m,r-k)*C(n,k). PROOF: Think of a set S that is the union of two disjoint sets T (with m elements) and U (with n elements). C(m+n,r) is the number of r-subsets of S. In each r-subset of S, some of the elements come from U and some come from T. The number of elements from U can be 0 or 1 or 2 or ... The number of r-subsets of S with k elements from U is C(n,k) (the number of ways to choose k elements from U) plus C(m,r-k) (the number of ways to choose the remaining r-k elements from T). QED FACT: C(2n,n) = Sum_{k=0}^n C(n,k)^2. PROOF: Follows from Vandermonde's identity with m = n. QED ------------------------------------------------------------------------ 5.5 Generalized Permutations and Combinations ------------------------------------------------------------------------ THEOREM: Number of r-permutations of a set of n objects **with repetition allowed** is n^r. PROOF: Product rule. QED EXAMPLE: Number of strings of length r from {a,b,...,z} is 26^r. THEOREM: There are C(n+r-1,r) r-combinations from a set with n elements when repetition is allowed. PROOF: Think of n boxes, representing the n elements. Then think of how many of the r chosen elements are drawn from each box and mark with stars. Can represent like this, for instance, with 5 boxes and 6 stars. Bars are between boxes. This diagram: | * * | | * * * | * indicates choosing 0 copies of the first element 2 copies of the second element 0 copies of the third element 3 copies of the fourth element 1 copy of the fifth elemente. In general, we have n-1 bars and r stars to arrange. So think of n-1+r positions, to be filled with the bars and stars. We can choose any r of the n-1+r positions for the stars, and then insert the bars in the remaining positions. The number of ways to choose the r positions from the stars from n-1+r available positions is C(n-1+r,r). QED EXAMPLE: A bakery has 4 kinds of cookies. How many ways are there to choose 6 cookies? Let n = 4 and r = 6 in previous theorem. Then we have C(4+6+1,6) = C(9,6) = 84. THEOREM: Number of permutations of n objects, with n_1 indistinguishable objects of type 1, n_2 indistinguishable objects of type 2, ..., and n_k indistinguishable objects of type k, is n!/(n_1!*n_2!*...*n_k!). PROOF: The n_1 objects of type 1 can be placed in the n positions of the overall permutation in C(n,n_1) ways, leaving n - n_1 positions free. The n_2 objects of type 2 can be placed in the remaining free positions in C(n-n_1,n_2) ways. Etc. So we get C(n,n_1)*C(n-n_1,n_2)*...*(Cn-n_1-...-n_{k-1},n_k). By algebra, this is n!/(n_1!*n_2!*...*n_k!). QED EXAMPLE: How many different strings can be made by reordering letters in SUCCESS? 3 S's, so let n_1 = 3 1 U, so let n_2 = 1 2 C's, so let n_3 = 2 1 E, so let n_4 = 1. And n = 7. So previous theorem gives us 7!/(3!*1!*2!*1!) = 420.