Discrete Structures for Computing Notes 6 ------------------------------------------------------------------------ Chapter 4: Induction and Recursion ------------------------------------------------------------------------ Many mathematical statements assert that some property is true for all positive integers. Ex: A set with n elements has 2^n subsets Ex: The sum of the first n positive integers is n(n+1)/2 A powerful technique for proving such properties is MATHEMATICAL INDUCTION. ------------------------------------------------------------------------ 4.1 Mathematical Induction ------------------------------------------------------------------------ Our goal is to prove that a propositional function P(n) is true, where the universe is all positive integers. Mathematical induction is a way to do this, using a 2-step process: (1) BASIS STEP: Show that P(1) is true. (2) INDUCTIVE STEP: Show that the conditional statement P(k) -> P(k+1) is true, for all positive integers k. P(k) is called the INDUCTIVE HYPOTHESIS. How do we do step (2)? * Assume that P(k) is true. * Show that under this assumption P(k+1) is true. Don't get confused about the inductive step! We are not just assuming that P(k) is true for every k. We are showing that *if* P(k) is true, then P(k+1) is true. Why is this a valid proof technique? That is, suppose we've done steps (1) and (2). Why would this show that P(n) is true for all n? One intuitive explanation relates this to climbing an infinitely long ladder: * basis step tells how to get on the first rung of the ladder * inductive step tells how to go from one rung to the next rung If you can get on and if you can get from one rung to the other, then you can get to every rung. In other words, P(1) is shown directly in step (1) P(2) is true because P(1) is true and step (2) shows that P(1) -> P(2) P(3) is true because P(2) is true and step (2) shows that P(2) -> P(3) P(4) is true because P(3) is true and step (2) shows taht P(3) -> P(4) etc. I.e., (P(1) /\ Ak(P(k) -> P(k+1))) -> AnP(n) Example: -------- Prove 1 + 2 + ... + n = n(n+1)/2 using mathematical induction. More precisely, let P(n) be the proposition that 1 + 2 + ... + n = n(n+1)/2. Basis Step: Show P(1). That is, show that 1 = 1(1+1)/2. The arithmetic works. Inductive Step: Show that for all k, P(k) -> P(k+1). That is, if 1 + 2 + ... + k = k(k+1)/2 (the inductive hypothesis), then 1 + 2 + ... + k + (k+1) = (k+1)(k+2)/2. General approach: We get to use as given the inductive hypothesis P(k). We then look at P(k+1) and try to break the problem down into pieces. Often, we can break it into a big piece and a little piece. We then use the inductive hypothesis to take care of the big piece and then do some work directly to handle the remaining little piece of the problem. So let's consider how to show P(k+1). The sum of the first k+1 positive integers is the sum of the first k positive integers, plus k+1. So we can use the formula for the first k+1 positive integers, and then do some arithmetic. 1 + 2 + ... + k + (k+1) = k(k+1)/2 + (k+1) by inductive hypothesis We hope that this equals (k+1)(k+2)/2. Does it? Yes! QED Example: ------- Prove 1 + 3 + 5 + ... + (2n-1) = n^2 using mathematical induction. Let P(n) be the statement 1 + 3 + 5 + ... + (2n-1) = n^2. Basis Step: Prove P(1). Substitute 1 for n in the equation: 1 = 2*1 - 1 = 1^2. Inductive Step: * Assume P(k) is true, i.e., assume 1 + 3 + 5 + ... + (2k-1) = k^2. * Prove that P(k+1) is true, i.e., prove 1 + 3 + 5 + ... + (2(k+1)-1) = (k+1)^2. Try same method as previous example: 1 + 3 + 5 + ... + (2k-1) + (2(k+1)-1) = k^2 + (2(k+1)-1) by the inductive hypothesis = k^2 + 2k + 2 - 1 = (k+1)^2 by algebra QED Sometimes we would like to prove something is true for all the nonnegative integers. We can still use mathematical induction, we just change where we start from. So the basis step will be for 0, instead of 1. Example: ------- Prove 1 + 2 + 2^2 + ... + 2^n = 2^{n+1} - 1 using mathematical induction, starting with n = 0. Let P(n) be the statement 1 + 2 + 2^2 + ... + 2^n = 2^{n+1} - 1. Basis step: Show P(0). That is, show that 2^0 = 2^1 - 1. Yes. Inductive Step: Assume P(k). That is, assume 1 + 2 + 2^2 + ... + 2^k = 2^{k+1} - 1. Now show P(k). That is, show 1 + 2 + 2^2 + ... + 2^k + 2^{k+1} = 2^{k+2} - 1. Try our usual strategy: 1 + 2 + 2^2 + ... + 2^k + 2^{k+1} = 2^{k+1} - 1 + 2^{k+1} = 2*2^{k+1} - 1 = 2^{k+2} - 1. QED More generally, we can start an inductive proof anywhere (doesn't have to be 0 or 1). Example: -------- Use mathematical induction to prove P(n), where P(n) is the statement Sum_{j=0}^n a*r^j = (a*r^{n+1} - a)/(r-1) when r != 1. Basis step: Show P(0) is true. (p. 270) QED You can also use mathematical induction to prove *inequalities*. Example: -------- Prove n < 2^n. Let P(n) be the statement that n < 2^n. Basis Step: Show P(1) is true. That is, show 1 < 2^1. OK. Inductive Step: Assume P(k) is true and show P(k+1) is true. That is, assume k < 2^k and show k+1 < 2^{k+1}. k + 1 < 2^k + 1 by inductive hypothesis < 2^k + 2^k since k >= 1 = 2*2^k = 2^{k+1}. QED Example: -------- Prove 2^n < n! for all n >= 4. (Note that when n = 1, 2^1 !< 1! since 2 > 1 when n = 2, 2^2 !< 2! since 4 > 2 when n = 3, 2^3 !< 3! since 8 > 6.) Let P(n) be the statement that 2^n < n!. Basis step: Show P(4) is true. That is, show 2^4 < 4!. This is true since 16 < 24. Inductive step: Assume P(k) is true for any k >= 4, and show P(k+1) is true. That is, assume 2^k < k! and show 2^{k+1} < (k+1)!. 2^{k+1} = 2*2^k < 2*(k!) by inductive hypothesis < (k+1)*(k!) since 2 < k+1 (remember k >= 4) = (k+1)! QED Example: -------- Use mathematical induction to prove that n^3 - n is divisible by 3 for all positive integers n. Let P(n) be the statement that n^3 - n is divisible by 3. Basis step: Show P(1). That is, show that 1^3 - 1 = 0 is divisible by 3. OK. Inductive step: Assume P(k) is true and show P(k+1) is true. That is, assume k^3 - k is divisible by 3 and show (k+1)^3 - (k+1) is divisible by 3. (k+1)^3 - (k_1) = k^3 + 3k^2 + 3k + 1 by algebra = (k^3 - k) + 3(k^2 + k) by algebra First term is divisible by 3, by inductive hypothesis. Second term is divisible by 3, since 3 is an explicit factor. Thus the entire expression is divisible by 3. QED Example: ------- Use mathematical induction to prove that if a set has n elements, then it has 2^n different subsets. Let P(n) be the statement that a set with n elements has 2^n different subsets. Basis step: Show P(0) is true. That is, show that if a set has no elements (is the empty set), then it has 2^0 = 1 subset. This is true since the only subset of the empty set is the empty set itself. Inductive step: Assume P(k) is true and show P(k+1) is true. That is, assume that any set with k elements has 2^k subsets, and prove that any set with k+1 elements has 2^{k+1) subsets. Let T be any set with 2^{k+1} subsets. Pick any element a in T, and let S be T - {a}. Since S has k subsets, it has 2^k subsets. For each subset X of S, there are two subsets of T: X and X U {a}. Thus there are twice as many subsets of T as there are of S, which makes 2*2^k = 2^{k+1}. QED Beware of errors in inductive proofs! Example of Erroneous Proof: --------------------------- Let P(n) be the statement that every set of n lines in the plane such that no two of the lines are parallel meet in a common point. Basis: Show P(2) is true. I.e., every pair of lines in the plane that are not parallel meet in a common point. This is obviously true. Inductive Step: Assume P(k) is true and show P(k+1) is true. Suppose the lines are L_1, L_2, ..., L_{k+1}. By the inductive hypothesis, L_1, ..., L_k meet in a common point, P1. Also by the inductive hypothesis, L_2, ..., L_{k+1} meet in a common point, P2. We now show P1 = P2. Assume in contradiction P1 != P2. Then the lines L_2, ..., L_k that contain both P1 and P2 must actually be the same line, since two points determine a line. But this contradicts the assumption that none of the lines are parallel. QED Where is the error? When n = 3, the set of lines containing both P1 and P2 consists solely of L_2. So there is no contradiction to assuming P1 != P2. ------------------------------------------------------------------------ 4.2 Strong Induction ------------------------------------------------------------------------ Sometimes in trying to do the inductive step of a proof by mathematical induction, it is not sufficient to assume that P(k) is true in order to prove P(k+1). Instead, we need to rely on some of the other previous values. STRONG INDUCTION is like regular induction, except that in the inductive step, we assume that P(1), P(2), ..., and P(k) are true in order to prove P(k+1). Example: Show that every integer greater than 1 can be written as the product of prime numbers. Let P(n) be the proposition that n equals the product of primes. We want to show P(n) is true for all n > 1. Basis Step: Show that P(2) is true. 2 = 2, done. Inductive Step: Assume P(2), ..., P(k) are true. I.e., *every* integer between 2 and k can be written as the product of primes (not just k). Show P(k+1), that k+1 can be written as the product of primes. Case 1: k+1 is a prime number. Then k+1 = k+1 and we are done. Case 2: k+1 is not a prime number. Then k+1 = a*b, where a and b are both integers between 2 and k. *** here is where we use the inductive hypothesis for strong induction! We cannot assume that a and b are *equal* to k. All we now is that they are between 2 and k. So we need to use the assumption that every number between 2 and k can be written as the product of primes. Let a = p_1 * p_2 * ... * p_x, where all the p_i's are primes. Let b = q_1 * q_2 * ... * q_y, where all the q_i's are primes. Then k+1 = a*b = p_1 * p_2 * ... * p_x * q_1 * q_2 * ... * q_y, which is the product of primes. QED MORE COMPLICATED BASIS CASES: With strong induction, we can handle situations where we have more than one basis case. There might be a situation where we need to do a direct proof of several small cases before we can do the inductive step. Example: Prove that if all you have are 4-cent stamps and 5-cent stamps, you can still make every amount of postage greater than or equal to 12 cents. Let P(n) be the statement that postage of n cents can be made using 4-cent and 5-cent stamps. Basis step(s): * 12 cents: 4 + 4 + 4 * 13 cents: 4 + 4 + 5 * 14 cents: 4 + 5 + 5 * 15 cents: 5 + 5 + 5 So we've proved P(12), P(13), P(14), and P(15). Inductive step: Assume P(12), P(13), ..., P(k) are true, and show that P(k+1) is true, where k >= 15. In particular, we can assume that P(k-3) is true. Why? Because k-3 is at leat 12, since k is at least 15. Now we can see that k+1 = (k-3) + 4. So we can use the postage for k-3 and add a 4-cent stamp to it. QED The next example has to do with computational geometry. Theorem: A simple polygon with n sides, n >= 3, can be triangulated into n-2 triangles. POLYGON: closed geometric figure consisting of a sequence of line segments, or SIDES with consecutive sides meeting at a common endpoint. A common endpoint is called a VERTEX. SIMPLE polygon: Nonconsecutive sides do not intersect. Pictures (p. 288) DIAGONAL: a line segment (other than a side) connecting two vertices. Some diagonals are INTERIOR (lie totally within the polygon), while others are not. TRIANGULATE: divide into triangles by adding nonintersecting diagonals. (Can do so in more than one way.) Pictures (p. 289): Now we know what the theorem means. We can prove it using strong induction. But we do need a basic lemma (helping fact) about polygons: LEMMA: Every simple polygon has an interior diagonal. We will not prove this (you can read the proof in the book: it is not a proof by induction). But we will use it to prove the theorem. Let T(n) be the statement that every simple polygon with n sides can be triangulated into n-2 triangles, n >= 3. BASIS STEP: Show T(3) is true, i.e., that every triangle can be triangulated. This is trivial, don't even need to add any diagonals. INDUCTIVE STEP: Assume T(3), T(4), ..., T(k) are all true and show that T(k+1) is true. Let P be a simple polygon with k+1 sides. Use Lemma 1 to split P into two pieces with an interior diagonal ab: The two pieces of P are also simple polygons. Call them Q and R. We don't know exactly how many sides Q and R have, as this depends on exactly what P looks like and exactly where ab is. But we can make some deductions about the number of sides s of Q and and the number of sides t of R: (a) s >= 3 and s <= (k+1) - 1 = k (b) t >= 3 and t <= (k+1) - 1 = k (c) k+1 = s + t - 2 Because of (a) and (b) above, we can use the (strong) inductive hypothesis to triangulate Q and R. Triangulation of Q produces s - 2 triangles. Triangulation of R produces t - 2 triangles. The resulting triangulation of P has s - 2 + t - 2 = s + t - 4 triangles. By (c) above, s + t - 4 = (k+1) - 2, so we have the desired number of triangles of P. QED ------------------------------------------------------------------------ 4.3 Recursive Definitions and Structural Induction ------------------------------------------------------------------------ Recursion is a way to define something in terms of itself. The trick is to make sure that the definition is not circular. The way to do it is similar to the inductive proof procedure we've been studying: * first, explicitly define a few objects * then, give rules for building new objects out of existing objects We can then prove properties of these objects using a form of induction called structural induction. RECURSIVELY DEFINED FUNCTION (with nonneg ints as domain): BASIS STEP: Specify f(0). INDUCTIVE STEP: Give a rule for specifying f(n+1) using some or all of f(0), ..., f(n). Example: * f(0) = 3 * f(n+1) = 2*f(n) + 3 Notice that f(1) = 9, f(2) = 21, f(3) = 45, f(4) = 93,... Example: Give a recursive definition of n! (factorial). * F(0) = 1 * F(n+1) = (n+1)*F(n) Notice that F(5) = 5*F(4) = 5*4*F(3) = 5*4*3*F(2) = 5*4*3*2*F(1) = 5*4*3*2*1*F(0) = 5*4*3*2*1*1 = 120. Example: Fibonacci numbers: * Basis: f_0 = 0, f_1 = 1 (two base cases) * Inductive: f_n = f_{n_1} + f_{n-2} for all n >= 2 Notice that * f_2 = f_1 + f_0 = 1 + 0 = 1 * f_3 = f_2 + f_1 = 1 + 1 = 2 * f_4 = f_3 + f_2 = 2 + 1 = 3 * f_5 = f_4 + f_3 = 3 + 2 = 5 Now we can prove properties about the Fibonacci numbers using induction, mirroring the inductive definition. Example: Show that f_n > ((1 + sqrt(5))/2)^{n-2} for all n >= 3. Let P(n) be the statement f_n > ((1 + sqrt(5))/2)^{n-2}. Basis step: A strong hint that we'll need two base cases in the proof is that the definition of Fibonacci numbers uses two base cases. P(3) is the statement f_3 > ((1 + sqrt(5))/2)^{3-2}. Since f_3 = 2 and (1 + sqrt(5))/2 is approx 1.62, P(3) is true. P(4) is the statement f_4 > ((1 + sqrt(5))/2)^{4-2}. Since f_4 = 3 and ((1 + sqrt(5))/2)^2 is approx 2.62, P(4) is true. Inductive step: Assume P(3), ..., P(k) are true and show P(k+1) is true. I.e., we need to show f_{k+1} > ((1 + sqrt(5))/2)^{k-1}. Let's use the recursive definition of f_{k+1} and see what we can get. Let alpha = (1 + sqrt(5))/2. f_{k+1} = f_k + f_{k-1} by definition of f_{k+1} > alpha^{k-2} + alpha^{k-3} by inductive hypothesis >= alpha^{k-1} ??? we would like this to be true. It turns out that alpha^{k-2} + alpha^{k-3} = alpha^{k-1}. Why? Let's assume this equality and see what it tells us about alpha. Divide both sides by alpha^{k-2} and get 1 + alpha = alpha^2 Equivalent to 0 = alpha^2 - alpha - 1 Solving for alpha using the quadratic equation tells us that alpha = (1 + sqrt(5))/2 or alpha = (1 - sqrt(5))/2 QED ---------- RECURSIVELY DEFINED SETS BASIS STEP: explicitly list a few elements that are in the set INDUCTIVE STEP: give rules for forming new elements of the set using those already known to be in the set. EXCLUSION RULE: Nothing else is in the set. ---------- Example: Define a set S of integers like this: * basis step: 3 is in S * recursive step: if x and y are in S, then x+y is in S. We can build up the elements of S: 3, 3+3 = 6, 3+6 = 9, 3+9 = 12, ... Intuitively, S is all positive multiples of 3. We will shortly see how to prove this rigorously. ---------- Example: The set Sigma^* of STRINGS over the alphabet (set of symbols) Sigma is defined: * the empty string lambda is in Sigma^* * if w is in Sigma^* and x is in Sigma, then wx is in Sigma^*. Example: Suppose Sigma = {0,1}. Then Sigma* contains lamda, lambda0 = 0, lambda1 = 1, 00, 01, 10, 11, etc. ---------- Example: The set of well-formed formulas (WFF) of propositional logic * Basis step: - T is in WFF - F is in WFF - s is in WFF for each propositional variable s * recursive step: Suppose E and F are in WFF. Then - (~E) is in WFF - (E /\ F) is in WFF - (E \/ F) is in WFF - (E -> F) is in WFF - (E <-> F) is in WFF So we can show, for instance, that (p -> F) -> F) is a well-formed formula. You can check that (p ~ /\ q) is not. ---------- A GRAPH is a structure consisting of VERTICES and EDGES (an edge connects two vertices). A ROOTED TREE is a special kind of graph that contains a distinguished vertex called the ROOT. The set of rooted trees is defined recursively: BASIS STEP: A single vertex r is a rooted tree and r is the root. RECURSIVE STEP: Suppose T1, T2, ..., Tn are disjoint rooted trees (i.e., have no vertices in common). Let r1, r2, ..., rn be the roots of the trees. Construct a new rooted tree as follows: * Let r be a "new" vertex, not in any of the trees T1, T2, ..., Tn. * add an edge from r to each of r1, r2, ..., rn. Examples: << Fig 2, p. 302 >> ---------- A special case of rooted trees are BINARY TREES. This structure is used extensively in computer science. There are several different flavors of binary tree. Here are recursive definitions of two version. EXTENDED BINARY TREE: <<< perhaps skip this definition >>> BASIS STEP: the empty set (no vertices and no edges) is an extended binary tree. RECURSIVE STEP: Suppose T1 and T2 are disjoint (no vertices in common) extended binary trees. Construct a new extended binary tree as follows: * Let r be a "new" vertex, not in T1 or T2. * If T1 is not the empty set, then add an edge from r to the root of T1. * If T2 is not the empty set, then add an edge from r to the root of T2. (Sometimes T1 and T2 are distinguished as "left" and "right" subtrees of the new tree.) Examples: << Fig 3, p. 303 >> ---------- FULL BINARY TREE: BASIS STEP: Any single vertex is a full binary tree. RECURSIVE STEP: Suppose T1 and T2 are disjoint full binary trees. Construct a new full binary tree as follows: * Let r be a "new" vertex, not in T1 or T2. * Add an edge from r to the root of T1. * Add an edge from r to the root of T2. (Sometimes T1 and T2 are distinguished as "left" and "right" subtrees of the new tree.) Examples: << Fig 4, p. 304 >> (Intuitive difference between extended and full binary trees: In an extended binary tree, every node has either 0, 1 or 2 children. In a full binary tree, every node has either 0 or 2 children.) --------------- STRUCTURAL INDUCTION: A way to prove something, using mathematical induction, about a recursively defined set. It works like this: BASIS STEP: Prove that the property is true for the elements of the step defined in the basis step of the definition. INDUCTIVE STEP: Assume that the property is true for the elements used in the recursive step of the definition to construct the new element. Prove that the property is true of the newly constructed element. In essence, structural induction is doing regular induction on the number of times that the recursive step is employed to create a new element. ---------- Example: Recall the set S defined recursively like this: BASIS: 3 is in S. RECURSIVE STEP: If x and y are in S, then x + y is in S. We can prove rigorously that S consists of all positive integers that are divisible by 3. Proof: Let A be the set of all positive integers divisible by 3. I.e., A = {3n : n > 0}. We want to prove A = S. First show A is a subset of S. Use (regular) induction. Let P(n) be the statement "3n is an element of S". We want to show P(n) is true for n = 1, 2, 3, ... Basis Step: P(1): 3*1 is an element of S. Since 3*1 = 3, this follows from the basis step of the definition of S. Inductive Step: Assume P(k) is true and show P(k+1) is true. P(k) states that 3*k is in S. Then by the recursive step of the definition of S, 3*k + 3 is in S. Since 3*k + 3 = 3(k+1), and 3(k+1) is in A, P(k+1) is true. Now show S is a subset of A. Use structural induction. Basis Step: The basis element of S is 3. So show 3 is is in A. True since 3 = 3*1. Inductive Step: The recursive step of the definition of S says that if x and y are in S, then x + y is in S. Assume x and y are in A. Then show x + y is in A. (Notice how we are mirroring the recursive step of the definition of S.) Let x = 3*a and y = 3*b. Then x + y = 3*(a+b), so x + y is in A. QED ---------- Recall the recursive definition of the set of well-formed formulas WFF: Basis Step: T, F, and every propositional variable s are in WFF. Recursive Step: If E and F are in WFF, then so are (~E), (E /\ F), (E \/ F), (E -> F), and (E <-> F). Theorem: Every well-formed formula contains an equal number of left and right parentheses. Proof: (By structural induction.) Basis step: Show the result is true for elements of WFF defined in the basis step. Each one of them has 0 left parens and 0 right parens. Inductive step: Pick any two elements E and F in WFF. Assume E has same number of left and right parens (call this number x), and F has same number of left and right parens (call thsi number y). Now consider all the ways that a new WFF can be formed using E and F, according to the recursive step of the definition: * (~E) : has x + 1 left parens and x + 1 right parens. * (E /\ F) : has x + y + 1 left parens and x + y + 1 right parens * (E \/ F) : ditto * (E -> F) : ditto * (E <-> F) : ditto QED -------------- Example: Recall the definition of full binary tree from above. Definition: The HEIGHT of a full binary tree T, h(T), is defined recursively: * Basis step: If T consists of a single vertex, then h(T) = 0. * Recursive step: If T is constructed out of T1 and T2, then h(T) = 1 + max(h(T1),h(T2)). Example of heights: Definition: Let T be a full binary tree. Then n(T), the number of vertices in T, is defined recursively like this: * Basis step: If T consists of a single vertex, then n(T) = 1. * Recursive step: If T is constructed out of T1 and T2, then n(T) = 1 + n(T1) + n(T2). Theorem: If T is a full binary tree, then n(T) <= 2^{h(T)+1} - 1. Proof: (Use structural induction.) Basis step: Consider full binary tree consisting of a single vertex. By definition, h(T) = 0 and n(T) = 1. Check formula: 1 <= 2^{0+1} - 1. True. Inductive step: Let T1 and T2 be any two full binary trees. Assume T1 and T2 satisfy the formula. That is, * n(T1) <= 2^{h(T1)+1} - 1 and * n(T2) <= 2^{h(T2)+1} - 1. Show that the full binary tree T constructed from T1 and T2 satisfies the formula. That is, * n(T) <= 2^{h(T)+1} - 1. By the recursive steps in the definitions, * n(T) = 1 + n(T1) + n(T2) and * h(T) = 1 + max(h(T1),h(T2)). Let's see how to combine the facts we know to get the desired goal: n(T) = 1 + n(T1) + n(T2) by recursive step in def. of n(T) <= 1 + (2^{h(T1)+1} - 1) + (2^{h(T2)+1} - 1) by inductive hypothesis = 2^{h(T1)+1} + 2^{h(T2)+1} - 1 by algebra <= 2*max(2^{h(T1)+1}, 2^{h(T2)+1}) - 1 since sum of two terms is at most twice the larger = 2*2^{max(h(T1),h(T2))+1 - 1 since max(2^x,2^y) = 2^{max(x,y)} = 2*2^{h(T)} - 1 by recursive step in def. of h(T) = 2^{h(T)+1} - 1 by algebra QED