D J J) Y COMPETITIVE PROGRAMMING PART -2 $ x Scanned with CamScanner6 Greedy algorithms 6.1 Coin problem 62 Scheduling........ 6.3 Tasks and deadlines 6.4 Minimizing sums 6.5 Data compression 7 Dynamic programming 7.1 Coin problem ........ 7.2 Longest increasing subsequence 7.3 Pathsinagrid ....... 7.4 Knapsack problems 7.5 Editdistance ........ 7.6 Counting tilings 8 Amortized analysis 8.1 Two pointers method 8.2. Nearest smaller elements . 8.3. Sliding window minimum . beceeeee 9 Range queries 9.1 Static array queries... . 92 Binary indexed tree 9.3 Segment tree . . . 9.4 Additional techniques 10 Bit manipulation 10.1 Bit representation . . . 10.2 Bit operations . . . 10.3 Representing sets 10.4 Bit optimizations 10.5 Dynamic programming II Graph algorithms 11 Basics of graphs 11.1 Graph terminology 11.2 Graph representation . . . 12 Graph traversal 12.1 Depth-first search 12.2 Breadth-first search .. . . 123 Applications 37 87 58 60 61 62 65 65 70 7 72 74 15 7 7 79 ceeeeeeeeeee BL 83 84 86 89 93 95 95 96 98 100 102 107 109 109 113 17 17 119 121 Scanned with CamScannerChapter 6 Greedy algorithms A greedy algorithm constructs a solution to the problem by always making a choice that looks the best at the moment. A greedy algorithm never takes back its choices, but directly constructs the final solution. For this reason, greedy algorithms are usually very efficient. ‘The difficulty in designing greedy algorithms is to find a greedy strategy that always produces an optimal solution to the problem. The locally optimal choices ina greedy algorithm should also be globally optimal. It is often difficult to argue that a greedy algorithm works. Coin problem As a first example, we consider a problem where we are given a set of coins and our task is to form a sum of money n using the coins. The values of the coins are coins = (e1,¢2,...,c#l, and each coin can be used as many times we want. What is the minimum number of coins needed? For example, if the coins are the euro coins (in cents) {1,2,5,10,20,50, 100,200} and n = 520, we need at least four coins. The optimal solution is to select coins 200 + 200 + 100 +20 whose sum is 520. Greedy algorithm A simple greedy algorithm to the problem always selects the largest possible coin, until the required sum of money has been constructed. This algorithm works in the example case, because we first select two 200 cent coins, then one 100 cent coin and finally one 20 cent coin. But does this algorithm always work? It turns out that if the coins are the euro coins, the greedy algorithm always works, ie,, it always produces a solution with the fewest possible number of coins. ‘The correctness of the algorithm can be shown as follow: First, each coin 1, 5, 10, 50 and 100 appears at most once in an optimal solution, because if the solution would contain two such coins, we could replace 57 Scanned with CamScannerthem by one coin and obtain a better solution. For example, if the solution would contain coins 5 +5, we could replace them by coin 10. In the same way, coins 2 and 20 appear at most twice in an optimal solution, because we could replace coins 2+2+2 by coins 5 +1 and coins 20 +20 +20 by coins 50+10. Moreover, an optimal solution cannot contain coins 2+2+1 or 20+20+10, because we could replace them by coins 5 and 50. Using these observations, we can show for each coin x that it is not possible to optimally construct a sum x or any larger sum by only using coins that are smaller than x. For example, if x = 100, the largest optimal sum using the smaller coins is 50+20+20+5+2+2=99. Thus, the greedy algorithm that always selects the largest coin produces the optimal solution. This example shows that it can be difficult to argue that a greedy algorithm works, even if the algorithm itself is simple, General case In the general case, the coin set can contain any coins and the greedy algorithm does not necessarily produce an optimal solution. We can prove that a greedy algorithm does not work by showing a counterex- ample where the algorithm gives a wrong answer. In this problem we can easily find a counterexample: if the coins are {1,3,4) and the target sum is 6, the greedy algorithm produces the solution 4 +1+1 while the optimal solution is 3+ 3. It is not known if the general coin problem can be solved using any greedy algorithm’. However, as we will see in Chapter 7, in some cases, the general problem can be efficiently solved using a dynamic programming algorithm that always gives the correct answer. Scheduling Many scheduling problems can be solved using greedy algorithms. A classic problem is as follows: Given n events with their starting and ending times, find a schedule that includes as many events as possible. It is not possible to select an event partially. For example, consider the following events: event starting time ending time 1 3 bamD 2 5 3 9 6 8 In this case the maximum number of events is two. For example, we can select events B and D as follows: THowever, itis possible to check in polynomial time if the greedy algorithm presented in this chapter works for a given set of eains (53) 58 Scanned with CamScannerIt is possible to invent several greedy algorithms for the problem, but which of them works in every case? Algorithm 1 ‘The first idea is to select as short events as possible. In the example case this algorithm selects the following events: es) Cc) bo) However, selecting short events is not always a correct strategy. For example, the algorithm fails in the following case: CI Co Coe If we select the short event, we can only select one event, However, it would be possible to select both long events. Algorithm 2 Another idea is to always select the next possible event that begins as early as possible. This algorithm selects the following events: a co) oo However, we can find a counterexample also for this algorithm. For example, in the following case, the algorithm only selects one event: eee co Co If we select the first event, it is not possible to select any other events. However, it would be possible to select the other two events. 59 Scanned with CamScannerAlgorithm 3 ‘The third idea is to always select the next possible event that ends as early as possible. This algorithm selects the following events: — Be Cc) bo It turns out that this algorithm always produces an optimal solution. The reason for this is that it is always an optimal choice to first select an event that ends as early as possible. After this, it is an optimal choice to select the next event using the same strategy, ete., until we cannot select any more events. ‘One way to argue that the algorithm works is to consider what happens if we first select an event that ends later than the event that ends as early as possible. Now, we will have at most an equal number of choices how we can select the next event. Hence, selecting an event that ends later can never yield a better solution, and the greedy algorithm is correct. Tasks and deadlines Let us now consider a problem where we are given n tasks with durations and deadlines and our task is to choose an order to perform the tasks. For each task, we earn d-x points where d is the task’s deadline and x is the moment when we finish the task. What is the largest possible total score we can obtain? For example, suppose that the tasks are as follows: task duration deadline A 4 2 B 3 5 c 2 7 D 4 5 In this case, an optimal schedule for the tasks is as follows: 0 5 10 (CB a In this solution, C yields 5 points, B yields 0 points, A yields -7 points and D yields 8 points, so the total score is -10. Surprisingly, the optimal solution to the problem does not depend on the deadlines at all, but a correct greedy strategy is to simply perform the tasks sorted by their durations in increasing order. The reason for this is that if we ever perform two tasks one after another such that the first task takes longer than the second task, we can obtain a better solution if we swap the tasks. For example, consider the following schedule: 60 Scanned with CamScannerHere a > b, so we should swap the tasks: yk) 6 a Now X gives b points less and ¥ gives a points more, so the total score increases by a—b > 0. In an optimal solution, for any two consecutive tasks, it must hold that the shorter task comes before the longer task. Thus, the tasks must be performed sorted by their durations. Minimizing sums We next consider a problem where we are given n numbers 1,2, task is to find a value x that minimizes the sum a, and our Jays" + lag —x]° 40+ lan = 21°. We focus on the cases ¢ = 1 and ¢=2. Case c=1 In this case, we should minimize the sum Jay x] + lay —x] +--+ Jay —2| For example, if the numbers are [1, which produces the sum 9,2,6], the best solution is to select x= 2 [1-2] + 12-2) +|9-2| + 12-2) + 16-2) = 12. In the general case, the best choice for x is the median of the numbers, ie., the middle number after sorting. For example, the list [1,2,9,2,6] becomes [1,2,2, 6,9] after sorting, so the median is 2. ‘The median is an optimal choice, because if x is smaller than the median, the sum becomes smaller by increasing x, and if x is larger then the median, the sum becomes smaller by decreasing x. Hence, the optimal sohution is that x is the median. If n is even and there are two medians, both medians and all values between them are optimal choices. 2 In this case, we should minimize the sum Case c (ay =x) +(a2— 2) +--+ (an— 2)" 61 Scanned with CamScannerFor example, if the numbers are [1,2,9,2,6], the best solution is to select x= 4 which produces the sum (1-4)? +(2-4)? + (9-4)? + (2-4)? + (6 - 4)? = 46, In the general case, the best choice for x is the average of the numbers. In the example the average is (1+2+9+2+6)5=4. This result can be derived by presenting the sum as follows: nx* -2x(ay +a +--+ ay) + (a4 +03 +---+a2) ‘The last part does not depend on x, so we can ignore it. The remaining parts form a function nx? -2xs where | +az+---+ay. This is a parabola opening upwards with roots x = 0 and x =2s/n, and the minimum value is the average of the roots x= s/n, ie,, the average of the numbers ay,a2,...,d. Data compression A binary code assigns for each character of a string a codeword that consists of bits, We can compress the string using the binary code by replacing each character by the corresponding codeword. For example, the following binary code assigns codewords for characters A-D: character codeword a 00 8 o1 c 10 o cn This is a constant-length code which means that the length of each codeword is the same. For example, we can compress the string AABACDACA as follows: 000001001011001000 Using this code, the length of the compressed string is 18 bits. However, we can compress the string better if we use a variable-length code where codewords may have different lengths. Then we can give short codewords for characters that appear often and long codewords for characters that appear rarely. It turns out that an optimal code for the above string is as follows: character codeword a 0 8 110 c 10 o 11 An optimal code produces a compressed string that is as short as possible. In this case, the compressed string using the optimal cod 001100 101110100, 62 Scanned with CamScannerso only 15 bits are needed instead of 18 bits. Thus, thanks to a better code it was possible to save 3 bits in the compressed string. We require that no codeword is a prefix of another codeword. For example, it is not allowed that a code would contain both codewords 10 and 1011. The reason for this is that we want to be able to generate the original string from the compressed string. If a codeword could he a prefix of another codeword, this would not always be possible. For example, the following code is not valid: character codeword a 10 8 u c 1011 D a Using this code, it would not be possible to know if the compressed string 1011 corresponds to the string AB or the string C. Huffman coding Huffman coding? is a greedy algorithm that constructs an optimal code for compressing a given string. The algorithm builds a binary tree based on the frequencies of the characters in the string, and each character's codeword can be read by following a path from the root to the corresponding node. A move to the left corresponds to bit 0, and a move to the right corresponds to bit 1 Initially, each character of the string is represented by a node whose weight is the number of times the character occurs in the string. Then at each step two nodes with minimum weights are combined by creating a new node whose weight is the sum of the weights of the original nodes. The process continues until all nodes have been combined. ‘Next we will see how Huffman coding creates the optimal code for the string AABACDACA, Initially, there are four nodes that correspond to the characters of the string: ® O®® @ A 8 c D ‘The node that represents character A has weight 5 because character A appears 5 times in the string. The other weights have been calculated in the same way. The first step is to combine the nodes that correspond to characters 8 and D, both with weight 1. The result is: 2D. A. Huffman discovered this method when solving a university course assignment and published the algorithm in 1952 [40], 63 Scanned with CamScannerAfter this, the nodes with weight 2 are combined: Now all nodes are in the tree, so the code is ready. The following codewords can be read from the tree: character codeword a 0 8 110 c 10 D a 64 Scanned with CamScannerChapter 7 Dynamic programming Dynamic programming is a technique that combines the correctness of com- plete search and the efficiency of greedy algorithms. Dynamic programming can be applied if the problem can be divided into overlapping subproblems that can be solved independently. ‘There are two uses for dynamic programming: + Finding an optimal solution: We want to find a solution that is as large as possible or as small as possible. * Counting the number of solutions: We want to calculate the total num- ber of possible solutions, We will first see how dynamic programming can be used to find an optimal solution, and then we will use the same idea for counting the solutions. Understanding dynamic programming is a milestone in every competitive programmer's career. While the basic idea is simple, the challenge is how to apply dynamic programming to different problems. This chapter introduces a set of classic problems that are a good starting point, Coin problem We first focus on a problem that we have already seen in Chapter 6: Given a set, of coin values coins ={¢1,¢2,...,¢g) and a target sum of money n, our task is to form the sum n using as few coins as possible. In Chapter 6, we solved the problem using a greedy algorithm that always chooses the largest possible coin. The greedy algorithm works, for example, when the coins are the euro coins, but in the general case the greedy algorithm does not necessarily produce an optimal solution, ‘Now is time to solve the problem efficiently using dynamic programming, so that the algorithm works for any coin set. The dynamic programming algorithm is based on a recursive function that goes through all possibilities how to form the sum, like a brute force algorithm. However, the dynamic programming algorithm is efficient because it uses memoization and calculates the answer to each subproblem only once. 65 Scanned with CamScannerRecursive formulation The idea in dynamic programming is to formulate the problem recursively so that the solution to the problem can be calculated from solutions to smaller subproblems. In the coin problem, a natural recursive problem is as follows: what is the smallest number of coins required to form a sum x? Let solve(x) denote the minimum number of coins required for a sum x. The values of the function depend on the values of the coins. For example, if coins = {1,8,41, the first values of the function are as follows: solve(0) = solve(1) solve(2) = solve(3) solve(4) = solve(5) solve(6) = solve(7) solve(8) = solve(9) solve(10) For example, solve(10) = 3, because at least 3 coins are needed to form the sum 10. The optimal solution is 3+3+4= 10. ‘The essential property of solve is that its values can be recursively calculated from its smaller values. The idea is to focus on the first coin that we choose for the sum. For example, in the above scenario, the first coin can be either 1, 3 or 4. If we first choose coin 1, the remaining task is to form the sum 9 using the minimum number of coins, which is a subproblem of the original problem. Of course, the same applies to coins 3 and 4. Thus, we can use the following recursive formula to calculate the minimum number of coins: ENR MEE HOS " cy solve(x) = min(solve(x—1) +1, solve(x—-3)+1, solve(x—4)+1). ‘The base case of the recursion is solve(0) = 0, because no coins are needed to form an empty sum. For example, solve(10) = solve(7)+ 1 = solve(4) + 2 = solve(0)+3=3. Now we are ready to give a general recursive function that calculates the minimum number of coins needed to form a sum x: co x<0 solve(x)=40 x=0 Mincccoirs SOlve(t—c)+1 x >0 First, if x <0, the value is co, because it is impossible to form a negative sum. of money. Then, if x= 0, the value is 0, because no coins are needed to form an 66 Scanned with CamScannerempty sum. Finally, if x > 0, the variable c goes through all possibilities how to choose the first coin of the sum. Once a recursive function that solves the problem has been found, we can directly implement a solution in C++ (the constant INF denotes infinity): int solve(int x) { if (x < @) return INF; if (x == @) return @ int best = INF; for (auto € : coins) ( best = min(best, solve(x-c)*1); ) return best; > Still, this function is not efficient, because there may be an exponential number of ways to construct the sum. However, next we will see how to make the function efficient using a technique called memoization, Using memoization ‘The idea of dynamic programming is to use memoization to efficiently calculate values of a recursive function. This means that the values of the function are stored in an array after calculating them. For each parameter, the value of the function is calculated recursively only once, and after this, the value can be directly retrieved from the array. In this problem, we use arrays bool readyiN]; int _value(N]; where ready[] indicates whether the value of solve() has been calculated, and if it is, value[x] contains this value. The constant N has been chosen so that, all required values fit in the arrays. ‘Now the function can be efficiently implemented as follows: int solve(int x) ( if (x < @) return INF; if (x == @) return @; if (readytx]) return valueCx]; int best = INF; for (auto € : coins) ( best = min(best, solve(x-c)+1); y valuelx] ready[x] return best; 67 Scanned with CamScanner‘The function handles the base cases x <0 and x =0 as previously. Then the function checks from ready(-] if solve(x) has already been stored in value[.x], and if it is, the function directly returns it. Otherwise the function calculates the value of solve(x) recursively and stores it in value[-x] This function works efficiently, because the answer for each parameter x is calculated recursively only once. After a value of solve(x) has been stored in valuefs!], it can be efficiently retrieved whenever the function will be called again with the parameter x. The time complexity of the algorithm is O(nk), where n is the target sum and k is the number of coins, Note that we can also iteratively construct the array value using a loop that, simply calculates all the values of solve for parameters 0...n: valuele] for (int x value[x] = INF for (auto € : coins) { if (re >= 0) { value(x] = min(value[x], value(x-c]+1); x) ( d ? In fact, most competitive programmers prefer this implementation, because it is shorter and has lower constant factors. From now an, we also use iterative implementations in our examples. Still, it is often easier to think about dynamic programming solutions in terms of recursive functions, Constructing a solution Sometimes we are asked both to find the value of an optimal solution and to give an example how such a solution can be constructed. In the coin problem, for example, we can declare another array that indicates for each sum of money the first coin in an optimal solution: ‘Then, we can modify the algorithm as follows: int first(N] value(@] For (int x xen: x) ( value[x] = INF; for (auto € : coins) ( if Gre >= @ 8&8 value(x-c]+) < valuefx]) { valuelx] = valuefx-c]+1; Firstlxd] 68 Scanned with CamScannerAfter this, the following code can be used to print the coins that appear in an optimal solution for the sum n: while (n> @) ( cout << first{n] << "\n' n-= first(n]; Counting the number of solutions Let us now consider another version of the coin problem where our task is to calculate the total number of ways to produce a sum x using the coins. For example, if coins = (1,3,4} and x = 5, there are a total of 6 ways: e1s1eititl oB41t1 © 14143 sta © 14341 eae Again, we can solve the problem recursively. Let solve(x) denote the number of ways we can form the sum x. For example, if coins = {1,3,4), then solve(5) = 6 and the recursive formula is solve(x) =solve(x - 1)+ solve(x~3)+ solvelx—4). ‘Then, the general recursive function is as follows: 0 x<0 solve(x)=4 1 x=0 Leceoins SOlve(x—c) x>0 Ifx <0, the value is 0, because there are no solutions. If x = 0, the value is 1, because there is only one way to form an empty sum. Otherwise we calculate the sum of all values of the form solve(x~c) where c is in coins. ‘The following code constructs an array count such that count[x] equals the value of solve(x) for 0 = @) { count£x] += count{x-c]; 69 Scanned with CamScannerOften the number of solutions is so large that it is not required to calculate the exact number but it is enough to give the answer modulo m where, for example, m= 10° +7. This can be done by changing the code so that all calculations are done modulo m. In the above code, it suffices to add the line count [x] %= m; after the line countLx] += countLx-c]; Now we have discussed all basic ideas of dynamic programming. Since dynamic programming can be used in many different situations, we will now go through a set of problems that show further examples about the possibilities of dynamic programming. Longest increasing subsequence Our first problem is to find the longest increasing subsequence in an array of n elements. This is a maximum-length sequence of array elements that goes from left to right, and each element in the sequence is larger than the previous element. For example, in the array o 12845 6 6l|2[5]il7|4]als the longest increasing subsequence contains 4 elements: o 12345 67 6[2[5]1]7] 4/8 UN" Let length(k) denote the length of the longest increasing subsequence that. ends at position k. Thus, if we calculate all values of length(k) where 0= @) possibletxILk] possible[x}{k] |= possible{x](k~ possibletx-wfk]]Ck-11; ? However, here is a better implementation that only uses a one-dimensional array possible[x] that indicates whether we can construct a subset with sum x. ‘The trick is to update the array from right to left for each new weight: possible(@] for (int k #1; k cen; ke) ( for (int «2M; x32 0; x--) { if (possible(x]) possibleLx+wlk]] = true; y > Note that the general idea presented here can be used in many knapsack problems. For example, if we are given objects with weights and values, we can determine for each weight sum the maximum value sum of a subset. 13 Scanned with CamScannerEdit distance ‘The edit distance or Levenshtein distance! is the minimum number of edi ing operations needed to transform a string into another string. The allowed editing operations are as follows: * insert a character (e.g. ABC —- ABCA) * remove a character (e.g. ABC — AC) * modify a character (e.g. ABC — ADC) For example, the edit distance between LOVE and NOVIE is 2, because we can first perform the operation LOVE — MOVE (modify) and then the operation MOVE + NOVIE (insert). This is the smallest possible number of operations, because it is clear that only one operation is not enough. Suppose that we are given a string x of length n and a string y of length m, and we want to calculate the edit distance between x and y. To solve the problem, we define a function distance(a, 6) that gives the edit distance between prefixes x{0...a] and yl0...5]. Thus, using this function, the edit distance between x and y equals distance(n~1,m- 1). We can calculate values of distance as follows: distance(a,b) = min(distance(a,b-1)+1, distance(a—1,b)+1, distance(a ~1,b— 1) + cost(a,b)) Here cost(a,b) = 0 if xa] = y[b], and otherwise cost(a,b) = 1. The formula considers the following ways to edit the string x: * distance(a,b —1): insert a character at the end of x * distance(a—1,b): remove the last character from x * distance(a~1,b~1): match or modify the last character of x In the two first cases, one editing operation is needed (insert or remove). In the last case, if x{a] = y[b], we can match the last characters without editing, and otherwise one editing operation is needed (modify) The following table shows the values of distance in the example case: MOVIE of1f2]3]4]5 tfafaf2lala[s ol2|2fij2is/4 v[s[3f2lilels ela{a[al2i2/2 Phe distance is named after V. I. Levenshtein who studied it in connection with binary codes (49), 74 Scanned with CamScannerThe lower-right corner of the table tells us that the edit distance between LOVE and MOVIE is 2. The table also shows how to construct the shartest sequence of editing operations. In this case the path is as follows: ¥ 1 w[wlo wlele|< wlelele wo) efe|olm m and the total number of solutions is 781. The problem can be solved using dynamic programming by going through the grid row by row. Each row in a solution can be represented as a string that contains m characters from the set (7,1, ,—. For example, the above solution consists of four rows that correspond to the following strings: *ncancan * ucdunnu + cacguun * cacgcsu Let count(k,x) denote the number of ways to construct a solution for rows 1...k of the grid such that string x corresponds to row k. It is possible to use dynamic programming here, because the state of a row is constrained only by the state of the previous row. ony Scanned with CamScannerA solution is valid if row 1 does not contain the character Li, row n does not contain the character /, and all consecutive rows are compatible. For example, the rows UC UNMU and CIC UUM are compatible, while the rows CIN IN and COCCI are not compatible. Since a row consists of m characters and there are four choices for each character, the number of distinct rows is at most 4”. Thus, the time complexity of the solution is O(n4?”) because we can go through the O(4") possible states for each row, and for each state, there are O(4”) possible states for the previous row. In practice, it is a good idea to rotate the grid so that the shorter side has length m, because the factor 42” dominates the time complexity. It is possible to make the solution more efficient by using a more compact representation for the rows. It turns out that it is sufficient to know which columns of the previous row contain the upper square of a vertical tile. Thus, we can represent a row using only characters 7 and D, where Cis a combination of characters U, C and 4. Using this representation, there are only 2” distinct rows and the time complexity is O(n 2”), Asa final note, there is also a surprising direct formula for calculating the number of tilings”: fz tm xb TL [] 4:€cos* = + cos’ —— : nat ma ‘This formula is very efficient, because it calculates the number of tilings in O(nm) time, but since the answer is a product of real numbers, a problem when using the formula is how to store the intermediate results accurately. Surprisingly, this formula was discovered in 1961 by two research teams [43, 67] that worked independently: 16 Scanned with CamScannerChapter 8 Amortized analysis ‘The time complexity of an algorithm is often easy to analyze just by examining the structure of the algorithm: what loops does the algorithm contain and how many times the loops are performed. However, sometimes a straightforward analysis does not give a true picture of the efficiency of the algorithm. Amortized analysis can be used to analyze algorithms that contain opera- tions whose time complexity varies. The idea is to estimate the total time used to all such operations during the execution of the algorithm, instead of focusing on individual operations. Two pointers method In the two pointers method, two pointers are used to iterate through the array values. Both pointers can move to one direction only, which ensures that the algorithm works efficiently. Next we discuss two problems that can be solved using the two pointers method. Subarray sum As the first example, consider a problem where we are given an array of n positive integers and a target sum x, and we want to find a subarray whose sum is x or report that there is no such subarray. For example, the array 1jaf2|sfifa contains a subarray whose s 1{3|ajsjilile ‘This problem can be solved in O(n) time by using the two pointers method. The idea is to maintain pointers that point to the first and last value of a subarray. On each turn, the left pointer moves one step to the right, and the right pointer moves to the right as long as the resulting subarray sum is at most x. If the sum becomes exactly x, a solution has been found. 7 Scanned with CamScannerAs an example, consider the following array and a target sum x = 8: ij3f2|sjilije|s ‘The initial subarray contains the values 1j3s|2}5|1jij2}3 T T Then, the left pointer moves one step to the right. The right pointer does not move, because otherwise the subarray sum would exceed x. 1/3]2]sjijij2|3 TT Again, the left pointer moves one step to the right, and this time the right pointer moves three steps to the right. The subarray sum is 2+5+1=8, 50a subarray whose sum is x has been found. 1{3|[2]sjijij2|3 T T ‘The running time of the algorithm depends on the number of steps the right pointer moves. While there is no useful upper bound on how many steps the pointer can move on a single turn. we know that the pointer moves a total of O(n) steps during the algorithm, because it only moves to the right. Since both the left and right pointer move O(n) steps during the algorithm, the algorithm works in O(n) time. 2SUM problem Another problem that can be solved using the two pointers method is the following problem, also known as the 2SUM problem: given an array of n numbers and a target sum x, find two array values such that their sum is x, or report that no such values exist ‘To solve the problem, we first sort the array values in increasing order. After that, we iterate through the array using two pointers. The left pointer starts at the first value and moves one step to the right on each turn. The right pointer begins at the last value and always moves to the left until the sum of the left and right value is at most x. If the sum is exactly x, a solution has been found. For example, consider the following array and a target sum x = 12: 1j4|{5/6|7] 9/9 |10 ‘The initial positions of the pointers are as follows. The sum of the values is 1+10=11 that is smaller than x 18 Scanned with CamScanner1|4|5/6|7]9]9 |10 T T Then the left pointer moves one step to the right. The right pointer moves three steps to the left, and the sum becomes 4+7=11 1|4]5/6|7] 9/9 |10 T T After this, the left pointer moves one step to the right again. The right pointer does not move, and a solution 5 +7 = 12 has been found. 1|4|5|6{7]9/ 9 |10 T T ‘The running time of the algorithm is O(nlogn), because it first sorts the array in O(nlogn) time, and then both pointers move O(n) steps. Note that it is possible to solve the problem in another way in O(n logn) time using binary search. In such a solution, we iterate through the array and for each array value, we try to find another value that yields the sum x. This can be done by performing n binary searches, each of which takes O(logn) time. A more difficult problem is the 3SUM problem that asks to find three array values whose sum is x. Using the idea of the above algorithm, this problem can be solved in O(n*) time!. Can you see how? Nearest smaller elements Amortized analysis is often used to estimate the number of operations performed ona data structure. The operations may be distributed unevenly so that most operations occur during a certain phase of the algorithm, but the total number of, the operations is limited As an example, consider the problem of finding for each array element the nearest smaller element, ie., the first smaller element that precedes the element in the array. It is possible that no such element exists, in which case the algorithm should report this. Next we will see how the problem can be efficiently solved using a stack structure. We go through the array from left to right and maintain a stack of array elements. At each array position, we remove elements from the stack until the top element is smaller than the current element, or the stack is empty. Then, we report that the top element is the nearest smaller element of the current element, or if the stack is empty, there is no such element. Finally, we add the current element to the stack. ‘As an example, consider the following array: ‘Ror a long time, it was thought that solving the 3SUM problem more efficiently than in O(n®) time would not be possible. However, in 2014, it turned out [30] that this isnot the case. 79 Scanned with CamScannerFirst, the elements 1, 3 and 4 are added to the stack, because each element is larger than the previous element. Thus, the nearest smaller clement of 4 is 3, and the nearest smaller element of 3 is 1. ‘The next element 2 is smaller than the two top elements in the stack. Thus, the elements 3 and 4 are removed from the stack, and then the element 2 is added to the stack. Its nearest smaller element is 1 1j3|4aja]s|aiaiea Then, the element 5 is larger than the element 2, so it will be added to the stack, and its nearest smaller element is 2: After this, the element 5 is removed from the stack and the elements 3 and 4 are added to the stack: Finally, all elements except 1 are removed from the stack and the last element 2 is added to the stack: The efficiency of the algorithm depends on the total number of stack opera- tions. If the current element is larger than the top element in the stack, it is directly added to the stack, which is efficient. However, sometimes the stack can contain several larger elements and it takes time to remove them. Still, each element is added exactly once to the stack and removed at most once from the stack. Thus, each element causes O(1) stack operations, and the algorithm works in O(n) time. 80 Scanned with CamScannerSliding window minimum A sliding window is a constant-size subarray that moves from left to right through the array. At each window position, we want to calculate some infor- mation about the elements inside the window. In this section, we focus on the problem of maintaining the sliding window minimum, which means that we should report the smallest value inside each window. ‘The sliding window minimum can be calculated using a similar idea that we used to calculate the nearest smaller elements. We maintain a queue where each element is larger than the previous element, and the first element always corresponds to the minimum element inside the window. After each window move, we remove elements from the end of the queue until the last queue element is smaller than the new window element, or the queue becomes empty. We also remove the first queue element if it is not inside the window anymore. Finally, we add the new window element to the end of the queue. As an example, consider the following array: 2|ila|siaf4ajije ‘Suppose that the size of the sliding window is 4. At the first window position, the smallest value is 1: 2|al4|s]3l4iif2 HHH) ‘Then the window moves one step right. The new element 3 is smaller than the elements 4 and 5 in the queue, so the elements 4 and 5 are removed from the queue and the element 3 is added to the queue. The smallest value is still 1 2|af4a]slial4iij2 After this, the window moves again, and the smallest element 1 does not belong to the window anymore. Thus, it is removed from the queue and the smallest value is now 3. Also the new element 4 is added to the queue. 2|ifa]s[sf4ajif2 ‘The next new element 1 is smaller than all elements in the queue. Thus, all elements are removed from the queue and it will only contain the element 1: 2|1|4|6]s}4}1]2 O) 81 Scanned with CamScannerFinally the window reaches its last position. The element 2 is added to the queue, but the smallest value inside the window is still 1. 2[1]4]5]afajzf2 GHz) Since each array clement is added to the queue exactly once and removed from the queue at most once, the algorithm works in O(n) time. 82 Scanned with CamScannerChapter 9 Range queries In this chapter, we discuss data structures that allow us to efficiently process range queries. In a range query, our task is to calculate a value based on a subarray of an array. Typical range queries are: * sumg(a, ): calculate the sum of values in range [a,b] * ming(a,): find the minimum value in range [a,b] * maxp(a,): find the maximum value in range [a,6] For example, consider the range [3,6] in the following array: 012345 67 1|3|[sjajejij3al4 In this case, sum,(3,6) = 14, ming(3,6) = 1 and max,(3,6) = 6. A simple way to process range queries is to use a loop that goes through all array values in the range. For example, the following function can be used to process stm queries on an array: int sum(int a, int b) { int s = 0; for (int dea; i s = arrayli] by it) ¢ ) » This function works in O(n) time, where n is the size of the array. Thus, we can process q queries in O(ng) time using the function. However, if both n and q are large, this approach is slow. Fortunately, it turns out that there are ways to process range queries much more efficiently. 83 Scanned with CamScannerStatic array queries We first focus on a situation where the array is static, i.e., the array values are never updated between the queries. In this case, it suffices to construct a static data structure that tells us the answer for any possible query. Sum queries We can easily process sum queries on a static array by constructing a prefix sum array. Each value in the prefix sum array equals the sum of values in the original array up to that position, ie., the value at position k is sum4(0, k). The prefix sum array can be constructed in O(n) time. For example, consider the following array: 123 ij3|4aisljelilaj2 ‘The corresponding prefix sum array is as follows O12 845 67 1| 4 | 8 |16|22|23|27|29 Since the prefix sum array contains all values of sum,(0,), we can caleulate any value of sum4(a, 6) in O(1) time as follows: sum, (a,b) = sumy(0,5) ~ sum(0,a~ 1) By defining sun,(0,-1)=0, the above formula also holds when a =0. For example, consider the range [3,6] O12 384 5 67 sielil4l2 * = 19. This sum can be calculated from two In this case sum,(3,6 values of the prefix sum array: o 12345 67 1| 4 | 8 |16|22]23|27|29 ‘Thus, sumg(3,6) = sumg(0,6) ~ sum,(0,2)=27-8=19. It is also possible to generalize this idea to higher dimensions. For example, ‘we can construct a two-dimensional prefix sum array that can be used to calculate the sum of any rectangular subarray in O(1) time. Each sum in such an array corresponds to a subarray that begins at the upper-left corner of the array. 84 Scanned with CamScanner‘The following picture illustrates the idea’ ID c B Al | I ‘The sum of the gray subarray can be calculated using the formula S(A)-S(B)-S(C)+S(D), where $(X) denotes the sum of values in a rectangular subarray from the upper- left corner to the position of X. Minimum queries Minimum queries are more difficult to process than sum queries, Still, there is a quite simple O(n logn) time preprocessing method after which we can answer any minimum query in O(1) time’. Note that since minimum and maximum queries can be processed similarly, we can focus on minimum queries. ‘The idea is to precalculate all values of ming(a, 6) where 6 ~a +1 (the length of the range) is a power of two. For example, for the array o12 345 67 aj3fa|siefilal2 the following values are calculated: a_b_ming(a,b) 0 1 2 ck ewnols aaaaeale b i 2 3 4 5 6 7 beHonen 3 4 5 6 saoneennolas aonkewroa ‘The number of precalculated values is O(n logn), because there are O(logn) range lengths that are powers of two. The values can be calculated efficiently using the recursive formula ming(a,b) = min(ming(a,a +w—1),ming(a+w,b)), This technique was introduced in [7] and sometimes called the sparse table method, There are also more sophisticated techniques (22] where the preprocessing time is only O(n), but such algorithms are not needed in competitive programming, 85 Scanned with CamScannerwhere 6-a +1 is a power of two and w = (b-a + 1)/2. Calculating all those values takes O(n logn) time. After this, any value of ming(a,b) can be calculated in O(1) time as a minimum, of two precalculated values. Let k be the largest power of two that does not exceed b-a+1, We can calculate the value of ming(a,b) using the formula ming(a,b) = min(ming(a,a +h ~1),ming(b~k +1,)). In the above formula, the range [a,b] is represented as the union of the ranges [a,a+k-1]) and (6 -k+ 1,6), both of length k. As an example, consider the range [1,6] 28 45 67 1{3[4]sjelil4[2 ‘The length of the range is 6, and the largest power of two that does not exceed 6 is 4. Thus the range [1,6] is the union of the ranges [1,4] and [3,6]: 012345 67 1js|4jslelijsi2 Since ming(1,4)=3 and ning(3,6)= 1, we conclude that ming(1,6) = 1. Binary indexed tree Abinary indexed tree or a Fenwick tree can be seen as a dynamic variant, of a prefix sum array. It supports two O(logn) time operations on an array: processing a range sum query and updating a value ‘The advantage of a binary indexed tree is that it allows us to efficiently update array values between sum queries. This would not be possible using a prefix sum array, because after each update, it would be necessary to build the whole prefix sum array again in O(n) time. Structure Even if the name of the structure is a binary indexed tree, it is usually represented as an array. In this section we assume that all arrays are one-indexed, because it makes the implementation easier. Let p(k) denote the largest power of two that divides k. We store a binary indexed tree as an array tree such that tree{he] = sumy(k— plk)+1,k), 2he binary indexed tree structure was presented by P. M. Fenwick in 1994 [21] 86 Scanned with CamScanneri.e., each position & contains the sum of values in a range of the original array whose length is p(k) and that ends at position &. For example, since p(6) = 2, tree{6] contains the value of sumg(5,6). For example, consider the following array’ 1j3{4a|sjelija ‘The corresponding binary indexed tree is as follows: 12345 678 1|4| 4/16] 6|7| 4 |29 The following picture shows more clearly how each value in the binary indexed tree corresponds to a range in the original array: 4 16 8 29 tsTe foe [74 oe rtd oyo;o};o 5 I Je FE Using a binary indexed tree, any value of sum,(1,#) can be calculated in Ollogn) time, because a range [1,k] can always be divided into O(logn) ranges whose sums are stored in the tree. For example, the range [1,7] consists of the following ranges: 45678 16| 6 [7 | 4 [29] i TELE e]a)a)h —_ | ‘Thus, we can calculate the corresponding sum as follows: sumg(1,7) = sumg(1,4) + sumg(5, 6) + sumg(7,7) = 16-+7 + To calculate the value of sum,(a,b) where a > 1, we can use the same trick that we used with prefix sum arrays: sumg(a,b) = sumy(1, 6) — sumg(1,@~ 1). 87 Scanned with CamScannerSince we can calculate both sumy(1,5) and sumg(1,a~1) in O(logn) time, the total time complexity is O(logn). ‘Then, after updating a value in the original array, several values in the binary indexed tree should be updated. For example, if the value at position 3 changes, the sums of the following ranges change: 8 29 Te iE Since each array element belongs to O(logn) ranges in the binary indexed tree, it suffices to update O(logn) values in the tree. Implementation ‘The operations of a binary indexed tree can be efficiently implemented using bit. operations. The key fact needed is that we can calculate any value of p(k) using the formula pik) = kB -k. ‘The following function calculates the value of sum,(1,) int sum(int k) { int 5 = 0; while (k >= 1) treetkl; kak; > The following function increases the array value at position k by x (x can be positive or negative): void add(int k, int x) { while (k = 1; k /= 2) { tree(k] = tree[2*k]+treel2sk+1]; ) 91 Scanned with CamScannerFirst the function updates the value at the bottom level of the tree. After this, the function updates the values of all internal tree nodes, until it reaches the top node of the tree. Both the above functions work in O(logn) time, because a segment tree of n elements consists of O(log n) levels, and the functions move one level higher in the tree at each step. Other queries ‘Segment trees can support all range queries where it is possible to divide a range into two parts, calculate the answer separately for both parts and then efficiently combine the answers. Examples of such queries are minimum and maximum, greatest common divisor, and bit operations and, or and xor. For example, the following segment tree supports minimum queries: In this case, every tree node contains the smallest value in the corresponding array range. The top node of the tree contains the smallest value in the whole array. The operations can be implemented like previously, but instead of sums, minima are calculated ‘The structure of a segment tree also allows us to use binary search for locating array elements. For example, if the tree supports minimum queries, we can find the position of an element with the smallest value in O(logn) time. For example, in the above tree, an element with the smallest value 1 can be found by traversing a path downwards from the top node: Scanned with CamScannerAdditional techniques Index compression A limitation in data structures that are built upon an array is that the elements are indexed using consecutive integers. Difficulties arise when large indices are needed. For example, if we wish to use the index 10°, the array should contain 10° elements which would require too much memory. However, we can often bypass this limitation by using index compression, where the original indices are replaced with indices 1,2,8, ete. This can be done if we know all the indices needed during the algorithm beforehand. ‘The idea is to replace each original index x with e(x) where c is a function that, compresses the indices. We require that the order of the indices does not change, so if a > k removes the & last bits from the number. For example, 14<<2= 56, because 14 and 56 correspond to 1110 and 111000. Similarly, 49 >> 3 = 6, because 49 and 6 correspond to 110001 and 110. Note that x << corresponds to multiplying x by 2', and x >> corresponds to dividing x by 2* rounded down to an integer. Applications Anumber of the form 1 << & has a one bit in position k and all other bits are zero, so we can use such numbers to access single bits of numbers. In particular, the kth bit of a number is one exactly when x & (1 << &) is not zero. The following code prints the bit representation of an int number x: for (int i= 31; i >= @; if (a(I ‘The following code goes through the subsets of a set x: int b do ( // process subset b J while (b=(b-x)8x); 99 Scanned with CamScannerBit optimizations Many algorithms can be optimized using bit operations. Such optimizations do not change the time complexity of the algorithm, but they may have a large impact on the actual running time of the code. In this section we discuss examples of such situations. Hamming distances ‘The Hamming distance hamning(a,5) between two strings a and of equal length is the number of positions where the strings differ. For example, hamming(01101, 11001) = 2. Consider the following problem: Given a list of n bit strings, each of length k, calculate the minimum Hamming distance between two strings in the list. For example, the answer for [00111,01101, 11110] is 2, because + hamming(00111,01101) © hanming(00111, 11110} © hamming(01101, 11110) A straightforward way to solve the problem is to go through all pairs of strings and calculate their Hamming distances, which yields an O(n?A) time algorithm. The following function can be used to calculate distances: int haming(string a, string b) { int d= @; for (int i= @; i best[1<