Greedy Algorithms • Greedy is an algorithmic paradigm that builds up a solution piece by piece, always choosing the next piece that offers the most clear and immediate benefit. • Greedy algorithms are used for optimization problems. An optimization problem can be solved using Greedy if the problem has the following property: At every step, we can make a choice that looks best at the moment, and we get the optimal solution of the complete problem • If a Greedy Algorithm can solve a problem, then it generally becomes the best method to solve that problem as the Greedy algorithms are in general more efficient than other techniques. But Greedy algorithms cannot always be applied. Examples of popular Greedy algorithms • Following are some standard algorithms that are Greedy algorithms. 1) Kruskal’s Minimum Spanning Tree (MST) : In Kruskal’s algorithm, we create a MST by picking edges one by one. The Greedy Choice is to pick the smallest weight edge that doesn’t cause a cycle in the MST constructed so far. 2) Prim’s Minimum Spanning Tree : In Prim’s algorithm also, we create a MST by picking edges one by one. We maintain two sets: set of the vertices already included in MST and the set of the vertices not yet included. The Greedy Choice is to pick the smallest weight edge that connects the two sets. 3) Dijkstra’s Shortest Path : The Dijkstra’s algorithm is very similar to Prim’s algorithm. The shortest path tree is built up, edge by edge. We maintain two sets: set of the vertices already included in the tree and the set of the vertices not yet included. The Greedy Choice is to pick the edge that connects the two sets and is on the smallest weight path from source to the set that contains not yet included vertices. 4) Huffman Coding : Huffman Coding is a loss - less compression technique. It assigns variable length bit codes to different characters. The Greedy Choice is to assign least bit length code to the most frequent character. Activity Selection problem • Let us consider the Activity Selection problem as our first example of Greedy algorithms. Following is the problem statement. You are given n activities with their start and finish times. Select the maximum number of activities that can be performed by a single person, assuming that a person can only work on a single activity at a time. Example: • Example 1 : Consider the following 3 activities sorted by by finish time. start[] = {10, 12, 20}; finish[] = {20, 25, 30}; A person can perform at most two activities. The maximum set of activities that can be executed is {0, 2} [ These are indexes in start[] and finish[] ] Example 1 • Consider the following 3 activities sorted by by finish time. • start[] = { 10 , 12, 20 }; • finish[] = { 20 , 25, 30 }; • Solution: • It is clear that a person can perform at most two activities. • The maximum set of activities that can be executed is {0, 2} [ A Person can perform activity 1 starting at time 10 to time 20. Next activity he can perform is with start time 20 and finish time 30. Activity Selection problem Algorithm • The greedy choice is to always pick the next activity whose finish time is least among the remaining activities and the start time is more than or equal to the finish time of previously selected activity. We can sort the activities according to their finishing time so that we always consider the next activity as minimum finishing time activity. • 1) Sort the activities according to their finishing time 2) Select the first activity from the sorted array and print it. 3) Do following for remaining activities in the sorted array. .......a) If the start time of this activity is greater than or equal to the finish time of previously selected activity then select this activity and print it. Activity Selection problem Algorithm 1) Sort the activities according to their finishing time 2) Select the first activity from the sorted array and print it. 3) Do following for remaining activities in the sorted array. ....... a) If the start time of this activity is greater than or equal to the finish time of previously selected activity then select this activity and print it. What is a 'Greedy algorithm'? A greedy algorithm, as the name suggests, always makes the choice that seems to be the best at that moment. This means that it makes a locally - optimal choice in the hope that this choice will lead to a globally - optimal solution. 2 Properties of Greedy Algorithms • If both of the properties below are true, a greedy algorithm can be used to solve the problem. • Greedy choice property • A global (overall) optimal solution can be reached by choosing the optimal choice at each step. • Optimal substructure • A problem has an optimal substructure if an optimal solution to the entire problem contains the optimal solutions to the sub - problems. How Greedy Algorithm Works? • A greedy algorithm selects a candidate greedily (local optimum) and adds it to the current solution provided that it doesn’t corrupt the feasibility. • If the solution obtained by above step is not final, repeat till global optimum or the final solution is obtained. Example of Greedy Algorithm • Kruskal’s Algorithm : This is a greedy algorithm used to find the minimum spanning tree of a graph. • Kruskal’s algorithm can be stated as follows: 0. Create a minimum spanning tree T that initially contains no edges, 1. Choose an edge e in G, where (a) e is not in T and ... (b) e is of minimum weight and ... (c) e does not create a cycle in T 2. If T does not contain all the vertices of G go to step 1. • According to the above algorithm we always choose the minimum weight edges. Hence Krushkal’s Algorithm will always give the correct result. Note that a Minimum Spanning Tree of V vertices must have at least V - 1 edges and should not contain cycle. So we did not pick any extra edge in above step. Advantages and disadvantages: • It is quite easy to come up with a greedy algorithm (or even multiple greedy algorithms) for a problem. • Analyzing the run time for greedy algorithms will generally be much easier than for other techniques (like Divide and conquer). For the Divide and conquer technique, it is not clear whether the technique is fast or slow. This is because at each level of recursion the size of gets smaller and the number of sub - problems increases. • The difficult part is that for greedy algorithms you have to work much harder to understand correctness issues Applications • Greedy algorithms mostly (but not always) fail to find the globally optimal solution, because they usually do not operate exhaustively on all the data. They can make commitments to certain choices too early which prevent them from finding the best overall solution later. For example, all known greedy coloring algorithms for the graph coloring problem and all other such problems do not consistently find optimum solutions. But, they are useful because they are quick to think up and often give good approximations to the optimum. • If a greedy algorithm can be proven to produce the global optimum for a given problem class, it typically becomes the method of choice because it is faster than other optimization methods. Examples of such greedy algorithms are Kruskal's algorithm and Prim's algorithm for finding minimum spanning trees , and the algorithm for finding optimum Huffman trees a greedy strategy does not always produce an optimal solution • However, in many problems, a greedy strategy does not produce an optimal solution. • For example, in the problem below, the greedy algorithm seeks to find the path with the largest sum. • It does this by selecting the largest available number at each step. The greedy algorithm fails to find the largest sum, however, because it makes decisions based only on the information it has at any one step, and without regard to the overall problem. The greedy algorithm fails to find the largest sum, however, because it makes decisions based only on the information it has at any one step, and without regard to the overall problem. Making change(selecting coins) problem