Using Heaps in Algorithms

Madhavan Mukund
https://www.cmi.ac.in/~madhavan

Programming, Data Structures and Algorithms using Python

Week 6
Priority queues and heaps

Priority queues support the following operations

insert()
delete max() or delete min()

Heaps are a tree based implementation of priority queues

insert(), delete max() / delete min() are both O(log n)
heapify() builds a heap from a list/array in time O(n)

Heap can be represented as a list/array

Simple index arithmetic to find parent and children of a node

What more do we need to use a heap in an algorithm?

Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 2/9
Dijkstra’s algorithm
def dijkstra(WMat,s):
Maintain two dictionaries with (rows,cols,x) = WMat.shape
vertices as keys infinity = np.max(WMat)*rows+1
(visited,distance) = ({},{})
visited, initially False for all v for v in range(rows):
distance, initially infinity for (visited[v],distance[v]) = (False,infinity)
distance[s] = 0
all v
for u in range(rows):
nextd = min([distance[v] for v in range(rows)
Set distance[s] to 0 if not visited[v]])
nextvlist = [v for v in range(rows)
Repeat, until all reachable vertices if (not visited[v]) and
are visited distance[v] == nextd]
if nextvlist == []:
Find unvisited vertex nextv with break
minimum distance nextv = min(nextvlist)
visited[nextv] = True
Set visited[nextv] to True
for v in range(cols):
Recompute distance[v] for if WMat[nextv,v,0] == 1 and (not visited[v]):
every neighbour v of nextv distance[v] = min(distance[v],distance[nextv]
+WMat[nextv,v,1])
return(distance)

Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 3/9
Dijkstra’s algorithm
def dijkstra(WMat,s):
Bottleneck (rows,cols,x) = WMat.shape
infinity = np.max(WMat)*rows+1
Find unvisited vertex j with (visited,distance) = ({},{})
minimum distance for v in range(rows):
(visited[v],distance[v]) = (False,infinity)
Naive implementation requires an distance[s] = 0
O(n) scan for u in range(rows):
nextd = min([distance[v] for v in range(rows)
if not visited[v]])
nextvlist = [v for v in range(rows)
if (not visited[v]) and
distance[v] == nextd]
if nextvlist == []:
break
nextv = min(nextvlist)
visited[nextv] = True
for v in range(cols):
if WMat[nextv,v,0] == 1 and (not visited[v]):
distance[v] = min(distance[v],distance[nextv]
+WMat[nextv,v,1])
return(distance)

Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 4/9
Dijkstra’s algorithm
def dijkstra(WMat,s):
Bottleneck (rows,cols,x) = WMat.shape
infinity = np.max(WMat)*rows+1
Find unvisited vertex j with (visited,distance) = ({},{})
minimum distance for v in range(rows):
(visited[v],distance[v]) = (False,infinity)
Naive implementation requires an distance[s] = 0
O(n) scan for u in range(rows):
nextd = min([distance[v] for v in range(rows)
Maintain unvisited vertices as a if not visited[v]])
nextvlist = [v for v in range(rows)
min-heap if (not visited[v]) and
delete min() in O(log n) time distance[v] == nextd]
if nextvlist == []:
break
nextv = min(nextvlist)
visited[nextv] = True
for v in range(cols):
if WMat[nextv,v,0] == 1 and (not visited[v]):
distance[v] = min(distance[v],distance[nextv]
+WMat[nextv,v,1])
return(distance)

Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 4/9
Dijkstra’s algorithm
def dijkstra(WMat,s):
Bottleneck (rows,cols,x) = WMat.shape
infinity = np.max(WMat)*rows+1
Find unvisited vertex j with (visited,distance) = ({},{})
minimum distance for v in range(rows):
(visited[v],distance[v]) = (False,infinity)
Naive implementation requires an distance[s] = 0
O(n) scan for u in range(rows):
nextd = min([distance[v] for v in range(rows)
Maintain unvisited vertices as a if not visited[v]])
nextvlist = [v for v in range(rows)
min-heap if (not visited[v]) and
delete min() in O(log n) time distance[v] == nextd]
if nextvlist == []:
But, also need to update distances break
nextv = min(nextvlist)
of neighbours visited[nextv] = True
for v in range(cols):
Unvisited neighbours’ distances
if WMat[nextv,v,0] == 1 and (not visited[v]):
are inside the min-heap distance[v] = min(distance[v],distance[nextv]
Updating a value is not a basic +WMat[nextv,v,1])
return(distance)
heap operation
Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 4/9
Updating values in a min-heap
Change 54 to 35
29

38 31

54 47 84 61

68 59 50

Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 5/9
Updating values in a min-heap
Change 54 to 35
Reducing a value can create a 29
violation with parent

38 31

35 47 84 61

68 59 50

Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 5/9
Updating values in a min-heap
Change 54 to 35
Reducing a value can create a 29
violation with parent
Swap upwards to restore heap,
similar to insert() 35 31

38 47 84 61

68 59 50

Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 5/9
Updating values in a min-heap
Change 54 to 35
Reducing a value can create a 29
violation with parent
Swap upwards to restore heap,
similar to insert() 35 31

Change 29 to 71
38 47 84 61

68 59 50

Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 5/9
Updating values in a min-heap
Change 54 to 35
Reducing a value can create a 71
violation with parent
Swap upwards to restore heap,
similar to insert() 35 31

Change 29 to 71
Increasing a value can create a 38 47 84 61
violation with child

68 59 50

Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 5/9
Updating values in a min-heap
Change 54 to 35
Reducing a value can create a 31
violation with parent
Swap upwards to restore heap,
similar to insert() 35 71

Change 29 to 71
Increasing a value can create a 38 47 84 61
violation with child
Swap downwards to restore
heap, similar to delete min()
68 59 50

Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 5/9
Updating values in a min-heap
Change 54 to 35
Reducing a value can create a 31
violation with parent
Swap upwards to restore heap,
similar to insert() 35 61

Change 29 to 71
Increasing a value can create a 38 47 84 71
violation with child
Swap downwards to restore
heap, similar to delete min()
68 59 50

Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 5/9
Updating values in a min-heap
Change 54 to 35
Reducing a value can create a 31
violation with parent
Swap upwards to restore heap,
similar to insert() 35 61

Change 29 to 71
Increasing a value can create a 38 47 84 71
violation with child
Swap downwards to restore
heap, similar to delete min()
68 59 50
Both updates are O(log n)
Are we done?

Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 5/9
Updating values in a min-heap
Change 54 to 35
Reducing a value can create a 31
violation with parent
Swap upwards to restore heap,
similar to insert() 35 61

Change 29 to 71
Increasing a value can create a 38 47 84 71
violation with child
Swap downwards to restore
heap, similar to delete min()
68 59 50
Both updates are O(log n)
Are we done?

Locate the node to update?

Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 5/9
Updating values in a min-heap
Maintain two additional 7:29
dictionaries
Vertices are
{0,1,...,n-1} 5:38 3:31
Heap positions are
{0, 1, . . . , n − 1}
VtoH maps vertices to 1:54 4:47 2:84 6:61
heap positions
HtoV maps heap
positions to vertices 0:68 8:59
VtoH 0 1 2 3 4 5 6 7 8
7 3 5 2 4 1 6 0 8
HtoV 0 1 2 3 4 5 6 7 8
7 5 3 1 4 2 6 0 8
Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 6/9
Updating values in a min-heap
Maintain two additional 7:29
dictionaries
Vertices are
{0,1,...,n-1} 5:38 3:31
Heap positions are
{0, 1, . . . , n − 1}
VtoH maps vertices to 1:35 4:47 2:84 6:61
heap positions
HtoV maps heap
positions to vertices 0:68 8:59

Update node 1 to 35 VtoH 0 1 2 3 4 5 6 7 8

7 3 5 2 4 1 6 0 8
HtoV 0 1 2 3 4 5 6 7 8
7 5 3 1 4 2 6 0 8
Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 6/9
Updating values in a min-heap
Maintain two additional 7:29
dictionaries
Vertices are
{0,1,...,n-1} 1:35 3:31
Heap positions are
{0, 1, . . . , n − 1}
VtoH maps vertices to 5:38 4:47 2:84 6:61
heap positions
HtoV maps heap
positions to vertices 0:68 8:59

Update node 1 to 35 VtoH 0 1 2 3 4 5 6 7 8

7 1 5 2 4 3 6 0 8
Update VtoH and HtoV HtoV 0 1 2 3 4 5 6 7 8
each time we swap values in 7 1 3 5 4 2 6 0 8
the heap
Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 6/9
Dijkstra’s algorithm
Using min-heaps 7:29
Identifying next vertex to
visit is O(log n)
Updating distance takes 5:38 3:31
O(log n) per neighbour
Adjacency list —
proportionally to degree 1:54 4:47 2:84 6:61

0:68 8:59
VtoH 0 1 2 3 4 5 6 7 8
7 3 5 2 4 1 6 0 8
HtoV 0 1 2 3 4 5 6 7 8
7 5 3 1 4 2 6 0 8
Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 7/9
Dijkstra’s algorithm
Using min-heaps 7:29
Identifying next vertex to
visit is O(log n)
Updating distance takes 5:38 3:31
O(log n) per neighbour
Adjacency list —
proportionally to degree 1:54 4:47 2:84 6:61

Cumulatively
O(n log n) to identify 0:68 8:59
vertices to visit across n
iterations VtoH 0 1 2 3 4 5 6 7 8
O(m log n) distance 7 3 5 2 4 1 6 0 8
updates overall HtoV 0 1 2 3 4 5 6 7 8
7 5 3 1 4 2 6 0 8
Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 7/9
Dijkstra’s algorithm
Using min-heaps 7:29
Identifying next vertex to
visit is O(log n)
Updating distance takes 5:38 3:31
O(log n) per neighbour
Adjacency list —
proportionally to degree 1:54 4:47 2:84 6:61

Cumulatively
O(n log n) to identify 0:68 8:59
vertices to visit across n
iterations VtoH 0 1 2 3 4 5 6 7 8
O(m log n) distance 7 3 5 2 4 1 6 0 8
updates overall HtoV 0 1 2 3 4 5 6 7 8
7 5 3 1 4 2 6 0 8
Overall O((m + n) log n)
Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 7/9
Heap sort
Start with an unordered list

Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 8/9
Heap sort
Start with an unordered list
Build a heap — O(n)

Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 8/9
Heap sort
Start with an unordered list
Build a heap — O(n)
Call delete max() n times to extract elements in descending order — O(n log n)

Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 8/9
Heap sort
Start with an unordered list
Build a heap — O(n)
Call delete max() n times to extract elements in descending order — O(n log n)
After each delete max(), heap shrinks by 1

Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 8/9
Heap sort
Start with an unordered list
Build a heap — O(n)
Call delete max() n times to extract elements in descending order — O(n log n)
After each delete max(), heap shrinks by 1
Store maximum value at the end of current heap

Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 8/9
Heap sort
Start with an unordered list
Build a heap — O(n)
Call delete max() n times to extract elements in descending order — O(n log n)
After each delete max(), heap shrinks by 1
Store maximum value at the end of current heap
In place O(n log n) sort

Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 8/9
Summary

Updating a value in a heap takes O(log n)

Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 9/9
Summary

Updating a value in a heap takes O(log n)

Need to maintain additional pointers to map values to heap positions and vice versa

Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 9/9
Summary

Updating a value in a heap takes O(log n)

Need to maintain additional pointers to map values to heap positions and vice versa
With this extended notion of heap, Dijkstra’s algorithm complexity improves from
O(n2 ) to O((m + n) log n)

Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 9/9
Summary

Updating a value in a heap takes O(log n)

Need to maintain additional pointers to map values to heap positions and vice versa
With this extended notion of heap, Dijkstra’s algorithm complexity improves from
O(n2 ) to O((m + n) log n)
In a similar way, improve Prim’s algorithm to O((m + n) log n)

Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 9/9
Summary

Updating a value in a heap takes O(log n)

Need to maintain additional pointers to map values to heap positions and vice versa
With this extended notion of heap, Dijkstra’s algorithm complexity improves from
O(n2 ) to O((m + n) log n)
In a similar way, improve Prim’s algorithm to O((m + n) log n)
Heaps can also be used to sort a list in place in O(n log n)

Madhavan Mukund Using Heaps in Algorithms PDSA using Python Week 6 9/9