{"id":"dsa-graphs-topological-sort","slug":"topological-sort","title":"Topological sort","text":"Some jobs must wait for others. A pizza can't go in the oven before the toppings are on, a course can't start before its prerequisites, and a program can't be built before the libraries it uses. These are directed graphs, where an edge from a to b says \"a comes before b\". An order of all the vertices in which every edge points forward, from earlier to later, is a **topological order**, and finding one is a **topological sort**.\n\nIt exists exactly when the graph has no directed **cycle**: in a, b, c, a, each of them would have to come before itself. A directed graph without cycles is a **DAG**, a directed acyclic graph.\n\n### Kahn's algorithm\n\nThe **in-degree** of a vertex, the number of edges coming in, is here the number of jobs it still waits for. A job with in-degree 0 waits for nothing and can go now. **Kahn's algorithm** works from that:\n\n1. Count every vertex's in-degree, in one pass over the edges.\n2. Put the vertices with in-degree 0 in a queue.\n3. Take a vertex from the queue and add it to the order. Its edges are now done, so lower the in-degree of the vertex at the other end of each one by 1. A vertex whose in-degree reaches 0 has nothing left to wait for, and joins the queue.\n4. When the queue is empty, the order holds every vertex, unless the graph has a cycle.\n\n```python\nfrom collections import deque\n\n\ndef topological_order(graph):\n    indegree = {v: 0 for v in graph}\n    for v in graph:\n        for w in graph[v]:\n            indegree[w] += 1\n    queue = deque([v for v in graph if indegree[v] == 0])\n    order = []\n    while queue:\n        v = queue.popleft()\n        order.append(v)\n        for w in graph[v]:\n            indegree[w] -= 1\n            if indegree[w] == 0:\n                queue.append(w)\n    return order\n\n\npizza = {\n    \"make dough\": [\"roll out\"],\n    \"make sauce\": [\"add toppings\"],\n    \"grate cheese\": [\"add toppings\"],\n    \"roll out\": [\"add toppings\"],\n    \"add toppings\": [\"bake\"],\n    \"heat oven\": [\"bake\"],\n    \"bake\": [],\n}\nprint(topological_order(pizza))\n```\n\n```output\n['make dough', 'make sauce', 'grate cheese', 'heat oven', 'roll out', 'add toppings', 'bake']\n```\n\nWhy it's right: a vertex joins the queue only when its in-degree reaches 0, that is, once every vertex with an edge into it is already in the order. So every edge points forward. Often many orders are right (the oven could be heated first), and Kahn's algorithm gives one of them. When a task wants one particular order, such as the smallest number first whenever there's a choice, use a heap from `heapq` in place of the deque.\n\n### Finding a cycle\n\nOn a cycle, each vertex waits for the one before it, so none of them ever reaches in-degree 0, and neither does anything that waits for them. The queue runs empty early, and the order comes out short:\n\n```python\nfrom collections import deque\n\n\ndef topological_order(graph):\n    indegree = {v: 0 for v in graph}\n    for v in graph:\n        for w in graph[v]:\n            indegree[w] += 1\n    queue = deque([v for v in graph if indegree[v] == 0])\n    order = []\n    while queue:\n        v = queue.popleft()\n        order.append(v)\n        for w in graph[v]:\n            indegree[w] -= 1\n            if indegree[w] == 0:\n                queue.append(w)\n    return order\n\n\nloop = {\"a\": [\"b\"], \"b\": [\"c\"], \"c\": [\"a\", \"d\"], \"d\": [], \"e\": [\"d\"]}\norder = topological_order(loop)\nprint(order)\nif len(order) < len(loop):\n    done = set(order)\n    print(\"stuck:\", [v for v in loop if v not in done])\n```\n\n```output\n['e']\nstuck: ['a', 'b', 'c', 'd']\n```\n\na, b and c are a cycle. d isn't on it, but it waits for c, so it's stuck too. Only e gets done.\n\nSo never take a short order for a finished one: compare its length with the number of vertices. The vertices left out are the ones on a cycle or behind one. (Another classic way to find a cycle is a DFS that marks each vertex \"in progress\" until everything after it is done: an edge back to a vertex in progress closes a cycle. Kahn's algorithm finds it without any recursion.)\n\n### What it costs\n\nCounting the in-degrees is one pass over the edges. After that, each vertex joins and leaves the queue once, and each edge lowers an in-degree once: O(V + E) time, and O(V + E) memory for the adjacency lists and the counts. With a heap instead of the deque, each vertex costs O(log V) more: O(V log V + E).\n\nThe tempting way is to look for a job that has nothing left to wait for, take it, and look again. Each look scans the vertices, once for every vertex taken: O(V²), billions of steps for 100,000 packages. Kahn's queue already holds the jobs that are ready, so nothing is ever searched for.\n","task":"Fill in the blanks of Kahn's algorithm: counting each vertex's in-degree, and the moment a vertex is ready to join the queue. The program should print:\n\n```\n[2, 3, 1, 4, 0]\n[1, 3]\n```\n","kind":"fill","seed":"from collections import deque\n\n\ndef order(n, edges):\n    indegree = [0] * n\n    after = [[] for _ in range(n)]\n    for a, b in edges:\n        after[a].append(b)\n        [[indegree[b] += 1]]\n    queue = deque([v for v in range(n) if indegree[v] == 0])\n    result = []\n    while queue:\n        v = queue.popleft()\n        result.append(v)\n        for w in after[v]:\n            indegree[w] -= 1\n            if [[indegree[w] == 0]]:\n                queue.append(w)\n    return result\n\n\nprint(order(5, [(3, 1), (1, 0), (3, 4), (4, 0), (2, 4)]))\nprint(order(4, [(3, 0), (0, 2), (2, 0), (1, 3)]))\n","solution":"from collections import deque\n\n\ndef order(n, edges):\n    indegree = [0] * n\n    after = [[] for _ in range(n)]\n    for a, b in edges:\n        after[a].append(b)\n        indegree[b] += 1\n    queue = deque([v for v in range(n) if indegree[v] == 0])\n    result = []\n    while queue:\n        v = queue.popleft()\n        result.append(v)\n        for w in after[v]:\n            indegree[w] -= 1\n            if indegree[w] == 0:\n                queue.append(w)\n    return result\n\n\nprint(order(5, [(3, 1), (1, 0), (3, 4), (4, 0), (2, 4)]))\nprint(order(4, [(3, 0), (0, 2), (2, 0), (1, 3)]))\n","hints":["An edge from a to b is one more thing b waits for, so it adds 1 to b's in-degree: `indegree[b] += 1`.","A vertex is ready when nothing is left to wait for, the moment its in-degree reaches 0: `indegree[w] == 0`."],"tests":[{"name":"Output","stdin":"","expect":"[2, 3, 1, 4, 0]\n[1, 3]","hidden":false}],"require":[],"forbid":[],"more":[{"task":"Packages numbered 0 to n - 1 must be installed so that each one comes after the packages it needs. `deps` lists pairs `(a, b)`: package a must be installed before package b. Write `install_order(n, deps)` that returns the order as a list, taking the smallest-numbered package whenever several are ready, or `None` when there's a cycle and no order exists.\n\nKeep the ready packages in a heap with `heapq`, so the smallest one comes out in O(log n). The hidden tests have 100,000 packages: looking for the next ready package with a scan of all of them is O(n²).\n\nFor example, here 0, 4 and 5 are ready at the start. After 5, both 1 and 2 are, and 1 goes first. So this program:\n\n```python\nprint(install_order(6, [(5, 2), (2, 3), (4, 3), (0, 1), (5, 1)]))\nprint(install_order(4, [(1, 3), (3, 2), (2, 1), (0, 2)]))\n```\n\nprints:\n\n```\n[0, 4, 5, 1, 2, 3]\nNone\n```\n","kind":"code","seed":"import heapq\n\n\ndef install_order(n, deps):\n    pass\n\n\nif __name__ == \"__main__\":\n    print(install_order(6, [(5, 2), (2, 3), (4, 3), (0, 1), (5, 1)]))\n    print(install_order(4, [(1, 3), (3, 2), (2, 1), (0, 2)]))\n","solution":"import heapq\n\n\ndef install_order(n, deps):\n    needs = [0] * n\n    after = [[] for _ in range(n)]\n    for a, b in deps:\n        after[a].append(b)\n        needs[b] += 1\n    ready = [p for p in range(n) if needs[p] == 0]\n    heapq.heapify(ready)\n    order = []\n    while ready:\n        p = heapq.heappop(ready)\n        order.append(p)\n        for q in after[p]:\n            needs[q] -= 1\n            if needs[q] == 0:\n                heapq.heappush(ready, q)\n    if len(order) < n:\n        return None\n    return order\n\n\nif __name__ == \"__main__\":\n    print(install_order(6, [(5, 2), (2, 3), (4, 3), (0, 1), (5, 1)]))\n    print(install_order(4, [(1, 3), (3, 2), (2, 1), (0, 2)]))\n","hints":["Count each package's in-degree and build the lists of what comes after each one, in one pass over `deps`. The packages with in-degree 0 are ready.","Put the ready packages in a list and `heapq.heapify` it. Then, while it isn't empty, `heapq.heappop` gives the smallest ready package: add it to the order, and lower the in-degree of each package after it. One that reaches 0 goes in with `heapq.heappush`.","When the heap runs empty, the order is complete only if it has all n packages. If it's shorter, the rest are on a cycle or wait for one: return `None`."],"tests":[{"name":"Test 1","stdin":"","call":"print(install_order(6, [(5, 2), (2, 3), (4, 3), (0, 1), (5, 1)]))\nprint(install_order(4, [(1, 3), (3, 2), (2, 1), (0, 2)]))\n","expect":"[0, 4, 5, 1, 2, 3]\nNone","hidden":false},{"name":"install_order(5, [])","stdin":"","call":"install_order(5, [])\n","expect":"[0, 1, 2, 3, 4]","hidden":true,"returns":true},{"name":"install_order(1, [])","stdin":"","call":"install_order(1, [])\n","expect":"[0]","hidden":true,"returns":true},{"name":"install_order(4, [(3, 1), (1, 2), (2, 0)])","stdin":"","call":"install_order(4, [(3, 1), (1, 2), (2, 0)])\n","expect":"[3, 1, 2, 0]","hidden":true,"returns":true},{"name":"install_order(5, [(4, 0), (4, 2), (0, 3), (2, 3), (3, 1)])","stdin":"","call":"install_order(5, [(4, 0), (4, 2), (0, 3), (2, 3), (3, 1)])\n","expect":"[4, 0, 2, 3, 1]","hidden":true,"returns":true},{"name":"install_order(5, [(4, 1), (3, 1), (2, 0)])","stdin":"","call":"install_order(5, [(4, 1), (3, 1), (2, 0)])\n","expect":"[2, 0, 3, 4, 1]","hidden":true,"returns":true},{"name":"print(install_order(5, [(3, 4), (4, 2), (2, 4), (0, 1)]))","stdin":"","call":"print(install_order(5, [(3, 4), (4, 2), (2, 4), (0, 1)]))\n","expect":"None","hidden":true},{"name":"Test 8","stdin":"","call":"order = install_order(100_000, [(i + 1, i) for i in range(99_999)])\nprint(len(order), order[:3], order[-1])\n","expect":"100000 [99999, 99998, 99997] 0","hidden":true,"big":true},{"name":"Test 9","stdin":"","call":"order = install_order(100_000, [(i, i + 50_000) for i in range(50_000)])\nprint(len(order), order[:3], order[49_999:50_002], order[-1])\n","expect":"100000 [0, 1, 2] [49999, 50000, 50001] 99999","hidden":true,"big":true},{"name":"print(install_order(100_000, [(i + 1, i) for i in range(99_999)] + [(0, 99_999)]))","stdin":"","call":"print(install_order(100_000, [(i + 1, i) for i in range(99_999)] + [(0, 99_999)]))\n","expect":"None","hidden":true,"big":true}],"require":[],"forbid":[]},{"task":"A school has n courses, numbered 0 to n - 1. `prereqs` lists pairs `(a, b)`: course a must be passed in an earlier term than course b. Any number of courses can be taken in the same term. Write `fewest_terms(n, prereqs)` that returns the fewest terms needed to pass every course, or -1 when that's impossible because of a cycle.\n\nRun Kahn's algorithm one **level** at a time, like the level-by-level BFS of the module on trees: the courses that are ready at the start make up the first term, and the courses they unlock make up the next. The hidden tests have 60,000 courses and up to 60,000 terms, so don't scan all the courses again for every term.\n\nFor example, courses 0 and 1 can be taken in the first term, then 2 and 4, and 3 last. So this program:\n\n```python\nprint(fewest_terms(5, [(0, 2), (1, 2), (2, 3), (1, 4)]))\nprint(fewest_terms(2, [(0, 1), (1, 0)]))\n```\n\nprints:\n\n```\n3\n-1\n```\n","kind":"code","seed":"def fewest_terms(n, prereqs):\n    pass\n\n\nif __name__ == \"__main__\":\n    print(fewest_terms(5, [(0, 2), (1, 2), (2, 3), (1, 4)]))\n    print(fewest_terms(2, [(0, 1), (1, 0)]))\n","solution":"def fewest_terms(n, prereqs):\n    needs = [0] * n\n    unlocks = [[] for _ in range(n)]\n    for a, b in prereqs:\n        unlocks[a].append(b)\n        needs[b] += 1\n    term = [c for c in range(n) if needs[c] == 0]\n    terms = 0\n    passed = 0\n    while term:\n        terms += 1\n        passed += len(term)\n        next_term = []\n        for course in term:\n            for later in unlocks[course]:\n                needs[later] -= 1\n                if needs[later] == 0:\n                    next_term.append(later)\n        term = next_term\n    if passed < n:\n        return -1\n    return terms\n\n\nif __name__ == \"__main__\":\n    print(fewest_terms(5, [(0, 2), (1, 2), (2, 3), (1, 4)]))\n    print(fewest_terms(2, [(0, 1), (1, 0)]))\n","hints":["Count the in-degrees and build the lists of the courses each course unlocks, as for Kahn's algorithm. The courses with in-degree 0 make up the first term.","For each term, go through its courses and lower the in-degree of every course they unlock. The ones that reach 0 make up the next term. Count the terms until a term is empty.","Count the courses passed too. If they're fewer than n at the end, some courses are on a cycle or behind one: return -1."],"tests":[{"name":"Test 1","stdin":"","call":"print(fewest_terms(5, [(0, 2), (1, 2), (2, 3), (1, 4)]))\nprint(fewest_terms(2, [(0, 1), (1, 0)]))\n","expect":"3\n-1","hidden":false},{"name":"fewest_terms(6, [])","stdin":"","call":"fewest_terms(6, [])\n","expect":"1","hidden":true,"returns":true},{"name":"fewest_terms(4, [(0, 1), (1, 2), (2, 3)])","stdin":"","call":"fewest_terms(4, [(0, 1), (1, 2), (2, 3)])\n","expect":"4","hidden":true,"returns":true},{"name":"fewest_terms(4, [(3, 2), (2, 1), (0, 1)])","stdin":"","call":"fewest_terms(4, [(3, 2), (2, 1), (0, 1)])\n","expect":"3","hidden":true,"returns":true},{"name":"fewest_terms(6, [(0, 1), (1, 2), (0, 3), (3, 4), (4, 5), (5, 2)])","stdin":"","call":"fewest_terms(6, [(0, 1), (1, 2), (0, 3), (3, 4), (4, 5), (5, 2)])\n","expect":"5","hidden":true,"returns":true},{"name":"fewest_terms(5, [(0, 1), (1, 2), (2, 3), (3, 1), (0, 4)])","stdin":"","call":"fewest_terms(5, [(0, 1), (1, 2), (2, 3), (3, 1), (0, 4)])\n","expect":"-1","hidden":true,"returns":true},{"name":"fewest_terms(60_000, [(i, i + 1) for i in range(59_999)])","stdin":"","call":"fewest_terms(60_000, [(i, i + 1) for i in range(59_999)])\n","expect":"60000","hidden":true,"big":true,"returns":true},{"name":"fewest_terms(60_000, [(i // 2, i) for i in range(1, 60_000)] + [(i, i + 1) for i in range(40_000, 59_999)])","stdin":"","call":"fewest_terms(60_000, [(i // 2, i) for i in range(1, 60_000)] + [(i, i + 1) for i in range(40_000, 59_999)])\n","expect":"20016","hidden":true,"big":true,"returns":true}],"require":[],"forbid":[]}]}