{"id":"dsa-complexity-big-o","slug":"big-o","title":"Big-O in practice","text":"Two programs can give the same answer while one finishes in a blink and the other runs for an hour. The computer isn't the difference. What matters is **how the work grows** as the input grows: with 10 values both are instant, with a million values only one of them is.\n\nYou could time a program with a clock, but the time changes from computer to computer and with whatever else the computer is doing. Counting steps doesn't. Here three functions count how many times their innermost line runs:\n\n```python\ndef one_loop(n):\n    steps = 0\n    for i in range(n):\n        steps += 1\n    return steps\n\n\ndef every_pair(n):\n    steps = 0\n    for i in range(n):\n        for j in range(i + 1, n):\n            steps += 1\n    return steps\n\n\ndef halving(n):\n    steps = 0\n    while n > 1:\n        n = n // 2\n        steps += 1\n    return steps\n\n\nfor n in [10, 100, 1000]:\n    print(n, one_loop(n), every_pair(n), halving(n))\n```\n\n```output\n10 10 45 3\n100 100 4950 6\n1000 1000 499500 9\n```\n\nRead the columns down. `one_loop` grows with n: ten times the input, ten times the work. `every_pair` meets each pair of values once, n × (n - 1) / 2 steps: ten times the input, about a hundred times the work. `halving` halves n until 1 is left, so ten times the input adds only about three steps.\n\n### Big-O\n\n**Big-O notation** names that growth and leaves out the details. `one_loop` is **O(n)**, `every_pair` is **O(n²)** and `halving` is **O(log n)**. Two more come up all the time: **O(1)**, the same work whatever the size, and **O(n log n)**, the cost of sorting.\n\n| Big-O | n = 1,000 | n = 1,000,000 | For example |\n| --- | --- | --- | --- |\n| O(1) | 1 | 1 | `values[i]`, `len(values)` |\n| O(log n) | 10 | 20 | binary search |\n| O(n) | 1,000 | 1,000,000 | one loop over the list |\n| O(n log n) | 10,000 | 20,000,000 | `sorted(values)` |\n| O(n²) | 1,000,000 | 1,000,000,000,000 | a loop over every pair |\n\nBig-O drops **constant factors**: `every_pair` does about n²/2 steps, and that is still O(n²). Two loops one after the other do 2n steps, still O(n). For big inputs, the growth beats any constant. Plain Python does roughly ten to fifty million simple steps a second, so an O(n²) loop over 200,000 values, 20 billion steps, runs for many minutes, while an O(n) loop over them takes a fraction of a second.\n\n### Loops you don't see\n\nA line can be a loop in disguise. `x in values` on a list, `values.index(x)`, `values.count(x)`, `values.remove(x)`, `max(values)` and `sum(values)` go through the items one by one: O(n) each. A slice `values[a:b]` copies its items, and `values.pop(0)` or `values.insert(0, x)` moves every other item by one place. Put one of them in a loop over the list and you have O(n²) without a second `for`. (To take items from the front again and again, use a `collections.deque`, the queue of the module on trees: it removes its first item in O(1).)\n\nThese two functions find the best profit from buying on one day and selling on a later day. Both are right:\n\n```python\ndef best_profit_slow(prices):\n    best = 0\n    for i in range(len(prices) - 1):\n        best = max(best, max(prices[i + 1:]) - prices[i])\n    return best\n\n\ndef best_profit(prices):\n    best = 0\n    lowest = prices[0]\n    for price in prices:\n        lowest = min(lowest, price)\n        best = max(best, price - lowest)\n    return best\n\n\nprices = [9, 11, 8, 5, 7, 10, 6]\nprint(best_profit_slow(prices))\nprint(best_profit(prices))\n```\n\n```output\n5\n5\n```\n\n`best_profit_slow` has one `for`, but `max(prices[i + 1:])` copies and scans the rest of the list in every round: O(n²) in all. `best_profit` remembers the lowest price so far instead, so each round is O(1): the function is O(n) time, with O(1) extra memory (two variables, whatever the length of the list). Remembering something as you go is the most common way to remove a hidden loop.\n\n### Sorting first\n\nSorting costs O(n log n), and it often pays: after sorting, equal and close values sit next to each other. To find out whether a list has a repeated value, comparing every pair is O(n²). In a sorted list, a repeated value sits right next to its twin, so one pass over the neighbors is enough: O(n log n) for the sort plus O(n) for the pass, O(n log n) in all.\n\n```python\ndef has_repeat(values):\n    ordered = sorted(values)\n    for i in range(len(ordered) - 1):\n        if ordered[i] == ordered[i + 1]:\n            return True\n    return False\n\n\nprint(has_repeat([31, 7, 19, 7, 2]))\nprint(has_repeat([31, 7, 19, 8, 2]))\n```\n\n```output\nTrue\nFalse\n```\n\n`sorted` makes a new sorted list, O(n) extra memory, and leaves the original as it was.\n","task":"What does this program print? Count the steps of each function by hand, and watch how each count grows when n doubles. Type each line it prints.\n","kind":"predict","seed":"def a(n):\n    steps = 0\n    for i in range(n):\n        for j in range(3):\n            steps += 1\n    return steps\n\n\ndef b(n):\n    steps = 0\n    for i in range(n):\n        for j in range(i):\n            steps += 1\n    return steps\n\n\ndef c(n):\n    steps = 0\n    while n > 1:\n        n = n // 2\n        steps += 1\n    return steps\n\n\nprint(a(8), b(8), c(8))\nprint(a(16), b(16), c(16))\n","solution":"def a(n):\n    steps = 0\n    for i in range(n):\n        for j in range(3):\n            steps += 1\n    return steps\n\n\ndef b(n):\n    steps = 0\n    for i in range(n):\n        for j in range(i):\n            steps += 1\n    return steps\n\n\ndef c(n):\n    steps = 0\n    while n > 1:\n        n = n // 2\n        steps += 1\n    return steps\n\n\nprint(a(8), b(8), c(8))\nprint(a(16), b(16), c(16))\n","hints":["In `a`, the inner loop always runs 3 times, whatever `i` is. In `b`, it runs `i` times: 0 times, then 1, then 2, and so on.","`b(8)` is 0 + 1 + 2 + ... + 7 = 28. `c` halves n until it reaches 1: 8, 4, 2, 1 is three halvings.","When n doubles, `a` doubles (O(n)), `b` grows about four times (O(n²)), and `c` grows by one (O(log n))."],"tests":[{"name":"Output","stdin":"","expect":"24 28 3\n48 120 4","hidden":false}],"require":[],"forbid":[],"lines":["24 28 3","48 120 4"],"more":[{"task":"A **record** is a value greater than every value before it. The first value is always a record. This function counts the records in a list, and it gives the right answers, but `max(values[:i])` is a hidden loop: it makes the function O(n²), and the hidden test has 200,000 values. Make it O(n), without changing what it returns.\n\nFor example, `count_records([3, 1, 4, 1, 5, 9, 2, 6])` returns (the records are 3, 4, 5 and 9):\n\n```\n4\n```\n","kind":"code","seed":"def count_records(values):\n    count = 0\n    for i in range(len(values)):\n        if i == 0 or values[i] > max(values[:i]):\n            count += 1\n    return count\n\n\nif __name__ == \"__main__\":\n    print(count_records([3, 1, 4, 1, 5, 9, 2, 6]))\n","solution":"def count_records(values):\n    count = 0\n    highest = None\n    for value in values:\n        if highest is None or value > highest:\n            count += 1\n            highest = value\n    return count\n\n\nif __name__ == \"__main__\":\n    print(count_records([3, 1, 4, 1, 5, 9, 2, 6]))\n","hints":["`max(values[:i])` is the highest value before position `i`. Instead of working it out again in every round, remember it as you go, as `best_profit` remembers the lowest price.","Keep a variable `highest`, the greatest value so far. A value greater than `highest` is a record, and it becomes the new `highest`.","Mind the first value and negative values: don't start `highest` at 0. Start it at `None` (and treat the first value as a record), or at `values[0]` when the list isn't empty."],"tests":[{"name":"count_records([3, 1, 4, 1, 5, 9, 2, 6])","stdin":"","call":"count_records([3, 1, 4, 1, 5, 9, 2, 6])\n","expect":"4","hidden":false,"returns":true},{"name":"count_records([])","stdin":"","call":"count_records([])\n","expect":"0","hidden":true,"returns":true},{"name":"count_records([5, 5, 5])","stdin":"","call":"count_records([5, 5, 5])\n","expect":"1","hidden":true,"returns":true},{"name":"count_records([-3, -1, -2, 0])","stdin":"","call":"count_records([-3, -1, -2, 0])\n","expect":"3","hidden":true,"returns":true},{"name":"count_records([10, 9, 8, 7])","stdin":"","call":"count_records([10, 9, 8, 7])\n","expect":"1","hidden":true,"returns":true},{"name":"count_records([i * 7919 % 200_003 for i in range(200_000)])","stdin":"","call":"count_records([i * 7919 % 200_003 for i in range(200_000)])\n","expect":"60","hidden":true,"big":true,"returns":true}],"require":[],"forbid":[]},{"task":"Write `smallest_gap(values)`: the smallest difference between two values of the list (the larger one minus the smaller one). The list has at least two values. Comparing every pair is O(n²), too slow for the hidden test's 100,000 values: sort first, and the closest values sit next to each other.\n\nFor example, `smallest_gap([40, 7, 22, 31, 15])` returns (22 - 15):\n\n```\n7\n```\n","kind":"code","seed":"def smallest_gap(values):\n    pass\n\n\nif __name__ == \"__main__\":\n    print(smallest_gap([40, 7, 22, 31, 15]))\n","solution":"def smallest_gap(values):\n    ordered = sorted(values)\n    best = ordered[1] - ordered[0]\n    for i in range(1, len(ordered) - 1):\n        gap = ordered[i + 1] - ordered[i]\n        if gap < best:\n            best = gap\n    return best\n\n\nif __name__ == \"__main__\":\n    print(smallest_gap([40, 7, 22, 31, 15]))\n","hints":["In a sorted list, the two closest values are always neighbors: any value between two others is closer to each of them than they are to each other.","Sort the values with `sorted`, then go through the neighbors `ordered[i]` and `ordered[i + 1]` and keep the smallest difference.","Start `best` from the first pair of neighbors, `ordered[1] - ordered[0]`, so you have a real difference to compare with."],"tests":[{"name":"smallest_gap([40, 7, 22, 31, 15])","stdin":"","call":"smallest_gap([40, 7, 22, 31, 15])\n","expect":"7","hidden":false,"returns":true},{"name":"smallest_gap([5, 1])","stdin":"","call":"smallest_gap([5, 1])\n","expect":"4","hidden":true,"returns":true},{"name":"smallest_gap([3, 9, 3])","stdin":"","call":"smallest_gap([3, 9, 3])\n","expect":"0","hidden":true,"returns":true},{"name":"smallest_gap([-10, 4, -2, 15])","stdin":"","call":"smallest_gap([-10, 4, -2, 15])\n","expect":"6","hidden":true,"returns":true},{"name":"smallest_gap([100, 1, 50, 120, 70])","stdin":"","call":"smallest_gap([100, 1, 50, 120, 70])\n","expect":"20","hidden":true,"returns":true},{"name":"smallest_gap([50, 2, 30, 49])","stdin":"","call":"smallest_gap([50, 2, 30, 49])\n","expect":"1","hidden":true,"returns":true},{"name":"smallest_gap([i * i % 1_000_003 for i in range(1, 100_001)])","stdin":"","call":"smallest_gap([i * i % 1_000_003 for i in range(1, 100_001)])\n","expect":"1","hidden":true,"big":true,"returns":true}],"require":[],"forbid":[]}]}