{"id":"dsa-hash-lru-cache","slug":"lru-cache","title":"Challenge: an LRU cache","text":"A **cache** keeps answers that were slow to get, so that asking again is fast: pages a browser downloaded, rows a program read from a database. A cache can't keep everything, so when it's full, something has to go. A **least recently used (LRU)** cache throws out the item that was used longest ago, betting that what you used lately you'll use again soon. It's a favorite interview question because it needs two things in O(1) at once: finding an item by its key, and finding the oldest one.\n\n### A dict remembers its order\n\nA dict keeps its keys in the order they were added. (Its table holds only positions, and the pairs themselves sit in a list in the order they came.) Giving a key that's already there a new value keeps its place; removing the key and adding it again moves it to the end:\n\n```pycon\n>>> weather = {\"Oslo\": 4, \"Lima\": 19, \"Rome\": 15}\n>>> weather[\"Oslo\"] = 6\n>>> list(weather)\n['Oslo', 'Lima', 'Rome']\n>>> temp = weather.pop(\"Oslo\")\n>>> weather[\"Oslo\"] = temp\n>>> list(weather)\n['Lima', 'Rome', 'Oslo']\n```\n\nSo the order of a dict can be the order of use: the most recent key at the end, the oldest at the front.\n\n### OrderedDict\n\n`collections.OrderedDict` is a dict made for exactly this. `move_to_end(key)` moves a key to the end, and `popitem(last=False)` removes the first pair, the oldest, and returns it. Both are O(1) on average: `move_to_end` finds the key by its hash, as a dict does, and `popitem(last=False)` takes the first key in the order and deletes it from the table, like `del`.\n\n```python\nfrom collections import OrderedDict\n\nweather = OrderedDict()\nfor city, temp in [(\"Oslo\", 4), (\"Lima\", 19), (\"Rome\", 15)]:\n    weather[city] = temp\nweather.move_to_end(\"Oslo\")\nprint(list(weather))\nprint(weather.popitem(last=False))\nprint(list(weather))\n```\n\n```output\n['Lima', 'Rome', 'Oslo']\n('Lima', 19)\n['Rome', 'Oslo']\n```\n\nBeside its hash table, an OrderedDict keeps its keys in a chain where each key knows the one before it and the one after it, so it can take a key out of the middle and put it at the end without moving anything else. Such a chain is called a **doubly linked list**. An interviewer may ask you to build the cache without OrderedDict, from a plain dict and a doubly linked list of your own, which is what OrderedDict does inside: the dict finds a key's link in the chain, and the link comes out and goes to the end in O(1). The module on trees builds a linked list. A plain dict's oldest key is `next(iter(d))`, but each key removed from the front leaves an empty place that iteration has to skip until the dict is rebuilt, so taking the oldest out again and again can get slow. For a cache, use OrderedDict.\n\n### The cache\n\nWith the most recently used key at the end:\n\n- `get(key)`: if the key is in the cache, move it to the end, since it was just used, and return its value. Otherwise return `None`.\n- `put(key, value)`: store the value and move the key to the end. If the cache now holds more than `capacity` keys, remove the oldest one, at the front.\n\nEach step is an O(1) dict operation, so both methods are O(1) on average, and the cache needs O(capacity) memory. The slow way keeps the order in a list, where `order.remove(key)` searches the list: O(capacity) for every `get`.\n\n### functools.lru_cache\n\nPython has an LRU cache for functions. `@lru_cache(maxsize=...)` above a function wraps it, the way `@property` wraps a method: a call with arguments seen before returns the stored result without running the function, and once `maxsize` results are stored, a new one pushes out the least recently used. `cache_info()` counts the **hits** (answers from the cache) and the **misses**:\n\n```python\nfrom functools import lru_cache\n\n\n@lru_cache(maxsize=2)\ndef slow_square(n):\n    print(\"working out\", n)\n    return n * n\n\n\nfor n in [3, 4, 3, 5, 4]:\n    slow_square(n)\nprint(slow_square.cache_info())\n```\n\n```output\nworking out 3\nworking out 4\nworking out 5\nworking out 4\nCacheInfo(hits=1, misses=4, maxsize=2, currsize=2)\n```\n\nThe second 3 is a hit and makes 3 the most recent, so 5 pushes out 4, and the last call has to work 4 out again. With `maxsize=None` the cache never forgets (`functools.cache` is the same), which is **memoization**: a recursive function such as Fibonacci then works out each number only once. The arguments become keys of a dict inside, so they must be hashable: a list argument stops with `TypeError: unhashable type: 'list'`.\n","task":"Write the class `LRUCache`. `LRUCache(capacity)` makes an empty cache for at most `capacity` keys (at least 1), and `get(key)` and `put(key, value)` work as described above. Both must be O(1): the hidden test makes 650,000 calls on a cache of 100,000 keys, reading keys from every part of the order and pushing out 350,000 of them. This program:\n\n```python\ncache = LRUCache(2)\ncache.put(\"a\", 1)\ncache.put(\"b\", 2)\nprint(cache.get(\"a\"))\ncache.put(\"c\", 3)\nprint(cache.get(\"b\"), cache.get(\"a\"), cache.get(\"c\"))\n```\n\nprints (reading `\"a\"` made `\"b\"` the least recently used, so `\"c\"` pushed `\"b\"` out):\n\n```\n1\nNone 1 3\n```\n","kind":"code","seed":"from collections import OrderedDict\n\n\nclass LRUCache:\n    def __init__(self, capacity):\n        self.capacity = capacity\n\n    def get(self, key):\n        pass\n\n    def put(self, key, value):\n        pass\n\n\nif __name__ == \"__main__\":\n    cache = LRUCache(2)\n    cache.put(\"a\", 1)\n    cache.put(\"b\", 2)\n    print(cache.get(\"a\"))\n    cache.put(\"c\", 3)\n    print(cache.get(\"b\"), cache.get(\"a\"), cache.get(\"c\"))\n","solution":"from collections import OrderedDict\n\n\nclass LRUCache:\n    def __init__(self, capacity):\n        self.capacity = capacity\n        self.items = OrderedDict()\n\n    def get(self, key):\n        if key not in self.items:\n            return None\n        self.items.move_to_end(key)\n        return self.items[key]\n\n    def put(self, key, value):\n        self.items[key] = value\n        self.items.move_to_end(key)\n        if len(self.items) > self.capacity:\n            self.items.popitem(last=False)\n\n\nif __name__ == \"__main__\":\n    cache = LRUCache(2)\n    cache.put(\"a\", 1)\n    cache.put(\"b\", 2)\n    print(cache.get(\"a\"))\n    cache.put(\"c\", 3)\n    print(cache.get(\"b\"), cache.get(\"a\"), cache.get(\"c\"))\n","hints":["Keep the pairs in an `OrderedDict`, oldest first. `get` returns `None` for a key that isn't there; otherwise it calls `move_to_end(key)` and returns the value.","`put` stores the value and calls `move_to_end(key)`: a key that was already there keeps its old place otherwise. If there are now more than `capacity` keys, `popitem(last=False)` removes the oldest.","Keeping the order in a list and calling `remove` on it costs O(capacity) for each call, whichever end holds the newest key, and so does finding the oldest key with `list(self.items)[0]`. A plain dict whose oldest key is `next(iter(...))` gets slow too, as it has to skip every key already removed from the front. All of these are too slow for the big test."],"tests":[{"name":"Test 1","stdin":"","call":"cache = LRUCache(2)\ncache.put(\"a\", 1)\ncache.put(\"b\", 2)\nprint(cache.get(\"a\"))\ncache.put(\"c\", 3)\nprint(cache.get(\"b\"), cache.get(\"a\"), cache.get(\"c\"))\n","expect":"1\nNone 1 3","hidden":false},{"name":"Test 2","stdin":"","call":"cache = LRUCache(2)\ncache.put(\"a\", 1)\ncache.put(\"b\", 2)\ncache.put(\"a\", 10)\ncache.put(\"c\", 3)\nprint(cache.get(\"a\"), cache.get(\"b\"), cache.get(\"c\"))\n","expect":"10 None 3","hidden":true},{"name":"Test 3","stdin":"","call":"cache = LRUCache(2)\ncache.put(\"a\", 1)\ncache.put(\"b\", 2)\ncache.put(\"b\", 20)\nprint(cache.get(\"a\"), cache.get(\"b\"))\ncache.put(\"c\", 3)\nprint(cache.get(\"a\"), cache.get(\"b\"), cache.get(\"c\"))\n","expect":"1 20\nNone 20 3","hidden":true},{"name":"Test 4","stdin":"","call":"cache = LRUCache(1)\ncache.put(\"x\", 1)\ncache.put(\"y\", 2)\nprint(cache.get(\"x\"), cache.get(\"y\"))\n","expect":"None 2","hidden":true},{"name":"Test 5","stdin":"","call":"cache = LRUCache(2)\ncache.put(1, \"one\")\ncache.put(2, \"two\")\nprint(cache.get(3))\ncache.put(4, \"four\")\nprint(cache.get(1), cache.get(2), cache.get(4))\n","expect":"None\nNone two four","hidden":true},{"name":"Test 6","stdin":"","call":"cache = LRUCache(3)\nfor key in [1, 2, 3]:\n    cache.put(key, key * 10)\ncache.get(1)\ncache.get(2)\ncache.put(4, 40)\ncache.put(5, 50)\nprint([cache.get(key) for key in [1, 2, 3, 4, 5]])\n","expect":"[None, 20, None, 40, 50]","hidden":true},{"name":"Test 7","stdin":"","call":"cache = LRUCache(100_000)\nfor i in range(100_000):\n    cache.put(i, i)\ntotal = 0\nfor i in range(100_000):\n    total += cache.get(i * 7 % 100_000) + cache.get((i - 5) * 7 % 100_000)\nfor i in range(100_000, 450_000):\n    cache.put(i, 0)\nprint(total, cache.get(99_999), cache.get(449_999))\n","expect":"9999900000 None 0","hidden":true,"big":true}],"require":[],"forbid":[],"more":[{"task":"What does this program print? Type each line it prints.\n","kind":"predict","seed":"from functools import lru_cache\n\n\n@lru_cache(maxsize=3)\ndef lookup(word):\n    print(\"miss\", word)\n    return len(word)\n\n\nfor word in [\"red\", \"blue\", \"red\", \"green\", \"pink\", \"blue\", \"red\"]:\n    lookup(word)\nprint(lookup.cache_info().hits)\n","solution":"from functools import lru_cache\n\n\n@lru_cache(maxsize=3)\ndef lookup(word):\n    print(\"miss\", word)\n    return len(word)\n\n\nfor word in [\"red\", \"blue\", \"red\", \"green\", \"pink\", \"blue\", \"red\"]:\n    lookup(word)\nprint(lookup.cache_info().hits)\n","hints":["The cache keeps the results for the 3 most recently used words. A word in the cache is a hit: nothing is printed, and it becomes the most recent.","Write down the cache after each call, oldest first. After `green` it holds `blue`, `red`, `green`. Which word does `pink` push out?","After `pink`, the cache holds `red`, `green`, `pink`, so the next `blue` is a miss, and it pushes out `red`."],"tests":[{"name":"Output","stdin":"","expect":"miss red\nmiss blue\nmiss green\nmiss pink\nmiss blue\nmiss red\n1","hidden":false}],"require":[],"forbid":[],"lines":["miss red","miss blue","miss green","miss pink","miss blue","miss red","1"]},{"task":"A computer's memory holds `capacity` pages, and a program asks for pages one after another. When the page it asks for is in memory, that's a hit, and the page becomes the most recently used. When it isn't, that's a **miss**: the page is loaded, and if the memory is already full, the least recently used page is dropped to make room.\n\nWrite `count_misses(requests, capacity)` that returns the number of misses, for a memory that starts empty. The hidden test has 300,000 requests and a capacity of 40,000.\n\nFor example, `count_misses([1, 2, 1, 3, 2, 4, 1, 2], 3)` returns (1, 2 and 3 load, 4 pushes out 1, and the next 1 pushes out 3):\n\n```\n5\n```\n","kind":"code","seed":"from collections import OrderedDict\n\n\ndef count_misses(requests, capacity):\n    pass\n\n\nif __name__ == \"__main__\":\n    print(count_misses([1, 2, 1, 3, 2, 4, 1, 2], 3))\n","solution":"from collections import OrderedDict\n\n\ndef count_misses(requests, capacity):\n    memory = OrderedDict()\n    misses = 0\n    for page in requests:\n        if page in memory:\n            memory.move_to_end(page)\n        else:\n            misses += 1\n            memory[page] = True\n            if len(memory) > capacity:\n                memory.popitem(last=False)\n    return misses\n\n\nif __name__ == \"__main__\":\n    print(count_misses([1, 2, 1, 3, 2, 4, 1, 2], 3))\n","hints":["Use an `OrderedDict` as the memory, with the least recently used page first. The values don't matter: store `True`.","A hit: `move_to_end(page)`. A miss: add one to the count, add the page, and if the memory now holds more than `capacity` pages, drop the oldest with `popitem(last=False)`.","A list with `in` and `remove` costs O(capacity) for every request: too slow for 300,000 requests."],"tests":[{"name":"count_misses([1, 2, 1, 3, 2, 4, 1, 2], 3)","stdin":"","call":"count_misses([1, 2, 1, 3, 2, 4, 1, 2], 3)\n","expect":"5","hidden":false,"returns":true},{"name":"count_misses([], 4)","stdin":"","call":"count_misses([], 4)\n","expect":"0","hidden":true,"returns":true},{"name":"count_misses([7, 7, 8, 7], 1)","stdin":"","call":"count_misses([7, 7, 8, 7], 1)\n","expect":"3","hidden":true,"returns":true},{"name":"count_misses([1, 2, 3, 1, 2, 3], 5)","stdin":"","call":"count_misses([1, 2, 3, 1, 2, 3], 5)\n","expect":"3","hidden":true,"returns":true},{"name":"count_misses([1, 2, 3, 4] * 3, 3)","stdin":"","call":"count_misses([1, 2, 3, 4] * 3, 3)\n","expect":"12","hidden":true,"returns":true},{"name":"count_misses([\"home\", \"news\", \"home\", \"mail\", \"news\", \"shop\", \"home\"], 2)","stdin":"","call":"count_misses([\"home\", \"news\", \"home\", \"mail\", \"news\", \"shop\", \"home\"], 2)\n","expect":"6","hidden":true,"returns":true},{"name":"count_misses([i * i % 100_003 for i in range(300_000)], 40_000)","stdin":"","call":"count_misses([i * i % 100_003 for i in range(300_000)], 40_000)\n","expect":"100002","hidden":true,"big":true,"returns":true}],"require":[],"forbid":[]}]}