{"id":"dsa-dp-lcs","slug":"lcs","title":"Longest common subsequence and edit distance","text":"Cross out some characters of a string, and what's left is a **subsequence** of it: the characters that stay keep their order, with gaps where others were. `\"ace\"` is a subsequence of `\"abcde\"`. Take two strings, and the longest string you can get from **both** by crossing out characters is their **longest common subsequence** (LCS). That's how `diff` and Git compare two versions of a file, line by line: the lines in the LCS are the ones nobody touched. Trying every subsequence is hopeless, since a string of n characters has 2ⁿ of them. DP works on **prefixes** instead, the first few characters of each string.\n\n### The table\n\n`length[i][j]` holds the length of the LCS of `a[:i]` and `b[:j]`, the first `i` characters of `a` and the first `j` of `b`. Each entry depends on the last characters of those prefixes, `a[i - 1]` and `b[j - 1]`:\n\n- When they **match**, the LCS of the two prefixes may as well finish with that character, after the LCS of the shorter prefixes: `length[i][j] = length[i - 1][j - 1] + 1`.\n- When they don't, a common subsequence can't finish with both, so one of the two can go. Try leaving out each and keep the longer result: `length[i][j] = max(length[i - 1][j], length[i][j - 1])`.\n\nAn empty prefix has nothing in common with anything, so row 0 and column 0 are all 0. That's why the table has `len(a) + 1` rows and `len(b) + 1` columns, and why the characters are `a[i - 1]` and `b[j - 1]`: entry `i` is about the first `i` characters, and the last of them is at index `i - 1`.\n\n```python\na, b = \"PYTHON\", \"TYPHOON\"\nlength = [[0] * (len(b) + 1) for _ in range(len(a) + 1)]\nfor i in range(1, len(a) + 1):\n    for j in range(1, len(b) + 1):\n        if a[i - 1] == b[j - 1]:\n            length[i][j] = length[i - 1][j - 1] + 1\n        else:\n            length[i][j] = max(length[i - 1][j], length[i][j - 1])\n\nprint(\"   \" + \"  \".join(b))\nfor i in range(1, len(a) + 1):\n    print(a[i - 1], length[i][1:])\n```\n\n```output\n   T  Y  P  H  O  O  N\nP [0, 0, 1, 1, 1, 1, 1]\nY [0, 1, 1, 1, 1, 1, 1]\nT [1, 1, 1, 1, 1, 1, 1]\nH [1, 1, 1, 2, 2, 2, 2]\nO [1, 1, 1, 2, 3, 3, 3]\nN [1, 1, 1, 2, 3, 3, 4]\n```\n\nThe bottom-right entry is the answer: the LCS is 4 characters long. The table has (n + 1) × (m + 1) entries for strings of n and m characters, each O(1): O(n × m) time and memory. For two strings of 1,000 characters, that's a million entries instead of 2¹⁰⁰⁰ subsequences.\n\nThe classic slip is to compare `a[i]` and `b[j]`, out of habit:\n\n```python\na, b = \"cat\", \"act\"\nlength = [[0] * (len(b) + 1) for _ in range(len(a) + 1)]\nfor i in range(1, len(a) + 1):\n    for j in range(1, len(b) + 1):\n        if a[i] == b[j]:              # should be a[i - 1] and b[j - 1]\n            length[i][j] = length[i - 1][j - 1] + 1\n        else:\n            length[i][j] = max(length[i - 1][j], length[i][j - 1])\n```\n\n```crash\nTraceback (most recent call last):\n  File \"main.py\", line 5, in <module>\n    if a[i] == b[j]:              # should be a[i - 1] and b[j - 1]\n               ~^^^\nIndexError: string index out of range\n```\n\nRow 1 is about `\"c\"`, but it compares `a[1]`, the `\"a\"`, and at `j = 3` it asks for `b[3]`, one past the end of `\"act\"`.\n\n### Getting the string back\n\nThe table holds lengths, but it also remembers which choice made each entry, so you can walk back from the bottom-right corner, `i = len(a)` and `j = len(b)`, until `i` or `j` is 0:\n\n- When `a[i - 1]` and `b[j - 1]` are the same character, it's part of the answer. Save it in a list, then step back in both strings: `i - 1` and `j - 1`.\n- Otherwise go to the neighbor the entry came from: up (`i - 1`) when `length[i - 1][j] >= length[i][j - 1]`, else left (`j - 1`).\n\nThe walk finds the characters from the end backward, so `found[::-1]` turns the list around. In the PYTHON table it starts at N, where the letters match, and finds `\"PHON\"`. Two strings often have several longest common subsequences (`\"THON\"` and `\"YHON\"` are just as long), and a different rule for ties finds a different one.\n\n### Edit distance\n\nThe **edit distance** (or Levenshtein distance) between two words is the fewest one-character changes that turn one into the other, where a change **inserts**, **deletes** or **replaces** a character. `\"brain\"` becomes `\"bread\"` in 3, for example by inserting e after the r, deleting the i and replacing the n with a d. A spell checker offers the words at the smallest distance from your typo.\n\n`dist[i][j]` holds the edit distance from `a[:i]` to `b[:j]`:\n\n- The edges come first. Emptying `i` characters takes `i` deletes, so `dist[i][0] = i`, and building `j` characters from nothing takes `j` inserts, so `dist[0][j] = j`. Leaving these edges at 0 is the common mistake: the distance from `\"abc\"` to `\"\"` then comes out 0.\n- When `a[i - 1]` and `b[j - 1]` are the same, the ends need no change, and what's left costs `dist[i - 1][j - 1]`.\n- When they're different, one change fixes the end, on top of what came before: a replace of `a[i - 1]` by `b[j - 1]` (`dist[i - 1][j - 1] + 1`), a delete of `a[i - 1]` (`dist[i - 1][j] + 1`) or an insert of `b[j - 1]` (`dist[i][j - 1] + 1`). `dist[i][j]` is the smallest of the three.\n\nHere is the table from `\"cat\"` (down) to `\"act\"` (across):\n\n| | `\"\"` | a | c | t |\n| --- | --- | --- | --- | --- |\n| `\"\"` | 0 | 1 | 2 | 3 |\n| c | 1 | 1 | 1 | 2 |\n| a | 2 | 1 | 2 | 2 |\n| t | 3 | 2 | 2 | **2** |\n\nSwapping two neighbors costs 2 edits. Like the LCS, the table costs O(n × m) time and memory, while the plain recursion tries up to three edits at every step: exponential.\n","task":"This program fills the edit distance table from `\"an\"` (down) to `\"can\"` (across), and prints it row by row. What does it print? Type each line it prints.\n","kind":"predict","seed":"a, b = \"an\", \"can\"\ndist = [[0] * 4 for _ in range(3)]\nfor i in range(3):\n    dist[i][0] = i\nfor j in range(4):\n    dist[0][j] = j\nfor i in range(1, 3):\n    for j in range(1, 4):\n        if a[i - 1] == b[j - 1]:\n            dist[i][j] = dist[i - 1][j - 1]\n        else:\n            dist[i][j] = 1 + min(dist[i - 1][j - 1],  # replace\n                                 dist[i - 1][j],      # delete\n                                 dist[i][j - 1])      # insert\nfor row in dist:\n    print(row)\n","solution":"a, b = \"an\", \"can\"\ndist = [[0] * 4 for _ in range(3)]\nfor i in range(3):\n    dist[i][0] = i\nfor j in range(4):\n    dist[0][j] = j\nfor i in range(1, 3):\n    for j in range(1, 4):\n        if a[i - 1] == b[j - 1]:\n            dist[i][j] = dist[i - 1][j - 1]\n        else:\n            dist[i][j] = 1 + min(dist[i - 1][j - 1],  # replace\n                                 dist[i - 1][j],      # delete\n                                 dist[i][j - 1])      # insert\nfor row in dist:\n    print(row)\n","hints":["The first row is the edge: 0, 1, 2, 3 inserts. The first column is 0, 1, 2 deletes.","Row 1 is for `\"a\"`. Against `\"c\"` the letters differ, so take 1 plus the smallest of the three neighbors. Against `\"ca\"` the last letters match: copy the entry diagonally up-left.","The last entry is the edit distance from `\"an\"` to `\"can\"`: one insert."],"tests":[{"name":"Output","stdin":"","expect":"[0, 1, 2, 3]\n[1, 1, 1, 2]\n[2, 2, 2, 1]","hidden":false}],"require":[],"forbid":[],"lines":["[0, 1, 2, 3]","[1, 1, 1, 2]","[2, 2, 2, 1]"],"more":[{"task":"Write `edit_distance(a, b)`: the smallest number of one-character inserts, deletes and replaces that turn the string `a` into the string `b`. Use the lesson's table `dist`: set its edges, then work out each other entry from the three next to it. The tests have strings of 600 and 450 characters, where trying every edit never ends.\n\nFor example, `print(edit_distance(\"brain\", \"bread\"), edit_distance(\"cat\", \"act\"))` prints:\n\n```\n3 2\n```\n","kind":"code","seed":"def edit_distance(a, b):\n    pass\n\n\nif __name__ == \"__main__\":\n    print(edit_distance(\"brain\", \"bread\"), edit_distance(\"cat\", \"act\"))\n","solution":"def edit_distance(a, b):\n    dist = [[0] * (len(b) + 1) for _ in range(len(a) + 1)]\n    for i in range(len(a) + 1):\n        dist[i][0] = i\n    for j in range(len(b) + 1):\n        dist[0][j] = j\n    for i in range(1, len(a) + 1):\n        for j in range(1, len(b) + 1):\n            if a[i - 1] == b[j - 1]:\n                dist[i][j] = dist[i - 1][j - 1]\n            else:\n                dist[i][j] = 1 + min(dist[i - 1][j - 1],  # replace\n                                     dist[i - 1][j],      # delete\n                                     dist[i][j - 1])      # insert\n    return dist[len(a)][len(b)]\n\n\nif __name__ == \"__main__\":\n    print(edit_distance(\"brain\", \"bread\"), edit_distance(\"cat\", \"act\"))\n","hints":["Make `dist` with `len(a) + 1` rows of `len(b) + 1` zeros. Then set the edges: `dist[i][0] = i` for every row, and `dist[0][j] = j` for every column.","For `i` and `j` from 1: equal characters `a[i - 1]` and `b[j - 1]` copy `dist[i - 1][j - 1]`. Otherwise it's 1 plus the smallest of `dist[i - 1][j - 1]`, `dist[i - 1][j]` and `dist[i][j - 1]`.","The answer is the bottom-right entry, `dist[len(a)][len(b)]`."],"tests":[{"name":"print(edit_distance(\"brain\", \"bread\"), edit_distance(\"cat\", \"act\"))","stdin":"","call":"print(edit_distance(\"brain\", \"bread\"), edit_distance(\"cat\", \"act\"))\n","expect":"3 2","hidden":false},{"name":"print(edit_distance(\"\", \"tea\"), edit_distance(\"cake\", \"\"), edit_distance(\"\", \"\"))","stdin":"","call":"print(edit_distance(\"\", \"tea\"), edit_distance(\"cake\", \"\"), edit_distance(\"\", \"\"))\n","expect":"3 4 0","hidden":true},{"name":"edit_distance(\"radar\", \"radar\")","stdin":"","call":"edit_distance(\"radar\", \"radar\")\n","expect":"0","hidden":true,"returns":true},{"name":"edit_distance(\"stone\", \"tones\")","stdin":"","call":"edit_distance(\"stone\", \"tones\")\n","expect":"2","hidden":true,"returns":true},{"name":"edit_distance(\"algorithm\", \"altruistic\")","stdin":"","call":"edit_distance(\"algorithm\", \"altruistic\")\n","expect":"6","hidden":true,"returns":true},{"name":"edit_distance(\"garden\", \"gardener\")","stdin":"","call":"edit_distance(\"garden\", \"gardener\")\n","expect":"2","hidden":true,"returns":true},{"name":"edit_distance(\"abcdefghijklmnopqrst\", \"ABCDEFGHIJKLMNOPQRST\")","stdin":"","call":"edit_distance(\"abcdefghijklmnopqrst\", \"ABCDEFGHIJKLMNOPQRST\")\n","expect":"20","hidden":true,"big":true,"returns":true},{"name":"edit_distance(\"\".join(chr(97 + i * 5 % 19) for i in range(600)), \"\".join(chr(97 + i * 8 % 21) for i in range(450)))","stdin":"","call":"edit_distance(\"\".join(chr(97 + i * 5 % 19) for i in range(600)), \"\".join(chr(97 + i * 8 % 21) for i in range(450)))\n","expect":"516","hidden":true,"big":true,"returns":true}],"require":[],"forbid":[{"pattern":"setrecursionlimit","message":"Don't raise the recursion limit: the browser's Python can crash. Fill the table with loops instead."}]},{"task":"Write `lcs(a, b)`, which returns one longest common subsequence of the strings `a` and `b`, built with the lesson's table and its walk back, going up on a tie. The tests have strings of 600 and 450 characters, so fill the table without recursion.\n\nFor example, `lcs(\"DYNAMIC\", \"PROGRAMMING\")` returns:\n\n```\n'AMI'\n```\n","kind":"code","seed":"def lcs(a, b):\n    pass\n\n\nif __name__ == \"__main__\":\n    print(repr(lcs(\"DYNAMIC\", \"PROGRAMMING\")))\n","solution":"def lcs(a, b):\n    length = [[0] * (len(b) + 1) for _ in range(len(a) + 1)]\n    for i in range(1, len(a) + 1):\n        for j in range(1, len(b) + 1):\n            if a[i - 1] == b[j - 1]:\n                length[i][j] = length[i - 1][j - 1] + 1\n            else:\n                length[i][j] = max(length[i - 1][j], length[i][j - 1])\n    found = []\n    i, j = len(a), len(b)\n    while i > 0 and j > 0:\n        if a[i - 1] == b[j - 1]:\n            found.append(a[i - 1])\n            i -= 1\n            j -= 1\n        elif length[i - 1][j] >= length[i][j - 1]:\n            i -= 1\n        else:\n            j -= 1\n    return \"\".join(found[::-1])\n\n\nif __name__ == \"__main__\":\n    print(repr(lcs(\"DYNAMIC\", \"PROGRAMMING\")))\n","hints":["Build `length` with `len(a) + 1` rows of `len(b) + 1` zeros, and fill it as the lesson's program does.","Then start at `i, j = len(a), len(b)` and step back until one of them reaches 0. On equal characters, append `a[i - 1]` to a list and decrease both. Otherwise decrease `i` when `length[i - 1][j] >= length[i][j - 1]`, and `j` when not.","The list holds the characters from the last one back to the first: `\"\".join(found[::-1])` gives the string."],"tests":[{"name":"lcs(\"DYNAMIC\", \"PROGRAMMING\")","stdin":"","call":"lcs(\"DYNAMIC\", \"PROGRAMMING\")\n","expect":"'AMI'","hidden":false,"returns":true},{"name":"lcs(\"PYTHON\", \"TYPHOON\")","stdin":"","call":"lcs(\"PYTHON\", \"TYPHOON\")\n","expect":"'PHON'","hidden":true,"returns":true},{"name":"lcs(\"ORANGE\", \"ARRANGE\")","stdin":"","call":"lcs(\"ORANGE\", \"ARRANGE\")\n","expect":"'RANGE'","hidden":true,"returns":true},{"name":"lcs(\"breakfast\", \"fastbreak\")","stdin":"","call":"lcs(\"breakfast\", \"fastbreak\")\n","expect":"'break'","hidden":true,"returns":true},{"name":"lcs(\"dog\", \"cat\")","stdin":"","call":"lcs(\"dog\", \"cat\")\n","expect":"''","hidden":true,"returns":true},{"name":"lcs(\"\", \"seven\")","stdin":"","call":"lcs(\"\", \"seven\")\n","expect":"''","hidden":true,"returns":true},{"name":"lcs(\"level\", \"level\")","stdin":"","call":"lcs(\"level\", \"level\")\n","expect":"'level'","hidden":true,"returns":true},{"name":"lcs(\"\".join(chr(97 + i * 5 % 19) for i in range(30)), \"\".join(chr(97 + i * 8 % 21) for i in range(30)))","stdin":"","call":"lcs(\"\".join(chr(97 + i * 5 % 19) for i in range(30)), \"\".join(chr(97 + i * 8 % 21) for i in range(30)))\n","expect":"'afqdob'","hidden":true,"big":true,"returns":true},{"name":"Test 9","stdin":"","call":"found = lcs(\"\".join(chr(97 + i * 5 % 19) for i in range(600)), \"\".join(chr(97 + i * 8 % 21) for i in range(450)))\nprint(len(found), found[:20])\n","expect":"116 afkpbglqchmrdiobrepc","hidden":true,"big":true}],"require":[],"forbid":[{"pattern":"setrecursionlimit","message":"Don't raise the recursion limit: the browser's Python can crash. Fill the table with loops instead."}]}]}