{"id":"regex-search","slug":"search-and-findall","title":"What regular expressions are","text":"String methods find exact text: `\"cat\" in text` or `text.find(\"cat\")`. But often you're looking for a *kind* of text: any number, a time of day, a word with a capital letter. A **regular expression** (a **regex** for short) describes such text with a **pattern**. Regular expressions are a small language of their own, and the same patterns work, with small differences, in most programming languages and in many text editors.\n\nPython's regular expressions are in the module **`re`**. The function `re.search(pattern, text)` looks for the first place in the text that matches the pattern:\n\n```python\nimport re\n\nnotes = [\"Meeting at 10\", \"Lunch with Pia\", \"Dentist at 3 pm\"]\nfor note in notes:\n    if re.search(r\"\\d\", note):\n        print(\"Has a time:\", note)\n```\n\n```output\nHas a time: Meeting at 10\nHas a time: Dentist at 3 pm\n```\n\nIn a pattern, most characters simply match themselves: the pattern `r\"Pia\"` matches the text Pia, with the same capitals. A few have a special meaning. `\\d` matches any one **digit**, 0 to 9.\n\n`re.search` returns a **match object** when it finds a match, and `None` when it doesn't. In an `if`, a match counts as true and `None` as false. The match object's method `group()` gives the text that matched:\n\n```pycon\n>>> import re\n>>> re.search(r\"\\d\", \"Room 42\")\n<re.Match object; span=(5, 6), match='4'>\n>>> re.search(r\"\\d\\d\", \"Room 42\").group()\n'42'\n>>> print(re.search(r\"\\d\", \"Room B\"))\nNone\n>>> re.search(r\"\\d\", \"Room 42\") is not None\nTrue\n```\n\n`span=(5, 6)` says where the match is: from index 5 up to index 6. When a function should return `True` or `False`, compare the result with `None`, as the last line does.\n\n### Raw strings\n\nThe patterns above are written `r\"...\"`. The `r` makes a **raw string**, in which Python leaves every backslash as it is. In an ordinary string, Python reads some backslash pairs itself: `\"\\n\"` is a line break and `\"\\t\"` a tab. Patterns are full of backslashes, so write them as raw strings, and the `re` module gets exactly what you typed:\n\n```pycon\n>>> len(\"\\n\")\n1\n>>> len(r\"\\n\")\n2\n>>> print(r\"\\d and \\n\")\n\\d and \\n\n```\n\nWithout the `r`, Python even warns about a pattern such as `\"\\d\"`: `SyntaxWarning: \"\\d\" is an invalid escape sequence`.\n\n### Finding every match\n\n`re.findall(pattern, text)` returns a list of all the matches, as strings. A `+` after a part of a pattern means \"one or more of it\" (step 3 shows more marks like it), so `\\d+` matches a whole run of digits: a number.\n\n```python\nimport re\n\ntext = \"3 apples, 12 pears and 150 grapes\"\nprint(re.findall(r\"\\d\", text))\nprint(re.findall(r\"\\d+\", text))\nnumbers = [int(number) for number in re.findall(r\"\\d+\", text)]\nprint(sum(numbers))\n```\n\n```output\n['3', '1', '2', '1', '5', '0']\n['3', '12', '150']\n165\n```\n\nThe matches are strings, so turn them into numbers with `int` before you calculate with them.\n","task":"A password needs at least one digit. Fill in the two blanks: the name of the `re` function that looks for a match, and the pattern for one digit. The program should print:\n\n```\nsunshine needs a digit\nsunsh1ne ok\n2fast ok\n```\n","kind":"fill","seed":"import re\n\npasswords = [\"sunshine\", \"sunsh1ne\", \"2fast\"]\nfor password in passwords:\n    if re.[[search]](r\"[[\\d]]\", password):\n        print(password, \"ok\")\n    else:\n        print(password, \"needs a digit\")\n","solution":"import re\n\npasswords = [\"sunshine\", \"sunsh1ne\", \"2fast\"]\nfor password in passwords:\n    if re.search(r\"\\d\", password):\n        print(password, \"ok\")\n    else:\n        print(password, \"needs a digit\")\n","hints":["The function that looks for the first match anywhere in the text returns a match or `None`.","The function is `re.search`, so the first blank is `search`. A digit is `\\d` in a pattern."],"tests":[{"name":"Output","stdin":"","expect":"sunshine needs a digit\nsunsh1ne ok\n2fast ok","hidden":false}],"require":[],"forbid":[],"more":[{"task":"What does this program print? Type each line it prints.\n","kind":"predict","seed":"import re\n\nprint(re.findall(r\"\\d\", \"R2-D2\"))\nprint(re.findall(r\"\\d+\", \"Call 555 0199 now\"))\nprint(re.search(r\"cat\", \"Concatenate\") is not None)\nprint(re.search(r\"Cat\", \"concatenate\"))\n","solution":"import re\n\nprint(re.findall(r\"\\d\", \"R2-D2\"))\nprint(re.findall(r\"\\d+\", \"Call 555 0199 now\"))\nprint(re.search(r\"cat\", \"Concatenate\") is not None)\nprint(re.search(r\"Cat\", \"concatenate\"))\n","hints":["`\\d` matches one digit at a time, and `\\d+` a whole run of digits.","The third line looks for `cat` anywhere in the text, and it's there, inside the word.","Patterns are case sensitive: `Cat` with a capital C isn't in `concatenate`, so `re.search` returns `None`."],"tests":[{"name":"Output","stdin":"","expect":"['2', '2']\n['555', '0199']\nTrue\nNone","hidden":false}],"require":[],"forbid":[],"lines":["['2', '2']","['555', '0199']","True","None"]},{"task":"Write a function `numbers_in(text)` that returns a list of the whole numbers in the text, as integers, in the order they appear. Use the `re` module. For example,\n\n```python\nprint(numbers_in(\"Gate 7 opens at 1930, row 12\"))\n```\n\nshould print:\n\n```\n[7, 1930, 12]\n```\n","kind":"code","seed":"import re\n\n\ndef numbers_in(text):\n    pass\n\n\nif __name__ == \"__main__\":\n    print(numbers_in(\"Gate 7 opens at 1930, row 12\"))\n","solution":"import re\n\n\ndef numbers_in(text):\n    return [int(number) for number in re.findall(r\"\\d+\", text)]\n\n\nif __name__ == \"__main__\":\n    print(numbers_in(\"Gate 7 opens at 1930, row 12\"))\n","hints":["`re.findall(r\"\\d+\", text)` gives every number in the text, as strings.","Turn each string into an integer with `int`. A list comprehension does it in one line."],"tests":[{"name":"print(numbers_in(\"Gate 7 opens at 1930, row 12\"))","stdin":"","call":"print(numbers_in(\"Gate 7 opens at 1930, row 12\"))\n","expect":"[7, 1930, 12]","hidden":false},{"name":"numbers_in(\"a1b22c333\")","stdin":"","call":"numbers_in(\"a1b22c333\")\n","expect":"[1, 22, 333]","hidden":true,"returns":true},{"name":"numbers_in(\"no numbers here\")","stdin":"","call":"numbers_in(\"no numbers here\")\n","expect":"[]","hidden":true,"returns":true}],"require":[{"pattern":"\\bre\\.(findall|search)\\s*\\(","message":"Find the numbers with the `re` module."}],"forbid":[]}]}