{"id":"reading-files-read-once","slug":"read-once","title":"Read the file once, use the data many times","text":"A file handle reads forward. A `for` loop over it moves through the file line by line, and when the loop ends, the handle is at the end of the file. A second loop over the same handle finds no lines left, so its body doesn't run at all. Python doesn't stop with an error: the program just gives a wrong answer, which makes this mistake easy to miss.\n\n```file laps.txt\n95\n88\n102\n91\n```\n\n```python\nwith open(\"laps.txt\") as file:\n    laps = 0\n    for line in file:\n        laps += 1\n    total = 0\n    for line in file:\n        total += int(line)\nprint(\"Laps:\", laps)\nprint(\"Total time:\", total)\n```\n\n```output\nLaps: 4\nTotal time: 0\n```\n\nThere are two ways to fix it. You can open the file again for the second loop, with a second `with open(...)`. Usually it's better to read the file once and store its contents in a list or a dictionary. A list can be looped over as many times as you like, and the rest of the program never touches the file again. With the same file:\n\n```file laps.txt\n95\n88\n102\n91\n```\n\n```python\nlaps = []\nwith open(\"laps.txt\") as file:\n    for line in file:\n        laps.append(int(line))\n\nprint(\"Laps:\", len(laps))\nprint(\"Total time:\", sum(laps))\nprint(\"Fastest:\", min(laps))\nfor lap in laps:\n    if lap > 100:\n        print(\"Slow lap:\", lap)\n```\n\n```output\nLaps: 4\nTotal time: 376\nFastest: 88\nSlow lap: 102\n```\n\n### Reading into a dictionary\n\nWhen each line is about something with a name, a dictionary is a good home for the data. In this file, each line is one customer's shopping basket: the customer's name, then everything in the basket. A function reads the file into a dictionary whose keys are the customers and whose values are lists of their items, and returns it. After that, the program asks the dictionary as many questions as it likes:\n\n```file baskets.csv\nMia;bread;milk;apples\nNoah;coffee;bananas\nLena;apples;bread;cheese;milk\n```\n\n```python\ndef read_baskets(filename):\n    baskets = {}\n    with open(filename) as file:\n        for line in file:\n            parts = line.strip().split(\";\")\n            baskets[parts[0]] = parts[1:]\n    return baskets\n\n\nbaskets = read_baskets(\"baskets.csv\")\nprint(baskets[\"Noah\"])\nfor customer, items in baskets.items():\n    print(f\"{customer}: {len(items)} items\")\nfor customer, items in baskets.items():\n    if \"bread\" in items:\n        print(customer, \"bought bread\")\n```\n\n```output\n['coffee', 'bananas']\nMia: 3 items\nNoah: 2 items\nLena: 4 items\nMia bought bread\nLena bought bread\n```\n\nThe slice `parts[1:]` is already a list: every field after the name. Here the fields are words, so the list can go into the dictionary as it is. When they are numbers, build the list with a loop that converts each field with `int`, as the rounds example in the previous step did, and store that list under the name. Then the numbers are converted once, while reading, and every later calculation can use them.\n\nTwo loops over the dictionary are no problem, unlike two loops over the file. And the rest of the program never sees a semicolon or a line break: only `read_baskets` knows what the file looks like.\n","task":"The program should print the number of days in the file `temperatures.txt` and their average temperature, but the average comes out wrong. Fix it, so that with this file\n\n```text\n12\n15\n17\n13\n14\n```\n\nit prints:\n\n```\nDays: 5\nAverage: 14.2\n```\n","kind":"code","seed":"with open(\"temperatures.txt\") as file:\n    count = 0\n    for line in file:\n        count += 1\n    total = 0\n    for line in file:\n        total += int(line)\nprint(\"Days:\", count)\nprint(\"Average:\", total / count)\n","solution":"temperatures = []\nwith open(\"temperatures.txt\") as file:\n    for line in file:\n        temperatures.append(int(line))\nprint(\"Days:\", len(temperatures))\nprint(\"Average:\", sum(temperatures) / len(temperatures))\n","hints":["The first loop reads the file to its end. What's left for the second loop?","Read the file once into a list of numbers with `append`, and do both calculations with the list after the `with` block.","`len`, `sum` and the list are all you need: the average is `sum(temperatures) / len(temperatures)`."],"tests":[{"name":"Test 1","stdin":"","files":{"temperatures.txt":"12\n15\n17\n13\n14\n"},"expect":"Days: 5\nAverage: 14.2","hidden":false},{"name":"Test 2","stdin":"","files":{"temperatures.txt":"-3\n1\n5\n1\n"},"expect":"Days: 4\nAverage: 1.0","hidden":true}],"require":[],"forbid":[],"more":[{"task":"Each line of the file `ratings.csv` has the title of a movie and the ratings people gave it, from 1 to 5. Write a function `read_ratings(filename)` that returns a dictionary whose keys are the titles and whose values are lists of the ratings as integers.\n\nWith this file\n\n```text\nMoon Rocks;4;5;3\nThe Long Road;2;3\nCats in Space;5;5;4;5\n```\n\n`print(read_ratings(\"ratings.csv\"))` prints:\n\n```\n{'Moon Rocks': [4, 5, 3], 'The Long Road': [2, 3], 'Cats in Space': [5, 5, 4, 5]}\n```\n","kind":"code","seed":"def read_ratings(filename):\n    pass\n\n\nif __name__ == \"__main__\":\n    print(read_ratings(\"ratings.csv\"))\n","solution":"def read_ratings(filename):\n    ratings = {}\n    with open(filename) as file:\n        for line in file:\n            parts = line.strip().split(\";\")\n            ratings[parts[0]] = []\n            for rating in parts[1:]:\n                ratings[parts[0]].append(int(rating))\n    return ratings\n\n\nif __name__ == \"__main__\":\n    print(read_ratings(\"ratings.csv\"))\n","hints":["Strip and split each line. The title is `parts[0]`, and the ratings are the slice `parts[1:]`.","Store an empty list under the title, then go through the ratings and `append` each one, converted with `int`.","Return the dictionary after the file has been read."],"tests":[{"name":"print(read_ratings(\"ratings.csv\"))","stdin":"","call":"print(read_ratings(\"ratings.csv\"))\n","files":{"ratings.csv":"Moon Rocks;4;5;3\nThe Long Road;2;3\nCats in Space;5;5;4;5\n"},"expect":"{'Moon Rocks': [4, 5, 3], 'The Long Road': [2, 3], 'Cats in Space': [5, 5, 4, 5]}","hidden":false},{"name":"read_ratings(\"books.csv\")","stdin":"","call":"read_ratings(\"books.csv\")\n","files":{"books.csv":"Deep Water;1\nSmall Gods;5;4\n"},"expect":"{'Deep Water': [1], 'Small Gods': [5, 4]}","hidden":true,"returns":true}],"require":[],"forbid":[]},{"task":"The starter code has a ready `read_ratings` function from the previous task. Write the function `best_rated(filename)`, which uses it and returns the title with the highest average rating. You can assume no two titles have the same average.\n\nWith this `ratings.csv`\n\n```text\nMoon Rocks;4;5;3\nThe Long Road;2;3\nCats in Space;5;5;4;5\n```\n\n`print(best_rated(\"ratings.csv\"))` prints:\n\n```\nCats in Space\n```\n","kind":"code","seed":"def read_ratings(filename):\n    ratings = {}\n    with open(filename) as file:\n        for line in file:\n            parts = line.strip().split(\";\")\n            ratings[parts[0]] = []\n            for rating in parts[1:]:\n                ratings[parts[0]].append(int(rating))\n    return ratings\n\n\ndef best_rated(filename):\n    pass\n\n\nif __name__ == \"__main__\":\n    print(best_rated(\"ratings.csv\"))\n","solution":"def read_ratings(filename):\n    ratings = {}\n    with open(filename) as file:\n        for line in file:\n            parts = line.strip().split(\";\")\n            ratings[parts[0]] = []\n            for rating in parts[1:]:\n                ratings[parts[0]].append(int(rating))\n    return ratings\n\n\ndef best_rated(filename):\n    ratings = read_ratings(filename)\n    best_title = \"\"\n    best_average = 0\n    for title, scores in ratings.items():\n        average = sum(scores) / len(scores)\n        if average > best_average:\n            best_title = title\n            best_average = average\n    return best_title\n\n\nif __name__ == \"__main__\":\n    print(best_rated(\"ratings.csv\"))\n","hints":["Start with `ratings = read_ratings(filename)`: from then on you work with a dictionary, not the file.","Go through `ratings.items()` and work out each title's average with `sum` and `len`.","Keep the best title and its average in two variables, and replace them whenever you find a higher average."],"tests":[{"name":"print(best_rated(\"ratings.csv\"))","stdin":"","call":"print(best_rated(\"ratings.csv\"))\n","files":{"ratings.csv":"Moon Rocks;4;5;3\nThe Long Road;2;3\nCats in Space;5;5;4;5\n"},"expect":"Cats in Space","hidden":false},{"name":"best_rated(\"books.csv\")","stdin":"","call":"best_rated(\"books.csv\")\n","files":{"books.csv":"Deep Water;3;4\nSmall Gods;5;4;5\nNight Train;2\n"},"expect":"'Small Gods'","hidden":true,"returns":true},{"name":"best_rated(\"games.csv\")","stdin":"","call":"best_rated(\"games.csv\")\n","files":{"games.csv":"Tetris;5;5\nSnake;4\n"},"expect":"'Tetris'","hidden":true,"returns":true},{"name":"best_rated(\"shows.csv\")","stdin":"","call":"best_rated(\"shows.csv\")\n","files":{"shows.csv":"Long Saga;3;5;3;3;3\nQuiz Night;4;4;3\nShort Gem;5\n"},"expect":"'Short Gem'","hidden":true,"returns":true}],"require":[],"forbid":[]}]}