Combining data from several files

Data that belongs together is often spread over several files. A shop might keep its product names in one file and the amounts in stock in another, because different people update them. What connects the files is a shared key: a value, such as a product code, that appears in both and means the same thing in each.

The usual way to combine such files is to read each one into a dictionary with the key as the dictionary's key, and then look the keys up:

code;name
P1;desk lamp
P2;office chair
P3;bookshelf

code;amount
P2;14
P1;3

names = {}
with open("products.csv") as file:
    for line in file:
        parts = line.strip().split(";")
        if parts[0] == "code":
            continue
        names[parts[0]] = parts[1]

amounts = {}
with open("stock.csv") as file:
    for line in file:
        parts = line.strip().split(";")
        if parts[0] == "code":
            continue
        amounts[parts[0]] = int(parts[1])

for code, name in names.items():
    if code in amounts:
        print(f"{name}: {amounts[code]} in stock")
    else:
        print(f"{name}: none in stock")

desk lamp: 3 in stock
office chair: 14 in stock
bookshelf: none in stock

Notice two things. The lines of the two files are in a different order, and that doesn't matter, because the dictionary finds each code wherever it was in the file. And P3 has no line in stock.csv at all: the in check handles that, where amounts[code] alone would stop the program with a KeyError.

One function, several files

When two files have the same shape, one function can read both. Programs also often ask the user which files to read, instead of having the names in the code:

101;Ana
102;Bo
103;Cy

103;cy@example.com
101;ana@example.com

def read_pairs(filename):
    pairs = {}
    with open(filename) as file:
        for line in file:
            parts = line.strip().split(";")
            pairs[parts[0]] = parts[1]
    return pairs


names = read_pairs(input("Names file: "))
emails = read_pairs(input("Emails file: "))
for number, name in names.items():
    if number in emails:
        print(f"{name}: {emails[number]}")
    else:
        print(f"{name}: no email")

names.csv
emails.csv

Names file: names.csv
Emails file: emails.csv
Ana: ana@example.com
Bo: no email
Cy: cy@example.com

The student numbers stay strings here: they are only compared with each other, never calculated with, so there's no need to convert them.

Many lines with the same key

Sometimes a file has many lines for the same key, such as one line for every sale of a product. Then the dictionary can collect a total for each key: the first time a key comes up, its total starts at 0, and every line adds to it.

totals = {}
with open("sales.csv") as file:
    for line in file:
        parts = line.strip().split(";")
        code = parts[0]
        if code not in totals:
            totals[code] = 0
        totals[code] += int(parts[1])