CSV file storage

Personal profile
Author's brief introduction ： Hello everyone , I am a W_chuanqi, A programming enthusiast
Personal home page ：W_chaunqi
Stand by me ： give the thumbs-up + Collection ️+ Leaving a message.
May you and I share ：“ If you are in the mire , The heart is also in the mire , Then all eyes are muddy ; If you are in the mire , And I miss Kun Peng , Then you can see 90000 miles of heaven and earth .”

4.3 CSV File store

​ CSV, Its full name is Comma-Separated Values, In Chinese, it is called comma separated value or character separated value , Its files store tabular data in plain text .CSV A file is a sequence of characters , It can consist of any number of records , Each record is separated by some line break . Each record consists of several fields , The separator between fields is another character or string , The most common are commas or tabs . But all records have exactly the same sequence of fields , Equivalent to a structured table in plain text form . It is better than Excel The file is more concise ,XLS The text is a spreadsheet , Include text 、 The number 、 Formula and format etc ,CSV These are not included in , Is plain text with specific characters as separators , Simple and clear structure . therefore , Sometimes use CSV It is convenient to store data . Let's take a look at Python Read data and write data CSV Documentation process .

1. write in

Let's take a look at the simplest example ：

import csv

with open('data.csv', 'w') as csvfile:
    writer = csv.writer(csvfile)
    writer.writerow(['id', 'name', 'age'])
    writer.writerow(['1001', 'Mike', '20'])
    writer.writerow(['1002', 'Bob', '22'])
    writer.writerow(['1003', 'Jordan', '21'])

​ Here, first open data.csv file , Then specify the open mode as w( Write now ), Get file handle , Then call csv Library writer Method to initialize the write object , Pass in the handle , And then call writerow Method to pass in the data of each row , This completes writing .

​ At the end of the run , Will generate a file named data.csv The file of , At this point, the data is successfully written . Open directly as text ,
The following will be displayed ：

You can see , write in CSV By default, the text of the file separates each record with a comma , Every call writerow Method to write a row of data . use Excel open data.csv The results of the file are shown in the figure .

​ If you want to change the separator between columns , You can pass in delimiter Parameters , The code is as follows ：

import csv

with open('data.csv', 'w') as csvfile:
    writer = csv.writer(csvfile, delimiter=' ')
    writer.writerow(['id', 'name', 'age'])
    writer.writerow(['1001', 'Mike', '20'])
    writer.writerow(['1002', 'Bob', '22'])
    writer.writerow(['1003', 'Jordan', '21'])

Be careful ：

There's an exception here , The reason is that this document is in WPS The door is open , Unable to write ...

​ Here, when initializing the writer object , Passed a space delimiter Parameters , At this time, the columns in the output result are separated by spaces , The contents are as follows :

in addition , We can also call writerows Method to write multiple lines at the same time , At this time, the parameter needs to be transferred to the two-dimensional list , for example ：

import csv

with open('data.csv', 'w') as csvfile:
    writer = csv.writer(csvfile)
    writer.writerow(['id', 'name', 'age'])
    writer.writerows(
        [['1001', 'Mike', '20'], ['1002', 'Bob', '22'], ['1003', 'Jordan', '21']])

​ The output is the same , The contents are as follows ：

​ But in general , Crawlers crawl structured data , We usually use dictionaries to represent this kind of data .csv The library also provides a way to write dictionaries , Examples are as follows ：

import csv

with open('data.csv', 'w') as csvfile:
    fieldnames = ['id', 'name', 'age']
    writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
    writer.writeheader()
    writer.writerow({
    'id': '1001', 'name': 'Mike', 'age': '20'})
    writer.writerow({
    'id': '1002', 'name': 'Bob', 'age': '22'})
    writer.writerow({
    'id': '1003', 'name': 'Jordan', 'age': '21'})

​ Here we first define 3 A field , use fieldnames Express , Then pass it on to Dictwriter Method to initialize a dictionary write object , And assign the object to writer Variable . Then call the writeheader Method first writes the header information , Call again writerow The method is passed on to the corresponding dictionary . The final result of writing people is exactly the same as before , The contents are as follows ：

​ In this way, the dictionary is written CSV In file .

​ in addition , If you want to append write , You can change the open mode of the file , Namely the open The second parameter of the function is changed to a, The code is as follows ：

import csv

with open('data.csv', 'a') as csvfile:
    fieldnames = ['id', 'name', 'age']
    writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
    writer.writerow({
    'id': '1004', 'name': 'Durant', 'age': '25'})

​ Execute this code again , The content of the document becomes ：

​ The result shows that , The data is appended to the file .

​ If you want to write Chinese content , We know that we may encounter the problem of character encoding , You need to give open Parameter specifies the encoding format . for example , Here write a line of data containing Chinese , The code is rewritten as follows ：

import csv

with open('data.csv', 'a', encoding='utf-8') as csvfile:
    fieldnames = ['id', 'name', 'age']
    writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
    writer.writerow({
    'id': '1004', 'name': ' Zhang Wei ', 'age': '25'})

If not here open Function to specify the encoding , Coding errors may occur .

in addition , If contact pandas Such as the library , You can call DataFrame Object's to_csv Method to write data to CSV In file .

This method requires installation pandas library , The installation command is :

pip install pandas

After installation , We can use it pandas The library saves data as CSV file , The example code is as follows ：

import pandas as pd

data = [{
    'id': '1001', 'name': 'Mike', 'age': '20'},
        {
    'id': '1002', 'name': 'Bob', 'age': '22'},
        {
    'id': '1003', 'name': 'Jordan', 'age': '21'}
        ]
df = pd.DataFrame(data)
df.to_csv('data.csv', index=False)

​ Here we first define several pieces of data , Every data is a dictionary , Then combine them into a list , The assignment is data. Then we use pandas Of DataFrame Class creates a DataFrame object , Parameters of the incoming data, And assign this object to df. Finally, we call df Of to_csv Method can also save data as CSV file .

2. Read

​ We can also use it csv Library to read CSV file . for example , Read out the contents of the file just written , The relevant code is as follows :

import csv
with open('data.csv', 'r', encoding='utf-8') as csvfile:
    reader = csv.reader(csvfile)
    for row in reader:
        print(row)

​ The operation results are as follows ：

​ Here we construct Reader object , Output the contents of each line in the file through traversal , Each line is a list . Be careful , If CSV The document contains Chinese , You also need to specify the file code .

​ in addition , We can also use pandas Of read_csv Method from CSV Read it from the file , for example ：

import pandas as pd
df = pd.read_csv('data.csv')
print(df)

The operation results are as follows ：

there df It's actually a DataFrame object , If you are familiar with this , You can directly use it to complete the analysis and processing of some data .

​ If you only want to read the data in the file , You can put df Then it is further transformed into a list or tuple , The example code is as follows ：

import pandas as pd
df = pd.read_csv('data.csv')
data = df.values.tolist()
print(data)

Here we call df Of valves attribute , Call again tolist Method , Data can be converted into list form , The operation results are as follows ：

in addition , Direct pair df Traverse line by line , You can also get list type results , The code is as follows :

import pandas as pd
df = pd.read_csv('data.csv')
for index, row in df.iterrows():
    print(row.tolist())

The operation results are as follows ：

You can see , We also get the result of list type .