Crawler small operation

take reponse The content is displayed in Chinese , And then str To dict

res=response.content.decode("unicode_escape")

res_dict=json.loads(res)

obtain token( Look yes localStorage still SessionStorage), need sleep Wait for a while token Pass it on to get

token = self.driver.execute_script('return localStorage.getItem("token");')
print('\ntoken by ：%s\n'%token)

obtain cookie

# obtain cookie
cookie_tag = self.driver.get_cookies()
# print(cookie_tag)
# Handle cookie In the format you need 
cookie = cookie_tag[0]['name'] + '=' + cookie_tag[0]['value']
print('\ncookie by ：%s\n'%cookie)

Read 、 Write file garbled

### Read 
f=open('{}'.format(file_catalogue+'/'+file_list[0]),mode='r',encoding='utf-8',errors='ignore')
contentJson=json.load(f)
print(contentJson)


### write in 
with open('student.json',mode='w',encoding='utf8') as f:
	json.dump(jsonA,f,ensure_ascii=False,index=2)  #ensure_ascii=False Don't let Chinese characters turn into ASCII
	#f.write(jsonA)  # It is not allowed to write... In the form of ordinary file    Because the suffix is json

Make the dictionary easier to view

print(json.dumps(dict_data,indent=1,ensure_ascii=False))

Turn the data you climb down into python type

json.loads(str(response.text),strict=False)