当前位置:网站首页>Get all stock data of big a
Get all stock data of big a
2022-07-05 12:12:00 【ziyi813】
Learn to practice reptile instances , Get stock data
Dongcai Weiwu , No crawler restrictions
https://www.eastmoney.com/robots.txt
obtain 4 Thousands of stock data are saved as csv
import requests
import re
import json
import time
def getApiRequest(url):
try:
headerParams = {
'user-agent' : 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36' }
r= requests.get(url, timeout=30, headers = headerParams)
r.raise_for_status()
return r
except:
return 'http request error !'
def rinseData(res):
try:
jsdata = re.findall(r'\(.*?\)', res.text)
dicts = json.loads(jsdata[0][1:len(jsdata[0])-1 ])
diff = dicts.get('data', []).get('diff', [])
return diff
except:
return []
def writeCsv(f, lslist):
try :
for item in lslist:
tmp = [item.get('f12'),item.get('f14'),item.get('f15'),item.get('f3'),item['f4'],item['f5'],item['f6'],item['f7'],item['f2'],item['f16'],item['f17'],item['f18'],item['f10'],item['f8'],item['f9'],item['f23'] ]
datastr = ''
for s in tmp:
datastr += str(s) + ','
f.write(datastr[:len(datastr)-1] + "\n")
except:
return 'write csv file Error'
def main():
total = 236
page = 20
start = time.perf_counter() # Starting time
f = open('gp.csv', 'w+', encoding='utf-8')
f.write(' Code , name , The latest price , applies , Up and down , volume ( hand ), turnover , The amplitude , The highest , The minimum , today , Yesterday , Volume ratio , Turnover rate , P / E ratio ( dynamic ), Market to net ratio \n')
for p in range(total):
a = "*" * p
b = "." * (total -p)
c = (p/total)*100
dur = time.perf_counter() - start
print("\r{:^3.0f}%[{}->{}]{:.2f}s".format(c,a,b,dur),end="")
url = 'http://81.push2.eastmoney.com/api/qt/clist/get?cb=jQuery1124017220261478010612_1636785244249&pn='+ str(p) +'&pz='+str(page)+'&po=1&np=1&ut=bd1d9ddb04089700cf9c27f6f7426281&fltt=2&invt=2&fid=f3&fs=m:0+t:6,m:0+t:80,m:1+t:2,m:1+t:23&fields=f1,f2,f3,f4,f5,f6,f7,f8,f9,f10,f12,f13,f14,f15,f16,f17,f18,f20,f21,f23,f24,f25,f22,f11,f62,f128,f136,f115,f152&_=1636785244294'
res = getApiRequest(url)
lslist = rinseData(res)
writeCsv(f, lslist)
f.close()
print("\n"+" end of execution ".center(total//2,"-"))
main()
边栏推荐
- 【load dataset】
- Tabbar configuration at the bottom of wechat applet
- 【云原生 | Kubernetes篇】Ingress案例实战(十三)
- 图像超分实验:SRCNN/FSRCNN
- Check the debug port information in rancher and do idea remote JVM debug
- pytorch-线性回归
- Design of music box based on assembly language
- Principle of persistence mechanism of redis
- 【 YOLOv3中Loss部分计算】
- Redis master-slave mode
猜你喜欢
How to make your products as expensive as possible
【TFLite, ONNX, CoreML, TensorRT Export】
Yolov5 target detection neural network -- calculation principle of loss function
Check the debug port information in rancher and do idea remote JVM debug
[untitled]
1 plug-in to handle advertisements in web pages
Reading notes of growth hacker
Simply solve the problem that the node in the redis cluster cannot read data (error) moved
【PyTorch预训练模型修改、增删特定层】
Mongodb replica set
随机推荐
Matlab imoverlay function (burn binary mask into two-dimensional image)
Semantic segmentation experiment: UNET network /msrc2 dataset
Embedded software architecture design - message interaction
15 methods in "understand series after reading" teach you to play with strings
【SingleShotMultiBoxDetector(SSD,单步多框目标检测)】
2022年国内云管平台厂商哪家好?为什么?
JS for loop number exception
Thoughts and suggestions on the construction of intelligent management and control system platform for safe production in petrochemical enterprises
How to clear floating?
Principle of redis cluster mode
vscode快捷键
Vscode shortcut key
Acid transaction theory
Codeforces Round #804 (Div. 2)
一款新型的智能家居WiFi选择方案——SimpleWiFi在无线智能家居中的应用
跨平台(32bit和64bit)的 printf 格式符 %lld 输出64位的解决方式
Proof of the thinking of Hanoi Tower problem
Tabbar configuration at the bottom of wechat applet
Swift - enables textview to be highly adaptive
ABAP table lookup program