当前位置:网站首页>Chapter contents of the romance of the Three Kingdoms
Chapter contents of the romance of the Three Kingdoms
2022-07-29 08:00:00 【Zhao [email protected]】
import requests
from bs4 import BeautifulSoup
from fake_useragent import UserAgent
# Crawl through the titles and contents of all chapters of the romance novels of the Three Kingdoms https://www.shicimingju.com/book/sanguoyanyi.html
if __name__ =='__main__':
headers={
"User-Agent":UserAgent().chrome
}
get_url='https://www.shicimingju.com/book/sanguoyanyi.html'
# Initiate request , Get a response
page_text=requests.get(url=get_url,headers=headers).text.encode('ISO-8859-1')
# Analyze the chapter title and chapter content in the home page
#1. Instantiation BeautifulSoup object , take html Data is loaded into the object
soup=BeautifulSoup(page_text,'lxml')
# print(soup)
#2. Analyze the chapter title and details page url
list_data=soup.select('.book-mulu > ul > li')
fp=open('./sanguo.text','w',encoding='utf-8')
for i in list_data:
title=i.a.text
detail_url='https://www.shicimingju.com/'+ i.a['href']
# Of the details page url Send a request ,
detail_text=requests.get(url=detail_url,headers=headers).text.encode('ISO-8859-1')
detail_soup=BeautifulSoup(detail_text,'lxml')
# Get chapter content
content=detail_soup.find('div',class_='chapter_content').text
# Persistent storage
fp.write(title+":"+content+"\n")
print(title,' Download complete ')
版权声明
本文为[Zhao [email protected]]所创,转载请带上原文链接,感谢
https://yzsam.com/2022/210/202207290520357516.html
边栏推荐
- NLP introduction + practice: Chapter 5: using the API in pytorch to realize linear regression
- [cryptography experiment] 0x00 install NTL Library
- Access数据库引入datagridview数据后,显示错误
- Compare three clock circuit schemes of single chip microcomputer
- File system I
- 输出1234无重复的三位数
- Shell script - global variables, local variables, environment variables
- What is the use of chat robots? What type? After reading these, you will understand!
- Cfdiv1+2-bash and a high math puzzle- (gcd+ summary of segment tree single point interval maintenance)
- Ionicons icon Encyclopedia
猜你喜欢

UE4 highlight official reference value

Convert source package to RPM package
![[cryoelectron microscope] relation4.0 - subtomogram tutorial](/img/5b/5364fbe68c495b67d9db5ed9bec2ac.png)
[cryoelectron microscope] relation4.0 - subtomogram tutorial

Compare three clock circuit schemes of single chip microcomputer

CDM - code division multiplexing (easy to understand)
![[WPF] realize language switching through dynamic / static resources](/img/23/1e089ce4a07128323824b25897a8c4.png)
[WPF] realize language switching through dynamic / static resources

Do you want to meet all the needs of customers

10 practical uses of NFT

Rotation in model space and rotation in world space
![[freeze electron microscope] analysis of the source code of the subtomogram alignment function of relion4.0 (for self use)](/img/fe/0efdd151f9661d5cd06a79b7266754.png)
[freeze electron microscope] analysis of the source code of the subtomogram alignment function of relion4.0 (for self use)
随机推荐
State machine DP (simple version)
C language problems
Dynamic Thresholds Buffer Management in a Shared Buffer Packet Switch论文总结
330. Complete the array as required
Very practical shell and shellcheck
Analyze the roadmap of 25 major DFI protocols and predict the seven major trends in the future of DFI
QT connects two qslite databases and reports an error qsqlquery:: exec: database not open
Day 014 二维数组练习
Shell script - global variables, local variables, environment variables
[introduction to cryoelectron microscopy] Caltech open class course notes part 3:image formation
Solve the problem that the disk is full due to large files
postman接口测试|js脚本之阻塞休眠和非阻塞休眠
The smallest positive number that a subset of an array cannot accumulate
Keyboard processing in jetpack compose
What is the use of chat robots? What type? After reading these, you will understand!
Data unit: bit, byte, word, word length
Unity - default rendering pipeline - sculpt shader
Up sampling deconvolution operation
LANDSCAPE
功能自动化测试实施的原则以及方法有哪些?