当前位置:网站首页>Chapter contents of the romance of the Three Kingdoms
Chapter contents of the romance of the Three Kingdoms
2022-07-29 08:00:00 【Zhao [email protected]】
import requests
from bs4 import BeautifulSoup
from fake_useragent import UserAgent
# Crawl through the titles and contents of all chapters of the romance novels of the Three Kingdoms https://www.shicimingju.com/book/sanguoyanyi.html
if __name__ =='__main__':
headers={
"User-Agent":UserAgent().chrome
}
get_url='https://www.shicimingju.com/book/sanguoyanyi.html'
# Initiate request , Get a response
page_text=requests.get(url=get_url,headers=headers).text.encode('ISO-8859-1')
# Analyze the chapter title and chapter content in the home page
#1. Instantiation BeautifulSoup object , take html Data is loaded into the object
soup=BeautifulSoup(page_text,'lxml')
# print(soup)
#2. Analyze the chapter title and details page url
list_data=soup.select('.book-mulu > ul > li')
fp=open('./sanguo.text','w',encoding='utf-8')
for i in list_data:
title=i.a.text
detail_url='https://www.shicimingju.com/'+ i.a['href']
# Of the details page url Send a request ,
detail_text=requests.get(url=detail_url,headers=headers).text.encode('ISO-8859-1')
detail_soup=BeautifulSoup(detail_text,'lxml')
# Get chapter content
content=detail_soup.find('div',class_='chapter_content').text
# Persistent storage
fp.write(title+":"+content+"\n")
print(title,' Download complete ')
版权声明
本文为[Zhao [email protected]]所创,转载请带上原文链接,感谢
https://yzsam.com/2022/210/202207290520357516.html
边栏推荐
- 准备esp32环境
- Solve the problem that CSDN cannot publish blog due to unknown copyright
- Phased learning about the entry-level application of SQL Server statements - necessary for job hunting (I)
- Resize2fs: bad magic number in super block
- 功能自动化测试实施的原则以及方法有哪些?
- Autojs微信研究:微信自动发送信息机器人最终成品(有效果演示)
- Ue4/ue5 C disk enlargement processing
- 20 hacker artifacts
- @Use of jsonserialize annotation
- Strongly connected component
猜你喜欢

Joseph Ring problem

V-Ray 5 acescg workflow settings
![[cryoelectron microscope | paper reading] emclarity: software for high-resolution cryoelectron tomography and sub fault averaging](/img/1e/9f05862288261e16a6b04508b9b292.png)
[cryoelectron microscope | paper reading] emclarity: software for high-resolution cryoelectron tomography and sub fault averaging

Compare three clock circuit schemes of single chip microcomputer

207. Curriculum

C language data type
![[deep learning] data preparation -pytorch custom image segmentation data set loading](/img/7d/61be445febc140027b5d9d16db8d2e.png)
[deep learning] data preparation -pytorch custom image segmentation data set loading

The new colleague wrote a few pieces of code, broke the system, and was blasted by the boss!
![[cryoEM] Introduction to FSC, Fourier shell correlation](/img/01/6a4f6041444abfd1d3ca7e0c14b524.png)
[cryoEM] Introduction to FSC, Fourier shell correlation

IonIcons图标大全
随机推荐
2022 Shenzhen Cup Title A: get rid of "scream effect" and "echo room effect" and get out of the "information cocoon room"
Unity beginner 3 - enemy movement control and setting of blood loss area (2D)
For the application challenge of smart city, shengteng AI gives a new solution
Data unit: bit, byte, word, word length
在一个sql文件中,上面定义一个测试表及数据,下面可以select* from 测试表
My entrepreneurial neighbors
[note] the art of research - (tell a good story and argument)
Phased learning about the entry-level application of SQL Server statements - necessary for job hunting (I)
Realize the effect of changing some colors of a paragraph of text
Ansible (automation software)
Explanation and closing method of server 135, 137, 138, 139, 445 and other ports
Technology sharing | quick intercom integrated dispatching system
Access数据库引入datagridview数据后,显示错误
[cryoelectron microscope] relation4.0 - subtomogram tutorial
Implementation of simple cubecap+fresnel shader in unity
CDM - code division multiplexing (easy to understand)
Some thoughts on growing into an architect
Compare three clock circuit schemes of single chip microcomputer
The database uses PSQL and JDBC to connect remotely and disconnect automatically from time to time
Basic introduction to pod