当前位置:网站首页>Three methods of script about login and cookies
Three methods of script about login and cookies
2022-07-06 01:07:00 【Keep a low profile】
Some websites require login , So this article records about login and cookies Three ways
Another article full of dry goods
Method 1
adopt settings.py
Here go to the website cookie Write to configuration file , In this way, you can load cookie 了
Put this 2 Configuration on
COOKIES_ENABLED = False
DEFAULT_REQUEST_HEADERS = {
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'Accept-Language': 'en',
'Cookie': ' Yours cookie Information '
}
This opens cookie 了 , That's all right.
Method 2
You still have to log in first , From the browser cookie Take it out
When sending a request , Get ready cookie, hold cookie Assemble into a dictionary , rewrite start_requests Method , Just pass the dictionary in
cookie This is probably the structure , Of course, it's much more complicated , So I took it out of the browser , Just do the following string parsing
start_requests It's inherited from spiders , Then here is to rewrite this class
Put this cookie close , Then throw it down parse This method
name=zhangsan;sex=female
def start_requests(self):
cookie = {
}
str_cookie = ' Yours cookie Information '
str_cookie_items_list = str_cookie.split(';')
for str_cookie_items in str_cookie_items_list:
str_cookie_kw = str_cookie_items.split('=')
cookie[str_cookie_kw[0].strip()] = str_cookie_kw[1].strip()
yield scrapy.Request(
url=self.start_urls[0],
method='get',
callback=self.parse,
cookies=cookie
)
def parse(self, response, **kwargs):
print(response.text)
Method 3
This is the login interface , And then jump start_url The page of ,scrapy Will keep session
Or rewrite spider This class of start_requests Method
import scrapy
class LoginSpiderSpider(scrapy.Spider):
name = 'login_spider'
allowed_domains = ['17k.com']
start_urls = ['https://user.17k.com/ck/author/shelf?page=1&appKey=2406394919']
def start_requests(self):
url = 'https://passport.17k.com/ck/user/login' # Login interface
username = 'username' # This is the parameter of login interface user name
password = 'password' # This is the parameter of login interface password
yield scrapy.Request(
url=url,
method='post', # Interface is post request
body=f'loginName={
username}&password={
password}',
callback=self.start_parse,
""" stay scrapy Return request Here, if you send post request , Then the requested parameters should be body, Nothing here data=data This thing , Then use body Words It has to be like the above Why login , We have to call back to start_parse, Not directly to parse? Because after logging in , This response Is the user information after login json, We have to start from here request To parse That's all right. """
)
def start_parse(self, response, **kwargs):
# print(response.json()) # Print the request body of successful login
yield scrapy.Request(
url=self.start_urls[0],
method='get',
callback=self.parse,
)
def parse(self, response, **kwargs):
print(response.json())
Method four
selenium To complete the login , take cookie, Add cookie,–> middleware
What if you want to know what happened , Let's hear the decomposition next time
边栏推荐
- Cannot resolve symbol error
- Interview must brush algorithm top101 backtracking article top34
- 可恢复保险丝特性测试
- [groovy] compile time metaprogramming (compile time method injection | method injection using buildfromspec, buildfromstring, buildfromcode)
- Synchronized and reentrantlock
- Questions about database: (5) query the barcode, location and reader number of each book in the inventory table
- cf:D. Insert a Progression【关于数组中的插入 + 绝对值的性质 + 贪心一头一尾最值】
- 云导DNS和知识科普以及课堂笔记
- 程序员成长第九篇:真实项目中的注意事项
- 激动人心,2022开放原子全球开源峰会报名火热开启
猜你喜欢
Mobilenet series (5): use pytorch to build mobilenetv3 and learn and train based on migration
Cve-2017-11882 reappearance
Study diary: February 13, 2022
Five challenges of ads-npu chip architecture design
SSH login is stuck and disconnected
SAP Spartacus home 页面读取 product 数据的请求的 population 逻辑
Installation and use of esxi
How to make your own robot
cf:H. Maximal AND【位运算练习 + k次操作 + 最大And】
[pat (basic level) practice] - [simple mathematics] 1062 simplest fraction
随机推荐
Dedecms plug-in free SEO plug-in summary
95后CV工程师晒出工资单,狠补了这个,真香...
如何制作自己的機器人
FFT learning notes (I think it is detailed)
Fibonacci number
Idea remotely submits spark tasks to the yarn cluster
Recoverable fuse characteristic test
Ubantu check cudnn and CUDA versions
Installation and use of esxi
MCU realizes OTA online upgrade process through UART
Some features of ECMAScript
In the era of industrial Internet, we will achieve enough development by relying on large industrial categories
Gartner发布2022-2023年八大网络安全趋势预测,零信任是起点,法规覆盖更广
Construction plan of Zhuhai food physical and chemical testing laboratory
Starting from 1.5, build a micro Service Framework - call chain tracking traceid
MIT doctoral thesis | robust and reliable intelligent system using neural symbol learning
程序员搞开源,读什么书最合适?
IP storage and query in MySQL
Leetcode study - day 35
devkit入门