当前位置:网站首页>What should I do if selenium is reversed?
What should I do if selenium is reversed?
2022-07-31 13:08:00 【BRYTLEVSON】
In the recent business needs, it was found that Taobao, Meituan, and Jingdong major websites have carried out anti-crawling on selenium;
It is speculated that the server side should judge some special fields and methods to identify crawlers. After all, seleniumThe environment and configuration of the chromedriver below and the real chrome are different.
In order to complete the project requirements, I tried some simple methods;
Method one:
The cookie is carried when the request is sent, and the Session state of requests is used to maintain it. Facts have proved that this method is simple and effective, and can be accessed successfully.But this is when the amount of data to be collected is relatively small.If you want fast capture, use aiohttp!
Method 2:
If js can be broken, break it!
Actual cases:
Method three:
mitmproxy request interception
- Set proxy ip for this machine
- Start mitmproxy
- Open chromewebdriver, check each JS file, whether there is driver word, find the final JS file
- Interference script
- Exit mitmproxy
Method four:
A type of headless browser using pyppeteer.Practical case listpyppeteer practical case
边栏推荐
- Istio微服务治理网格的全方面可视化监控(微服务架构展示、资源监控、流量监控、链路监控)
- C#中+=的用法
- 【OpenCV】-边缘检测汇总示例
- Anaconda安装labelImg图像标注软件
- Indoor real-time laser SLAM control method based on biological excitation neural network
- SAP message TK 248 solved
- PHP序列化:eval
- NPM 使用介绍
- 操作符详解
- Error EPERM operation not permitted, mkdir ‘Dsoftwarenodejsnode_cache_cacach两种解决办法
猜你喜欢

The 2nd activity of the TOGAF10 Standard Reading Club continues wonderfully, and the highlights will be reviewed!

NameNode (NN) 和SecondaryNameNode (2NN)工作机制

ERROR 1064 (42000) You have an error in your SQL syntax; check the manual that corresponds to your

IDEA找不到Database解决方法

中望3D 2023正式发布,设计仿真制造一体化缩短产品开发周期

架构实战营|模块8

深圳某游戏研发公司每个工位都装监控,网友:堪比“坐牢”!

2022年最新重庆建筑安全员模拟题库及答案

电商rpa是什么意思?跟电商rpi是一个意思吗?

Error EPERM operation not permitted, mkdir ‘Dsoftwarenodejsnode_cache_cacach两种解决办法
随机推荐
如何使用StarUML画类图[通俗易懂]
Banyan Tree Loan GPU Hardware Architecture
FastAPI encapsulates a generic response
函数递归1.0
NameNode故障处理的两种方法
IDEA连接MySQL数据库并执行SQL查询操作
纷享销客罗旭对话元气森林黄晓枫:零售数字化的终点不是创新,而是数据
架构实战营|模块8
C#控件CheckBox的使用
ERROR 1064 (42000) You have an error in your SQL syntax; check the manual that corresponds to your
【OpenCV】-边缘检测汇总示例
C#中+=的用法
ERROR 1819 (HY000) Your password does not satisfy the current policy requirements
战略进攻能力的重要性,要远远高于战略防守能力
PyQt5 rapid development and actual combat 10.2 compound interest calculation && 10.3 refresh blog clicks
NameNode (NN) 和SecondaryNameNode (2NN)工作机制
[Shader] Shader official example [easy to understand]
关于我放弃考研这件事儿
sqlalchemy 判断一个array 类型的字段是否和一个array有至少一个一致的数据
alert(1) (haozi.me)靶场练习