当前位置:网站首页>What should I do if selenium is reversed?
What should I do if selenium is reversed?
2022-07-31 13:08:00 【BRYTLEVSON】
In the recent business needs, it was found that Taobao, Meituan, and Jingdong major websites have carried out anti-crawling on selenium;
It is speculated that the server side should judge some special fields and methods to identify crawlers. After all, seleniumThe environment and configuration of the chromedriver below and the real chrome are different.
In order to complete the project requirements, I tried some simple methods;
Method one:
The cookie is carried when the request is sent, and the Session state of requests is used to maintain it. Facts have proved that this method is simple and effective, and can be accessed successfully.But this is when the amount of data to be collected is relatively small.If you want fast capture, use aiohttp!
Method 2:
If js can be broken, break it!
Actual cases:
Method three:
mitmproxy request interception
- Set proxy ip for this machine
- Start mitmproxy
- Open chromewebdriver, check each JS file, whether there is driver word, find the final JS file
- Interference script
- Exit mitmproxy
Method four:
A type of headless browser using pyppeteer.Practical case listpyppeteer practical case
边栏推荐
- 【牛客刷题-SQL大厂面试真题】NO3.电商场景(某东商城)
- 查看Mysql数据库版本
- The function of SQL GROUP BY dependence
- Exploring Plain Vision Transformer Backbones for Object Detection Paper Reading Notes
- Using SQL Server FOR XML and FOR JSON syntax on other RDBMSs with jOOQ
- Flutter键盘可见性
- 尚硅谷–MySQL–基础篇(P1~P95)
- SAP message TK 248 solved
- SAP 电商云 Spartacus UI 和 Accelerator UI 里的 ASM 模块
- 基于模糊预测与扩展卡尔曼滤波的野值剔除方法
猜你喜欢

C#获得网卡信息 NetworkInterface IPInterfaceProperties

C#使用ComboBox控件

攻防演练丨赛宁红方管控平台走进广东三地 助力数字政府网络安全建设

alert(1) (haozi.me)靶场练习

365-day challenge LeetCode1000 questions - Day 044 Maximum element in the layer and level traversal

IDEA找不到Database解决方法
尚硅谷–MySQL–基础篇(P1~P95)

ECCV2022:在Transformer上进行递归,不增参数,计算量还少!

【OpenCV】-边缘检测汇总示例

五种数据提交方式的优化
随机推荐
centos7安装mysql5.7步骤(图解版)
关于MySQL主从复制的数据同步延迟问题
使用openssl命令生成证书和对应的私钥,私钥签名,公钥验签
IDEA连接MySQL数据库并执行SQL查询操作
Fully Dynamically Constrained Robot Efficient Time-Optimal Trajectory Planning
滑窗法切分数据
SAP e-commerce cloud Spartacus SSR Optimization Engine execution sequence of several timeouts
WPF中TabControl动态获取当前选中的TabItem
PyQt5快速开发与实战 10.1 获取城市天气预报
Talk about the message display mechanism on the SAP product UI
sqlalchemy 判断一个array 类型的字段是否和一个array有至少一个一致的数据
PyQt5 rapid development and actual combat 10.1 Get city weather forecast
go中select语句
ASM module in SAP Ecommerce Cloud Spartacus UI and Accelerator UI
How does the SAP ABAP OData service support the $filter (filter) operation trial version
手撕Verilog PWM呼吸灯
Flutter键盘可见性
C#中+=的用法
C#控件StatusStrip使用
基于去噪自编码器的故障隔离与识别方法