当前位置:网站首页>Anti crawler mechanism solution: JS code generates random strings locally
Anti crawler mechanism solution: JS code generates random strings locally
2022-07-29 02:06:00 【Gloomy】
This is an article in Python Notes in learning .
May we make progress together .
Problem description
Previous web Generally, there is no anti crawler mechanism on the end , But now we encounter this problem , It's about to cross this mountain , Conquer it .
In the process of learning reptiles , I climbed a dictionary , As a result, people crawled backwards ,error.
{
“errorCode”:50}
Analyze and solve
Let's look at it first “ Review element ”, That is to choose “ Check ”, Shortcut key :Ctrl + Shift + I
choice network, Find the network request for translation , Then check it out Headers
Results found , stay Request URL in , There is one more website “_o”, Generally, delete this item , To avoid anti crawler mechanism .
follow-up
This can indeed run successfully , however , It's not that simple .
In the view Form Data Find out :
Unexpectedly, it has the same translation as before without anti crawler mechanism From Data The content is different , There are a few more , There are also a few items missing .
Two of them are the key signs of anti crawler :
salt : salt ? Encryption function , It's about time , Is the timestamp of system time conversion .
sign: User's signature string , By constant 、 Time stamp 、 The character composition of user translation , There are specific ways to encrypt , Therefore, each translated character corresponds to sign Is not the same .
If you don't want to delete “_o” To solve , We need to understand their anti crawler mechanism , Find the encryption method , So as to deal with anti reptiles ——
Find all of them in the web page source code “JS file ”, Find related The suffix is “fanyi.js” Of ( It's easy to find , Blue link ), After jumping, I got a full page of strange and disordered code .
Copy code to Code formatter In , You can format less strange code .
Search for “salt”, You can see that salt and sign How to define , Crack the encryption principle .
边栏推荐
- Have you ever encountered the situation that the IP is blocked when crawling web pages?
- (arxiv-2018) 重新审视基于视频的 Person ReID 的时间建模
- Why does stonedb dare to call it the only open source MySQL native HTAP database in the industry?
- 数学建模——红酒品质分类
- Make logic an optimization example in sigma DSP - data distributor
- LeetCode 112:路径总和
- 【云原生与5G】微服务加持5G核心网
- Stonedb invites you to participate in the open source community monthly meeting!
- 数学建模——自来水管道铺设问题
- Force deduction brush question (2): sum of three numbers
猜你喜欢

Overview of Qualcomm 5g intelligent platform

数学建模——派出所选址
![[10:00 public class]: application exploration of Kwai gpu/fpga/asic heterogeneous platform](/img/e7/1d06eba0e50eeb91d2d5da7524f4af.png)
[10:00 public class]: application exploration of Kwai gpu/fpga/asic heterogeneous platform

Lm13 morphological quantification momentum period analysis

数据平台数据接入实践

What is the function of data parsing?

【流放之路-第八章】

Process -- user address space and kernel address space

2022年编程语言排名,官方数据来了,让人大开眼界

Yocto project download and compilation
随机推荐
druid. The performance of IO + tranquility real-time tasks is summarized with the help of 2020 double 11
Leetcode 113: path sum II
Lxml web page capture the most complete strategy
Qt 内存管理小技巧
数学建模——红酒品质分类
【流放之路-第四章】
【云原生与5G】微服务加持5G核心网
2022年编程语言排名,官方数据来了,让人大开眼界
Explanation of yocto project directory structure
Qt源码分析--QObject(4)
Sword finger offer special assault edition day 13
Come on, handwritten RPC S2 serialization exploration
MySQL high performance optimization notes (including 578 pages of notes PDF document), collected
[the road of Exile - Chapter 8]
数学建模——永冻土层上关于路基热传导问题
Mysql存储json格式数据
Make logic an optimization example in sigma DSP - data distributor
(cvpr-2019) selective kernel network
Thirty years of MPEG audio coding
JS dom2 and dom3