Selenium uses proxy IP

One 、 Application agent IP

If a user visits a website many times , It may be recognized as a reptile , This limits its clients ip The interview of , For some more formal websites , The anti climbing system is very strong , This is most likely to happen , So sometimes it's necessary to use agents IP, I usually choose to use random dynamic agents ip, In this way, it can ensure that a random user rather than a fixed user can be accessed every time .

Don't talk much , register IPIDEA go in , Free for registration 100M Traffic , If you have special needs, you can buy it again ：

http://www.ipidea.net/?utm-source=gejing&utm-keyword=?gejing

Generate API:

Click the generate link

Copy the link package and save it , I'll use it later .

Two 、 stay selenium Using agents IP actual combat ( One )

Set the basic format of the proxy ：

import requests
proxies = {
    'http': 'http://222.89.32.159:21079',
    'https': 'http://222.89.32.159:21079'
}
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.163 Safari/537.36"
}
res = requests.get(url=urls,headers=headers,proxies=proxies)

I've been thinking for a long time and haven't figured out which websites are strong in anti climbing , So I just found a website to test , You can try to visit the educational administration system of your school again ,12360,facebook etc. …

Target website ：

https://www.taobao.com/

So positioning is easy ：

driver.find_element_by_name('q')

I once wrote about using an agent to play with a crawler , yes requests modular , But as an individual , The more I learn later , Find out selenium Use more and more ,requests Gradually abandoned , So here is a supplementary selenium Add proxy .

It's very simple ：

ops.add_argument('--proxy-server=http://%s' % a) # Add proxy 

Notice the a The format is ：ip:port

Be careful ： Using agents ip Modules need to be installed selenium-wire：

pip install selenium-wire

You should be ：

from seleniumwire import webdriver

instead of ：

from selenium import webdriver

For example X Treasure search ：XX mobile phone

Complete code ：

from selenium import webdriver
from fake_useragent import UserAgent
from selenium.webdriver.chrome.options import Options

headers = {'User-Agent': UserAgent().random}
ops = Options()

driver = webdriver.Chrome(r'D:e Safe browser download chromedriver.exe')

api_url = ' Let you copy the agent api link '

driver.get(api_url)

a = driver.find_element_by_xpath('/html/body/pre').text  #  Access to the agent 

ops.add_argument('--proxy-server=http://%s' % a) # Add proxy 

driver.delete_all_cookies()  # clear cookies

driver.get('https://www.taobao.com/')

driver.find_element_by_name('q').send_keys(' Huawei mobile phones ')

Next, click the button ：

Identify the element where you click , And then use click Click to ：

from selenium.webdriver import ActionChains

b= driver.find_element_by_class_name('search-button')  # Location search 

ActionChains(driver).click(b).perform()

Maybe it triggered the anti pickpocketing mechanism ？ Need to log in , I don't know my X Po account password , Casually demonstrate and enter … The rest is done by yourself
Here is account password analysis ：


So the location account password is as follows , I set the account number ：chuanchuan, Input of password setting ：123456 Fabricate , Operate according to your actual account number , I won't go on , It's just positioning points

driver.find_element_by_name('fm-login-id').send_keys('chuanchuan')  #  Enter account 

driver.find_element_by_name('fm-login-password').send_keys('123456')  #  Input password 

The effect is as follows ：

3、 ... and 、 stay selenium Using agents IP actual combat ( Two )

Be careful ： Climb out with proxy wang Need foreign environment , In order to demonstrate, I had to buy a foreign environment test , Please have a look at ： Foreign environment server
such as ：

https://www.facebook.com/

Analysis account password login ：

The code is ：

from fake_useragent import UserAgent
import requests
from selenium import webdriver
from selenium.webdriver import ChromeOptions

headers = {'User-Agent': UserAgent().random}

api_url = ' Copy your api'

res = requests.post(api_url, headers=headers, verify=True)
PROXY = res.text
print(PROXY)

ops = ChromeOptions()

ops.add_argument('--proxy-server=%s' % PROXY)  #  Add proxy 

driver = webdriver.Chrome(r'D:e Safe browser download chromedriver.exe', chrome_options=ops)

driver.get("https://m.facebook.com/")
driver.find_element_by_name('email').send_keys(" Your account number ")
driver.find_element_by_name('pass').send_keys(' Your password ')
#  Button 
btnSubmit = driver.find_element_by_name('login')
btnSubmit.click()

The effect is as follows ：

My account is blocked , Subsequent operations will not continue to demonstrate , According to what I said selenium Knowledge points operate by themselves , Nothing more than point-to-point positioning, positioning and saving .

3、 ... and 、selenium Single element positioning practice review

3.1 Location filling

Take Microsoft search engine as an example ：

https://cn.bing.com/?mkt=zh-CN

analysis ：

therefore ：

from selenium import webdriver

driver = webdriver.Chrome(r'D:e Safe browser download chromedriver.exe')

driver.get('https://cn.bing.com/?mkt=zh-CN')

driver.find_element_by_name('q').send_keys('')

as follows ：

You can also do it in two ways ：

driver.find_element_by_id('sb_form_q').send_keys('')


driver.find_element_by_class_name('sb_form_q').send_keys('')

send_keys Function means filling in information .

3.2 Click on the search

analysis ：id perhaps class

b=driver.find_element_by_id('search_icon')

ActionChains(driver).click(b).perform()

as follows ：

It's on it id location , hold class Write down the positioning ：

b = driver.find_element_by_class_name('search')
ActionChains(driver).click(b).perform()

Interestingly , location class The time is ：search instead of search icon tooltip, I think it may be because of this space , Fortunately, I have reptile experience , Otherwise, the dead card is not positioned correctly .

3.3 Complete code

agent api Please replace with your , Apply according to my method ：

# coding=gbk
"""
 author ： Sichuan 
 official account ： Play with big data 
@ Time   : 2022/3/3 17:11
 Group ：428335755
"""
from selenium import webdriver
from selenium.webdriver import ActionChains

driver = webdriver.Chrome(r'D:e Safe browser download chromedriver.exe')

driver.get('https://cn.bing.com/?mkt=zh-CN')

# driver.find_element_by_name('q').send_keys('')

# driver.find_element_by_id('sb_form_q').send_keys('')

driver.find_element_by_class_name('sb_form_q').send_keys('')

# b=driver.find_element_by_id('search_icon')
b = driver.find_element_by_class_name('search')
ActionChains(driver).click(b).perform()