Learn the mitmproxy packet capturing tool from scratch

Man In The Middle

mitm yes Man In The Middle An acronym for , It means someone in the middle , indicate mitmproxy It's an agent , You can intercept requests , Realize network packet capture . Well known network packet capture tools include Fiddler、Charles、HttpWatch、WireShark、BurpSuite and Postman etc. ,mitmproxy relatively speaking , Not so well known , Its unique advantage over the above tools is , It provides Python API, You can write Python The code records the traffic of the network request , So as to collect interface information and turn Into interface automation use cases, etc .mitmproxy Official documents are not only mitmproxy Introduction to the use of tools , And the principle of agent implementation , It is very helpful for us to learn and master network knowledge .

Installation and use

pip install

Log in and copy

       

        pip install mitmproxy
       
       

        
1.
       

mitmproxy yes Python Developed , use pip You can install it successfully . Officials have given binary installation packages and other methods , I think as a model Python Development tools , use pip Direct installation is more Pythonic Of .

Command to start

Can pass 3 Command start mitmproxy：

• • 
    
• ​mitmproxy Console
• ​mitmweb Webpage
• ​mitmdump Command line ​
  • 
    

Their contents are the same , It's just UI It's just different .

Set up agent manually

After opening mitmproxy Listen by default http://localhost:8080, You need to set up a proxy , Set the proxy server to this Address .

Windows You can set it like this ：

​ After setting, be sure to click the Save button , Otherwise, the agency will not take effect .​

actually ,Charles And other network packet capture tools will automatically configure the agent , and mitmproxy You need to set up the proxy manually , The development team explained that the browser version and configuration often change , So they suggest setting it manually on the Internet according to the environmental search method .

Download the certificate

visit http://mitm.it/, If the agent configuration does not take effect , After opening, it will be this interface ：

Normal word , The certificate download page will appear ：

Select the corresponding certificate according to the platform and install it OK 了 .

Use

mitmproxy The usage of other packet capture tools is similar .

The official video demonstrates mitmproxy How to use ：

​ ​https://docs.mitmproxy.org/stable/mitmproxytutorial-userinterface/​​

And it introduces 5 Operation mode ：

​ ​https://docs.mitmproxy.org/stable/concepts-modes/​​

The reader is free to read Next .

mitmproxy working principle

1. 1. 
      
2. client （ This machine ） Connect to the proxy server （​ ​http://127.0.0.1:8080​​）, The proxy server connects to the server .
3. The client sends a request to the proxy server , The proxy server sends the request to the server .
   1. 
      

All requests go through mitmproxy, Request interception can be realized .

​ about HTTP Come on , Directly with the help of mitmproxy Just pass the request . And for HTTPS Come on , There is one SSL/TLS Safety certification , Certificate must be installed , The server will think mitmproxy Your request is trustworthy , The request will be delivered successfully .​

HTTPS The agency process is much more complicated , The client follows mitmproxy Establishing a connection （1​_{2 Step ）, Then conduct security authentication （3}​6 Step ）, Last pass request （7~8 Step ）.

Reverse proxy

It is assumed that... Is used locally FastAPI Started Mock service , Address by http://127.0.0.1:5000, adopt Postman Interface , How do you use it? mitmproxy To intercept requests ？ This requires the use of reverse proxy .

First run in reverse proxy mode mitmproxy, The server port is 5000, The listening port is 8000：

Log in and copy

       

        mitmproxy --mode reverse:http://127.0.0.1:5000 --listen-host 127.0.0.1 --listen-port 8000
       
       

        
1.
       

Then send the request http://127.0.0.1:5000/login Inside 5000 Port changed to 8000, visit ：

​ ​http://127.0.0.1:8000/login​​

You can be in mitmproxy See the request ：

So you can use it locally Mock Service to debug intercepting requests Python Code. .

Reverse proxy , There is no need to manually set up the agent on this machine .

Python API

Python API yes mitmproxy Features of ：

In the Python The intercepted request is processed in the code .

addons It provides a lot of hook function , such as request：

Log in and copy

       

        """
        
Basic skeleton of a mitmproxy addon.
        

        
Run as follows: mitmproxy -s anatomy.py
        
"""
        
from mitmproxy import ctx
        

        

        
class Counter:
        
    def __init__(self):
        
        self.num = 0
        

        
    def request(self, flow):
        
        self.num = self.num + 1
        
        ctx.log.info("We've seen %d flows" % self.num)
        

        

        
addons = [
        
    Counter()
        
]
       
       

        
1.
        
2.
        
3.
        
4.
        
5.
        
6.
        
7.
        
8.
        
9.
        
10.
        
11.
        
12.
        
13.
        
14.
        
15.
        
16.
        
17.
        
18.
        
19.
        
20.
       

response：

Log in and copy

       

        """Add an HTTP header to each response."""
        

        

        
class AddHeader:
        
    def __init__(self):
        
        self.num = 0
        

        
    def response(self, flow):
        
        self.num = self.num + 1
        
        flow.response.headers["count"] = str(self.num)
        

        

        
addons = [
        
    AddHeader()
        
]
       
       

        
1.
        
2.
        
3.
        
4.
        
5.
        
6.
        
7.
        
8.
        
9.
        
10.
        
11.
        
12.
        
13.
        
14.
        
15.
       

Add at startup ​​-s​​ Parameter specifies the script ：

Log in and copy

       

        mitmdump -s ./anatomy.py
       
       

        
1.
       

The code will take effect when the request is intercepted .

more hook Please refer to the official description of the function ：

​ ​https://docs.mitmproxy.org/stable/api/events.html​​

No more details here .

​ And about how to use mitmproxy Recording traffic automatically generates parts of automated use cases , I will integrate it into tep In the new version , Then I'll make an introduction , Coming soon .​

Reference material ：

mitmproxy Official documents ​ ​https://docs.mitmproxy.org/stable/​​

official account 【 Test and development gang 】

Copyright Affirming ： this paper Original articles for bloggers , turn Please keep it original text link And author .