Dare to climb here, you're not far from prison, reptile reverse actual combat case

Put the important things first , Please look carefully ：

This article focuses on learning and communicating crawler technology , In especial JS Reverse technology for the purpose . Everyone should focus on learning the ideas and processes , Please don't imitate the website cited in the article , If you want to crawl , Please climb no more than 3 page , Otherwise, we will be responsible for the consequences .

Crawl It!

Reptile technology is the first step in 10000 years —— Analyze the web page structure and data loading method ：

The result is as soon as you open the web page , It seems that the website maintenance personnel have also gone for the holiday . Then come back from vacation and continue writing ..

This article pauses on 2021.2.11 In the morning 10:25

The Spring Festival holiday is over in the blink of an eye , I'm back “ Do things ” 了 ！

• open url, Display data page .

• adopt Chrome Browser's Network Tab can know , Turning pages is done by ajax Realized .

The response data is as follows ：

"95780ba0943730051dccb5fe3918f9fe1b6f2130681f99d5620c5497aa480f133ef544506ed439f
4bddb1ca63d7c066964c776ea098085c5cd9112e7cbcf533c5823e8b4bb0d542e5e2591fb6a46134
7bbb39b1ed0de45bce0d2512d6a0834f5c1d80d738d51820fb78c1aa2481fd43c825924bc101ad71
917930074d902df9764fe7f29f5a6cb917ea3e617ca76a348186f29d547a8b2bd4044cc9d9036fe2
b22e953f4aeea13705d9d1583300371749d5a882f6c50176739ca1171a50db15aae8c6550591dd8a
e62ad68dc69935e3fcf57379cc625c94acabb2effc4d5505f214f94a2a0e50110d9647d20bb189f4
4acad6184304fa46dff0048b0f58dd1b5d8a945794bbe9e66898b8e363192a79b84cde408f90a02c
9c9b85df2b6d5de973fc00874b223acd2da7df8642ce1e1e64aa6ff550e7b96db782d348f8209d5f
d59b9e26a2aa7ee9c4621cfc5a6d757d91e3862469c39b3b5152054a50667e0f8dfad00fafe55917
e938398c09fc682becfd9aa54f68ce6fd7e023cc7db416727a1572389c798d85f1cd348ff918a4d0
9e67c41a719ac57b6c06a05e954b84a6f0303192b740d3d5ed9c9cc4fe8f0c4de71aa12edc33069d
55f311894109c8c686745fcd22ed2cb33f5deeb6ca9af98c0097e86b450e98a9b1011587011014a3
5b63a3d55c17d92849f56d5e20e76847238c37620d2098b96c153accc9df0df7722868ff86028415
1bf7d7d0aa78ffeb89c882d592994ff09a235180f6f1ff6092fd60ce7eae7c69fc30ba47239f513e
08b9eb3e530cf08647933ea18d2c34c4cb9515b82d4153172311afb9af5d8bdd770d9d9410e75381
cf0f2aa2ca5cb33ccf3276ed0c7dfeda58b7c953bd7225b4395b07e922b75c26c3b5ee302ac3f0ae
40dc31a55ef92afdc7909c7f2e900e1ae8bf92764571c5a5ff27671fbce00c27fc379d64ca61d84f
95558bd2af9b83dd58349922543b8c8e0dc43dead3a102025ce19f4db76f5346b5bbd8784cefa854
51ef22ccd3967f1f6b2eb6f3a9114cc7f26ce107b25985c35ac9a82f11b637fd4a52c2c4f6966f05
3ec48eea3d056268304e58540c3eac82dd70e668e2b24707daeb5d1026b6ad7e7efb638bd51838f7
18c690d0ca4f421246e09432169a0c84ee96adcd4082f5796f474c636ed6d19702e8e5c45421e98a
56d94324160196673363f729ec7cc207f478fbaa865af04ab243cd1a5d3cf54b22393a04aeff51fd
42d33a3d76d11814a540332db0d9f61172e99b3f08f75b081367da26edae0ee7ff7dcab16883ab28
82e48eb95177cdf3d337a1836c949182e7bd5b155aef25c87f399664a367fbe961f1e532295dbce5
0c5a1f7acd891849bae2e6169eb8b9bfbe62e85e740686bc4d3a1ff47cc224507f7be3578751d571
6f006eebdced3c82eabbd500e55b8ff610cef3d947a317f103bf2cbde3fdfc2a0bfdc7e26bdcfee7
9a6967fd66eb01fc84e5c07718747c603e52719d045fdd19ff01f4a00921db26c1761fb744eebf73
262688e1802da1c1a0320196dc721f1a604512589fd23cc01895760881ba4a7af28e5e66b921a14b
85b7f86cd317e691aadf81a8d2b463318058490b19293a35d2fe87ffdecd220cd24fbee8d4073c26
bc61a3a0ad31377db1e660bb5b15c397d15ff820832ffb46f4008ced73bb1133dc957e6e3ec3d829
309addb1aa1e3a431d06b1c8aef6c43ed040f5ca311910bc0d70c34d8aca94d5b7084cfe46fb793d
e87a027f7bfe4d60489d05d3a0e661e9405a768548cdb69bb8a8e52aa0c775684b3e30f18cc059c9
38b451b3376a1736a21df7809791ed8144a893cd831245c64321064f45c67721413018e6bfef3ebe
572cda88723a970ae924cdde1b0a205dbf80b2e5aa55e6baac55b4a30232ee3bad287a40d4916a42
414f6805955ad4cb32bc1c7a4946d77cc9aa4582add2e68976a9a85003acec58f21327c30750e7c1
0615b94a66912b93237e5cc5568a5a53b7214801d3f9903bff3f445e33cc9fda9d19137917abbee4
c08d178095e5956acbb22dcd91639943ace2c24510b7206d92d011f44c055973c7e25c3de3553a5d
ad8457f0b0023e5cfef2f66ed0288b320f37e8313d061b9b75add7c7848ad722e616cad7c7c12359
018c34f723e37afaf21d466654312495a08beb81cb77e2c2bae6d0151fdaa68df623a70618cd6d3c
b66c1475c8eb24829742f10b90bfb9b5794f5eeba7226f7530ba86a69804cf8fbb4c6985c843c0c4
0a3f5bf88bdf55437b7bfcb0bb52f3bae7ba8ee035923b0dcceb144a30f7d2559103d512b4c786c1
710ebba3cedde63e99eee1c5f6ff7991d4b013ba8f2b223bd791baa5e6962a687415b947efb5dea9
07e3dc7d3eb19d9adbc4e9498c5362460ceb945533e3c4c049e93ce9711ca776d54279a01a226998
c5160efd85151618f565c38e9beed091441d2bd1e7677e032d13a25817f051241ca97995188f45cb
be1602636e855f827f35122b6b8cc887f013402b032fc4d1e24939c7cf2c5e686fdb4c4565022ba6
11726a8b73b5c7f8e77502d73b56b58f14239eebe1fcf356b20367e41a7434ce793b738a6d2b7f7d
3a6753a950641bce3fe01428fd777ddadab33fdde61b9fc56c527d89f52ec4cce77fca7ff693ba1d
7e45225eff225fded5e6708e73c20f6fb72b8ce8b080f3f1f055d5b4b624d0b4020a599b9dbebd9e
88c567cd894a94d846022de3298b171556155bec89323dc12a49137f06acb0f3fbad45aa55f94ebc
31458ccb0289dc403540b98e3a36d9b48a975bbb055dccfc57d87a6213f1fc57db70919dab450c3f
21da6daec9d0df7a847fd632a3e9a882b0e14e15565d06c7f437e7e69e9648f1cb9b022d6118d08e
7a0407d15a820b3249bff0ce986c8dc05ca7eca74b6302080a24c898437e1bbc5d380eb93a783710
407fa1d5389c127ab201e2508cdbc63b51a9d5c546d6bcee9f5e0b3a4b92239509884edc05c27aa1
76f64a21f4a7d4bfe997b4a4fb1b95c9f7e1cf1a85c33df25848be1ae83fe4a948095c38d66dbe10
73d8fa6e1bda1fc8a2913d8731d78b1bd6ed8dda5597da4e63d9dadbb8eadb1612198fc88342afd6
dd2442170683a5d55d27dea2475bf57fec14663906dbb6beec6c91a012f909e01626bf9a1a2a452f
c696b52e4edabe499f349cfeff364b28d8bf82aa9a5cfaae80c4b92e22f03c58b3cca43363f1a64e
cfa7c652de4e676b3d18029e0fb606e8ca0e5f2960b16705eeb7d34f48ef3d6f1a6f2e9dd2ce32fb
44af7cfff512488d7bfe6f57f40b5681a18bf2b6a60827b8aa888c81afbcaff7063fc4aeed89b137
5d35f4f7af1ce3cf9c5b3c81c4175651a981f755f422c30713d3e6ca4f3763f95c0f0b1d1f51c32b
769157a7c0a25f050560bb851fbf581a7186d515bd88e100fc91a56b39b86da71e8d028485358e58
31b00d2e653485ee7f404cf8848a681f3fafceb06138e090fc2caf2b03a4978173e1c6e71821ada8
9ceba3302cc16cfaa4b9bf472fa79156772e5ffdab8b44606e258176fa651d13e7388c13ee17d965
4b3e5564c24ccf478015014da5b0dcc8deb213cb0558c185339b22c490a332b4d4f9cf700cf2662d
ec0f815c8418e2573621713717463e4606b5903f66d7bc0581369a1586e84ebd7ba27277aecbd67f
737e35ab87ac78c428588bd7a1a26e4bc2834779d3399e06790bf1cb1ea77f665a449b9e6b3f459d
bd14bbf1534b4f19292f35fbb5942ec505fcf654235d8e1b556843c87b6b868b4e049733e85f551f
78e3587dd6223183f038be2482259638a0650cf6eff6f1bf5383d4774e5b1e5bd20dc839df78ae23
cca22d23d30f2fa1a49307220474a371666908619339f83cc7ae3fcd37a37db640a06970b39db055
a2a8a0d067fe72516b9b9eccc51a69dcd19e961bfe3b131e8130ec2e648d20aafafaceb7f05f03e5
a4e75cd3dadece5d15cdb26c7e6c39a4fcf400490a668e4221f4c0129cfeb46078775392218c65db
37935c172fc5c10f2b0079a195698acd00f736071593604a1e088db94b29319b75f759cc0e6fb3a4
e226a2305dfb7ad0f0f3b9b02b229953e030ac50fe75560f7a98ccfff5a3dd06617349b4fd1ee5ac
db33789b5724e5c3e4441d13a11974b11406bca931d7077b0b2e1e9c4c7a6eb7245fc5779c1cf922
0f64e3bbd4c1fc1cd5000c879700dcb35e5fd77508fa96063de2c7531ae658bf048156b7678b4b41
bf66f00eb6198503671668115bcfcea46ab1a07a7c077d339bb24f65757fe8cfc944cbc1ab7aae39
b19c40619f28d86e58b1f39e52202ff220092ded48323531264e8271129c644ca7e6c4ee03ccbae9
02b990257b1a55daec1b919f33fff441198653032d1026ac1b404de3562c6f851664fd534b04e544
c43dd82ac7ebeaccc7a9a479678c2371665e99815e6a0c60b8b66a21fb0275dd5ee2c26cd69ddff0
b173b40b90e6a602a30c5f0b33e8b933de6f13520af43adf5b87e22ee5af4c307a27b2f788feaefc
668d87af90bf442df137fd26c8e29a6d40c5cb5fbd551752ef0660477013614528661901067c6d64
a95917b9b419180c4d3d3f0a1edb8521d90700ad70a2f4d0717d5bdc24766802ece3be32dc45bfa0
79cd7ff464608926728f9b395f609f53e42c7ba4136abd566d53d0b1819fbef45358df803eadd071
a250c3ca1abd7c2426975dbdfc5dc8369c79e785a4f6c0ed5a7152ec4912df7bf97914abca3f0b4f
616905ed128809a5ead9d9fe57425a16a32fb0ff3e81e409718814eca8226676d3cc8df61b8b0827
c5c54696a5c5c25f697fb842ca316a36a09dfb48b6fc9f5fdc3717affb3477d98259126821e2fedb
292daba634bff50e1e6136dfc83a4441bc8f2ed5350792b6c4668e76f5db7d64b4f40e94bb749471
19881f0077536a67422278bf0cc97614c2f7800ad08fc3e04e5ea3b010a6c16af0a8e16dfb15b780
e6eb284f197ce9d3d370536bb10c1f48762e7842a651875c0d47c530fc9fd5705fad0b0f3a533500
153e810ad751ff15af761011399a162029a77b4fe937edd5b8987ac01e2c89f1642716fe83627124
7f26c983bea9c09ae415a7d367f5a3bfd1d5cf3e3eafd235d0a222869a6d2f3403937a2f6bb0b7eb
6568c743a39ee90bb7b8efd6d1b5e863da2cae9642eedb758d44f2dd749a6327ee586de842042c7c
ae67b09fd95c57aca18b6a25321440c04bdb056ae12b593fc2f96a45f608d0d0a96e15ad6fff2bbb
bf906b02451af38cfce206ec1004de390f3fb724ddafd903dd31d15fabe97d7a540fda85467014d2
0ea9e3a5a92e88c9718ac6ebd35b7047c333a4a8dcf56b66039b2b77fda17f9e6a741801a43410b6
d4db887d713b83a5db7a1dae485312f452d805da5cf8e385c656b10754359ef3031222fcb3fa3f79
de5acd68630ebc075a86b63d100c05f472057401e9bbc70e931550337f6536b69bda901e8503de8a
b9b0216a0edaea8117381f7e0a91e1abd1e4c3f8009ce0eda0802fc896248754b35d85b76b8ec569
917451a3ec2dadc3a2d07b7f5fb0bcc6c5a9fc5e1280310fbf7dc4bf060228ed3c997f35ab16d44c
0c5dceec43281120738882cc22b1ee5172d41613723c8da59fa6a70438868e2aa216e152435ed196
53866ebcc82721b1a524ebe0dde2a60647924e0d9ea4e391e2fc042f46b93658dc73087360b60969
cf50615daf3a7fe37f24691aad9114b8b8b2c010115236100c007d2e0754aff82e380d9147b18b5d
1489bdfb5753d0ad58854212935801080ad49054b9a599ecaafaa29c2194ef5cf98515e60dfdbce4
61f190b7ea166946b7d3b584b0f137ece880feb9431c5d6732bd3302c1147dbd03edf1aedc8c509e
0a6238a74aae81acd9cab9c1c4d8aeed9a0ac8f9a8a32a93217600e5a43c807677caa2954fb1e065
51b74336c8f37c83edd238f3b2b9434949300a6dc5f00f08ecc2fc8b4ba329c2217bc03e8e6e3457
e92d72c824ef97a8e35ebe395bd4e8944074fd64d6f59e672daba63bb7dcf416ec1d47400d186a4f
882721cbc6a6809b92aab8e0e66f6a6f82b35fc254558afadfa857179c245c626672b8a82c99b194
8761c98707206cbe7ecab8ffee47e9a7577504f78a59059b232a8ed9600a9516cb1619a42e471b23
0b4074d1df1acbb07834ece68090ca8f242fb126e688881004e7751c3f47a18c9b62994e4bbe2aff
fdc007f125234cdddc007b138a4eadaab2bc94f4d645e3adf00d490e4e2283d7dfff39bd3b0c5759
4ca2696c549281887f82776d98e1bab01f031d5767a71053122976a6e59fb2a13f07c98b5e45d530
1bb11d75bb83fb0c8c1090cd5089ff73eb07b6ffc775e5da3194702c2bf8b01f139457158475a8a2
dab9657e4321ebf05f52b64c604e6338fe1801c81db8120c553b641e3b3344d700a91d25412c90fc
981abe075450ed73fd9145d5b9626983272a47d60b087d30a76ede332fdaef790e14583d81dfc89b
8206a120badd4db7b11d73bc2ce882fea80d0ebb86d835c786bf81c63c3855d873535ad17ab183cd
4"

The messy characters returned by the request are the encrypted data .（ Yes, of course , Your server gives me encrypted data , You must decrypt the data locally if you want to display it on the browser page , You certainly won't show this pile of garbled code .）

• Then we need to find the corresponding js Decryption process .

Through the power of nine cows and two tigers, we finally found the place to decrypt , This step is the most time-consuming , The most likely step to baldness .

The main ways to find the reverse entry point are ：

• Keyword search , Break point debugging
• utilize XHR To set breakpoints ;
• Check the call stack .

This web page I typed XHR The breakpoint + Search for keywords decrtpt( Decrypt ) The way to find this encryption .

Let's take a look at the debug data ~

{
    "code":200,"data":
{
    "list":[{
    "QY_FR_NAME":" Miao Cunxu ","QY_REGION":"350200","QY_NAME":" Xiamen Xinghaiwan Engineering Management Co., Ltd ","QY_REGION_NAME":" Fujian Province - Xiamen City ","QY_ORG_CODE":"913502112601177740","COLLECT_TIME":1477106448000,"RN":31,"QY_ID":"616160676166636361616466606865656369","QY_SRC_TYPE":"0","OLD_CODE":"260117774"},
{
    "QY_FR_NAME":" Chen Youliang ","QY_REGION":"330800","QY_NAME":" Zhejiang Zhoudao Engineering Management Co., Ltd ","QY_REGION_NAME":" Zhejiang Province - Quzhou City ","QY_ORG_CODE":"91330802330012039P","COLLECT_TIME":1477106529000,"RN":32,"QY_ID":"616160676166636361616466606865656261","QY_SRC_TYPE":"0","OLD_CODE":"330012039"},
{
    "QY_FR_NAME":" Xu Bingshui ","QY_REGION":"370100","QY_NAME":" Shandong Zhengda Construction Supervision Co., Ltd ","QY_REGION_NAME":" Shandong Province - Jinan City ","QY_ORG_CODE":"913701022671825366","COLLECT_TIME":1477106425000,"RN":33,"QY_ID":"616160676166636361616466606865656260","QY_SRC_TYPE":"0","OLD_CODE":"267182536"},
{
    "QY_FR_NAME":" Wang Zhong ","QY_REGION":"510100","QY_NAME":" KaiBang Construction Management Group Co., Ltd ","QY_REGION_NAME":" Sichuan Province - Chengdu ","QY_ORG_CODE":"91510105725354703M","COLLECT_TIME":1477106469000,"RN":34,"QY_ID":"616160676166636361616466606865656263","QY_SRC_TYPE":"0","OLD_CODE":"725354703"},
{
    "QY_FR_NAME":" Liu Yinuo ","QY_REGION":"371300","QY_NAME":" Linyi Hongli Construction Engineering Supervision Consulting Co., Ltd ","QY_REGION_NAME":" Shandong Province - Linyi City ","QY_ORG_CODE":"913713005543587173","COLLECT_TIME":1477106454000,"RN":35,"QY_ID":"616160676166636361616466606865656262","QY_SRC_TYPE":"0","OLD_CODE":"554358717"},
{
    "QY_FR_NAME":" Wang Hongjuan ","QY_REGION":"410100","QY_NAME":" Henan Zhongjian Engineering Technology Co., Ltd ","QY_REGION_NAME":" Henan province - Zhengzhou city ","QY_ORG_CODE":"914101057919311268","COLLECT_TIME":1477106526000,"RN":36,"QY_ID":"616160676166636361616466606865656265","QY_SRC_TYPE":"0","OLD_CODE":"791931126"},
{
    "QY_FR_NAME":" Lin Jin ","QY_REGION":"350100","QY_NAME":" Fujian Jianxin Engineering Management Group Co., Ltd ","QY_REGION_NAME":" Fujian Province - Fuzhou City ","QY_ORG_CODE":"913501001544135960","COLLECT_TIME":1477106407000,"RN":37,"QY_ID":"616160676166636361616466606865656264","QY_SRC_TYPE":"0","OLD_CODE":"154413596"},
{
    "QY_FR_NAME":" Yang gaobao ","QY_REGION":"421200","QY_NAME":" Jiayu County Jiatai Construction Supervision Co., Ltd ","QY_REGION_NAME":" Hubei province - Xianning City ","QY_ORG_CODE":"91421221737145124M","COLLECT_TIME":1477106486000,"RN":38,"QY_ID":"616160676166636361616466606865656267","QY_SRC_TYPE":"0","OLD_CODE":"737145124"},
{
    "QY_FR_NAME":" Jiang Yabin ","QY_REGION":"410800","QY_NAME":" Henan synergy Engineering Supervision Co., Ltd ","QY_REGION_NAME":" Henan province - Jiaozuo City ","QY_ORG_CODE":"91410800555731456W","COLLECT_TIME":1477106405000,"RN":39,"QY_ID":"616160676166636361616466606865656266","QY_SRC_TYPE":"0","OLD_CODE":"555731456"},
{
    "QY_FR_NAME":" Huang Xiangjun ","QY_REGION":"450100","QY_NAME":" Guangxi Nanning Jianke Engineering Supervision Co., Ltd ","QY_REGION_NAME":" Guangxi Zhuang Autonomous Region - Nanning city ","QY_ORG_CODE":"914501031983159639","COLLECT_TIME":1477106406000,"RN":40,"QY_ID":"616160676166636361616466606865656269","QY_SRC_TYPE":"0","OLD_CODE":"198315963"},
{
    "QY_FR_NAME":" Liao Yuan ","QY_REGION":"510100","QY_NAME":" Chengdu Hailong Engineering Project Management Co., Ltd ","QY_REGION_NAME":" Sichuan Province - Chengdu ","QY_ORG_CODE":"91510100201986152W","COLLECT_TIME":1477106437000,"RN":41,"QY_ID":"616160676166636361616466606865656268","QY_SRC_TYPE":"0","OLD_CODE":"201986152"},
{
    "QY_FR_NAME":" Zou Yun ","QY_REGION":"360500","QY_NAME":" Xinyu YONGGU Engineering Supervision Co., Ltd ","QY_REGION_NAME":" Jiangxi Province - Xinyu City ","QY_ORG_CODE":"91360500723935352F","COLLECT_TIME":1477109337000,"RN":42,"QY_ID":"616160676166636361616466606865656561","QY_SRC_TYPE":"0","OLD_CODE":"723935352"},
{
    "QY_FR_NAME":" Wang Wenquan ","QY_REGION":"371300","QY_NAME":" Shandong zhenghang Engineering Consulting Co., Ltd ","QY_REGION_NAME":" Shandong Province - Linyi City ","QY_ORG_CODE":"91371302092199458A","COLLECT_TIME":1477106426000,"RN":43,"QY_ID":"616160676166636361616466606865656560","QY_SRC_TYPE":"0","OLD_CODE":"092199458"},
{
    "QY_FR_NAME":" Zhang Hongjian ","QY_REGION":"530100","QY_NAME":" Yunnan jinji'an construction consulting and Supervision Co., Ltd ","QY_REGION_NAME":" Yunnan Province - Kunming ","QY_ORG_CODE":"91530000719414685K","COLLECT_TIME":1479833531000,"RN":44,"QY_ID":"616160676166636361616466606865656567","QY_SRC_TYPE":"0","OLD_CODE":"719414685"},
{
    "QY_FR_NAME":" Zhang Zhiming ","QY_REGION":"610700","QY_NAME":" Hanzhong Zhicheng Construction Engineering Supervision Co., Ltd ","QY_REGION_NAME":" Shaanxi Province - Hanzhong City ","QY_ORG_CODE":"91610725667992132H","COLLECT_TIME":1477106417000,"RN":45,"QY_ID":"616160676166636361616466606865656568","QY_SRC_TYPE":"0","OLD_CODE":"667992132"}],
"pageNum":2,"pageSize":15,"total":450},"message":"","success":true}

From the picture above , We can know that the decryption process consists of a h function Realized ,h function Put the parameters passed in t（ It's that big string of encrypted characters ）, adopt AES After decryption, a json Format string , This is the data we want .

Let's start with python To simulate this process ：

OK, We can go through requests Get the encrypted data , The next step is to decrypt the characters .

This is the second time-consuming step in the reverse process , Pick the js The code then uses pyexecjs Module to get decrypted data ; Or put js The code to use
python Rewrite ( Is to put js The code logic is changed to python Of ).

According to my knowledge , This is a AES encryption , The pattern is CBC, The fill pattern is Pkcs7, Just know the offset again iv And the key key How much is the , This decryption process can be completely broken , According to the position parameters of the function ,f The corresponding is the secret key key.

It doesn't take much time to come ~~~ It's next to , Get the key and vi after , You can use Python Medium Crypto The module carries out AES Decrypt . The detailed code is as follows ：

import requests
from Crypto.Cipher import AES
from binascii import b2a_hex, a2b_hex


#  towards url Initiate request , Parameters page_num Is the number of pages requested 
def request(page_num):
  headers = {
    
    'Connection': 'keep-alive',
    'Pragma': 'no-cache',
    'Cache-Control': 'no-cache',
    'Accept': 'application/json, text/plain, */*',
    'timeout': '30000',
    'accessToken': '',
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.104 Safari/537.36',
    'Referer': 'http://jzsc.mohurd.gov.cn/data/company',
    'Accept-Language': 'zh-CN,zh;q=0.9',
  }

  params = (
    ('pg', page_num),
    ('pgsz', '15'),
    ('total', '450'),
  )
  response = requests.get('http://jzsc.mohurd.xxx.cn/api/webApi/dataservice/query/comp/list', headers=headers, params=params, verify=False)
  return response.text


#  Decrypt the response data returned from the request , Parameters text For response data 
def decrypt(text):
  key = 'jo8j9wGw%6HbxfFn'.encode('utf-8')  // secret key 
  iv = '0123456789ABCDEF'.encode('utf-8')  // Encryption offset 
  mode = AES.MODE_CBC      // Encryption mode 
  cryptos = AES.new(key, mode, iv)
  plain_text = cryptos.decrypt(a2b_hex(text))
  return bytes.decode(plain_text).rstrip('\0')

#  get data , Let's observe whether the printed data is decrypted data 
def get_data():
  for i in range(3):
    print(decrypt(request(str(i))))

if __name__ == '__main__':
  get_data()

After code execution , We got the data on the webpage ：

{
    "code":200,"data":{
    "list":[{
    "QY_FR_NAME":" Liao Luyun ","QY_REGION":"360500","QY_NAME":" Jiangxi Tiangong water and land engineering design Co., Ltd ","QY_REGION_NAME":" Jiangxi Province - Xinyu City ","QY_ORG_CODE":"91360500079012847X","COLLECT_TIME":1477106416000,"RN":1,"QY_ID":"616160676166636361616466606865626860","QY_SRC_TYPE":"0","OLD_CODE":"079012847"},{
    "QY_FR_NAME":" Shan Jun ","QY_REGION":"360500","QY_NAME":" Jiangxi Zhongyu architectural design and Research Institute Co., Ltd ","QY_REGION_NAME":" Jiangxi Province - Xinyu City ","QY_ORG_CODE":"91360500159864134W","COLLECT_TIME":1477106400000,"RN":2,"QY_ID":"616160676166636361616466606865626863","QY_SRC_TYPE":"0","OLD_CODE":"159864134"},{
    "QY_FR_NAME":" Lin Maoqing ","QY_REGION":"420100","QY_NAME":" Wuhan Maoye Construction Engineering Co., Ltd ","QY_REGION_NAME":" Hubei province - Wuhan City ","QY_ORG_CODE":"91420106074476087K","COLLECT_TIME":1477106413000,"RN":3,"QY_ID":"616160676166636361616466606865626862","QY_SRC_TYPE":"0","OLD_CODE":"074476087"},{
    "QY_FR_NAME":" Tu Ming ","QY_REGION":"360100","QY_NAME":" Nanchang water conservancy planning and Design Institute ","QY_REGION_NAME":" Jiangxi Province - Nanchang City ","QY_ORG_CODE":"91360100MA35J54Y86","COLLECT_TIME":1477106440000,"RN":4,"QY_ID":"616160676166636361616466606865626865","QY_SRC_TYPE":"0","OLD_CODE":"491102609"},{
    "QY_FR_NAME":" Guolinjiang ","QY_REGION":"410100","QY_NAME":" Bowei Zhucheng Design Co., Ltd ","QY_REGION_NAME":" Henan province - Zhengzhou city ","QY_ORG_CODE":"914101025651418126","COLLECT_TIME":1477106411000,"RN":5,"QY_ID":"616160676166636361616466606865626864","QY_SRC_TYPE":"0","OLD_CODE":"565141812"},{
    "QY_FR_NAME":" Guyahong ","QY_REGION":"440100","QY_NAME":" Guangdong Boya Information Technology Co., Ltd ","QY_REGION_NAME":" Guangdong province, - guangzhou ","QY_ORG_CODE":"914401017163873948","COLLECT_TIME":1477106545000,"RN":6,"QY_ID":"616160676166636361616466606865626867","QY_SRC_TYPE":"0","OLD_CODE":"716387394"},{
    "QY_FR_NAME":" Hongxichang ","QY_REGION":"440300","QY_NAME":" Shenzhen Dasheng Construction Group Co., Ltd ","QY_REGION_NAME":" Guangdong province, - shenzhen ","QY_ORG_CODE":"91440300564201676L","COLLECT_TIME":1477106522000,"RN":7,"QY_ID":"616160676166636361616466606865626869","QY_SRC_TYPE":"0","OLD_CODE":"564201676"},{
    "QY_FR_NAME":" Zheng Li ","QY_REGION":"440300","QY_NAME":" Shenzhen Qisheng decoration design Engineering Co., Ltd ","QY_REGION_NAME":" Guangdong province, - shenzhen ","QY_ORG_CODE":"91440300087044558G","COLLECT_TIME":1477106425000,"RN":8,"QY_ID":"616160676166636361616466606865626868","QY_SRC_TYPE":"0","OLD_CODE":"087044558"},{
    "QY_FR_NAME":" Li Sheng ","QY_REGION":"440100","QY_NAME":" Guangdong Beisi Control Technology Co., Ltd ","QY_REGION_NAME":" Guangdong province, - guangzhou ","QY_ORG_CODE":"91440000735024820F","COLLECT_TIME":1477106400000,"RN":9,"QY_ID":"616160676166636361616466606865656160","QY_SRC_TYPE":"0","OLD_CODE":"735024820"},{
    "QY_FR_NAME":" Zhang XiaoCong ","QY_REGION":"440600","QY_NAME":" Guangdong Tianyuan Architectural Design Co., Ltd ","QY_REGION_NAME":" Guangdong province, - Foshan city, ","QY_ORG_CODE":"91440606725095377Y","COLLECT_TIME":1477106406000,"RN":10,"QY_ID":"616160676166636361616466606865656163","QY_SRC_TYPE":"0","OLD_CODE":"725095377"},{
    "QY_FR_NAME":" Chen Fen ","QY_REGION":"440100","QY_NAME":" Shenzhen Zhidao Landscape Design Co., Ltd ","QY_REGION_NAME":" Guangdong province, - guangzhou ","QY_ORG_CODE":"91440300758618904A","COLLECT_TIME":1477106842000,"RN":11,"QY_ID":"616160676166636361616466606865656162","QY_SRC_TYPE":"0","OLD_CODE":"758618904"},{
    "QY_FR_NAME":" Huang Ronghui ","QY_REGION":"440500","QY_NAME":" Shantou Zhenqiao Decoration Engineering Corporation ","QY_REGION_NAME":" Guangdong province, - Shantou city ","QY_ORG_CODE":"914405001927423079","COLLECT_TIME":1477106439000,"RN":12,"QY_ID":"616160676166636361616466606865656165","QY_SRC_TYPE":"0","OLD_CODE":"192742307"},{
    "QY_FR_NAME":" Li Jian ","QY_REGION":"440300","QY_NAME":" Shenzhen Senlei Yiming design consultant 
 Ask limited company ","QY_REGION_NAME":" Guangdong province, - shenzhen ","QY_ORG_CODE":"914403007663577081","COLLECT_TIME":1477106424000,"RN":13,"QY_ID":"616160676166636361616466606865656164","QY_SRC_TYPE":"0","OLD_CODE":"766357708"},{"QY_FR_NAME":" Zhang Yumei ","QY_REGION":"450100","QY_NAME":" Guangxi Xipu Engineering Design Co., Ltd ","QY_REGION_NAME":" Guangxi Zhuang Autonomous Region - Nanning city ","QY_ORG_CODE":"669733898","COLLECT_TIME":1477106445000,"RN":14,"QY_ID":"616160676166636361616466606865656167","QY_SRC_TYPE":"0","OLD_CODE":"669733898"},{"QY_FR_NAME":" Peng Wei ","QY_REGION":"440100","QY_NAME":" Guangzhou Water Engineering Co., Ltd ","QY_REGION_NAME":" Guangdong province, - guangzhou ","QY_ORG_CODE":"91440101190437587Y","COLLECT_TIME":1477106420000,"RN":15,"QY_ID":"616160676166636361616466606865656061","QY_SRC_TYPE":"0","OLD_CODE":"190437587"}],"pageNum":0,"pageSize":15,"total":513826},"message":"","success":true}
{
    "code":200,"data":{
    "list":[{
    "QY_FR_NAME":" Wu fan ","QY_REGION":"510100","QY_NAME":" Four seasons wonderful Design Group Co., Ltd ","QY_REGION_NAME":" Sichuan Province - Chengdu ","QY_ORG_CODE":"915100000858139239","COLLECT_TIME":1477106410000,"RN":16,"QY_ID":"616160676166636361616466606865656060","QY_SRC_TYPE":"0","OLD_CODE":"085813923"},{
    "QY_FR_NAME":" Wang Hongwu ","QY_REGION":"510100","QY_NAME":" Chengdu City Tiandi Engineering Design Consulting Co., Ltd ","QY_REGION_NAME":" Sichuan Province - Chengdu ","QY_ORG_CODE":"91510105762288418C","COLLECT_TIME":1477106422000,"RN":17,"QY_ID":"616160676166636361616466606865656063","QY_SRC_TYPE":"0","OLD_CODE":"762288418"},{
    "QY_FR_NAME":" Yuan Liang ","QY_REGION":"610100","QY_NAME":" Shaanxi shouchuang Tiancheng Engineering Technology Co., Ltd ","QY_REGION_NAME":" Shaanxi Province - Xi'an City ","QY_ORG_CODE":"91610000684783500F","COLLECT_TIME":1477106808000,"RN":18,"QY_ID":"616160676166636361616466606865656062","QY_SRC_TYPE":"0","OLD_CODE":"684783500"},{
    "QY_FR_NAME":" Bai Jinhan ","QY_REGION":"510100","QY_NAME":" Sichuan Taixing Decoration Engineering Co., Ltd ","QY_REGION_NAME":" Sichuan Province - Chengdu ","QY_ORG_CODE":"915100002018730434","COLLECT_TIME":1477106445000,"RN":19,"QY_ID":"616160676166636361616466606865656065","QY_SRC_TYPE":"0","OLD_CODE":"201873043"},{
    "QY_FR_NAME":" Yellow wintersweet ","QY_REGION":"510100","QY_NAME":" Chengdu Sanyuan Architectural Survey and Design Co., Ltd ","QY_REGION_NAME":" Sichuan Province - Chengdu ","QY_ORG_CODE":"9151018479216087X2","COLLECT_TIME":1477106448000,"RN":20,"QY_ID":"616160676166636361616466606865656064","QY_SRC_TYPE":"0","OLD_CODE":"202625084"},{
    "QY_FR_NAME":" Duan Li ","QY_REGION":"510100","QY_NAME":" Sichuan Ruishi Zhongzheng Construction Engineering Co., Ltd ","QY_REGION_NAME":" Sichuan Province - Chengdu ","QY_ORG_CODE":"91510115592094097F","COLLECT_TIME":1477106696000,"RN":21,"QY_ID":"616160676166636361616466606865656067","QY_SRC_TYPE":"0","OLD_CODE":"592094097"},{
    "QY_FR_NAME":" Lou Yongjun ","QY_REGION":"110000","QY_NAME":" Beijing Huaqing Technology Engineering Management Co., Ltd ","QY_REGION_NAME":" The Beijing municipal ","QY_ORG_CODE":"9111010872146477XF","COLLECT_TIME":1477106408000,"RN":22,"QY_ID":"616160676166636361616466606865656066","QY_SRC_TYPE":"0","OLD_CODE":"72146477X"},{
    "QY_FR_NAME":" Chen Jigang ","QY_REGION":"210100","QY_NAME":" Shenyang Huasheng Engineering Construction Supervision Co., Ltd ","QY_REGION_NAME":" Liaoning Province - Shenyang 
","QY_ORG_CODE":"91210103242650807A","COLLECT_TIME":1477106423000,"RN":23,"QY_ID":"616160676166636361616466606865656069","QY_SRC_TYPE":"0","OLD_CODE":"242650807"},{"QY_FR_NAME":" Liuhaifu ","QY_REGION":"230600","QY_NAME":" Daqing Daguang Engineering Construction Supervision Co., Ltd ","QY_REGION_NAME":" Heilongjiang Province - Daqing City ","QY_ORG_CODE":"912306047253128885","COLLECT_TIME":1477106451000,"RN":24,"QY_ID":"616160676166636361616466606865656361","QY_SRC_TYPE":"0","OLD_CODE":"725312888"},{"QY_FR_NAME":" Li Huaibin ","QY_REGION":"211400","QY_NAME":" Huludao Haiwei project 
 Construction Supervision Co., Ltd ","QY_REGION_NAME":" Liaoning Province - Huludao City ","QY_ORG_CODE":"912114007164286430","COLLECT_TIME":1477106432000,"RN":25,"QY_ID":"616160676166636361616466606865656360","QY_SRC_TYPE":"0","OLD_CODE":"716428643"},{"QY_FR_NAME":" Wang Feng ","QY_REGION":"120000","QY_NAME":" Tianjin Port Oil Engineering Construction Supervision Co., Ltd ","QY_REGION_NAME":" tianjin ","QY_ORG_CODE":"9112011674668236XP","COLLECT_TIME":1477107655000,"RN":26,"QY_ID":"616160676166636361616466606865656363","QY_SRC_TYPE":"0","OLD_CODE":"74668236X"},{"QY_FR_NAME":" Lu Minjie ","QY_REGION":"310000","QY_NAME":" Shanghai Shenyi Engineering Consulting Co., Ltd ","QY_REGION_NAME":" Shanghai ","QY_ORG_CODE":"913101181345318787","COLLECT_TIME":1477106405000,"RN":27,"QY_ID":"616160676166636361616466606865656362","QY_SRC_TYPE":"0","OLD_CODE":"134531878"},{"QY_FR_NAME":" Gao Shuangxuan ","QY_REGION":"130100","QY_NAME":" Shijiazhuang Zhongtian Engineering Construction Supervision Co., Ltd ","QY_REGION_NAME":" Hebei Province - Shijiazhuang city ","QY_ORG_CODE":"9113010060118700X3","COLLECT_TIME":1477106403000,"RN":28,"QY_ID":"616160676166636361616466606865656365","QY_SRC_TYPE":"0","OLD_CODE":"60118700X"},{"QY_FR_NAME":" Fan Guangyi ","QY_REGION":"231000","QY_NAME":" Mudanjiang Longhua Engineering Construction Supervision Co., Ltd ","QY_REGION_NAME":" Heilongjiang Province - Mudanjiang City ","QY_ORG_CODE":"912310007026319900","COLLECT_TIME":1477106420000,"RN":29,"QY_ID":"616160676166636361616466606865656367","QY_SRC_TYPE":"0","OLD_CODE":"702631990"},{"QY_FR_NAME":" Wang Guohua ","QY_REGION":"320100","QY_NAME":" Jiangsu communications engineering consulting and Supervision Co., Ltd ","QY_REGION_NAME":" Jiangsu Province - nanjing ","QY_ORG_CODE":"913200001347697633","COLLECT_TIME":1477106402000,"RN":30,"QY_ID":"616160676166636361616466606865656366","QY_SRC_TYPE":"0","OLD_CODE":"134769763"}],"pageNum":1,"pageSize":15,"total":450},"message":"","success":true}

{
    "code":200,"data":{
    "list":[{
    "QY_FR_NAME":" Miao Cunxu ","QY_REGION":"350200","QY_NAME":" Xiamen Xinghaiwan Engineering Management Co., Ltd ","QY_REGION_NAME":" Fujian Province - Xiamen City ","QY_ORG_CODE":"913502112601177740","COLLECT_TIME":1477106448000,"RN":31,"QY_ID":"616160676166636361616466606865656369","QY_SRC_TYPE":"0","OLD_CODE":"260117774"},{
    "QY_FR_NAME":" Chen Youliang ","QY_REGION":"330800","QY_NAME":" Zhejiang Zhoudao Engineering Management Co., Ltd ","QY_REGION_NAME":" Zhejiang Province - Quzhou City ","QY_ORG_CODE":"91330802330012039P","COLLECT_TIME":1477106529000,"RN":32,"QY_ID":"616160676166636361616466606865656261","QY_SRC_TYPE":"0","OLD_CODE":"330012039"},{
    "QY_FR_NAME":" Xu Bingshui ","QY_REGION":"370100","QY_NAME":" Shandong Zhengda Construction Supervision Co., Ltd ","QY_REGION_NAME":" Shandong Province - Jinan City ","QY_ORG_CODE":"913701022671825366","COLLECT_TIME":1477106425000,"RN":33,"QY_ID":"616160676166636361616466606865656260","QY_SRC_TYPE":"0","OLD_CODE":"267182536"},{
    "QY_FR_NAME":" Wang Zhong ","QY_REGION":"510100","QY_NAME":" KaiBang Construction Management Group Co., Ltd ","QY_REGION_NAME":" Sichuan Province - Chengdu ","QY_ORG_CODE":"91510105725354703M","COLLECT_TIME":1477106469000,"RN":34,"QY_ID":"616160676166636361616466606865656263","QY_SRC_TYPE":"0","OLD_CODE":"725354703"},{
    "QY_FR_NAME":" Liu Yinuo ","QY_REGION":"371300","QY_NAME":" Linyi Hongli Construction Engineering Supervision Consulting Co., Ltd ","QY_REGION_NAME":" Shandong Province - Linyi City ","QY_ORG_CODE":"913713005543587173","COLLECT_TIME":1477106454000,"RN":35,"QY_ID":"616160676166636361616466606865656262","QY_SRC_TYPE":"0","OLD_CODE":"554358717"},{
    "QY_FR_NAME":" Wang Hongjuan ","QY_REGION":"410100","QY_NAME":" Henan Zhongjian Engineering Technology Co., Ltd ","QY_REGION_NAME":" Henan province - Zhengzhou city ","QY_ORG_CODE":"914101057919311268","COLLECT_TIME":1477106526000,"RN":36,"QY_ID":"616160676166636361616466606865656265","QY_SRC_TYPE":"0","OLD_CODE":"791931126"},{
    "QY_FR_NAME":" Lin Jin ","QY_REGION":"350100","QY_NAME":" Fujian Jianxin Engineering Management Group Co., Ltd ","QY_REGION_NAME":" Fujian Province - Fuzhou City ","QY_ORG_CODE":"913501001544135960","COLLECT_TIME":1477106407000,"RN":37,"QY_ID":"616160676166636361616466606865656264","QY_SRC_TYPE":"0","OLD_CODE":"154413596"},{
    "QY_FR_NAME":" Yang gaobao ","QY_REGION":"421200","QY_NAME":" Jiayu County Jiatai Construction Supervision Co., Ltd ","QY_REGION_NAME":" Hubei province - Xianning City ","QY_ORG_CODE":"91421221737145124M","COLLECT_TIME":1477106486000,"RN":38,"QY_ID":"616160676166636361616466606865656267","QY_SRC_TYPE":"0","OLD_CODE":"737145124"},{
    "QY_FR_NAME":" Jiang Yabin ","QY_REGION":"410800","QY_NAME":" Henan synergy Engineering Supervision Co., Ltd ","QY_REGION_NAME":" Henan province - Jiaozuo City ","QY_ORG_CODE":"91410800555731456W","COLLECT_TIME":1477106405000,"RN":39,"QY_ID":"616160676166636361616466606865656266","QY_SRC_TYPE":"0","OLD_CODE":"555731456"},{
    "QY_FR_NAME":" Huang Xiangjun ","QY_REGION":"450100","QY_NAME":" Guangxi Nanning Jianke Engineering Supervision Co., Ltd 
","QY_REGION_NAME":" Guangxi Zhuang Autonomous Region - Nanning city ","QY_ORG_CODE":"914501031983159639","COLLECT_TIME":1477106406000,"RN":40,"QY_ID":"616160676166636361616466606865656269","QY_SRC_TYPE":"0","OLD_CODE":"198315963"},{"QY_FR_NAME":" Liao Yuan ","QY_REGION":"510100","QY_NAME":" Chengdu 
 Hailong Engineering Project Management Co., Ltd ","QY_REGION_NAME":" Sichuan Province - Chengdu ","QY_ORG_CODE":"91510100201986152W","COLLECT_TIME":1477106437000,"RN":41,"QY_ID":"616160676166636361616466606865656268","QY_SRC_TYPE":"0","OLD_CODE":"201986152"},{"QY_FR_NAME":" Zou Yun ","QY_REGION":"360500","QY_NAME":" Xinyu YONGGU Engineering Supervision Co., Ltd ","QY_REGION_NAME":" Jiangxi Province - Xinyu City ","QY_ORG_CODE":"91360500723935352F","COLLECT_TIME":1477109337000,"RN":42,"QY_ID":"616160676166636361616466606865656561","QY_SRC_TYPE":"0","OLD_CODE":"723935352"},{"QY_FR_NAME":" Wang Wenquan ","QY_REGION":"371300","QY_NAME":" Shandong zhenghang Engineering Consulting Co., Ltd ","QY_REGION_NAME":" Shandong Province - Linyi City ","QY_ORG_CODE":"91371302092199458A","COLLECT_TIME":1477106426000,"RN":43,"QY_ID":"616160676166636361616466606865656560","QY_SRC_TYPE":"0","OLD_CODE":"092199458"},{"QY_FR_NAME":" Zhang Hongjian ","QY_REGION":"530100","QY_NAME":" Yunnan jinji'an construction consulting and Supervision Co., Ltd ","QY_REGION_NAME":" Yunnan Province - Kunming ","QY_ORG_CODE":"91530000719414685K","COLLECT_TIME":1479833531000,"RN":44,"QY_ID":"616160676166636361616466606865656567","QY_SRC_TYPE":"0","OLD_CODE":"719414685"},{"QY_FR_NAME":" Zhang Zhiming ","QY_REGION":"610700","QY_NAME":" Hanzhong Zhicheng Construction Engineering Supervision Co., Ltd ","QY_REGION_NAME":" Shaanxi Province - Hanzhong City ","QY_ORG_CODE":"91610725667992132H","COLLECT_TIME":1477106417000,"RN":45,"QY_ID":"616160676166636361616466606865656568","QY_SRC_TYPE":"0","OLD_CODE":"667992132"}],"pageNum":2,"pageSize":15,"total":450},"message":"","success":true}

Come here , The crawling data section is all over .

About Python Technology reserve

Learn from good examples Python Whether it's employment or sideline, it's good to make money , But learn to Python Still have a learning plan . Finally, let's share a complete set of Python Learning materials , For those who want to learn Python Let's have a little help ！

One 、Python Learning routes in all directions

Python All directions are Python Sort out the common technical points , Form a summary of knowledge points in various fields , The use of it is , You can find the corresponding learning resources according to the above knowledge points , Make sure you learn more comprehensively .

Two 、 Learning software

If a worker wants to do a good job, he must sharpen his tools first . Study Python Common development software is here , It saves you a lot of time .

3、 ... and 、 Getting started video

When we were watching videos to learn , You can't just move your eyes and brain without hands , A more scientific way to learn is to use them after understanding , At this time, the hand training program is very suitable .

Four 、 Practical cases

Optical theory is useless , Learn to knock together , Do it , Can you apply what you have learned to practice , At this time, we can make some practical cases to learn .

5、 ... and 、 Interview information

We learn Python Must be to find a well paid job , The following interview questions are from Ali 、 tencent 、 The latest interview materials of big Internet companies such as byte , And the leader Ali gave an authoritative answer , After brushing this set of interview materials, I believe everyone can find a satisfactory job .

This full version of Python A full set of learning materials has been uploaded CSDN, Friends can scan the bottom of wechat if necessary CSDN The official two-dimensional code is free 【 Guarantee 100% free 】