当前位置:网站首页>nodejs爬虫
nodejs爬虫
2022-07-06 09:25:00 【星时代曹波涛】
nodejs也可以玩爬虫
需要通过npm下在两个模块。
npm init -y
npm install -S axios cheerio
创建index.js文件
const cheerio = require("cheerio")
const axios = require("axios")
const fs = require("fs")
if (!fs.existsSync("download")) {
fs.mkdirSync("download", 777)
}
console.log("开始爬取")
axios.get("http://www.scsoul.top/list/1/20").then(r => {
const $ = cheerio.load(r.data)
$("img").each((index, item) => {
if (index != 0) {
var url = $(item).attr("src");
console.log("开始下载" + url)
axios.get(url, {
responseType: 'stream' }).then(resp => {
let writer = fs.createWriteStream("./download/" + index + ".png");
resp.data.pipe(writer);
console.log(index+".png下载完成")
})
}
})
})
cmd输入
node index.js
就可以看到文件已经被下载下来了
边栏推荐
- FSM and I2C experiment report
- Crawler series (9): item+pipeline data storage
- Eslint--- error: newline required at end of file but not found (EOL last) solution
- Your wechat nickname may be betraying you
- Brief introduction to libevent
- 编程到底难在哪里?
- Research Report on market supply and demand and strategy of Chinese graphic screen printing equipment industry
- 0 - 1 problème de sac à dos (1)
- Lab 8 文件系统
- ucorelab4
猜你喜欢
毕业才知道IT专业大学生毕业前必做的1010件事
Scoring system based on 485 bus
TCP的三次握手与四次挥手
ucore lab 2
LeetCode#62. Different paths
ucore lab 6
51 lines of code, self-made TX to MySQL software!
Learning record: understand systick system timer and write delay function
STM32 learning record: play with keys to control buzzer and led
Flex --- detailed explanation of flex layout attributes
随机推荐
Cost accounting [20]
Cost accounting [13]
ucore lab 2
Printing quality inspection and verification system Industry Research Report - market status analysis and development prospect forecast
Research Report on pharmaceutical R & D outsourcing service industry - market status analysis and development prospect forecast
JS --- BOM details of JS (V)
Perinatal Software Industry Research Report - market status analysis and development prospect forecast
China chart recorder market trend report, technology dynamic innovation and market forecast
Research Report on market supply and demand and strategy of China's land incineration plant industry
VS2019初步使用
Flex --- detailed explanation of flex layout attributes
Word macro operation: convert the automatic number in the document into editable text type
区间和------离散化
Cost accounting [13]
STM32 how to use stlink download program: light LED running light (Library version)
Learning record: use stm32f1 watchdog
Leetcode notes - dynamic planning -day6
Research Report on market supply and demand and strategy of geosynthetics industry in China
C语言学习笔记
Intensive learning notes: Sutton book Chapter III exercise explanation (ex17~ex29)