当前位置:网站首页>How are the open source Netease cloud music API projects implemented?
How are the open source Netease cloud music API projects implemented?
2022-07-06 07:05:00 【Xiaolin on the corner】
Last article You deserve this high-value open-source third-party Netease cloud music player This paper introduces an open source third-party Netease cloud music player , In this article, let's have a detailed understanding of Netease cloud music api
project NeteaseCloudMusicApi Implementation principle of .
NeteaseCloudMusicApi Use Node.js
Development , There are two main frameworks and libraries , One Web Application development framework Express, A request Library Axios, You should be familiar with these two, but I don't want to introduce them more .
establish express application
The entry file of the project is /app.js
:
async function start() {
require('./server').serveNcmApi({
checkVersion: true,
})
}
start()
Called /server.js
Of documents serveNcmApi
Method , Let's go to this file ,serveNcmApi
The simplified method is as follows :
async function serveNcmApi(options) {
const port = Number(options.port || process.env.PORT || '3000')
const host = options.host || process.env.HOST || ''
const app = await consturctServer(options.moduleDefs)
const appExt = app
appExt.server = app.listen(port, host, () => {
console.log(`server running @ http://${
host ? host : 'localhost'}:${
port}`)
})
return appExt
}
It mainly starts listening to the specified port , So the main logic of creating applications is consturctServer
Method :
async function consturctServer(moduleDefs) {
// Create an application
const app = express()
// Set to true, The client's IP The address is understood as X-Forwarded-* The leftmost entry in the header
app.set('trust proxy', true)
/** * To configure CORS & Pre inspection request */
app.use((req, res, next) => {
if (req.path !== '/' && !req.path.includes('.')) {
res.set({
'Access-Control-Allow-Credentials': true, // Cross domain , Allow the client to carry authentication information , such as cookie, meanwhile , The front end also needs to be set when sending requests withCredentials: true
'Access-Control-Allow-Origin': req.headers.origin || '*', // Domain names that allow cross domain requests , Set to * On behalf of allowing all domain names
'Access-Control-Allow-Headers': 'X-Requested-With,Content-Type', // Used for pre inspection request (options) List the allowed custom headers on the server , If the request sent by the front end contains a custom request header , And the header is not included in Access-Control-Allow-Headers in , Then the request cannot be successfully initiated
'Access-Control-Allow-Methods': 'PUT,POST,GET,DELETE,OPTIONS', // It is ideal to set the request method allowed by cross domain requests
'Content-Type': 'application/json; charset=utf-8', // Set the type and code of response data
})
}
// OPTIONS Request for pre inspection , Complex requests will send a pre check request before sending the real request , Get server support for Access-Control-Allow-xxx Related information , Determine whether it is necessary to send a real request later , Return status code 204 On behalf of the successful request , But there's no content
req.method === 'OPTIONS' ? res.status(204).end() : next()
})
// ...
}
First, I created a Express
application , Then set it as a trust agent , stay Express
get ip
Usually by req.ip
and req.ips
,trust proxy
The default value is false
, In this case req.ips
Value is empty , When set to true
when ,req.ip
The value of will start from the request header X-Forwarded-For
Take the leftmost value ,req.ips
Will contain X-Forwarded-For
All of the head ip
Address .
X-Forwarded-For
The format of the header is as follows :
X-Forwarded-For: client1, proxy1, proxy2
Value through a comma + Space
The multiple ip
Address discrimination , The leftmost client1
Is the original client ip
Address , Every time the proxy server successfully receives a request , Just put Source of the request ip
Address Add to the right .
Take the example above , This request passed through two proxy servers :proxy1
and proxy2
. Request by the client1
issue , here XFF
It's empty. , here we are proxy1
when ,proxy1
hold client1
Add to XFF
in , Then the request is sent to proxy2
, adopt proxy2
When ,proxy1
Be added to XFF
in , Then the request is sent to the final server , After arrival proxy2
Be added to XFF
in .
But it's very easy to forge this field , So when contemporary reason is not believable , This field is not necessarily reliable , But normally XFF
Last of ip
The address must be the last proxy server ip
Address , This will be more reliable .
Then set the cross domain response header , The setting here is the key to allowing websites with different domain names to request successfully .
continue :
async function consturctServer(moduleDefs) {
// ...
/** * analysis Cookie */
app.use((req, _, next) => {
req.cookies = {
}
//;(req.headers.cookie || '').split(/\s*;\s*/).forEach((pair) => { // Polynomial regular expression //
// Read from the request header cookie,cookie The format is :name=value;name2=value2..., So first according to ; Cut into arrays
;(req.headers.cookie || '').split(/;\s+|(?<!\s)\s+$/g).forEach((pair) => {
let crack = pair.indexOf('=')
// Skip directly without value
if (crack < 1 || crack == pair.length - 1) return
// take cookie Save to cookies On the object
req.cookies[decode(pair.slice(0, crack)).trim()] = decode(
pair.slice(crack + 1),
).trim()
})
next()
})
/** * Request body parsing and file upload processing */
app.use(express.json())
app.use(express.urlencoded({
extended: false }))
app.use(fileUpload())
/** * take public The files in the directory are provided as static files */
app.use(express.static(path.join(__dirname, 'public')))
/** * Cache request , Within two minutes, the same request will read data from the cache , Will not send requests to Netease cloud music server */
app.use(cache('2 minutes', (_, res) => res.statusCode === 200))
// ...
}
Next, I registered some middleware , Used to resolve cookie
、 Processing request body, etc , In addition, interface caching is also done , Prevent too frequent requests for Netease cloud music servers from being blocked .
continue :
async function consturctServer(moduleDefs) {
// ...
/** * Special routing */
const special = {
'daily_signin.js': '/daily_signin',
'fm_trash.js': '/fm_trash',
'personal_fm.js': '/personal_fm',
}
/** * load /module All modules in the directory , Each module corresponds to an interface */
const moduleDefinitions =
moduleDefs ||
(await getModulesDefinitions(path.join(__dirname, 'module'), special))
// ...
}
Next, load /module
All modules in the directory :
Each module represents a request for Netease cloud music interface , For example, get album details album_detail.js
:
Module loading method getModulesDefinitions
as follows :
async function getModulesDefinitions(
modulesPath, specificRoute, doRequire = true,
) {
const files = await fs.promises.readdir(modulesPath)
const parseRoute = (fileName) =>
specificRoute && fileName in specificRoute
? specificRoute[fileName]
: `/${
fileName.replace(/\.js$/i, '').replace(/_/g, '/')}`
// Traverse all the files in the directory
const modules = files
.reverse()
.filter((file) => file.endsWith('.js'))// Filter out js file
.map((file) => {
const identifier = file.split('.').shift()// Module identifier
const route = parseRoute(file)// The route corresponding to the module
const modulePath = path.join(modulesPath, file)// The module path
const module = doRequire ? require(modulePath) : modulePath// Load module
return {
identifier, route, module }
})
return modules
}
As just now album_detail.js
Module as an example , The data returned is as follows :
{
identifier: 'album_detail',
route: '/album/detail',
module: () => {
/* Module content */}
}
The next step is to register routes :
async function consturctServer(moduleDefs) {
// ...
for (const moduleDef of moduleDefinitions) {
// Registered routing
app.use(moduleDef.route, async (req, res) => {
// cookie You can also query parameters 、 The request body is uploaded
;[req.query, req.body].forEach((item) => {
if (typeof item.cookie === 'string') {
// take cookie The string is converted to json type
item.cookie = cookieToJson(decode(item.cookie))
}
})
// hold cookie、 Query parameters 、 Request header 、 Documents are all integrated , Pass it to each module as a parameter
let query = Object.assign(
{
},
{
cookie: req.cookies },
req.query,
req.body,
req.files,
)
try {
// Execute the module method , That is, initiate a request for Netease cloud music interface
const moduleResponse = await moduleDef.module(query, (...params) => {
// Parameter injection client IP
const obj = [...params]
// Handle ip, In order to achieve IPv4-IPv6 Interworking ,IPv4 The address will be added before ::ffff:
let ip = req.ip
if (ip.substr(0, 7) == '::ffff:') {
ip = ip.substr(7)
}
obj[3] = {
...obj[3],
ip,
}
return request(...obj)
})
// After the request is successful , Get... In the response cookie, And through Set-Cookie Response header to this cookie Set to the front-end browser
const cookies = moduleResponse.cookie
if (Array.isArray(cookies) && cookies.length > 0) {
if (req.protocol === 'https') {
// Remove cross domain requests cookie Of SameSite Limit , This attribute is used to restrict third parties Cookie, So as to reduce the safety risk
res.append(
'Set-Cookie',
cookies.map((cookie) => {
return cookie + '; SameSite=None; Secure'
}),
)
} else {
res.append('Set-Cookie', cookies)
}
}
// Reply to the request
res.status(moduleResponse.status).send(moduleResponse.body)
} catch (moduleResponse) {
// Request failure handling
// No responder , return 404
if (!moduleResponse.body) {
res.status(404).send({
code: 404,
data: null,
msg: 'Not Found',
})
return
}
// 301 The delegate called the interface that needs to be logged in , But I didn't log in
if (moduleResponse.body.code == '301')
moduleResponse.body.msg = ' Need to log in '
res.append('Set-Cookie', moduleResponse.cookie)
res.status(moduleResponse.status).send(moduleResponse.body)
}
})
}
return app
}
The logic is clear , Register each module as a route , After receiving the corresponding request , take cookie
、 Query parameters 、 The request body is passed to the corresponding module , Then request the interface of Netease cloud music , If the request succeeds , Then deal with the return of Netease cloud music interface cookie
, Finally, return all the data to the front end , If the interface fails , Then carry out the corresponding processing .
From the query parameters and request body of the request cookie
It may not be easy to understand , because cookie
It is usually brought from the request body , This should be done mainly to support Node.js
Call inside :
After the request is successful , If the returned data exists cookie
, Then some processing will be carried out , First of all, if it is https
Request , Then it will be set SameSite=None; Secure
,SameSite
yes Cookie
A property in , To limit third parties Cookie
, So as to reduce the safety risk .Chrome 51
Start adding this attribute , To prevent CSRF
Attacks and user tracking , There are three optional values :strict/lax/none
, The default is lax
, For example, when the domain name is https://123.com
In the page of https://456.com
Interface of domain name , By default, in addition to navigating to 123
Of get
Except for requests , Other requests will not be carried 123
Domain name cookie
, If set to strict
More strictly , Not at all cookie
, So this project is to facilitate cross domain calls , Set to none
, No restrictions , Set to none
At the same time, you need to set Secure
attribute .
Finally through Set-Cookie
The response header will cookie
Write to the browser on the front end .
Send a request
Next, let's take a look at the... Used to send the request request
Method , This method is /util/request.js
file , First, some modules are introduced :
const encrypt = require('./crypto')
const axios = require('axios')
const PacProxyAgent = require('pac-proxy-agent')
const http = require('http')
const https = require('https')
const tunnel = require('tunnel')
const {
URLSearchParams, URL } = require('url')
const config = require('../util/config.json')
// ...
Then is the specific method of sending the request createRequest
, It's a long way to go , Let's take a slow look at :
const createRequest = (method, url, data = {
}, options) => {
return new Promise((resolve, reject) => {
let headers = {
'User-Agent': chooseUserAgent(options.ua) }
// ...
})
}
The function returns one Promise
, First, define a request header object , And add the User-Agent
head , This header will save the browser type 、 Version number 、 Rendering engine , And the operating system 、 edition 、CPU Type information , The standard format is :
Browser logo ( Operating system identification ; Encryption level identification ; Browser language ) Rendering engine logo Version information
Needless to say , This header is obviously forged to cheat the server , Let it think that the request is from the browser , Instead of also coming from the server .
By default, there are several dead User-Agent
The head is selected randomly :
const chooseUserAgent = (ua = false) => {
const userAgentList = {
mobile: [
'Mozilla/5.0 (iPhone; CPU iPhone OS 13_5_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.1.1 Mobile/15E148 Safari/604.1',
'Mozilla/5.0 (Linux; Android 9; PCT-AL10) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.64 HuaweiBrowser/10.0.3.311 Mobile Safari/537.36',
// ...
],
pc: [
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:80.0) Gecko/20100101 Firefox/80.0',
'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:80.0) Gecko/20100101 Firefox/80.0',
// ...
],
}
let realUserAgentList =
userAgentList[ua] || userAgentList.mobile.concat(userAgentList.pc)
return ['mobile', 'pc', false].indexOf(ua) > -1
? realUserAgentList[Math.floor(Math.random() * realUserAgentList.length)]
: ua
}
Continue to look at :
const createRequest = (method, url, data = {
}, options) => {
return new Promise((resolve, reject) => {
// ...
// If it is post request , Modify encoding format
if (method.toUpperCase() === 'POST')
headers['Content-Type'] = 'application/x-www-form-urlencoded'
// forge Referer head
if (url.includes('music.163.com'))
headers['Referer'] = 'https://music.163.com'
// Set up ip Head
let ip = options.realIP || options.ip || ''
if (ip) {
headers['X-Real-IP'] = ip
headers['X-Forwarded-For'] = ip
}
// ...
})
}
Continue to set several header fields ,Axios
The default encoding format is json
, and POST
Requests generally use application/x-www-form-urlencoded
Coding format .
Referer
The header represents the page where the request is sent url
, For example https://123.com
Intra page call https://456.com
Interface ,Referer
The header will be set to https://123.com
, This head is usually used for anti-theft chains . So this header is also forged to deceive the server that the request comes from their own page .
Next, we set two ip
Head ,realIP
The front-end manual transmission is required :
continue :
const createRequest = (method, url, data = {
}, options) => {
return new Promise((resolve, reject) => {
// ...
// Set up cookie
if (typeof options.cookie === 'object') {
if (!options.cookie.MUSIC_U) {
// tourists
if (!options.cookie.MUSIC_A) {
options.cookie.MUSIC_A = config.anonymous_token
}
}
headers['Cookie'] = Object.keys(options.cookie)
.map(
(key) =>
encodeURIComponent(key) +
'=' +
encodeURIComponent(options.cookie[key]),
)
.join('; ')
} else if (options.cookie) {
headers['Cookie'] = options.cookie
}
// ...
})
}
Next set up cookie
, There are two types , One is object type , This situation cookie
It usually comes from query parameters or request body , The other is string , This is what you ask the head to bring in under normal circumstances .MUSIC_U
It should be after login cookie
了 ,MUSIC_A
It should be a token
, Calling some interfaces without login may report an error , So there will be a tourist token
:
continue :
const createRequest = (method, url, data = {
}, options) => {
return new Promise((resolve, reject) => {
// ...
if (options.crypto === 'weapi') {
let csrfToken = (headers['Cookie'] || '').match(/_csrf=([^(;|$)]+)/)
data.csrf_token = csrfToken ? csrfToken[1] : ''
data = encrypt.weapi(data)
url = url.replace(/\w*api/, 'weapi')
} else if (options.crypto === 'linuxapi') {
data = encrypt.linuxapi({
method: method,
url: url.replace(/\w*api/, 'api'),
params: data,
})
headers['User-Agent'] =
'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36'
url = 'https://music.163.com/api/linux/forward'
} else if (options.crypto === 'eapi') {
const cookie = options.cookie || {
}
const csrfToken = cookie['__csrf'] || ''
const header = {
osver: cookie.osver, // System version
deviceId: cookie.deviceId, //encrypt.base64.encode(imei + '\t02:00:00:00:00:00\t5106025eb79a5247\t70ffbaac7')
appver: cookie.appver || '8.7.01', // app edition
versioncode: cookie.versioncode || '140', // Version number
mobilename: cookie.mobilename, // equipment model
buildver: cookie.buildver || Date.now().toString().substr(0, 10),
resolution: cookie.resolution || '1920x1080', // Device resolution
__csrf: csrfToken,
os: cookie.os || 'android',
channel: cookie.channel,
requestId: `${
Date.now()}_${
Math.floor(Math.random() * 1000) .toString() .padStart(4, '0')}`,
}
if (cookie.MUSIC_U) header['MUSIC_U'] = cookie.MUSIC_U
if (cookie.MUSIC_A) header['MUSIC_A'] = cookie.MUSIC_A
headers['Cookie'] = Object.keys(header)
.map(
(key) =>
encodeURIComponent(key) + '=' + encodeURIComponent(header[key]),
)
.join('; ')
data.header = header
data = encrypt.eapi(options.url, data)
url = url.replace(/\w*api/, 'eapi')
}
// ...
})
}
This code will be difficult to understand , The author did not understand , Anyway, roughly speaking, this project uses four types of Netease cloud music interfaces :weapi
、linuxapi
、eapi
、api
, such as :
https://music.163.com/weapi/vipmall/albumproduct/detail
https://music.163.com/eapi/activate/initProfile
https://music.163.com/api/album/detail/dynamic
Request parameters for each type of interface 、 Encryption methods are different , So it needs to be handled separately :
such as weapi
:
let csrfToken = (headers['Cookie'] || '').match(/_csrf=([^(;|$)]+)/)
data.csrf_token = csrfToken ? csrfToken[1] : ''
data = encrypt.weapi(data)
url = url.replace(/\w*api/, 'weapi')
take cookie
Medium _csrf
The value is taken out and added to the request data , Then encrypt the data :
const weapi = (object) => {
const text = JSON.stringify(object)
const secretKey = crypto
.randomBytes(16)
.map((n) => base62.charAt(n % 62).charCodeAt())
return {
params: aesEncrypt(
Buffer.from(
aesEncrypt(Buffer.from(text), 'cbc', presetKey, iv).toString('base64'),
),
'cbc',
secretKey,
iv,
).toString('base64'),
encSecKey: rsaEncrypt(secretKey.reverse(), publicKey).toString('hex'),
}
}
See other encryption algorithms :crypto.js.
As for how to know these , Or Netease cloud music insiders ( Almost impossible ), Or reverse it , For example, the interface of the web version , Open console , Send a request , Find the location in the source code , Breaking point , View the request data structure , Read the compressed or confused source code and try it slowly , All in all , Pay tribute to these big guys .
continue :
const createRequest = (method, url, data = {
}, options) => {
return new Promise((resolve, reject) => {
// ...
// Data structure of response
const answer = {
status: 500, body: {
}, cookie: [] }
// Request configuration
let settings = {
method: method,
url: url,
headers: headers,
data: new URLSearchParams(data).toString(),
httpAgent: new http.Agent({
keepAlive: true }),
httpsAgent: new https.Agent({
keepAlive: true }),
}
if (options.crypto === 'eapi') settings.encoding = null
// Configure agent
if (options.proxy) {
if (options.proxy.indexOf('pac') > -1) {
settings.httpAgent = new PacProxyAgent(options.proxy)
settings.httpsAgent = new PacProxyAgent(options.proxy)
} else {
const purl = new URL(options.proxy)
if (purl.hostname) {
const agent = tunnel.httpsOverHttp({
proxy: {
host: purl.hostname,
port: purl.port || 80,
},
})
settings.httpsAgent = agent
settings.httpAgent = agent
settings.proxy = false
} else {
console.error(' Invalid proxy configuration , Don't use agents ')
}
}
} else {
settings.proxy = false
}
if (options.crypto === 'eapi') {
settings = {
...settings,
responseType: 'arraybuffer',
}
}
// ...
})
}
Here we mainly define the data structure of the response 、 Defines the requested configuration data , And aiming at eapi
Did some special treatment , The most important thing is the related configuration of the agent .
Agent
yes Node.js
Of HTTP
A class in the module , Responsible for managing the http
Persistence and reuse of client connections . It maintains a queue of pending requests for a given host and port , Reuse a single socket connection for each request , Until the queue is empty , The socket is either destroyed , Or put it in the pool , In the pool, it will be used again to request to the same host and port , In short, it saves every launch http
The time required to recreate the socket when requesting , Increase of efficiency .
pac
Refers to agent automatic configuration , In fact, it contains a javascript
Function text file , This function will decide whether to connect directly or through a proxy , It's a little easier than writing an agent directly , Of course, you need to configure options.proxy
Is the remote address of this file , The format is :'pac+【pac File address 】+'
.pac-proxy-agent
The module will provide a http.Agent
Realization , It will be based on the specified PAC
The proxy file determines which HTTP
、HTTPS
or SOCKS
agent , Or direct connection .
As for why to use tunnel
modular , The author searched it and still didn't understand , It may be to solve http
The interface of the protocol requests Netease cloud music https
The problem of protocol interface failure ? Friends who know can explain in the comment area ~
Last :
const createRequest = (method, url, data = {
}, options) => {
return new Promise((resolve, reject) => {
// ...
axios(settings)
.then((res) => {
const body = res.data
// Will respond to set-cookie In the header cookie Take out , Save directly to the response object
answer.cookie = (res.headers['set-cookie'] || []).map((x) =>
x.replace(/\s*Domain=[^(;|$)]+;*/, ''),// Remove domain name restrictions
)
try {
// eapi The returned data is also encrypted , Need to decrypt
if (options.crypto === 'eapi') {
answer.body = JSON.parse(encrypt.decrypt(body).toString())
} else {
answer.body = body
}
answer.status = answer.body.code || res.status
// Unify these status codes as 200, Both represent success
if (
[201, 302, 400, 502, 800, 801, 802, 803].indexOf(answer.body.code) > -1
) {
// Special status code
answer.status = 200
}
} catch (e) {
try {
answer.body = JSON.parse(body.toString())
} catch (err) {
answer.body = body
}
answer.status = res.status
}
answer.status =
100 < answer.status && answer.status < 600 ? answer.status : 400
// Status code 200 On behalf of success , Everything else represents failure
if (answer.status === 200) resolve(answer)
else reject(answer)
})
.catch((err) => {
answer.status = 502
answer.body = {
code: 502, msg: err }
reject(answer)
})
})
}
Commands are less used now Axios
Sent the request , Handled the response cookie
, Save to the response object , Convenient for subsequent use , In addition, some status codes are processed , You can see try-catch
Use more , As for why , It's estimated that we should try more to know what went wrong , If you are interested, you can try it yourself .
summary
This article through the source point of view to understand NeteaseCloudMusicApi Implementation principle of the project , You can see that the whole process is relatively simple . It's nothing more than a request broker , The difficulty lies in finding these interfaces , And reverse analyze the parameters of each interface , Encryption method , Decryption method . Finally, let me remind you , This project is only for learning , Please do not engage in commercial activities or conduct acts of copyright destruction ~
边栏推荐
- Explain in detail the functions and underlying implementation logic of the groups sets statement in SQL
- 18. Multi level page table and fast table
- [advanced software testing step 1] basic knowledge of automated testing
- Interface automation test framework: pytest+allure+excel
- Librosa audio processing tutorial
- 漏了监控:Zabbix对Eureka instance状态监控
- Wechat official account infinite callback authorization system source code, launched in the whole network
- BUU的MISC(不定时更新)
- Bio model realizes multi person chat
- leetcode6109. 知道秘密的人数(中等,周赛)
猜你喜欢
[server data recovery] case of offline data recovery of two hard disks of IBM server RAID5
首发织梦百度推送插件全自动收录优化seo收录模块
Top test sharing: if you want to change careers, you must consider these issues clearly!
leetcode6109. 知道秘密的人数(中等,周赛)
Depth residual network
接口自动化测试框架:Pytest+Allure+Excel
ROS学习_基础
Development of entity developer database application
Chapter 7 - thread pool of shared model
[brush questions] how can we correctly meet the interview?
随机推荐
Explain in detail the functions and underlying implementation logic of the groups sets statement in SQL
[advanced software testing step 1] basic knowledge of automated testing
这个高颜值的开源第三方网易云音乐播放器你值得拥有
Basic commands of MySQL
leetcode6109. 知道秘密的人数(中等,周赛)
leetcode1020. 飞地的数量(中等)
指尖上的 NFT|在 G2 上评价 Ambire,有机会获得限量版收藏品
【Hot100】739. Daily temperature
中青看点阅读新闻
Embed UE4 program into QT interface display
《从0到1:CTFer成长之路》书籍配套题目(周更)
Leetcode59. spiral matrix II (medium)
编译,连接 -- 笔记 -2
微信脑力比拼答题小程序_支持流量主带最新题库文件
Call, apply, bind rewrite, easy to understand with comments
SEO学习的最好方式:搜索引擎
Pallet management in SAP SD delivery process
leetcode1020. Number of enclaves (medium)
ROS学习_基础
作者已死?AI正用藝術征服人類