Imook nodejs personal learning notes (log)

summary ： Use fs Of api Read and write files , Keep the log in the text , Set up crontab Split logs regularly .

7.1 journal

• Access log access log In the system （ The most important log in the system ）
• Custom log Custom events Custom errors, etc

nodejs File operation nodejs stream

• The development and use of logs
• Split the log 、 Log content analysis

Writing a file is asynchronous , There is no need to write immediately . use redis Too wasteful .mysql Table structure , Not suitable for , You don't have to deploy , And waste . The cost of documentation is relatively low .

7.2 nodejs File operations

const fs = require('fs');
const path = require('path');

const fileName = path.resolve(__dirname, 'data.log');

//  Read file contents 
fs.readFile(fileName, (err, data) => {
    if (err) {
        console.log(err);
        return
    }
    // data  It's binary , Need to convert to string 
    console.log(data.toString());
})

//  write file 
const content = " Ms. Hu is still a fairy ";
const opt = {
    flag: 'a'
}
fs.writeFile(fileName, content, opt, (err) => {
    if (err) {
        console.log(err);
        return;
    }
})

//  Determine if the file exists 
fs.exists(fileName, (exists) => {
    console.log("exists:", exists);
})

7.3 stream Introduce

Performance bottleneck :IO、 The Internet IO、 file IO .
Compared with CPU Computing and memory reading and writing ,IO The main characteristics of slow .
How to be in Limited hardware resources Next , Improve IO Operating efficiency .

The concept of flow ：
Loading while watching . Save a lot of hardware resources .

7.45 Read and write log

read

const fs = require('fs')
const path = require('path')
const readline = require('readline')
//  file name 
const fileName = path.join(__dirname, '../', '../', 'logs', 'access.log');

// readStream  important 
const readStream = fs.createReadStream(fileName)

//  establish readline object   important 
const rl = readline.createInterface({
    input: readStream
})

let chromeNum = 0
let sum = 0

//  Read line by line 
rl.on('line', (lineData) => {
    if (!lineData) {
        return
    }
    //  Record the total number of rows 
    sum++

    const arr = lineData.split(' -- ')
    if (arr[2] && arr[2].indexOf('Chrome') > 0) {
        //  Add up  chrome  The number of 
        chromeNum++
    }
})

//  Listen read complete 
rl.on('close', () => {
    console.log('chrome  Proportion ：' + chromeNum / sum)
})

Write

//  Write the log 
function writeLog(writeStream, log) {
    writeStream.write(log + '\n')  //  Key code 
}

//  Generate  write Stream
function createWriteStream(fileName) {
    const fullFileName = path.join(__dirname, '../', '../', 'logs', fileName)
    const writeStream = fs.createWriteStream(fullFileName, {
        flags: 'a'
    })
    return writeStream
}

//  Write access logs 
const accessWriteStream = createWriteStream('access.log')
function access(log) {
    writeLog(accessWriteStream, log)
}

module.exports = {
    access
}

7.7 Log splitting , Log analysis

In order to be more efficient , use shell Script split log .crontab, Scheduled tasks .

#!/bin/sh
cd /Users/huziyin/Documents/Self-study/imook-nodejs/blog-1/logs
cp access.log $(date +%Y-%m-%d).access.log
echo "">access.log