Viewing splitchunks code segmentation from MPX resource construction optimization

High quality resource sharing

Learning route guidance （ Click unlock ）	Knowledge orientation	Crowd positioning
🧡 Python Actual wechat ordering applet 🧡	Progressive class	This course is python flask+ Perfect combination of wechat applet , From the deployment of Tencent to the launch of the project , Create a full stack ordering system .
Python Quantitative trading practice	beginner	Take you hand in hand to create an easy to expand 、 More secure 、 More efficient quantitative trading system

background

MPX It is an enhanced applet cross end framework produced by Didi , Its core is the enhancement of native applet functions . Specific use is not the scope of this article , To learn more, you can go to the official website to learn more .

Back to the point , Use MPX I have been developing small programs for some time , The framework has its own set of build output strategies for shared resources between different packages , There is such a paragraph on its official website ：

Summarize the key two points ：

• pure js resources ： The main package reference outputs the main package , Or the shared data among sub packages is also output to the main package
• Not js resources , Include wxml、 style 、 Pictures, etc ： The main package reference outputs the main package , Shared among subcontractors will be output to their respective subcontractors

I'm curious MPX How does the interior achieve the above effect , In especial js resources , So I read it @mpxjs/[email protected] Uncover the details of its implementation .

mpx How to achieve

Let's start with a brief introduction MPX How to integrate the discrete file structure of small programs , It's based on webpack Packaged and built , The user is in Webpack Only one entry file needs to be configured in the configuration app.mpx, It will be based on Dependency analysis and Dynamic addition entry To integrate discrete files of small programs ,,loader Can parse json In the configuration file pages Domain and usingComponents Domain Path declared in , By dynamically adding entry To add these files to Webpack In the building system of , And recursively execute this process , Until all that is used throughout the project .mpx All the files are added .

The key is coming. ,MPX Before the output , Its with the aid of the webpack Of SplitChunksPlugin The ability to extract reusable modules to an external bundle in , Ensure that the resulting package does not contain duplicate modules .

js Output of the resource module

@mpxjs/webpack-plugin The plug-in is MPX be based on webapck The core of building , It will be in webpack All modules are built finishMoudles Hook to implement the build output strategy , Mainly configuration SplitChunks Of cacheGroup, follow-up webpack The code optimization phase will be based on SplitChunks Configuration to output code .

apply(compiler) {

    ...
    //  Get webpack Default configuration object splitChunks
    let splitChunksOptions = compiler.options.optimization.splitChunks
    //  Delete splitChunks After the configuration ,webpack There will be no instantiation inside SplitChunkPlugin
    delete compiler.options.optimization.splitChunks
    // SplitChunkPlugin The instantiation of is by mpx To take over , In this way, you can get the actual example and follow it up options Amendment 
    let splitChunksPlugin = new SplitChunksPlugin(splitChunksOptions)
    splitChunksPlugin.apply(compiler)
    ...
    
    compilation.hooks.finishModules.tap('MpxWebpackPlugin', (modules) => {
            //  Automatically follow up subcontract configuration changes splitChunksPlugin To configure 
            if (splitChunksPlugin) {
              let needInit = false
              Object.keys(mpx.componentsMap).forEach((packageName) => {
                if (!splitChunksOptions.cacheGroups.hasOwnProperty(packageName)) {
                  needInit = true
                  splitChunksOptions.cacheGroups[packageName] = getPackageCacheGroup(packageName)
                }
              })
              if (needInit) {
                splitChunksPlugin.options = SplitChunksPlugin.normalizeOptions(splitChunksOptions)
              }
           }
    })

You can see that when all modules are built , For different packageName To generate its corresponding cacheGroups, Mainly reflected in getPackageCacheGroup Method implementation . And then to get SplitChunksPlugin The handle to the instance , For its options Perform rewrite normalization .

function isChunkInPackage (chunkName, packageName) {
  return (new RegExp(`^${packageName}\\/`)).test(chunkName)
}

function getPackageCacheGroup (packageName) {
  if (packageName === 'main') {
    return {
      name: 'bundle',
      minChunks: 2,
      chunks: 'all'
    }
  } else {
    return {
      test: (module, chunks) => {
        return chunks.every((chunk) => {
          return isChunkInPackage(chunk.name, packageName)
        })
      },
      name: `${packageName}/bundle`,
      minChunks: 2,
      minSize: 1000,
      priority: 100,
      chunks: 'all'
    }
  }
}

getPackageCacheGroup A code splitting group is generated for each package of the applet , That is, generate the corresponding for each package cacheGroups.

For example, an applet project has a main package and A、B Two subcontractors , It generates cacheGroups The contents are as follows ：

{
  default: {
    automaticNamePrefix: '',
    reuseExistingChunk: true,
    minChunks: 2,
    priority: -20
  },
  vendors: {
    automaticNamePrefix: 'vendors',
    test: /[\\/]node\_modules[\\/]/,
    priority: -10
  },
  main: { name: 'bundle', minChunks: 2, chunks: 'all' },
  A: {
    test: [Function: test],
    name: 'A/bundle',
    minChunks: 2,
    minSize: 1000,
    priority: 100,
    chunks: 'all'
  },
  B: {
    test: [Function: test],
    name: 'B/bundle',
    minChunks: 2,
    minSize: 1000,
    priority: 100,
    chunks: 'all'
  }

Subcontract code split output bundle Is the highest priority （priority: 100）, Therefore, packaging in subcontracting will be given priority ; Otherwise, it will execute main Code packaging rules in , It handles the packaging of shared modules between all packages and the reused modules in the main package .

Let's take a look at the packaging rules for subcontracting and main packages .

1、 For modules in the subcontract :

{
  test: (module, chunks) => {
    //  All that depend on the current module chunks Whether they are all under the current subcontracting chunk
    return chunks.every((chunk) => {
      return isChunkInPackage(chunk.name, packageName)
    })
  },
  name: `${packageName}/bundle`,
  minChunks: 2,
  minSize: 1000,
  priority: 100,
  chunks: 'all'
}

The modules in the subcontract are extracted from the modules under the current subcontract bundle In file , Need to meet ：

• This module is not referenced by other packages , Including the main package and other subcontracts （test Functional logic ）
• At least under the subcontract 2 individual chunk quote （minChunks:2）
• After pulling away bundle The size shall meet at least about 1kb（minSize: 1000）

2、 For modules in the main package ：

{
  name: 'bundle',
  minChunks: 2,
  chunks: 'all'
}

The main package will be extracted bundle The conditions of the document ：

• The module is at least 2 individual chunk quote （minChunks:2）, This chunk Do not distinguish between... In the main subcontract chunk

3、 For modules shared between subcontractors ：

Modules shared between subcontractors , dissatisfaction SplitChunks Subcontracting exclusive rules set for each subcontracting , That is, the module is only referenced in the current subcontracting , Not referenced in other packages .

test: (module, chunks) => {
    return chunks.every((chunk) => {
      return isChunkInPackage(chunk.name, packageName)
    })
}

therefore , It will eventually arrive at main Of cacheGroup In the packaging rules of , That is, in the packaging rules of the main package .

such ,MPX By configuring SplitChunksOptions.cacheGroups In the main package js Shared by modules and subcontractors js Modules are output to the main package , Modules individually referenced by subcontracting are output to the current subcontracting .

Components and static resources

For components and static resources ,MPX stay webpack Built thisCompilation In the hook function, there will be compilation Mount a package on the __mpx__ object , Contains static resources 、 Component resources 、 Page resources and other attributes , It also contains static non js Output processing of resources, etc ：

compiler.hooks.thisCompilation.tap('MpxWebpackPlugin', (compilation, { normalModuleFactory }) => {
    ...
    if (!compilation.\_\_mpx\_\_) {
        mpx = compilation.\_\_mpx\_\_ = {
             ...
             componentsMap: {
                main: {}
              },
              //  Static resources ( picture , typeface , Stand alone style ) etc. , Record according to the package , Redundant storage , ditto 
              staticResourcesMap: {
                main: {}
              },
              ...
              //  The output rules for components and static resources are as follows ：
              // 1.  The resources referenced by the main package are output to the main package 
              // 2.  The resources referenced by subcontracting and the main package are output to the main package , Do not repeat output in current subcontracting 
              // 3.  Resources referenced by subcontracting and no other packages are output to the current subcontracting 
              // 4.  Resources referenced by subcontracting and other subcontracting , Repeat output to current subcontracting 
          getPackageInfo: ({ resource, outputPath, resourceType = 'components', warn }) => {
            let packageRoot = ''
            let packageName = 'main'
            const { resourcePath } = parseRequest(resource)
            const currentPackageRoot = mpx.currentPackageRoot
            const currentPackageName = currentPackageRoot || 'main'
            const resourceMap = mpx[`${resourceType}Map`]
            const isIndependent = mpx.independentSubpackagesMap[currentPackageRoot]
            //  All references in the main package use the resources in the main package , No additional output 
            if (!resourceMap.main[resourcePath] || isIndependent) {
              packageRoot = currentPackageRoot
              packageName = currentPackageName
              ...
            }
            resourceMap[packageName] = resourceMap[packageName] || {}
            const currentResourceMap = resourceMap[packageName]

            let alreadyOutputed = false
            if (outputPath) {
              outputPath = toPosix(path.join(packageRoot, outputPath))
              //  If output has been performed before , There is no need to repeat 
              if (currentResourceMap[resourcePath] === outputPath) {
                alreadyOutputed = true
              } else {
                currentResourceMap[resourcePath] = outputPath
              }
            } else {
              currentResourceMap[resourcePath] = true
            }

            return {
              packageName,
              packageRoot,
              outputPath,
              alreadyOutputed
            }
          },
          ...
        }
    }
}
 Fold  

such webpack Build compile non js Resource will be called compilation.__mpx__.getPackageInfo Method returns non js The output path of static resources , In this method, the output rules of the following resources are formulated ：

1. The resources referenced by the main package are output to the main package
2. The resources referenced by subcontracting and the main package are output to the main package , Do not repeat output in current subcontracting
3. Resources referenced by subcontracting and no other packages are output to the current subcontracting
4. Resources referenced by subcontracting and other subcontracting , Repeat output to current subcontracting

such ,mpx When dealing with static resources in a project , This method will be called to obtain the output path of static resources .

Here is a simple example to illustrate ,

For example, for the picture in the project, it will call @mpxjs/webpack-plugin Provided url-loader To deal with , And webpack Of url-loader similar , For pictures smaller than the specified size limit In the process of base64 Handle , Otherwise use file-loader To output pictures （ You need to call getPackageInfo Method to get the output path of the picture ）, Related codes ：

 let outputPath

  if (options.publicPath) { //  first loader Configured publicPath
    outputPath = url
    if (options.outputPathCDN) {
      if (typeof options.outputPathCDN === 'function') {
        outputPath = options.outputPathCDN(outputPath, this.resourcePath, context)
      } else {
        outputPath = toPosix(path.join(options.outputPathCDN, outputPath))
      }
    }
  } else {
      //  otherwise , call getPackageInfo Get the output path 
    url = outputPath = mpx.getPackageInfo({
      resource: this.resource,
      outputPath: url,
      resourceType: 'staticResources',
      warn: (err) => {
        this.emitWarning(err)
      }
    }).outputPath
  }
  
  ...
  
  this.emitFile(outputPath, content);
  
  ...

Final , The image resource will call compilation.__mpx__.getPackageInfo Method to obtain the output path of image resources for output .

For the same css resources 、wxml resources as well as json resources ,mpx The internal is extracted by creating a sub compiler , There is no in-depth introduction here .

splitChunks Usage of

webpack Of splitChunks Plug ins are used for code splitting , From the above analysis, we can see that MPX The interior is built-in splitChunks Of cacheGroups Configuration items to Take the initiative Implementation of the applet js The module implements partition optimization .webpack There are three common ways to split code ：

• Multi entry segmentation ：webpack Of entry Manual entry for configuration item configuration , Also available compilation.addEntry Program added entry
• Dynamic import ： Separate the code through the inline function of the module , Such as through import('./a')
• Prevent duplication ： Use splitChunks Come and go and separate chunk

The first two are common in our daily development , The third is through webpack Of optimization.splitChunks Configuration items .

Usually ,webpack Configuration item optimization.splitChunks There will be a default configuration for code splitting , We said above MPX Generating for different packages cacheGroups when , Careful students will find that the package we finally generated has two more configuration groups ：

{
    default: {
    automaticNamePrefix: '',
    reuseExistingChunk: true,
    minChunks: 2,
    priority: -20
  },
  vendors: {
    automaticNamePrefix: 'vendors',
    test: /[\\/]node\_modules[\\/]/,
    priority: -10
  },
  ...
}

This is a webpack by optimization.splitChunks.cacheGroups Configured default group , besides optimization.splitChunks There are other default configuration items , As shown in the following code ：

splitChunks: {
    chunks: "async",
    minSize: 30000,
    minChunks: 1,
    maxAsyncRequests: 5,
    maxInitialRequests: 3,
    automaticNameDelimiter: '~',
    name: true,
    cacheGroups: {
        vendors: {
            test: /[\\/]node\_modules[\\/]/,
            priority: -10
        },
    default: {
            minChunks: 2,
            priority: -20,
            reuseExistingChunk: true
        }
    }
}

The implementation effect of the above default configuration is ： Meet the following 4 Conditional module code will be extracted into new chunk

• come from node_modules Module in , Or at least by 2 individual chunk Reusable module code
• Separated chunk Must be greater than or equal to 3000byte, about 30kb
• Load asynchronously on demand chunk when , The maximum number of parallel requests does not exceed 5 individual
• When the page is initially loaded , The maximum number of parallel requests does not exceed 3 individual

Let's introduce the functions of these configuration items ：

• chunks： Express webpack For which chunk Segmentation , Optional value is async、all、initial
  • async： For asynchronous loaded chunks Segmentation
  • initial： Initial for non asynchronous loading chunks Segmentation
  • all： For all chunks Segmentation
• minSize: After the split chunk Minimum size to be satisfied , Otherwise, it will not be split
• minChunks: Indicates that a module should be at least minChunks individual chunk Only when it is contained can it be divided
• maxAsyncRequests: Indicates that load on demand is asynchronous chunk when , The maximum number of parallel requests ; This number includes the asynchrony of the current request chunk And what it depends on chunk Request
• maxInitialRequests: Indicates loading entry chunk when , The maximum number of parallel requests
• automaticNameDelimiter: Represents a split chunk The name of the connector , The default is . Such as chunkvendors.js
• name: Set up chunk The name of the file , The default is true, Express splitChunks be based on chunk and cacheGroups Of key Automatic naming .
• cacheGroups: It allows you to configure multiple groups , Implementation of fine segmentation code ;
  • This object configures property inheritance splitChunks Middle Division cacheGroups All attributes except , These attribute value overrides can be reconfigured in this object splitChunks The value in
  • This object also has some unique properties such as test、priority and reuseExistingChunk etc.

in the light of cacheGroups Configuration adds a little ：

cacheGroups Each configured group can be configured according to test Set conditions , accord with test Module of condition , It's assigned to this group . Modules can be referenced by multiple groups , But eventually it will be based on priority To decide which group to pack into .