Understand the principle of bytecode enhancement technology through the jvm-sandbox source code

Introduce

JVM  Sandbox container is a kind of  JVM  Non intrusive operation period of  AOP  Solution . adopt  JVM-SANDBOX  Can be restarted without , Don't invade the target  jvm  Code enhancement of the target method under the premise of .

No intrusion , Class isolation , Pluggable , multi-tenancy , High compatibility is its characteristic ,JVM-SANDBOX  It is a relatively low-level code enhancement framework, which can do a lot of things , For example, online system flow control 、 Request recording of online system 、 Playback of results , Online fault location and so on . Such as open source projects  

jvm-sandbox-repeater

 , 

chaosblade

  Are built on this basis , You can go to  github  To understand , Project address ： 

https://github.com/alibaba/jvm-sandbox

In this article, we will analyze  JVM-SANDBOX  The underlying source code , So as to understand the core implementation of a bytecode enhancement framework .

Architecture design

JVM Sandbox  built-in  HTTP  The server （Jetty） Accept user instructions to manage the module , For example, activation , frozen , Refresh etc . Through custom  ClassLoader  Class isolation , be based on  Java  Virtual machine tool interface （JVMTI）, utilize  ASM  The framework enhances the code of the target method .

JVM-SANDBOX  Divided into several sub projects , among  agent,core,spy  Is the main library .agent: Sandbox started agent ,core: Sandbox core ,spy: Sandbox spy class ( Code enhanced buried point class ). The figure below describes  sandbox  The most important module in the overall architecture / function , These abilities are all in  agent,core,spy  Provided in three subprojects .

• Module control management ： Responsible for managing the  sandbox  Own module and user-defined module , For example, module loading , Activate , frozen , uninstall

• Event monitoring processing ： User defined module implementation  Event  Interface for customized processing of enhanced events , Wait for the event distribution processor to trigger .

• Sandbox event distribution processor ： After enhancing the target method, three links will be added to the target method , Before method call  
  BEFORE
  、
  After calling  
  RETURN
  、
  Call throw exception  
  THROWS
  、
  When the code is executed to these three phases, the dispatcher will assign it to the corresponding event listening executor for execution .

• Code weaving framework ： adopt  ASM  The framework relies on  JVMTI  Modify the bytecode of the target method , So as to complete the ability of code enhancement .

• Retrieve filter ： When the user creates an enhanced event on the target method , The sandbox will be right for the target  jvm  Match and filter the target classes and methods in . Match to the target classes and methods set by the user for enhancement , To filter out  sandbox  Internal classes and  jvm  Classes that are considered immutable .

• Load class retrieval :  Get the class collection that needs to be enhanced and rely on the retrieval filter module

• HTTP  The server ： adopt  http  The protocol communicates with the client （sandbox.sh  It can be understood as client ） The essence is through  curl  Command to send instructions to the sandbox  http  The server . For example, module loading , Activate , frozen , Uninstall and other instructions

Related technology

Before the analysis, briefly introduce some underlying technologies that will be used in the bytecode enhancement framework , Essentially, any bytecode enhancement framework is built around these underlying technologies in the upper layer .

JVM TI

JVM TI（JVM TOOL INTERFACE,JVM  Tool interface ） yes  JVM  A set of pairs provided  JVM  Tool interface for operation . adopt  JVMTI  Can realize the right  JVM  A variety of operations , It registers various event hooks through the interface , stay  JVM  The predefined hook is triggered when the event is triggered , To achieve each  JVM  The response to the incident , Events include class file loading 、 Exception generation and capture 、 Thread start and end 、 Enter and exit critical area 、 Member variable modification 、GC  Start and end 、 Method call in and out 、 Critical area competition and waiting 、VM  Start and exit, etc .

When  JVM  Class file loading hook event will be triggered when loading class file  ClassFileLoadHook, triggering  Instrumentation  Class library  ClassFileTransformer ( Bytecode Converter ) Of  transform  Method , stay  transfrom  Method can convert bytecode

Instrumentation

Instrumentation  yes  JVM  The basic library provided can dynamically modify the loaded classes at run time .java agent  yes  JVM TI  An implementation of the interface ,Instrumentation  Instances can only pass  agent  Of  premain  perhaps  agentmian  Method .

load  agent  There are two common ways

• Add... To the startup script -javaagent  Parameter this way will be accompanied by  JVM  Start together ,agent  We need to provide famous  premain  Method , seeing the name of a thing one thinks of its function  premain  Is in  main  Method is executed before it runs , Then I will run the main program  main  Method , This requires developers to confirm the processing logic and parameter content of the agent before the application starts . This kind of hanging on  agent  The advantage of this method is if  agent  Starting requires loading a large number of classes , With  jvm  Loading directly at startup will not cause  JVM  Stuck or  CPU  shake , The disadvantage is that it is not flexible enough .

• utilize  Attach API  stay  JVM  The mount can be completed without restarting during operation ,agent  Need to provide  agentmain  Method , Plug and play mode is very flexible ,Attach API  Provides an add-on to  Java  The mechanism of virtual machine , Use this  API  Attach to the target virtual machine and load its tool agent into the virtual machine , Essentially, it provides and goals  jvm  The ability to communicate . For example, we often use  jastck,jmap  Such orders are all used  attach api  First with the goal  jvm  Establish communication and then execute the command . But if  agent  Starting requires loading a large number of classes, which may lead to the target  jvm  There's a Caton ,cpu  Jitter, etc

stay  Instrumentation  Interface provides multiple  api  Used to manage and operate bytecode . We can focus on the following ：

• addTransformer: Register bytecode converter , After registering a bytecode converter , All class loads are processed by bytecode converters .

• retransformClasses  Back to the  JVM  Bytecode conversion of loaded classes

• removeTransformer  Delete the registered bytecode converter , After deletion, the newly loaded class will not be processed by bytecode converter , But already “ enhance ” The used classes will still be preserved

ClassFileTransformer

ClassFileTransformer（ Bytecode Converter ） It's an interface , There is only... In the interface  transform  One way .

byte[] transform( ClassLoader loader,
 String className,
 Class<?> classBeingRedefined,
 ProtectionDomain protectionDomain,
 byte[] classfileBuffer)
 throws IllegalClassFormatException;

stay  transform  You can return the converted bytecode  byte[], Can pass  Instrumentation#addTransformer  Method registers the implemented bytecode converter , Once registered, the bytecode converter will be triggered at the right time .

• When loading a new class , for example  ClassLoader.defineClass

• When redefining classes , for example  Instrumentation.redefineClasses 

• When re converting classes , for example  Instrumentation.retransformClasses 

Bytecode generation

stay  ClassFileTransformer#transform  Method will return the converted bytecode  byte[], How to dynamically generate bytecode , This is about to mention the famous  ASM  The framework .

ASM  It's a universal  Java  Bytecode operation and analysis framework . It can modify existing classes in binary form or dynamically generate classes

.

Many open source projects using bytecode enhancement technology are based on  ASM  Build , Such as  CGLIB,Groovy,Kotlin  Compiler and so on ,ASM  Some common byte code conversion and analysis algorithms are provided , You can build custom tools for complex transformations and code analysis ; Several core classes ：

• ClassReader： The main function of this class is to read bytecode files , And then inform... Of the read data  ClassVisitor;

• ClassVisitor： For generating and transforming compiled classes  ASM API  be based on  ClassVisitor  abstract class , receive  ClassReader  Send a message to  method  Access requests for , And replace it with another customized  MethodVisitor

• ClassWriter： It is inherited from  ClassVisitor, It is mainly used to generate classes ;

The general implementation process is to first load the original  Class  file , Then access all elements through visitor mode , In the process of access, the elements are transformed , Finally, regenerate a bytecode  byte[], Pictured :

ClassLoader

Another feature class isolation in the sandbox is through the implementation of customization  classLoader  To complete ,ClassLoader（ Class loader ）, stay  java  All classes in must be loaded correctly by the class loader before they can run , Class loader is only used to implement class loading actions , But it's in  Java  The role of the program is far beyond the class loading stage .

For any class , It must be established by the class loader that loads it and the class itself  Java  Uniqueness in virtual machines , Every classloader , All have a separate class namespace . This sentence can be expressed more popularly : Compare two classes whether or not “ equal ”, It makes sense only if the two classes are loaded by the same classloader , otherwise , Even if the two classes come from the same  Class  file , Be the same  Java  Virtual machine loading , As long as the classloaders that load them are different , Then these two classes must not be equal .

Types of Loaders

stay  java8  The default  jvm  Three class loaders will be provided , Namely

• Start class loader (Bootstrap Class Loader): This class loader is responsible for loading and storing in  <JAVA_HOME>\lib  Catalog , Or be -Xbootclasspath  Stored in the path specified by the , And it's  Java  Virtual machines can   Identification of the ( Identify by file name , Such as  rt .jar、t ools.jar, The name does not match the class library even if placed in  lib  The directory will not be loaded ) class   The library is loaded into the memory of the virtual machine

• Extend the classloader (Extension Class Loader): This class loader is in class  sun.misc.Launcher$ExtClassLoader  China and Israel  Java  In the form of code . It's responsible for loading <JAVA_HOM E>\lib\ext  Directory , Or be  java.ext.dirs  The system variables are   All class libraries in the specified path .

• Application class loader (Application Class Loader): This class loader consists of  sun.misc.Launcher$App ClassLoader  To achieve . Because the application class loader is  ClassLoader  Class  getSystem- ClassLoader() Return value of method , So in some cases, it is also called “ system class loader ”. It is responsible for loading the user classpath  (ClassPath) All the class libraries on , Developers can also use this classloader directly in the code . If there is no... In the application   I have customized my own class loader , In general, this is the default classloader in the program .&nbsp;

• Custom class loaders : As the name suggests, it is a class loader customized by developers

Parent delegation model

The class loaders provided by the three systems are introduced above , So how do they collaborate with each other and with custom class loaders ？

The collaboration relationship of various class loaders is shown in Figure , This hierarchical relationship is the class loader “ Parent delegation model ”, In addition to the top-level bootloader , The rest of the class loaders should have their own parent class loaders .

The process of the parent delegation model is : If a class loader receives a class load request , It doesn't try to load the class itself first , Instead, delegate the request to the parent loader to complete , This is true of class loaders at every level , So all load requests should eventually be passed to the top-level boot loader , Only when the parent loader feeds back that it cannot complete the load request ( It did not find the required class in its search scope ) when , The child loader will try to load itself .

Use the parental delegation model to organize the relationships between class loaders , One obvious benefit is  Java  With its class loader, the class in has a   A hierarchical relationship with priority . For example, class  java. lang.Object, It's stored in  rt . jar  In , No matter which class loader wants to load this class , Finally, it is delegated to the startup class loader at the top of the model for loading , therefore  Object  class   The same class can be guaranteed in various class loader environments of the program . conversely , If you don't use the parental delegation model , All by each   If the class loader loads itself , If the user also wrote a name called  java.lang.Object  Class , And put it in the program  ClassPath  in , There will be many different  Object  class ,Java  There is no guarantee for the most basic behavior in the type system , The app will be a mess &nbsp;.

Custom class loaders

Generally, custom class loaders only need to inherit  URLClassLoader, rewrite  findClass  The method can , Of course, you can also inherit directly  ClassLoader, however  

ClassLoader  Load only  classpath  The following classes , and  URLClassLoader  You can load classes in any path .

URLClassLoader  yes  ClassLoader  Subclasses of , It is used to point from  JAR  Files and directories  URL  Search path to load classes and resources . That is to say through  URLClassLoader  You can load the specified  jar  Medium  class  Into memory .

Sandbox isolation strategy

In understanding  classloader  After relevant knowledge , Let's see  jvm-sandbox  The official class isolation strategy map provided

Customized in sandbox  SandboxClassLoader  as well as  ModuleJarClassLoader  To load the classes inside the sandbox and the classes in the module respectively ,sandbox agent  It is from  AppClassLoader  Loading , and  sandbox spy  Spies use  BootstrapClassLoader  Loading , The purpose is to use the parental delegation loading model , Ensure that spies can be targeted correctly  JVM  load , Thus implanted into the target  jvm  Complete the interaction with business code .

SPI

SPI  Full name  Service Provider Interface, yes  Java  A set of functions provided to be implemented or extended by a third party  API, It can be used to enable framework extensions and replace components .Java SPI  It's actually “ Interface based programming ＋ The strategy pattern ＋ The configuration file ” Dynamic loading mechanism of composite implementation , Provided through  interface  seek  implement  Methods . Be similar to  IOC  Thought , Move control of the assembly out of the program , To achieve decoupling .

About  SPI  The underlying implementation of is not analyzed here , The implementation of the custom module in the sandbox makes use of  SPI  Mechanism , When we customize the module in addition to the implementation  Module interface, You still need to follow  SPI  Our agreement is  resource/META-services  In the following  com.alibaba.jvm.sandbox.api.Module  For the file name , The content of the document is  Module  Implementation classpath of . When using  SPI  When loading the implementation class, you need to pass a  classLoader, This  classLoader  yes  moduleClassLoader,moduleClassLoader  Chinese is inherited from  URLClassLoader,SPI  Load the class to find the current  classLoader  The corresponding resource path , To find a match  Module interface  Path file , Then load the corresponding implementation class .

Adaptive scene ： The caller... As needed , Use 、 Extend or replace the implementation strategy .

The boot process

stay  sandbox  The startup process involves  Agent  mount , Customize  classloader  load  sandbox  Inner class , initialization  http  The server , Module loading and initialization .

mount  Agent

JVM-SANDBOX  For two kinds of  agent  All mount modes support .

• -javaagent  Mount directly in the startup script

• utilize  attach api  Mount at runtime

If it is loaded directly at startup, it is directly in the target  JVM  Add... To the startup script of -javaagent  Appoint  sandbox-agent.jar  that will do . When executed ./sandbox -p pid( The goal is  jvm  process  id) It is used.  attach api  In a way that  agent  Mount of

When running the above instructions, execute  sandbox.sh  in  attach_jvm func, stay  attach_jvm  China is through  java -jar  start-up  sandbox-core, And designated  sandbox-agent.jar  Path address and other parameters .

function attach_jvm() {
 # attach target jvm
 "${SANDBOX_JAVA_HOME}/bin/java" \
 ${SANDBOX_JVM_OPS} \
 -jar "${SANDBOX_LIB_DIR}/sandbox-core.jar" \
 "${TARGET_JVM_PID}" \
 "${SANDBOX_LIB_DIR}/sandbox-agent.jar" \
 "home=${SANDBOX_HOME_DIR};token=${token};server.ip=${TARGET_SERVER_IP};server.port=${TARGET_SERVER_PORT};namespace=${TARGET_NAMESPACE}" ||
 exit_on_err 1 "attach JVM ${TARGET_JVM_PID} fail."
}

stay  sandbox-core  Project  pom.xml  The document specifies  core  The startup class of the program  CoreLauncher.java

<manifest>
 <mainClass>com.alibaba.jvm.sandbox.core.CoreLauncher</mainClass>
</manifest>

stay  CoreLauncher  In the use of  Attach api  To the goal  jvm  Conduct  jvm-sandbox agent  mount .

private void attachAgent(final String targetJvmPid,
 final String agentJarPath,
 final String cfg) throws Exception {
 vmObj = VirtualMachine.attach(targetJvmPid);
 if (vmObj != null) {
 vmObj.loadAgent(agentJarPath, cfg);
 }
}

load  Agent

stay  agent  There are only two in the project  java  file ：AgentLaucher.java、SandboxClassLoader.java

stay  agent  Project  pom.xml  Of the program is specified in  Premain-Class  as well as  Agent-Class, Corresponding to the above two  agent  How to mount , When using -javaagent  mount  agent  Will load  Premain-Class  Of  premain  Method , When using  Attach api  mount  agent  Will load  Agent-Class  Of  agentmain  Method

<manifestEntries>
 <Premain-Class>com.alibaba.jvm.sandbox.agent.AgentLauncher</Premain-Class>
 <Agent-Class>com.alibaba.jvm.sandbox.agent.AgentLauncher</Agent-Class>
 <Can-Redefine-Classes>true</Can-Redefine-Classes>
 <Can-Retransform-Classes>true</Can-Retransform-Classes>
</manifestEntries>

&nbsp;

In two  main  The parameters received in the method are the same, respectively ：String featureString  and  Instrumentation inst

• String featureString  Is the execution script execution in the script  attach  Parameters passed during   Such as  sandbox  Path address ,token,namspace( Tenant ) etc.

• Instrumentation  yes  JVM  The basic library provided can dynamically modify the loaded classes at run time , obtain  Instrumentation  Instances can only pass  premain  perhaps  agentmian  Method parameters .

// Start loading  -javaagent
public static void premain(String featureString, Instrumentation inst) {
 LAUNCH_MODE = LAUNCH_MODE_AGENT;
 install(toFeatureMap(featureString), inst);
}
// Dynamic loading  attach api
public static void agentmain(String featureString, Instrumentation inst) {
 LAUNCH_MODE = LAUNCH_MODE_ATTACH;
 final Map<String, String> featureMap = toFeatureMap(featureString); // Parse parameters
 writeAttachResult(
 getNamespace(featureMap), // Get the ownership of the tenant  namespace
 getToken(featureMap), // obtain  token
 install(featureMap, inst)
 );
}

No matter which way you mount  agent  The core is all in  install  In the method ,install  Is to start loading  agent  Business logic of

initialization  Agent

stay  install  Method to complete the pair  agent  The initialization , In the process of initialization, a custom  SandboxClassLoader  Load the sandbox class , Realize the isolation between the inner class of sandbox and business class .

Spy  Spy class

stay  install  Will first use  Instrumentation  Examples will  sandbox-spy.jar  Add to  BootstrapClassLoader  Within the search scope of .

//  take  Spy  Injection into  BootstrapClassLoader
inst.appendToBootstrapClassLoaderSearch(new JarFile(new File(
 getSandboxSpyJarPath(home)
 // SANDBOX_SPY_JAR_PATH
)));

Why do you do this , Let's understand before we explain  Spy  What are spies . In the world view of sandbox , Any one of them  Java  Method calls can be decomposed into  BEFORE、RETURN  and  THROWS  Three links , Therefore, the event detection and process control mechanism of the corresponding link are extended in three links .

// BEFORE
try {
 /*
 * do something...
 */
 // RETURN
 return;
} catch (Throwable cause) {
 // THROWS
}

and  Spy  The spy class is realized  before,return,throws  Wait for the hook function . When will  Spy  When the spy class buys points into the business code , When the class loading mechanism is triggered, the parent delegation model can be used to look up layer by layer , stay  BootstrapClassLoader  You can put  Spy  The spy class is loaded correctly , And in the  Spy  The spy class of has built-in sandbox  SpyHandler. This completes the communication between the target class and the sandbox kernel . The essential class enhancement strategy is shown in the following figure :

In the load  agent  perform  install  The first step in the method is to put  Spy  Spy class is added to  BootstrapClassLoader  Within the search scope of , In this way, when the business code is enhanced  classloader  Parent delegate mechanism ,Spy  The spy class must be loaded correctly ,Spy  Spies will  before,return,throws  Wait for the hook function to be smoothly injected into the business code .

SandBoxClassLoader

Will be  Spy  Spy class is added to  BootstrapClassLoader  Create after  SandBoxClassLoader, The purpose is to use custom  classLoader  You can minimize the impact on the target  JVM  Invasion of .

final ClassLoader sandboxClassLoader = loadOrDefineClassLoader(
 namespace,
 getSandboxCoreJarPath(home)
 // SANDBOX_CORE_JAR_PATH
);

In the custom  SandboxClassLoader  Loading classes in breaks the parental delegation model , First let yourself load , If the self loading fails, then delegate the loading upward . The purpose of this is to know clearly  SandboxClassLoader  Only sandboxes will be loaded  jar  File class , And these  jia  The file path is not in the destination  JVM  Of  ClasssLoader  Searchable path , So there is no point in loading up delegates , Destroy the parental delegation model , Priority self loading performance is better .

 @Override
 protected synchronized Class<?> loadClass(String name, boolean resolve) throws ClassNotFoundException {
 final Class<?> loadedClass = findLoadedClass(name);
 if (loadedClass != null) {
 return loadedClass;
 }
 try {
 Class<?> aClass = findClass(name);
 if (resolve) {
 resolveClass(aClass);
 }
 return aClass;
 } catch (Exception e) {
 return super.loadClass(name, resolve);
 }
 }

start-up  HTTP  service

After creating  SandboxClassLoader  Later, they will use  SandboxClassLoader  load  core.jar  Proxy class in （ProxyCoreServer）, Then the reflection calls  ProxyCoreServer  Of  bind  Method to initialize  HTTP  Service and load all modules .

@Override
public synchronized void bind(final CoreConfigure cfg, final Instrumentation inst) throws IOException {
 this.cfg = cfg;
 try {
 initializer.initProcess(new Initializer.Processor() {
 @Override
 public void process() throws Throwable {
 logger.info("initializing server. cfg={}", cfg);
 jvmSandbox = new JvmSandbox(cfg, inst);
 initHttpServer(); // initialization  http server
 initJettyContextHandler(); // initialization  jetty context  processor
 httpServer.start();//  start-up  http server
 }
 });
 //  Initialize and load all modules
 try {
 jvmSandbox.getCoreModuleManager().reset();
 } catch (Throwable cause) {
 logger.warn("reset occur error when initializing.", cause);
 }
 ......
}

stay  bind  Will initialize  HttpServer  And initialize the context handler , Will be bound in the initialization context handler /module/http/* Of  Servlet  by  ModuleHttpServlet, In this way, when we operate the module, we will go through it first  ModuleHttpServlet  Match and distribute to specific modules to execute commands .

private void initJettyContextHandler() {
 ......
 // module-http-servlet
 final String pathSpec = "/module/http/*";
 logger.info("initializing http-handler. path={}", contextPath + pathSpec);
 context.addServlet(
 new ServletHolder(new ModuleHttpServlet(cfg, jvmSandbox.getCoreModuleManager())),
 pathSpec
 );

 httpServer.setHandler(context);
}

Summary

stay  sandbox  When starting, you first need to use  Attach API  perhaps  java -javaagent  Hang on  agent.agent  You need to provide the corresponding  agetmain  Methods or  premain  Method , stay  agent  Corresponding  main  Method .

First of all, will  spy  Spy class is added to  BootstrapClassLoader  To prepare for later code enhancements , The ultimate goal is to make the goal  JVM  Business class can communicate with sandbox , Then create a custom  SandboxClassLoader  It is used to load inner classes of sandbox to realize class isolation , Finally start  HttpServer  And initialize the corresponding  Servlet, After startup, load and initialize all  module

Module management

Currently in  sandbox  There are two places in the storage module :

• $sandbox_home/module/
  Sandbox system module directory , By configuration item  system_module  Define . A general management module for storing sandboxes , For example, for the sandbox module management function  module-mgr  modular , Future module operation quality monitoring module 、 The security verification modules will also be stored here , Distribute with the release of sandbox . The system module is not refreshed (-f)、** Forced to refresh (-F) The impact of function , Only container reset (-R)** It enables the sandbox to reload all modules under the system module directory .

• $sandbox_home/sandbox-module/
  Sandbox user module directory , from  sandbox.properties  Configuration item for  user_module  Define , The default is ${HOME}/.sandbox-module/. It is generally used to store modules developed by users . Self developed modules often face frequent version upgrades , When dynamic hot plug replacement of modules is required , Can pass ** Refresh (-f) Or force refresh (-F)** To finish reloading .

Module life cycle

There are four states of modules in the sandbox

• 【 Load module 】 Loaded correctly by sandbox , The sandbox will allow the module to command accordingly 、 Code insertion and other actions

• 【 Activation module 】 After loading successfully, the default is frozen , The code needs to be activated actively . The module can only listen to sandbox events when it is active

• 【 Freeze module 】 After entering the frozen state , All sandbox events previously listened to will be masked . It should be noted that , The frozen module will not return the code plug of event listening , Only  delete()、wathcing() Or the plug-in code will be cleaned up when the module is unloaded

• 【 Unload sandbox 】 I won't see this module again , All resources previously allocated to this module will be recycled , Classes that include modules that have been listening for events will be removed from listening instrumentation , Clean and tidy without sequelae

&nbsp;

The state of the module corresponds to the life cycle of the module , The life cycle of a module in the sandbox is as follows :

MODULE_LOAD( Module loading ),MODULE_UNLOAD( Module uninstall ),MODULE_ACTIVE( Module activation ),MODULE_FROZE( Module freeze ),MODULE_LOAD_COMPLETED( Module loading completed )

Custom module

Custom modules need to be implemented  Module  Interface , Statement @Information  Annotations specify unique  id

@Information(id = "broken-clock-tinker")
public class BrokenClockTinkerModule implements Module {

 @Resource
 private ModuleEventWatcher moduleEventWatcher;

 @Command("repairCheckState")
 public void repairCheckState() {

 new EventWatchBuilder(moduleEventWatcher)
 .onClass("com.taobao.demo.Clock")
 .onBehavior("checkState")
 .onWatch(new AdviceListener() {

 /**
 *  Intercept {@code com.taobao.demo.Clock#checkState()} Method , When this method throws an exception, it will be
 * AdviceListener#afterThrowing() Intercepted
 */
 @Override
 protected void afterThrowing(Advice advice) throws Throwable {

 //  Here it is , You can go through  ProcessController  To change the execution process of the original method
 //  The meaning of the code here is ： Change the behavior of throwing exceptions in the original method , Change to return immediately ;void  Return value with  null  Express
 ProcessController.returnImmediately(null);
 }
 });

 }

}


Module loading

Initialize... During startup  http server  The last step of is to load and initialize all  module

 jvmSandbox.getCoreModuleManager().reset();

Let's take a look at loading  module  What has been done , Because this is the initialization operation , So in  reset  First, force the uninstallation of all modules , Avoid having previously loaded modules . Then traverse  moduleLibDirArray  Load module ,moduleLibDirArray  It refers to the storage path of the module ,

stay  reset  Will traverse these two paths to load the modules under the path .

public synchronized CoreModuleManager reset() throws ModuleException {

 logger.info("resetting all loaded modules:{}", loadedModuleBOMap.keySet());

 // 1.  Force uninstall of all modules
 unloadAll();

 // 2.  Load all modules
 for (final File moduleLibDir : moduleLibDirArray) {
 //  User module loading Directory , Load all modules in the user module directory
 //  Verify the access rights of the module
 if (moduleLibDir.exists() && moduleLibDir.canRead()) {
 new ModuleLibLoader(moduleLibDir, cfg.getLaunchMode())
 .load(
 new InnerModuleJarLoadCallback(),
 new InnerModuleLoadCallback()
 );
 } else {
 logger.warn("module-lib not access, ignore flush load this lib. path={}", moduleLibDir);
 }
 }

 return this;
}

After calling methods layer by layer, you will go to  ModuleJarLoader.java  Of  load  In the method , The parameters received here are the above  InnerModuleLoadCallback  object , In this method, we can see that another  ModuleJarClassLoader, Understand this literally  ClassLoader  Mainly to load the class of the module .

After creation, it specifies the thread usage of the current loading module  ModuleJarClassLoader, In this way, when the classes in the module are actually loaded later, they can be used by default  ModuleJarClassLoader  了 .

void load(final ModuleLoadCallback mCb) throws IOException {

 boolean hasModuleLoadedSuccessFlag = false;
 ModuleJarClassLoader moduleJarClassLoader = null;
 logger.info("prepare loading module-jar={};", moduleJarFile);
 try {
 moduleJarClassLoader = new ModuleJarClassLoader(moduleJarFile);

 final ClassLoader preTCL = Thread.currentThread().getContextClassLoader();
 Thread.currentThread().setContextClassLoader(moduleJarClassLoader);

 try {
 hasModuleLoadedSuccessFlag = loadingModules(moduleJarClassLoader, mCb);
 } finally {
 Thread.currentThread().setContextClassLoader(preTCL);
 }

ModuleJarClassLoader

ModuleJarClassLoader  Inherited from a man named  RoutingURLClassLoader, Its construction method is to call the parent class  RoutingURLClassLoader  And passed two parameters , Both parameters are modules to be loaded  jar  file .

public class ModuleJarClassLoader extends RoutingURLClassLoader {

 private ModuleJarClassLoader(final File moduleJarFile,
 final File tempModuleJarFile) throws IOException {
 super(
 new URL[]{new URL("file:" + tempModuleJarFile.getPath())},
 new Routing(
 ModuleJarClassLoader.class.getClassLoader(),
 "^com\\.alibaba\\.jvm\\.sandbox\\.api\\..*",
 "^javax\\.servlet\\..*",
 "^javax\\.annotation\\.Resource.*$"
 )
 );
 }

RoutingURLClassLoader

RoutingURLClassLoader  In, we focus on the construction method and  loadClass  Method , Received in the construction method  ModuleJarClassLoader  The module to be loaded passed  jar  Path and  Routing  object ,Routing  Object contains  ModuleJarClassLoader  Of  ClassLoader（SandboxClassLoader） And some regular matching Classpaths .

adopt  loadClass  The implementation of the method is not difficult to see , When you want to load  className  If the regular matching is successful, you can delegate directly  SandboxClassLoader  load , The advantage of this is that each module does not need to load these general classes . When you want to load  className  Regular matching failed , Load by itself , If the loading is unsuccessful, continue to delegate loading upward .

public class RoutingURLClassLoader extends URLClassLoader {

 private static final Logger logger = LoggerFactory.getLogger(RoutingURLClassLoader.class);
 private final ClassLoadingLock classLoadingLock = new ClassLoadingLock();
 private final Routing[] routingArray;

 public RoutingURLClassLoader(final URL[] urls,
 final Routing... routingArray) {
 super(urls);
 this.routingArray = routingArray;
 }
 
 @Override
 protected Class<?> loadClass(final String javaClassName, final boolean resolve) throws ClassNotFoundException {
 return classLoadingLock.loadingInLock(javaClassName, new ClassLoadingLock.ClassLoading() {
 @Override
 public Class<?> loadClass(String javaClassName) throws ClassNotFoundException {
 //  The priority query class loads the routing table , If you hit the routing rule , From the... In the routing table first  ClassLoader  Complete class loading
 if (ArrayUtils.isNotEmpty(routingArray)) {
 for (final Routing routing : routingArray) {
 if (!routing.isHit(javaClassName)) {
 continue;
 }
 final ClassLoader routingClassLoader = routing.classLoader;
 try {
 return routingClassLoader.loadClass(javaClassName);
 } catch (Exception cause) {
 //  If at present  routingClassLoader  Could not find the class that should be loaded first ( There is no possibilities , But it does not rule out that there is a class named with the same name )
 //  Exceptions should be ignored at this time , Continue loading
 // ignore...
 }
 }
 }
 //  First go through the cache of loaded classes , If you don't hit , Then continue to load
 final Class<?> loadedClass = findLoadedClass(javaClassName);
 if (loadedClass != null) {
 return loadedClass;
 }
 try {
 Class<?> aClass = findClass(javaClassName);
 if (resolve) {
 resolveClass(aClass);
 }
 return aClass;
 } catch (Exception cause) {
 DelegateBizClassLoader delegateBizClassLoader = BusinessClassLoaderHolder.getBussinessClassLoader();
 try {
 if(null != delegateBizClassLoader){
 return delegateBizClassLoader.loadClass(javaClassName,resolve);
 }
 } catch (Exception e) {
 // Ignore exceptions , Continue loading
 }
 return RoutingURLClassLoader.super.loadClass(javaClassName, resolve);
 }
 }
 });
}
}


stay  ModuleJarLoader.java  Of  load  In addition to creating  ModuleJarClassLoader, Also called  loadModules  Method , Using in methods  ServiceLoader(SPI) load  Module  Interface implementation , Then traverse to check whether the interface implementation meets the requirements , For example, whether there is @Infomation  annotation , The module is unique  id  Is it legal , Whether the starting mode of the module is consistent with that of the sandbox . When these pre checks pass , call  ModuleLoadCallback.load  Load the module .

private boolean loadingModules(final ModuleJarClassLoader moduleClassLoader,
 final ModuleLoadCallback mCb) {

 final Set<String> loadedModuleUniqueIds = new LinkedHashSet<String>();
 final ServiceLoader<Module> moduleServiceLoader = ServiceLoader.load(Module.class, moduleClassLoader);
 final Iterator<Module> moduleIt = moduleServiceLoader.iterator();
 while (moduleIt.hasNext()) {

 final Module module;
 try {
 module = moduleIt.next();
 } catch (Throwable cause) {
 logger.warn(&quot;loading module instance failed: instance occur error, will be ignored. module-jar={}&quot;, moduleJarFile, cause);
 continue;
 }

 final Class<?> classOfModule = module.getClass();

 //  Determine whether the module implements @Information  Mark
 if (!classOfModule.isAnnotationPresent(Information.class)) {
 logger.warn(&quot;loading module instance failed: not implements @Information, will be ignored. class={};module-jar={};&quot;,
 classOfModule,
 moduleJarFile
 );
 continue;
 }

 final Information info = classOfModule.getAnnotation(Information.class);
 final String uniqueId = info.id();

 //  Judgment module  ID  Is it legal
 if (StringUtils.isBlank(uniqueId)) {
 logger.warn(&quot;loading module instance failed: @Information.id is missing, will be ignored. class={};module-jar={};&quot;,
 classOfModule,
 moduleJarFile
 );
 continue;
 }

 //  Judge whether the startup mode required by the module matches the startup mode of the container
 if (!ArrayUtils.contains(info.mode(), mode)) {
 logger.warn(&quot;loading module instance failed: launch-mode is not match module required, will be ignored. module={};launch-mode={};required-mode={};class={};module-jar={};&quot;,
 uniqueId,
 mode,
 StringUtils.join(info.mode(), &quot;,&quot;),
 classOfModule,
 moduleJarFile
 );
 continue;
 }

 try {
 if (null != mCb) {
 mCb.onLoad(uniqueId, classOfModule, module, moduleJarFile, moduleClassLoader);
 }
 }
 ......
 }

@Infomation

annotation @Infomation  It is defined inside the sandbox , It is mainly used to describe the information of the module . on top  loadingModules  Method used in  SPI  After loading the corresponding module implementation class , Will find the corresponding @Infomation  Do some pre check operations for information .

@Information(id = "broken-clock-tinker")
public class BrokenClockTinkerModule implements Module 

onLoad( Real load )

Above  loadingModules  According to  SPI  The mechanism has obtained the implementation class of the module , stay  loadingModules  In the end, it will call  ModuleLoadCallback  Of  onLoad  Method ,onLoad  call  DefaultCoreModuleManager#load  Method here is to really load the contents of the module implementation class .

The loading process is mainly divided into the following steps :

• Initialization module information  CoreModule  stay  coreModule  There is  module  Implementation class of ,moduleJar,ModuleClassLoader, And the most important set of module class converters  sandboxClassFileTransformers, Class converter is the key to bytecode enhancement , The code enhancement chapter will introduce in detail

• Note injection @Resource  resources , for example  ModuleEventWatcher  Event observer . About  ModuleEventWatcher  The code enhancement chapter will introduce in detail

• Notify the corresponding implementation of module loading in the life cycle （module  Implementation classes can implement  Module  Interface and sandbox module life cycle interface  ModuleLifecyle  Interface ）, stay  ModuleLifecyle  In the method corresponding to the interface , Users can customize and implement business logic .

• Activate the module and notify the corresponding implementation of the module activation in the lifecycle , Only the activated module can respond to the enhanced event of the module .

• Make the module unique  id  And current  coreModule  Instance stored in module list , The module list is a global  ConcurrentHashMap. Remember when  Httpserver  It will be initialized at startup  ModuleHttpServlet, stay  ModuleHttpServlet  When receiving the request, the module will be parsed from the parameters  id, So as to get the corresponding  coreModule.

• Notify the corresponding implementation of module loading completion in the life cycle

private synchronized void load(final String uniqueId,
 final Module module,
 final File moduleJarFile,
 final ModuleJarClassLoader moduleClassLoader) throws ModuleException {
 ......
 //  Initialization module information
 final CoreModule coreModule = new CoreModule(uniqueId, moduleJarFile, moduleClassLoader, module);
 //  Inject @Resource  resources
 injectResourceOnLoadIfNecessary(coreModule);
 //  Notification lifecycle , Module loading
 callAndFireModuleLifeCycle(coreModule, MODULE_LOAD);
 //  Set to loaded
 coreModule.markLoaded(true);
 //  If the module is marked, it is automatically activated when loaded , You need to activate the module after loading
 markActiveOnLoadIfNecessary(coreModule);
 //  Register in the module list
 loadedModuleBOMap.put(uniqueId, coreModule);
 //  Notification lifecycle , Module loading completed
 callAndFireModuleLifeCycle(coreModule, MODULE_LOAD_COMPLETED);


}

Summary

During module loading , First, we will traverse the storage  module  The path of , Then through the customized module  ClassLoader(ModuleJarClassLoader) With  SPI  Load module by  jar  In the document  Module  Interface implementation class , After loading successfully, the module will be pre checked , After passing the check, the implementation class will be injected  Resouce  Resources and notify the corresponding lifecycle events , The final will be  CoreModule  Register in the module list .

Module activation

After activation, the module can receive user-defined instructions , The custom instruction is actually through @Command  Annotations are customized in the module  http  Interface . stay  http  You can do anything in the implementation method corresponding to the interface , For example, trigger code enhancement .

stay  sandbox.sh  The startup script of can be seen by using -a  Parameter activates the specified module , The essence will send a request call &quot;sandbox-module-mgr/active&quot;,sandbox-module-mgr  It is a general sandbox system management module

# -a active module
[[ -n ${OP_MODULE_ACTIVE} ]] &&
 sandbox_curl_with_exit "sandbox-module-mgr/active" "&ids=${ARG_MODULE_ACTIVE}"

stay  sandbox-mgr-module  Project  ModuleMgrModule  Class , find  active  Method definition , because  sandbox-mgr-module  Belongs to the system module , In the module loading described above, the system module will also be loaded , So when used -a  When the parameter activates the module, it will also pass  ModuleHttpServlet  Distribute the request to  ModuleMgrModule  Of  active  Method .

stay  active  Parameters will be parsed in , Get the module to activate  id, Through the module  id  Find the corresponding  module  example （ Remember the introduction in the module loading chapter that when the module is loaded, the module's  coreModule  Store in  loadedModuleBOMap  In the! , there  search  The essence is from  map  Find the corresponding  coreModule）

obtain  module  Implement... In the class @Information  annotation  id, Determine whether it has been activated , Activate if not

@Command("active")
public void active(final Map<String, String> param,
 final PrintWriter writer) throws ModuleException {
 int total = 0;
 final String idsStringPattern = getParamWithDefault(param, &quot;ids&quot;, EMPTY);
 for (final Module module : search(idsStringPattern)) {
 final Information info = module.getClass().getAnnotation(Information.class);
 final boolean isActivated = moduleManager.isActivated(info.id());
 if (!isActivated) {
 try {
 moduleManager.active(info.id());
 total++;
 } catch (ModuleException me) {
 logger.warn(&quot;active module[id={};] occur error={}.&quot;, me.getUniqueId(), me.getErrorCode(), me);
 }// try
 } else {
 total++;
 }
 }// for
 output(writer, &quot;total %s module activated.&quot;, total);
}

Summary

The essence of activation is to  coreModule  Example of  isActivated  Tag set to  true, In this way, the module can receive the event triggered by code enhancement

Module freeze

Module freezing is to mark the loaded module with freeze , The user-defined enhancement code still exists after freezing , It's just that sandboxes don't handle user-defined code enhancement logic .

In the startup script , Freezing instructions are also sent to  sandbox-module-mgr  Sandbox system management module

# -A frozen module
[[ -n ${OP_MODULE_FROZEN} ]] &&
 sandbox_curl_with_exit "sandbox-module-mgr/frozen" "&ids=${ARG_MODULE_FROZEN}

Summary

The processing process of module freezing is basically the same as that of module activation , But one more step is to freeze the event handler .

The event handler will be introduced in the chapter of code enhancement and event processing .

Module uninstall

Module unloading will clean the whole module from the sandbox , All resources previously allocated to this module will be recycled , Classes that include modules that have been listening for events will be removed from listening instrumentation , Clean and tidy without sequelae

In the startup script , Freezing instructions are also sent to  sandbox-module-mgr  Sandbox system management module

# -u unload module
[[ -n ${OP_MODULE_UNLOAD} ]] &&
 sandbox_curl_with_exit "sandbox-module-mgr/unload" "&action=unload&ids=${ARG_MODULE_UNLOAD}"

The process of module unloading

• Try to freeze the module , Let the event handler pause from executing user-defined enhanced logic .

• Notify the corresponding implementation of the lifecycle module uninstall

• Delete the corresponding module from the module registry

• Mark uninstall ,isLoaded=true

• Release resources , One step in the module loading process is injection @Resource  resources , At injection  ModuleEventWatcher  Resource time , It's actually built  ReleaseResource  object , And implemented  release  Method . stay  ModuleEventWatcher.delete  Will use  Instrumentation  Class library delete class enhancement converter  SandboxClassFileTransformer, In this way, when new classes are loaded, they will not be implanted  spy  Spies . But the loaded classes always have spyware code . adopt  Instrumentation  The class library re renders the bytecode of the target class .

@Override
public void delete(final int watcherId,
 final Progress progress) {
 final Set<Matcher> waitingRemoveMatcherSet = new LinkedHashSet<Matcher>();
 //  Find out what to delete  SandboxClassFileTransformer
 final Iterator<SandboxClassFileTransformer> cftIt = coreModule.getSandboxClassFileTransformers().iterator();
 int cCnt = 0, mCnt = 0;
 while (cftIt.hasNext()) {
 final SandboxClassFileTransformer sandboxClassFileTransformer = cftIt.next();
 if (watcherId == sandboxClassFileTransformer.getWatchId()) {
 //  Freeze all associated code enhancements
 EventListenerHandler.getSingleton()
 .frozen(sandboxClassFileTransformer.getListenerId());
 //  stay  JVM  Remove the hit  ClassFileTransformer
 inst.removeTransformer(sandboxClassFileTransformer);
 //  Count
 cCnt += sandboxClassFileTransformer.getAffectStatistic().cCnt();
 mCnt += sandboxClassFileTransformer.getAffectStatistic().mCnt();
 //  Append to the filter set to be deleted
 waitingRemoveMatcherSet.add(sandboxClassFileTransformer.getMatcher());
 //  Clear the  SandboxClassFileTransformer
 cftIt.remove();
 }
 }
 //  Find the class collection that needs to be deleted and re rendered
 final List<Class<?>> waitingReTransformClasses = classDataSource.findForReTransform(
 new GroupMatcher.Or(waitingRemoveMatcherSet.toArray(new Matcher[0]))
 );
 logger.info(&quot;watch={} in module={} found {} classes for delete.&quot;,
 watcherId,
 coreModule.getUniqueId(),
 waitingReTransformClasses.size()
 );
 beginProgress(progress, waitingReTransformClasses.size());
 try {
 //  application  JVM
 reTransformClasses(watcherId, waitingReTransformClasses, progress);
 } finally {
 finishProgress(progress, cCnt, mCnt);
 }
}

• close  ModuleJarClassLoader, You can load all the module classes from  jvm  Clean up . It will be used before closing  SPI  Find the module  ModuleJarUnloadSPI  Implementation class of interface , The notification module has been uninstalled and can do some custom business processing .

public void closeIfPossible() {
 onJarUnLoadCompleted();
 try {

 //  If it is  JDK7+ Version of , URLClassLoader  Realized  Closeable  Interface , Just call it directly
 if (this instanceof Closeable) {
 logger.debug("JDK is 1.7+, use URLClassLoader[file={}].close()", moduleJarFile);
 try {
 ((Closeable)this).close();
 } catch (Throwable cause) {
 logger.warn("close ModuleJarClassLoader[file={}] failed. JDK7+", moduleJarFile, cause);
 }
 return;
 }
 .......
 }
 }

Summary

During module unloading, the processing of events will be suspended first , And then use it  Instrumentation  Uninstall the enhanced bytecode and restore the original bytecode , Finally close  ModuleClassLoader

Code enhancements

This chapter will focus on how to use  sandbox  Enhance the code and the implementation principle behind it

Custom module

Refer to the official  demo  Before enhancing the code, we first customize the module   As shown below ：

You need to implement  Module  Interface , Statement @Information  Annotations specify unique  id, Definition  ModuleEventWatcher  annotation , Declare custom @Command  Processing method of instructions .

@Information(id = "broken-clock-tinker")
public class BrokenClockTinkerModule implements Module {

 @Resource
 private ModuleEventWatcher moduleEventWatcher;

 @Command("repairCheckState")
 public void repairCheckState() {

 new EventWatchBuilder(moduleEventWatcher)
 .onClass("com.taobao.demo.Clock")
 .onBehavior("checkState")
 .onWatch(new AdviceListener() {

 /**
 *  Intercept {@code com.taobao.demo.Clock#checkState()} Method , When this method throws an exception, it will be
 * AdviceListener#afterThrowing() Intercepted
 */
 @Override
 protected void afterThrowing(Advice advice) throws Throwable {

 //  Here it is , You can go through  ProcessController  To change the execution process of the original method
 //  The meaning of the code here is ： Change the behavior of throwing exceptions in the original method , Change to return immediately ;void  Return value with  null  Express
 ProcessController.returnImmediately(null);
 }
 });

 }

}

demo  Will use  EventWatchBuilder  Yes  Clock  Class  checkState  Method for code enhancement , When  checkState  Method will execute after throwing an exception  afterThrowing  The code in the method , Ignore the exception and return directly .

watch

Focus on  onWatch  Method , After layers of calls, you will arrive  DefaultModuleEventWatch  Of  watch  Method , and  watch  Method is the entry that really triggers code enhancement .

• stay  watch  Method will first create  SandboxClassFileTransformer  Sandbox converter , And register it in  CoreModule  in

• utilize  Instrumentation  The class library  addTransformer api  register  SandboxClassFileTransformer  After the sandbox class converter instance registers the class converter , All subsequent class loads will go through  SandboxClassFileTransformer

• Find a collection of classes to render , utilize  matcher  Object to find the current  jvm  Loaded classes matcher  The object is  EventWatchBuilder  The wrapper object of the target class and target method specified in , stay  matcher  Object specifies the matching rule .

• Render the found class （ Code enhancements ）, utilize  Instrumentation  Of  retransformClasses  Method re pair  JVM  Bytecode conversion of loaded classes .

private int watch(final Matcher matcher,
 final EventListener listener,
 final Progress progress,
 final Event.Type... eventType) {
 final int watchId = watchIdSequencer.next();
 //  Add... To the corresponding module  ClassFileTransformer
 final SandboxClassFileTransformer sandClassFileTransformer = new SandboxClassFileTransformer(
 watchId, coreModule.getUniqueId(), matcher, listener, isEnableUnsafe, eventType, namespace);

 //  Sign up to  CoreModule  in
 coreModule.getSandboxClassFileTransformers().add(sandClassFileTransformer);

 // here  addTransformer  after , The next class loading will go through  sandClassFileTransformer
 inst.addTransformer(sandClassFileTransformer, true);

 //  Find a collection of classes to render
 final List<Class<?>> waitingReTransformClasses = classDataSource.findForReTransform(matcher);
 logger.info(&quot;watch={} in module={} found {} classes for watch(ing).&quot;,
 watchId,
 coreModule.getUniqueId(),
 waitingReTransformClasses.size()
 );

 int cCnt = 0, mCnt = 0;

 //  Progress notification start
 beginProgress(progress, waitingReTransformClasses.size());
 try {

 //  application  JVM
 reTransformClasses(watchId,waitingReTransformClasses, progress);

 //  Count
 cCnt += sandClassFileTransformer.getAffectStatistic().cCnt();
 mCnt += sandClassFileTransformer.getAffectStatistic().mCnt();


 //  Activate the enhanced class
 if (coreModule.isActivated()) {
 final int listenerId = sandClassFileTransformer.getListenerId();
 EventListenerHandler.getSingleton()
 .active(listenerId, listener, eventType);
 }

 } finally {
 finishProgress(progress, cCnt, mCnt);
 }

 return watchId;

Code enhancements

In the chapter of basic knowledge, it is introduced that bytecode conversion mainly needs to be realized  ClassFileTransformer  Interface , Then register the implementation class to  Instrumentation  Bytecode conversion can be triggered at the right time , It can also be done through  Instrumentation  Of  api  for example  retransformClasses  Manual trigger .

stay  SandboxClassFileTransformer#_transform  Methods are mainly about whether matching is the target class and target method we want to enhance when the class is loaded or redefined , If so, use the code enhancement framework of sandbox to generate bytecode , If not, ignore and do not handle .

private byte[] _transform(final ClassLoader loader,
 final String internalClassName,
 final Class<?> classBeingRedefined,
 final byte[] srcByteCodeArray) {
 //  If not on  unsafe  switch , It is not allowed to enhance from  BootStrapClassLoader  Class
 if (!isEnableUnsafe
 && null == loader) {
 logger.debug(&quot;transform ignore {}, class from bootstrap but unsafe.enable=false.&quot;, internalClassName);
 return null;
 }

 final ClassStructure classStructure = getClassStructure(loader, classBeingRedefined, srcByteCodeArray);
 final MatchingResult matchingResult = new UnsupportedMatcher(loader, isEnableUnsafe).and(matcher).matching(classStructure);
 final Set<String> behaviorSignCodes = matchingResult.getBehaviorSignCodes();

 //  If a behavior doesn't match, you don't have to continue
 if (!matchingResult.isMatched()) {
 logger.debug(&quot;transform ignore {}, no behaviors matched in loader={}&quot;, internalClassName, loader);
 return null;
 }

 //  Start class matching
 try {
 final byte[] toByteCodeArray = new EventEnhancer().toByteCodeArray(
 loader,
 srcByteCodeArray,
 behaviorSignCodes,
 namespace,
 listenerId,
 eventTypeArray
 );
 if (srcByteCodeArray == toByteCodeArray) {
 logger.debug(&quot;transform ignore {}, nothing changed in loader={}&quot;, internalClassName, loader);
 return null;
 }

 // statistic affect
 affectStatistic.statisticAffect(loader, internalClassName, behaviorSignCodes);

 logger.info(&quot;transform {} finished, by module={} in loader={}&quot;, internalClassName, uniqueId, loader);
 return toByteCodeArray;
 } catch (Throwable cause) {
 logger.warn(&quot;transform {} failed, by module={} in loader={}&quot;, internalClassName, uniqueId, loader, cause);
 return null;
 }
}

stay  EventEnhancer#toByteCodeArray  Use in method  ASM  The framework enhances the code , adopt  ASM  Of  ClassReader  Read the target class bytecode , And then through  ClassWriter  Write the converted bytecode to  dump  Folder , And return the converted bytecode .

@Override
public byte[] toByteCodeArray(final ClassLoader targetClassLoader,
 final byte[] byteCodeArray,
 final Set<String> signCodes,
 final String namespace,
 final int listenerId,
 final Event.Type[] eventTypeArray) {
 //  Returns the enhanced bytecode
 final ClassReader cr = new ClassReader(byteCodeArray);
 final ClassWriter cw = createClassWriter(targetClassLoader, cr);
 final int targetClassLoaderObjectID = ObjectIDs.instance.identity(targetClassLoader);
 cr.accept(
 new EventWeaver(
 ASM7, cw, namespace, listenerId,
 targetClassLoaderObjectID,
 cr.getClassName(),
 signCodes,
 eventTypeArray
 ),
 EXPAND_FRAMES
 );
 return dumpClassIfNecessary(cr.getClassName(), cw.toByteArray());
}

The transformation process is through inheritance  ClassVisitor  Abstract classes implement method event weavers  EventWeaver, stay  EventWeaver  The target method will be matched in , If the match is successful, weave in  Spy  Buried point method in spy class . The specific code is in  EventWeaver#visitMethod  In the method , The content is too long to paste .

Summary

Use... In the code enhancement process  Instrumentation  The class library  addTransformer api  register  SandboxClassFileTransformer  Sandbox class converter instance , When the class is loaded 、 Class redefines 、 Class conversion will trigger  SandboxClassFileTransformer  Of  transformer  Method to convert bytecode , The essence is to use  ASM  The framework will  SPY  Methods in spy class are woven into target methods , Finally, the enhanced bytecode is output  byte[]

Event handling

In the code enhancement chapter, it is introduced how to weave the spy class into the target method. The generated bytecode will reapply the generated class to  jvm  in , How can the enhanced method be implemented into the user-defined enhanced logic ？

take  spyMethodOnBefore  give an example , Suppose we monitor the target method  Before  event , Because the enhanced code will be executed before the target method is executed  spyMethodOnBefore  Method , stay  spyMethodOnBefore  You will find the corresponding  SpyHandler  Spy processor instance  EventListenHandler.

public static Ret spyMethodOnBefore(final Object[] argumentArray,
 final String namespace,
 final int listenerId,
 final int targetClassLoaderObjectID,
 final String javaClassName,
 final String javaMethodName,
 final String javaMethodDesc,
 final Object target) throws Throwable {
 final Thread thread = Thread.currentThread();
 if (selfCallBarrier.isEnter(thread)) {
 return Ret.RET_NONE;
 }
 final SelfCallBarrier.Node node = selfCallBarrier.enter(thread);
 try {
 final SpyHandler spyHandler = namespaceSpyHandlerMap.get(namespace);
 if (null == spyHandler) {
 return Ret.RET_NONE;
 }
 return spyHandler.handleOnBefore(
 listenerId, targetClassLoaderObjectID, argumentArray,
 javaClassName,
 javaMethodName,
 javaMethodDesc,
 target
 );
 } catch (Throwable cause) {
 handleException(cause);
 return Ret.RET_NONE;
 } finally {
 selfCallBarrier.exit(thread, node);
 }
}

EventListenHandler

EventListenHandler  yes  SpyHandler  Implementation class of interface , stay  handleOnBefore  Will pass  listenerID  Find the event handler  processor, There are user-defined in the event handler  listenr  example , In this way, you can call back  listener  Of  OnEvent  Method , Execute the code written by the real user in the custom module .

@Override
public Spy.Ret handleOnBefore(int listenerId, int targetClassLoaderObjectID, Object[] argumentArray, String javaClassName, String javaMethodName, String javaMethodDesc, Object target) throws Throwable {

 //  Events generated in the guard area do not need to be responded
 if (SandboxProtector.instance.isInProtecting()) {
 logger.debug("listener={} is in protecting, ignore processing before-event", listenerId);
 return newInstanceForNone();
 }

 //  Get event handler
 final EventProcessor processor = mappingOfEventProcessor.get(listenerId);

 //  If not already registered , Then return directly , Do nothing
 if (null == processor) {
 logger.debug("listener={} is not activated, ignore processing before-event.", listenerId);
 return newInstanceForNone();
 }

 //  Get call trace information
 final EventProcessor.Process process = processor.processRef.get();

 //  If the current processing  ID  Be ignored , Return immediately
 if (process.isIgnoreProcess()) {
 logger.debug("listener={} is marked ignore process!", listenerId);
 return newInstanceForNone();
 }

 //  call  ID
 final int invokeId = invokeIdSequencer.getAndIncrement();
 process.pushInvokeId(invokeId);

 //  Call the process  ID
 final int processId = process.getProcessId();

 final ClassLoader javaClassLoader = ObjectIDs.instance.getObject(targetClassLoaderObjectID);
 // Place the business class loader
 BusinessClassLoaderHolder.setBussinessClassLoader(javaClassLoader);
 final BeforeEvent event = process.getEventFactory().makeBeforeEvent(
 processId,
 invokeId,
 javaClassLoader,
 javaClassName,
 javaMethodName,
 javaMethodDesc,
 target,
 argumentArray
 );
 try {
 return handleEvent(listenerId, processId, invokeId, event, processor);
 } finally {
 process.getEventFactory().returnEvent(event);
 }
}

EventProcessor

EventProcessor  The event handler contains user-defined  listener  as well as  listenerId, The stack of calls is recorded when the enhanced code is executed .

In code enhancement  watch  In the chapter ,watch  The enhanced class will be activated at the end of method execution , Here is mainly to  listenerId  As  key, take  EventProcessor  As  value  Deposit to  map  in . In this way, when an event is received, it can be based on  listenerId  Find the corresponding event handler

public void active(final int listenerId,
 final EventListener listener,
 final Event.Type[] eventTypes) {
 mappingOfEventProcessor.put(listenerId, new EventProcessor(listenerId, listener, eventTypes));
 logger.info("activated listener[id={};target={};] event={}",
 listenerId,
 listener,
 join(eventTypes, ",")
 );
}

listenerId

listenerId  It can be understood as that the user-defined module performs  watch  Will generate a globally unique  id, Through this  id  To bind the corresponding event handler

listener

The ultimate goal of event handlers is callback  listener  The method in ,listener  It's our custom module that makes the target method  watch  The parameter passed by , stay  listener  Of  onEvent  perhaps  AdviceListener  The logic to be enhanced for the target method can be implemented arbitrarily in each buried point method of .

Summary

In the event processing process, it passes  SpyHandler  Implementation class of  EventListenHandler  Distribution of events , Match to the event handler corresponding to the current event  EventProcessor, stay  EventProcessor  Will really call back the user implementation  listener, So as to complete the real user-defined code enhancement logic .

summary

JVM-SANDBOX  It's a kind of  JVM  Non intrusive operation period of  AOP  Solution , Through it, we can easily develop many interesting projects, such as recording and playback 、 fault simulation 、 Dynamic logs 、 Row link acquisition and so on , In this paper, we introduce  JVM-SANDBOX  Implementation details , Understand its core implementation .

Review the key points :

• JVM-SANDBOX  be based on  JVM TI(JVM  Tool interface ) Realization  agent  To the goal  jvm  Mount , utilize  Instrumentation  Class libraries register custom class converters （SandboxClassFileTransformer）, Use in class converters  ASM  The framework weaves methods of the target class  Spy  Buried point method in spy class , So as to realize byte code enhancement .

• JVM-SANDBOX  utilize  ClassLoader  Parent delegation model , Customize  SandboxClassLoader  Load sandbox inner class (spy-core), Use customization  ModulerJarClassLoader  and  SPI  Mechanism to load custom modules , So as to realize class isolation .

• Pluggable and multi tenant features are more inclined to  JVM-SANDBOX  Internal business logic , It is described in detail in the module management chapter module loading and unloading .

Finally, explain why you should be right  jvm-sandbox  Source code analysis , It is because it is used in work and in the open source community  jvm-sandbox  And found some  bug  For example, multiple module loading and unloading will lead to  metaspace oom etc. . If you have energy in the back, you will be right  bug  And the solution process .