One . The phenomenon
A while ago, the company's online environment Java Because OOM Abnormal alarm of , As a result, the whole service is unavailable and pulled out of the cluster , Local simulations reproduce the following phenomena :
The solution was to add metaspace The capacity of ：-XX:MaxMetaspaceSize=500m, From the original default 256m Change it to 500m, Although it didn't appear again oom, But this is just a temporary solution , Through the company's monitoring system metaspace Usage is still on the rise , Moreover, with the increasing number of business visits, it is possible to reach the threshold .
Two . analysis
Metaspace Meta space mainly stores metadata information of classes , The class description information loaded in our application , Such as class name 、 attribute 、 Method 、 Access restrictions, etc , Store in a certain structure Metaspace in .
Thus we can see that metaspace Space growth is due to reflection class loading , Class loading generated by dynamic proxy , in other words Metaspace There is a relationship between the size of the data loaded and the size of the class , The more classes you load metaspace The more memory it takes up .
Because we know the business scenario at that time, the number of visits to the order details interface of a mail service suddenly increased , And view log Of eroor Log found that most of the problems were reported by the order details interface first :java.lang.OutOfMemoryError: Metaspace
Here I'm testing the environment Java Applied jvm Increase in -XX:+TraceClassLoading -XX:+TraceClassUnloading Record the loading and unloading of the class , And then through jmeter Multiple threads call the order details interface to simulate metaspace Spillover phenomenon , Found in catalina.out In addition to the classes used in business, there are also a large number of reflection classes in the file , as follows ：
It is not normal for these reflection classes to be loaded and unloaded frequently , adopt Arthas Diagnostic tools （Java Online diagnosis is one of the best tools Arthas） Looking at the call chain, we find that every time the interface is called is realized by reflection .
At present, our projects are all based on SOA The framework provides access to the outside world , From above sun.reflect The callers of can also see that 
As can be seen from the above figure, when calling the underlying interface, the instance of the class is obtained through reflection , Look at the framework's underlying code implementation to confirm 
Also returned to the underlying interface json Reflection is also used in data deserialization 
Continue with the code, you can see that the implementation of these reflections will use java.lang.Class Inside ReflectionData object 
ReflectionData It's an internal static class that's cached , The attributes inside are the attributes we need to use when doing reflection operations Field, Method Method And constructors, etc . But there's a problem reflectionData Be being SoftReference Modified by soft reference , Here's the picture 
If it is a soft reference, it may be recycled when the memory space is insufficient , If you recycle it, you can only get it again by reflection .
and SoftReference Whether it's recycled or not SoftRefLRUPolicyMSPerMB The values of the parameters are related , Check out our online JVM Configuration discovery for XX:SoftRefLRUPolicyMSPerMB This parameter is set to 0
SoftRefLRUPolicyMSPerMB This parameter roughly means every 1M Free space can be maintained SoftReference The lifetime of the object ( The unit is ms millisecond ),LRU yes Least Recently Used Abbreviation , Least recently used .
This value jvm The default is 1000ms, If it is set to 0, This will cause the soft reference object to be recycled immediately , This will lead to frequent re generation of new classes , And can't achieve the effect of reuse .
There's a lot of sun.reflect.GeneratedSerializationConstructorAccessor,GeneratedMethodAccessor That's how it came about .
I change this parameter back to the default value -XX:SoftRefLRUPolicyMSPerMB=1000 (1 second ), It has been released to the production environment for verification , It came down after it was released , So far, it has basically stabilized 
After adjustment, there is basically no fluctuation again
3、 ... and . summary
- At present, it is mainly through modification JVM Of
-XX:SoftRefLRUPolicyMSPerMBValue to solve metaspace Rising issues , We will continue to observe the changes , Adjust the parameters properly . As for why this parameter was previously set to 0, Still need to find ops To confirm the . - Our application needs a lot of RPC Interaction , Belong to I/O Intensive business , Use SOA,Dubbo We all have similar problems , Through the above source code analysis, we can see that this is unavoidable ( Unless it's a different serialization protocol , such as
hessian, Do not use the method of reflection to assign values ) Including the use of itself Spring Many parts of the framework are also implemented through reflection, such as AOP, And what we often use in burying sitesJsonUtilsTools , adopt dump Files can also see that there are a lot of attribute copy and reflection operations .
Therefore, in the normal business code development, if we encounter the operation of two object assignment, we should try our best to implement it in the way of reflection , Consider the following code ：
The object copy operation here uses apache common-beanutils.jar Medium BeanUtils, The underlying layer of this class uses javabeans+ Reflection implementation , Poor performance , Memory overhead is large , When the system is highly concurrent, it is easy to cause Metaspace Space is growing too fast , It is not recommended to use .
If there are few fields, you can assign values directly , More words can be used Cglib Of BeanCopier class ,BeanCopier The bottom of the class is to use asm Bytecode operation mode to copy objects , Performance loss and memory overhead are relatively small .
Or use MapStruct This helps you generate set、get Tools for methods , It works better .
Source of the article ：http://javakk.com/160.html
