Introduction to distributed cache / cache cluster

development environment ：

System：Windows

JavaEE Server：tomcat5.0.2.8、tomcat6

JavaSDK： jdk6+

IDE：eclipse、MyEclipse 6.6

 

Development dependency Library ：

JDK6、 JavaEE5、ehcache-core-2.5.2.jar

Email：[email protected]

Blog：hoojo The blog of _CSDN Blog -JavaEE,OpenSource frame ,JavaScript Domain Blogger

hoojo - Blog Garden

http://hoojo.blogjava.net

 

front 2 This article introduces to Ehcache Integrate Spring Use page 、 Object caching  Ehcache Integrate Spring Use page 、 Object caching - hoojo - Blog Garden  

stay Spring、Hibernate Use in Ehcache cache   stay Spring、Hibernate Use in Ehcache cache - hoojo - Blog Garden

 

One 、 Introduction to caching system

EhCache It's pure. Java In process caching framework for , With fast 、 Characteristics such as ability , yes Hibernate In the default CacheProvider.

EhCache Application Architecture , The picture below is EhCache Location in the application ：

EhCache  The main characteristics of the system are ：

1. Fast 、 Essence ;

2. Simple ;

3. Multiple caching strategies ;

4. There are two levels of caching data ： Memory and disk , So there's no need to worry about capacity ;

5. The cached data will be written to disk during virtual machine restart ;

6. Can pass RMI、 Pluggable API And so on ;

7. Listening interface with cache and cache manager ;

8. Supports multiple cache manager instances , And multiple cache areas of an instance ;

9. Provide Hibernate Cache implementation of ;

because EhCache It's the caching system in the process , Once the application is deployed in a cluster environment , Each node maintains its own cached data , When a node updates the cached data , These updated data cannot be shared among other nodes , This will not only reduce the efficiency of node operation , And it will cause the data to be out of sync . For example, a website uses A、B Two nodes are deployed as clusters , When A After the node's cache is updated , and B Before the node cache is updated, users may browse the page , Later, the updated data , There will be data that has not been updated , Although we can also pass Session Sticky Technology to lock users on a node , But for some interactive or right and wrong Web In terms of system ,Session Sticky Obviously not very suitable for .

So we need to use EhCache The cluster solution of .

from 1.2 Version start ,Ehcache You can use distributed caching .EhCache  from 1.7  Version start , Five cluster schemes are supported , Namely ：

• Terracotta

• RMI

• JMS

• JGroups

• EhCache Server

The three most commonly used cluster methods , Namely RMI、JGroups as well as EhCache Server . This paper mainly introduces RMI The way .

The distributed feature is based on plugin The way to achieve .Ehcache It comes with some default distributed cache plug-ins , These plug-ins can meet the needs of most applications . If you need to use other plug-ins, you need to develop them yourself , Developers can view distribution Package source code and JavaDoc To achieve it . Although not necessary , Understand some when using distributed caching ehcahce The design idea of is also helpful . See the distributed cache design page for details . The following sections show how to make distributed plug-ins the same as ehcache Working together .

Here are some important aspects of distributed caching ：

• How do you know about other caches in a clustered environment ？

• What is the form of distributed messages ？

• What needs to be replicated ？ increase （Puts）, to update （Updates） Or failure （Expiries）？

• How to copy ？ Synchronous or asynchronous ？

To install distributed caching , You need to configure a PeerProvider、 One CacheManagerPeerListener,

They are for a CacheManager It's the overall situation . Every... That performs distributed operations cache Add one cacheEventListener To send messages .

 

Two 、 Cluster cache concept and its configuration

Correct element type

Only serializable elements can be copied . Some operations , For example, remove , You only need the key value of the element, not the whole element ; In such an operation, even if the element is not serializable, but the key value is serializable, it can be copied .

Members found that （Peer Discovery）

Ehcache When clustering, there is a cache The concept of group . Every cache It's all other cache One of the peer, There is no lord cache The existence of . Just now we asked a question ： How do you know about other caches in a clustered environment ？ This problem can be named member discovery （Peer Discovery）.

Ehcache Two mechanisms are provided for member discovery , It's like a car ： Manual and automatic . To use a built-in member discovery mechanism, you need to ehcache Specified in the configuration file of cacheManagerPeerProviderFactory Elemental class The attribute is

net.sf.ehcache.distribution.RMICacheManagerPeerProviderFactory.

Automatic member discovery

The automatic discovery method uses TCP Broadcast mechanism to determine and maintain a broadcast group . It only needs a simple configuration to automatically add and remove members from the group . You don't need any knowledge of optimizing servers in the cluster , This is recommended by default .

Members send one... To the group every second “ heartbeat ”. If a member 5 It will be removed by the group if no signal is sent for seconds . If a new member sends a “ heartbeat ” It will be added to the group .

Anyone who has replication installed with this configuration cache Will be discovered by other members and identified as available .

To set up automatic member discovery , You need to specify the ehcache In profile cacheManagerPeerProviderFactory Elemental properties attribute , It looks like this ：
peerDiscovery=automatic

multicastGroupAddress=multicast address | multicast host name

multicastGroupPort=port

timeToLive=0-255 (timeToLive Properties are described in the FAQ section )

Example

Suppose you have two servers in a cluster . You want to sync sampleCache1 and sampleCache2. Each independent server should have such a configuration ：
To configure server1 and server2

<cacheManagerPeerProviderFactory

class="net.sf.ehcache.distribution.RMICacheManagerPeerProviderFactory"

properties="peerDiscovery=automatic, multicastGroupAddress=230.0.0.1,

multicastGroupPort=4446, timeToLive=32"/>

Manual member discovery

For manual member configuration, you need to know the of each listener IP Address and port . Members cannot be dynamically added and removed at run time . When it is technically difficult to use broadcast, you can manually discover members , For example, there is a router between servers in the cluster that cannot transmit broadcast packets . You can also use manual member discovery for one-way data replication , Only let server2 know server1, and server1 I do not know! server2.

Configure manual member discovery , You need to specify the ehcache In profile cacheManagerPeerProviderFactory Of properties attribute , Like this ：
peerDiscovery=manual rmiUrls=//server:port/cacheName, //server:port/cacheName ...
rmiUrls The server is configured cache peers A list of . Be careful not to duplicate the configuration .

Example

Suppose you have two servers in a cluster . You want to sync sampleCache1 and sampleCache2. The following is the configuration required for each server ：
To configure server1

<cacheManagerPeerProviderFactory

class="net.sf.ehcache.distribution.RMICacheManagerPeerProviderFactory"

properties="peerDiscovery=manual,

rmiUrls=//server2:40001/sampleCache11|//server2:40001/sampleCache12"/>

To configure server2

<cacheManagerPeerProviderFactory

class="net.sf.ehcache.distribution.RMICacheManagerPeerProviderFactory"

properties="peerDiscovery=manual,

rmiUrls=//server1:40001/sampleCache11|//server1:40001/sampleCache12"/>

To configure CacheManagerPeerListener

Every CacheManagerPeerListener Listening from members to the current CacheManager The news of . To configure CacheManagerPeerListener One needs to be specified CacheManagerPeerListenerFactory, It is implemented as a plug-in , Used to create CacheManagerPeerListener.

cacheManagerPeerListenerFactory The attributes of are ：
class – A complete factory class name .
properties – Attributes that only make sense for this factory , Separated by commas .

Ehcache There is a built-in based on RMI Distribution system . Its listener is RMICacheManagerPeerListener, This listener can be used

RMICacheManagerPeerListenerFactory To configure the .

<cacheManagerPeerListenerFactory

class="net.sf.ehcache.distribution.RMICacheManagerPeerListenerFactory"

properties="hostName=localhost, port=40001,

socketTimeoutMillis=2000"/>

The valid attributes are ：
hostname ( Optional ) – Name of the server running the listener . Indicates the address of the member of the cluster group , It is also the interface you want to control to receive messages from the cluster .

stay CacheManager During initialization, you will check hostname Is it available .

If hostName Unavailable ,CacheManager The startup will be rejected and an exception will be thrown if the connection is rejected .

If specified ,hostname Will use InetAddress.getLocalHost().getHostAddress() To get .

Warning ： Don't put localhost Configure as local address 127.0.0.1, Because it is not visible in the network, it will not be able to receive information from the remote server and therefore cannot be copied . There are multiple... On the same machine CacheManager When , You should only use localhost To configure the .

port – The port on which the listener listens .
socketTimeoutMillis ( Optional ) – Socket Time out . The default is 2000ms. When you socket The synchronous cache request address is far away , Not a local area network . You may need to make this time larger , Otherwise, the synchronization cache may fail due to delay .

To configure CacheReplicators

Each to be synchronized cache You need to set one to send a message to CacheManagerr Cached event listeners for member replication messages of . This work has to be done for each cache Add one to the configuration of cacheEventListenerFactory Element to complete .

<!-- Sample cache named sampleCache2. -->

<cache name="sampleCache2"

maxElementsInMemory="10"

eternal="false"

timeToIdleSeconds="100"

timeToLiveSeconds="100"

overflowToDisk="false">

<cacheEventListenerFactory class="net.sf.ehcache.distribution.RMICacheReplicatorFactory"

properties="replicateAsynchronously=true,replicatePuts=true, replicateUpdates=true, replicateUpdatesViaCopy=false, replicateRemovals=true "/>

</cache>

class – Use net.sf.ehcache.distribution.RMICacheReplicatorFactory

This factory supports the following properties ：
replicatePuts=true | false – When a new element is added to the cache, whether to copy it to other peers. The default is true.
replicateUpdates=true | false – Whether to copy an element that already exists in the cache when it is covered . The default is true.
replicateRemovals= true | false – Whether to copy when the element is removed . The default is true.
replicateAsynchronously=true | false – Replication is asynchronous （ Designated as true when ） It's still synchronous （ Designated as false when ）. The default is true.
replicatePutsViaCopy=true | false – When a new element is copied to another cache Whether or not to copy is specified as true Copy , The default is true.
replicateUpdatesViaCopy=true | false – When an element is copied to another cache Whether or not to copy （ Designated as true Copy ）, The default is true.

You can use ehcache To reduce the workload of configuration , The default behavior is to copy everything asynchronously ; You can reduce as in the following example RMICacheReplicatorFactory Property configuration for ：

<!-- Sample cache named sampleCache4. All missing RMICacheReplicatorFactory properties default to true -->

<cache name="sampleCache4"

maxElementsInMemory="10"

eternal="true"

overflowToDisk="false"

memoryStoreEvictionPolicy="LFU">

<cacheEventListenerFactory class="net.sf.ehcache.distribution.RMICacheReplicatorFactory"/>

</cache>

 

Common questions

Windows Upper Tomcat

There is one Tomcat Or is it JDK Of bug, stay tomcat If tomcat If there are spaces in the installation path of , When it starts RMI The listener will fail . See http://archives.java.sun.com/cgi-bin/wa?A2=ind0205&L=rmi-users&P=797 and http://www.ontotext.com/kim/doc/sys-doc/faq-howto-bugs/known-bugs.html.

Because in Windows Installation on Tomcat The default is installed in “Program Files” In a folder , So this problem often happens .

Broadcast blocking

automatic peer discovery It is closely related to broadcasting . The broadcast may be blocked by the route , image Xen and VMWare This virtualization technology can also block the broadcast . If these are all turned on , You may also want to open the configuration of your network card . A simple way to tell whether the broadcast is effective ,

That's using ehcache remote debugger Look at “ heartbeat ” Is it available .

The radio doesn't go far enough or it goes too far

You can set up badly misnamed time to live To control the distance of broadcasting . Use the radio IP When the agreement ,timeToLive The value of is the field or range that the packet can pass . The agreement is as follows ：

0 Is limited to the same server

1 Is restricted to the same subnet

32 Is limited to the same website

64 Is limited to the same region

128 Is limited to the same continent

255 There is no limit

Translator press ： The translation of the above materials is not accurate enough , Please find the original text by yourself .

stay Java The default value in the implementation is 1, That is, it spreads in the same subnet . change timeToLive Attributes can limit or extend the scope of propagation .

 

3、 ... and 、 RMI Mode cache cluster / Configure distributed cache

RMI yes Java A remote method call technology of , It's a point-to-point, based on Java The communication mode of the object .EhCache from 1.2 From the beginning, the version supports RMI Cache cluster based on . In a clustered environment EhCache The keys and values of all cache objects must be serializable , That is, it must be realized java.io.Serializable Interface , This also needs to be observed in other cluster modes .

 

     The picture below is RMI Structure diagram of cluster mode ：

      

use RMI In cluster mode , Each node in the cluster is a peer relationship , There is no concept of a master node or a slave node , Therefore, there must be a mechanism between nodes to know each other , You must know the information of other nodes , Including the host address 、 Port number, etc .EhCache There are two ways to discover nodes ： Manual configuration and automatic discovery . The manual configuration mode requires configuring the connection information of all other nodes in each node , Once the nodes in the cluster change , The cache needs to be reconfigured .

because RMI yes Java Built in supported technologies in , Therefore use RMI In cluster mode , There is no need to introduce other Jar package ,EhCache It comes with support RMI The function of clustering . Use RMI The cluster mode needs to be in ehcache.xml Defined in the configuration file cacheManagerPeerProviderFactory node .

 

      The distributed synchronous cache should make the cache Know each other's cache, be called Peer Discovery（ Members found that ） EHCache There are two ways to implement member discovery ：

1、 Manual search

A、 stay ehcache.xml Middle configuration PeerDiscovery Member discovery object

      Server1 To configure , Configure local hostName、port yes 400001, Monitor separately 192.168.8.32:400002 Of mobileCache and 192.168.5.231:400003 Of mobileCache. Notice the mobileCache Is the name of the cache , Corresponding to server2、server3 Of cache Configuration of .

<?xml version="1.0" encoding="gbk"?>

<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="ehcache.xsd">

 <diskStore path="java.io.tmpdir"/>

 <!-- 

  Cluster caches in multiple servers , Here is to synchronize the cache of some servers 

 server1 hostName:192.168.8.9 port:400001 cacheName:mobileCache

 server2 hostName:192.168.8.32 port:400002 cacheName:mobileCache

 server3 hostName:192.168.8.231 port:400003 cacheName:mobileCache

  Be careful ： Of each server to synchronize the cache RMI signal communication socket Ports are different , Pay attention to settings during configuration 

 -->

 <!-- server1  Of cacheManagerPeerProviderFactory To configure  -->

 <cacheManagerPeerProviderFactory 

 class="net.sf.ehcache.distribution.RMICacheManagerPeerProviderFactory" 

 properties="hostName=localhost,

 port=400001,

 socketTimeoutMillis=2000,

 peerDiscovery=manual,

 rmiUrls=//192.168.8.32:400002/mobileCache|//192.168.5.231:400003/mobileCache"

 />

</ehcache>

Note above cacheManagerPeerProviderFactory The element appears in diskStore Next

Also in your other 2 Add configuration on servers

Server2, Configure local host,port by 400002, Synchronize separately 192.168.8.9:400001 Of mobileCache and 192.168.5.231:400003 Of mobileCache

<!-- server2  Of cacheManagerPeerProviderFactory To configure  -->

<cacheManagerPeerProviderFactory 

 class="net.sf.ehcache.distribution.RMICacheManagerPeerProviderFactory" 

 properties="hostName=localhost,

 port=400002,

 socketTimeoutMillis=2000,

 peerDiscovery=manual,

 rmiUrls=//192.168.8.9:400001/mobileCache|//192.168.5.231:400003/mobileCache"

/>

Server3, Configure local host,port by 400003, Synchronize separately 192.168.8.9:400001 Of mobileCache Cache and 192.168.8.32:400002 Of mobileCache cache

<!-- server3  Of cacheManagerPeerProviderFactory To configure  -->

<cacheManagerPeerProviderFactory 

 class="net.sf.ehcache.distribution.RMICacheManagerPeerProviderFactory" 

 properties="hostName=localhost,

 port=400003,

 socketTimeoutMillis=2000,

 peerDiscovery=manual,

 rmiUrls=//192.168.8.9:400001/mobileCache|//192.168.8.32:400002/mobileCache"

/>

This configures manual discovery on three different servers cache Of PeerProvider The configuration discovered by the member . It is worth noting that you are configuring rmiUrls Pay special attention to url It can't be repeated , And port 、 The addresses are all right .

If specified ,hostname Will use InetAddress.getLocalHost().getHostAddress() To get .

Warning ： Don't put localhost Configure as local address 127.0.0.1, Because it is not visible in the network, it will not be able to receive information from the remote server and therefore cannot be copied . There are multiple... On the same machine CacheManager When , You should only use localhost To configure the .

B、 Next, configure cache and cache synchronization listening , It needs to be in each server ehcache.xml Add to file cache Configuration and cacheEventListenerFactory、cacheLoaderFactory Configuration of

<defaultCache maxElementsInMemory="10000" eternal="false" timeToIdleSeconds="30" timeToLiveSeconds="30" overflowToDisk="false"/>

<!-- 

  Configure custom cache 

 maxElementsInMemory： Maximum number of objects allowed to be created in the cache 

 eternal： Whether the objects in the cache are permanent , If it is , Timeout will be ignored , Objects never expire .

 timeToIdleSeconds： The maximum time the cache data is idle , In other words, if a cache has not been accessed for a long time, it will be destroyed , If the value is  0  It means that elements can pause for an infinite amount of time .

 timeToLiveSeconds： Cache data lifetime , The maximum lifetime of a cache object , After that time, it will be destroyed , This only works if the element is not permanently resident , If the value is 0 It means that elements can pause for an infinite amount of time .

 overflowToDisk： When there is not enough memory , Whether to enable disk caching .

 memoryStoreEvictionPolicy： The elimination algorithm after the cache is full .

  Update the cache every hour （1 Hours expired ） 

-->

<cache name="mobileCache"

 maxElementsInMemory="10000"

 eternal="false"

 overflowToDisk="true"

 timeToIdleSeconds="1800"

 timeToLiveSeconds="3600"

 memoryStoreEvictionPolicy="LFU">

 <!-- 

 RMI Cache distribution synchronization lookup  class Use net.sf.ehcache.distribution.RMICacheReplicatorFactory

  This factory supports the following properties ：

 replicatePuts=true | false –  When a new element is added to the cache, whether to copy it to other peers. The default is true.

 replicateUpdates=true | false –  Whether to copy an element that already exists in the cache when it is covered . The default is true.

 replicateRemovals= true | false –  Whether to copy when the element is removed . The default is true.

 replicateAsynchronously=true | false –  Replication is asynchronous  Designated as true when , It's still synchronous , Designated as false when . The default is true.

 replicatePutsViaCopy=true | false –  When a new element is copied to another cache Whether or not to copy  Designated as true Copy , The default is true.

 replicateUpdatesViaCopy=true | false –  When an element is copied to another cache Whether or not to copy  Designated as true Copy , The default is true.

 asynchronousReplicationIntervalMillis=1000

 -->

 <!--  monitor RMI Synchronize cache object configuration   Register the corresponding cache listening class , Used to handle cache events , Such as put,remove,update, and expire -->

 <cacheEventListenerFactory

 class="net.sf.ehcache.distribution.RMICacheReplicatorFactory"

 properties="replicateAsynchronously=true, 

 replicatePuts=true, 

 replicateUpdates=true,

 replicateUpdatesViaCopy=false, 

 replicateRemovals=true "/>

 <!--  Used to initialize the cache , And automatic setting  -->

 <bootstrapCacheLoaderFactory class="net.sf.ehcache.bootstrap.BootstrapCacheLoaderFactory"/>

</cache>

C、 That's it 3 Configuration of servers , Here's how server1 Complete ehcache.xml Configuration of

<?xml version="1.0" encoding="gbk"?>

<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="ehcache.xsd">

 <diskStore path="java.io.tmpdir"/>

 <!-- 

  Cluster caches in multiple servers , Here is to synchronize the cache of some servers 

 server1 hostName:192.168.8.9 port:400001 cacheName:mobileCache

 server2 hostName:192.168.8.32 port:400002 cacheName:mobileCache

 server3 hostName:192.168.8.231 port:400003 cacheName:mobileCache

  Note the of each server to synchronize the cache RMI signal communication socket Ports are different , Pay attention to settings during configuration 

 -->

 <!-- server1  Of cacheManagerPeerProviderFactory To configure  -->

 <cacheManagerPeerProviderFactory 

 class="net.sf.ehcache.distribution.RMICacheManagerPeerProviderFactory" 

 properties="hostName=localhost,

 port=400001,

 socketTimeoutMillis=2000,

 peerDiscovery=manual,

 rmiUrls=//192.168.8.32:400002/mobileCache|//192.168.5.231:400003/mobileCache"

 />

 <defaultCache maxElementsInMemory="10000" eternal="false" timeToIdleSeconds="30" timeToLiveSeconds="30" overflowToDisk="false"/>

 <!-- 

  Configure custom cache 

 maxElementsInMemory： Maximum number of objects allowed to be created in the cache 

 eternal： Whether the objects in the cache are permanent , If it is , Timeout will be ignored , Objects never expire .

 timeToIdleSeconds： The maximum time the cache data is idle , In other words, if a cache has not been accessed for a long time, it will be destroyed ,

  If the value is  0  It means that elements can pause for an infinite amount of time .

 timeToLiveSeconds： Cache data lifetime , The maximum lifetime of a cache object , After that time, it will be destroyed ,

  This only works if the element is not permanently resident , If the value is 0 It means that elements can pause for an infinite amount of time .

 overflowToDisk： When there is not enough memory , Whether to enable disk caching .

 memoryStoreEvictionPolicy： The elimination algorithm after the cache is full .

  Update the cache every hour （1 Hours expired ） 

 -->

 <cache name="mobileCache"

 maxElementsInMemory="10000"

 eternal="false"

 overflowToDisk="true"

 timeToIdleSeconds="1800"

 timeToLiveSeconds="3600"

 memoryStoreEvictionPolicy="LFU">

 <!-- 

 RMI Cache distribution synchronization lookup  class Use net.sf.ehcache.distribution.RMICacheReplicatorFactory

  This factory supports the following properties ：

 replicatePuts=true | false –  When a new element is added to the cache, whether to copy it to other peers. The default is true.

 replicateUpdates=true | false –  Whether to copy an element that already exists in the cache when it is covered . The default is true.

 replicateRemovals= true | false –  Whether to copy when the element is removed . The default is true.

 replicateAsynchronously=true | false –  Replication is asynchronous  Designated as true when , It's still synchronous , Designated as false when . The default is true.

 replicatePutsViaCopy=true | false –  When a new element is copied to another cache Whether or not to copy  Designated as true Copy , The default is true.

 replicateUpdatesViaCopy=true | false –  When an element is copied to another cache Whether or not to copy  Designated as true Copy , The default is true.

 asynchronousReplicationIntervalMillis=1000

 -->

 <!--  monitor RMI Synchronize cache object configuration   Register the corresponding cache listening class , Used to handle cache events , Such as put,remove,update, and expire -->

 <cacheEventListenerFactory

 class="net.sf.ehcache.distribution.RMICacheReplicatorFactory"

 properties="replicateAsynchronously=true, 

 replicatePuts=true, 

 replicateUpdates=true,

 replicateUpdatesViaCopy=false, 

 replicateRemovals=true "/>

 <!--  Used to initialize the cache , And automatic setting  -->

 <bootstrapCacheLoaderFactory class="net.sf.ehcache.bootstrap.BootstrapCacheLoaderFactory"/>

 </cache>

</ehcache>

2、 Auto discovery

Automatic discovery configuration is a little different from manual discovery , Other places are basically the same . Also in ehcache.xml Add configuration , The configuration is as follows

<!--

 Search the cache on a network segment 

timeToLive

 0 Is limited to the same server 

 1 Is restricted to the same subnet 

 32 Is limited to the same website 

 64 Is limited to the same region

 128 Is limited to the same continent 

 255 There is no limit 

-->

<cacheManagerPeerProviderFactory

 class="net.sf.ehcache.distribution.RMICacheManagerPeerProviderFactory"

 properties="peerDiscovery=automatic, multicastGroupAddress=192.168.0.1,

 multicastGroupPort=400004, timeToLive=32"

/>

Other configurations are the same as those for manual search , I won't go into that here . About ehcache Other cache configuration methods will not be introduced here , You can study it yourself . You can refer to ：