当前位置:网站首页>SPI primary key generation strategy for shardingsphere JDBC

SPI primary key generation strategy for shardingsphere JDBC

2022-06-13 05:46:00 Coffee is not bitter**


ShardingSphere The primary key generation strategy is to use the snowflake algorithm and UUID Two ways . These two are mainly through SPI The way to achieve , Its main purpose is also to expand , You can also implement the interface by yourself , Custom primary key generation policy .

One 、 What is the SPI?

details java SPI Detailed explanation of mechanism

Two 、 Source code parsing primary key generation strategy

1) Source code search path map

 Insert picture description here

2) The source code parsing

shardingKeyGenerator = containsKeyGeneratorConfiguration(tableRuleConfig)
                ? new ShardingKeyGeneratorServiceLoader().newService(tableRuleConfig.getKeyGeneratorConfig().getType(), tableRuleConfig.getKeyGeneratorConfig().getProperties()) : null;

The logic code above , analysis , If the partition algorithm value can be obtained from the configuration file , Through ShardingKeyGeneratorServiceLoader To parse the generation policy , Otherwise it just returns null.

public final class ShardingKeyGeneratorServiceLoader extends TypeBasedSPIServiceLoader<ShardingKeyGenerator> {
    
    
    static {
    
        //SPI:  Load the primary key generation policy 
        NewInstanceServiceLoader.register(ShardingKeyGenerator.class);
    }
    
    public ShardingKeyGeneratorServiceLoader() {
    
        super(ShardingKeyGenerator.class);
    }
}

Let's drive ShardingKeyGeneratorServiceLoader see , He is registered ShardingKeyGenerator class , In fact, it is to register all subclasses , That is, load the implementation class into memory and save it . The registration process is SPI Place reflected .
 Insert picture description here
adopt SPI What classes are loaded ? Let's see ShardingKeyGenerator, according to ShardingKeyGenerator To find the configuration file :
 Insert picture description here

We found two implementation classes loaded in the file , One is SNOWFLAKE, One is UUID.
Put one of them SnowflakeShardingKeyGenerator Click in to see , Found that the parent class has two methods :

  • getType()
  • generateKey()
    getType The returned string is the primary key generation policy configured in the configuration file .

3)UUID

open UUID The primary key generation implementation class of UUIDShardingKeyGenerator, We found that its generation rule is only UUID.randomUUID() Such a line of code , forehead ~ The feeling of silence in my heart is a little too simple !
UUID Although we can achieve global uniqueness , But it is still not recommended as a primary key , Because we do business in real life, whether it is user_id still order_id Primary keys are mostly integers , and UUID It's a 32 Bit string .
Its storage and query to MySQL High performance consumption , and MySQL Officials have also made clear their recommendations , The primary key should be as short as possible , As a database primary key UUID The disorder of the data also leads to frequent changes in data locations , Seriously affect performance . In a word, it is actually : In order to ensure performance while saving resources , Try not to take UUID.


/** * UUID key generator. */
@Getter
@Setter
public final class UUIDShardingKeyGenerator implements ShardingKeyGenerator {
    
    
    private Properties properties = new Properties();
    
    @Override
    public String getType() {
    
        return "UUID";
    }
    
    @Override
    public synchronized Comparable<?> generateKey() {
    
        return UUID.randomUUID().toString().replaceAll("-", "");
    }
}

4)SNOWFLAKE

SNOWFLAKE Is the default primary key generation scheme , Generate a 64bit Long integer (Long) data .
It can ensure that the primary keys of different processes are not repeated , Ordering of primary keys of the same process . Binary form contains 4 part , The sub table from high to low is :1bit Sign bit 、41bit Time stamp bit 、10bit Working process bit and 12bit Serial number position .
 Insert picture description here

Let's see generateKey Method , At a glance, it is all time related logic , This is an algorithm that depends heavily on server time , And those who rely on server time will encounter a thorny problem : Clock back .

There is a network time protocol in the Internet ntp Full name (Network Time Protocol) , Specifically used to synchronize 、 Calibrate the time of each computer in the network . This is why our smart phones don't need to set the watch , But time is the same .
Server clock callback can cause duplicate ID,SNOWFLAKE In the scheme, the original snowflake algorithm is improved , Added a maximum tolerated clock callback in milliseconds .
If the time of clock callback exceeds the maximum tolerance threshold of milliseconds , The program reports an error directly ; If it's within tolerance , Default distributed primary key generator , It will wait for the clock to synchronize to the time of the last primary key generation before continuing to work .
Maximum number of clock callback milliseconds tolerated , The default value is 0, You can use the properties max.tolerate.time.difference.milliseconds Set up .

#  Maximum number of clock callback milliseconds tolerated 
spring.shardingsphere.sharding.tables.course.key-generator.max.tolerate.time.difference.milliseconds=5
@Override
    public synchronized Comparable<?> generateKey() {
    
        // Current system time in milliseconds 
        long currentMilliseconds = timeService.getCurrentMillis();
        // Judge whether the waiting time is poor , if necessary , Then the waiting time goes by , And then get the current system time 
        if (waitTolerateTimeDifferenceIfNeed(currentMilliseconds)) {
    
            currentMilliseconds = timeService.getCurrentMillis();
        }
        // If the last millisecond and   The current system time is the same in milliseconds , In the same millisecond 
        if (lastMilliseconds == currentMilliseconds) {
    
            /** * & Bit and operator : Both numbers are binary , If all the corresponding bits are 1, The result is 1, Otherwise 0 *  When the sequence is 4095 when ,4095+1 After the new sequence and mask bit and operation, the result is 0 *  When the sequence is other values , Neither the bits nor the result of the operation will be 0 *  That is, the maximum value has been used in this millisecond sequence 4096, At this time, you need to take down a millisecond time value  */
            if (0L == (sequence = (sequence + 1) & SEQUENCE_MASK)) {
    
                currentMilliseconds = waitUntilNextTime(currentMilliseconds);
            }
        } else {
    
            // The last millisecond has passed , Reset the sequence value to -1
            vibrateSequenceOffset();
            sequence = sequenceOffset;
        }
        lastMilliseconds = currentMilliseconds;
        /** * XX......XX XX000000 00000000 00000000  Time difference  XX * XXXXXX XXXX0000 00000000  machine ID XX * XXXX XXXXXXXX  Serial number  XX *  Three parts | Bitwise OR operation : If all the corresponding bits are 0, The result is 0, Otherwise 1 */
        return ((currentMilliseconds - EPOCH) << TIMESTAMP_LEFT_SHIFT_BITS) | (getWorkerId() << WORKER_ID_LEFT_SHIFT_BITS) | sequence;
    }

3、 ... and 、 Custom primary key generation policy

In fact, it is relatively simple to implement a custom primary key generator , There are only two steps .

1) First step : Realization ShardingKeyGenerator Interface , And rewrite its internal methods

Use time to obtain the primary key and increase it automatically ID, Joined the Atomic Ensure the atomicity of high concurrence .

/** * @description: Custom primary key generation strategy  * @author: huoyajing * @time: 2021/12/22 11:44  In the morning  */

public class MyKeyGenerator implements ShardingKeyGenerator {
    

	/** *  Add atomic operation , Avoid high and give duplicate values  */
	private AtomicLong atomicLong=new AtomicLong();

	/** *  The core approach - Generate primary key ID * @return */
	@Override
	public Comparable<?> generateKey() {
    
		LocalDateTime localDateTime=LocalDateTime.now();
		String valueString = DateTimeFormatter.ofPattern("HHmmssSSS").format(localDateTime);
		return Long.parseLong(valueString+atomicLong.incrementAndGet());
	}

	/** *  Custom generation scheme type  * @return */
	@Override
	public String getType() {
    
		return "DATETIMEKEY";
	}

	@Override
	public Properties getProperties() {
    
		return null;
	}

	@Override
	public void setProperties(Properties properties) {
    

	}
}

2)META-INF/services Configure the customized primary key generation policy path in the file

Do not write wrong file path :

org.apache.shardingsphere.spi.keygen.ShardingKeyGenerator

Add our own customized primary key generation policy class path to it

com.huohuo.sharding.service.generator.MyKeyGenerator

 Insert picture description here

3) test

Revise it key-generator.type, Change to custom value

spring.shardingsphere.sharding.tables.course.key-generator.column=id
spring.shardingsphere.sharding.tables.course.key-generator.type=DATETIMEKEY

Test class :

@Test
	public void addCourse() {
    
		for (int i = 1; i <= 10; i++) {
    
			Course c = new Course();
			c.setName("shardingsphere");
			c.setType(Long.valueOf(i));
			courseMapper.insert(c);
		}
	}

Four 、 summary

Why do you want to implement a custom primary key generation policy , Through analysis, we know whether it is snow or UUID, Poor readability , We can customize the implementation of policies with business attributes .

原网站

版权声明
本文为[Coffee is not bitter**]所创,转载请带上原文链接,感谢
https://yzsam.com/2022/02/202202280507384909.html