Didi's distributed ID generator (tinyid), easy to use

Want to make APP Same thing as WeChat , Can run small programs smoothly ？ | Experience will send you to Xinjiang 、 Huawei 、 Cherry keyboard ！>>>

Don't understand distributed ID The students of the generator , Let's review the previous  《9 Species distributed ID generation 》

Tinyid It is a distributed model developed by Didi ID System ,Tinyid Is in Meituan （Leaf） Of leaf-segment The algorithm is based on the upgrade , It not only supports the database multi master mode , It also provides tinyid-client The access mode of the client , More convenient to use . But wameituan （Leaf） The difference is ,Tinyid Only support segments one mode does not support snowflake mode .

Tinyid Characteristics of

• Globally unique long type ID
• Increasing trend id
• Provide http and java-client Mode access
• Batch access is supported ID
• Support generation 1,3,5,7,9... Sequential ID
• Support multiple db Configuration of

Applicable scenario ： Only care about ID It's the number. , A system of increasing trends , Can tolerate ID Discontinuous , Can tolerate ID Waste of

Not applicable to the scene ： Like an order ID The business of , Because of the creation of ID Most of it is continuous , Easy to sweep 、 Or calculate the order quantity and other information

Tinyid principle

Tinyid It is based on segment mode , Let's talk about the principle of segment mode ： That is to get auto increment from the database in batches ID, One segment range at a time from the database , for example  (1,1000]  representative 1000 individual ID, The business service generates segments locally 1~1000 Self increasing of ID And load it into memory ..

Tinyid The available segments are loaded into memory , And generate... In memory ID, The available number segment is acquired for the first time ID Time to load , For example, when the use of the current segment reaches a certain proportion , The system will load the next available number segment asynchronously , This ensures that there are always available segments in memory , In order to be available for a period of time after the number sending service is down ID.

Schematic diagram is as follows ：

Tinyid Schematic diagram

Tinyid Realization

Tinyid Of GitHub Address ： https://github.com/didi/tinyid.git

Tinyid There are two ways to call , Based on the Tinyid-server Provided http The way , Another kind Tinyid-client Client mode . No matter which way to call , build Tinyid You have to build a watch in advance tiny_id_info、tiny_id_token.

CREATE TABLE `tiny_id_info` (
  `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT ' Since the primary key ',
  `biz_type` varchar(63) NOT NULL DEFAULT '' COMMENT ' Business types , only ',
  `begin_id` bigint(20) NOT NULL DEFAULT '0' COMMENT ' Start id, Record only the initial value , No other meaning . On initialization begin_id and max_id It should be the same ',
  `max_id` bigint(20) NOT NULL DEFAULT '0' COMMENT ' At present, the biggest id',
  `step` int(11) DEFAULT '0' COMMENT ' step ',
  `delta` int(11) NOT NULL DEFAULT '1' COMMENT ' Every time id The incremental ',
  `remainder` int(11) NOT NULL DEFAULT '0' COMMENT ' remainder ',
  `create_time` timestamp NOT NULL DEFAULT '2010-01-01 00:00:00' COMMENT ' Creation time ',
  `update_time` timestamp NOT NULL DEFAULT '2010-01-01 00:00:00' COMMENT ' Update time ',
  `version` bigint(20) NOT NULL DEFAULT '0' COMMENT ' Version number ',
  PRIMARY KEY (`id`),
  UNIQUE KEY `uniq_biz_type` (`biz_type`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8 COMMENT 'id Information sheet ';

CREATE TABLE `tiny_id_token` (
  `id` int(11) unsigned NOT NULL AUTO_INCREMENT COMMENT ' Self increasing id',
  `token` varchar(255) NOT NULL DEFAULT '' COMMENT 'token',
  `biz_type` varchar(63) NOT NULL DEFAULT '' COMMENT ' this token Accessible business type ID ',
  `remark` varchar(255) NOT NULL DEFAULT '' COMMENT ' remarks ',
  `create_time` timestamp NOT NULL DEFAULT '2010-01-01 00:00:00' COMMENT ' Creation time ',
  `update_time` timestamp NOT NULL DEFAULT '2010-01-01 00:00:00' COMMENT ' Update time ',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8 COMMENT 'token Information sheet ';

INSERT INTO `tiny_id_info` (`id`, `biz_type`, `begin_id`, `max_id`, `step`, `delta`, `remainder`, `create_time`, `update_time`, `version`)
VALUES
 (1, 'test', 1, 1, 100000, 1, 0, '2018-07-21 23:52:58', '2018-07-22 23:19:27', 1);

INSERT INTO `tiny_id_info` (`id`, `biz_type`, `begin_id`, `max_id`, `step`, `delta`, `remainder`, `create_time`, `update_time`, `version`)
VALUES
 (2, 'test_odd', 1, 1, 100000, 2, 1, '2018-07-21 23:52:58', '2018-07-23 00:39:24', 3);


INSERT INTO `tiny_id_token` (`id`, `token`, `biz_type`, `remark`, `create_time`, `update_time`)
VALUES
 (1, '0f673adf80504e2eaa552f5d791b644c', 'test', '1', '2017-12-14 16:36:46', '2017-12-14 16:36:48');

INSERT INTO `tiny_id_token` (`id`, `token`, `biz_type`, `remark`, `create_time`, `update_time`)
VALUES
 (2, '0f673adf80504e2eaa552f5d791b644c', 'test_odd', '1', '2017-12-14 16:36:46', '2017-12-14 16:36:48');

tiny_id_info Table is the data table of specific business party number segment information max_id ： Maximum value of segment

step： step , That is the length of the segment

biz_type： Business types

Segment acquisition right max_id Field once update operation ,update max_id= max_id + step, If the update is successful, the new segment is successful , The new segment range is (max_id ,max_id +step].

tiny_id_token It's a list of permissions , At present token Which business segment information can be operated .

modify tinyid-server in  \offline\application.properties  File configuration database , because tinyid Support more databases master Pattern , You can configure multiple database information . start-up  TinyIdServerApplication  Test it .

datasource.tinyid.primary.driver-class-name=com.mysql.jdbc.Driver
datasource.tinyid.primary.url=jdbc:mysql://127.0.0.1:3306/xin-master?autoReconnect=true&useUnicode=true&characterEncoding=UTF-8
datasource.tinyid.primary.username=junkang
datasource.tinyid.primary.password=junkang
datasource.tinyid.primary.testOnBorrow=false
datasource.tinyid.primary.maxActive=10

datasource.tinyid.secondary.driver-class-name=com.mysql.jdbc.Driver
datasource.tinyid.secondary.url=jdbc:mysql://localhost:3306/db2?autoReconnect=true&useUnicode=true&characterEncoding=UTF-8
datasource.tinyid.secondary.username=root
datasource.tinyid.secondary.password=123456
datasource.tinyid.secondary.testOnBorrow=false
datasource.tinyid.secondary.maxActive=10

1、Http The way

tinyid There are four inside http Interface to get ID And section .

package com.xiaoju.uemc.tinyid.server.controller;

/**
 * @author du_imba
 */


@RestController
@RequestMapping("/id/")
public class IdContronller {

    private static final Logger logger = LoggerFactory.getLogger(IdContronller.class);
    @Autowired
    private IdGeneratorFactoryServer idGeneratorFactoryServer;
    @Autowired
    private SegmentIdService segmentIdService;
    @Autowired
    private TinyIdTokenService tinyIdTokenService;
    @Value("${batch.size.max}")
    private Integer batchSizeMax;

    @RequestMapping("nextId")
    public Response<List<Long>> nextId(String bizType, Integer batchSize, String token) {
        Response<List<Long>> response = new Response<>();
        try {
            IdGenerator idGenerator = idGeneratorFactoryServer.getIdGenerator(bizType);
            List<Long> ids = idGenerator.nextId(newBatchSize);
            response.setData(ids);
        } catch (Exception e) {
            response.setCode(ErrorCode.SYS_ERR.getCode());
            response.setMessage(e.getMessage());
            logger.error("nextId error", e);
        }
        return response;
    }

    

    @RequestMapping("nextIdSimple")
    public String nextIdSimple(String bizType, Integer batchSize, String token) {
        String response = "";
        try {
            IdGenerator idGenerator = idGeneratorFactoryServer.getIdGenerator(bizType);
            if (newBatchSize == 1) {
                Long id = idGenerator.nextId();
                response = id + "";
            } else {
                List<Long> idList = idGenerator.nextId(newBatchSize);
                StringBuilder sb = new StringBuilder();
                for (Long id : idList) {
                    sb.append(id).append(",");
                }
                response = sb.deleteCharAt(sb.length() - 1).toString();
            }
        } catch (Exception e) {
            logger.error("nextIdSimple error", e);
        }
        return response;
    }

    @RequestMapping("nextSegmentId")
    public Response<SegmentId> nextSegmentId(String bizType, String token) {
        try {
            SegmentId segmentId = segmentIdService.getNextSegmentId(bizType);
            response.setData(segmentId);
        } catch (Exception e) {
            response.setCode(ErrorCode.SYS_ERR.getCode());
            response.setMessage(e.getMessage());
            logger.error("nextSegmentId error", e);
        }
        return response;
    }

    @RequestMapping("nextSegmentIdSimple")
    public String nextSegmentIdSimple(String bizType, String token) {
        String response = "";
        try {
            SegmentId segmentId = segmentIdService.getNextSegmentId(bizType);
            response = segmentId.getCurrentId() + "," + segmentId.getLoadingId() + "," + segmentId.getMaxId()
                    + "," + segmentId.getDelta() + "," + segmentId.getRemainder();
        } catch (Exception e) {
            logger.error("nextSegmentIdSimple error", e);
        }
        return response;
    }

}

nextId、nextIdSimple It's all about getting the next ID,nextSegmentIdSimple、getNextSegmentId Is to get the next available number segment . The difference is whether the interface has a return state .

nextId:
'http://localhost:9999/tinyid/id/nextId?bizType=test&token=0f673adf80504e2eaa552f5d791b644c'
response ：
{
 "data": [2],
 "code": 200,
 "message": ""
}

nextId Simple:
'http://localhost:9999/tinyid/id/nextIdSimple?bizType=test&token=0f673adf80504e2eaa552f5d791b644c'
response: 3

2、Tinyid-client client

If you don't want to pass http The way ,Tinyid-client The client is also a good choice .

quote  tinyid-server package

<dependency>
    <groupId>com.xiaoju.uemc.tinyid</groupId>
    <artifactId>tinyid-client</artifactId>
    <version>${tinyid.version}</version>
</dependency>

start-up  tinyid-server Package the project and get  tinyid-server-0.1.0-SNAPSHOT.jar , Set version ${tinyid.version} by 0.1.0-SNAPSHOT.

In our project  application.properties  Middle configuration  tinyid-server Service request address and User's identity token

tinyid.server=127.0.0.1:9999
tinyid.token=0f673adf80504e2eaa552f5d791b644c```

stay Java Code calls TinyId It's also very simple. , Just one line of code .

//  According to the type of business   Get a single ID
Long id = TinyId.nextId("test");

//  According to the type of business   Batch acquisition 10 individual ID
List<Long> ids = TinyId.nextId("test", 10);    

Tinyid The source code implementation of the whole project is also relatively simple , Like interacting with database more directly jdbcTemplate Realization

@Override
public TinyIdInfo queryByBizType(String bizType) {
    String sql = "select id, biz_type, begin_id, max_id," +
            " step, delta, remainder, create_time, update_time, version" +
            " from tiny_id_info where biz_type = ?";
    List<TinyIdInfo> list = jdbcTemplate.query(sql, new Object[]{bizType}, new TinyIdInfoRowMapper());
    if(list == null || list.isEmpty()) {
        return null;
    }
    return list.get(0);
}

summary

Two methods are recommended Tinyid-client, This way, ID Generate... For the local , Length of section No (step) The longer the , Supported by qps The greater the , If the segment is set large enough , be qps Can be up to 1000w+. and tinyid-client  Yes tinyid-server  Access becomes low frequency , To reduce the server End pressure .