Myrpc version 4

RPC The concept of

Background knowledge

• RPC Basic concepts of , Core functions

common RPC frame

Duboo Basic function

1. Telematics
2. Transparent remote procedure call based on interface method
3. Load balancing
4. Service registry

RPC The process

client Call the remote method -> request serialize -> Protocol code -> Network transmission -> Server side -> Deserialization request -> Call the local method to get response -> serialize -> code ->……

---

Version iteration process

Catalog

from 0 At the beginning RPC Iterative process of ：

• version0 edition ： Complete a with less than 100 lines of code RPC Example
• version1 edition ： Improve the common message format （request,response）, The client's dynamic proxy completes the request Encapsulation of message formats
• version2 edition ： Support the server to expose multiple service interfaces , Server program abstraction , Normalization
• version3 edition ： Use a high-performance network framework netty The realization of network communication , And the refactoring of client code
• version4 edition ： Custom message format , Support multiple serialization methods （java Native , json…）
• version5 edition : Implementation of server registration and discovery ,zookeeper As a registry
• version6 edition : Implementation of load balancing strategy

---

4.MyRPC edition 4

Background knowledge

• Various serialization methods and comparisons ,（Java Native serialization , json,protobuf,kryo…）
• Custom protocol
• java IO understand
• TCP Stick package problem , Solution

Questions in this section

• How to design and complete your own protocol ?

  answer ： Realize it by yourself encode And decode

Upgrade process

In front of RPC In the version , We all use java Its own serialization method , In fact, the performance of this method is very low （ Serialized byte array , Decoding and encoding speed ）, And in netty Server side , We use netty It comes with an encoder , A simple transmission The length of the message （4 Bytes ）| Serialized data Formatted data . Here we want to customize our transmission format and codec .

The following is my preliminary design of my custom format , Read the message type first （Requst, Response, ping, pong）, Serialization mode （ Native , json,kryo, protobuf…）, Plus the message length ： Prevent sticking , Then read according to the length data

Message type （2Byte）	Serialization mode 2Byte	The length of the message 4Byte
Serialized Data….	Serialized Data…	Serialized Data….

Prerequisite treatment ： maven pom Introduce in the file fastjson package , RPCResponse You need to add DataType Field , Because other serialization methods （json） Can't get Data The type of ,

upgrade 1： Custom codec

Serializer interface ：

public interface Serializer {
    
    //  Serialize objects into byte arrays 
    byte[] serialize(Object obj);
    //  Deserialize from byte array to message ,  Use java The self-contained serialization method is not used messageType You can also get the corresponding object （ The serialized byte array contains class information ）
    //  For other methods, the message format should be specified , According to message Convert to the corresponding object 
    Object deserialize(byte[] bytes, int messageType);
    //  Returns the sequencer used , Which is 
    // 0：java It has its own serialization method , 1: json Serialization mode 
    int getType();
    //  Take out the serializer according to the sequence number , For the time being, there are two ways to implement , Other ways are needed , Just implement this interface 
    static Serializer getSerializerByCode(int code){
    
        switch (code){
    
            case 0:
                return new ObjectSerializer();
            case 1:
                return new JsonSerializer();
            default:
                return null;
        }
    }
}

encode class

/** *  In turn, it is written in the customized message format , The incoming data is request perhaps response *  Need to hold a serialize device , Responsible for serializing the incoming objects into byte arrays  */
@AllArgsConstructor
public class MyEncode extends MessageToByteEncoder {
    
    private Serializer serializer;
    
    @Override
    protected void encode(ChannelHandlerContext ctx, Object msg, ByteBuf out) throws Exception {
    
        System.out.println(msg.getClass());
        //  Write message type 
        if(msg instanceof RPCRequest){
    
            out.writeShort(MessageType.REQUEST.getCode());
        }
        else if(msg instanceof RPCResponse){
    
            out.writeShort(MessageType.RESPONSE.getCode());
        }
        //  Write serialization mode 
        out.writeShort(serializer.getType());
        //  Get the serialized array 
        byte[] serialize = serializer.serialize(msg);
        //  Write length 
        out.writeInt(serialize.length);
        //  Write serialized byte array 
        out.writeBytes(serialize);
    }
}

decode class

/** *  Decode the data according to the custom message format  */
@AllArgsConstructor
public class MyDecode extends ByteToMessageDecoder {
    
    @Override
    protected void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception {
    
        // 1.  Read message type 
        short messageType = in.readShort();
        //  Now it only supports request And response request 
        if(messageType != MessageType.REQUEST.getCode() &&
                messageType != MessageType.RESPONSE.getCode()){
    
            System.out.println(" This data is not supported at the moment ");
            return;
        }
        // 2.  Read serialized type 
        short serializerType = in.readShort();
        //  Get the corresponding serializer according to the type 
        Serializer serializer = Serializer.getSerializerByCode(serializerType);
        if(serializer == null)throw new RuntimeException(" There is no corresponding serializer ");
        // 3.  Read the byte length after data serialization 
        int length = in.readInt();
        // 4.  Read serialized array 
        byte[] bytes = new byte[length];
        in.readBytes(bytes);
        //  Decode the byte array with the corresponding serializer 
        Object deserialize = serializer.deserialize(bytes, messageType);
        out.add(deserialize);
    }
}

upgrade 2： Support for different serializers

Java Native serialization

public class ObjectSerializer implements Serializer{
    
    //  utilize java IO  object  ->  Byte array 
    @Override
    public byte[] serialize(Object obj) {
    
        byte[] bytes = null;
        ByteArrayOutputStream bos = new ByteArrayOutputStream();
        try {
    
            ObjectOutputStream oos = new ObjectOutputStream(bos);
            oos.writeObject(obj);
            oos.flush();
            bytes = bos.toByteArray();
            oos.close();
            bos.close();
        } catch (IOException e) {
    
            e.printStackTrace();
        }
        return bytes;
    }
    //  Byte array  ->  object 
    @Override
    public Object deserialize(byte[] bytes, int messageType) {
    
        Object obj = null;
        ByteArrayInputStream bis = new ByteArrayInputStream(bytes);
        try {
    
            ObjectInputStream ois = new ObjectInputStream(bis);
            obj = ois.readObject();
            ois.close();
            bis.close();
        } catch (IOException | ClassNotFoundException e) {
    
            e.printStackTrace();
        }
        return obj;
    }
    // 0  representative java Native serializer 
    @Override
    public int getType() {
    
        return 0;
    }
}

Json Serializer

/** *  because json Serialization is done by converting objects into strings , lost Data Class information of the object , therefore deserialize need  *  Understand the class information of the object , According to the class information JsonObject ->  The corresponding object  */
public class JsonSerializer implements Serializer{
    
    @Override
    public byte[] serialize(Object obj) {
    
        byte[] bytes = JSONObject.toJSONBytes(obj);
        return bytes;
    }
    @Override
    public Object deserialize(byte[] bytes, int messageType) {
    
        Object obj = null;
        //  The messages transmitted are divided into request And response
        switch (messageType){
    
            case 0:
                RPCRequest request = JSON.parseObject(bytes, RPCRequest.class);
                Object[] objects = new Object[request.getParams().length];
                //  hold json The string is converted to the corresponding object , fastjson Basic data types can be read , No conversion 
                for(int i = 0; i < objects.length; i++){
    
                    Class<?> paramsType = request.getParamsTypes()[i];
                    if (!paramsType.isAssignableFrom(request.getParams()[i].getClass())){
    
                        objects[i] = JSONObject.toJavaObject((JSONObject) request.getParams()[i],request.getParamsTypes()[i]);
                    }else{
    
                        objects[i] = request.getParams()[i];
                    }
                }
                request.setParams(objects);
                obj = request;
                break;
            case 1:
                RPCResponse response = JSON.parseObject(bytes, RPCResponse.class);
                Class<?> dataType = response.getDataType();
                if(! dataType.isAssignableFrom(response.getData().getClass())){
    
                    response.setData(JSONObject.toJavaObject((JSONObject) response.getData(),dataType));
                }
                obj = response;
                break;
            default:
                System.out.println(" This message is not supported for the time being ");
                throw new RuntimeException();
        }
        return obj;
    }
    
    // 1  Represents the json Serialization mode 
    @Override
    public int getType() {
    
        return 1;
    }
}

result

stay netty Initialization class （ client , Server side ） Add a decoder in the, and use the decoder defined by yourself ：

public class NettyClientInitializer extends ChannelInitializer<SocketChannel> {
    
    @Override
    protected void initChannel(SocketChannel ch) throws Exception {
    
        ChannelPipeline pipeline = ch.pipeline();
        //  Use a custom codec 
        pipeline.addLast(new MyDecode());
        //  Encoding requires an incoming serializer , Here is json, And support ObjectSerializer, You can also implement other 
        pipeline.addLast(new MyEncode(new JsonSerializer()));
        pipeline.addLast(new NettyClientHandler());
    }
}

Successful startup !

summary

In this version , The message format we define ourselves , Make it support multiple message types , Serialization mode , Use the message header plus length to solve the sticky packet problem

also , Realized ObjectSerializer And JsonSerializer Two kinds of serializers , It can also be easily extended to other serialization methods （ Realization Serialize Interface ）.

The biggest pain point of this version

• Communication between server and client host And port Must know in advance , Every client must know the corresponding service ip And port number , And if the service hangs up or changes its address , It's very troublesome. . Scalability is not strong

---