Im instant messaging development: Practice of offline messages and historical messages

In today's mobile Internet era ,IM Such products have become an indispensable part of our life . Like wechat 、 nailing 、QQ And so on are typical IM Social products with core functions . There are also some applications, although IM Function is not the core , but IM Capability is also an extremely important part of its whole application , Like online games 、 E-commerce live broadcast and other applications .

stay IM Under the premise of more and more extensive technology application scenarios , For instant messaging IM It is more and more necessary to learn and master technology .

stay IM In a huge technical system , The message system is undoubtedly the core , And in the message system , The most critical part is the distribution and storage of messages , Offline messages and historical messages are unavoidable technical points in this key link .

IM The general practice of message delivery

In general IM In the message system , For real-time messages 、 offline message 、 The historical news is probably the following technical ideas .

For online users ： Messages are sent directly to online recipients in real time , When the message is sent , The server does not store messages on the ground .

For offline users ： The server will store the message to the offline library , When the user logs in , Pull offline messages away from the offline Library , Then the server will delete the offline message .

The disadvantage of this implementation is that the message is not persistent , The message roaming cannot be supported , Reduces the reliability of messages .

（PS： actually , In fact, this is not a disadvantage , Because it is not necessary to store historical messages in some scenarios , The so-called message roaming capability is not necessary , For example, wechat .）

And in the message system we designed , As long as the server receives the message from the sender , While forwarding to the receiver, the message will also be stored on the ground in the offline database and historical message library , The landing of historical messages can also support message roaming and other related functions .

What are offline messages and historical messages ？

About offline messages and historical messages , On the technical , We define it this way .

1） offline message ：

Offline messages are users （ The receiver ） Messages received while offline , Most of these messages are concerned by users , It has certain timeliness .

In terms of our system experience , By default, our offline messages only save the messages of the last seven days .

user （ The receiver ） After the next login, you will get all these offline messages , Then, the client performs offline message processing according to the chat session UI Exhibition （ For example, display an unread message bubble ）.

（PS： The possibility of users offline is technically composed of many situations , For example, the other party is not online 、 The other party's network is broken 、 The other party's mobile phone crashed 、 The server made an error while sending, etc , Strictly speaking —— As long as the message cannot be sent in real time , All count “ offline message ”.）

Historical news ：

Historical messages store all chat messages of users , These messages include outgoing messages and received messages .

When the client gets the history message , It is usually obtained by paging according to the session .

In terms of our system experience , The storage time of historical messages is designed to be half a year by default , Of course, this time can be determined according to the actual product operation rules , There are no hard and fast rules .

When the user sends a chat message to the server , First it goes into the messaging system , The message system distributes and stores messages .

In the process ： For online recipients , Will choose to push the message directly . But when the receiver is not online or the message push fails , There will also be other ways to get messages , For example, the receiver will take the initiative to pull the unreceived messages from the server . But it is unknown when and where the receiver will pull the message from the server , So that's the point of storing messages in the offline library .

The message system stores the process offline , In order not to affect the stability of the whole system , We used MQ Message queuing IO Uncouple , So chat messages are actually stored asynchronously in the offline library （ adopt MQ Slow progress IO Uncouple , This is also the usual practice ）.

After distributing the message ： The message service will synchronize a message data to the historical message service , The historical message service will also store messages on the ground .

For new client devices ： There will be a need to synchronize messages （ The so-called message roaming capability ）, And this is the main function of historical news . In the historical message base , The client can pull the full amount of historical messages of any session .

    1） Our storage medium for offline messages is Redis;

    2） Historical news we chose HBase.

Why choose different storage media , In fact, we are considering different business scenarios and read-write modes of offline messages and historical messages .

Let's focus on the difference between offline message storage and historical message storage .

Offline message storage mode ——“ I want to write ”

For the storage mode of offline messages, we use diffusion writing .

Each user has their own separate in box and outbox ：

    1） The inbox stores all messages that need to be synchronized to the receiving end ;

    2） All messages sent by the sender are stored in the Outbox .

Take single chat as an example ： In a conversation between two people , The message will be written twice , That is, the sender's outbox and the receiver's inbox .

And in the group scenario ： Writing will be magnified more （ Spread ）, If there are N personal , That group message will be spread and written N Time .

To sum up ：

    1） The advantage of diffusion writing is ： The logic of the receiver will be very clear and simple , Just read it once from your in box , This greatly reduces the reading pressure required to synchronize messages ;

    2） The disadvantage of diffusion writing is ： The write is magnified exponentially , Especially for the group scenario .

Historical message storage mode ——“ Spread reading ”

The storage mode of historical messages is diffusion reading . Instant messaging chat software app Development can consult weikeyun .

Because in historical news , Each session holds the full amount of messages for the entire session . In diffuse reading mode , Messages are saved only once per session .

Contrast diffusion write mode , The advantages and disadvantages of diffusion reading are as follows ：

    1） Advantage is ： The number of writes is greatly reduced , Especially for group messages , Just save it once ;

    2） The disadvantage is that ： Receiving messages at the receiving end is very complex and inefficient , Because in this mode, if the client wants to pull all messages, it can only synchronize once per session , Reading will be magnified , And there may be many invalid reads , Because some conversations may have no news at all .