Composition of pulsar messages

Pulsar be based on Release - A subscription model （ Commonly abbreviated as pub-sub）, In this mode , producer （producers） The message is （messages） Post to topic （topics）; consumer （consumers） Subscribe to these topics （topics）, Then process the incoming message （messages）, And report to the agent after processing （broker） Send a confirmation message .

When a consumer subscribes to a message , Even if the consumer disconnects ,Pulsar It will be Keep all messages . Only when the consumer confirms that all messages have been successfully processed ,Pulsar Will discard the reserved message .

If a message fails to be consumed , Consumers want to consume this message again , Can be opened Message re delivery mechanism , request broker Resend this message .

news （messages） yes  Pulsar Basic “ Company ”. The following table lists the components of the message .

Component	Description
Value / data payload	The data carried by the message. All Pulsar messages contain raw bytes, although message data can also conform to data schemas. //  The data carried by the message . all Pulsar Messages contain raw bytes , Although the message data can also conform to the data pattern .
Key	The key (string type) of the message. It is a short name of message key or partition key. Messages are optionally tagged with keys, which is useful for features like topic compaction. // ​ Key of the message （ String type ）. It's news key Or zoning key For short . Messages can optionally be marked with keys , This is very useful for compressing topics, etc .
Properties	An optional key/value map of user-defined properties. // A key value pair attribute that can be customized by the user
Producer name	The name of the producer who produces the message. If you do not specify a producer name, the default name is used. //  The name of the producer that generated the message . If no producer name is specified , The default name is used .
Topic name	The name of the topic that the message is published to.
Schema version	The version number of the schema that the message is produced with. // The version number of the schema used to generate the message .
Sequence ID	Each Pulsar message belongs to an ordered sequence on its topic. The sequence ID of a message is initially assigned by its producer, indicating its order in that sequence, and can also be customized. // Every one of them Pulsar message stay topic There is an ordered sequence in , Sequence ID By default, it is assigned by its producer , Indicates its order in the sequence , You can also customize it . Sequence ID can be used for message deduplication. If brokerDeduplicationEnabled is set to true, the sequence ID of each message is unique within a producer of a topic (non-partitioned) or a partition. //  Sequence ID Can be used to de duplicate messages . If BrokerDeplicationEnabled Set to true, Then the sequence of each message ID In the theme （ Non partition ） Or partition is the only producer .
Message ID	The message ID of a message is assigned by bookies as soon as the message is persistently stored. Message ID indicates a message’s specific position in a ledger and is unique within a Pulsar cluster. // news ID Persistent by bookies Distribute , Specify the specific location of the message in the message classification , And in Pulsar It's the only one in the cluster .
Publish time	The timestamp of when the message is published. The timestamp is automatically applied by the producer. // Time of news release , This time is automatically provided by the producer
Event time	An optional timestamp attached to a message by applications. For example, applications attach a timestamp on when the message is processed. If nothing is set to event time, the value is 0. // Optional timestamp that the application attaches to the message . for example , The application processes the message with a timestamp . If nothing is set to the event time , The value is 0.

news （messages） The default size is 5 MB. You can also use the following configuration , Configure message size .

in the broker.conf file.

# The max size of a message (in bytes).
maxMessageSize=5242880

In the bookkeeper.conf file.

# The max size of the netty frame (in bytes). Any messages received larger than this value are rejected. The default value is 5 MB.
nettyMaxFrameSizeBytes=5253120

For more information on Pulsar messages, see Pulsar binary protocol.