Kafuka learning path (I) Kafuka installation and simple use

Hello everyone , I meet you again , I'm your friend, Quan Jun .

One , Installation environment and software version

linux	centOs6 64
jdk	jdk-8u191-linux-x64.tar.gz
zookeeper	zookeeper-3.4.10.tar.gz
kafuka	kafka_2.11-0.11.0.2

Two , install

## decompression 
-rwxrw-rw-.  1 root root 42136632 Jun 11 01:55 kafka_2.11-0.11.0.2.tgz
drwxr-xr-x. 12 1001 1001     4096 Jun 11 05:35 zookeeper-3.4.10
[[email protected] module]# tar -xvf kafka_2.11-0.11.0.2.tgz 
[[email protected] kafka_2.11-0.11.0.2]# ll
total 56
drwxr-xr-x. 3 root root  4096 Nov 10  2017 bin
drwxr-xr-x. 2 root root  4096 Nov 10  2017 config
drwxr-xr-x. 2 root root  4096 Jun 11 20:09 libs
-rw-r--r--. 1 root root 28824 Nov 10  2017 LICENSE
drwxr-xr-x. 2 root root  4096 Jun 11 20:10 logs
-rw-r--r--. 1 root root   336 Nov 10  2017 NOTICE
drwxr-xr-x. 2 root root  4096 Nov 10  2017 site-docs

## Add log folder 
[[email protected] kafka_2.11-0.11.0.2]# mkdir logs
[[email protected] kafka_2.11-0.11.0.2]# ll

# Modify configuration file 
[[email protected] kafka_2.11-0.11.0.2]# cd config/
[[email protected] config]# vim server.properties 
broker.id=1  #broker The global unique number of , Can't repeat ( Mine is with zk Of myid equally )
delete.topic.enable=true
listeners=PLAINTEXT://192.168.8.132:9092
log.dirs=/opt/module/kafka_2.11-0.11.0.2/logs
zookeeper.connect=192.168.8.129:2181,192.168.8.132:2181,192.168.8.133:2181

Copy

3、 ... and , Start and create partitions using

notes ：zookeeper On the premise that the cluster starts normally

# start-up 
[[email protected] kafka_2.11-0.11.0.2]# bin/kafka-server-start.sh config/server.properties &

# close 
[[email protected] kafka_2.11-0.11.0.2]# bin/kafka-server-stop.sh stop

## establish topic
#topic  Definition topic name 
#replication-factor   Define the number of copies 
#partitions   Define the number of partitions 
[[email protected] kafka_2.11-0.11.0.2]# bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 3 --topic test 

bin/kafka-topics.sh  --zookeeper localhost:2181 --create --replication-factor 3 --partitions 3 --topic test 
Created topic "test".  
## see topic  list 
[[email protected] kafka_2.11-0.11.0.2]# bin/kafka-topics.sh --zookeeper localhost:2181 --list
test

## Check the details 
[[email protected] kafka_2.11-0.11.0.2]# bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic test
Topic:test      PartitionCount:3        ReplicationFactor:3     Configs:        MarkedForDeletion:true
        Topic: test     Partition: 0    Leader: -1      Replicas: 0,1,2 Isr: 2
        Topic: test     Partition: 1    Leader: -1      Replicas: 1,2,0 Isr: 2
        Topic: test     Partition: 2    Leader: -1      Replicas: 2,0,1 Isr: 2

## Delete topic
[[email protected] kafka_2.11-0.11.0.2]# bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic test
Topic test is marked for deletion.
Note: This will have no impact if delete.topic.enable is not set to true.
# If there is one in the cluster kafuka No settings  delete.topic.enable=true ,
# It won't delete , You need to restart all , Delete it again 
# Delete the mark after success 
[[email protected] kafka_2.11-0.11.0.2]#  bin/kafka-topics.sh --zookeeper localhost:2181 --list
test - marked for deletion
# Again zk Delete the registered node in 
rmr /brokers/topics/【topic name】

Copy

Four , Easy to use

## Send a message (localhost  It has to be local ip)
bin/kafka-console-producer.sh --broker-list localhost:9092 --topic topicTest

## News consumption (localhost  It has to be local ip)
bin/kafka-console-consumer.sh --zookeeper localhost:2181 --from-beginning --topic topicTest

# production 
[[email protected] kafka_2.11-0.11.0.2]# bin/kafka-console-producer.sh --broker-list 192.168.8.129:9092 --topic test1
>123
>123
>123
>123
>123
>123
>123
>hool^H^H
>holl
>hello
>hello
>

# consumption 1
[[email protected] kafka_2.11-0.11.0.2]#  bin/kafka-console-consumer.sh --zookeeper 192.168.8.132:2181 --from-beginning --topic test1
Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].
123
123
123
123
123
123
123
hool
holl
hello
hello

# consumption 2, Quit halfway and come in , Early news will be out of order 
[[email protected] kafka_2.11-0.11.0.2]# bin/kafka-console-consumer.sh --zookeeper 192.168.8.133:2181 --from-beginning --topic test1
Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].
123
123
123
hello
123
123
hool
123
123
holl
hello

Copy

Publisher ： Full stack programmer stack length , Reprint please indicate the source ：https://javaforall.cn/131623.html Link to the original text ：https://javaforall.cn