kafka集群配置

 4台虚拟机 

192.168.130.62

192.168.130.63

192.168.130.64

192.168.130.65

1.和hbase公用服务器， jdk,zookeeper已配置好，不需要重新配置

[datalink@slave1 kafka_2.11-0.9.0.0]$ java -version
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

[datalink@slave1 kafka_2.11-0.9.0.0]$ zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/module/zookeeper/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: follower

 2.下载kafka

[datalink@slave1 kafka_2.11-0.9.0.0]$ wget https://archive.apache.org/dist/kafka/0.9.0.0/kafka_2.11-0.9.0.0.tgz

 3.解压

  tar -zxvf kafka_2.11-0.9.0.0.tgz -C /opt/module/

 4.将解压文件拷贝到各个节点

 scp -r kafka_2.11-0.9.0.0/ datalink@slave2:/opt/module/
 scp -r kafka_2.11-0.9.0.0/ datalink@slave3:/opt/module/
 scp -r kafka_2.11-0.9.0.0/ datalink@slave4:/opt/module/

5.配置环境变量  /etc/profile

export KAFKA_HOME=/opt/module/kafka_2.11-0.9.0.0
export PATH=$PATH:$KAFKA_HOME/bin

6.修改各个服务器配置文件/opt/module/kafka_2.11-0.9.0.0/config/server.properties

broker.id=0

listeners=PLAINTEXT://192.168.130.62:9092

zookeeper.connect=192.168.130.62:2181,192.168.130.63:2181,192.168.130.64:2181,192.168.130.65:2181

说明：

broker.id   kafka集群的多台server，每个server为一个kafka实例,也叫broker，id唯一

listeners监听器，其实就是告诉外部连接者要通过什么协议访问指定主机名和端口开放的 Kafka 服务

zookeeper.connect  zookeeper服务器

7，每台服务器上启动kafka

bin/kafka-server-start.sh -daemon config/server.properties

8.创建一个主题

kafka-topics.sh --create --zookeeper 192.168.1.47:2181 --replication-factor 3 --partitions 1 --topic test-topic

9.查看主题列表

[datalink@slave1 ~]$ kafka-topics.sh --list --zookeeper 192.168.130.65:2181
test-topic

10.查看主题信息

[datalink@slave1 ~]$ kafka-topics.sh --describe --zookeeper 192.168.130.65:2181 --topic test-topic
Topic:test-topic PartitionCount:1 ReplicationFactor:3 Configs:
Topic: test-topic Partition: 0 Leader: 2 Replicas: 2,1,3 Isr: 2,1,3