kafka 單機版安裝教程

kafka 單機版安裝教程

安裝zookeeper

netstat -nltp | grep 2181檢查是否已被佔用

  1. netstat -apn | grep 2181

開啟2181,

mkdir /usr/local/kafka

wget archive.apache.org/dist

tar -zxvf zookeeper-3.3.6.tar.gz

vim /etc/profile

使之立即生效

source /etc/profile

在主目錄下創建data和logs兩個目錄用於存儲數據和日誌:

在conf目錄下新建zoo.cfg文件,寫入以下內容保存:

tickTime=2000

initLimit=10

syncLimit=5

dataDir=/usr/local/zookeeper/data

dataLogDir=/usr/local/zookeeper/logs

clientPort=2181

進入bin目錄,啟動、停止、重啟分和查看當前節點狀態(包括集群中是何角色)別執行:

./zkServer.sh start

./zkServer.sh stop

./zkServer.sh restart

./zkServer.sh status

安裝kafka

wget archive.apache.org/dist

tar -zxvf kafka_2.8.0-0.8.0.tar.gz

配置kafka,修改文件server.properties

vim server.properties

上圖中的log.dirs結合自己所建的文件夾路徑填寫,沒有嚴格要求

配置kafka下zookeeper

mkdir /usr/local/kafka/zookeeper #創建zookeeper目錄

mkdir /usr/local/kafka/log/zookeeper #創建zookeeper日誌目錄

修改日誌文件目錄

vim kafkastart.sh

/usr/local/kafka/bin/zookeeper-server-start.sh /usr/local/kafka/config/zookeeper.properties &

sleep 3

/usr/local/kafka/bin/kafka-server-start.sh /usr/local/kafka/config/server.properties

vim kafkastop.sh

/usr/local/kafka/bin/zookeeper-server-stop.sh /usr/local/kafka/config/zookeeper.properties &

sleep 3

/usr/local/kafka/bin/kafka-server-stop.sh /usr/local/kafka/config/server.properties

設置腳本開機啟動自動執行

vim /etc/rc.d/rc.local

touch /var/lock/subsys/local

sh /root/kafkastart.sh &

sh /root/kafkastart.sh

sh /root/kafkastop.sh

bin/kafka-create-topic.sh --partition 1 --replica 1 --zookeeper 192.168.217.129:2181 --topic test

消費者:bin/kafka-console-consumer.sh --zookeeper 210.38.32.138:2181 --topic test --from-beginning

生產者: bin/kafka-console-producer.sh --broker-list 210.38.32.138:9092 --topic test

zookeper開機自啟動: /etc/profile

cd /etc/rc.d/init.d/

touch zookeeper

chmod +x zookeeper

vim zookeeper

#!/bin/bash

#chkconfig: 2345 10 90

#description: service zookeeper

export JAVA_HOME=/home/jdk/jdk1.7.0_45

export ZOO_LOG_DIR=/opt/zookeeper/log

export ZOOKEEPER_HOME=/usr/local/zookeeper

case "$1" in

start) su root ${ZOOKEEPER_HOME}/bin/zkServer.sh start;;

start-foreground) su root ${ZOOKEEPER_HOME}/bin/zkServer.sh start-foreground;;

stop) su root ${ZOOKEEPER_HOME}/bin/zkServer.sh stop;;

status) su root ${ZOOKEEPER_HOME}/bin/zkServer.sh status;;

restart) su root ${ZOOKEEPER_HOME}/bin/zkServer.sh restart;;

upgrade)su root ${ZOOKEEPER_HOME}/bin/zkServer.sh upgrade;;

print-cmd)su root ${ZOOKEEPER_HOME}/bin/zkServer.sh print-cmd;;

*) echo "requirestart|start-foreground|stop|status|restart|print-cmd";;

esac

把zookeeper這個腳本添加到開機啟動項裡面,命令是:

chkconfig --add zookeeper

如果想看看是否添加成功,命令是: service zookeeper status

chkconfig --list

推薦閱讀:

為何流處理最近很火,而同根的複雜事件處理一直不溫不火?
kafka消費者組數量較大對性能有什麼影響?
Kafka設計解析(七)- 流式處理的新貴Kafka Stream
kafka解決了什麼問題?
如何使用Kafka在生產環境構建大規模機器學習

TAG:Hadoop | Kafka | 計算機科學 |