Open
Description
本文简单记录一下如何搭建Apache Kafka 开发环境
前言
我这里用的是MBP，如果是Windows用户那么下面的命令以及一些路径可能会有差异，比如脚本后缀应该是.bat.
安装相关库&&工具
- JDK1.8
- Gradle
- Zookeeper
构建Kafka
- https://github.com/apache/kafka， 下载源码
- 执行以下命令
cd kafka
gradle idea
#这里会下载一坨依赖，慢慢等，看到Build Successful说明成功了引入IDE
-
首先需要安装Scala插件
-
配置日志，这里需要将根目录下的
conf/log4j.properties拷贝到/core/src/main/resources/log4j.properties,resources文件夹不存在则新建一个 -
配置运行时设置,直接参考下图即可
-
点击运行，第一次启动时Idea会先编译整个项目，因此可能会比较慢，日志如下则表示启动成功:
[2017-11-15 21:11:47,436] INFO [Transaction Marker Channel Manager 0]: Starting (kafka.coordinator.transaction.TransactionMarkerChannelManager)
[2017-11-15 21:11:47,436] INFO [TransactionCoordinator id=0] Startup complete. (kafka.coordinator.transaction.TransactionCoordinator)
[2017-11-15 21:11:47,470] INFO [/config/changes-event-process-thread]: Starting (kafka.common.ZkNodeChangeNotificationListener$ChangeEventProcessThread)
[2017-11-15 21:11:47,504] INFO Creating /brokers/ids/0 (is it secure? false) (kafka.utils.ZKCheckedEphemeral)
[2017-11-15 21:11:47,509] INFO Result of znode creation is: OK (kafka.utils.ZKCheckedEphemeral)
[2017-11-15 21:11:47,510] INFO Registered broker 0 at path /brokers/ids/0 with addresses: EndPoint(192.168.1.103,9092,ListenerName(PLAINTEXT),PLAINTEXT) (kafka.utils.ZkUtils)
[2017-11-15 21:11:47,515] WARN Error while loading kafka-version.properties :null (org.apache.kafka.common.utils.AppInfoParser)
[2017-11-15 21:11:47,516] INFO Kafka version : unknown (org.apache.kafka.common.utils.AppInfoParser)
[2017-11-15 21:11:47,516] INFO Kafka commitId : unknown (org.apache.kafka.common.utils.AppInfoParser)
[2017-11-15 21:11:47,517] INFO [KafkaServer id=0] started (kafka.server.KafkaServer)
验证
- 首先来创建一个topic,单个replication，单个partitions
> bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
Created topic "test".现在就可以看到topic已经创建成功了:
bin/kafka-topics.sh --list --zookeeper localhost:2181 !10280
test- 发几个消息试试
> bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
Hello World
你好,世界！- 新开一个窗口，用于消费消息:
> bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning
Hello World
你好,世界！Metadata
Metadata
Assignees
Labels
No labels