關于在雲伺服器部署docker後,部署kafka,外網無法通路解決方式
這裡主要測試的華為雲伺服器,但是我想所有的雲伺服器應該是大同小異的,其次是外網通路階段,kafka部署後無法通路,以及在容器中更改了server.properties,重新開機容器還是不正确,通過查找發現伺服器可以做以下修改之後就能成功使用kafka tools和代碼通路kafka了
這裡使用的kafka和zookeeper
docker pull wurstmeister/zookeeper
docker pull wurstmeister/kafka 1.進入hosts檔案
vi /etc/hosts
//添加公網ip
公網ip 2.删除已經部署好的kafka,也可以不做删除,但重新給端口,命名就可以了
docker run -d --name kafka -p 9093:9093 -e KAFKA_BROKER_ID=0 -e KAFKA_ZOOKEEPER_CONNECT=(一定要是公網ip,否則遠端是無法操作kafka的,kaka tools也是無法使用的):2181 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://(一定要是公網ip,否則遠端是無法操作kafka的,kaka tools也是無法使用的):9092 -e KAFKA_LISTENERS=PLAINTEXT://0.0.0.0:9092 -t wurstmeister/kafka 這個時候其實已經可以通路kafka了,但是有些人還是無法通路,究其原因是因為在雲伺服器的出入站規則中沒有增加2181端口和個人給的kafka端口,是以無法通路的,以下是我的出入站規則
3.現在就能通過代碼測試是否可以通路了,我這裡建立了一個test的topic;然後還可以通過kafka tools檢視是否有kafka了.
package com.project.spark.util
/**
* @Author ：
* @Date ： Create in 2019-07-17 14:40
* @Description ：<描述>
*/
object Producer extends App {
import java.util.Properties
import org.apache.kafka.clients.producer.{KafkaProducer, ProducerRecord}
val topic = "par"
val brokers = "公網ip:9092"
val props = new Properties()
props.put("bootstrap.servers", brokers)
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer")
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer")
val producer = new KafkaProducer[String, String](props)
val t = System.currentTimeMillis()
var msg = "啟動成功"
val record = new ProducerRecord[String, String](topic, "key", msg)
producer.send(record)
producer.close()
} 關于有些小夥伴在這些都做成功之後,在控制台begin topic的時候發現有以下錯誤
[2019-07-18 05:45:26,746] WARN [Consumer clientId=consumer-1, groupId=console-consumer-74208] Connection to node -1 (/192.168.0.152:9093) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2019-07-18 05:45:26,849] WARN [Consumer clientId=consumer-1, groupId=console-consumer-74208] Connection to node -1 (/192.168.0.152:9093) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2019-07-18 05:45:26,950] WARN [Consumer clientId=consumer-1, groupId=console-consumer-74208] Connection to node -1 (/192.168.0.152:9093) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2019-07-18 05:45:27,152] WARN [Consumer clientId=consumer-1, groupId=console-consumer-74208] Connection to node -1 (/192.168.0.152:9093) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2019-07-18 05:45:27,654] WARN [Consumer clientId=consumer-1, groupId=console-consumer-74208] Connection to node -1 (/192.168.0.152:9093) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
[2019-07-18 05:45:28,558] WARN [Consumer clientId=consumer-1, groupId=console-consumer-74208] Connection to node -1 (/192.168.0.152:9093) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient) 這個原因是因為你開啟的時候ip沒有寫公網ip,是以會造成無法連接配接,下面是我測試的幾條指令
kafka-console-consumer.sh --bootstrap-server 公網ip:給的端口 --topic par --from-beginning
kafka-console-consumer.sh --bootstrap-server 公網ip:給的端口 --topic par --from-beginning