3. 檔案加鎖失敗 Failed to acquire lock on file .lock in
4. 發送消息報錯 UNKNOWN_TOPIC_OR_PARTITION
5. Error while reading checkpoint file xxxx/cleaner-offset-checkpoint
6. InconsistentBrokerMetadataException
7. log.dir相關異常 Failed to load xxx during broker startup
8. meta.properties 版本資訊不對
日常運維
問題排查
怎麼能夠少了滴滴開源的
1. Leader的epoch過時
The leader epoch in the request is older than the epoch on the broker
--
Partition $topicPartition marked as failed
解決方法
說明 目前分區的Leader的epoch比Broker的epoch老
是以導緻follow去fetchleader的時候報錯;
隻要重新發生一下Leader選舉就行了;
2. 修改Broker.id出現異常
Configured broker.id 0 doesn't match stored broker.id 1 in meta.properties.
If you moved your data, make sure your configured broker.id matches. If you intend to create a new broker, you should remove all data in your data directories (log.dirs).
出現這種情況一般是 你可能中途修改了Broker的配置
broker.id
; 又或者修改了
log.dir
路徑,然後這個路徑之前存在;
你可以看看
log.dir
檔案夾下面的
meta.properties
#Wed Jun 23 17:59:02 CST 2021
broker.id=0
version=0
cluster.id=0
Failed to acquire lock on file .lock in /Users/xxxx/work/IdeaPj/xxx/kafka/kafka-logs-0. A Kafka instance in another process or thread is using this directory.
異常原因:
Broker在啟動的時候,會把
log.dirs
加上一個檔案鎖,以防其他程式對它進行篡改;
出現這種異常表示已經有一個程式對檔案夾加上了鎖了; 是以擷取失敗;
解決方法
這個時候你要檢查一下,這個Broker是否已經啟動過了,或者兩個Broke中
log.dirs
配置了相同的檔案夾;
如果上面你确定沒有問題,那你還可以把相應的檔案夾的
.lock
檔案删掉; 強制去掉鎖檔案; (不建議這樣操作)
4. 發送消息報錯 UNKNOWN_TOPIC_OR_PARTITION
WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id : {test80=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient)
5. Error while reading checkpoint file xxxx/cleaner-offset-checkpoint
Error while reading checkpoint file /Users/shirenchuang/work/IdeaPj/didi_source/kafka/kafka-logs-2/cleaner-offset-checkpoint
6. InconsistentBrokerMetadataException
kafka.common.InconsistentBrokerMetadataException: BrokerMetadata is not consistent across log.dirs. This could happen if multiple brokers shared a log directory (log.dirs) or partial data was manually copied from another broker. Found:
- kafka-logs-0 -> BrokerMetadata(brokerId=0, clusterId=0)
- kafka-logs-1 -> BrokerMetadata(brokerId=1, clusterId=0)
7. log.dir相關異常 Failed to load xxx during broker startup
Failed to load ${dir.getAbsolutePath} during broker startup
異常原因:
啟動的時候讀取檔案夾
log.dirs
檔案裡面的
meta.properties
的時候抛IOException,讀取失敗
解決方法
查詢一下是不是對應的dir中的檔案
meta.properties
有什麼異常(是否有權限讀取等等)
Duplicate log directory found: xxxx
異常原因:
log.dirs
設定的檔案夾重複了;比如:
log.dirs=kafka-logs-0,kafka-logs-0
解決方法
檢查一下是不是設定重複了
Found directory /xxxx/kafka/kafka-logs-0/test, 'test' is not in the form of topic-partition or topic-partition.uniqueId-delete (if marked for deletion).
Kafka's log directories (and children) should only contain Kafka topic data.
異常原因:
log.dirs
檔案夾中存在不符合條件的檔案夾,一般裡面的檔案夾的格式都是
topic-分區号
,
topic-分區号-future
,
topic-分區号-delete
解決方法
自檢一下不合格的檔案夾
8. meta.properties 版本資訊不對
[2021-07-21 13:38:19,246][ERROR][main]: Failed to create or validate data directory /Users/xxx/kafka/kafka-logs-0
java.io.IOException: Failed to load /Users/xxxx/kafka/kafka-logs-0 during broker startup