天天看点

Kafka生产环境中的错误。

最近写完storm实时统计pv.uv.ip的项目后,前端服务器用flume进行收集,逐步把所有的服务器都增加上。增加的差不多时。Kafka报了如下类似错误:

查了各种资料,发现是由于Kafka走的默认配置。发送的消息超过了最大字节导致的。修改如下参数即可:

 <code>replica.fetch.max.bytes</code> - this will allow

for the replicas in the brokers to send messages within the cluster and

make sure the messages are replicated correctly. If this is too small,

then the message will never be replicated, and therefore, the consumer

will never see the message because the message will never be committed

(fully replicated).

<code>message.max.bytes</code> - this is the largest size of the message that can be received by the broker from a producer.

修改完成之后一切正常。

继续阅读