通过pykafka接收Kafka消息队列

时间:2021-02-02 05:33:35
没有Kafka环境,所以也没有进行验证。感觉今后应该能用到,所以借抄在此,备查。
pykafka使用示例,自动消费最新消息,不重复消费:
# -* coding:utf8 *-
from pykafka import KafkaClient

host = '192.168.200.38'
client = KafkaClient(hosts="%s:9092" % host)

print client.topics

# 生产者
# topicdocu = client.topics['task_pull']
# producer = topicdocu.get_producer()
# for i in range(4):
# print i
# producer.produce('test message ' + str(i ** 2))
# producer.stop()

# 消费者
topic = client.topics['task_push']
consumer = topic.get_simple_consumer(consumer_group='test', auto_commit_enable=True, consumer_id='test')
for message in consumer:
if message is not None:
print message.offset, message.value