精华内容
下载资源
问答
  • kafka停止consumer,producer等进程

    千次阅读 2018-04-22 17:28:18
    kafka停止consumer,producer等进程 1.官网给出的方案: You can now stop the console consumer, the console producer, the Wordcount application, the Kafka broker and the ZooKeeper server in order via ...

    kafka停止consumer,producer等进程

    1.官网给出的方案:
    You can now stop the console consumer, the console producer, the Wordcount application, the Kafka broker and the ZooKeeper server in order via Ctrl-C.

    展开全文
  • kafka-server-stop.sh不起作用,kafka关不了 修改kafka-server-stop.sh文件 cd ~/modules/kafka_2.11/bin vi kafka-server-stop.sh 此时可以把PIDS=$(ps ax | grep -i ‘kafka.Kafka’ | grep java | grep -v grep | ...

    kafka-server-stop.sh不起作用,kafka关不了

    修改kafka-server-stop.sh文件
    cd ~/modules/kafka_2.11/bin

    vi kafka-server-stop.sh

    此时可以把PIDS=$(ps ax | grep -i ‘kafka.Kafka’ | grep java | grep -v grep | awk ‘{print $1}’)这行代码修改为

    PIDS=$(jps -lm | grep -i ‘kafka.Kafka’ | awk ‘{print $1}’)

    展开全文
  • Kafka启动停止脚本

    2020-05-08 22:31:39
    #! /bin/bash case $1 in "start"){ for i in hadoop102 hadoop103 hadoop104 do echo " --------启动 $i Kafka-------" # 用于KafkaManager监控 ssh $i "export JMX_PORT=9988 &... /opt/module/kafka/bin
    #! /bin/bash
    
    case $1 in
    "start"){
            for i in hadoop102 hadoop103 hadoop104
            do
                    echo " --------启动 $i Kafka-------"
                    # 用于KafkaManager监控
                    ssh $i "export JMX_PORT=9988 && /opt/module/kafka/bin/kafka-server-start.sh -daemon /opt/module/kafka/config/server.properties "
            done
    };;
    "stop"){
            for i in hadoop102 hadoop103 hadoop104
            do
                    echo " --------停止 $i Kafka-------"
                    ssh $i "/opt/module/kafka/bin/kafka-server-stop.sh stop"
            done
    };;
    esac
    
    

    说明:路径和主机名根据自己情况修改

    展开全文
  • kafka启动停止脚本

    2018-11-24 19:52:34
    linux常见文件 cat>...kafka启动脚本 #!/bin/bash echo "start kafka>>>" #cd /home/software/kafka_2.11-0.11.0.3 bin/zookeeper-server-start.sh -daemon config/zookeeper.properti...

    linux常见文件

    cat>>start.sh

    ctrl+d推出保存

    kafka启动脚本

    #!/bin/bash
    echo "start kafka>>>"
    #cd /home/software/kafka_2.11-0.11.0.3
    bin/zookeeper-server-start.sh -daemon config/zookeeper.properties
    sleep 1
    bin/kafka-server-start.sh -daemon config/server.properties


    kafka停止脚本

    #!/bin/bash
    echo  "kafka stop >>>?"
    bin/kafka-server-stop.sh
    sleep 1
    bin/zookeeper-server-stop.sh

    常见命令

    生产者客户端
    bin/kafka-console-producer.sh --broker-list 192.168.43.122:9092 --topic test
    bin/kafka-console-producer.sh --broker-list iod.kafka:6667 --topic test
    bin/kafka-console-producer.sh --broker-list 192.168.43.161:9092 --topic test

    消费者客户端
    bin/kafka-console-consumer.sh --bootstrap-server 192.168.43.122:9092 --topic vehicle_passlog--from-beginning

    bin/kafka-console-consumer.sh --bootstrap-server 192.168.43.161:9092 --topic test
    bin/kafka-console-consumer.sh --bootstrap-server iod.kafka:6667 --topic vehicle_passlog
    bin/kafka-console-consumer.sh --zookeeper iod.kafka:2181 --topic iod_gcjlout

    启动zookeeper
    nohup bin/zookeeper-server-start.sh config/zookeeper.properties >/dev/null &
    bin/zookeeper-server-start.sh -daemon config/zookeeper.properties

    启动kafka
    nohup bin/kafka-server-start.sh config/server.properties &
    bin/kafka-server-start.sh -daemon config/server.properties


    查看topic列表
    bin/kafka-topics.sh --list --zookeeper 192.168.43.122:2181
    bin/kafka-topics.sh --list --zookeeper 61.16.43.64:2181

    查看topic详情
    bin/kafka-topics.sh --zookeeper 192.168.43.122:2181 --describe --topic test

    查看消费情况
    bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --zookeeper localhost:2181 --group zhsq --topic vehicle_passlog

    bin/kafka-consumer-offset-checker.sh --zookeeper 61.16.43.64:2181 --topic iod_gcjlout --group zhsq

    创建topic ,1个副本6个分区,副本数小于broker数,一个分区只能同时被一个消费者线程消费,一般分区数量等于线程数量,
    bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 6 --topic test

    删除topic
    bin/kafka-topics.sh --delete --zookeeper localhost:2181 --topic test

    后台启动kafka
    bin/kafka-server-start.sh -daemon config/server.properties

    hosts目录
    C:\Windows\System32\drivers\etc


    auto.offset.reset
    如果存在已经提交的offest时,不管设置为earliest 或者latest 都会从已经提交的offest处开始消费
    如果不存在已经提交的offest时,earliest 表示从头开始消费,latest 表示从最新的数据消费,也就是新产生的数据.

    展开全文
  • kafka consumer 停止消费topic

    万次阅读 2018-04-07 12:23:10
    现象在kafka consumer (以 kafka1.0.0为例)消费 topic 时,常常会出现程序还在运行,但是已经不消费消息了(kafka producer正常生产消息),使用kafka命令查看,kafka 已经没有consumer 的信息了。实验用例实验 kafka ...
  • 请教个问题,为什么storm加了任务后,开始运行特别快,然后越来越慢,然后就停止了,再然后爆出kafka连接失败,然后任务停止,等一会,又启动,又是快速处理,我确认kakfa中日志根本没有读完 而且,每次重启任务的...
  • 单个启动停止Kafka

    2020-12-27 14:42:54
    前提是已经配置好了$KAFKA_HOME 环境变量 启动Kafka: sh $KAFKA_HOME/bin/kafka-server-start.sh -daemon $KAFKA_HOME/config/server.properties ...停止Kafka: sh $KAFKA_HOME/bin/kafka-server-stop.sh
  • 启动kafka: sh /app/pet_kafka_xxxx_cluster/bin/kafka-server-start.sh -daemon /app/pet_kafka_xxxx_cluster/config/server....停止kafka:/app/pet_kafka_xxxx_cluster/bin/kafka-server-stop.sh pet...
  • brew install kafka ==> zookeeper To have launchd start zookeeper now and restart at login: brew services start zookeeper Or, if you don't want/need a background service you can just run: zk...
  • kafka-python 停止消费

    千次阅读 2018-07-12 13:45:03
    kafka-python 停止消费使用kafka-python时,不消费数据,也没有异常换为pykafka库后解决使用例子from pykafka import KafkaClient client = KafkaClient(hosts='127.0.0.1:9092')topic = client.topics['logsget'] ...
  • vim kf.sh #! /bin/bash case $1 in "start"){ for i in hadoop102 hadoop103 hadoop104 ... ssh $i "/export/servers/kafka/bin/kafka-server-start.sh -daemon /export/servers/kafka/config/server.propertie
  • kafka 基本命令(较新版本的kafka

    千次阅读 2019-09-05 11:14:21
    # 指定对应的配置文件启动 kafka-server-start.sh /hadoop/kafka/config/server.properties # -daemon 以后台的方式启动 kafka-server-start.sh -daemon /hadoop/kafka/...kafka停止命令 kafka-server-stop.sh 或...
  • kafka shutdown停止很慢问题 在数据量大的时候,consumer一次抓取数据的数据很多,进入到业务处理的数据可能有很多, 假设一次poll有1万条数据进入业务程序,而且业务程序是和poll绑定在一起线程...
  • kafka 版本 kafka_2.11-0.10.2.1   因为kafka没有批量启动脚本,每次都需要在各个broker节点上启动kafka服务。比较麻烦。   这里自定义一个kafka启动的脚本: #!/bin/bash   BROKERS="hdp-01 hdp-02 ...
  • kafka:启动停止方式

    千次阅读 2019-05-30 00:04:30
    perf_2.8.0*.jar:/usr/local/kafka/bin/../clients/build/libs//kafka-clients*.jar:/usr/local/kafka/bin/../examples/build/libs//kafka-examples*.jar:/usr/local/kafka/bin/../contrib/hadoop-consumer/build/...

空空如也

空空如也

1 2 3 4 5 ... 20
收藏数 588
精华内容 235
关键字:

kafka停止