QBoard » Big Data » Big Data - Data Ingestion Tools : Sqoop, Flume, Kafka, Nifi.. » Leader Not Available Kafka in Console Producer

Leader Not Available Kafka in Console Producer

  • I am trying to use Kafka.
    All configurations are done properly but when I try to produce message from console I keep getting the following error

    WARN Error while fetching metadata with correlation id 39 : 
         {4-3-16-topic1=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)

    Kafka version: 2.11-0.9.0.0

      October 1, 2020 5:02 PM IST
    0
  • It could be related to advertised.host.name setting in your server.properties. What could happen is that your producer is trying to find out who is the leader for a given partition, figures out its advertised.host.name and advertised.port and tries to connect. If these settings are not configured correctly it then may think that the leader is unavailable.
      October 3, 2020 12:15 PM IST
    1
  • I tried all the recommendations listed here. What worked for me was to go to server.propertiesand add:

    port = 9092
    advertised.host.name = localhost 

    Leave listeners and advertised_listeners commented out.

      October 3, 2020 12:16 PM IST
    1
  • What solved it for me is to set listeners like so:

    advertised.listeners = PLAINTEXT://my.public.ip:9092
    listeners = PLAINTEXT://0.0.0.0:9092
    

    This makes KAFKA broker listen to all interfaces

      October 5, 2020 12:32 PM IST
    1
    • Jasmine Chacko
      Jasmine Chacko This should be the accepted answer. Works for multi node configuration and makes a lot of sense
      October 5, 2020
  • I'm using kafka_2.12-0.10.2.1:

    vi config/server.properties

    add below line:

    listeners=PLAINTEXT://localhost:9092
    • No need to change the advertised.listeners as it picks up the value from std listener property.

    Hostname and port the broker will advertise to producers and consumers. If not set,

    • it uses the value for "listeners" if configured

    Otherwise, it will use the value returned from java.net.InetAddress.getCanonicalHostName().

    stop the Kafka broker:

    bin/kafka-server-stop.sh

    restart broker:

    bin/kafka-server-start.sh -daemon config/server.properties

    and now you should not see any issues.

    This post was edited by Laksh Nath at October 5, 2020 12:41 PM IST
      October 5, 2020 12:40 PM IST
    0
  • When LEADER_NOT_AVAILABLE error throws, just restart the kafka broker:

    /bin/kafka-server-stop.sh

    followed by

    /bin/kafka-server-start.sh config/server.properties

    (Note: Zookeeper must be running by this time ,if you do otherway it wont work )

      October 5, 2020 12:52 PM IST
    0