How to catch warning “Broker may not be available” at the Spring Kafka Listener
I am working on a POC for implementing a kafka cluster in my project. I have setup a kafka cluster in my local machine with 3 brokers. Now I am sending messages to the Kafka server using Spring MVC REST service which is internally using Spring Kafka to produce and consume messages to and from the Kafka cluster. Now i am trying to send alerts when the consumer is unable to receive messages from the topic is down. I shutdown the only broker to which the consumer is connected. I am not getting any exception in my logs but i got he following warning messages.
0:20:35.500 [TEST_GROUP-0-C-1] WARN
o.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1,
groupId=TEST_GROUP] Connection to node 2147483645 could not be
established. Broker may not be available.
Is it possible to catch this warning message so that i can send alert when my consumer has lost connection? Below is my consumer code.
private static final Logger LOGGER = LoggerFactory.getLogger(ListenerServiceImpl.class);
@Autowired
Dao<RnMessage> messageDao;
@Autowired
MessageService messageService;
@KafkaListener(id = "TEST_GROUP", topics = "TESTQUEUE", errorHandler="eventQueueMessageListenerExceptionHandler")
public void listenMessageInQueue(String msg) {
try {
//String str = new String(msg, "UTF-8");
LOGGER.info("receiving payload='{}'", msg);
messageDao.saveMessage(msg);
messageService.sendMessageToOutQueue(msg);
}catch(Exception e) {
e.printStackTrace();
}
}
java spring apache-kafka spring-kafka
asked Dec 27 '18 at 15:48
Venu
9212
add a comment |
I am working on a POC for implementing a kafka cluster in my project. I have setup a kafka cluster in my local machine with 3 brokers. Now I am sending messages to the Kafka server using Spring MVC REST service which is internally using Spring Kafka to produce and consume messages to and from the Kafka cluster. Now i am trying to send alerts when the consumer is unable to receive messages from the topic is down. I shutdown the only broker to which the consumer is connected. I am not getting any exception in my logs but i got he following warning messages.
0:20:35.500 [TEST_GROUP-0-C-1] WARN
o.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1,
groupId=TEST_GROUP] Connection to node 2147483645 could not be
established. Broker may not be available.
Is it possible to catch this warning message so that i can send alert when my consumer has lost connection? Below is my consumer code.
private static final Logger LOGGER = LoggerFactory.getLogger(ListenerServiceImpl.class);
@Autowired
Dao<RnMessage> messageDao;
@Autowired
MessageService messageService;
@KafkaListener(id = "TEST_GROUP", topics = "TESTQUEUE", errorHandler="eventQueueMessageListenerExceptionHandler")
public void listenMessageInQueue(String msg) {
try {
//String str = new String(msg, "UTF-8");
LOGGER.info("receiving payload='{}'", msg);
messageDao.saveMessage(msg);
messageService.sendMessageToOutQueue(msg);
}catch(Exception e) {
e.printStackTrace();
}
}
java spring apache-kafka spring-kafka
asked Dec 27 '18 at 15:48
Venu
9212
show consumer configs, and also check zookeeper and kafka server are up
– Deadpool
Dec 27 '18 at 16:11
1
With a different setup, you would ship your logs to something like Splunk or Elasticsearch, which are configured with alerting on specific log conditions. Then, you wouldn't be adding boilerplate altering code into all your Kafka clients
– cricket_007
Dec 27 '18 at 16:17
add a comment |
I am working on a POC for implementing a kafka cluster in my project. I have setup a kafka cluster in my local machine with 3 brokers. Now I am sending messages to the Kafka server using Spring MVC REST service which is internally using Spring Kafka to produce and consume messages to and from the Kafka cluster. Now i am trying to send alerts when the consumer is unable to receive messages from the topic is down. I shutdown the only broker to which the consumer is connected. I am not getting any exception in my logs but i got he following warning messages.
0:20:35.500 [TEST_GROUP-0-C-1] WARN
o.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1,
groupId=TEST_GROUP] Connection to node 2147483645 could not be
established. Broker may not be available.
Is it possible to catch this warning message so that i can send alert when my consumer has lost connection? Below is my consumer code.
private static final Logger LOGGER = LoggerFactory.getLogger(ListenerServiceImpl.class);
@Autowired
Dao<RnMessage> messageDao;
@Autowired
MessageService messageService;
@KafkaListener(id = "TEST_GROUP", topics = "TESTQUEUE", errorHandler="eventQueueMessageListenerExceptionHandler")
public void listenMessageInQueue(String msg) {
try {
//String str = new String(msg, "UTF-8");
LOGGER.info("receiving payload='{}'", msg);
messageDao.saveMessage(msg);
messageService.sendMessageToOutQueue(msg);
}catch(Exception e) {
e.printStackTrace();
}
}
java spring apache-kafka spring-kafka
asked Dec 27 '18 at 15:48
Venu
9212
I am working on a POC for implementing a kafka cluster in my project. I have setup a kafka cluster in my local machine with 3 brokers. Now I am sending messages to the Kafka server using Spring MVC REST service which is internally using Spring Kafka to produce and consume messages to and from the Kafka cluster. Now i am trying to send alerts when the consumer is unable to receive messages from the topic is down. I shutdown the only broker to which the consumer is connected. I am not getting any exception in my logs but i got he following warning messages.
0:20:35.500 [TEST_GROUP-0-C-1] WARN
o.apache.kafka.clients.NetworkClient - [Consumer clientId=consumer-1,
groupId=TEST_GROUP] Connection to node 2147483645 could not be
established. Broker may not be available.
Is it possible to catch this warning message so that i can send alert when my consumer has lost connection? Below is my consumer code.
private static final Logger LOGGER = LoggerFactory.getLogger(ListenerServiceImpl.class);
@Autowired
Dao<RnMessage> messageDao;
@Autowired
MessageService messageService;
@KafkaListener(id = "TEST_GROUP", topics = "TESTQUEUE", errorHandler="eventQueueMessageListenerExceptionHandler")
public void listenMessageInQueue(String msg) {
try {
//String str = new String(msg, "UTF-8");
LOGGER.info("receiving payload='{}'", msg);
messageDao.saveMessage(msg);
messageService.sendMessageToOutQueue(msg);
}catch(Exception e) {
e.printStackTrace();
}
}
java spring apache-kafka spring-kafka
java spring apache-kafka spring-kafka
asked Dec 27 '18 at 15:48
Venu
9212
asked Dec 27 '18 at 15:48
Venu
9212
asked Dec 27 '18 at 15:48
Venu
9212
asked Dec 27 '18 at 15:48
Venu
9212
asked Dec 27 '18 at 15:48
Venu
9212
9212
show consumer configs, and also check zookeeper and kafka server are up
– Deadpool
Dec 27 '18 at 16:11
1
With a different setup, you would ship your logs to something like Splunk or Elasticsearch, which are configured with alerting on specific log conditions. Then, you wouldn't be adding boilerplate altering code into all your Kafka clients
– cricket_007
Dec 27 '18 at 16:17
add a comment |
show consumer configs, and also check zookeeper and kafka server are up
– Deadpool
Dec 27 '18 at 16:11
1
With a different setup, you would ship your logs to something like Splunk or Elasticsearch, which are configured with alerting on specific log conditions. Then, you wouldn't be adding boilerplate altering code into all your Kafka clients
– cricket_007
Dec 27 '18 at 16:17
show consumer configs, and also check zookeeper and kafka server are up
– Deadpool
Dec 27 '18 at 16:11
show consumer configs, and also check zookeeper and kafka server are up
– Deadpool
Dec 27 '18 at 16:11
1
1
With a different setup, you would ship your logs to something like Splunk or Elasticsearch, which are configured with alerting on specific log conditions. Then, you wouldn't be adding boilerplate altering code into all your Kafka clients
– cricket_007
Dec 27 '18 at 16:17
With a different setup, you would ship your logs to something like Splunk or Elasticsearch, which are configured with alerting on specific log conditions. Then, you wouldn't be adding boilerplate altering code into all your Kafka clients
– cricket_007
Dec 27 '18 at 16:17
add a comment |
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53947559%2fhow-to-catch-warning-broker-may-not-be-available-at-the-spring-kafka-listener%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53947559%2fhow-to-catch-warning-broker-may-not-be-available-at-the-spring-kafka-listener%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
show consumer configs, and also check zookeeper and kafka server are up
– Deadpool
Dec 27 '18 at 16:11
1
With a different setup, you would ship your logs to something like Splunk or Elasticsearch, which are configured with alerting on specific log conditions. Then, you wouldn't be adding boilerplate altering code into all your Kafka clients
– cricket_007
Dec 27 '18 at 16:17