Find duplicate message in Service bus topics

Multi tool use
I want to find duplicate messages in ServiceBusTopics.Below is my logic.If I find any duplicate message am adding to the list and sending to another service bus topic. But I wasn't able to read all the service bus messages that are present in the subscription before looping through the list.I want to read all the messages from the sevicebus and then loop through it.I'm not sure how should I stop the execution of loops until I read all the messages from suscription
private static string _serviceBusConn = "servicebusconnectionstring";
private static string _serviceBustopic = "topic1";
private static string _topic = "test_v1";
static void Main(string args)
{
IList<string> items = new List<string>();
int count;
IList<string> itemsRepeated = new List<string>();
var subClient = SubscriptionClient.CreateFromConnectionString(_serviceBusConn, _serviceBustopic, "DevTest");
subClient.OnMessage(m =>
{
Stream stream = m.GetBody<Stream>();
StreamReader reader = new StreamReader(stream);
string s = reader.ReadToEnd();
Console.WriteLine(s);
items.Add(s);
});
List<string> copy1= new List<string>(items);
List<string> copy2 = new List<string>(items);
foreach (var item in copy1)
{
count = 0;
foreach (var itemtopic in copy2)
{
count++;
Console.WriteLine("{0}{1}{2}", items, itemtopic, count);
if (item.Equals(itemtopic))
{
count++;
}
if (count > 1)
{
Console.WriteLine(count);
itemsRepeated.Add(itemtopic);
}
}
}
foreach (var repeateditem in itemsRepeated)
{
SendMessage(repeateditem);
}
}
static void SendMessage(string message)
{
var topicClient = TopicClient.CreateFromConnectionString(_serviceBusConn, _topic);
var msg = new BrokeredMessage(message);
topicClient.Send(msg);
}
c# azureservicebus
add a comment |
I want to find duplicate messages in ServiceBusTopics.Below is my logic.If I find any duplicate message am adding to the list and sending to another service bus topic. But I wasn't able to read all the service bus messages that are present in the subscription before looping through the list.I want to read all the messages from the sevicebus and then loop through it.I'm not sure how should I stop the execution of loops until I read all the messages from suscription
private static string _serviceBusConn = "servicebusconnectionstring";
private static string _serviceBustopic = "topic1";
private static string _topic = "test_v1";
static void Main(string args)
{
IList<string> items = new List<string>();
int count;
IList<string> itemsRepeated = new List<string>();
var subClient = SubscriptionClient.CreateFromConnectionString(_serviceBusConn, _serviceBustopic, "DevTest");
subClient.OnMessage(m =>
{
Stream stream = m.GetBody<Stream>();
StreamReader reader = new StreamReader(stream);
string s = reader.ReadToEnd();
Console.WriteLine(s);
items.Add(s);
});
List<string> copy1= new List<string>(items);
List<string> copy2 = new List<string>(items);
foreach (var item in copy1)
{
count = 0;
foreach (var itemtopic in copy2)
{
count++;
Console.WriteLine("{0}{1}{2}", items, itemtopic, count);
if (item.Equals(itemtopic))
{
count++;
}
if (count > 1)
{
Console.WriteLine(count);
itemsRepeated.Add(itemtopic);
}
}
}
foreach (var repeateditem in itemsRepeated)
{
SendMessage(repeateditem);
}
}
static void SendMessage(string message)
{
var topicClient = TopicClient.CreateFromConnectionString(_serviceBusConn, _topic);
var msg = new BrokeredMessage(message);
topicClient.Send(msg);
}
c# azureservicebus
add a comment |
I want to find duplicate messages in ServiceBusTopics.Below is my logic.If I find any duplicate message am adding to the list and sending to another service bus topic. But I wasn't able to read all the service bus messages that are present in the subscription before looping through the list.I want to read all the messages from the sevicebus and then loop through it.I'm not sure how should I stop the execution of loops until I read all the messages from suscription
private static string _serviceBusConn = "servicebusconnectionstring";
private static string _serviceBustopic = "topic1";
private static string _topic = "test_v1";
static void Main(string args)
{
IList<string> items = new List<string>();
int count;
IList<string> itemsRepeated = new List<string>();
var subClient = SubscriptionClient.CreateFromConnectionString(_serviceBusConn, _serviceBustopic, "DevTest");
subClient.OnMessage(m =>
{
Stream stream = m.GetBody<Stream>();
StreamReader reader = new StreamReader(stream);
string s = reader.ReadToEnd();
Console.WriteLine(s);
items.Add(s);
});
List<string> copy1= new List<string>(items);
List<string> copy2 = new List<string>(items);
foreach (var item in copy1)
{
count = 0;
foreach (var itemtopic in copy2)
{
count++;
Console.WriteLine("{0}{1}{2}", items, itemtopic, count);
if (item.Equals(itemtopic))
{
count++;
}
if (count > 1)
{
Console.WriteLine(count);
itemsRepeated.Add(itemtopic);
}
}
}
foreach (var repeateditem in itemsRepeated)
{
SendMessage(repeateditem);
}
}
static void SendMessage(string message)
{
var topicClient = TopicClient.CreateFromConnectionString(_serviceBusConn, _topic);
var msg = new BrokeredMessage(message);
topicClient.Send(msg);
}
c# azureservicebus
I want to find duplicate messages in ServiceBusTopics.Below is my logic.If I find any duplicate message am adding to the list and sending to another service bus topic. But I wasn't able to read all the service bus messages that are present in the subscription before looping through the list.I want to read all the messages from the sevicebus and then loop through it.I'm not sure how should I stop the execution of loops until I read all the messages from suscription
private static string _serviceBusConn = "servicebusconnectionstring";
private static string _serviceBustopic = "topic1";
private static string _topic = "test_v1";
static void Main(string args)
{
IList<string> items = new List<string>();
int count;
IList<string> itemsRepeated = new List<string>();
var subClient = SubscriptionClient.CreateFromConnectionString(_serviceBusConn, _serviceBustopic, "DevTest");
subClient.OnMessage(m =>
{
Stream stream = m.GetBody<Stream>();
StreamReader reader = new StreamReader(stream);
string s = reader.ReadToEnd();
Console.WriteLine(s);
items.Add(s);
});
List<string> copy1= new List<string>(items);
List<string> copy2 = new List<string>(items);
foreach (var item in copy1)
{
count = 0;
foreach (var itemtopic in copy2)
{
count++;
Console.WriteLine("{0}{1}{2}", items, itemtopic, count);
if (item.Equals(itemtopic))
{
count++;
}
if (count > 1)
{
Console.WriteLine(count);
itemsRepeated.Add(itemtopic);
}
}
}
foreach (var repeateditem in itemsRepeated)
{
SendMessage(repeateditem);
}
}
static void SendMessage(string message)
{
var topicClient = TopicClient.CreateFromConnectionString(_serviceBusConn, _topic);
var msg = new BrokeredMessage(message);
topicClient.Send(msg);
}
c# azureservicebus
c# azureservicebus
asked Jan 2 at 20:00
saraswathisaraswathi
246
246
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
You can use ReceiveBatch(Int32) method to receive messages from a topic subscription in a batch. You can get the number of messages in a topic subscription using Getsubscription() method of namespace manager.
var namespaceManager = NamespaceManager.CreateFromConnectionString(connnectionString);
var subscriptionDescription = namespaceManager.GetSubscription(topicName, subscriptionName);
var totalMessageCount= subscriptionDescription .MessageCount;
Now you can call the ReceiveBatch(minimumMessageCount) within a loop and terminate the loop when the received message count reaches the totalMessageCount.
int receivedMessageCount = 0;
List<BrokeredMessage> MessageList = new List<BrokeredMessage>();
do{
var messageList = subClient.ReceiveBatch(100);
receivedMessageCount += messageList.Count;
MessageList.AddRange(messageList);
}while(receivedMessageCount < totalMessageCount);
Now the MessageList will have all the messages in the topic subscription.You can use your custom logic on the message list to perform duplicate detection and forward that to another subscription.
Note: To receive all the messages from a ServiceBusTopic, you have to receive all the messages from all the topic subscriptions within that topic.
add a comment |
if (count > 1)
{
Console.WriteLine(count);
itemsRepeated.Add(itemtopic);
}
you can break the loop by using break like this
if (count > 1)
{
Console.WriteLine(count);
itemsRepeated.Add(itemtopic);
break;
}
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54012435%2ffind-duplicate-message-in-service-bus-topics%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
You can use ReceiveBatch(Int32) method to receive messages from a topic subscription in a batch. You can get the number of messages in a topic subscription using Getsubscription() method of namespace manager.
var namespaceManager = NamespaceManager.CreateFromConnectionString(connnectionString);
var subscriptionDescription = namespaceManager.GetSubscription(topicName, subscriptionName);
var totalMessageCount= subscriptionDescription .MessageCount;
Now you can call the ReceiveBatch(minimumMessageCount) within a loop and terminate the loop when the received message count reaches the totalMessageCount.
int receivedMessageCount = 0;
List<BrokeredMessage> MessageList = new List<BrokeredMessage>();
do{
var messageList = subClient.ReceiveBatch(100);
receivedMessageCount += messageList.Count;
MessageList.AddRange(messageList);
}while(receivedMessageCount < totalMessageCount);
Now the MessageList will have all the messages in the topic subscription.You can use your custom logic on the message list to perform duplicate detection and forward that to another subscription.
Note: To receive all the messages from a ServiceBusTopic, you have to receive all the messages from all the topic subscriptions within that topic.
add a comment |
You can use ReceiveBatch(Int32) method to receive messages from a topic subscription in a batch. You can get the number of messages in a topic subscription using Getsubscription() method of namespace manager.
var namespaceManager = NamespaceManager.CreateFromConnectionString(connnectionString);
var subscriptionDescription = namespaceManager.GetSubscription(topicName, subscriptionName);
var totalMessageCount= subscriptionDescription .MessageCount;
Now you can call the ReceiveBatch(minimumMessageCount) within a loop and terminate the loop when the received message count reaches the totalMessageCount.
int receivedMessageCount = 0;
List<BrokeredMessage> MessageList = new List<BrokeredMessage>();
do{
var messageList = subClient.ReceiveBatch(100);
receivedMessageCount += messageList.Count;
MessageList.AddRange(messageList);
}while(receivedMessageCount < totalMessageCount);
Now the MessageList will have all the messages in the topic subscription.You can use your custom logic on the message list to perform duplicate detection and forward that to another subscription.
Note: To receive all the messages from a ServiceBusTopic, you have to receive all the messages from all the topic subscriptions within that topic.
add a comment |
You can use ReceiveBatch(Int32) method to receive messages from a topic subscription in a batch. You can get the number of messages in a topic subscription using Getsubscription() method of namespace manager.
var namespaceManager = NamespaceManager.CreateFromConnectionString(connnectionString);
var subscriptionDescription = namespaceManager.GetSubscription(topicName, subscriptionName);
var totalMessageCount= subscriptionDescription .MessageCount;
Now you can call the ReceiveBatch(minimumMessageCount) within a loop and terminate the loop when the received message count reaches the totalMessageCount.
int receivedMessageCount = 0;
List<BrokeredMessage> MessageList = new List<BrokeredMessage>();
do{
var messageList = subClient.ReceiveBatch(100);
receivedMessageCount += messageList.Count;
MessageList.AddRange(messageList);
}while(receivedMessageCount < totalMessageCount);
Now the MessageList will have all the messages in the topic subscription.You can use your custom logic on the message list to perform duplicate detection and forward that to another subscription.
Note: To receive all the messages from a ServiceBusTopic, you have to receive all the messages from all the topic subscriptions within that topic.
You can use ReceiveBatch(Int32) method to receive messages from a topic subscription in a batch. You can get the number of messages in a topic subscription using Getsubscription() method of namespace manager.
var namespaceManager = NamespaceManager.CreateFromConnectionString(connnectionString);
var subscriptionDescription = namespaceManager.GetSubscription(topicName, subscriptionName);
var totalMessageCount= subscriptionDescription .MessageCount;
Now you can call the ReceiveBatch(minimumMessageCount) within a loop and terminate the loop when the received message count reaches the totalMessageCount.
int receivedMessageCount = 0;
List<BrokeredMessage> MessageList = new List<BrokeredMessage>();
do{
var messageList = subClient.ReceiveBatch(100);
receivedMessageCount += messageList.Count;
MessageList.AddRange(messageList);
}while(receivedMessageCount < totalMessageCount);
Now the MessageList will have all the messages in the topic subscription.You can use your custom logic on the message list to perform duplicate detection and forward that to another subscription.
Note: To receive all the messages from a ServiceBusTopic, you have to receive all the messages from all the topic subscriptions within that topic.
edited Jan 3 at 14:07
answered Jan 3 at 4:01


Ranjith eswaranRanjith eswaran
181112
181112
add a comment |
add a comment |
if (count > 1)
{
Console.WriteLine(count);
itemsRepeated.Add(itemtopic);
}
you can break the loop by using break like this
if (count > 1)
{
Console.WriteLine(count);
itemsRepeated.Add(itemtopic);
break;
}
add a comment |
if (count > 1)
{
Console.WriteLine(count);
itemsRepeated.Add(itemtopic);
}
you can break the loop by using break like this
if (count > 1)
{
Console.WriteLine(count);
itemsRepeated.Add(itemtopic);
break;
}
add a comment |
if (count > 1)
{
Console.WriteLine(count);
itemsRepeated.Add(itemtopic);
}
you can break the loop by using break like this
if (count > 1)
{
Console.WriteLine(count);
itemsRepeated.Add(itemtopic);
break;
}
if (count > 1)
{
Console.WriteLine(count);
itemsRepeated.Add(itemtopic);
}
you can break the loop by using break like this
if (count > 1)
{
Console.WriteLine(count);
itemsRepeated.Add(itemtopic);
break;
}
answered Jan 3 at 4:32


ShubhamShubham
1925
1925
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54012435%2ffind-duplicate-message-in-service-bus-topics%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
eIsoG5pSLrVAiRk2Fy5