How many users can MongooseIM handle on the PostgreSQL backend?

Multi tool use
Multi tool use












0















I am working on MongooseIM server with PostgreSQL as backend. I need all the basic features like:




  1. One to One chat with presence.

  2. Group Chat (Muc light only)

  3. MAM storage and all other basic features.


I can scale MongooseIM with clustering (with Mnesia for storing sessions). But I feel I may face bottle neck from the backend side using PostgreSQL. What is the rough estimate of the number of users that can be handled by PostgreSQL without the must have need for shift on No-sql like Riak. Because I don't want to mess-up the backend from the scalability point of view.



Example - Suppose I need to handle >10 Million users. Can I use PostgreSQL for that. If yes, then do I need special techniques like sharding in Postgresql to continue to work with it or at some point I must have to be shifted to Riak or other Nosql. What do you recommend?










share|improve this question





























    0















    I am working on MongooseIM server with PostgreSQL as backend. I need all the basic features like:




    1. One to One chat with presence.

    2. Group Chat (Muc light only)

    3. MAM storage and all other basic features.


    I can scale MongooseIM with clustering (with Mnesia for storing sessions). But I feel I may face bottle neck from the backend side using PostgreSQL. What is the rough estimate of the number of users that can be handled by PostgreSQL without the must have need for shift on No-sql like Riak. Because I don't want to mess-up the backend from the scalability point of view.



    Example - Suppose I need to handle >10 Million users. Can I use PostgreSQL for that. If yes, then do I need special techniques like sharding in Postgresql to continue to work with it or at some point I must have to be shifted to Riak or other Nosql. What do you recommend?










    share|improve this question



























      0












      0








      0








      I am working on MongooseIM server with PostgreSQL as backend. I need all the basic features like:




      1. One to One chat with presence.

      2. Group Chat (Muc light only)

      3. MAM storage and all other basic features.


      I can scale MongooseIM with clustering (with Mnesia for storing sessions). But I feel I may face bottle neck from the backend side using PostgreSQL. What is the rough estimate of the number of users that can be handled by PostgreSQL without the must have need for shift on No-sql like Riak. Because I don't want to mess-up the backend from the scalability point of view.



      Example - Suppose I need to handle >10 Million users. Can I use PostgreSQL for that. If yes, then do I need special techniques like sharding in Postgresql to continue to work with it or at some point I must have to be shifted to Riak or other Nosql. What do you recommend?










      share|improve this question
















      I am working on MongooseIM server with PostgreSQL as backend. I need all the basic features like:




      1. One to One chat with presence.

      2. Group Chat (Muc light only)

      3. MAM storage and all other basic features.


      I can scale MongooseIM with clustering (with Mnesia for storing sessions). But I feel I may face bottle neck from the backend side using PostgreSQL. What is the rough estimate of the number of users that can be handled by PostgreSQL without the must have need for shift on No-sql like Riak. Because I don't want to mess-up the backend from the scalability point of view.



      Example - Suppose I need to handle >10 Million users. Can I use PostgreSQL for that. If yes, then do I need special techniques like sharding in Postgresql to continue to work with it or at some point I must have to be shifted to Riak or other Nosql. What do you recommend?







      sql postgresql nosql mongoose-im






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Jan 2 at 10:33







      Shubham1164

















      asked Dec 29 '18 at 18:03









      Shubham1164Shubham1164

      7117




      7117
























          1 Answer
          1






          active

          oldest

          votes


















          1














          This is a complex matter. First of all, it depends a lot on the set of features you have enabled in MongooseIM, so any load tests done by others might not reflect your particular configuration and environment.



          Moreover, to actually be able to monitor all the important parameters in a system reaching tens of thousands of users, you'll probably need client-side performance tracking (delivery acks, roundtrip time) - this is not standardised by XMPP XEPs though, so implementations vary.



          This being said, Postgres will probably only allow you to scale into hundreds of thousands of online users. It might require table sharding and user sharding across MongooseIM clusters using XMPP federation to allow for communication between all users. DB nodes in this architecture are associated with MIM clusters, therefore do not share data across cluster boundaries. Setups along these lines are known to work in production.



          If you have to handle millions of users, you most likely have to switch from PostgreSQL to a database which can itself be more easily scaled. MongooseIM supports Cassandra, which currently seems to be the best bet. Personally, I'd curious to find out how TiKV (let me know, please, if you try that one) fares in such setups. It pretends to be MySQL on the wire, so should also work with MongooseIM, but operationally behaves similarly to Cassandra: automated replication, cluster scalability, fault tolerance, etc.



          Ultimately, though, you'll only know by testing yourself. Instagram shares some interesting stories about Postgres at scale which show the amount of maintenance required. Alas, load tests reaching millions of users are prohibitively expensive due to cloud costs, so such test results are not common.






          share|improve this answer
























          • Thank erszcz, that was very informative. It seems like I must not under-estimate the potencial of Postgresql and should not blindly pick up any nosql tech. Although nosql like Cassandra surely can be a wise option. But if Instagram can manage their work in Postgresql then I must take it serious too

            – Shubham1164
            Jan 2 at 14:23











          Your Answer






          StackExchange.ifUsing("editor", function () {
          StackExchange.using("externalEditor", function () {
          StackExchange.using("snippets", function () {
          StackExchange.snippets.init();
          });
          });
          }, "code-snippets");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "1"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53972068%2fhow-many-users-can-mongooseim-handle-on-the-postgresql-backend%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          1














          This is a complex matter. First of all, it depends a lot on the set of features you have enabled in MongooseIM, so any load tests done by others might not reflect your particular configuration and environment.



          Moreover, to actually be able to monitor all the important parameters in a system reaching tens of thousands of users, you'll probably need client-side performance tracking (delivery acks, roundtrip time) - this is not standardised by XMPP XEPs though, so implementations vary.



          This being said, Postgres will probably only allow you to scale into hundreds of thousands of online users. It might require table sharding and user sharding across MongooseIM clusters using XMPP federation to allow for communication between all users. DB nodes in this architecture are associated with MIM clusters, therefore do not share data across cluster boundaries. Setups along these lines are known to work in production.



          If you have to handle millions of users, you most likely have to switch from PostgreSQL to a database which can itself be more easily scaled. MongooseIM supports Cassandra, which currently seems to be the best bet. Personally, I'd curious to find out how TiKV (let me know, please, if you try that one) fares in such setups. It pretends to be MySQL on the wire, so should also work with MongooseIM, but operationally behaves similarly to Cassandra: automated replication, cluster scalability, fault tolerance, etc.



          Ultimately, though, you'll only know by testing yourself. Instagram shares some interesting stories about Postgres at scale which show the amount of maintenance required. Alas, load tests reaching millions of users are prohibitively expensive due to cloud costs, so such test results are not common.






          share|improve this answer
























          • Thank erszcz, that was very informative. It seems like I must not under-estimate the potencial of Postgresql and should not blindly pick up any nosql tech. Although nosql like Cassandra surely can be a wise option. But if Instagram can manage their work in Postgresql then I must take it serious too

            – Shubham1164
            Jan 2 at 14:23
















          1














          This is a complex matter. First of all, it depends a lot on the set of features you have enabled in MongooseIM, so any load tests done by others might not reflect your particular configuration and environment.



          Moreover, to actually be able to monitor all the important parameters in a system reaching tens of thousands of users, you'll probably need client-side performance tracking (delivery acks, roundtrip time) - this is not standardised by XMPP XEPs though, so implementations vary.



          This being said, Postgres will probably only allow you to scale into hundreds of thousands of online users. It might require table sharding and user sharding across MongooseIM clusters using XMPP federation to allow for communication between all users. DB nodes in this architecture are associated with MIM clusters, therefore do not share data across cluster boundaries. Setups along these lines are known to work in production.



          If you have to handle millions of users, you most likely have to switch from PostgreSQL to a database which can itself be more easily scaled. MongooseIM supports Cassandra, which currently seems to be the best bet. Personally, I'd curious to find out how TiKV (let me know, please, if you try that one) fares in such setups. It pretends to be MySQL on the wire, so should also work with MongooseIM, but operationally behaves similarly to Cassandra: automated replication, cluster scalability, fault tolerance, etc.



          Ultimately, though, you'll only know by testing yourself. Instagram shares some interesting stories about Postgres at scale which show the amount of maintenance required. Alas, load tests reaching millions of users are prohibitively expensive due to cloud costs, so such test results are not common.






          share|improve this answer
























          • Thank erszcz, that was very informative. It seems like I must not under-estimate the potencial of Postgresql and should not blindly pick up any nosql tech. Although nosql like Cassandra surely can be a wise option. But if Instagram can manage their work in Postgresql then I must take it serious too

            – Shubham1164
            Jan 2 at 14:23














          1












          1








          1







          This is a complex matter. First of all, it depends a lot on the set of features you have enabled in MongooseIM, so any load tests done by others might not reflect your particular configuration and environment.



          Moreover, to actually be able to monitor all the important parameters in a system reaching tens of thousands of users, you'll probably need client-side performance tracking (delivery acks, roundtrip time) - this is not standardised by XMPP XEPs though, so implementations vary.



          This being said, Postgres will probably only allow you to scale into hundreds of thousands of online users. It might require table sharding and user sharding across MongooseIM clusters using XMPP federation to allow for communication between all users. DB nodes in this architecture are associated with MIM clusters, therefore do not share data across cluster boundaries. Setups along these lines are known to work in production.



          If you have to handle millions of users, you most likely have to switch from PostgreSQL to a database which can itself be more easily scaled. MongooseIM supports Cassandra, which currently seems to be the best bet. Personally, I'd curious to find out how TiKV (let me know, please, if you try that one) fares in such setups. It pretends to be MySQL on the wire, so should also work with MongooseIM, but operationally behaves similarly to Cassandra: automated replication, cluster scalability, fault tolerance, etc.



          Ultimately, though, you'll only know by testing yourself. Instagram shares some interesting stories about Postgres at scale which show the amount of maintenance required. Alas, load tests reaching millions of users are prohibitively expensive due to cloud costs, so such test results are not common.






          share|improve this answer













          This is a complex matter. First of all, it depends a lot on the set of features you have enabled in MongooseIM, so any load tests done by others might not reflect your particular configuration and environment.



          Moreover, to actually be able to monitor all the important parameters in a system reaching tens of thousands of users, you'll probably need client-side performance tracking (delivery acks, roundtrip time) - this is not standardised by XMPP XEPs though, so implementations vary.



          This being said, Postgres will probably only allow you to scale into hundreds of thousands of online users. It might require table sharding and user sharding across MongooseIM clusters using XMPP federation to allow for communication between all users. DB nodes in this architecture are associated with MIM clusters, therefore do not share data across cluster boundaries. Setups along these lines are known to work in production.



          If you have to handle millions of users, you most likely have to switch from PostgreSQL to a database which can itself be more easily scaled. MongooseIM supports Cassandra, which currently seems to be the best bet. Personally, I'd curious to find out how TiKV (let me know, please, if you try that one) fares in such setups. It pretends to be MySQL on the wire, so should also work with MongooseIM, but operationally behaves similarly to Cassandra: automated replication, cluster scalability, fault tolerance, etc.



          Ultimately, though, you'll only know by testing yourself. Instagram shares some interesting stories about Postgres at scale which show the amount of maintenance required. Alas, load tests reaching millions of users are prohibitively expensive due to cloud costs, so such test results are not common.







          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered Jan 2 at 11:23









          erszczerszcz

          1,059715




          1,059715













          • Thank erszcz, that was very informative. It seems like I must not under-estimate the potencial of Postgresql and should not blindly pick up any nosql tech. Although nosql like Cassandra surely can be a wise option. But if Instagram can manage their work in Postgresql then I must take it serious too

            – Shubham1164
            Jan 2 at 14:23



















          • Thank erszcz, that was very informative. It seems like I must not under-estimate the potencial of Postgresql and should not blindly pick up any nosql tech. Although nosql like Cassandra surely can be a wise option. But if Instagram can manage their work in Postgresql then I must take it serious too

            – Shubham1164
            Jan 2 at 14:23

















          Thank erszcz, that was very informative. It seems like I must not under-estimate the potencial of Postgresql and should not blindly pick up any nosql tech. Although nosql like Cassandra surely can be a wise option. But if Instagram can manage their work in Postgresql then I must take it serious too

          – Shubham1164
          Jan 2 at 14:23





          Thank erszcz, that was very informative. It seems like I must not under-estimate the potencial of Postgresql and should not blindly pick up any nosql tech. Although nosql like Cassandra surely can be a wise option. But if Instagram can manage their work in Postgresql then I must take it serious too

          – Shubham1164
          Jan 2 at 14:23




















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53972068%2fhow-many-users-can-mongooseim-handle-on-the-postgresql-backend%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          ez,HWuYsJWhxerfWY,14I,Np4QyZ,57f,T,UFRakWfb,VaA 7NM1ZIvm Y7PBNq7WSdeHFAn4wxgXx,dd,D2ZyTGQ kA2Uiv3
          Nx2Rvpkt UK0c9IB MKdiRCqYnYkMTWBMijCpMTl4GTh2si8YTsvPV50B4n5pRTzB9uLDyW76,oN0NWO6H,U yD,p

          Popular posts from this blog

          Monofisismo

          Angular Downloading a file using contenturl with Basic Authentication

          Olmecas