Apache Kafka Connect With Springboot

Multi tool use
Multi tool use












1















I'm trying to find examples of kafka connect with springboot. It looks like there is no spring boot integration for kafka connect. Can some one point me in the right direction to be able to listen to changes on mysql db?










share|improve this question





























    1















    I'm trying to find examples of kafka connect with springboot. It looks like there is no spring boot integration for kafka connect. Can some one point me in the right direction to be able to listen to changes on mysql db?










    share|improve this question



























      1












      1








      1


      1






      I'm trying to find examples of kafka connect with springboot. It looks like there is no spring boot integration for kafka connect. Can some one point me in the right direction to be able to listen to changes on mysql db?










      share|improve this question
















      I'm trying to find examples of kafka connect with springboot. It looks like there is no spring boot integration for kafka connect. Can some one point me in the right direction to be able to listen to changes on mysql db?







      spring-boot apache-kafka apache-kafka-connect






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Dec 30 '18 at 5:38









      cricket_007

      81k1142110




      81k1142110










      asked Dec 30 '18 at 0:31









      user3310115user3310115

      250311




      250311
























          1 Answer
          1






          active

          oldest

          votes


















          2














          Kafka Connect doesn't really need Spring Boot because there is nothing for you to code for it, and it really works best when ran in distrbited-mode, as a cluster, not embedded within other applications. I suppose if you did want to do it, then you could copy relevent portions of the source code, but that of course isn't using Spring Boot, and you'd have to wire it all yourself



          The framework itself consists of a few core Java dependencies that have already been written (Debezium or Confluent JDBC Connector, for your mysql example), and two config files. One for Kafka Connect to know the bootstrap servers, serializers, etc. and another for the actual MySQL connector.






          share|improve this answer


























          • So then how can we integrate with a java application. Can you point out to some examples

            – user3310115
            Dec 30 '18 at 10:29











          • So its something like download docker image, and change config and run right? And one more question connector need to be install on database to stream the changes? Or its just a config changes?

            – Deadpool
            Dec 30 '18 at 15:36











          • @user331 I suggest searching for "Debezium Embedded" if you want to do this. As I said, out of the box, Kafka Connect isn't exactly designed to be ran as part of another application

            – cricket_007
            Dec 31 '18 at 2:03













          • @Deadpool Docker is just one option, not required, but yes, just configs. And depends on the database, like Mongo/MySql/Postgres use the replication oplog. Couchbase has its own CDC protocols, though, for example. These act over the network, though, not running on the database server itself

            – cricket_007
            Dec 31 '18 at 2:06











          • i'm using postgres database, so it will use the replication oplog, but i have a question some of the organizations does not allow access to log file, then any other options? @cricket_007

            – Deadpool
            Dec 31 '18 at 2:09













          Your Answer






          StackExchange.ifUsing("editor", function () {
          StackExchange.using("externalEditor", function () {
          StackExchange.using("snippets", function () {
          StackExchange.snippets.init();
          });
          });
          }, "code-snippets");

          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "1"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53974402%2fapache-kafka-connect-with-springboot%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          2














          Kafka Connect doesn't really need Spring Boot because there is nothing for you to code for it, and it really works best when ran in distrbited-mode, as a cluster, not embedded within other applications. I suppose if you did want to do it, then you could copy relevent portions of the source code, but that of course isn't using Spring Boot, and you'd have to wire it all yourself



          The framework itself consists of a few core Java dependencies that have already been written (Debezium or Confluent JDBC Connector, for your mysql example), and two config files. One for Kafka Connect to know the bootstrap servers, serializers, etc. and another for the actual MySQL connector.






          share|improve this answer


























          • So then how can we integrate with a java application. Can you point out to some examples

            – user3310115
            Dec 30 '18 at 10:29











          • So its something like download docker image, and change config and run right? And one more question connector need to be install on database to stream the changes? Or its just a config changes?

            – Deadpool
            Dec 30 '18 at 15:36











          • @user331 I suggest searching for "Debezium Embedded" if you want to do this. As I said, out of the box, Kafka Connect isn't exactly designed to be ran as part of another application

            – cricket_007
            Dec 31 '18 at 2:03













          • @Deadpool Docker is just one option, not required, but yes, just configs. And depends on the database, like Mongo/MySql/Postgres use the replication oplog. Couchbase has its own CDC protocols, though, for example. These act over the network, though, not running on the database server itself

            – cricket_007
            Dec 31 '18 at 2:06











          • i'm using postgres database, so it will use the replication oplog, but i have a question some of the organizations does not allow access to log file, then any other options? @cricket_007

            – Deadpool
            Dec 31 '18 at 2:09


















          2














          Kafka Connect doesn't really need Spring Boot because there is nothing for you to code for it, and it really works best when ran in distrbited-mode, as a cluster, not embedded within other applications. I suppose if you did want to do it, then you could copy relevent portions of the source code, but that of course isn't using Spring Boot, and you'd have to wire it all yourself



          The framework itself consists of a few core Java dependencies that have already been written (Debezium or Confluent JDBC Connector, for your mysql example), and two config files. One for Kafka Connect to know the bootstrap servers, serializers, etc. and another for the actual MySQL connector.






          share|improve this answer


























          • So then how can we integrate with a java application. Can you point out to some examples

            – user3310115
            Dec 30 '18 at 10:29











          • So its something like download docker image, and change config and run right? And one more question connector need to be install on database to stream the changes? Or its just a config changes?

            – Deadpool
            Dec 30 '18 at 15:36











          • @user331 I suggest searching for "Debezium Embedded" if you want to do this. As I said, out of the box, Kafka Connect isn't exactly designed to be ran as part of another application

            – cricket_007
            Dec 31 '18 at 2:03













          • @Deadpool Docker is just one option, not required, but yes, just configs. And depends on the database, like Mongo/MySql/Postgres use the replication oplog. Couchbase has its own CDC protocols, though, for example. These act over the network, though, not running on the database server itself

            – cricket_007
            Dec 31 '18 at 2:06











          • i'm using postgres database, so it will use the replication oplog, but i have a question some of the organizations does not allow access to log file, then any other options? @cricket_007

            – Deadpool
            Dec 31 '18 at 2:09
















          2












          2








          2







          Kafka Connect doesn't really need Spring Boot because there is nothing for you to code for it, and it really works best when ran in distrbited-mode, as a cluster, not embedded within other applications. I suppose if you did want to do it, then you could copy relevent portions of the source code, but that of course isn't using Spring Boot, and you'd have to wire it all yourself



          The framework itself consists of a few core Java dependencies that have already been written (Debezium or Confluent JDBC Connector, for your mysql example), and two config files. One for Kafka Connect to know the bootstrap servers, serializers, etc. and another for the actual MySQL connector.






          share|improve this answer















          Kafka Connect doesn't really need Spring Boot because there is nothing for you to code for it, and it really works best when ran in distrbited-mode, as a cluster, not embedded within other applications. I suppose if you did want to do it, then you could copy relevent portions of the source code, but that of course isn't using Spring Boot, and you'd have to wire it all yourself



          The framework itself consists of a few core Java dependencies that have already been written (Debezium or Confluent JDBC Connector, for your mysql example), and two config files. One for Kafka Connect to know the bootstrap servers, serializers, etc. and another for the actual MySQL connector.







          share|improve this answer














          share|improve this answer



          share|improve this answer








          edited Jan 5 at 23:17

























          answered Dec 30 '18 at 5:39









          cricket_007cricket_007

          81k1142110




          81k1142110













          • So then how can we integrate with a java application. Can you point out to some examples

            – user3310115
            Dec 30 '18 at 10:29











          • So its something like download docker image, and change config and run right? And one more question connector need to be install on database to stream the changes? Or its just a config changes?

            – Deadpool
            Dec 30 '18 at 15:36











          • @user331 I suggest searching for "Debezium Embedded" if you want to do this. As I said, out of the box, Kafka Connect isn't exactly designed to be ran as part of another application

            – cricket_007
            Dec 31 '18 at 2:03













          • @Deadpool Docker is just one option, not required, but yes, just configs. And depends on the database, like Mongo/MySql/Postgres use the replication oplog. Couchbase has its own CDC protocols, though, for example. These act over the network, though, not running on the database server itself

            – cricket_007
            Dec 31 '18 at 2:06











          • i'm using postgres database, so it will use the replication oplog, but i have a question some of the organizations does not allow access to log file, then any other options? @cricket_007

            – Deadpool
            Dec 31 '18 at 2:09





















          • So then how can we integrate with a java application. Can you point out to some examples

            – user3310115
            Dec 30 '18 at 10:29











          • So its something like download docker image, and change config and run right? And one more question connector need to be install on database to stream the changes? Or its just a config changes?

            – Deadpool
            Dec 30 '18 at 15:36











          • @user331 I suggest searching for "Debezium Embedded" if you want to do this. As I said, out of the box, Kafka Connect isn't exactly designed to be ran as part of another application

            – cricket_007
            Dec 31 '18 at 2:03













          • @Deadpool Docker is just one option, not required, but yes, just configs. And depends on the database, like Mongo/MySql/Postgres use the replication oplog. Couchbase has its own CDC protocols, though, for example. These act over the network, though, not running on the database server itself

            – cricket_007
            Dec 31 '18 at 2:06











          • i'm using postgres database, so it will use the replication oplog, but i have a question some of the organizations does not allow access to log file, then any other options? @cricket_007

            – Deadpool
            Dec 31 '18 at 2:09



















          So then how can we integrate with a java application. Can you point out to some examples

          – user3310115
          Dec 30 '18 at 10:29





          So then how can we integrate with a java application. Can you point out to some examples

          – user3310115
          Dec 30 '18 at 10:29













          So its something like download docker image, and change config and run right? And one more question connector need to be install on database to stream the changes? Or its just a config changes?

          – Deadpool
          Dec 30 '18 at 15:36





          So its something like download docker image, and change config and run right? And one more question connector need to be install on database to stream the changes? Or its just a config changes?

          – Deadpool
          Dec 30 '18 at 15:36













          @user331 I suggest searching for "Debezium Embedded" if you want to do this. As I said, out of the box, Kafka Connect isn't exactly designed to be ran as part of another application

          – cricket_007
          Dec 31 '18 at 2:03







          @user331 I suggest searching for "Debezium Embedded" if you want to do this. As I said, out of the box, Kafka Connect isn't exactly designed to be ran as part of another application

          – cricket_007
          Dec 31 '18 at 2:03















          @Deadpool Docker is just one option, not required, but yes, just configs. And depends on the database, like Mongo/MySql/Postgres use the replication oplog. Couchbase has its own CDC protocols, though, for example. These act over the network, though, not running on the database server itself

          – cricket_007
          Dec 31 '18 at 2:06





          @Deadpool Docker is just one option, not required, but yes, just configs. And depends on the database, like Mongo/MySql/Postgres use the replication oplog. Couchbase has its own CDC protocols, though, for example. These act over the network, though, not running on the database server itself

          – cricket_007
          Dec 31 '18 at 2:06













          i'm using postgres database, so it will use the replication oplog, but i have a question some of the organizations does not allow access to log file, then any other options? @cricket_007

          – Deadpool
          Dec 31 '18 at 2:09







          i'm using postgres database, so it will use the replication oplog, but i have a question some of the organizations does not allow access to log file, then any other options? @cricket_007

          – Deadpool
          Dec 31 '18 at 2:09




















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Stack Overflow!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53974402%2fapache-kafka-connect-with-springboot%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          hrPIERG2GUX240zmzUMR,P0KX gZ6FFyh F7o2 Ik9a,DporV,Z2Q,l,l,JDjm023ByO,1f AR2cNOm9LRI 8JbN,NPDab27o
          puXkpCLW,iBKVa39zjl2XTTTAs gncRpf9i5M gj2a,fW 6qD8A5cWHZksxYCOJHYBWLL,T1,Y5B1C j D5uhm

          Popular posts from this blog

          Monofisismo

          Angular Downloading a file using contenturl with Basic Authentication

          Olmecas