How to re-run all failed tasks in Apache Airflow?

Multi tool use
Multi tool use












0














I have a Apache Airflow DAG with tens of thousands of tasks and after a run, say a handful of them failed.



I fixed the bug that caused some tasks to fail and I would like to re-run ONLY FAILED TASKS.
This SO post suggests using the GUI to "clear" failed task:



How to restart a failed task on Airflow



This approach works if you have a handful number of failed tasks.



I am wondering if we can bypass the GUI and do it problematically, through command line something like:



airflow_clear_failed_tasks dag_id execution_data 









share|improve this question



























    0














    I have a Apache Airflow DAG with tens of thousands of tasks and after a run, say a handful of them failed.



    I fixed the bug that caused some tasks to fail and I would like to re-run ONLY FAILED TASKS.
    This SO post suggests using the GUI to "clear" failed task:



    How to restart a failed task on Airflow



    This approach works if you have a handful number of failed tasks.



    I am wondering if we can bypass the GUI and do it problematically, through command line something like:



    airflow_clear_failed_tasks dag_id execution_data 









    share|improve this question

























      0












      0








      0







      I have a Apache Airflow DAG with tens of thousands of tasks and after a run, say a handful of them failed.



      I fixed the bug that caused some tasks to fail and I would like to re-run ONLY FAILED TASKS.
      This SO post suggests using the GUI to "clear" failed task:



      How to restart a failed task on Airflow



      This approach works if you have a handful number of failed tasks.



      I am wondering if we can bypass the GUI and do it problematically, through command line something like:



      airflow_clear_failed_tasks dag_id execution_data 









      share|improve this question













      I have a Apache Airflow DAG with tens of thousands of tasks and after a run, say a handful of them failed.



      I fixed the bug that caused some tasks to fail and I would like to re-run ONLY FAILED TASKS.
      This SO post suggests using the GUI to "clear" failed task:



      How to restart a failed task on Airflow



      This approach works if you have a handful number of failed tasks.



      I am wondering if we can bypass the GUI and do it problematically, through command line something like:



      airflow_clear_failed_tasks dag_id execution_data 






      airflow






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked Dec 27 at 13:32









      motam79

      864923




      864923
























          1 Answer
          1






          active

          oldest

          votes


















          1














          Use the following command to clear only failed tasks:



          airflow clear [-s START_DATE] [-e END_DATE] --only_failed dag_id


          Documentation: https://airflow.readthedocs.io/en/stable/cli.html#clear






          share|improve this answer





















            Your Answer






            StackExchange.ifUsing("editor", function () {
            StackExchange.using("externalEditor", function () {
            StackExchange.using("snippets", function () {
            StackExchange.snippets.init();
            });
            });
            }, "code-snippets");

            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "1"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53945945%2fhow-to-re-run-all-failed-tasks-in-apache-airflow%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            1 Answer
            1






            active

            oldest

            votes








            1 Answer
            1






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            1














            Use the following command to clear only failed tasks:



            airflow clear [-s START_DATE] [-e END_DATE] --only_failed dag_id


            Documentation: https://airflow.readthedocs.io/en/stable/cli.html#clear






            share|improve this answer


























              1














              Use the following command to clear only failed tasks:



              airflow clear [-s START_DATE] [-e END_DATE] --only_failed dag_id


              Documentation: https://airflow.readthedocs.io/en/stable/cli.html#clear






              share|improve this answer
























                1












                1








                1






                Use the following command to clear only failed tasks:



                airflow clear [-s START_DATE] [-e END_DATE] --only_failed dag_id


                Documentation: https://airflow.readthedocs.io/en/stable/cli.html#clear






                share|improve this answer












                Use the following command to clear only failed tasks:



                airflow clear [-s START_DATE] [-e END_DATE] --only_failed dag_id


                Documentation: https://airflow.readthedocs.io/en/stable/cli.html#clear







                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered Dec 27 at 13:52









                kaxil

                2,483827




                2,483827






























                    draft saved

                    draft discarded




















































                    Thanks for contributing an answer to Stack Overflow!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    To learn more, see our tips on writing great answers.





                    Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


                    Please pay close attention to the following guidance:


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53945945%2fhow-to-re-run-all-failed-tasks-in-apache-airflow%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    49hf NXPoH8OM94Fgo5uqwlO,UxNMM,t,o IK 5R,i9xFhSXpOA
                    ZiWh2,Hb,aVKBqcalYx8yOHB9qyS,dIEjODj,5mAuoAhQXyzKm2MGtRqfPlWPd4wJYzM4F7Z,XNTArxsHyKyyclsbmidi N

                    Popular posts from this blog

                    Monofisismo

                    Angular Downloading a file using contenturl with Basic Authentication

                    Olmecas