How to save variables before killing a (too long) running python script?












1















I've been running a python script using:




  • python

  • Anacondaon

  • linux server (4.15.0-36-generic #39~16.04.1-Ubuntu)


For two weeks now and I think it will take more than a month to finish. This is already too long and so I would like to kill the process but before that I would like to save some lists that have been constructed during the last two weeks.



So far I tried:



1) using pyrasite-shell with one the PID associated with my python script (I've more than 20 PIDs associated with my script according to htop) but it does not work (no shell opens).



2) I also tried to find some kernel address using ps -aux | grep python3 and then connect to this kernel via something like ipython console --existing d3d55f0d-52f1-4660-a43b-1fedb698452f.json and then look for local variables with locals()in the prompt shell but this did not work (none of my lists appear in the list of local variables).



Ideally, I would like to connect the python kernel running my code and save all the current lists via pickle. Or any other idea to retrieve the lists would be great.










share|improve this question

























  • how about a finally statement with try/catch?

    – SRT HellKitty
    Jan 3 at 16:20






  • 2





    Colloquially you setup a logging system like this before you run your script...have you tried rconsole?

    – Matt Messersmith
    Jan 3 at 16:20











  • No, I've not tried rconsole (and I did not find any information about it). What is it? do you have some link that may explain how that works? Thanks.

    – guillaume lecue
    Jan 4 at 10:18
















1















I've been running a python script using:




  • python

  • Anacondaon

  • linux server (4.15.0-36-generic #39~16.04.1-Ubuntu)


For two weeks now and I think it will take more than a month to finish. This is already too long and so I would like to kill the process but before that I would like to save some lists that have been constructed during the last two weeks.



So far I tried:



1) using pyrasite-shell with one the PID associated with my python script (I've more than 20 PIDs associated with my script according to htop) but it does not work (no shell opens).



2) I also tried to find some kernel address using ps -aux | grep python3 and then connect to this kernel via something like ipython console --existing d3d55f0d-52f1-4660-a43b-1fedb698452f.json and then look for local variables with locals()in the prompt shell but this did not work (none of my lists appear in the list of local variables).



Ideally, I would like to connect the python kernel running my code and save all the current lists via pickle. Or any other idea to retrieve the lists would be great.










share|improve this question

























  • how about a finally statement with try/catch?

    – SRT HellKitty
    Jan 3 at 16:20






  • 2





    Colloquially you setup a logging system like this before you run your script...have you tried rconsole?

    – Matt Messersmith
    Jan 3 at 16:20











  • No, I've not tried rconsole (and I did not find any information about it). What is it? do you have some link that may explain how that works? Thanks.

    – guillaume lecue
    Jan 4 at 10:18














1












1








1








I've been running a python script using:




  • python

  • Anacondaon

  • linux server (4.15.0-36-generic #39~16.04.1-Ubuntu)


For two weeks now and I think it will take more than a month to finish. This is already too long and so I would like to kill the process but before that I would like to save some lists that have been constructed during the last two weeks.



So far I tried:



1) using pyrasite-shell with one the PID associated with my python script (I've more than 20 PIDs associated with my script according to htop) but it does not work (no shell opens).



2) I also tried to find some kernel address using ps -aux | grep python3 and then connect to this kernel via something like ipython console --existing d3d55f0d-52f1-4660-a43b-1fedb698452f.json and then look for local variables with locals()in the prompt shell but this did not work (none of my lists appear in the list of local variables).



Ideally, I would like to connect the python kernel running my code and save all the current lists via pickle. Or any other idea to retrieve the lists would be great.










share|improve this question
















I've been running a python script using:




  • python

  • Anacondaon

  • linux server (4.15.0-36-generic #39~16.04.1-Ubuntu)


For two weeks now and I think it will take more than a month to finish. This is already too long and so I would like to kill the process but before that I would like to save some lists that have been constructed during the last two weeks.



So far I tried:



1) using pyrasite-shell with one the PID associated with my python script (I've more than 20 PIDs associated with my script according to htop) but it does not work (no shell opens).



2) I also tried to find some kernel address using ps -aux | grep python3 and then connect to this kernel via something like ipython console --existing d3d55f0d-52f1-4660-a43b-1fedb698452f.json and then look for local variables with locals()in the prompt shell but this did not work (none of my lists appear in the list of local variables).



Ideally, I would like to connect the python kernel running my code and save all the current lists via pickle. Or any other idea to retrieve the lists would be great.







python-3.x






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Jan 3 at 16:15









jalazbe

511318




511318










asked Jan 3 at 14:44









guillaume lecueguillaume lecue

84




84













  • how about a finally statement with try/catch?

    – SRT HellKitty
    Jan 3 at 16:20






  • 2





    Colloquially you setup a logging system like this before you run your script...have you tried rconsole?

    – Matt Messersmith
    Jan 3 at 16:20











  • No, I've not tried rconsole (and I did not find any information about it). What is it? do you have some link that may explain how that works? Thanks.

    – guillaume lecue
    Jan 4 at 10:18



















  • how about a finally statement with try/catch?

    – SRT HellKitty
    Jan 3 at 16:20






  • 2





    Colloquially you setup a logging system like this before you run your script...have you tried rconsole?

    – Matt Messersmith
    Jan 3 at 16:20











  • No, I've not tried rconsole (and I did not find any information about it). What is it? do you have some link that may explain how that works? Thanks.

    – guillaume lecue
    Jan 4 at 10:18

















how about a finally statement with try/catch?

– SRT HellKitty
Jan 3 at 16:20





how about a finally statement with try/catch?

– SRT HellKitty
Jan 3 at 16:20




2




2





Colloquially you setup a logging system like this before you run your script...have you tried rconsole?

– Matt Messersmith
Jan 3 at 16:20





Colloquially you setup a logging system like this before you run your script...have you tried rconsole?

– Matt Messersmith
Jan 3 at 16:20













No, I've not tried rconsole (and I did not find any information about it). What is it? do you have some link that may explain how that works? Thanks.

– guillaume lecue
Jan 4 at 10:18





No, I've not tried rconsole (and I did not find any information about it). What is it? do you have some link that may explain how that works? Thanks.

– guillaume lecue
Jan 4 at 10:18












0






active

oldest

votes












Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54024538%2fhow-to-save-variables-before-killing-a-too-long-running-python-script%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























0






active

oldest

votes








0






active

oldest

votes









active

oldest

votes






active

oldest

votes
















draft saved

draft discarded




















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54024538%2fhow-to-save-variables-before-killing-a-too-long-running-python-script%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Monofisismo

Angular Downloading a file using contenturl with Basic Authentication

Olmecas