will io direction operation lock the file?





.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}







-1















i have a growing nginx log file about 20G already, and i wish to rotate it.



1, i mv the old log file to a new log file



2, i do > old_log_file.log to truncate the old log file in about 2~3 seconds



if there's a lock(write lock?) on the old log file when i doing the truncating(about 2~3 seconds)?



at that 2~3s period, nginx returns 502 for waiting to append logs to old log file until lock released?



thank you for explaining.










share|improve this question























  • mv old_log_file.log new_log_file.log && touch old_log_file.log

    – David C. Rankin
    Jan 5 at 9:02











  • i don't think your comand would work, because mv does not change the inode of old log file. And nginx would still log into the "new_log_file" because of the inode. And you create a new log file named as olg log file, but the inode changed, nginx wouldn't write into it until nginx -s reload the config file.

    – touchstone
    Jan 7 at 6:08











  • If that is how nginx associates the log, then you may well be correct. Then why not cp -a old_log_file.log new_log_file.log && :> old_log_file.log?

    – David C. Rankin
    Jan 7 at 6:17











  • i tried 2 truncating methods: > and 'logrotate', they all caused the nginx output 502 -- very short period, after then 200. and logrotate's handling procedure is as you said truncate after cp. now i have no idea what to debug this bug...

    – touchstone
    Jan 7 at 6:22











  • It is not a bug, it is a feature.

    – Basile Starynkevitch
    Jan 7 at 8:23


















-1















i have a growing nginx log file about 20G already, and i wish to rotate it.



1, i mv the old log file to a new log file



2, i do > old_log_file.log to truncate the old log file in about 2~3 seconds



if there's a lock(write lock?) on the old log file when i doing the truncating(about 2~3 seconds)?



at that 2~3s period, nginx returns 502 for waiting to append logs to old log file until lock released?



thank you for explaining.










share|improve this question























  • mv old_log_file.log new_log_file.log && touch old_log_file.log

    – David C. Rankin
    Jan 5 at 9:02











  • i don't think your comand would work, because mv does not change the inode of old log file. And nginx would still log into the "new_log_file" because of the inode. And you create a new log file named as olg log file, but the inode changed, nginx wouldn't write into it until nginx -s reload the config file.

    – touchstone
    Jan 7 at 6:08











  • If that is how nginx associates the log, then you may well be correct. Then why not cp -a old_log_file.log new_log_file.log && :> old_log_file.log?

    – David C. Rankin
    Jan 7 at 6:17











  • i tried 2 truncating methods: > and 'logrotate', they all caused the nginx output 502 -- very short period, after then 200. and logrotate's handling procedure is as you said truncate after cp. now i have no idea what to debug this bug...

    – touchstone
    Jan 7 at 6:22











  • It is not a bug, it is a feature.

    – Basile Starynkevitch
    Jan 7 at 8:23














-1












-1








-1








i have a growing nginx log file about 20G already, and i wish to rotate it.



1, i mv the old log file to a new log file



2, i do > old_log_file.log to truncate the old log file in about 2~3 seconds



if there's a lock(write lock?) on the old log file when i doing the truncating(about 2~3 seconds)?



at that 2~3s period, nginx returns 502 for waiting to append logs to old log file until lock released?



thank you for explaining.










share|improve this question














i have a growing nginx log file about 20G already, and i wish to rotate it.



1, i mv the old log file to a new log file



2, i do > old_log_file.log to truncate the old log file in about 2~3 seconds



if there's a lock(write lock?) on the old log file when i doing the truncating(about 2~3 seconds)?



at that 2~3s period, nginx returns 502 for waiting to append logs to old log file until lock released?



thank you for explaining.







linux redirect io truncate






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Jan 4 at 6:21









touchstonetouchstone

16718




16718













  • mv old_log_file.log new_log_file.log && touch old_log_file.log

    – David C. Rankin
    Jan 5 at 9:02











  • i don't think your comand would work, because mv does not change the inode of old log file. And nginx would still log into the "new_log_file" because of the inode. And you create a new log file named as olg log file, but the inode changed, nginx wouldn't write into it until nginx -s reload the config file.

    – touchstone
    Jan 7 at 6:08











  • If that is how nginx associates the log, then you may well be correct. Then why not cp -a old_log_file.log new_log_file.log && :> old_log_file.log?

    – David C. Rankin
    Jan 7 at 6:17











  • i tried 2 truncating methods: > and 'logrotate', they all caused the nginx output 502 -- very short period, after then 200. and logrotate's handling procedure is as you said truncate after cp. now i have no idea what to debug this bug...

    – touchstone
    Jan 7 at 6:22











  • It is not a bug, it is a feature.

    – Basile Starynkevitch
    Jan 7 at 8:23



















  • mv old_log_file.log new_log_file.log && touch old_log_file.log

    – David C. Rankin
    Jan 5 at 9:02











  • i don't think your comand would work, because mv does not change the inode of old log file. And nginx would still log into the "new_log_file" because of the inode. And you create a new log file named as olg log file, but the inode changed, nginx wouldn't write into it until nginx -s reload the config file.

    – touchstone
    Jan 7 at 6:08











  • If that is how nginx associates the log, then you may well be correct. Then why not cp -a old_log_file.log new_log_file.log && :> old_log_file.log?

    – David C. Rankin
    Jan 7 at 6:17











  • i tried 2 truncating methods: > and 'logrotate', they all caused the nginx output 502 -- very short period, after then 200. and logrotate's handling procedure is as you said truncate after cp. now i have no idea what to debug this bug...

    – touchstone
    Jan 7 at 6:22











  • It is not a bug, it is a feature.

    – Basile Starynkevitch
    Jan 7 at 8:23

















mv old_log_file.log new_log_file.log && touch old_log_file.log

– David C. Rankin
Jan 5 at 9:02





mv old_log_file.log new_log_file.log && touch old_log_file.log

– David C. Rankin
Jan 5 at 9:02













i don't think your comand would work, because mv does not change the inode of old log file. And nginx would still log into the "new_log_file" because of the inode. And you create a new log file named as olg log file, but the inode changed, nginx wouldn't write into it until nginx -s reload the config file.

– touchstone
Jan 7 at 6:08





i don't think your comand would work, because mv does not change the inode of old log file. And nginx would still log into the "new_log_file" because of the inode. And you create a new log file named as olg log file, but the inode changed, nginx wouldn't write into it until nginx -s reload the config file.

– touchstone
Jan 7 at 6:08













If that is how nginx associates the log, then you may well be correct. Then why not cp -a old_log_file.log new_log_file.log && :> old_log_file.log?

– David C. Rankin
Jan 7 at 6:17





If that is how nginx associates the log, then you may well be correct. Then why not cp -a old_log_file.log new_log_file.log && :> old_log_file.log?

– David C. Rankin
Jan 7 at 6:17













i tried 2 truncating methods: > and 'logrotate', they all caused the nginx output 502 -- very short period, after then 200. and logrotate's handling procedure is as you said truncate after cp. now i have no idea what to debug this bug...

– touchstone
Jan 7 at 6:22





i tried 2 truncating methods: > and 'logrotate', they all caused the nginx output 502 -- very short period, after then 200. and logrotate's handling procedure is as you said truncate after cp. now i have no idea what to debug this bug...

– touchstone
Jan 7 at 6:22













It is not a bug, it is a feature.

– Basile Starynkevitch
Jan 7 at 8:23





It is not a bug, it is a feature.

– Basile Starynkevitch
Jan 7 at 8:23












2 Answers
2






active

oldest

votes


















1














On Linux, there is (almost) no mandatory file locks (more precisely, there used to be some mandatory locking feature in the kernel, but it is deprecated and you really should avoid using it). File locking happens with flock(2) or lockf(3) and is advisory and should be explicit (e.g. with flock(1) command, or some program calling flock or lockf).



So every locking related to files is practically a convention between all the software using that file (and mv(1) or the redirection by your shell don't use file locking).



Remember that a file on Linux is mostly an i-node (see inode(7)) which could have zero, one or several file paths (see path_resolution(7) and be aware of link(2), rename(2), unlink(2)) and used thru some file descriptor. Read ALP (and perhaps Operating Systems: Three Easy Pieces) for more.



No file locking happens in the scenario of your question (and the i-nodes and file descriptors involved are independent).



Consider using logrotate(8).



Some software provide a way to reload their configuration and re-open log files. You should read the documentation of your nginx.






share|improve this answer


























  • logrotate 's source code shows that it first cp the old one then truncate the old one, inode doesn't change. and actually i used logrotate after > causes 502 issue, but the issues still exist, then i asked this question. sorry for my english.

    – touchstone
    Jan 7 at 5:57













  • logrotate and kill -USR1 nginx.pid which would reload the config file, all i tried does not solve the issue.

    – touchstone
    Jan 7 at 6:04











  • fine. i learned that "> " doesn't lock the file even though 2~3s' truncating at least. thank you a lot, i would keep looking for answers for my 502.

    – touchstone
    Jan 7 at 6:15











  • Reason in terms of i-nodes and file descriptors. You'll understand better what happens in your computer

    – Basile Starynkevitch
    Jan 7 at 6:36











  • maybe, the file offset is decreasing quickly as > is truncating and at the same time nginx is appending logs with a shrinking offset (i don't know how to describe a unstable offset)

    – touchstone
    Jan 7 at 8:22



















1














It depends on application if it locks the file. Application that generates this log file must have option to clear log file. One example is in editor like vim file can be externally modified while it is still open in editor.






share|improve this answer
























    Your Answer






    StackExchange.ifUsing("editor", function () {
    StackExchange.using("externalEditor", function () {
    StackExchange.using("snippets", function () {
    StackExchange.snippets.init();
    });
    });
    }, "code-snippets");

    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "1"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54033940%2fwill-io-direction-operation-lock-the-file%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    2 Answers
    2






    active

    oldest

    votes








    2 Answers
    2






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    1














    On Linux, there is (almost) no mandatory file locks (more precisely, there used to be some mandatory locking feature in the kernel, but it is deprecated and you really should avoid using it). File locking happens with flock(2) or lockf(3) and is advisory and should be explicit (e.g. with flock(1) command, or some program calling flock or lockf).



    So every locking related to files is practically a convention between all the software using that file (and mv(1) or the redirection by your shell don't use file locking).



    Remember that a file on Linux is mostly an i-node (see inode(7)) which could have zero, one or several file paths (see path_resolution(7) and be aware of link(2), rename(2), unlink(2)) and used thru some file descriptor. Read ALP (and perhaps Operating Systems: Three Easy Pieces) for more.



    No file locking happens in the scenario of your question (and the i-nodes and file descriptors involved are independent).



    Consider using logrotate(8).



    Some software provide a way to reload their configuration and re-open log files. You should read the documentation of your nginx.






    share|improve this answer


























    • logrotate 's source code shows that it first cp the old one then truncate the old one, inode doesn't change. and actually i used logrotate after > causes 502 issue, but the issues still exist, then i asked this question. sorry for my english.

      – touchstone
      Jan 7 at 5:57













    • logrotate and kill -USR1 nginx.pid which would reload the config file, all i tried does not solve the issue.

      – touchstone
      Jan 7 at 6:04











    • fine. i learned that "> " doesn't lock the file even though 2~3s' truncating at least. thank you a lot, i would keep looking for answers for my 502.

      – touchstone
      Jan 7 at 6:15











    • Reason in terms of i-nodes and file descriptors. You'll understand better what happens in your computer

      – Basile Starynkevitch
      Jan 7 at 6:36











    • maybe, the file offset is decreasing quickly as > is truncating and at the same time nginx is appending logs with a shrinking offset (i don't know how to describe a unstable offset)

      – touchstone
      Jan 7 at 8:22
















    1














    On Linux, there is (almost) no mandatory file locks (more precisely, there used to be some mandatory locking feature in the kernel, but it is deprecated and you really should avoid using it). File locking happens with flock(2) or lockf(3) and is advisory and should be explicit (e.g. with flock(1) command, or some program calling flock or lockf).



    So every locking related to files is practically a convention between all the software using that file (and mv(1) or the redirection by your shell don't use file locking).



    Remember that a file on Linux is mostly an i-node (see inode(7)) which could have zero, one or several file paths (see path_resolution(7) and be aware of link(2), rename(2), unlink(2)) and used thru some file descriptor. Read ALP (and perhaps Operating Systems: Three Easy Pieces) for more.



    No file locking happens in the scenario of your question (and the i-nodes and file descriptors involved are independent).



    Consider using logrotate(8).



    Some software provide a way to reload their configuration and re-open log files. You should read the documentation of your nginx.






    share|improve this answer


























    • logrotate 's source code shows that it first cp the old one then truncate the old one, inode doesn't change. and actually i used logrotate after > causes 502 issue, but the issues still exist, then i asked this question. sorry for my english.

      – touchstone
      Jan 7 at 5:57













    • logrotate and kill -USR1 nginx.pid which would reload the config file, all i tried does not solve the issue.

      – touchstone
      Jan 7 at 6:04











    • fine. i learned that "> " doesn't lock the file even though 2~3s' truncating at least. thank you a lot, i would keep looking for answers for my 502.

      – touchstone
      Jan 7 at 6:15











    • Reason in terms of i-nodes and file descriptors. You'll understand better what happens in your computer

      – Basile Starynkevitch
      Jan 7 at 6:36











    • maybe, the file offset is decreasing quickly as > is truncating and at the same time nginx is appending logs with a shrinking offset (i don't know how to describe a unstable offset)

      – touchstone
      Jan 7 at 8:22














    1












    1








    1







    On Linux, there is (almost) no mandatory file locks (more precisely, there used to be some mandatory locking feature in the kernel, but it is deprecated and you really should avoid using it). File locking happens with flock(2) or lockf(3) and is advisory and should be explicit (e.g. with flock(1) command, or some program calling flock or lockf).



    So every locking related to files is practically a convention between all the software using that file (and mv(1) or the redirection by your shell don't use file locking).



    Remember that a file on Linux is mostly an i-node (see inode(7)) which could have zero, one or several file paths (see path_resolution(7) and be aware of link(2), rename(2), unlink(2)) and used thru some file descriptor. Read ALP (and perhaps Operating Systems: Three Easy Pieces) for more.



    No file locking happens in the scenario of your question (and the i-nodes and file descriptors involved are independent).



    Consider using logrotate(8).



    Some software provide a way to reload their configuration and re-open log files. You should read the documentation of your nginx.






    share|improve this answer















    On Linux, there is (almost) no mandatory file locks (more precisely, there used to be some mandatory locking feature in the kernel, but it is deprecated and you really should avoid using it). File locking happens with flock(2) or lockf(3) and is advisory and should be explicit (e.g. with flock(1) command, or some program calling flock or lockf).



    So every locking related to files is practically a convention between all the software using that file (and mv(1) or the redirection by your shell don't use file locking).



    Remember that a file on Linux is mostly an i-node (see inode(7)) which could have zero, one or several file paths (see path_resolution(7) and be aware of link(2), rename(2), unlink(2)) and used thru some file descriptor. Read ALP (and perhaps Operating Systems: Three Easy Pieces) for more.



    No file locking happens in the scenario of your question (and the i-nodes and file descriptors involved are independent).



    Consider using logrotate(8).



    Some software provide a way to reload their configuration and re-open log files. You should read the documentation of your nginx.







    share|improve this answer














    share|improve this answer



    share|improve this answer








    edited Jan 5 at 17:24

























    answered Jan 5 at 8:38









    Basile StarynkevitchBasile Starynkevitch

    180k13174375




    180k13174375













    • logrotate 's source code shows that it first cp the old one then truncate the old one, inode doesn't change. and actually i used logrotate after > causes 502 issue, but the issues still exist, then i asked this question. sorry for my english.

      – touchstone
      Jan 7 at 5:57













    • logrotate and kill -USR1 nginx.pid which would reload the config file, all i tried does not solve the issue.

      – touchstone
      Jan 7 at 6:04











    • fine. i learned that "> " doesn't lock the file even though 2~3s' truncating at least. thank you a lot, i would keep looking for answers for my 502.

      – touchstone
      Jan 7 at 6:15











    • Reason in terms of i-nodes and file descriptors. You'll understand better what happens in your computer

      – Basile Starynkevitch
      Jan 7 at 6:36











    • maybe, the file offset is decreasing quickly as > is truncating and at the same time nginx is appending logs with a shrinking offset (i don't know how to describe a unstable offset)

      – touchstone
      Jan 7 at 8:22



















    • logrotate 's source code shows that it first cp the old one then truncate the old one, inode doesn't change. and actually i used logrotate after > causes 502 issue, but the issues still exist, then i asked this question. sorry for my english.

      – touchstone
      Jan 7 at 5:57













    • logrotate and kill -USR1 nginx.pid which would reload the config file, all i tried does not solve the issue.

      – touchstone
      Jan 7 at 6:04











    • fine. i learned that "> " doesn't lock the file even though 2~3s' truncating at least. thank you a lot, i would keep looking for answers for my 502.

      – touchstone
      Jan 7 at 6:15











    • Reason in terms of i-nodes and file descriptors. You'll understand better what happens in your computer

      – Basile Starynkevitch
      Jan 7 at 6:36











    • maybe, the file offset is decreasing quickly as > is truncating and at the same time nginx is appending logs with a shrinking offset (i don't know how to describe a unstable offset)

      – touchstone
      Jan 7 at 8:22

















    logrotate 's source code shows that it first cp the old one then truncate the old one, inode doesn't change. and actually i used logrotate after > causes 502 issue, but the issues still exist, then i asked this question. sorry for my english.

    – touchstone
    Jan 7 at 5:57







    logrotate 's source code shows that it first cp the old one then truncate the old one, inode doesn't change. and actually i used logrotate after > causes 502 issue, but the issues still exist, then i asked this question. sorry for my english.

    – touchstone
    Jan 7 at 5:57















    logrotate and kill -USR1 nginx.pid which would reload the config file, all i tried does not solve the issue.

    – touchstone
    Jan 7 at 6:04





    logrotate and kill -USR1 nginx.pid which would reload the config file, all i tried does not solve the issue.

    – touchstone
    Jan 7 at 6:04













    fine. i learned that "> " doesn't lock the file even though 2~3s' truncating at least. thank you a lot, i would keep looking for answers for my 502.

    – touchstone
    Jan 7 at 6:15





    fine. i learned that "> " doesn't lock the file even though 2~3s' truncating at least. thank you a lot, i would keep looking for answers for my 502.

    – touchstone
    Jan 7 at 6:15













    Reason in terms of i-nodes and file descriptors. You'll understand better what happens in your computer

    – Basile Starynkevitch
    Jan 7 at 6:36





    Reason in terms of i-nodes and file descriptors. You'll understand better what happens in your computer

    – Basile Starynkevitch
    Jan 7 at 6:36













    maybe, the file offset is decreasing quickly as > is truncating and at the same time nginx is appending logs with a shrinking offset (i don't know how to describe a unstable offset)

    – touchstone
    Jan 7 at 8:22





    maybe, the file offset is decreasing quickly as > is truncating and at the same time nginx is appending logs with a shrinking offset (i don't know how to describe a unstable offset)

    – touchstone
    Jan 7 at 8:22













    1














    It depends on application if it locks the file. Application that generates this log file must have option to clear log file. One example is in editor like vim file can be externally modified while it is still open in editor.






    share|improve this answer




























      1














      It depends on application if it locks the file. Application that generates this log file must have option to clear log file. One example is in editor like vim file can be externally modified while it is still open in editor.






      share|improve this answer


























        1












        1








        1







        It depends on application if it locks the file. Application that generates this log file must have option to clear log file. One example is in editor like vim file can be externally modified while it is still open in editor.






        share|improve this answer













        It depends on application if it locks the file. Application that generates this log file must have option to clear log file. One example is in editor like vim file can be externally modified while it is still open in editor.







        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Jan 5 at 8:36









        anandanand

        1337




        1337






























            draft saved

            draft discarded




















































            Thanks for contributing an answer to Stack Overflow!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54033940%2fwill-io-direction-operation-lock-the-file%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Monofisismo

            Angular Downloading a file using contenturl with Basic Authentication

            Olmecas