will io direction operation lock the file?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}
i have a growing nginx log file about 20G already, and i wish to rotate it.
1, i mv the old log file to a new log file
2, i do > old_log_file.log to truncate the old log file in about 2~3 seconds
if there's a lock(write lock?) on the old log file when i doing the truncating(about 2~3 seconds)?
at that 2~3s period, nginx returns 502 for waiting to append logs to old log file until lock released?
thank you for explaining.
linux redirect io truncate
|
show 1 more comment
i have a growing nginx log file about 20G already, and i wish to rotate it.
1, i mv the old log file to a new log file
2, i do > old_log_file.log to truncate the old log file in about 2~3 seconds
if there's a lock(write lock?) on the old log file when i doing the truncating(about 2~3 seconds)?
at that 2~3s period, nginx returns 502 for waiting to append logs to old log file until lock released?
thank you for explaining.
linux redirect io truncate
mv old_log_file.log new_log_file.log && touch old_log_file.log
– David C. Rankin
Jan 5 at 9:02
i don't think your comand would work, becausemvdoes not change theinodeof old log file. Andnginxwould still log into the "new_log_file" because of theinode. And you create a new log file named as olg log file, but theinodechanged, nginx wouldn't write into it untilnginx -s reloadthe config file.
– touchstone
Jan 7 at 6:08
If that is hownginxassociates the log, then you may well be correct. Then why notcp -a old_log_file.log new_log_file.log && :> old_log_file.log?
– David C. Rankin
Jan 7 at 6:17
i tried 2 truncating methods:>and 'logrotate', they all caused the nginx output 502 -- very short period, after then 200. andlogrotate's handling procedure is as you said truncate aftercp. now i have no idea what to debug this bug...
– touchstone
Jan 7 at 6:22
It is not a bug, it is a feature.
– Basile Starynkevitch
Jan 7 at 8:23
|
show 1 more comment
i have a growing nginx log file about 20G already, and i wish to rotate it.
1, i mv the old log file to a new log file
2, i do > old_log_file.log to truncate the old log file in about 2~3 seconds
if there's a lock(write lock?) on the old log file when i doing the truncating(about 2~3 seconds)?
at that 2~3s period, nginx returns 502 for waiting to append logs to old log file until lock released?
thank you for explaining.
linux redirect io truncate
i have a growing nginx log file about 20G already, and i wish to rotate it.
1, i mv the old log file to a new log file
2, i do > old_log_file.log to truncate the old log file in about 2~3 seconds
if there's a lock(write lock?) on the old log file when i doing the truncating(about 2~3 seconds)?
at that 2~3s period, nginx returns 502 for waiting to append logs to old log file until lock released?
thank you for explaining.
linux redirect io truncate
linux redirect io truncate
asked Jan 4 at 6:21
touchstonetouchstone
16718
16718
mv old_log_file.log new_log_file.log && touch old_log_file.log
– David C. Rankin
Jan 5 at 9:02
i don't think your comand would work, becausemvdoes not change theinodeof old log file. Andnginxwould still log into the "new_log_file" because of theinode. And you create a new log file named as olg log file, but theinodechanged, nginx wouldn't write into it untilnginx -s reloadthe config file.
– touchstone
Jan 7 at 6:08
If that is hownginxassociates the log, then you may well be correct. Then why notcp -a old_log_file.log new_log_file.log && :> old_log_file.log?
– David C. Rankin
Jan 7 at 6:17
i tried 2 truncating methods:>and 'logrotate', they all caused the nginx output 502 -- very short period, after then 200. andlogrotate's handling procedure is as you said truncate aftercp. now i have no idea what to debug this bug...
– touchstone
Jan 7 at 6:22
It is not a bug, it is a feature.
– Basile Starynkevitch
Jan 7 at 8:23
|
show 1 more comment
mv old_log_file.log new_log_file.log && touch old_log_file.log
– David C. Rankin
Jan 5 at 9:02
i don't think your comand would work, becausemvdoes not change theinodeof old log file. Andnginxwould still log into the "new_log_file" because of theinode. And you create a new log file named as olg log file, but theinodechanged, nginx wouldn't write into it untilnginx -s reloadthe config file.
– touchstone
Jan 7 at 6:08
If that is hownginxassociates the log, then you may well be correct. Then why notcp -a old_log_file.log new_log_file.log && :> old_log_file.log?
– David C. Rankin
Jan 7 at 6:17
i tried 2 truncating methods:>and 'logrotate', they all caused the nginx output 502 -- very short period, after then 200. andlogrotate's handling procedure is as you said truncate aftercp. now i have no idea what to debug this bug...
– touchstone
Jan 7 at 6:22
It is not a bug, it is a feature.
– Basile Starynkevitch
Jan 7 at 8:23
mv old_log_file.log new_log_file.log && touch old_log_file.log– David C. Rankin
Jan 5 at 9:02
mv old_log_file.log new_log_file.log && touch old_log_file.log– David C. Rankin
Jan 5 at 9:02
i don't think your comand would work, because
mv does not change the inode of old log file. And nginx would still log into the "new_log_file" because of the inode. And you create a new log file named as olg log file, but the inode changed, nginx wouldn't write into it until nginx -s reload the config file.– touchstone
Jan 7 at 6:08
i don't think your comand would work, because
mv does not change the inode of old log file. And nginx would still log into the "new_log_file" because of the inode. And you create a new log file named as olg log file, but the inode changed, nginx wouldn't write into it until nginx -s reload the config file.– touchstone
Jan 7 at 6:08
If that is how
nginx associates the log, then you may well be correct. Then why not cp -a old_log_file.log new_log_file.log && :> old_log_file.log?– David C. Rankin
Jan 7 at 6:17
If that is how
nginx associates the log, then you may well be correct. Then why not cp -a old_log_file.log new_log_file.log && :> old_log_file.log?– David C. Rankin
Jan 7 at 6:17
i tried 2 truncating methods:
> and 'logrotate', they all caused the nginx output 502 -- very short period, after then 200. and logrotate's handling procedure is as you said truncate after cp. now i have no idea what to debug this bug...– touchstone
Jan 7 at 6:22
i tried 2 truncating methods:
> and 'logrotate', they all caused the nginx output 502 -- very short period, after then 200. and logrotate's handling procedure is as you said truncate after cp. now i have no idea what to debug this bug...– touchstone
Jan 7 at 6:22
It is not a bug, it is a feature.
– Basile Starynkevitch
Jan 7 at 8:23
It is not a bug, it is a feature.
– Basile Starynkevitch
Jan 7 at 8:23
|
show 1 more comment
2 Answers
2
active
oldest
votes
On Linux, there is (almost) no mandatory file locks (more precisely, there used to be some mandatory locking feature in the kernel, but it is deprecated and you really should avoid using it). File locking happens with flock(2) or lockf(3) and is advisory and should be explicit (e.g. with flock(1) command, or some program calling flock or lockf).
So every locking related to files is practically a convention between all the software using that file (and mv(1) or the redirection by your shell don't use file locking).
Remember that a file on Linux is mostly an i-node (see inode(7)) which could have zero, one or several file paths (see path_resolution(7) and be aware of link(2), rename(2), unlink(2)) and used thru some file descriptor. Read ALP (and perhaps Operating Systems: Three Easy Pieces) for more.
No file locking happens in the scenario of your question (and the i-nodes and file descriptors involved are independent).
Consider using logrotate(8).
Some software provide a way to reload their configuration and re-open log files. You should read the documentation of your nginx.
logrotate's source code shows that it firstcpthe old one then truncate the old one,inodedoesn't change. and actually i usedlogrotateafter>causes 502 issue, but the issues still exist, then i asked this question. sorry for my english.
– touchstone
Jan 7 at 5:57
logrotateandkill -USR1 nginx.pidwhich would reload the config file, all i tried does not solve the issue.
– touchstone
Jan 7 at 6:04
fine. i learned that ">" doesn't lock the file even though 2~3s' truncating at least. thank you a lot, i would keep looking for answers for my 502.
– touchstone
Jan 7 at 6:15
Reason in terms of i-nodes and file descriptors. You'll understand better what happens in your computer
– Basile Starynkevitch
Jan 7 at 6:36
maybe, the file offset is decreasing quickly as>is truncating and at the same time nginx is appending logs with a shrinking offset (i don't know how to describe a unstable offset)
– touchstone
Jan 7 at 8:22
add a comment |
It depends on application if it locks the file. Application that generates this log file must have option to clear log file. One example is in editor like vim file can be externally modified while it is still open in editor.
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54033940%2fwill-io-direction-operation-lock-the-file%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
On Linux, there is (almost) no mandatory file locks (more precisely, there used to be some mandatory locking feature in the kernel, but it is deprecated and you really should avoid using it). File locking happens with flock(2) or lockf(3) and is advisory and should be explicit (e.g. with flock(1) command, or some program calling flock or lockf).
So every locking related to files is practically a convention between all the software using that file (and mv(1) or the redirection by your shell don't use file locking).
Remember that a file on Linux is mostly an i-node (see inode(7)) which could have zero, one or several file paths (see path_resolution(7) and be aware of link(2), rename(2), unlink(2)) and used thru some file descriptor. Read ALP (and perhaps Operating Systems: Three Easy Pieces) for more.
No file locking happens in the scenario of your question (and the i-nodes and file descriptors involved are independent).
Consider using logrotate(8).
Some software provide a way to reload their configuration and re-open log files. You should read the documentation of your nginx.
logrotate's source code shows that it firstcpthe old one then truncate the old one,inodedoesn't change. and actually i usedlogrotateafter>causes 502 issue, but the issues still exist, then i asked this question. sorry for my english.
– touchstone
Jan 7 at 5:57
logrotateandkill -USR1 nginx.pidwhich would reload the config file, all i tried does not solve the issue.
– touchstone
Jan 7 at 6:04
fine. i learned that ">" doesn't lock the file even though 2~3s' truncating at least. thank you a lot, i would keep looking for answers for my 502.
– touchstone
Jan 7 at 6:15
Reason in terms of i-nodes and file descriptors. You'll understand better what happens in your computer
– Basile Starynkevitch
Jan 7 at 6:36
maybe, the file offset is decreasing quickly as>is truncating and at the same time nginx is appending logs with a shrinking offset (i don't know how to describe a unstable offset)
– touchstone
Jan 7 at 8:22
add a comment |
On Linux, there is (almost) no mandatory file locks (more precisely, there used to be some mandatory locking feature in the kernel, but it is deprecated and you really should avoid using it). File locking happens with flock(2) or lockf(3) and is advisory and should be explicit (e.g. with flock(1) command, or some program calling flock or lockf).
So every locking related to files is practically a convention between all the software using that file (and mv(1) or the redirection by your shell don't use file locking).
Remember that a file on Linux is mostly an i-node (see inode(7)) which could have zero, one or several file paths (see path_resolution(7) and be aware of link(2), rename(2), unlink(2)) and used thru some file descriptor. Read ALP (and perhaps Operating Systems: Three Easy Pieces) for more.
No file locking happens in the scenario of your question (and the i-nodes and file descriptors involved are independent).
Consider using logrotate(8).
Some software provide a way to reload their configuration and re-open log files. You should read the documentation of your nginx.
logrotate's source code shows that it firstcpthe old one then truncate the old one,inodedoesn't change. and actually i usedlogrotateafter>causes 502 issue, but the issues still exist, then i asked this question. sorry for my english.
– touchstone
Jan 7 at 5:57
logrotateandkill -USR1 nginx.pidwhich would reload the config file, all i tried does not solve the issue.
– touchstone
Jan 7 at 6:04
fine. i learned that ">" doesn't lock the file even though 2~3s' truncating at least. thank you a lot, i would keep looking for answers for my 502.
– touchstone
Jan 7 at 6:15
Reason in terms of i-nodes and file descriptors. You'll understand better what happens in your computer
– Basile Starynkevitch
Jan 7 at 6:36
maybe, the file offset is decreasing quickly as>is truncating and at the same time nginx is appending logs with a shrinking offset (i don't know how to describe a unstable offset)
– touchstone
Jan 7 at 8:22
add a comment |
On Linux, there is (almost) no mandatory file locks (more precisely, there used to be some mandatory locking feature in the kernel, but it is deprecated and you really should avoid using it). File locking happens with flock(2) or lockf(3) and is advisory and should be explicit (e.g. with flock(1) command, or some program calling flock or lockf).
So every locking related to files is practically a convention between all the software using that file (and mv(1) or the redirection by your shell don't use file locking).
Remember that a file on Linux is mostly an i-node (see inode(7)) which could have zero, one or several file paths (see path_resolution(7) and be aware of link(2), rename(2), unlink(2)) and used thru some file descriptor. Read ALP (and perhaps Operating Systems: Three Easy Pieces) for more.
No file locking happens in the scenario of your question (and the i-nodes and file descriptors involved are independent).
Consider using logrotate(8).
Some software provide a way to reload their configuration and re-open log files. You should read the documentation of your nginx.
On Linux, there is (almost) no mandatory file locks (more precisely, there used to be some mandatory locking feature in the kernel, but it is deprecated and you really should avoid using it). File locking happens with flock(2) or lockf(3) and is advisory and should be explicit (e.g. with flock(1) command, or some program calling flock or lockf).
So every locking related to files is practically a convention between all the software using that file (and mv(1) or the redirection by your shell don't use file locking).
Remember that a file on Linux is mostly an i-node (see inode(7)) which could have zero, one or several file paths (see path_resolution(7) and be aware of link(2), rename(2), unlink(2)) and used thru some file descriptor. Read ALP (and perhaps Operating Systems: Three Easy Pieces) for more.
No file locking happens in the scenario of your question (and the i-nodes and file descriptors involved are independent).
Consider using logrotate(8).
Some software provide a way to reload their configuration and re-open log files. You should read the documentation of your nginx.
edited Jan 5 at 17:24
answered Jan 5 at 8:38
Basile StarynkevitchBasile Starynkevitch
180k13174375
180k13174375
logrotate's source code shows that it firstcpthe old one then truncate the old one,inodedoesn't change. and actually i usedlogrotateafter>causes 502 issue, but the issues still exist, then i asked this question. sorry for my english.
– touchstone
Jan 7 at 5:57
logrotateandkill -USR1 nginx.pidwhich would reload the config file, all i tried does not solve the issue.
– touchstone
Jan 7 at 6:04
fine. i learned that ">" doesn't lock the file even though 2~3s' truncating at least. thank you a lot, i would keep looking for answers for my 502.
– touchstone
Jan 7 at 6:15
Reason in terms of i-nodes and file descriptors. You'll understand better what happens in your computer
– Basile Starynkevitch
Jan 7 at 6:36
maybe, the file offset is decreasing quickly as>is truncating and at the same time nginx is appending logs with a shrinking offset (i don't know how to describe a unstable offset)
– touchstone
Jan 7 at 8:22
add a comment |
logrotate's source code shows that it firstcpthe old one then truncate the old one,inodedoesn't change. and actually i usedlogrotateafter>causes 502 issue, but the issues still exist, then i asked this question. sorry for my english.
– touchstone
Jan 7 at 5:57
logrotateandkill -USR1 nginx.pidwhich would reload the config file, all i tried does not solve the issue.
– touchstone
Jan 7 at 6:04
fine. i learned that ">" doesn't lock the file even though 2~3s' truncating at least. thank you a lot, i would keep looking for answers for my 502.
– touchstone
Jan 7 at 6:15
Reason in terms of i-nodes and file descriptors. You'll understand better what happens in your computer
– Basile Starynkevitch
Jan 7 at 6:36
maybe, the file offset is decreasing quickly as>is truncating and at the same time nginx is appending logs with a shrinking offset (i don't know how to describe a unstable offset)
– touchstone
Jan 7 at 8:22
logrotate 's source code shows that it first cp the old one then truncate the old one, inode doesn't change. and actually i used logrotate after > causes 502 issue, but the issues still exist, then i asked this question. sorry for my english.– touchstone
Jan 7 at 5:57
logrotate 's source code shows that it first cp the old one then truncate the old one, inode doesn't change. and actually i used logrotate after > causes 502 issue, but the issues still exist, then i asked this question. sorry for my english.– touchstone
Jan 7 at 5:57
logrotate and kill -USR1 nginx.pid which would reload the config file, all i tried does not solve the issue.– touchstone
Jan 7 at 6:04
logrotate and kill -USR1 nginx.pid which would reload the config file, all i tried does not solve the issue.– touchstone
Jan 7 at 6:04
fine. i learned that "
> " doesn't lock the file even though 2~3s' truncating at least. thank you a lot, i would keep looking for answers for my 502.– touchstone
Jan 7 at 6:15
fine. i learned that "
> " doesn't lock the file even though 2~3s' truncating at least. thank you a lot, i would keep looking for answers for my 502.– touchstone
Jan 7 at 6:15
Reason in terms of i-nodes and file descriptors. You'll understand better what happens in your computer
– Basile Starynkevitch
Jan 7 at 6:36
Reason in terms of i-nodes and file descriptors. You'll understand better what happens in your computer
– Basile Starynkevitch
Jan 7 at 6:36
maybe, the file offset is decreasing quickly as
> is truncating and at the same time nginx is appending logs with a shrinking offset (i don't know how to describe a unstable offset)– touchstone
Jan 7 at 8:22
maybe, the file offset is decreasing quickly as
> is truncating and at the same time nginx is appending logs with a shrinking offset (i don't know how to describe a unstable offset)– touchstone
Jan 7 at 8:22
add a comment |
It depends on application if it locks the file. Application that generates this log file must have option to clear log file. One example is in editor like vim file can be externally modified while it is still open in editor.
add a comment |
It depends on application if it locks the file. Application that generates this log file must have option to clear log file. One example is in editor like vim file can be externally modified while it is still open in editor.
add a comment |
It depends on application if it locks the file. Application that generates this log file must have option to clear log file. One example is in editor like vim file can be externally modified while it is still open in editor.
It depends on application if it locks the file. Application that generates this log file must have option to clear log file. One example is in editor like vim file can be externally modified while it is still open in editor.
answered Jan 5 at 8:36
anandanand
1337
1337
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54033940%2fwill-io-direction-operation-lock-the-file%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
mv old_log_file.log new_log_file.log && touch old_log_file.log– David C. Rankin
Jan 5 at 9:02
i don't think your comand would work, because
mvdoes not change theinodeof old log file. Andnginxwould still log into the "new_log_file" because of theinode. And you create a new log file named as olg log file, but theinodechanged, nginx wouldn't write into it untilnginx -s reloadthe config file.– touchstone
Jan 7 at 6:08
If that is how
nginxassociates the log, then you may well be correct. Then why notcp -a old_log_file.log new_log_file.log && :> old_log_file.log?– David C. Rankin
Jan 7 at 6:17
i tried 2 truncating methods:
>and 'logrotate', they all caused the nginx output 502 -- very short period, after then 200. andlogrotate's handling procedure is as you said truncate aftercp. now i have no idea what to debug this bug...– touchstone
Jan 7 at 6:22
It is not a bug, it is a feature.
– Basile Starynkevitch
Jan 7 at 8:23