Right way to manage application generated files
tl;dr
In my node.js application I create pdf documents. What is the best/right way to save them? Right now I use node.js fileserver and shell.js to do it.
I am working on a node.js web application to manage apartments and tenants for learning purpose and on some point I create PDF Documents that I want to save under a path
/documents/building_name/apartment_name/tenant_name/year/example.pfd
Now if the user wants to change the building, apartment or tenant name via an http PUT request I change the database but also the want to change the path.
Well both works but I can't write good tests for these functions.
Now a friend told me that it's a bad practice to save documents on a file server and I better should use BLOB.
On the other side google doesn't really agree on using blobs
So what is the right way to save documents?
Thanks
Amit
mysql node.js blob
asked Jan 2 at 12:21
AmitAmit
63
add a comment |
tl;dr
In my node.js application I create pdf documents. What is the best/right way to save them? Right now I use node.js fileserver and shell.js to do it.
I am working on a node.js web application to manage apartments and tenants for learning purpose and on some point I create PDF Documents that I want to save under a path
/documents/building_name/apartment_name/tenant_name/year/example.pfd
Now if the user wants to change the building, apartment or tenant name via an http PUT request I change the database but also the want to change the path.
Well both works but I can't write good tests for these functions.
Now a friend told me that it's a bad practice to save documents on a file server and I better should use BLOB.
On the other side google doesn't really agree on using blobs
So what is the right way to save documents?
Thanks
Amit
mysql node.js blob
asked Jan 2 at 12:21
AmitAmit
63
add a comment |
tl;dr
In my node.js application I create pdf documents. What is the best/right way to save them? Right now I use node.js fileserver and shell.js to do it.
I am working on a node.js web application to manage apartments and tenants for learning purpose and on some point I create PDF Documents that I want to save under a path
/documents/building_name/apartment_name/tenant_name/year/example.pfd
Now if the user wants to change the building, apartment or tenant name via an http PUT request I change the database but also the want to change the path.
Well both works but I can't write good tests for these functions.
Now a friend told me that it's a bad practice to save documents on a file server and I better should use BLOB.
On the other side google doesn't really agree on using blobs
So what is the right way to save documents?
Thanks
Amit
mysql node.js blob
asked Jan 2 at 12:21
AmitAmit
63
tl;dr
In my node.js application I create pdf documents. What is the best/right way to save them? Right now I use node.js fileserver and shell.js to do it.
I am working on a node.js web application to manage apartments and tenants for learning purpose and on some point I create PDF Documents that I want to save under a path
/documents/building_name/apartment_name/tenant_name/year/example.pfd
Now if the user wants to change the building, apartment or tenant name via an http PUT request I change the database but also the want to change the path.
Well both works but I can't write good tests for these functions.
Now a friend told me that it's a bad practice to save documents on a file server and I better should use BLOB.
On the other side google doesn't really agree on using blobs
So what is the right way to save documents?
Thanks
Amit
mysql node.js blob
mysql node.js blob
asked Jan 2 at 12:21
AmitAmit
63
asked Jan 2 at 12:21
AmitAmit
63
asked Jan 2 at 12:21
AmitAmit
63
asked Jan 2 at 12:21
AmitAmit
63
asked Jan 2 at 12:21
AmitAmit
63
63
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
You should first define a source of truth. Unless you're legally obliged to keep copies of those files and they are not being accessed very often, I wouldn't even bother storing those at all and just generate them upon request.
If not, keep the DB clean, blobs will make it huge. Put them into cold storage (again assuming they are not being accessed too frequently) without those paths. If the paths are reliant on often changing information, that can't be performant for neither the file server nor your system.
Instead store a revision number in your DB that the file can be found under and limit the path structure to information that rarely change.
Like {building}/{apartment}/{tenant}_{revision}.pfd
That - depending on your backup structure - will allow you to time-travel if necessary and doesn't force a re-index all the time.
Note: I don't know too much about your use case.
answered Jan 2 at 16:46
FredFred
3,07411425
Well, before I started studying computer science I worked in the real estate sector. I just needed an app idea and I can better estimate what a user would need if he would do my last work, but the project ist for purely educational purposes. So there are no direct legal obligations or prohibitions. The database is normalized. If I anyway have to save data like revision in the database would It be a smart idea to save all documents in the same folder and then having a table documents (id PK, tenant_id, year, revision) and when searching for a document just look for /documents/{id}.pdf?
– Amit
Jan 3 at 15:11
Sure, I was under the assumption you wanted to have at least some path structure.
– Fred
Jan 3 at 15:47
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54006311%2fright-way-to-manage-application-generated-files%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
You should first define a source of truth. Unless you're legally obliged to keep copies of those files and they are not being accessed very often, I wouldn't even bother storing those at all and just generate them upon request.
If not, keep the DB clean, blobs will make it huge. Put them into cold storage (again assuming they are not being accessed too frequently) without those paths. If the paths are reliant on often changing information, that can't be performant for neither the file server nor your system.
Instead store a revision number in your DB that the file can be found under and limit the path structure to information that rarely change.
Like {building}/{apartment}/{tenant}_{revision}.pfd
That - depending on your backup structure - will allow you to time-travel if necessary and doesn't force a re-index all the time.
Note: I don't know too much about your use case.
answered Jan 2 at 16:46
FredFred
3,07411425
Well, before I started studying computer science I worked in the real estate sector. I just needed an app idea and I can better estimate what a user would need if he would do my last work, but the project ist for purely educational purposes. So there are no direct legal obligations or prohibitions. The database is normalized. If I anyway have to save data like revision in the database would It be a smart idea to save all documents in the same folder and then having a table documents (id PK, tenant_id, year, revision) and when searching for a document just look for /documents/{id}.pdf?
– Amit
Jan 3 at 15:11
Sure, I was under the assumption you wanted to have at least some path structure.
– Fred
Jan 3 at 15:47
add a comment |
You should first define a source of truth. Unless you're legally obliged to keep copies of those files and they are not being accessed very often, I wouldn't even bother storing those at all and just generate them upon request.
If not, keep the DB clean, blobs will make it huge. Put them into cold storage (again assuming they are not being accessed too frequently) without those paths. If the paths are reliant on often changing information, that can't be performant for neither the file server nor your system.
Instead store a revision number in your DB that the file can be found under and limit the path structure to information that rarely change.
Like {building}/{apartment}/{tenant}_{revision}.pfd
That - depending on your backup structure - will allow you to time-travel if necessary and doesn't force a re-index all the time.
Note: I don't know too much about your use case.
answered Jan 2 at 16:46
FredFred
3,07411425
Well, before I started studying computer science I worked in the real estate sector. I just needed an app idea and I can better estimate what a user would need if he would do my last work, but the project ist for purely educational purposes. So there are no direct legal obligations or prohibitions. The database is normalized. If I anyway have to save data like revision in the database would It be a smart idea to save all documents in the same folder and then having a table documents (id PK, tenant_id, year, revision) and when searching for a document just look for /documents/{id}.pdf?
– Amit
Jan 3 at 15:11
Sure, I was under the assumption you wanted to have at least some path structure.
– Fred
Jan 3 at 15:47
add a comment |
You should first define a source of truth. Unless you're legally obliged to keep copies of those files and they are not being accessed very often, I wouldn't even bother storing those at all and just generate them upon request.
If not, keep the DB clean, blobs will make it huge. Put them into cold storage (again assuming they are not being accessed too frequently) without those paths. If the paths are reliant on often changing information, that can't be performant for neither the file server nor your system.
Instead store a revision number in your DB that the file can be found under and limit the path structure to information that rarely change.
Like {building}/{apartment}/{tenant}_{revision}.pfd
That - depending on your backup structure - will allow you to time-travel if necessary and doesn't force a re-index all the time.
Note: I don't know too much about your use case.
answered Jan 2 at 16:46
FredFred
3,07411425
You should first define a source of truth. Unless you're legally obliged to keep copies of those files and they are not being accessed very often, I wouldn't even bother storing those at all and just generate them upon request.
If not, keep the DB clean, blobs will make it huge. Put them into cold storage (again assuming they are not being accessed too frequently) without those paths. If the paths are reliant on often changing information, that can't be performant for neither the file server nor your system.
Instead store a revision number in your DB that the file can be found under and limit the path structure to information that rarely change.
Like {building}/{apartment}/{tenant}_{revision}.pfd
That - depending on your backup structure - will allow you to time-travel if necessary and doesn't force a re-index all the time.
Note: I don't know too much about your use case.
answered Jan 2 at 16:46
FredFred
3,07411425
answered Jan 2 at 16:46
FredFred
3,07411425
answered Jan 2 at 16:46
FredFred
3,07411425
answered Jan 2 at 16:46
FredFred
3,07411425
3,07411425
Well, before I started studying computer science I worked in the real estate sector. I just needed an app idea and I can better estimate what a user would need if he would do my last work, but the project ist for purely educational purposes. So there are no direct legal obligations or prohibitions. The database is normalized. If I anyway have to save data like revision in the database would It be a smart idea to save all documents in the same folder and then having a table documents (id PK, tenant_id, year, revision) and when searching for a document just look for /documents/{id}.pdf?
– Amit
Jan 3 at 15:11
Sure, I was under the assumption you wanted to have at least some path structure.
– Fred
Jan 3 at 15:47
add a comment |
Well, before I started studying computer science I worked in the real estate sector. I just needed an app idea and I can better estimate what a user would need if he would do my last work, but the project ist for purely educational purposes. So there are no direct legal obligations or prohibitions. The database is normalized. If I anyway have to save data like revision in the database would It be a smart idea to save all documents in the same folder and then having a table documents (id PK, tenant_id, year, revision) and when searching for a document just look for /documents/{id}.pdf?
– Amit
Jan 3 at 15:11
Sure, I was under the assumption you wanted to have at least some path structure.
– Fred
Jan 3 at 15:47
Well, before I started studying computer science I worked in the real estate sector. I just needed an app idea and I can better estimate what a user would need if he would do my last work, but the project ist for purely educational purposes. So there are no direct legal obligations or prohibitions. The database is normalized. If I anyway have to save data like revision in the database would It be a smart idea to save all documents in the same folder and then having a table documents (id PK, tenant_id, year, revision) and when searching for a document just look for /documents/{id}.pdf?
– Amit
Jan 3 at 15:11
Well, before I started studying computer science I worked in the real estate sector. I just needed an app idea and I can better estimate what a user would need if he would do my last work, but the project ist for purely educational purposes. So there are no direct legal obligations or prohibitions. The database is normalized. If I anyway have to save data like revision in the database would It be a smart idea to save all documents in the same folder and then having a table documents (id PK, tenant_id, year, revision) and when searching for a document just look for /documents/{id}.pdf?
– Amit
Jan 3 at 15:11
Sure, I was under the assumption you wanted to have at least some path structure.
– Fred
Jan 3 at 15:47
Sure, I was under the assumption you wanted to have at least some path structure.
– Fred
Jan 3 at 15:47
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54006311%2fright-way-to-manage-application-generated-files%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
