Prevent Duplicate Documents in MongoDB?
I'm trying to figure out the best way to prevent duplicate documents from being saved in MongoDB.
Right now my form takes the user_url from the user. The logic is:
Check if the user_url is valid. (dns.lookup)
If user_url is new, save it to the database and return url_ID.
If user_url is old, just return the url_ID.
I think my 2 options are:
var findOneURL = function(user_url, done) {
URL.findOne({
orig_url: user_url
}, (err, data) => {
if (err) {
done(err);
}
done(null, data);
})
}
or
var findEditThenSave = function(user_url, done) {
URL.findById(user_url, (err, data) => {
if (err) {
done(err);
}
data.save((err, data) => {
if (err) {
done(err);
}
done(null, data);
});
})
};
It's not working terribly well at the moment but this is the live version:
https://kindly-fisherman.glitch.me/
EDIT2: I think I got it working properly now. Here's my logic:
Saving to database: dns.lookup -> findByURL -> (If it doesn't exist) -> countURLs -> findAndUpdateURL -> Return what was saved to database.
OR -> (If it exists) -> Return the record.
Retrieving from database: findByID
javascript node.js mongodb
asked Jan 2 at 2:38
Adam WeilerAdam Weiler
829
add a comment |
I'm trying to figure out the best way to prevent duplicate documents from being saved in MongoDB.
Right now my form takes the user_url from the user. The logic is:
Check if the user_url is valid. (dns.lookup)
If user_url is new, save it to the database and return url_ID.
If user_url is old, just return the url_ID.
I think my 2 options are:
var findOneURL = function(user_url, done) {
URL.findOne({
orig_url: user_url
}, (err, data) => {
if (err) {
done(err);
}
done(null, data);
})
}
or
var findEditThenSave = function(user_url, done) {
URL.findById(user_url, (err, data) => {
if (err) {
done(err);
}
data.save((err, data) => {
if (err) {
done(err);
}
done(null, data);
});
})
};
It's not working terribly well at the moment but this is the live version:
https://kindly-fisherman.glitch.me/
EDIT2: I think I got it working properly now. Here's my logic:
Saving to database: dns.lookup -> findByURL -> (If it doesn't exist) -> countURLs -> findAndUpdateURL -> Return what was saved to database.
OR -> (If it exists) -> Return the record.
Retrieving from database: findByID
javascript node.js mongodb
asked Jan 2 at 2:38
Adam WeilerAdam Weiler
829
Since you are usingmongooseyou can useunique: trueproperty in the model. mongoosejs.com/docs/2.7.x/docs/schematypes.html
– Shams Nahid
Jan 2 at 4:04
I would suggest to use unique index on single field. docs.mongodb.com/manual/core/index-unique. This will throw an error while inserting duplicate row. Based on the error code you can easily find out this.
– Amaranadh Meda
Jan 2 at 5:56
add a comment |
I'm trying to figure out the best way to prevent duplicate documents from being saved in MongoDB.
Right now my form takes the user_url from the user. The logic is:
Check if the user_url is valid. (dns.lookup)
If user_url is new, save it to the database and return url_ID.
If user_url is old, just return the url_ID.
I think my 2 options are:
var findOneURL = function(user_url, done) {
URL.findOne({
orig_url: user_url
}, (err, data) => {
if (err) {
done(err);
}
done(null, data);
})
}
or
var findEditThenSave = function(user_url, done) {
URL.findById(user_url, (err, data) => {
if (err) {
done(err);
}
data.save((err, data) => {
if (err) {
done(err);
}
done(null, data);
});
})
};
It's not working terribly well at the moment but this is the live version:
https://kindly-fisherman.glitch.me/
EDIT2: I think I got it working properly now. Here's my logic:
Saving to database: dns.lookup -> findByURL -> (If it doesn't exist) -> countURLs -> findAndUpdateURL -> Return what was saved to database.
OR -> (If it exists) -> Return the record.
Retrieving from database: findByID
javascript node.js mongodb
asked Jan 2 at 2:38
Adam WeilerAdam Weiler
829
I'm trying to figure out the best way to prevent duplicate documents from being saved in MongoDB.
Right now my form takes the user_url from the user. The logic is:
Check if the user_url is valid. (dns.lookup)
If user_url is new, save it to the database and return url_ID.
If user_url is old, just return the url_ID.
I think my 2 options are:
var findOneURL = function(user_url, done) {
URL.findOne({
orig_url: user_url
}, (err, data) => {
if (err) {
done(err);
}
done(null, data);
})
}
or
var findEditThenSave = function(user_url, done) {
URL.findById(user_url, (err, data) => {
if (err) {
done(err);
}
data.save((err, data) => {
if (err) {
done(err);
}
done(null, data);
});
})
};
It's not working terribly well at the moment but this is the live version:
https://kindly-fisherman.glitch.me/
EDIT2: I think I got it working properly now. Here's my logic:
Saving to database: dns.lookup -> findByURL -> (If it doesn't exist) -> countURLs -> findAndUpdateURL -> Return what was saved to database.
OR -> (If it exists) -> Return the record.
Retrieving from database: findByID
javascript node.js mongodb
javascript node.js mongodb
asked Jan 2 at 2:38
Adam WeilerAdam Weiler
829
asked Jan 2 at 2:38
Adam WeilerAdam Weiler
829
edited Jan 3 at 4:01
Adam Weiler
asked Jan 2 at 2:38
Adam WeilerAdam Weiler
829
asked Jan 2 at 2:38
Adam WeilerAdam Weiler
829
asked Jan 2 at 2:38
Adam WeilerAdam Weiler
829
829
Since you are usingmongooseyou can useunique: trueproperty in the model. mongoosejs.com/docs/2.7.x/docs/schematypes.html
– Shams Nahid
Jan 2 at 4:04
I would suggest to use unique index on single field. docs.mongodb.com/manual/core/index-unique. This will throw an error while inserting duplicate row. Based on the error code you can easily find out this.
– Amaranadh Meda
Jan 2 at 5:56
add a comment |
Since you are usingmongooseyou can useunique: trueproperty in the model. mongoosejs.com/docs/2.7.x/docs/schematypes.html
– Shams Nahid
Jan 2 at 4:04
I would suggest to use unique index on single field. docs.mongodb.com/manual/core/index-unique. This will throw an error while inserting duplicate row. Based on the error code you can easily find out this.
– Amaranadh Meda
Jan 2 at 5:56
Since you are using
mongoose you can use unique: true property in the model. mongoosejs.com/docs/2.7.x/docs/schematypes.html– Shams Nahid
Jan 2 at 4:04
Since you are using
mongoose you can use unique: true property in the model. mongoosejs.com/docs/2.7.x/docs/schematypes.html– Shams Nahid
Jan 2 at 4:04
I would suggest to use unique index on single field. docs.mongodb.com/manual/core/index-unique. This will throw an error while inserting duplicate row. Based on the error code you can easily find out this.
– Amaranadh Meda
Jan 2 at 5:56
I would suggest to use unique index on single field. docs.mongodb.com/manual/core/index-unique. This will throw an error while inserting duplicate row. Based on the error code you can easily find out this.
– Amaranadh Meda
Jan 2 at 5:56
add a comment |
1 Answer
1
active
oldest
votes
The best choice is findOneAndUpdate query with upsert and returnNewDocument options
db.collection.findOneAndUpdate({ orig_url: user_url }, { $set: { orig_url: user_url }}, { upsert: true, returnNewDocument: true })
In mongoose
URL.findOneAndUpdate({orig_url: user_url }, { $set: { orig_url: user_url }}, { upsert: true, new: true }, (err, data) => {
if (err) {
done(err);
}
// data will contain your document
done(null, data);
});
upsert option specifies whether to insert document if not found, new (returnNewDocument in mongo's console) - whether to return old or updated document - if false (default is false) you will have null for inserted documents.
answered Jan 2 at 3:11
styopdevstyopdev
1,45922342
I replaced the .save with .findOneAndUpdate. It works if the record already exists (ie: www.google.ca), but when I try to add a new record it's throwing an error. EDIT: Oh, it's updating the original record to the new count! And then when I try to add a new item it's confused because it wants to use the new count as well. ... I'll go try & sort it out.
– Adam Weiler
Jan 3 at 2:57
@AdamWeiler sorry Adam, I dont know much aboutcount's logic, if you give me more details I will be able to help you
– styopdev
Jan 3 at 6:46
1
tl;dr: It's working okay now. Basically I created a flaw in the database. I had 6 records, and while I was working I used findOneAndUpdate, and accidentally gave a record "id=7". 6 records, but "id=7". Count returns how many records exist. For adding a new record I run Count and add 1, so it'll always be a new number. But, since I'd already used "id=7", the new record was being rejected since it also wanted "id=7". As soon as I fixed the database, everything was golden.
– Adam Weiler
Jan 3 at 17:43
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54000626%2fprevent-duplicate-documents-in-mongodb%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
The best choice is findOneAndUpdate query with upsert and returnNewDocument options
db.collection.findOneAndUpdate({ orig_url: user_url }, { $set: { orig_url: user_url }}, { upsert: true, returnNewDocument: true })
In mongoose
URL.findOneAndUpdate({orig_url: user_url }, { $set: { orig_url: user_url }}, { upsert: true, new: true }, (err, data) => {
if (err) {
done(err);
}
// data will contain your document
done(null, data);
});
upsert option specifies whether to insert document if not found, new (returnNewDocument in mongo's console) - whether to return old or updated document - if false (default is false) you will have null for inserted documents.
answered Jan 2 at 3:11
styopdevstyopdev
1,45922342
I replaced the .save with .findOneAndUpdate. It works if the record already exists (ie: www.google.ca), but when I try to add a new record it's throwing an error. EDIT: Oh, it's updating the original record to the new count! And then when I try to add a new item it's confused because it wants to use the new count as well. ... I'll go try & sort it out.
– Adam Weiler
Jan 3 at 2:57
@AdamWeiler sorry Adam, I dont know much aboutcount's logic, if you give me more details I will be able to help you
– styopdev
Jan 3 at 6:46
1
tl;dr: It's working okay now. Basically I created a flaw in the database. I had 6 records, and while I was working I used findOneAndUpdate, and accidentally gave a record "id=7". 6 records, but "id=7". Count returns how many records exist. For adding a new record I run Count and add 1, so it'll always be a new number. But, since I'd already used "id=7", the new record was being rejected since it also wanted "id=7". As soon as I fixed the database, everything was golden.
– Adam Weiler
Jan 3 at 17:43
add a comment |
The best choice is findOneAndUpdate query with upsert and returnNewDocument options
db.collection.findOneAndUpdate({ orig_url: user_url }, { $set: { orig_url: user_url }}, { upsert: true, returnNewDocument: true })
In mongoose
URL.findOneAndUpdate({orig_url: user_url }, { $set: { orig_url: user_url }}, { upsert: true, new: true }, (err, data) => {
if (err) {
done(err);
}
// data will contain your document
done(null, data);
});
upsert option specifies whether to insert document if not found, new (returnNewDocument in mongo's console) - whether to return old or updated document - if false (default is false) you will have null for inserted documents.
answered Jan 2 at 3:11
styopdevstyopdev
1,45922342
I replaced the .save with .findOneAndUpdate. It works if the record already exists (ie: www.google.ca), but when I try to add a new record it's throwing an error. EDIT: Oh, it's updating the original record to the new count! And then when I try to add a new item it's confused because it wants to use the new count as well. ... I'll go try & sort it out.
– Adam Weiler
Jan 3 at 2:57
@AdamWeiler sorry Adam, I dont know much aboutcount's logic, if you give me more details I will be able to help you
– styopdev
Jan 3 at 6:46
1
tl;dr: It's working okay now. Basically I created a flaw in the database. I had 6 records, and while I was working I used findOneAndUpdate, and accidentally gave a record "id=7". 6 records, but "id=7". Count returns how many records exist. For adding a new record I run Count and add 1, so it'll always be a new number. But, since I'd already used "id=7", the new record was being rejected since it also wanted "id=7". As soon as I fixed the database, everything was golden.
– Adam Weiler
Jan 3 at 17:43
add a comment |
The best choice is findOneAndUpdate query with upsert and returnNewDocument options
db.collection.findOneAndUpdate({ orig_url: user_url }, { $set: { orig_url: user_url }}, { upsert: true, returnNewDocument: true })
In mongoose
URL.findOneAndUpdate({orig_url: user_url }, { $set: { orig_url: user_url }}, { upsert: true, new: true }, (err, data) => {
if (err) {
done(err);
}
// data will contain your document
done(null, data);
});
upsert option specifies whether to insert document if not found, new (returnNewDocument in mongo's console) - whether to return old or updated document - if false (default is false) you will have null for inserted documents.
answered Jan 2 at 3:11
styopdevstyopdev
1,45922342
The best choice is findOneAndUpdate query with upsert and returnNewDocument options
db.collection.findOneAndUpdate({ orig_url: user_url }, { $set: { orig_url: user_url }}, { upsert: true, returnNewDocument: true })
In mongoose
URL.findOneAndUpdate({orig_url: user_url }, { $set: { orig_url: user_url }}, { upsert: true, new: true }, (err, data) => {
if (err) {
done(err);
}
// data will contain your document
done(null, data);
});
upsert option specifies whether to insert document if not found, new (returnNewDocument in mongo's console) - whether to return old or updated document - if false (default is false) you will have null for inserted documents.
answered Jan 2 at 3:11
styopdevstyopdev
1,45922342
edited Jan 2 at 3:16
answered Jan 2 at 3:11
styopdevstyopdev
1,45922342
answered Jan 2 at 3:11
styopdevstyopdev
1,45922342
answered Jan 2 at 3:11
styopdevstyopdev
1,45922342
1,45922342
I replaced the .save with .findOneAndUpdate. It works if the record already exists (ie: www.google.ca), but when I try to add a new record it's throwing an error. EDIT: Oh, it's updating the original record to the new count! And then when I try to add a new item it's confused because it wants to use the new count as well. ... I'll go try & sort it out.
– Adam Weiler
Jan 3 at 2:57
@AdamWeiler sorry Adam, I dont know much aboutcount's logic, if you give me more details I will be able to help you
– styopdev
Jan 3 at 6:46
1
tl;dr: It's working okay now. Basically I created a flaw in the database. I had 6 records, and while I was working I used findOneAndUpdate, and accidentally gave a record "id=7". 6 records, but "id=7". Count returns how many records exist. For adding a new record I run Count and add 1, so it'll always be a new number. But, since I'd already used "id=7", the new record was being rejected since it also wanted "id=7". As soon as I fixed the database, everything was golden.
– Adam Weiler
Jan 3 at 17:43
add a comment |
I replaced the .save with .findOneAndUpdate. It works if the record already exists (ie: www.google.ca), but when I try to add a new record it's throwing an error. EDIT: Oh, it's updating the original record to the new count! And then when I try to add a new item it's confused because it wants to use the new count as well. ... I'll go try & sort it out.
– Adam Weiler
Jan 3 at 2:57
@AdamWeiler sorry Adam, I dont know much aboutcount's logic, if you give me more details I will be able to help you
– styopdev
Jan 3 at 6:46
1
tl;dr: It's working okay now. Basically I created a flaw in the database. I had 6 records, and while I was working I used findOneAndUpdate, and accidentally gave a record "id=7". 6 records, but "id=7". Count returns how many records exist. For adding a new record I run Count and add 1, so it'll always be a new number. But, since I'd already used "id=7", the new record was being rejected since it also wanted "id=7". As soon as I fixed the database, everything was golden.
– Adam Weiler
Jan 3 at 17:43
I replaced the .save with .findOneAndUpdate. It works if the record already exists (ie: www.google.ca), but when I try to add a new record it's throwing an error. EDIT: Oh, it's updating the original record to the new count! And then when I try to add a new item it's confused because it wants to use the new count as well. ... I'll go try & sort it out.
– Adam Weiler
Jan 3 at 2:57
I replaced the .save with .findOneAndUpdate. It works if the record already exists (ie: www.google.ca), but when I try to add a new record it's throwing an error. EDIT: Oh, it's updating the original record to the new count! And then when I try to add a new item it's confused because it wants to use the new count as well. ... I'll go try & sort it out.
– Adam Weiler
Jan 3 at 2:57
@AdamWeiler sorry Adam, I dont know much about
count's logic, if you give me more details I will be able to help you– styopdev
Jan 3 at 6:46
@AdamWeiler sorry Adam, I dont know much about
count's logic, if you give me more details I will be able to help you– styopdev
Jan 3 at 6:46
1
1
tl;dr: It's working okay now. Basically I created a flaw in the database. I had 6 records, and while I was working I used findOneAndUpdate, and accidentally gave a record "id=7". 6 records, but "id=7". Count returns how many records exist. For adding a new record I run Count and add 1, so it'll always be a new number. But, since I'd already used "id=7", the new record was being rejected since it also wanted "id=7". As soon as I fixed the database, everything was golden.
– Adam Weiler
Jan 3 at 17:43
tl;dr: It's working okay now. Basically I created a flaw in the database. I had 6 records, and while I was working I used findOneAndUpdate, and accidentally gave a record "id=7". 6 records, but "id=7". Count returns how many records exist. For adding a new record I run Count and add 1, so it'll always be a new number. But, since I'd already used "id=7", the new record was being rejected since it also wanted "id=7". As soon as I fixed the database, everything was golden.
– Adam Weiler
Jan 3 at 17:43
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54000626%2fprevent-duplicate-documents-in-mongodb%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Since you are using
mongooseyou can useunique: trueproperty in the model. mongoosejs.com/docs/2.7.x/docs/schematypes.html– Shams Nahid
Jan 2 at 4:04
I would suggest to use unique index on single field. docs.mongodb.com/manual/core/index-unique. This will throw an error while inserting duplicate row. Based on the error code you can easily find out this.
– Amaranadh Meda
Jan 2 at 5:56