Is there a better way to replace rows from one dataframe to another based on a single columns' data?
I'm attempting to replace all matching rows in one dataframe with rows from another dataframe when a specific condition is satisfied. The number of rows in both dataframes will differ, but the number of columns are the same.
Do to the sensitive nature of the data that im dealing with i can give an example but not the real data itself.
An example of the dataframe could be something like this:
Location Tickets Destination Product Tare Net Scale Value1 Value2 ...
500 012 ID 01A 20 40 60 0.01 0.00
500 013 PA 02L 10 300 310 12.01 5
The other dataframe would have different values in each respective column, except for the 'Tickets' column. I would like to replace the data from all rows in df1 with any data from df2 that has a matching ticket number.
df1.loc[df1.Tickets.isin(df2.Tickets), :] =
df2.loc[df2.Tickets.isin(df1.Tickets), :].values
When i had run the code i get this error off and on:
ValueError: Must have equal len keys and value when setting with an ndarray
Not sure why or what is causing it because the code works on certain dataframes and not on others. The dataframes it is working on have different numbers of rows but the same columns, which is exactly the same for the dataframes it's not working on. I know their is something to this but i can't seem to find it.
Any help is greatly appreciated!
Update:
interestingly enough when i isolate the tickets to be replaced from df1 in a separate dataframe, i get no errors when i replace them with df2. This leads me to believe i have duplicate tickets in df1 that are causing my error but on further inspection i can find no duplicate tickets in df1.
python-3.x pandas
edited Jan 3 at 20:09
Rinzler786
71111
asked Jan 3 at 17:05
J_MillarJ_Millar
133
|
show 4 more comments
I'm attempting to replace all matching rows in one dataframe with rows from another dataframe when a specific condition is satisfied. The number of rows in both dataframes will differ, but the number of columns are the same.
Do to the sensitive nature of the data that im dealing with i can give an example but not the real data itself.
An example of the dataframe could be something like this:
Location Tickets Destination Product Tare Net Scale Value1 Value2 ...
500 012 ID 01A 20 40 60 0.01 0.00
500 013 PA 02L 10 300 310 12.01 5
The other dataframe would have different values in each respective column, except for the 'Tickets' column. I would like to replace the data from all rows in df1 with any data from df2 that has a matching ticket number.
df1.loc[df1.Tickets.isin(df2.Tickets), :] =
df2.loc[df2.Tickets.isin(df1.Tickets), :].values
When i had run the code i get this error off and on:
ValueError: Must have equal len keys and value when setting with an ndarray
Not sure why or what is causing it because the code works on certain dataframes and not on others. The dataframes it is working on have different numbers of rows but the same columns, which is exactly the same for the dataframes it's not working on. I know their is something to this but i can't seem to find it.
Any help is greatly appreciated!
Update:
interestingly enough when i isolate the tickets to be replaced from df1 in a separate dataframe, i get no errors when i replace them with df2. This leads me to believe i have duplicate tickets in df1 that are causing my error but on further inspection i can find no duplicate tickets in df1.
python-3.x pandas
edited Jan 3 at 20:09
Rinzler786
71111
asked Jan 3 at 17:05
J_MillarJ_Millar
133
Are all the values for'Ticket'unique within a dataframe? Also, I think this will only work if the order is the same in both dataframes, from a quick glance.
– busybear
Jan 3 at 17:09
Yes all ticket values are unique in say the df1 dataframe, and all ticket numbers in df2 are unique from one another. They will however be the same from df1 to df2 in select rows
– J_Millar
Jan 3 at 17:16
I will sort_values in order to get the second dataframe in order as it is not in ascending or descending order and get back to you shortly
– J_Millar
Jan 3 at 17:21
Your error might occur when there is no match between datasets.
– busybear
Jan 3 at 17:27
when i ordered the second dataset i have a new error that is: ValueError: shape mismatch: value array of shape (111,62) could not be broadcast to indexing result of shape (115,62)
– J_Millar
Jan 3 at 17:28
|
show 4 more comments
I'm attempting to replace all matching rows in one dataframe with rows from another dataframe when a specific condition is satisfied. The number of rows in both dataframes will differ, but the number of columns are the same.
Do to the sensitive nature of the data that im dealing with i can give an example but not the real data itself.
An example of the dataframe could be something like this:
Location Tickets Destination Product Tare Net Scale Value1 Value2 ...
500 012 ID 01A 20 40 60 0.01 0.00
500 013 PA 02L 10 300 310 12.01 5
The other dataframe would have different values in each respective column, except for the 'Tickets' column. I would like to replace the data from all rows in df1 with any data from df2 that has a matching ticket number.
df1.loc[df1.Tickets.isin(df2.Tickets), :] =
df2.loc[df2.Tickets.isin(df1.Tickets), :].values
When i had run the code i get this error off and on:
ValueError: Must have equal len keys and value when setting with an ndarray
Not sure why or what is causing it because the code works on certain dataframes and not on others. The dataframes it is working on have different numbers of rows but the same columns, which is exactly the same for the dataframes it's not working on. I know their is something to this but i can't seem to find it.
Any help is greatly appreciated!
Update:
interestingly enough when i isolate the tickets to be replaced from df1 in a separate dataframe, i get no errors when i replace them with df2. This leads me to believe i have duplicate tickets in df1 that are causing my error but on further inspection i can find no duplicate tickets in df1.
python-3.x pandas
edited Jan 3 at 20:09
Rinzler786
71111
asked Jan 3 at 17:05
J_MillarJ_Millar
133
I'm attempting to replace all matching rows in one dataframe with rows from another dataframe when a specific condition is satisfied. The number of rows in both dataframes will differ, but the number of columns are the same.
Do to the sensitive nature of the data that im dealing with i can give an example but not the real data itself.
An example of the dataframe could be something like this:
Location Tickets Destination Product Tare Net Scale Value1 Value2 ...
500 012 ID 01A 20 40 60 0.01 0.00
500 013 PA 02L 10 300 310 12.01 5
The other dataframe would have different values in each respective column, except for the 'Tickets' column. I would like to replace the data from all rows in df1 with any data from df2 that has a matching ticket number.
df1.loc[df1.Tickets.isin(df2.Tickets), :] =
df2.loc[df2.Tickets.isin(df1.Tickets), :].values
When i had run the code i get this error off and on:
ValueError: Must have equal len keys and value when setting with an ndarray
Not sure why or what is causing it because the code works on certain dataframes and not on others. The dataframes it is working on have different numbers of rows but the same columns, which is exactly the same for the dataframes it's not working on. I know their is something to this but i can't seem to find it.
Any help is greatly appreciated!
Update:
interestingly enough when i isolate the tickets to be replaced from df1 in a separate dataframe, i get no errors when i replace them with df2. This leads me to believe i have duplicate tickets in df1 that are causing my error but on further inspection i can find no duplicate tickets in df1.
python-3.x pandas
python-3.x pandas
edited Jan 3 at 20:09
Rinzler786
71111
asked Jan 3 at 17:05
J_MillarJ_Millar
133
edited Jan 3 at 20:09
Rinzler786
71111
asked Jan 3 at 17:05
J_MillarJ_Millar
133
edited Jan 3 at 20:09
Rinzler786
71111
edited Jan 3 at 20:09
Rinzler786
71111
edited Jan 3 at 20:09
Rinzler786
71111
71111
asked Jan 3 at 17:05
J_MillarJ_Millar
133
asked Jan 3 at 17:05
J_MillarJ_Millar
133
asked Jan 3 at 17:05
J_MillarJ_Millar
133
133
Are all the values for'Ticket'unique within a dataframe? Also, I think this will only work if the order is the same in both dataframes, from a quick glance.
– busybear
Jan 3 at 17:09
Yes all ticket values are unique in say the df1 dataframe, and all ticket numbers in df2 are unique from one another. They will however be the same from df1 to df2 in select rows
– J_Millar
Jan 3 at 17:16
I will sort_values in order to get the second dataframe in order as it is not in ascending or descending order and get back to you shortly
– J_Millar
Jan 3 at 17:21
Your error might occur when there is no match between datasets.
– busybear
Jan 3 at 17:27
when i ordered the second dataset i have a new error that is: ValueError: shape mismatch: value array of shape (111,62) could not be broadcast to indexing result of shape (115,62)
– J_Millar
Jan 3 at 17:28
|
show 4 more comments
Are all the values for'Ticket'unique within a dataframe? Also, I think this will only work if the order is the same in both dataframes, from a quick glance.
– busybear
Jan 3 at 17:09
Yes all ticket values are unique in say the df1 dataframe, and all ticket numbers in df2 are unique from one another. They will however be the same from df1 to df2 in select rows
– J_Millar
Jan 3 at 17:16
I will sort_values in order to get the second dataframe in order as it is not in ascending or descending order and get back to you shortly
– J_Millar
Jan 3 at 17:21
Your error might occur when there is no match between datasets.
– busybear
Jan 3 at 17:27
when i ordered the second dataset i have a new error that is: ValueError: shape mismatch: value array of shape (111,62) could not be broadcast to indexing result of shape (115,62)
– J_Millar
Jan 3 at 17:28
Are all the values for
'Ticket' unique within a dataframe? Also, I think this will only work if the order is the same in both dataframes, from a quick glance.– busybear
Jan 3 at 17:09
Are all the values for
'Ticket' unique within a dataframe? Also, I think this will only work if the order is the same in both dataframes, from a quick glance.– busybear
Jan 3 at 17:09
Yes all ticket values are unique in say the df1 dataframe, and all ticket numbers in df2 are unique from one another. They will however be the same from df1 to df2 in select rows
– J_Millar
Jan 3 at 17:16
Yes all ticket values are unique in say the df1 dataframe, and all ticket numbers in df2 are unique from one another. They will however be the same from df1 to df2 in select rows
– J_Millar
Jan 3 at 17:16
I will sort_values in order to get the second dataframe in order as it is not in ascending or descending order and get back to you shortly
– J_Millar
Jan 3 at 17:21
I will sort_values in order to get the second dataframe in order as it is not in ascending or descending order and get back to you shortly
– J_Millar
Jan 3 at 17:21
Your error might occur when there is no match between datasets.
– busybear
Jan 3 at 17:27
Your error might occur when there is no match between datasets.
– busybear
Jan 3 at 17:27
when i ordered the second dataset i have a new error that is: ValueError: shape mismatch: value array of shape (111,62) could not be broadcast to indexing result of shape (115,62)
– J_Millar
Jan 3 at 17:28
when i ordered the second dataset i have a new error that is: ValueError: shape mismatch: value array of shape (111,62) could not be broadcast to indexing result of shape (115,62)
– J_Millar
Jan 3 at 17:28
|
show 4 more comments
0
active
oldest
votes
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54026725%2fis-there-a-better-way-to-replace-rows-from-one-dataframe-to-another-based-on-a-s%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54026725%2fis-there-a-better-way-to-replace-rows-from-one-dataframe-to-another-based-on-a-s%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Are all the values for
'Ticket'unique within a dataframe? Also, I think this will only work if the order is the same in both dataframes, from a quick glance.– busybear
Jan 3 at 17:09
Yes all ticket values are unique in say the df1 dataframe, and all ticket numbers in df2 are unique from one another. They will however be the same from df1 to df2 in select rows
– J_Millar
Jan 3 at 17:16
I will sort_values in order to get the second dataframe in order as it is not in ascending or descending order and get back to you shortly
– J_Millar
Jan 3 at 17:21
Your error might occur when there is no match between datasets.
– busybear
Jan 3 at 17:27
when i ordered the second dataset i have a new error that is: ValueError: shape mismatch: value array of shape (111,62) could not be broadcast to indexing result of shape (115,62)
– J_Millar
Jan 3 at 17:28