Pandas: iterate over a row and adding the value to an empty column
Hello there I would like to iterate over the row CPB% and add the computations to a related column called 'Proba'. My dataframe looks like this: 
What I tried so far looks like this:
bins = np.linspace(0, 1, num=100)
dCPB = df['CPB%']
df['binnedB'] = pd.cut(dCPB, bins)
dfnew = pd.DataFrame(pd.cut(df['CPB%'], bins=bins).value_counts()).sort_index(ascending = True)
dfnew['binned'] = dfnew.index
total = dfnew['CPB%'].sum()
idx = total
for index,row in dfnew.iterrows():
idx = idx - row['CPB%']
row['Proba'] = float(idx) / float(total)
But my iteration does not update my empty column Proba, any idea why? Thanks!
python pandas dataframe series
edited Dec 31 '18 at 11:46
jpp
100k2161111
asked Dec 31 '18 at 11:24
Viktor.wViktor.w
1037
add a comment |
Hello there I would like to iterate over the row CPB% and add the computations to a related column called 'Proba'. My dataframe looks like this:
What I tried so far looks like this:
bins = np.linspace(0, 1, num=100)
dCPB = df['CPB%']
df['binnedB'] = pd.cut(dCPB, bins)
dfnew = pd.DataFrame(pd.cut(df['CPB%'], bins=bins).value_counts()).sort_index(ascending = True)
dfnew['binned'] = dfnew.index
total = dfnew['CPB%'].sum()
idx = total
for index,row in dfnew.iterrows():
idx = idx - row['CPB%']
row['Proba'] = float(idx) / float(total)
But my iteration does not update my empty column Proba, any idea why? Thanks!
python pandas dataframe series
edited Dec 31 '18 at 11:46
jpp
100k2161111
asked Dec 31 '18 at 11:24
Viktor.wViktor.w
1037
2
Please paste your dataframe as text and not an image
– Dan
Dec 31 '18 at 11:40
add a comment |
Hello there I would like to iterate over the row CPB% and add the computations to a related column called 'Proba'. My dataframe looks like this:
What I tried so far looks like this:
bins = np.linspace(0, 1, num=100)
dCPB = df['CPB%']
df['binnedB'] = pd.cut(dCPB, bins)
dfnew = pd.DataFrame(pd.cut(df['CPB%'], bins=bins).value_counts()).sort_index(ascending = True)
dfnew['binned'] = dfnew.index
total = dfnew['CPB%'].sum()
idx = total
for index,row in dfnew.iterrows():
idx = idx - row['CPB%']
row['Proba'] = float(idx) / float(total)
But my iteration does not update my empty column Proba, any idea why? Thanks!
python pandas dataframe series
edited Dec 31 '18 at 11:46
jpp
100k2161111
asked Dec 31 '18 at 11:24
Viktor.wViktor.w
1037
Hello there I would like to iterate over the row CPB% and add the computations to a related column called 'Proba'. My dataframe looks like this:
What I tried so far looks like this:
bins = np.linspace(0, 1, num=100)
dCPB = df['CPB%']
df['binnedB'] = pd.cut(dCPB, bins)
dfnew = pd.DataFrame(pd.cut(df['CPB%'], bins=bins).value_counts()).sort_index(ascending = True)
dfnew['binned'] = dfnew.index
total = dfnew['CPB%'].sum()
idx = total
for index,row in dfnew.iterrows():
idx = idx - row['CPB%']
row['Proba'] = float(idx) / float(total)
But my iteration does not update my empty column Proba, any idea why? Thanks!
python pandas dataframe series
python pandas dataframe series
edited Dec 31 '18 at 11:46
jpp
100k2161111
asked Dec 31 '18 at 11:24
Viktor.wViktor.w
1037
edited Dec 31 '18 at 11:46
jpp
100k2161111
asked Dec 31 '18 at 11:24
Viktor.wViktor.w
1037
edited Dec 31 '18 at 11:46
jpp
100k2161111
edited Dec 31 '18 at 11:46
jpp
100k2161111
edited Dec 31 '18 at 11:46
jpp
100k2161111
100k2161111
asked Dec 31 '18 at 11:24
Viktor.wViktor.w
1037
asked Dec 31 '18 at 11:24
Viktor.wViktor.w
1037
asked Dec 31 '18 at 11:24
Viktor.wViktor.w
1037
1037
2
Please paste your dataframe as text and not an image
– Dan
Dec 31 '18 at 11:40
add a comment |
2
Please paste your dataframe as text and not an image
– Dan
Dec 31 '18 at 11:40
2
2
Please paste your dataframe as text and not an image
– Dan
Dec 31 '18 at 11:40
Please paste your dataframe as text and not an image
– Dan
Dec 31 '18 at 11:40
add a comment |
2 Answers
2
active
oldest
votes
I think the problem is, you are assigning the result back to the row, which doesn't get stored anywhere. instead you can do:
proba =
for index, row in dfnew.iterrows():
idx = idx - row['CPB%']
proba.append(float(idx) / float(total))
dfnew['Proba'] = proba
However, this is not the best way, you can use .apply with axis=1 to do row-wise calculations on a data frame.
answered Dec 31 '18 at 11:32
YOLOYOLO
5,4721424
Yes thanks but I have another issue error: 'Cannot concatenate object of type "<type 'float'>"; only pd.Series
– Viktor.w
Dec 31 '18 at 11:40
1
You would need to useproba =instead ofproba = pd.Series()here. But not really necessary when you can usecumsum.
– jpp
Dec 31 '18 at 11:46
add a comment |
You can use pd.Series.cumsum to perform your iterative deductions:
total = dfnew['CPB%'].sum()
dfnew['Proba'] = 1 - df['CPB%'].cumsum() / total
With Pandas you should look to vectorise algorithms, which usually involves column-wise operations as opposed to a row-wise for loop. Here's a complete demonstration:
df = pd.DataFrame({'A': list(range(1, 7))})
def jpp(df):
total = df['A'].sum()
df['Proba'] = 1 - df['A'].cumsum() / total
return df
def yolo(df):
total = df['A'].sum()
idx = total
proba =
for index, row in df.iterrows():
idx = idx - row['A']
proba.append(float(idx) / float(total))
df['Proba'] = proba
return df
# check results are the same
assert df.pipe(jpp).equals(df.pipe(yolo))
%timeit df.pipe(jpp) # 691 µs
%timeit df.pipe(yolo) # 840 µs
answered Dec 31 '18 at 11:43
jppjpp
100k2161111
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53986877%2fpandas-iterate-over-a-row-and-adding-the-value-to-an-empty-column%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
I think the problem is, you are assigning the result back to the row, which doesn't get stored anywhere. instead you can do:
proba =
for index, row in dfnew.iterrows():
idx = idx - row['CPB%']
proba.append(float(idx) / float(total))
dfnew['Proba'] = proba
However, this is not the best way, you can use .apply with axis=1 to do row-wise calculations on a data frame.
answered Dec 31 '18 at 11:32
YOLOYOLO
5,4721424
Yes thanks but I have another issue error: 'Cannot concatenate object of type "<type 'float'>"; only pd.Series
– Viktor.w
Dec 31 '18 at 11:40
1
You would need to useproba =instead ofproba = pd.Series()here. But not really necessary when you can usecumsum.
– jpp
Dec 31 '18 at 11:46
add a comment |
I think the problem is, you are assigning the result back to the row, which doesn't get stored anywhere. instead you can do:
proba =
for index, row in dfnew.iterrows():
idx = idx - row['CPB%']
proba.append(float(idx) / float(total))
dfnew['Proba'] = proba
However, this is not the best way, you can use .apply with axis=1 to do row-wise calculations on a data frame.
answered Dec 31 '18 at 11:32
YOLOYOLO
5,4721424
Yes thanks but I have another issue error: 'Cannot concatenate object of type "<type 'float'>"; only pd.Series
– Viktor.w
Dec 31 '18 at 11:40
1
You would need to useproba =instead ofproba = pd.Series()here. But not really necessary when you can usecumsum.
– jpp
Dec 31 '18 at 11:46
add a comment |
I think the problem is, you are assigning the result back to the row, which doesn't get stored anywhere. instead you can do:
proba =
for index, row in dfnew.iterrows():
idx = idx - row['CPB%']
proba.append(float(idx) / float(total))
dfnew['Proba'] = proba
However, this is not the best way, you can use .apply with axis=1 to do row-wise calculations on a data frame.
answered Dec 31 '18 at 11:32
YOLOYOLO
5,4721424
I think the problem is, you are assigning the result back to the row, which doesn't get stored anywhere. instead you can do:
proba =
for index, row in dfnew.iterrows():
idx = idx - row['CPB%']
proba.append(float(idx) / float(total))
dfnew['Proba'] = proba
However, this is not the best way, you can use .apply with axis=1 to do row-wise calculations on a data frame.
answered Dec 31 '18 at 11:32
YOLOYOLO
5,4721424
edited Dec 31 '18 at 12:08
answered Dec 31 '18 at 11:32
YOLOYOLO
5,4721424
answered Dec 31 '18 at 11:32
YOLOYOLO
5,4721424
answered Dec 31 '18 at 11:32
YOLOYOLO
5,4721424
5,4721424
Yes thanks but I have another issue error: 'Cannot concatenate object of type "<type 'float'>"; only pd.Series
– Viktor.w
Dec 31 '18 at 11:40
1
You would need to useproba =instead ofproba = pd.Series()here. But not really necessary when you can usecumsum.
– jpp
Dec 31 '18 at 11:46
add a comment |
Yes thanks but I have another issue error: 'Cannot concatenate object of type "<type 'float'>"; only pd.Series
– Viktor.w
Dec 31 '18 at 11:40
1
You would need to useproba =instead ofproba = pd.Series()here. But not really necessary when you can usecumsum.
– jpp
Dec 31 '18 at 11:46
Yes thanks but I have another issue error: 'Cannot concatenate object of type "<type 'float'>"; only pd.Series
– Viktor.w
Dec 31 '18 at 11:40
Yes thanks but I have another issue error: 'Cannot concatenate object of type "<type 'float'>"; only pd.Series
– Viktor.w
Dec 31 '18 at 11:40
1
1
You would need to use
proba = instead of proba = pd.Series() here. But not really necessary when you can use cumsum.– jpp
Dec 31 '18 at 11:46
You would need to use
proba = instead of proba = pd.Series() here. But not really necessary when you can use cumsum.– jpp
Dec 31 '18 at 11:46
add a comment |
You can use pd.Series.cumsum to perform your iterative deductions:
total = dfnew['CPB%'].sum()
dfnew['Proba'] = 1 - df['CPB%'].cumsum() / total
With Pandas you should look to vectorise algorithms, which usually involves column-wise operations as opposed to a row-wise for loop. Here's a complete demonstration:
df = pd.DataFrame({'A': list(range(1, 7))})
def jpp(df):
total = df['A'].sum()
df['Proba'] = 1 - df['A'].cumsum() / total
return df
def yolo(df):
total = df['A'].sum()
idx = total
proba =
for index, row in df.iterrows():
idx = idx - row['A']
proba.append(float(idx) / float(total))
df['Proba'] = proba
return df
# check results are the same
assert df.pipe(jpp).equals(df.pipe(yolo))
%timeit df.pipe(jpp) # 691 µs
%timeit df.pipe(yolo) # 840 µs
answered Dec 31 '18 at 11:43
jppjpp
100k2161111
add a comment |
You can use pd.Series.cumsum to perform your iterative deductions:
total = dfnew['CPB%'].sum()
dfnew['Proba'] = 1 - df['CPB%'].cumsum() / total
With Pandas you should look to vectorise algorithms, which usually involves column-wise operations as opposed to a row-wise for loop. Here's a complete demonstration:
df = pd.DataFrame({'A': list(range(1, 7))})
def jpp(df):
total = df['A'].sum()
df['Proba'] = 1 - df['A'].cumsum() / total
return df
def yolo(df):
total = df['A'].sum()
idx = total
proba =
for index, row in df.iterrows():
idx = idx - row['A']
proba.append(float(idx) / float(total))
df['Proba'] = proba
return df
# check results are the same
assert df.pipe(jpp).equals(df.pipe(yolo))
%timeit df.pipe(jpp) # 691 µs
%timeit df.pipe(yolo) # 840 µs
answered Dec 31 '18 at 11:43
jppjpp
100k2161111
add a comment |
You can use pd.Series.cumsum to perform your iterative deductions:
total = dfnew['CPB%'].sum()
dfnew['Proba'] = 1 - df['CPB%'].cumsum() / total
With Pandas you should look to vectorise algorithms, which usually involves column-wise operations as opposed to a row-wise for loop. Here's a complete demonstration:
df = pd.DataFrame({'A': list(range(1, 7))})
def jpp(df):
total = df['A'].sum()
df['Proba'] = 1 - df['A'].cumsum() / total
return df
def yolo(df):
total = df['A'].sum()
idx = total
proba =
for index, row in df.iterrows():
idx = idx - row['A']
proba.append(float(idx) / float(total))
df['Proba'] = proba
return df
# check results are the same
assert df.pipe(jpp).equals(df.pipe(yolo))
%timeit df.pipe(jpp) # 691 µs
%timeit df.pipe(yolo) # 840 µs
answered Dec 31 '18 at 11:43
jppjpp
100k2161111
You can use pd.Series.cumsum to perform your iterative deductions:
total = dfnew['CPB%'].sum()
dfnew['Proba'] = 1 - df['CPB%'].cumsum() / total
With Pandas you should look to vectorise algorithms, which usually involves column-wise operations as opposed to a row-wise for loop. Here's a complete demonstration:
df = pd.DataFrame({'A': list(range(1, 7))})
def jpp(df):
total = df['A'].sum()
df['Proba'] = 1 - df['A'].cumsum() / total
return df
def yolo(df):
total = df['A'].sum()
idx = total
proba =
for index, row in df.iterrows():
idx = idx - row['A']
proba.append(float(idx) / float(total))
df['Proba'] = proba
return df
# check results are the same
assert df.pipe(jpp).equals(df.pipe(yolo))
%timeit df.pipe(jpp) # 691 µs
%timeit df.pipe(yolo) # 840 µs
answered Dec 31 '18 at 11:43
jppjpp
100k2161111
answered Dec 31 '18 at 11:43
jppjpp
100k2161111
answered Dec 31 '18 at 11:43
jppjpp
100k2161111
answered Dec 31 '18 at 11:43
jppjpp
100k2161111
100k2161111
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53986877%2fpandas-iterate-over-a-row-and-adding-the-value-to-an-empty-column%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
2
Please paste your dataframe as text and not an image
– Dan
Dec 31 '18 at 11:40