Doubts regarding `Understanding Keras LSTMs`
I am new to LSTMs and going through the Understanding Keras LSTMs and had some silly doubts related to a beautiful answer by Daniel Moller.
Here are some of my doubts:
There are 2 ways specified under the
Achieving one to manysection where it’s written that we can usestateful=Trueto recurrently take the output of one step and serve it as the input of the next step (needs output_features == input_features).
In the
One to many with repeat vectordiagram, the repeated vector is fed as input in all the time-step, whereas in theOne to many with stateful=Truethe output is fed as input in the next time step. So, aren't we changing the way the layers work by using thestateful=True?
Which of the above 2 approaches (using the repeat vector OR feeding the previous time-step output as the next input) should be followed when building an RNN?
Under the
One to many with stateful=Truesection, to change the behaviour ofone to many, in the code for manual loop for prediction, how will we know thesteps_to_predictvariable because we don't know the ouput sequence length in advance.
I also did not understand the way the entire model is using the
last_step outputto generate thenext_step ouput. It has confused me about the working ofmodel.predict()function. I mean, doesn'tmodel.predict()simultaneously predict the entire output sequences at once rather than looping through theno. of output sequences(whose value I still don't know) to be generated and doingmodel.predict()to predict a specific time-step output in a given iteration?
I couldn't understand the entire of
Many to manycase. Any other link would be helpful.
I understand that we use
model.reset_states()to make sure that a new batch is independent of the previous batch. But, Do we manually create batches of sequence such that one batch follows another batch or doesKerasinstateful=Truemode automatically divides the sequence into such batches.
If it's done manually then, why would anyone divide the dataset into such batches in which a part of a sequence is in one batch and the other in the next batch?
At last, what are the practical implementation or examples/use-cases where
stateful=Truewould be used(because this seems to be something unusual)? I am learning LSTMs and this is the first time I've been introduced tostatefulin Keras.
Can anyone help me in explaining my silly questions so that I can be clear on LSTM implementation in Keras?
keras deep-learning lstm rnn
asked Dec 28 '18 at 7:28
Ajay Singh NegiAjay Singh Negi
314217
add a comment |
I am new to LSTMs and going through the Understanding Keras LSTMs and had some silly doubts related to a beautiful answer by Daniel Moller.
Here are some of my doubts:
There are 2 ways specified under the
Achieving one to manysection where it’s written that we can usestateful=Trueto recurrently take the output of one step and serve it as the input of the next step (needs output_features == input_features).
In the
One to many with repeat vectordiagram, the repeated vector is fed as input in all the time-step, whereas in theOne to many with stateful=Truethe output is fed as input in the next time step. So, aren't we changing the way the layers work by using thestateful=True?
Which of the above 2 approaches (using the repeat vector OR feeding the previous time-step output as the next input) should be followed when building an RNN?
Under the
One to many with stateful=Truesection, to change the behaviour ofone to many, in the code for manual loop for prediction, how will we know thesteps_to_predictvariable because we don't know the ouput sequence length in advance.
I also did not understand the way the entire model is using the
last_step outputto generate thenext_step ouput. It has confused me about the working ofmodel.predict()function. I mean, doesn'tmodel.predict()simultaneously predict the entire output sequences at once rather than looping through theno. of output sequences(whose value I still don't know) to be generated and doingmodel.predict()to predict a specific time-step output in a given iteration?
I couldn't understand the entire of
Many to manycase. Any other link would be helpful.
I understand that we use
model.reset_states()to make sure that a new batch is independent of the previous batch. But, Do we manually create batches of sequence such that one batch follows another batch or doesKerasinstateful=Truemode automatically divides the sequence into such batches.
If it's done manually then, why would anyone divide the dataset into such batches in which a part of a sequence is in one batch and the other in the next batch?
At last, what are the practical implementation or examples/use-cases where
stateful=Truewould be used(because this seems to be something unusual)? I am learning LSTMs and this is the first time I've been introduced tostatefulin Keras.
Can anyone help me in explaining my silly questions so that I can be clear on LSTM implementation in Keras?
keras deep-learning lstm rnn
asked Dec 28 '18 at 7:28
Ajay Singh NegiAjay Singh Negi
314217
add a comment |
I am new to LSTMs and going through the Understanding Keras LSTMs and had some silly doubts related to a beautiful answer by Daniel Moller.
Here are some of my doubts:
There are 2 ways specified under the
Achieving one to manysection where it’s written that we can usestateful=Trueto recurrently take the output of one step and serve it as the input of the next step (needs output_features == input_features).
In the
One to many with repeat vectordiagram, the repeated vector is fed as input in all the time-step, whereas in theOne to many with stateful=Truethe output is fed as input in the next time step. So, aren't we changing the way the layers work by using thestateful=True?
Which of the above 2 approaches (using the repeat vector OR feeding the previous time-step output as the next input) should be followed when building an RNN?
Under the
One to many with stateful=Truesection, to change the behaviour ofone to many, in the code for manual loop for prediction, how will we know thesteps_to_predictvariable because we don't know the ouput sequence length in advance.
I also did not understand the way the entire model is using the
last_step outputto generate thenext_step ouput. It has confused me about the working ofmodel.predict()function. I mean, doesn'tmodel.predict()simultaneously predict the entire output sequences at once rather than looping through theno. of output sequences(whose value I still don't know) to be generated and doingmodel.predict()to predict a specific time-step output in a given iteration?
I couldn't understand the entire of
Many to manycase. Any other link would be helpful.
I understand that we use
model.reset_states()to make sure that a new batch is independent of the previous batch. But, Do we manually create batches of sequence such that one batch follows another batch or doesKerasinstateful=Truemode automatically divides the sequence into such batches.
If it's done manually then, why would anyone divide the dataset into such batches in which a part of a sequence is in one batch and the other in the next batch?
At last, what are the practical implementation or examples/use-cases where
stateful=Truewould be used(because this seems to be something unusual)? I am learning LSTMs and this is the first time I've been introduced tostatefulin Keras.
Can anyone help me in explaining my silly questions so that I can be clear on LSTM implementation in Keras?
keras deep-learning lstm rnn
asked Dec 28 '18 at 7:28
Ajay Singh NegiAjay Singh Negi
314217
I am new to LSTMs and going through the Understanding Keras LSTMs and had some silly doubts related to a beautiful answer by Daniel Moller.
Here are some of my doubts:
There are 2 ways specified under the
Achieving one to manysection where it’s written that we can usestateful=Trueto recurrently take the output of one step and serve it as the input of the next step (needs output_features == input_features).
In the
One to many with repeat vectordiagram, the repeated vector is fed as input in all the time-step, whereas in theOne to many with stateful=Truethe output is fed as input in the next time step. So, aren't we changing the way the layers work by using thestateful=True?
Which of the above 2 approaches (using the repeat vector OR feeding the previous time-step output as the next input) should be followed when building an RNN?
Under the
One to many with stateful=Truesection, to change the behaviour ofone to many, in the code for manual loop for prediction, how will we know thesteps_to_predictvariable because we don't know the ouput sequence length in advance.
I also did not understand the way the entire model is using the
last_step outputto generate thenext_step ouput. It has confused me about the working ofmodel.predict()function. I mean, doesn'tmodel.predict()simultaneously predict the entire output sequences at once rather than looping through theno. of output sequences(whose value I still don't know) to be generated and doingmodel.predict()to predict a specific time-step output in a given iteration?
I couldn't understand the entire of
Many to manycase. Any other link would be helpful.
I understand that we use
model.reset_states()to make sure that a new batch is independent of the previous batch. But, Do we manually create batches of sequence such that one batch follows another batch or doesKerasinstateful=Truemode automatically divides the sequence into such batches.
If it's done manually then, why would anyone divide the dataset into such batches in which a part of a sequence is in one batch and the other in the next batch?
At last, what are the practical implementation or examples/use-cases where
stateful=Truewould be used(because this seems to be something unusual)? I am learning LSTMs and this is the first time I've been introduced tostatefulin Keras.
Can anyone help me in explaining my silly questions so that I can be clear on LSTM implementation in Keras?
keras deep-learning lstm rnn
keras deep-learning lstm rnn
asked Dec 28 '18 at 7:28
Ajay Singh NegiAjay Singh Negi
314217
asked Dec 28 '18 at 7:28
Ajay Singh NegiAjay Singh Negi
314217
edited Dec 29 '18 at 1:58
Ajay Singh Negi
asked Dec 28 '18 at 7:28
Ajay Singh NegiAjay Singh Negi
314217
asked Dec 28 '18 at 7:28
Ajay Singh NegiAjay Singh Negi
314217
asked Dec 28 '18 at 7:28
Ajay Singh NegiAjay Singh Negi
314217
314217
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53955093%2fdoubts-regarding-understanding-keras-lstms%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53955093%2fdoubts-regarding-understanding-keras-lstms%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
