Splitting up long-running multi-row request
I have a python Flask server a javascript Vue.js client.
The server is a REST API, it has a long-running (10sec) request that responds with a json of the form:
{
"foo": "bar",
"data": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
}
The reason why it takes so long is because it's contacting a third-party service to get the data in the array. This third-party service is paginated and it takes about 1sec to return each page, which is how I get all 10 parts of my response data array. So potentially, part 1/10 of my array is obtained after only a 1sec wait. I would like to return it as soon as possible so that the client can display the first entry but keep the connection open and keep populating the remaining events as it gets them, and then close when it's all done.
Imagine the result is a 10 row table for example, at time=1s I'd show row 1, time=2s add row 2... etc etc...
It's possible that data obtained in a later part have repercussion on data obtained earlier. So I would like to be able to, for example, add row 8 and update row 2 at the same time when time=8s.
What are the best practices to split up such a long-running request, from both the server and client perspective? SSE? Websocket? Streaming?
The array is not particularly long in terms of bytesize, it's purely a time concern.
PS: I would like to avoid refactoring the third-party request if possible (into 10 parallel requests for example, because i don't necessarily know the pagination parameters in advance)
javascript rest api vue.js flask
asked Dec 27 at 14:24
thsr
133
add a comment |
I have a python Flask server a javascript Vue.js client.
The server is a REST API, it has a long-running (10sec) request that responds with a json of the form:
{
"foo": "bar",
"data": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
}
The reason why it takes so long is because it's contacting a third-party service to get the data in the array. This third-party service is paginated and it takes about 1sec to return each page, which is how I get all 10 parts of my response data array. So potentially, part 1/10 of my array is obtained after only a 1sec wait. I would like to return it as soon as possible so that the client can display the first entry but keep the connection open and keep populating the remaining events as it gets them, and then close when it's all done.
Imagine the result is a 10 row table for example, at time=1s I'd show row 1, time=2s add row 2... etc etc...
It's possible that data obtained in a later part have repercussion on data obtained earlier. So I would like to be able to, for example, add row 8 and update row 2 at the same time when time=8s.
What are the best practices to split up such a long-running request, from both the server and client perspective? SSE? Websocket? Streaming?
The array is not particularly long in terms of bytesize, it's purely a time concern.
PS: I would like to avoid refactoring the third-party request if possible (into 10 parallel requests for example, because i don't necessarily know the pagination parameters in advance)
javascript rest api vue.js flask
asked Dec 27 at 14:24
thsr
133
1
In your case I'd be tempted to use SSE/EventSource. But WebSockets would also do the trick. Don't know about streaming; if you're talking about fetch readable streams, browser support looks really spotty
– David784
Dec 27 at 15:48
add a comment |
I have a python Flask server a javascript Vue.js client.
The server is a REST API, it has a long-running (10sec) request that responds with a json of the form:
{
"foo": "bar",
"data": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
}
The reason why it takes so long is because it's contacting a third-party service to get the data in the array. This third-party service is paginated and it takes about 1sec to return each page, which is how I get all 10 parts of my response data array. So potentially, part 1/10 of my array is obtained after only a 1sec wait. I would like to return it as soon as possible so that the client can display the first entry but keep the connection open and keep populating the remaining events as it gets them, and then close when it's all done.
Imagine the result is a 10 row table for example, at time=1s I'd show row 1, time=2s add row 2... etc etc...
It's possible that data obtained in a later part have repercussion on data obtained earlier. So I would like to be able to, for example, add row 8 and update row 2 at the same time when time=8s.
What are the best practices to split up such a long-running request, from both the server and client perspective? SSE? Websocket? Streaming?
The array is not particularly long in terms of bytesize, it's purely a time concern.
PS: I would like to avoid refactoring the third-party request if possible (into 10 parallel requests for example, because i don't necessarily know the pagination parameters in advance)
javascript rest api vue.js flask
asked Dec 27 at 14:24
thsr
133
I have a python Flask server a javascript Vue.js client.
The server is a REST API, it has a long-running (10sec) request that responds with a json of the form:
{
"foo": "bar",
"data": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
}
The reason why it takes so long is because it's contacting a third-party service to get the data in the array. This third-party service is paginated and it takes about 1sec to return each page, which is how I get all 10 parts of my response data array. So potentially, part 1/10 of my array is obtained after only a 1sec wait. I would like to return it as soon as possible so that the client can display the first entry but keep the connection open and keep populating the remaining events as it gets them, and then close when it's all done.
Imagine the result is a 10 row table for example, at time=1s I'd show row 1, time=2s add row 2... etc etc...
It's possible that data obtained in a later part have repercussion on data obtained earlier. So I would like to be able to, for example, add row 8 and update row 2 at the same time when time=8s.
What are the best practices to split up such a long-running request, from both the server and client perspective? SSE? Websocket? Streaming?
The array is not particularly long in terms of bytesize, it's purely a time concern.
PS: I would like to avoid refactoring the third-party request if possible (into 10 parallel requests for example, because i don't necessarily know the pagination parameters in advance)
javascript rest api vue.js flask
javascript rest api vue.js flask
asked Dec 27 at 14:24
thsr
133
asked Dec 27 at 14:24
thsr
133
asked Dec 27 at 14:24
thsr
133
asked Dec 27 at 14:24
thsr
133
asked Dec 27 at 14:24
thsr
133
133
1
In your case I'd be tempted to use SSE/EventSource. But WebSockets would also do the trick. Don't know about streaming; if you're talking about fetch readable streams, browser support looks really spotty
– David784
Dec 27 at 15:48
add a comment |
1
In your case I'd be tempted to use SSE/EventSource. But WebSockets would also do the trick. Don't know about streaming; if you're talking about fetch readable streams, browser support looks really spotty
– David784
Dec 27 at 15:48
1
1
In your case I'd be tempted to use SSE/EventSource. But WebSockets would also do the trick. Don't know about streaming; if you're talking about fetch readable streams, browser support looks really spotty
– David784
Dec 27 at 15:48
In your case I'd be tempted to use SSE/EventSource. But WebSockets would also do the trick. Don't know about streaming; if you're talking about fetch readable streams, browser support looks really spotty
– David784
Dec 27 at 15:48
add a comment |
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53946584%2fsplitting-up-long-running-multi-row-request%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53946584%2fsplitting-up-long-running-multi-row-request%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
In your case I'd be tempted to use SSE/EventSource. But WebSockets would also do the trick. Don't know about streaming; if you're talking about fetch readable streams, browser support looks really spotty
– David784
Dec 27 at 15:48