Scrapy returning 302 for some network
I'm getting 302 while scraping using scrapy in some specific network. It is working in my office network but not in my home network. It is working in my one mobile network but not working (getting 302)in another mobile network. Do you guys know why this is happening?
python-3.x scrapy scrapy-spider
asked Jan 2 at 17:05
Arojit GhoshArojit Ghosh
65
add a comment |
I'm getting 302 while scraping using scrapy in some specific network. It is working in my office network but not in my home network. It is working in my one mobile network but not working (getting 302)in another mobile network. Do you guys know why this is happening?
python-3.x scrapy scrapy-spider
asked Jan 2 at 17:05
Arojit GhoshArojit Ghosh
65
add a comment |
I'm getting 302 while scraping using scrapy in some specific network. It is working in my office network but not in my home network. It is working in my one mobile network but not working (getting 302)in another mobile network. Do you guys know why this is happening?
python-3.x scrapy scrapy-spider
asked Jan 2 at 17:05
Arojit GhoshArojit Ghosh
65
I'm getting 302 while scraping using scrapy in some specific network. It is working in my office network but not in my home network. It is working in my one mobile network but not working (getting 302)in another mobile network. Do you guys know why this is happening?
python-3.x scrapy scrapy-spider
python-3.x scrapy scrapy-spider
asked Jan 2 at 17:05
Arojit GhoshArojit Ghosh
65
asked Jan 2 at 17:05
Arojit GhoshArojit Ghosh
65
asked Jan 2 at 17:05
Arojit GhoshArojit Ghosh
65
asked Jan 2 at 17:05
Arojit GhoshArojit Ghosh
65
asked Jan 2 at 17:05
Arojit GhoshArojit Ghosh
65
65
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
Without knowing which specific URL does it, I can only guess that those IP addresses that get a 302 might have been flagged by an anti-bot software solution and are being redirected to some kind of CAPTCHA.
Try using Splash, see if the results are the same.
answered Jan 16 at 11:37
GallaecioGallaecio
9091923
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54010354%2fscrapy-returning-302-for-some-network%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Without knowing which specific URL does it, I can only guess that those IP addresses that get a 302 might have been flagged by an anti-bot software solution and are being redirected to some kind of CAPTCHA.
Try using Splash, see if the results are the same.
answered Jan 16 at 11:37
GallaecioGallaecio
9091923
add a comment |
Without knowing which specific URL does it, I can only guess that those IP addresses that get a 302 might have been flagged by an anti-bot software solution and are being redirected to some kind of CAPTCHA.
Try using Splash, see if the results are the same.
answered Jan 16 at 11:37
GallaecioGallaecio
9091923
add a comment |
Without knowing which specific URL does it, I can only guess that those IP addresses that get a 302 might have been flagged by an anti-bot software solution and are being redirected to some kind of CAPTCHA.
Try using Splash, see if the results are the same.
answered Jan 16 at 11:37
GallaecioGallaecio
9091923
Without knowing which specific URL does it, I can only guess that those IP addresses that get a 302 might have been flagged by an anti-bot software solution and are being redirected to some kind of CAPTCHA.
Try using Splash, see if the results are the same.
answered Jan 16 at 11:37
GallaecioGallaecio
9091923
answered Jan 16 at 11:37
GallaecioGallaecio
9091923
answered Jan 16 at 11:37
GallaecioGallaecio
9091923
answered Jan 16 at 11:37
GallaecioGallaecio
9091923
9091923
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f54010354%2fscrapy-returning-302-for-some-network%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
