Creating social graph via twitter4j
i would like to create social graph based on users and followers from Twitter.
I'm using twitter4j library. I'm iterating over graph starting from selected user and then i'm getting his followers, then recursively function is executed for his followers. The problem is i receive rate limit very quickly.
public void getFollowers(Twitter twitter, String twitterScreenName)
{
try
{
IDs followerIDs = twitter.getFollowersIDs(twitterScreenName, -1);
long followerIdList = followerIDs.getIDs();
if (followerIdList.length > 0)
{
String screenName = "";
for(long id: followerIdList)
{
twitter4j.User user = twitter.showUser(id);
screenName = user.getScreenName();
System.out.println("Name: " + screenName);
getFollowers(twitter, screenName);
}
}
}
catch(TwitterException e)
{
e.getMessage();
}
}
The result is unsatisfactory because i can't get deeper to followers of user followers and so on. I know i could wait 15 minutes and again restore program working but it would last too long and graph is very poor.
Is it another way to bypass that problem or other tools similar to twitter4j which can solve problem of creating graph? Thanks for any help.
java twitter graph twitter4j social
asked Dec 29 '18 at 13:33
Maciej MMaciej M
11
|
show 6 more comments
i would like to create social graph based on users and followers from Twitter.
I'm using twitter4j library. I'm iterating over graph starting from selected user and then i'm getting his followers, then recursively function is executed for his followers. The problem is i receive rate limit very quickly.
public void getFollowers(Twitter twitter, String twitterScreenName)
{
try
{
IDs followerIDs = twitter.getFollowersIDs(twitterScreenName, -1);
long followerIdList = followerIDs.getIDs();
if (followerIdList.length > 0)
{
String screenName = "";
for(long id: followerIdList)
{
twitter4j.User user = twitter.showUser(id);
screenName = user.getScreenName();
System.out.println("Name: " + screenName);
getFollowers(twitter, screenName);
}
}
}
catch(TwitterException e)
{
e.getMessage();
}
}
The result is unsatisfactory because i can't get deeper to followers of user followers and so on. I know i could wait 15 minutes and again restore program working but it would last too long and graph is very poor.
Is it another way to bypass that problem or other tools similar to twitter4j which can solve problem of creating graph? Thanks for any help.
java twitter graph twitter4j social
asked Dec 29 '18 at 13:33
Maciej MMaciej M
11
1
No, this is a restriction placed by Twitter itself. You are going to have to limit this somehow.
– Joe C
Dec 29 '18 at 13:38
so it is impossible to create large graph this way in reasonable time?
– Maciej M
Dec 29 '18 at 13:40
1
I think this is intended, it's a shared service by Twitter. Otherwise, People would and could load the whole dataset and consume all resources. For any other analysis, a dump can by purchased from Twitter or one might to use older dumps form academic projects maybe
– AKSW
Dec 29 '18 at 13:40
Such a graph might be possible if you're willing to pay Twitter for an increased rate limit. But if not, then no, not possible.
– Joe C
Dec 29 '18 at 13:42
According to the documentation at developer.twitter.com/en/docs/accounts-and-users/… you can request 15 times in 15 minutes, so you should "sleep" for 60 secs between each request.
– JeffProd
Dec 29 '18 at 15:52
|
show 6 more comments
i would like to create social graph based on users and followers from Twitter.
I'm using twitter4j library. I'm iterating over graph starting from selected user and then i'm getting his followers, then recursively function is executed for his followers. The problem is i receive rate limit very quickly.
public void getFollowers(Twitter twitter, String twitterScreenName)
{
try
{
IDs followerIDs = twitter.getFollowersIDs(twitterScreenName, -1);
long followerIdList = followerIDs.getIDs();
if (followerIdList.length > 0)
{
String screenName = "";
for(long id: followerIdList)
{
twitter4j.User user = twitter.showUser(id);
screenName = user.getScreenName();
System.out.println("Name: " + screenName);
getFollowers(twitter, screenName);
}
}
}
catch(TwitterException e)
{
e.getMessage();
}
}
The result is unsatisfactory because i can't get deeper to followers of user followers and so on. I know i could wait 15 minutes and again restore program working but it would last too long and graph is very poor.
Is it another way to bypass that problem or other tools similar to twitter4j which can solve problem of creating graph? Thanks for any help.
java twitter graph twitter4j social
asked Dec 29 '18 at 13:33
Maciej MMaciej M
11
i would like to create social graph based on users and followers from Twitter.
I'm using twitter4j library. I'm iterating over graph starting from selected user and then i'm getting his followers, then recursively function is executed for his followers. The problem is i receive rate limit very quickly.
public void getFollowers(Twitter twitter, String twitterScreenName)
{
try
{
IDs followerIDs = twitter.getFollowersIDs(twitterScreenName, -1);
long followerIdList = followerIDs.getIDs();
if (followerIdList.length > 0)
{
String screenName = "";
for(long id: followerIdList)
{
twitter4j.User user = twitter.showUser(id);
screenName = user.getScreenName();
System.out.println("Name: " + screenName);
getFollowers(twitter, screenName);
}
}
}
catch(TwitterException e)
{
e.getMessage();
}
}
The result is unsatisfactory because i can't get deeper to followers of user followers and so on. I know i could wait 15 minutes and again restore program working but it would last too long and graph is very poor.
Is it another way to bypass that problem or other tools similar to twitter4j which can solve problem of creating graph? Thanks for any help.
java twitter graph twitter4j social
java twitter graph twitter4j social
asked Dec 29 '18 at 13:33
Maciej MMaciej M
11
asked Dec 29 '18 at 13:33
Maciej MMaciej M
11
edited Dec 29 '18 at 13:56
Maciej M
asked Dec 29 '18 at 13:33
Maciej MMaciej M
11
asked Dec 29 '18 at 13:33
Maciej MMaciej M
11
asked Dec 29 '18 at 13:33
Maciej MMaciej M
11
11
1
No, this is a restriction placed by Twitter itself. You are going to have to limit this somehow.
– Joe C
Dec 29 '18 at 13:38
so it is impossible to create large graph this way in reasonable time?
– Maciej M
Dec 29 '18 at 13:40
1
I think this is intended, it's a shared service by Twitter. Otherwise, People would and could load the whole dataset and consume all resources. For any other analysis, a dump can by purchased from Twitter or one might to use older dumps form academic projects maybe
– AKSW
Dec 29 '18 at 13:40
Such a graph might be possible if you're willing to pay Twitter for an increased rate limit. But if not, then no, not possible.
– Joe C
Dec 29 '18 at 13:42
According to the documentation at developer.twitter.com/en/docs/accounts-and-users/… you can request 15 times in 15 minutes, so you should "sleep" for 60 secs between each request.
– JeffProd
Dec 29 '18 at 15:52
|
show 6 more comments
1
No, this is a restriction placed by Twitter itself. You are going to have to limit this somehow.
– Joe C
Dec 29 '18 at 13:38
so it is impossible to create large graph this way in reasonable time?
– Maciej M
Dec 29 '18 at 13:40
1
I think this is intended, it's a shared service by Twitter. Otherwise, People would and could load the whole dataset and consume all resources. For any other analysis, a dump can by purchased from Twitter or one might to use older dumps form academic projects maybe
– AKSW
Dec 29 '18 at 13:40
Such a graph might be possible if you're willing to pay Twitter for an increased rate limit. But if not, then no, not possible.
– Joe C
Dec 29 '18 at 13:42
According to the documentation at developer.twitter.com/en/docs/accounts-and-users/… you can request 15 times in 15 minutes, so you should "sleep" for 60 secs between each request.
– JeffProd
Dec 29 '18 at 15:52
1
1
No, this is a restriction placed by Twitter itself. You are going to have to limit this somehow.
– Joe C
Dec 29 '18 at 13:38
No, this is a restriction placed by Twitter itself. You are going to have to limit this somehow.
– Joe C
Dec 29 '18 at 13:38
so it is impossible to create large graph this way in reasonable time?
– Maciej M
Dec 29 '18 at 13:40
so it is impossible to create large graph this way in reasonable time?
– Maciej M
Dec 29 '18 at 13:40
1
1
I think this is intended, it's a shared service by Twitter. Otherwise, People would and could load the whole dataset and consume all resources. For any other analysis, a dump can by purchased from Twitter or one might to use older dumps form academic projects maybe
– AKSW
Dec 29 '18 at 13:40
I think this is intended, it's a shared service by Twitter. Otherwise, People would and could load the whole dataset and consume all resources. For any other analysis, a dump can by purchased from Twitter or one might to use older dumps form academic projects maybe
– AKSW
Dec 29 '18 at 13:40
Such a graph might be possible if you're willing to pay Twitter for an increased rate limit. But if not, then no, not possible.
– Joe C
Dec 29 '18 at 13:42
Such a graph might be possible if you're willing to pay Twitter for an increased rate limit. But if not, then no, not possible.
– Joe C
Dec 29 '18 at 13:42
According to the documentation at developer.twitter.com/en/docs/accounts-and-users/… you can request 15 times in 15 minutes, so you should "sleep" for 60 secs between each request.
– JeffProd
Dec 29 '18 at 15:52
According to the documentation at developer.twitter.com/en/docs/accounts-and-users/… you can request 15 times in 15 minutes, so you should "sleep" for 60 secs between each request.
– JeffProd
Dec 29 '18 at 15:52
|
show 6 more comments
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53970033%2fcreating-social-graph-via-twitter4j%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53970033%2fcreating-social-graph-via-twitter4j%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
No, this is a restriction placed by Twitter itself. You are going to have to limit this somehow.
– Joe C
Dec 29 '18 at 13:38
so it is impossible to create large graph this way in reasonable time?
– Maciej M
Dec 29 '18 at 13:40
1
I think this is intended, it's a shared service by Twitter. Otherwise, People would and could load the whole dataset and consume all resources. For any other analysis, a dump can by purchased from Twitter or one might to use older dumps form academic projects maybe
– AKSW
Dec 29 '18 at 13:40
Such a graph might be possible if you're willing to pay Twitter for an increased rate limit. But if not, then no, not possible.
– Joe C
Dec 29 '18 at 13:42
According to the documentation at developer.twitter.com/en/docs/accounts-and-users/… you can request 15 times in 15 minutes, so you should "sleep" for 60 secs between each request.
– JeffProd
Dec 29 '18 at 15:52