MATLAB error: Matrix is too large to return linear indices
I am trying to find all non-zeros elements in a large sparse matrix K(19683000x19683000). I used find to first return the linear indices.
val = K(K~=0);
inds = find(K~=0);
[j, i] = ind2sub(size(K), inds);
% [j, i] = find(K~=0);
i = full(i);
j = full(j);
This gave some error:
Error using find
Matrix is too large to return linear indices.
Use [i,j] = find(S) for sparse matrix.
Error in (line 82)
inds = find(K~=0);
Error in run (line 64)
evalin('caller', [script ';']);
Any idea what is happening and how I can avoid it?
matlab
asked Dec 29 '18 at 18:57
colddiecolddie
4771623
add a comment |
I am trying to find all non-zeros elements in a large sparse matrix K(19683000x19683000). I used find to first return the linear indices.
val = K(K~=0);
inds = find(K~=0);
[j, i] = ind2sub(size(K), inds);
% [j, i] = find(K~=0);
i = full(i);
j = full(j);
This gave some error:
Error using find
Matrix is too large to return linear indices.
Use [i,j] = find(S) for sparse matrix.
Error in (line 82)
inds = find(K~=0);
Error in run (line 64)
evalin('caller', [script ';']);
Any idea what is happening and how I can avoid it?
matlab
asked Dec 29 '18 at 18:57
colddiecolddie
4771623
I have never found that error, probably because I never used matrices so large. The error is likely related with the fact thatdoublevariables can exactly represent integers up to2^53-1only. The number of elements of your matrix does not exceed that limit, but it is near
– Luis Mendo
Dec 29 '18 at 19:15
Maybe you can sidestep the issue (up to2^53-1) by getting the row and column indices, and then computing the linear index from those manually:[ii,jj] = find(K); inds = ii + (jj-1)*size(K,1)
– Luis Mendo
Dec 29 '18 at 19:19
@LuisMendo indeed this is a solution for me. Thanks. Maybe you can put your suggestion as an answer so I can accept it?
– colddie
Jan 2 at 15:18
Glad it worked. I rephrased a bit and added as an answer
– Luis Mendo
Jan 3 at 0:58
add a comment |
I am trying to find all non-zeros elements in a large sparse matrix K(19683000x19683000). I used find to first return the linear indices.
val = K(K~=0);
inds = find(K~=0);
[j, i] = ind2sub(size(K), inds);
% [j, i] = find(K~=0);
i = full(i);
j = full(j);
This gave some error:
Error using find
Matrix is too large to return linear indices.
Use [i,j] = find(S) for sparse matrix.
Error in (line 82)
inds = find(K~=0);
Error in run (line 64)
evalin('caller', [script ';']);
Any idea what is happening and how I can avoid it?
matlab
asked Dec 29 '18 at 18:57
colddiecolddie
4771623
I am trying to find all non-zeros elements in a large sparse matrix K(19683000x19683000). I used find to first return the linear indices.
val = K(K~=0);
inds = find(K~=0);
[j, i] = ind2sub(size(K), inds);
% [j, i] = find(K~=0);
i = full(i);
j = full(j);
This gave some error:
Error using find
Matrix is too large to return linear indices.
Use [i,j] = find(S) for sparse matrix.
Error in (line 82)
inds = find(K~=0);
Error in run (line 64)
evalin('caller', [script ';']);
Any idea what is happening and how I can avoid it?
matlab
matlab
asked Dec 29 '18 at 18:57
colddiecolddie
4771623
asked Dec 29 '18 at 18:57
colddiecolddie
4771623
asked Dec 29 '18 at 18:57
colddiecolddie
4771623
asked Dec 29 '18 at 18:57
colddiecolddie
4771623
asked Dec 29 '18 at 18:57
colddiecolddie
4771623
4771623
I have never found that error, probably because I never used matrices so large. The error is likely related with the fact thatdoublevariables can exactly represent integers up to2^53-1only. The number of elements of your matrix does not exceed that limit, but it is near
– Luis Mendo
Dec 29 '18 at 19:15
Maybe you can sidestep the issue (up to2^53-1) by getting the row and column indices, and then computing the linear index from those manually:[ii,jj] = find(K); inds = ii + (jj-1)*size(K,1)
– Luis Mendo
Dec 29 '18 at 19:19
@LuisMendo indeed this is a solution for me. Thanks. Maybe you can put your suggestion as an answer so I can accept it?
– colddie
Jan 2 at 15:18
Glad it worked. I rephrased a bit and added as an answer
– Luis Mendo
Jan 3 at 0:58
add a comment |
I have never found that error, probably because I never used matrices so large. The error is likely related with the fact thatdoublevariables can exactly represent integers up to2^53-1only. The number of elements of your matrix does not exceed that limit, but it is near
– Luis Mendo
Dec 29 '18 at 19:15
Maybe you can sidestep the issue (up to2^53-1) by getting the row and column indices, and then computing the linear index from those manually:[ii,jj] = find(K); inds = ii + (jj-1)*size(K,1)
– Luis Mendo
Dec 29 '18 at 19:19
@LuisMendo indeed this is a solution for me. Thanks. Maybe you can put your suggestion as an answer so I can accept it?
– colddie
Jan 2 at 15:18
Glad it worked. I rephrased a bit and added as an answer
– Luis Mendo
Jan 3 at 0:58
I have never found that error, probably because I never used matrices so large. The error is likely related with the fact that
double variables can exactly represent integers up to 2^53-1 only. The number of elements of your matrix does not exceed that limit, but it is near– Luis Mendo
Dec 29 '18 at 19:15
I have never found that error, probably because I never used matrices so large. The error is likely related with the fact that
double variables can exactly represent integers up to 2^53-1 only. The number of elements of your matrix does not exceed that limit, but it is near– Luis Mendo
Dec 29 '18 at 19:15
Maybe you can sidestep the issue (up to
2^53-1) by getting the row and column indices, and then computing the linear index from those manually: [ii,jj] = find(K); inds = ii + (jj-1)*size(K,1)– Luis Mendo
Dec 29 '18 at 19:19
Maybe you can sidestep the issue (up to
2^53-1) by getting the row and column indices, and then computing the linear index from those manually: [ii,jj] = find(K); inds = ii + (jj-1)*size(K,1)– Luis Mendo
Dec 29 '18 at 19:19
@LuisMendo indeed this is a solution for me. Thanks. Maybe you can put your suggestion as an answer so I can accept it?
– colddie
Jan 2 at 15:18
@LuisMendo indeed this is a solution for me. Thanks. Maybe you can put your suggestion as an answer so I can accept it?
– colddie
Jan 2 at 15:18
Glad it worked. I rephrased a bit and added as an answer
– Luis Mendo
Jan 3 at 0:58
Glad it worked. I rephrased a bit and added as an answer
– Luis Mendo
Jan 3 at 0:58
add a comment |
1 Answer
1
active
oldest
votes
I am not familiar with that error (probably because I never used matrices so large). The error is likely related with the fact that double variables can exactly represent integers up to 2^53 only. The number of elements of your matrix does not exceed that limit, but it is near.
Maybe you can sidestep the issue, up to 2^53, by getting the row and column indices and then computing the linear index from those manually:
[ii,jj] = find(K);
inds = ii + (jj-1)*size(K,1);
If needed, you can push the limit up to 2^64 by using a uint64 linear index instead of double:
[ii,jj] = find(K);
inds = uint64(ii) + (uint64(jj)-1)*size(K,1);
answered Jan 3 at 0:57
Luis MendoLuis Mendo
93.1k1154122
1
There's a built in function to do the conversion: sub2ind
– Philip
Jan 3 at 1:02
@Philip Sure. I got into the habit of doing it manually because it's slightly faster :-)
– Luis Mendo
Jan 3 at 1:05
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53972495%2fmatlab-error-matrix-is-too-large-to-return-linear-indices%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
I am not familiar with that error (probably because I never used matrices so large). The error is likely related with the fact that double variables can exactly represent integers up to 2^53 only. The number of elements of your matrix does not exceed that limit, but it is near.
Maybe you can sidestep the issue, up to 2^53, by getting the row and column indices and then computing the linear index from those manually:
[ii,jj] = find(K);
inds = ii + (jj-1)*size(K,1);
If needed, you can push the limit up to 2^64 by using a uint64 linear index instead of double:
[ii,jj] = find(K);
inds = uint64(ii) + (uint64(jj)-1)*size(K,1);
answered Jan 3 at 0:57
Luis MendoLuis Mendo
93.1k1154122
1
There's a built in function to do the conversion: sub2ind
– Philip
Jan 3 at 1:02
@Philip Sure. I got into the habit of doing it manually because it's slightly faster :-)
– Luis Mendo
Jan 3 at 1:05
add a comment |
I am not familiar with that error (probably because I never used matrices so large). The error is likely related with the fact that double variables can exactly represent integers up to 2^53 only. The number of elements of your matrix does not exceed that limit, but it is near.
Maybe you can sidestep the issue, up to 2^53, by getting the row and column indices and then computing the linear index from those manually:
[ii,jj] = find(K);
inds = ii + (jj-1)*size(K,1);
If needed, you can push the limit up to 2^64 by using a uint64 linear index instead of double:
[ii,jj] = find(K);
inds = uint64(ii) + (uint64(jj)-1)*size(K,1);
answered Jan 3 at 0:57
Luis MendoLuis Mendo
93.1k1154122
1
There's a built in function to do the conversion: sub2ind
– Philip
Jan 3 at 1:02
@Philip Sure. I got into the habit of doing it manually because it's slightly faster :-)
– Luis Mendo
Jan 3 at 1:05
add a comment |
I am not familiar with that error (probably because I never used matrices so large). The error is likely related with the fact that double variables can exactly represent integers up to 2^53 only. The number of elements of your matrix does not exceed that limit, but it is near.
Maybe you can sidestep the issue, up to 2^53, by getting the row and column indices and then computing the linear index from those manually:
[ii,jj] = find(K);
inds = ii + (jj-1)*size(K,1);
If needed, you can push the limit up to 2^64 by using a uint64 linear index instead of double:
[ii,jj] = find(K);
inds = uint64(ii) + (uint64(jj)-1)*size(K,1);
answered Jan 3 at 0:57
Luis MendoLuis Mendo
93.1k1154122
I am not familiar with that error (probably because I never used matrices so large). The error is likely related with the fact that double variables can exactly represent integers up to 2^53 only. The number of elements of your matrix does not exceed that limit, but it is near.
Maybe you can sidestep the issue, up to 2^53, by getting the row and column indices and then computing the linear index from those manually:
[ii,jj] = find(K);
inds = ii + (jj-1)*size(K,1);
If needed, you can push the limit up to 2^64 by using a uint64 linear index instead of double:
[ii,jj] = find(K);
inds = uint64(ii) + (uint64(jj)-1)*size(K,1);
answered Jan 3 at 0:57
Luis MendoLuis Mendo
93.1k1154122
answered Jan 3 at 0:57
Luis MendoLuis Mendo
93.1k1154122
answered Jan 3 at 0:57
Luis MendoLuis Mendo
93.1k1154122
answered Jan 3 at 0:57
Luis MendoLuis Mendo
93.1k1154122
93.1k1154122
1
There's a built in function to do the conversion: sub2ind
– Philip
Jan 3 at 1:02
@Philip Sure. I got into the habit of doing it manually because it's slightly faster :-)
– Luis Mendo
Jan 3 at 1:05
add a comment |
1
There's a built in function to do the conversion: sub2ind
– Philip
Jan 3 at 1:02
@Philip Sure. I got into the habit of doing it manually because it's slightly faster :-)
– Luis Mendo
Jan 3 at 1:05
1
1
There's a built in function to do the conversion: sub2ind
– Philip
Jan 3 at 1:02
There's a built in function to do the conversion: sub2ind
– Philip
Jan 3 at 1:02
@Philip Sure. I got into the habit of doing it manually because it's slightly faster :-)
– Luis Mendo
Jan 3 at 1:05
@Philip Sure. I got into the habit of doing it manually because it's slightly faster :-)
– Luis Mendo
Jan 3 at 1:05
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53972495%2fmatlab-error-matrix-is-too-large-to-return-linear-indices%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
I have never found that error, probably because I never used matrices so large. The error is likely related with the fact that
doublevariables can exactly represent integers up to2^53-1only. The number of elements of your matrix does not exceed that limit, but it is near– Luis Mendo
Dec 29 '18 at 19:15
Maybe you can sidestep the issue (up to
2^53-1) by getting the row and column indices, and then computing the linear index from those manually:[ii,jj] = find(K); inds = ii + (jj-1)*size(K,1)– Luis Mendo
Dec 29 '18 at 19:19
@LuisMendo indeed this is a solution for me. Thanks. Maybe you can put your suggestion as an answer so I can accept it?
– colddie
Jan 2 at 15:18
Glad it worked. I rephrased a bit and added as an answer
– Luis Mendo
Jan 3 at 0:58