Numba: How to turn on/off just-in-time compilation programmatically (setting NUMBA_DISABLE_JIT environment...
I have written a script which is intended to test the efficacy of the @numba.jit decorators that I've added to several functions. I want to first exercise the annotated functions without the just-in-time compilation, then do it again with the @numba.jit decorators in play, in order to then compare the two results.
I've tried doing this by modifying the value of the NUMBA_DISABLE_JIT environment variable via os.environ, but I'm not sure yet that this is having the desired effect. For example:
# run first without and then with numba's just-in-time compilation
for flag in [1, 0]:
# enable/disable numba's just-in-time compilation
os.environ["NUMBA_DISABLE_JIT"] = str(flag)
# time an arbitrary number of iterations of the JIT decorated function
start = time.time()
for _ in range(1000):
expensive_function()
end = time.time()
# display the elapsed time
if flag == 0:
preposition = "with"
else:
preposition = "without"
print("Elapsed time " + preposition + " numba: {t}".format(t=(end - start)))
Is the setting of the environment variable DISABLE_NUMBA_JIT above actually having the effect of disabling/enabling the JIT compilation of all functions decorated with @numba.jit as I assume? If not then is there a better way to skin this cat?
Thanks in advance for any comments/suggestions.
python jit numba
asked Dec 27 '18 at 16:10
James Adams
3,261125283
add a comment |
I have written a script which is intended to test the efficacy of the @numba.jit decorators that I've added to several functions. I want to first exercise the annotated functions without the just-in-time compilation, then do it again with the @numba.jit decorators in play, in order to then compare the two results.
I've tried doing this by modifying the value of the NUMBA_DISABLE_JIT environment variable via os.environ, but I'm not sure yet that this is having the desired effect. For example:
# run first without and then with numba's just-in-time compilation
for flag in [1, 0]:
# enable/disable numba's just-in-time compilation
os.environ["NUMBA_DISABLE_JIT"] = str(flag)
# time an arbitrary number of iterations of the JIT decorated function
start = time.time()
for _ in range(1000):
expensive_function()
end = time.time()
# display the elapsed time
if flag == 0:
preposition = "with"
else:
preposition = "without"
print("Elapsed time " + preposition + " numba: {t}".format(t=(end - start)))
Is the setting of the environment variable DISABLE_NUMBA_JIT above actually having the effect of disabling/enabling the JIT compilation of all functions decorated with @numba.jit as I assume? If not then is there a better way to skin this cat?
Thanks in advance for any comments/suggestions.
python jit numba
asked Dec 27 '18 at 16:10
James Adams
3,261125283
add a comment |
I have written a script which is intended to test the efficacy of the @numba.jit decorators that I've added to several functions. I want to first exercise the annotated functions without the just-in-time compilation, then do it again with the @numba.jit decorators in play, in order to then compare the two results.
I've tried doing this by modifying the value of the NUMBA_DISABLE_JIT environment variable via os.environ, but I'm not sure yet that this is having the desired effect. For example:
# run first without and then with numba's just-in-time compilation
for flag in [1, 0]:
# enable/disable numba's just-in-time compilation
os.environ["NUMBA_DISABLE_JIT"] = str(flag)
# time an arbitrary number of iterations of the JIT decorated function
start = time.time()
for _ in range(1000):
expensive_function()
end = time.time()
# display the elapsed time
if flag == 0:
preposition = "with"
else:
preposition = "without"
print("Elapsed time " + preposition + " numba: {t}".format(t=(end - start)))
Is the setting of the environment variable DISABLE_NUMBA_JIT above actually having the effect of disabling/enabling the JIT compilation of all functions decorated with @numba.jit as I assume? If not then is there a better way to skin this cat?
Thanks in advance for any comments/suggestions.
python jit numba
asked Dec 27 '18 at 16:10
James Adams
3,261125283
I have written a script which is intended to test the efficacy of the @numba.jit decorators that I've added to several functions. I want to first exercise the annotated functions without the just-in-time compilation, then do it again with the @numba.jit decorators in play, in order to then compare the two results.
I've tried doing this by modifying the value of the NUMBA_DISABLE_JIT environment variable via os.environ, but I'm not sure yet that this is having the desired effect. For example:
# run first without and then with numba's just-in-time compilation
for flag in [1, 0]:
# enable/disable numba's just-in-time compilation
os.environ["NUMBA_DISABLE_JIT"] = str(flag)
# time an arbitrary number of iterations of the JIT decorated function
start = time.time()
for _ in range(1000):
expensive_function()
end = time.time()
# display the elapsed time
if flag == 0:
preposition = "with"
else:
preposition = "without"
print("Elapsed time " + preposition + " numba: {t}".format(t=(end - start)))
Is the setting of the environment variable DISABLE_NUMBA_JIT above actually having the effect of disabling/enabling the JIT compilation of all functions decorated with @numba.jit as I assume? If not then is there a better way to skin this cat?
Thanks in advance for any comments/suggestions.
python jit numba
python jit numba
asked Dec 27 '18 at 16:10
James Adams
3,261125283
asked Dec 27 '18 at 16:10
James Adams
3,261125283
asked Dec 27 '18 at 16:10
James Adams
3,261125283
asked Dec 27 '18 at 16:10
James Adams
3,261125283
asked Dec 27 '18 at 16:10
James Adams
3,261125283
3,261125283
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
I think that flag only has impact on the first call of expensive_function so isn't doing what you you'd like.
With numba you can always access the original python function with .py_func, so that could be a simpler way to do this
import numba
@numba.njit
def expensive_function(arr):
ans = 0.0
for a in arr:
ans += a
return ans
arr = np.random.randn(1_000_000)
In [21]: %timeit expensive_function(arr)
# 1.51 ms ± 24.4 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
%timeit expensive_function.py_func(arr)
# 134 ms ± 11 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
answered Dec 27 '18 at 17:20
chrisb
23.5k63337
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53947836%2fnumba-how-to-turn-on-off-just-in-time-compilation-programmatically-setting-num%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
I think that flag only has impact on the first call of expensive_function so isn't doing what you you'd like.
With numba you can always access the original python function with .py_func, so that could be a simpler way to do this
import numba
@numba.njit
def expensive_function(arr):
ans = 0.0
for a in arr:
ans += a
return ans
arr = np.random.randn(1_000_000)
In [21]: %timeit expensive_function(arr)
# 1.51 ms ± 24.4 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
%timeit expensive_function.py_func(arr)
# 134 ms ± 11 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
answered Dec 27 '18 at 17:20
chrisb
23.5k63337
add a comment |
I think that flag only has impact on the first call of expensive_function so isn't doing what you you'd like.
With numba you can always access the original python function with .py_func, so that could be a simpler way to do this
import numba
@numba.njit
def expensive_function(arr):
ans = 0.0
for a in arr:
ans += a
return ans
arr = np.random.randn(1_000_000)
In [21]: %timeit expensive_function(arr)
# 1.51 ms ± 24.4 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
%timeit expensive_function.py_func(arr)
# 134 ms ± 11 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
answered Dec 27 '18 at 17:20
chrisb
23.5k63337
add a comment |
I think that flag only has impact on the first call of expensive_function so isn't doing what you you'd like.
With numba you can always access the original python function with .py_func, so that could be a simpler way to do this
import numba
@numba.njit
def expensive_function(arr):
ans = 0.0
for a in arr:
ans += a
return ans
arr = np.random.randn(1_000_000)
In [21]: %timeit expensive_function(arr)
# 1.51 ms ± 24.4 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
%timeit expensive_function.py_func(arr)
# 134 ms ± 11 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
answered Dec 27 '18 at 17:20
chrisb
23.5k63337
I think that flag only has impact on the first call of expensive_function so isn't doing what you you'd like.
With numba you can always access the original python function with .py_func, so that could be a simpler way to do this
import numba
@numba.njit
def expensive_function(arr):
ans = 0.0
for a in arr:
ans += a
return ans
arr = np.random.randn(1_000_000)
In [21]: %timeit expensive_function(arr)
# 1.51 ms ± 24.4 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
%timeit expensive_function.py_func(arr)
# 134 ms ± 11 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
answered Dec 27 '18 at 17:20
chrisb
23.5k63337
answered Dec 27 '18 at 17:20
chrisb
23.5k63337
answered Dec 27 '18 at 17:20
chrisb
23.5k63337
answered Dec 27 '18 at 17:20
chrisb
23.5k63337
23.5k63337
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53947836%2fnumba-how-to-turn-on-off-just-in-time-compilation-programmatically-setting-num%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
