Does Mathematica reuse previous computations?
$begingroup$
I am doing an analysis of experimental results in which I need to repeat the same GaussianFilter
hundred of times on different data. As explained in the documentation, GaussianFilter
just convolves the data with a Gaussian kernel. Does it recompute the kernel every time I call the function, or will it somehow preserve and reuse the previous kernel? Would it be more efficient computationally for me to precompute the kernel (which I could do easily by applying GaussianFilter
to a KroneckerDelta
array), then do hundreds of ListConvolve
s instead of hundreds of GaussianFilter
s?
numerics convolution
$endgroup$
|
show 2 more comments
$begingroup$
I am doing an analysis of experimental results in which I need to repeat the same GaussianFilter
hundred of times on different data. As explained in the documentation, GaussianFilter
just convolves the data with a Gaussian kernel. Does it recompute the kernel every time I call the function, or will it somehow preserve and reuse the previous kernel? Would it be more efficient computationally for me to precompute the kernel (which I could do easily by applying GaussianFilter
to a KroneckerDelta
array), then do hundreds of ListConvolve
s instead of hundreds of GaussianFilter
s?
numerics convolution
$endgroup$
2
$begingroup$
My first guess is thatGaussianFilter
employs FFT (FFT, multiply, inverse FFT). In this case, it would be a very bad idea to replace it by a convolution unless the convolution is also implemented by FFT: The naive way of convolution would require $O(n^2)$ flops for a vector of length $n$ while the FFT method would need only $O(n , log(n))$ flops.
$endgroup$
– Henrik Schumacher
yesterday
2
$begingroup$
Best way would be to just try it and compare theRepeatedTimings
.
$endgroup$
– Henrik Schumacher
yesterday
3
$begingroup$
@HenrikSchumacher I'm pretty sure ListConvolve also uses FFT (based on its performance).
$endgroup$
– Szabolcs
yesterday
1
$begingroup$
ListConvolve does indeed use FFT.
$endgroup$
– Leon Avery
yesterday
4
$begingroup$
Indeed it does.
$endgroup$
– J. M. is slightly pensive♦
yesterday
|
show 2 more comments
$begingroup$
I am doing an analysis of experimental results in which I need to repeat the same GaussianFilter
hundred of times on different data. As explained in the documentation, GaussianFilter
just convolves the data with a Gaussian kernel. Does it recompute the kernel every time I call the function, or will it somehow preserve and reuse the previous kernel? Would it be more efficient computationally for me to precompute the kernel (which I could do easily by applying GaussianFilter
to a KroneckerDelta
array), then do hundreds of ListConvolve
s instead of hundreds of GaussianFilter
s?
numerics convolution
$endgroup$
I am doing an analysis of experimental results in which I need to repeat the same GaussianFilter
hundred of times on different data. As explained in the documentation, GaussianFilter
just convolves the data with a Gaussian kernel. Does it recompute the kernel every time I call the function, or will it somehow preserve and reuse the previous kernel? Would it be more efficient computationally for me to precompute the kernel (which I could do easily by applying GaussianFilter
to a KroneckerDelta
array), then do hundreds of ListConvolve
s instead of hundreds of GaussianFilter
s?
numerics convolution
numerics convolution
asked yesterday
Leon AveryLeon Avery
695318
695318
2
$begingroup$
My first guess is thatGaussianFilter
employs FFT (FFT, multiply, inverse FFT). In this case, it would be a very bad idea to replace it by a convolution unless the convolution is also implemented by FFT: The naive way of convolution would require $O(n^2)$ flops for a vector of length $n$ while the FFT method would need only $O(n , log(n))$ flops.
$endgroup$
– Henrik Schumacher
yesterday
2
$begingroup$
Best way would be to just try it and compare theRepeatedTimings
.
$endgroup$
– Henrik Schumacher
yesterday
3
$begingroup$
@HenrikSchumacher I'm pretty sure ListConvolve also uses FFT (based on its performance).
$endgroup$
– Szabolcs
yesterday
1
$begingroup$
ListConvolve does indeed use FFT.
$endgroup$
– Leon Avery
yesterday
4
$begingroup$
Indeed it does.
$endgroup$
– J. M. is slightly pensive♦
yesterday
|
show 2 more comments
2
$begingroup$
My first guess is thatGaussianFilter
employs FFT (FFT, multiply, inverse FFT). In this case, it would be a very bad idea to replace it by a convolution unless the convolution is also implemented by FFT: The naive way of convolution would require $O(n^2)$ flops for a vector of length $n$ while the FFT method would need only $O(n , log(n))$ flops.
$endgroup$
– Henrik Schumacher
yesterday
2
$begingroup$
Best way would be to just try it and compare theRepeatedTimings
.
$endgroup$
– Henrik Schumacher
yesterday
3
$begingroup$
@HenrikSchumacher I'm pretty sure ListConvolve also uses FFT (based on its performance).
$endgroup$
– Szabolcs
yesterday
1
$begingroup$
ListConvolve does indeed use FFT.
$endgroup$
– Leon Avery
yesterday
4
$begingroup$
Indeed it does.
$endgroup$
– J. M. is slightly pensive♦
yesterday
2
2
$begingroup$
My first guess is that
GaussianFilter
employs FFT (FFT, multiply, inverse FFT). In this case, it would be a very bad idea to replace it by a convolution unless the convolution is also implemented by FFT: The naive way of convolution would require $O(n^2)$ flops for a vector of length $n$ while the FFT method would need only $O(n , log(n))$ flops.$endgroup$
– Henrik Schumacher
yesterday
$begingroup$
My first guess is that
GaussianFilter
employs FFT (FFT, multiply, inverse FFT). In this case, it would be a very bad idea to replace it by a convolution unless the convolution is also implemented by FFT: The naive way of convolution would require $O(n^2)$ flops for a vector of length $n$ while the FFT method would need only $O(n , log(n))$ flops.$endgroup$
– Henrik Schumacher
yesterday
2
2
$begingroup$
Best way would be to just try it and compare the
RepeatedTimings
.$endgroup$
– Henrik Schumacher
yesterday
$begingroup$
Best way would be to just try it and compare the
RepeatedTimings
.$endgroup$
– Henrik Schumacher
yesterday
3
3
$begingroup$
@HenrikSchumacher I'm pretty sure ListConvolve also uses FFT (based on its performance).
$endgroup$
– Szabolcs
yesterday
$begingroup$
@HenrikSchumacher I'm pretty sure ListConvolve also uses FFT (based on its performance).
$endgroup$
– Szabolcs
yesterday
1
1
$begingroup$
ListConvolve does indeed use FFT.
$endgroup$
– Leon Avery
yesterday
$begingroup$
ListConvolve does indeed use FFT.
$endgroup$
– Leon Avery
yesterday
4
4
$begingroup$
Indeed it does.
$endgroup$
– J. M. is slightly pensive♦
yesterday
$begingroup$
Indeed it does.
$endgroup$
– J. M. is slightly pensive♦
yesterday
|
show 2 more comments
2 Answers
2
active
oldest
votes
$begingroup$
Here I implemented three different versions of Gaussian filtering (for periodic data). It took me a while to adjust the constants and still some of them might be wrong.
Prepare the Gaussian kernel
n = 200000;
σ = .1;
t = Subdivide[-1. Pi, 1. Pi, n - 1];
ker = 1/Sqrt[2 Pi]/ σ Exp[-(t/σ)^2/2];
ker = Join[ker[[Quotient[n,2] + 1 ;;]], ker[[;; Quotient[n,2]]]];
Generate noisy function
u = Sin[t] + Cos[2 t] + 1.5 Cos[3 t] + .5 RandomReal[{-1, 1}, Length[t]];
The three methods with their timings:
kerhat = 2 Pi/Sqrt[N@n] Fourier[ker];
vConvolve = (2. Pi/n) ListConvolve[ker, u, {-1, -1}]; // RepeatedTiming // First
vFFT = Re[Fourier[InverseFourier[u] kerhat]]; // RepeatedTiming // First
vFilter = GaussianFilter[u, 1./(Pi) σ n, Padding -> "Periodic"]; // RepeatedTiming // First
0.0043
0.0057
0.054
ListLinePlot[{u, vFFT, vFilter, vConvolve}]
From further experiments with different values for n
, GaussianFilter
seems to be slower by a factor 10-20 over a wide range of n
(from n = 1000
to n = 1000000
). So it seems that it does use some FFT-based method (because it has the same speed asymptotics) but maybe some crucial part of the algorithm is not compiled (the factor 10 is somewhat an indicator for that) or does not use the fastest FFT implementation possible. A bit weird.
So, to my own surprise, your idea of computing the kernel once does help but for quite unexpected reasons.
$endgroup$
1
$begingroup$
IIRC, it's using a discrete Gaussian kernel, which involves non-compilable modified Bessel functions, so that might be the reason for the added computational effort you observe.
$endgroup$
– J. M. is slightly pensive♦
yesterday
$begingroup$
Yes, that is one reason I proposed using GaussianFilter itself to generate the kernel.
$endgroup$
– Leon Avery
yesterday
$begingroup$
If I addMethod -> "Gaussian"
to theGaussianFilter
call, it's about as fast as the other two
$endgroup$
– Niki Estner
12 hours ago
2
$begingroup$
@NikiEstner Ah, that's good to know! Somewhat unexpected that we have to call out twice for a"Gaussian"
in order to get one, isn't it?
$endgroup$
– Henrik Schumacher
11 hours ago
add a comment |
$begingroup$
It's hard to know for sure, but one way to test for caching is to apply a single command to lots of data sets, or to apply the command separately to each set. For instance:
n = 5000;
data = RandomReal[{-1, 1}, {n, 10000}];
GaussianFilter[#, 100] & /@ data; // AbsoluteTiming
Do[GaussianFilter[data[[i]], 100], {i, n}] // AbsoluteTiming
Do[GaussianFilter[data[[i]], 100 + RandomInteger[{-15, 15}]], {i, n}] // AbsoluteTiming
The second line generates 5000 different sets of data, each 10000 length. The third applies one Gaussian filter to all the data sets. The third line applies a separate GaussianFilter to each set. The final line forces the GaussianFilter to recompute its kernel. The timings are pretty much the same. This suggests that whatever is happening, the time needed to calculate the Gaussian filter parameters is pretty negligeable.
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "387"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmathematica.stackexchange.com%2fquestions%2f193328%2fdoes-mathematica-reuse-previous-computations%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
Here I implemented three different versions of Gaussian filtering (for periodic data). It took me a while to adjust the constants and still some of them might be wrong.
Prepare the Gaussian kernel
n = 200000;
σ = .1;
t = Subdivide[-1. Pi, 1. Pi, n - 1];
ker = 1/Sqrt[2 Pi]/ σ Exp[-(t/σ)^2/2];
ker = Join[ker[[Quotient[n,2] + 1 ;;]], ker[[;; Quotient[n,2]]]];
Generate noisy function
u = Sin[t] + Cos[2 t] + 1.5 Cos[3 t] + .5 RandomReal[{-1, 1}, Length[t]];
The three methods with their timings:
kerhat = 2 Pi/Sqrt[N@n] Fourier[ker];
vConvolve = (2. Pi/n) ListConvolve[ker, u, {-1, -1}]; // RepeatedTiming // First
vFFT = Re[Fourier[InverseFourier[u] kerhat]]; // RepeatedTiming // First
vFilter = GaussianFilter[u, 1./(Pi) σ n, Padding -> "Periodic"]; // RepeatedTiming // First
0.0043
0.0057
0.054
ListLinePlot[{u, vFFT, vFilter, vConvolve}]
From further experiments with different values for n
, GaussianFilter
seems to be slower by a factor 10-20 over a wide range of n
(from n = 1000
to n = 1000000
). So it seems that it does use some FFT-based method (because it has the same speed asymptotics) but maybe some crucial part of the algorithm is not compiled (the factor 10 is somewhat an indicator for that) or does not use the fastest FFT implementation possible. A bit weird.
So, to my own surprise, your idea of computing the kernel once does help but for quite unexpected reasons.
$endgroup$
1
$begingroup$
IIRC, it's using a discrete Gaussian kernel, which involves non-compilable modified Bessel functions, so that might be the reason for the added computational effort you observe.
$endgroup$
– J. M. is slightly pensive♦
yesterday
$begingroup$
Yes, that is one reason I proposed using GaussianFilter itself to generate the kernel.
$endgroup$
– Leon Avery
yesterday
$begingroup$
If I addMethod -> "Gaussian"
to theGaussianFilter
call, it's about as fast as the other two
$endgroup$
– Niki Estner
12 hours ago
2
$begingroup$
@NikiEstner Ah, that's good to know! Somewhat unexpected that we have to call out twice for a"Gaussian"
in order to get one, isn't it?
$endgroup$
– Henrik Schumacher
11 hours ago
add a comment |
$begingroup$
Here I implemented three different versions of Gaussian filtering (for periodic data). It took me a while to adjust the constants and still some of them might be wrong.
Prepare the Gaussian kernel
n = 200000;
σ = .1;
t = Subdivide[-1. Pi, 1. Pi, n - 1];
ker = 1/Sqrt[2 Pi]/ σ Exp[-(t/σ)^2/2];
ker = Join[ker[[Quotient[n,2] + 1 ;;]], ker[[;; Quotient[n,2]]]];
Generate noisy function
u = Sin[t] + Cos[2 t] + 1.5 Cos[3 t] + .5 RandomReal[{-1, 1}, Length[t]];
The three methods with their timings:
kerhat = 2 Pi/Sqrt[N@n] Fourier[ker];
vConvolve = (2. Pi/n) ListConvolve[ker, u, {-1, -1}]; // RepeatedTiming // First
vFFT = Re[Fourier[InverseFourier[u] kerhat]]; // RepeatedTiming // First
vFilter = GaussianFilter[u, 1./(Pi) σ n, Padding -> "Periodic"]; // RepeatedTiming // First
0.0043
0.0057
0.054
ListLinePlot[{u, vFFT, vFilter, vConvolve}]
From further experiments with different values for n
, GaussianFilter
seems to be slower by a factor 10-20 over a wide range of n
(from n = 1000
to n = 1000000
). So it seems that it does use some FFT-based method (because it has the same speed asymptotics) but maybe some crucial part of the algorithm is not compiled (the factor 10 is somewhat an indicator for that) or does not use the fastest FFT implementation possible. A bit weird.
So, to my own surprise, your idea of computing the kernel once does help but for quite unexpected reasons.
$endgroup$
1
$begingroup$
IIRC, it's using a discrete Gaussian kernel, which involves non-compilable modified Bessel functions, so that might be the reason for the added computational effort you observe.
$endgroup$
– J. M. is slightly pensive♦
yesterday
$begingroup$
Yes, that is one reason I proposed using GaussianFilter itself to generate the kernel.
$endgroup$
– Leon Avery
yesterday
$begingroup$
If I addMethod -> "Gaussian"
to theGaussianFilter
call, it's about as fast as the other two
$endgroup$
– Niki Estner
12 hours ago
2
$begingroup$
@NikiEstner Ah, that's good to know! Somewhat unexpected that we have to call out twice for a"Gaussian"
in order to get one, isn't it?
$endgroup$
– Henrik Schumacher
11 hours ago
add a comment |
$begingroup$
Here I implemented three different versions of Gaussian filtering (for periodic data). It took me a while to adjust the constants and still some of them might be wrong.
Prepare the Gaussian kernel
n = 200000;
σ = .1;
t = Subdivide[-1. Pi, 1. Pi, n - 1];
ker = 1/Sqrt[2 Pi]/ σ Exp[-(t/σ)^2/2];
ker = Join[ker[[Quotient[n,2] + 1 ;;]], ker[[;; Quotient[n,2]]]];
Generate noisy function
u = Sin[t] + Cos[2 t] + 1.5 Cos[3 t] + .5 RandomReal[{-1, 1}, Length[t]];
The three methods with their timings:
kerhat = 2 Pi/Sqrt[N@n] Fourier[ker];
vConvolve = (2. Pi/n) ListConvolve[ker, u, {-1, -1}]; // RepeatedTiming // First
vFFT = Re[Fourier[InverseFourier[u] kerhat]]; // RepeatedTiming // First
vFilter = GaussianFilter[u, 1./(Pi) σ n, Padding -> "Periodic"]; // RepeatedTiming // First
0.0043
0.0057
0.054
ListLinePlot[{u, vFFT, vFilter, vConvolve}]
From further experiments with different values for n
, GaussianFilter
seems to be slower by a factor 10-20 over a wide range of n
(from n = 1000
to n = 1000000
). So it seems that it does use some FFT-based method (because it has the same speed asymptotics) but maybe some crucial part of the algorithm is not compiled (the factor 10 is somewhat an indicator for that) or does not use the fastest FFT implementation possible. A bit weird.
So, to my own surprise, your idea of computing the kernel once does help but for quite unexpected reasons.
$endgroup$
Here I implemented three different versions of Gaussian filtering (for periodic data). It took me a while to adjust the constants and still some of them might be wrong.
Prepare the Gaussian kernel
n = 200000;
σ = .1;
t = Subdivide[-1. Pi, 1. Pi, n - 1];
ker = 1/Sqrt[2 Pi]/ σ Exp[-(t/σ)^2/2];
ker = Join[ker[[Quotient[n,2] + 1 ;;]], ker[[;; Quotient[n,2]]]];
Generate noisy function
u = Sin[t] + Cos[2 t] + 1.5 Cos[3 t] + .5 RandomReal[{-1, 1}, Length[t]];
The three methods with their timings:
kerhat = 2 Pi/Sqrt[N@n] Fourier[ker];
vConvolve = (2. Pi/n) ListConvolve[ker, u, {-1, -1}]; // RepeatedTiming // First
vFFT = Re[Fourier[InverseFourier[u] kerhat]]; // RepeatedTiming // First
vFilter = GaussianFilter[u, 1./(Pi) σ n, Padding -> "Periodic"]; // RepeatedTiming // First
0.0043
0.0057
0.054
ListLinePlot[{u, vFFT, vFilter, vConvolve}]
From further experiments with different values for n
, GaussianFilter
seems to be slower by a factor 10-20 over a wide range of n
(from n = 1000
to n = 1000000
). So it seems that it does use some FFT-based method (because it has the same speed asymptotics) but maybe some crucial part of the algorithm is not compiled (the factor 10 is somewhat an indicator for that) or does not use the fastest FFT implementation possible. A bit weird.
So, to my own surprise, your idea of computing the kernel once does help but for quite unexpected reasons.
edited yesterday
answered yesterday
Henrik SchumacherHenrik Schumacher
56.7k577157
56.7k577157
1
$begingroup$
IIRC, it's using a discrete Gaussian kernel, which involves non-compilable modified Bessel functions, so that might be the reason for the added computational effort you observe.
$endgroup$
– J. M. is slightly pensive♦
yesterday
$begingroup$
Yes, that is one reason I proposed using GaussianFilter itself to generate the kernel.
$endgroup$
– Leon Avery
yesterday
$begingroup$
If I addMethod -> "Gaussian"
to theGaussianFilter
call, it's about as fast as the other two
$endgroup$
– Niki Estner
12 hours ago
2
$begingroup$
@NikiEstner Ah, that's good to know! Somewhat unexpected that we have to call out twice for a"Gaussian"
in order to get one, isn't it?
$endgroup$
– Henrik Schumacher
11 hours ago
add a comment |
1
$begingroup$
IIRC, it's using a discrete Gaussian kernel, which involves non-compilable modified Bessel functions, so that might be the reason for the added computational effort you observe.
$endgroup$
– J. M. is slightly pensive♦
yesterday
$begingroup$
Yes, that is one reason I proposed using GaussianFilter itself to generate the kernel.
$endgroup$
– Leon Avery
yesterday
$begingroup$
If I addMethod -> "Gaussian"
to theGaussianFilter
call, it's about as fast as the other two
$endgroup$
– Niki Estner
12 hours ago
2
$begingroup$
@NikiEstner Ah, that's good to know! Somewhat unexpected that we have to call out twice for a"Gaussian"
in order to get one, isn't it?
$endgroup$
– Henrik Schumacher
11 hours ago
1
1
$begingroup$
IIRC, it's using a discrete Gaussian kernel, which involves non-compilable modified Bessel functions, so that might be the reason for the added computational effort you observe.
$endgroup$
– J. M. is slightly pensive♦
yesterday
$begingroup$
IIRC, it's using a discrete Gaussian kernel, which involves non-compilable modified Bessel functions, so that might be the reason for the added computational effort you observe.
$endgroup$
– J. M. is slightly pensive♦
yesterday
$begingroup$
Yes, that is one reason I proposed using GaussianFilter itself to generate the kernel.
$endgroup$
– Leon Avery
yesterday
$begingroup$
Yes, that is one reason I proposed using GaussianFilter itself to generate the kernel.
$endgroup$
– Leon Avery
yesterday
$begingroup$
If I add
Method -> "Gaussian"
to the GaussianFilter
call, it's about as fast as the other two$endgroup$
– Niki Estner
12 hours ago
$begingroup$
If I add
Method -> "Gaussian"
to the GaussianFilter
call, it's about as fast as the other two$endgroup$
– Niki Estner
12 hours ago
2
2
$begingroup$
@NikiEstner Ah, that's good to know! Somewhat unexpected that we have to call out twice for a
"Gaussian"
in order to get one, isn't it?$endgroup$
– Henrik Schumacher
11 hours ago
$begingroup$
@NikiEstner Ah, that's good to know! Somewhat unexpected that we have to call out twice for a
"Gaussian"
in order to get one, isn't it?$endgroup$
– Henrik Schumacher
11 hours ago
add a comment |
$begingroup$
It's hard to know for sure, but one way to test for caching is to apply a single command to lots of data sets, or to apply the command separately to each set. For instance:
n = 5000;
data = RandomReal[{-1, 1}, {n, 10000}];
GaussianFilter[#, 100] & /@ data; // AbsoluteTiming
Do[GaussianFilter[data[[i]], 100], {i, n}] // AbsoluteTiming
Do[GaussianFilter[data[[i]], 100 + RandomInteger[{-15, 15}]], {i, n}] // AbsoluteTiming
The second line generates 5000 different sets of data, each 10000 length. The third applies one Gaussian filter to all the data sets. The third line applies a separate GaussianFilter to each set. The final line forces the GaussianFilter to recompute its kernel. The timings are pretty much the same. This suggests that whatever is happening, the time needed to calculate the Gaussian filter parameters is pretty negligeable.
$endgroup$
add a comment |
$begingroup$
It's hard to know for sure, but one way to test for caching is to apply a single command to lots of data sets, or to apply the command separately to each set. For instance:
n = 5000;
data = RandomReal[{-1, 1}, {n, 10000}];
GaussianFilter[#, 100] & /@ data; // AbsoluteTiming
Do[GaussianFilter[data[[i]], 100], {i, n}] // AbsoluteTiming
Do[GaussianFilter[data[[i]], 100 + RandomInteger[{-15, 15}]], {i, n}] // AbsoluteTiming
The second line generates 5000 different sets of data, each 10000 length. The third applies one Gaussian filter to all the data sets. The third line applies a separate GaussianFilter to each set. The final line forces the GaussianFilter to recompute its kernel. The timings are pretty much the same. This suggests that whatever is happening, the time needed to calculate the Gaussian filter parameters is pretty negligeable.
$endgroup$
add a comment |
$begingroup$
It's hard to know for sure, but one way to test for caching is to apply a single command to lots of data sets, or to apply the command separately to each set. For instance:
n = 5000;
data = RandomReal[{-1, 1}, {n, 10000}];
GaussianFilter[#, 100] & /@ data; // AbsoluteTiming
Do[GaussianFilter[data[[i]], 100], {i, n}] // AbsoluteTiming
Do[GaussianFilter[data[[i]], 100 + RandomInteger[{-15, 15}]], {i, n}] // AbsoluteTiming
The second line generates 5000 different sets of data, each 10000 length. The third applies one Gaussian filter to all the data sets. The third line applies a separate GaussianFilter to each set. The final line forces the GaussianFilter to recompute its kernel. The timings are pretty much the same. This suggests that whatever is happening, the time needed to calculate the Gaussian filter parameters is pretty negligeable.
$endgroup$
It's hard to know for sure, but one way to test for caching is to apply a single command to lots of data sets, or to apply the command separately to each set. For instance:
n = 5000;
data = RandomReal[{-1, 1}, {n, 10000}];
GaussianFilter[#, 100] & /@ data; // AbsoluteTiming
Do[GaussianFilter[data[[i]], 100], {i, n}] // AbsoluteTiming
Do[GaussianFilter[data[[i]], 100 + RandomInteger[{-15, 15}]], {i, n}] // AbsoluteTiming
The second line generates 5000 different sets of data, each 10000 length. The third applies one Gaussian filter to all the data sets. The third line applies a separate GaussianFilter to each set. The final line forces the GaussianFilter to recompute its kernel. The timings are pretty much the same. This suggests that whatever is happening, the time needed to calculate the Gaussian filter parameters is pretty negligeable.
answered yesterday
bill sbill s
54.3k377156
54.3k377156
add a comment |
add a comment |
Thanks for contributing an answer to Mathematica Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmathematica.stackexchange.com%2fquestions%2f193328%2fdoes-mathematica-reuse-previous-computations%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
2
$begingroup$
My first guess is that
GaussianFilter
employs FFT (FFT, multiply, inverse FFT). In this case, it would be a very bad idea to replace it by a convolution unless the convolution is also implemented by FFT: The naive way of convolution would require $O(n^2)$ flops for a vector of length $n$ while the FFT method would need only $O(n , log(n))$ flops.$endgroup$
– Henrik Schumacher
yesterday
2
$begingroup$
Best way would be to just try it and compare the
RepeatedTimings
.$endgroup$
– Henrik Schumacher
yesterday
3
$begingroup$
@HenrikSchumacher I'm pretty sure ListConvolve also uses FFT (based on its performance).
$endgroup$
– Szabolcs
yesterday
1
$begingroup$
ListConvolve does indeed use FFT.
$endgroup$
– Leon Avery
yesterday
4
$begingroup$
Indeed it does.
$endgroup$
– J. M. is slightly pensive♦
yesterday