What should I do with a temp np ndarray created in the cython PyObject in __dealloc__
I'm trying to change some code and have to create a np.ndarray in a cython class
cdef class T:
def __cinit__(self, X):
# copy a memory space by the numpy way
cdef np.ndarray temp = np.ndarray(shape=X.shape, dtype=X.dtype)
self.X_fake = <DTYPE_t*> temp.data
def __dealloc__(self):
# should I free this?
free(self.X_fake)
I have found nothing in numpy or cython doc. I know if I use malloc to create a DTYPE_t* pointer, it's easy to work, I just wanna know that whether should I free the self.X_fake, or how can I dealloc the temp ndarray, will cython dealloc a numpy class by gc workflow?
python numpy cython
asked Nov 22 '18 at 3:57
SladarSladar
409
add a comment |
I'm trying to change some code and have to create a np.ndarray in a cython class
cdef class T:
def __cinit__(self, X):
# copy a memory space by the numpy way
cdef np.ndarray temp = np.ndarray(shape=X.shape, dtype=X.dtype)
self.X_fake = <DTYPE_t*> temp.data
def __dealloc__(self):
# should I free this?
free(self.X_fake)
I have found nothing in numpy or cython doc. I know if I use malloc to create a DTYPE_t* pointer, it's easy to work, I just wanna know that whether should I free the self.X_fake, or how can I dealloc the temp ndarray, will cython dealloc a numpy class by gc workflow?
python numpy cython
asked Nov 22 '18 at 3:57
SladarSladar
409
In you special case (when it compiles), I wouldn't touchX_fakeat all, because it is a dangling pointer as soon as the program is outside of__cinit__.
– ead
Nov 22 '18 at 5:23
Use ndarray class to create a memory space instead of malloc. Your answer is that the temp will dealloc after cinit, so the pointer will not hold a malloced memory space?
– Sladar
Nov 22 '18 at 5:32
I wouldn't call it an answer. But yes, this is what will happen. I also would not says that a pointer "holds memory" - it just points to an memory address, which might no longer be accessible and shouldn't be used.
– ead
Nov 22 '18 at 5:38
Yes I mean that pointer points to the memory block, but after cinit ends, that memory block will be free, which means it would be malloced to another variable.
– Sladar
Nov 22 '18 at 6:47
add a comment |
I'm trying to change some code and have to create a np.ndarray in a cython class
cdef class T:
def __cinit__(self, X):
# copy a memory space by the numpy way
cdef np.ndarray temp = np.ndarray(shape=X.shape, dtype=X.dtype)
self.X_fake = <DTYPE_t*> temp.data
def __dealloc__(self):
# should I free this?
free(self.X_fake)
I have found nothing in numpy or cython doc. I know if I use malloc to create a DTYPE_t* pointer, it's easy to work, I just wanna know that whether should I free the self.X_fake, or how can I dealloc the temp ndarray, will cython dealloc a numpy class by gc workflow?
python numpy cython
asked Nov 22 '18 at 3:57
SladarSladar
409
I'm trying to change some code and have to create a np.ndarray in a cython class
cdef class T:
def __cinit__(self, X):
# copy a memory space by the numpy way
cdef np.ndarray temp = np.ndarray(shape=X.shape, dtype=X.dtype)
self.X_fake = <DTYPE_t*> temp.data
def __dealloc__(self):
# should I free this?
free(self.X_fake)
I have found nothing in numpy or cython doc. I know if I use malloc to create a DTYPE_t* pointer, it's easy to work, I just wanna know that whether should I free the self.X_fake, or how can I dealloc the temp ndarray, will cython dealloc a numpy class by gc workflow?
python numpy cython
python numpy cython
asked Nov 22 '18 at 3:57
SladarSladar
409
asked Nov 22 '18 at 3:57
SladarSladar
409
asked Nov 22 '18 at 3:57
SladarSladar
409
asked Nov 22 '18 at 3:57
SladarSladar
409
asked Nov 22 '18 at 3:57
SladarSladar
409
409
In you special case (when it compiles), I wouldn't touchX_fakeat all, because it is a dangling pointer as soon as the program is outside of__cinit__.
– ead
Nov 22 '18 at 5:23
Use ndarray class to create a memory space instead of malloc. Your answer is that the temp will dealloc after cinit, so the pointer will not hold a malloced memory space?
– Sladar
Nov 22 '18 at 5:32
I wouldn't call it an answer. But yes, this is what will happen. I also would not says that a pointer "holds memory" - it just points to an memory address, which might no longer be accessible and shouldn't be used.
– ead
Nov 22 '18 at 5:38
Yes I mean that pointer points to the memory block, but after cinit ends, that memory block will be free, which means it would be malloced to another variable.
– Sladar
Nov 22 '18 at 6:47
add a comment |
In you special case (when it compiles), I wouldn't touchX_fakeat all, because it is a dangling pointer as soon as the program is outside of__cinit__.
– ead
Nov 22 '18 at 5:23
Use ndarray class to create a memory space instead of malloc. Your answer is that the temp will dealloc after cinit, so the pointer will not hold a malloced memory space?
– Sladar
Nov 22 '18 at 5:32
I wouldn't call it an answer. But yes, this is what will happen. I also would not says that a pointer "holds memory" - it just points to an memory address, which might no longer be accessible and shouldn't be used.
– ead
Nov 22 '18 at 5:38
Yes I mean that pointer points to the memory block, but after cinit ends, that memory block will be free, which means it would be malloced to another variable.
– Sladar
Nov 22 '18 at 6:47
In you special case (when it compiles), I wouldn't touch
X_fake at all, because it is a dangling pointer as soon as the program is outside of __cinit__.– ead
Nov 22 '18 at 5:23
In you special case (when it compiles), I wouldn't touch
X_fake at all, because it is a dangling pointer as soon as the program is outside of __cinit__.– ead
Nov 22 '18 at 5:23
Use ndarray class to create a memory space instead of malloc. Your answer is that the temp will dealloc after cinit, so the pointer will not hold a malloced memory space?
– Sladar
Nov 22 '18 at 5:32
Use ndarray class to create a memory space instead of malloc. Your answer is that the temp will dealloc after cinit, so the pointer will not hold a malloced memory space?
– Sladar
Nov 22 '18 at 5:32
I wouldn't call it an answer. But yes, this is what will happen. I also would not says that a pointer "holds memory" - it just points to an memory address, which might no longer be accessible and shouldn't be used.
– ead
Nov 22 '18 at 5:38
I wouldn't call it an answer. But yes, this is what will happen. I also would not says that a pointer "holds memory" - it just points to an memory address, which might no longer be accessible and shouldn't be used.
– ead
Nov 22 '18 at 5:38
Yes I mean that pointer points to the memory block, but after cinit ends, that memory block will be free, which means it would be malloced to another variable.
– Sladar
Nov 22 '18 at 6:47
Yes I mean that pointer points to the memory block, but after cinit ends, that memory block will be free, which means it would be malloced to another variable.
– Sladar
Nov 22 '18 at 6:47
add a comment |
1 Answer
1
active
oldest
votes
As @ead has pointed out, currently temp is an automatic variable that will go out of scope at the end of __cinit__. This means that self.X_fake will be pointing to nothing, and likely cause you problems. Here's the simplest way to fix your code:
import numpy as np
cimport numpy as np
ctypedef np.float64_t DTYPE_t
cdef class T:
cdef np.ndarray temp
cdef DTYPE_t* X_fake
def __cinit__(self, X):
# copy a memory space by the numpy way
self.temp = np.ndarray(shape=X.shape, dtype=X.dtype)
self.X_fake = <DTYPE_t*> self.temp.data
This way, temp will share the lifetime of the T instance to which it belongs. You don't need a __dealloc__. temp will get automatically cleaned up along with everything else in a given instance of T when it goes out of scope/gets garbage collected.
Also, you'll know if you have to free the memory belonging to a pointer, since in that case you'll have created it with malloc in the first place. If you didn't malloc it, don't free it.
answered Nov 22 '18 at 9:10
teltel
7,39621431
Thanks for your answer, I know the way to use malloc and free, just don't really know how to deal with ndarray.
– Sladar
Nov 22 '18 at 9:48
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53423660%2fwhat-should-i-do-with-a-temp-np-ndarray-created-in-the-cython-pyobject-in-deal%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
As @ead has pointed out, currently temp is an automatic variable that will go out of scope at the end of __cinit__. This means that self.X_fake will be pointing to nothing, and likely cause you problems. Here's the simplest way to fix your code:
import numpy as np
cimport numpy as np
ctypedef np.float64_t DTYPE_t
cdef class T:
cdef np.ndarray temp
cdef DTYPE_t* X_fake
def __cinit__(self, X):
# copy a memory space by the numpy way
self.temp = np.ndarray(shape=X.shape, dtype=X.dtype)
self.X_fake = <DTYPE_t*> self.temp.data
This way, temp will share the lifetime of the T instance to which it belongs. You don't need a __dealloc__. temp will get automatically cleaned up along with everything else in a given instance of T when it goes out of scope/gets garbage collected.
Also, you'll know if you have to free the memory belonging to a pointer, since in that case you'll have created it with malloc in the first place. If you didn't malloc it, don't free it.
answered Nov 22 '18 at 9:10
teltel
7,39621431
Thanks for your answer, I know the way to use malloc and free, just don't really know how to deal with ndarray.
– Sladar
Nov 22 '18 at 9:48
add a comment |
As @ead has pointed out, currently temp is an automatic variable that will go out of scope at the end of __cinit__. This means that self.X_fake will be pointing to nothing, and likely cause you problems. Here's the simplest way to fix your code:
import numpy as np
cimport numpy as np
ctypedef np.float64_t DTYPE_t
cdef class T:
cdef np.ndarray temp
cdef DTYPE_t* X_fake
def __cinit__(self, X):
# copy a memory space by the numpy way
self.temp = np.ndarray(shape=X.shape, dtype=X.dtype)
self.X_fake = <DTYPE_t*> self.temp.data
This way, temp will share the lifetime of the T instance to which it belongs. You don't need a __dealloc__. temp will get automatically cleaned up along with everything else in a given instance of T when it goes out of scope/gets garbage collected.
Also, you'll know if you have to free the memory belonging to a pointer, since in that case you'll have created it with malloc in the first place. If you didn't malloc it, don't free it.
answered Nov 22 '18 at 9:10
teltel
7,39621431
Thanks for your answer, I know the way to use malloc and free, just don't really know how to deal with ndarray.
– Sladar
Nov 22 '18 at 9:48
add a comment |
As @ead has pointed out, currently temp is an automatic variable that will go out of scope at the end of __cinit__. This means that self.X_fake will be pointing to nothing, and likely cause you problems. Here's the simplest way to fix your code:
import numpy as np
cimport numpy as np
ctypedef np.float64_t DTYPE_t
cdef class T:
cdef np.ndarray temp
cdef DTYPE_t* X_fake
def __cinit__(self, X):
# copy a memory space by the numpy way
self.temp = np.ndarray(shape=X.shape, dtype=X.dtype)
self.X_fake = <DTYPE_t*> self.temp.data
This way, temp will share the lifetime of the T instance to which it belongs. You don't need a __dealloc__. temp will get automatically cleaned up along with everything else in a given instance of T when it goes out of scope/gets garbage collected.
Also, you'll know if you have to free the memory belonging to a pointer, since in that case you'll have created it with malloc in the first place. If you didn't malloc it, don't free it.
answered Nov 22 '18 at 9:10
teltel
7,39621431
As @ead has pointed out, currently temp is an automatic variable that will go out of scope at the end of __cinit__. This means that self.X_fake will be pointing to nothing, and likely cause you problems. Here's the simplest way to fix your code:
import numpy as np
cimport numpy as np
ctypedef np.float64_t DTYPE_t
cdef class T:
cdef np.ndarray temp
cdef DTYPE_t* X_fake
def __cinit__(self, X):
# copy a memory space by the numpy way
self.temp = np.ndarray(shape=X.shape, dtype=X.dtype)
self.X_fake = <DTYPE_t*> self.temp.data
This way, temp will share the lifetime of the T instance to which it belongs. You don't need a __dealloc__. temp will get automatically cleaned up along with everything else in a given instance of T when it goes out of scope/gets garbage collected.
Also, you'll know if you have to free the memory belonging to a pointer, since in that case you'll have created it with malloc in the first place. If you didn't malloc it, don't free it.
answered Nov 22 '18 at 9:10
teltel
7,39621431
answered Nov 22 '18 at 9:10
teltel
7,39621431
answered Nov 22 '18 at 9:10
teltel
7,39621431
answered Nov 22 '18 at 9:10
teltel
7,39621431
7,39621431
Thanks for your answer, I know the way to use malloc and free, just don't really know how to deal with ndarray.
– Sladar
Nov 22 '18 at 9:48
add a comment |
Thanks for your answer, I know the way to use malloc and free, just don't really know how to deal with ndarray.
– Sladar
Nov 22 '18 at 9:48
Thanks for your answer, I know the way to use malloc and free, just don't really know how to deal with ndarray.
– Sladar
Nov 22 '18 at 9:48
Thanks for your answer, I know the way to use malloc and free, just don't really know how to deal with ndarray.
– Sladar
Nov 22 '18 at 9:48
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53423660%2fwhat-should-i-do-with-a-temp-np-ndarray-created-in-the-cython-pyobject-in-deal%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
In you special case (when it compiles), I wouldn't touch
X_fakeat all, because it is a dangling pointer as soon as the program is outside of__cinit__.– ead
Nov 22 '18 at 5:23
Use ndarray class to create a memory space instead of malloc. Your answer is that the temp will dealloc after cinit, so the pointer will not hold a malloced memory space?
– Sladar
Nov 22 '18 at 5:32
I wouldn't call it an answer. But yes, this is what will happen. I also would not says that a pointer "holds memory" - it just points to an memory address, which might no longer be accessible and shouldn't be used.
– ead
Nov 22 '18 at 5:38
Yes I mean that pointer points to the memory block, but after cinit ends, that memory block will be free, which means it would be malloced to another variable.
– Sladar
Nov 22 '18 at 6:47