Python Pandas - Understanding inplace=True

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ height:90px;width:728px;box-sizing:border-box;
}

In the pandas library many times there is an option to change the object inplace such as with the following statement...

df.dropna(axis='index', how='all', inplace=True)

I am curious what is being returned as well as how the object is handled when inplace=True is passed vs. when inplace=False.

Are all operations modifying self when inplace=True? And when inplace=False is a new object created immediately such as new_df = self and then new_df is returned?

asked May 10 '17 at 13:08

Aran Freel

65521020

5

Yes, inplace=True returns None inplace=False returns a copy of the object with the operation performed. The docs are pretty clear on this, is there something that is confusing with a specific part? Spefically If True, do operation inplace and return None.

– EdChum
May 10 '17 at 13:09

1

Correctly said..

– Aditya
May 10 '17 at 13:10

I am subclassing the DataFrame object and with an operation such as merge it doesn't seem possible to do it inplace... self = self.merge(new_df, how='left', on='column2' I am not sure that it is possible to reassign self

– Aran Freel
May 10 '17 at 13:12

1

You're correct that DataFrame.merge has no inplace argument. It returns a DataFrame, so no issue reassigning.

– JAV
May 11 '17 at 5:01

Can someone also highlight the advantages of using it in terms of resource consumption?

– markroxor
Apr 3 at 12:03

add a comment |

In the pandas library many times there is an option to change the object inplace such as with the following statement...

df.dropna(axis='index', how='all', inplace=True)

I am curious what is being returned as well as how the object is handled when inplace=True is passed vs. when inplace=False.

Are all operations modifying self when inplace=True? And when inplace=False is a new object created immediately such as new_df = self and then new_df is returned?

asked May 10 '17 at 13:08

Aran Freel

65521020

5

Yes, inplace=True returns None inplace=False returns a copy of the object with the operation performed. The docs are pretty clear on this, is there something that is confusing with a specific part? Spefically If True, do operation inplace and return None.

– EdChum
May 10 '17 at 13:09

1

Correctly said..

– Aditya
May 10 '17 at 13:10

I am subclassing the DataFrame object and with an operation such as merge it doesn't seem possible to do it inplace... self = self.merge(new_df, how='left', on='column2' I am not sure that it is possible to reassign self

– Aran Freel
May 10 '17 at 13:12

1

You're correct that DataFrame.merge has no inplace argument. It returns a DataFrame, so no issue reassigning.

– JAV
May 11 '17 at 5:01

Can someone also highlight the advantages of using it in terms of resource consumption?

– markroxor
Apr 3 at 12:03

add a comment |

In the pandas library many times there is an option to change the object inplace such as with the following statement...

df.dropna(axis='index', how='all', inplace=True)

I am curious what is being returned as well as how the object is handled when inplace=True is passed vs. when inplace=False.

Are all operations modifying self when inplace=True? And when inplace=False is a new object created immediately such as new_df = self and then new_df is returned?

asked May 10 '17 at 13:08

Aran Freel

65521020

In the pandas library many times there is an option to change the object inplace such as with the following statement...

df.dropna(axis='index', how='all', inplace=True)

I am curious what is being returned as well as how the object is handled when inplace=True is passed vs. when inplace=False.

Are all operations modifying self when inplace=True? And when inplace=False is a new object created immediately such as new_df = self and then new_df is returned?

python pandas in-place

asked May 10 '17 at 13:08

Aran Freel

65521020

asked May 10 '17 at 13:08

Aran Freel

65521020

asked May 10 '17 at 13:08

Aran Freel

65521020

asked May 10 '17 at 13:08

Aran Freel

65521020

asked May 10 '17 at 13:08

Aran Freel

65521020

5

Yes, inplace=True returns None inplace=False returns a copy of the object with the operation performed. The docs are pretty clear on this, is there something that is confusing with a specific part? Spefically If True, do operation inplace and return None.

– EdChum
May 10 '17 at 13:09

1

Correctly said..

– Aditya
May 10 '17 at 13:10

I am subclassing the DataFrame object and with an operation such as merge it doesn't seem possible to do it inplace... self = self.merge(new_df, how='left', on='column2' I am not sure that it is possible to reassign self

– Aran Freel
May 10 '17 at 13:12

1

You're correct that DataFrame.merge has no inplace argument. It returns a DataFrame, so no issue reassigning.

– JAV
May 11 '17 at 5:01

Can someone also highlight the advantages of using it in terms of resource consumption?

– markroxor
Apr 3 at 12:03

add a comment |

5

Yes, inplace=True returns None inplace=False returns a copy of the object with the operation performed. The docs are pretty clear on this, is there something that is confusing with a specific part? Spefically If True, do operation inplace and return None.

– EdChum
May 10 '17 at 13:09

1

Correctly said..

– Aditya
May 10 '17 at 13:10

I am subclassing the DataFrame object and with an operation such as merge it doesn't seem possible to do it inplace... self = self.merge(new_df, how='left', on='column2' I am not sure that it is possible to reassign self

– Aran Freel
May 10 '17 at 13:12

1

You're correct that DataFrame.merge has no inplace argument. It returns a DataFrame, so no issue reassigning.

– JAV
May 11 '17 at 5:01

Can someone also highlight the advantages of using it in terms of resource consumption?

– markroxor
Apr 3 at 12:03

Yes, inplace=True returns None inplace=False returns a copy of the object with the operation performed. The docs are pretty clear on this, is there something that is confusing with a specific part? Spefically If True, do operation inplace and return None.

– EdChum
May 10 '17 at 13:09

Correctly said..

– Aditya
May 10 '17 at 13:10

I am subclassing the DataFrame object and with an operation such as merge it doesn't seem possible to do it inplace... self = self.merge(new_df, how='left', on='column2' I am not sure that it is possible to reassign self

– Aran Freel
May 10 '17 at 13:12

You're correct that DataFrame.merge has no inplace argument. It returns a DataFrame, so no issue reassigning.

– JAV
May 11 '17 at 5:01

Can someone also highlight the advantages of using it in terms of resource consumption?

– markroxor
Apr 3 at 12:03

add a comment |

3 Answers
3

active

oldest

votes

When inplace=True is passed, the data is renamed in place (it returns nothing), so you'd use:

df.an_operation(inplace=True)

When inplace=False is passed (this is the default value, so isn't necessary), performs the operation and returns a copy of the object, so you'd use:

df = df.an_operation(inplace=False)

So:

if inplace == False:

    Assign your result to a new variable

else

    No need to assign

edited Feb 12 at 14:40

answered Aug 24 '17 at 8:02

ECH

1,08421539

Would I be right in thinking that inplace is only an option for methods which alter existing data, but not for methods which 'reshape' the data. For instance, I can .set_index(inplace=True) as this applies values to the existing index, but can't .reindex(inplace=True) because this could create extra rows on the DataFrame that didn't exist in the previous array?

– ac24
Mar 13 '18 at 22:49

4

The method .dropna() accepts inplace=True and can most definitely reshape the dataframe, so no.

– jorijnsmit
Aug 26 '18 at 13:46

1

You have to be careful here. @ac24 is actually more or less right. While dropna returns a dataframe of different shape, it doesn’t actually reshape the underlying data — it merely returns a mask over it (when inplace=False), which can lead to the dreaded SettingWithCopyWarning. Only when there are no more references to the old array of values will pandas reshape according to the mask. A better rule of thumb is: inplace is available when the operation doesn’t require allocating a new backing ndarray of values.

– BallpointBen
Feb 27 at 5:08

Perfection!! +1

– Kaushal28
Mar 17 at 11:42

add a comment |

The way I use it is

# Have to assign back to dataframe (because it is a new copy)

df = df.some_operation(inplace=False)

# No need to assign back to dataframe (because it is on the same copy)

df.some_operation(inplace=True)

CONCLUSION:

 if inplace is False

      Assign to a new variable;

 else

      No need to assign

edited Sep 14 '18 at 4:27

answered Mar 4 '18 at 2:43

Nabin

6,34654372

1

Hi @Nabin, Thats way too clear for anyone working on Pandas and Numpy :-)

– Vetrivel PS
Dec 27 '18 at 7:52

add a comment |

I usually use with numpy.

you use inplace=True, if you don't want to save the updated data to the same variable

data["column1"].where(data["column1"]< 5, inplace=True)

this is same as...

data["column1"] = data["column1"].where(data["column1"]< 5)

answered Sep 13 '18 at 18:50

hyukkyulee

433

add a comment |

Your Answer

StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});

}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f43893457%2fpython-pandas-understanding-inplace-true%23new-answer', 'question_page');
}
);

Post as a guest

Name

Required, but never shown

3 Answers
3

active

oldest

votes

3 Answers
3

active

oldest

votes

When inplace=True is passed, the data is renamed in place (it returns nothing), so you'd use:

df.an_operation(inplace=True)

When inplace=False is passed (this is the default value, so isn't necessary), performs the operation and returns a copy of the object, so you'd use:

df = df.an_operation(inplace=False)

So:

if inplace == False:

    Assign your result to a new variable

else

    No need to assign

edited Feb 12 at 14:40

answered Aug 24 '17 at 8:02

ECH

1,08421539

Would I be right in thinking that inplace is only an option for methods which alter existing data, but not for methods which 'reshape' the data. For instance, I can .set_index(inplace=True) as this applies values to the existing index, but can't .reindex(inplace=True) because this could create extra rows on the DataFrame that didn't exist in the previous array?

– ac24
Mar 13 '18 at 22:49

4

The method .dropna() accepts inplace=True and can most definitely reshape the dataframe, so no.

– jorijnsmit
Aug 26 '18 at 13:46

1

You have to be careful here. @ac24 is actually more or less right. While dropna returns a dataframe of different shape, it doesn’t actually reshape the underlying data — it merely returns a mask over it (when inplace=False), which can lead to the dreaded SettingWithCopyWarning. Only when there are no more references to the old array of values will pandas reshape according to the mask. A better rule of thumb is: inplace is available when the operation doesn’t require allocating a new backing ndarray of values.

– BallpointBen
Feb 27 at 5:08

Perfection!! +1

– Kaushal28
Mar 17 at 11:42

add a comment |

When inplace=True is passed, the data is renamed in place (it returns nothing), so you'd use:

df.an_operation(inplace=True)

When inplace=False is passed (this is the default value, so isn't necessary), performs the operation and returns a copy of the object, so you'd use:

df = df.an_operation(inplace=False)

So:

if inplace == False:

    Assign your result to a new variable

else

    No need to assign

edited Feb 12 at 14:40

answered Aug 24 '17 at 8:02

ECH

1,08421539

Would I be right in thinking that inplace is only an option for methods which alter existing data, but not for methods which 'reshape' the data. For instance, I can .set_index(inplace=True) as this applies values to the existing index, but can't .reindex(inplace=True) because this could create extra rows on the DataFrame that didn't exist in the previous array?

– ac24
Mar 13 '18 at 22:49

4

The method .dropna() accepts inplace=True and can most definitely reshape the dataframe, so no.

– jorijnsmit
Aug 26 '18 at 13:46

1

You have to be careful here. @ac24 is actually more or less right. While dropna returns a dataframe of different shape, it doesn’t actually reshape the underlying data — it merely returns a mask over it (when inplace=False), which can lead to the dreaded SettingWithCopyWarning. Only when there are no more references to the old array of values will pandas reshape according to the mask. A better rule of thumb is: inplace is available when the operation doesn’t require allocating a new backing ndarray of values.

– BallpointBen
Feb 27 at 5:08

Perfection!! +1

– Kaushal28
Mar 17 at 11:42

add a comment |

When inplace=True is passed, the data is renamed in place (it returns nothing), so you'd use:

df.an_operation(inplace=True)

When inplace=False is passed (this is the default value, so isn't necessary), performs the operation and returns a copy of the object, so you'd use:

df = df.an_operation(inplace=False)

So:

if inplace == False:

    Assign your result to a new variable

else

    No need to assign

edited Feb 12 at 14:40

answered Aug 24 '17 at 8:02

ECH

1,08421539

When inplace=True is passed, the data is renamed in place (it returns nothing), so you'd use:

df.an_operation(inplace=True)

When inplace=False is passed (this is the default value, so isn't necessary), performs the operation and returns a copy of the object, so you'd use:

df = df.an_operation(inplace=False)

So:

if inplace == False:

    Assign your result to a new variable

else

    No need to assign

edited Feb 12 at 14:40

answered Aug 24 '17 at 8:02

ECH

1,08421539

edited Feb 12 at 14:40

answered Aug 24 '17 at 8:02

ECH

1,08421539

answered Aug 24 '17 at 8:02

ECH

1,08421539

answered Aug 24 '17 at 8:02

ECH

1,08421539

Would I be right in thinking that inplace is only an option for methods which alter existing data, but not for methods which 'reshape' the data. For instance, I can .set_index(inplace=True) as this applies values to the existing index, but can't .reindex(inplace=True) because this could create extra rows on the DataFrame that didn't exist in the previous array?

– ac24
Mar 13 '18 at 22:49

4

The method .dropna() accepts inplace=True and can most definitely reshape the dataframe, so no.

– jorijnsmit
Aug 26 '18 at 13:46

1

You have to be careful here. @ac24 is actually more or less right. While dropna returns a dataframe of different shape, it doesn’t actually reshape the underlying data — it merely returns a mask over it (when inplace=False), which can lead to the dreaded SettingWithCopyWarning. Only when there are no more references to the old array of values will pandas reshape according to the mask. A better rule of thumb is: inplace is available when the operation doesn’t require allocating a new backing ndarray of values.

– BallpointBen
Feb 27 at 5:08

Perfection!! +1

– Kaushal28
Mar 17 at 11:42

add a comment |

Would I be right in thinking that inplace is only an option for methods which alter existing data, but not for methods which 'reshape' the data. For instance, I can .set_index(inplace=True) as this applies values to the existing index, but can't .reindex(inplace=True) because this could create extra rows on the DataFrame that didn't exist in the previous array?

– ac24
Mar 13 '18 at 22:49

4

The method .dropna() accepts inplace=True and can most definitely reshape the dataframe, so no.

– jorijnsmit
Aug 26 '18 at 13:46

1

You have to be careful here. @ac24 is actually more or less right. While dropna returns a dataframe of different shape, it doesn’t actually reshape the underlying data — it merely returns a mask over it (when inplace=False), which can lead to the dreaded SettingWithCopyWarning. Only when there are no more references to the old array of values will pandas reshape according to the mask. A better rule of thumb is: inplace is available when the operation doesn’t require allocating a new backing ndarray of values.

– BallpointBen
Feb 27 at 5:08

Perfection!! +1

– Kaushal28
Mar 17 at 11:42

Would I be right in thinking that inplace is only an option for methods which alter existing data, but not for methods which 'reshape' the data. For instance, I can .set_index(inplace=True) as this applies values to the existing index, but can't .reindex(inplace=True) because this could create extra rows on the DataFrame that didn't exist in the previous array?

– ac24
Mar 13 '18 at 22:49

The method .dropna() accepts inplace=True and can most definitely reshape the dataframe, so no.

– jorijnsmit
Aug 26 '18 at 13:46

You have to be careful here. @ac24 is actually more or less right. While dropna returns a dataframe of different shape, it doesn’t actually reshape the underlying data — it merely returns a mask over it (when inplace=False), which can lead to the dreaded SettingWithCopyWarning. Only when there are no more references to the old array of values will pandas reshape according to the mask. A better rule of thumb is: inplace is available when the operation doesn’t require allocating a new backing ndarray of values.

– BallpointBen
Feb 27 at 5:08

Perfection!! +1

– Kaushal28
Mar 17 at 11:42

add a comment |

The way I use it is

# Have to assign back to dataframe (because it is a new copy)

df = df.some_operation(inplace=False)

# No need to assign back to dataframe (because it is on the same copy)

df.some_operation(inplace=True)

CONCLUSION:

 if inplace is False

      Assign to a new variable;

 else

      No need to assign

edited Sep 14 '18 at 4:27

answered Mar 4 '18 at 2:43

Nabin

6,34654372

1

Hi @Nabin, Thats way too clear for anyone working on Pandas and Numpy :-)

– Vetrivel PS
Dec 27 '18 at 7:52

add a comment |

The way I use it is

# Have to assign back to dataframe (because it is a new copy)

df = df.some_operation(inplace=False)

# No need to assign back to dataframe (because it is on the same copy)

df.some_operation(inplace=True)

CONCLUSION:

 if inplace is False

      Assign to a new variable;

 else

      No need to assign

edited Sep 14 '18 at 4:27

answered Mar 4 '18 at 2:43

Nabin

6,34654372

1

Hi @Nabin, Thats way too clear for anyone working on Pandas and Numpy :-)

– Vetrivel PS
Dec 27 '18 at 7:52

add a comment |

The way I use it is

# Have to assign back to dataframe (because it is a new copy)

df = df.some_operation(inplace=False)

# No need to assign back to dataframe (because it is on the same copy)

df.some_operation(inplace=True)

CONCLUSION:

 if inplace is False

      Assign to a new variable;

 else

      No need to assign

edited Sep 14 '18 at 4:27

answered Mar 4 '18 at 2:43

Nabin

6,34654372

The way I use it is

# Have to assign back to dataframe (because it is a new copy)

df = df.some_operation(inplace=False)

# No need to assign back to dataframe (because it is on the same copy)

df.some_operation(inplace=True)

CONCLUSION:

 if inplace is False

      Assign to a new variable;

 else

      No need to assign

edited Sep 14 '18 at 4:27

answered Mar 4 '18 at 2:43

Nabin

6,34654372

edited Sep 14 '18 at 4:27

answered Mar 4 '18 at 2:43

Nabin

6,34654372

answered Mar 4 '18 at 2:43

Nabin

6,34654372

answered Mar 4 '18 at 2:43

Nabin

6,34654372

1

Hi @Nabin, Thats way too clear for anyone working on Pandas and Numpy :-)

– Vetrivel PS
Dec 27 '18 at 7:52

add a comment |

1

Hi @Nabin, Thats way too clear for anyone working on Pandas and Numpy :-)

– Vetrivel PS
Dec 27 '18 at 7:52

Hi @Nabin, Thats way too clear for anyone working on Pandas and Numpy :-)

– Vetrivel PS
Dec 27 '18 at 7:52

add a comment |

I usually use with numpy.

you use inplace=True, if you don't want to save the updated data to the same variable

data["column1"].where(data["column1"]< 5, inplace=True)

this is same as...

data["column1"] = data["column1"].where(data["column1"]< 5)

answered Sep 13 '18 at 18:50

hyukkyulee

433

add a comment |

I usually use with numpy.

you use inplace=True, if you don't want to save the updated data to the same variable

data["column1"].where(data["column1"]< 5, inplace=True)

this is same as...

data["column1"] = data["column1"].where(data["column1"]< 5)

answered Sep 13 '18 at 18:50

hyukkyulee

433

add a comment |

I usually use with numpy.

you use inplace=True, if you don't want to save the updated data to the same variable

data["column1"].where(data["column1"]< 5, inplace=True)

this is same as...

data["column1"] = data["column1"].where(data["column1"]< 5)

answered Sep 13 '18 at 18:50

hyukkyulee

433

I usually use with numpy.

you use inplace=True, if you don't want to save the updated data to the same variable

data["column1"].where(data["column1"]< 5, inplace=True)

this is same as...

data["column1"] = data["column1"].where(data["column1"]< 5)

answered Sep 13 '18 at 18:50

hyukkyulee

433

answered Sep 13 '18 at 18:50

hyukkyulee

433

answered Sep 13 '18 at 18:50

hyukkyulee

433

answered Sep 13 '18 at 18:50

hyukkyulee

433

add a comment |

draft saved

draft discarded

Thanks for contributing an answer to Stack Overflow!

Please be sure to answer the question. Provide details and share your research!

But avoid …

Asking for help, clarification, or responding to other answers.

Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Post as a guest

Name

Required, but never shown

Name

Required, but never shown

Name

Required, but never shown

This page is only for reference, If you need detailed information, please check here

搜尋此網誌

Argthtjtr