Remove any empty fields in a loop?
A list has many paths of certain csv's.
How to check if each csv in every loop has any empty columns and delete them if they are.
Code:
for i in list1:
if (list1.columns = '').any():
i.remove that column
Hope this explains what I am talking about.
python pandas
asked Nov 20 '18 at 11:30
user10671234
376
add a comment |
A list has many paths of certain csv's.
How to check if each csv in every loop has any empty columns and delete them if they are.
Code:
for i in list1:
if (list1.columns = '').any():
i.remove that column
Hope this explains what I am talking about.
python pandas
asked Nov 20 '18 at 11:30
user10671234
376
add a comment |
A list has many paths of certain csv's.
How to check if each csv in every loop has any empty columns and delete them if they are.
Code:
for i in list1:
if (list1.columns = '').any():
i.remove that column
Hope this explains what I am talking about.
python pandas
asked Nov 20 '18 at 11:30
user10671234
376
A list has many paths of certain csv's.
How to check if each csv in every loop has any empty columns and delete them if they are.
Code:
for i in list1:
if (list1.columns = '').any():
i.remove that column
Hope this explains what I am talking about.
python pandas
python pandas
asked Nov 20 '18 at 11:30
user10671234
376
asked Nov 20 '18 at 11:30
user10671234
376
asked Nov 20 '18 at 11:30
user10671234
376
asked Nov 20 '18 at 11:30
user10671234
376
asked Nov 20 '18 at 11:30
user10671234
376
376
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
Sample:
df = pd.DataFrame({
'':list('abcdef'),
'B':[4,5,4,5,5,np.nan],
'C':[''] * 6,
'D':[np.nan] * 6,
'E':[5,3,6,9,2,4],
'F':list('aaabb') + ['']
})
print (df)
B C D E F
0 a 4.0 NaN 5 a
1 b 5.0 NaN 3 a
2 c 4.0 NaN 6 a
3 d 5.0 NaN 9 b
4 e 5.0 NaN 2 b
5 f NaN NaN 4
Removed first column, because empty column name - it means filtering only columns with no empty values with loc and boolean indexing:
df1 = df.loc[:, df.columns != '']
print (df1)
B C D E F
0 4.0 NaN 5 a
1 5.0 NaN 3 a
2 4.0 NaN 6 a
3 5.0 NaN 9 b
4 5.0 NaN 2 b
5 NaN NaN 4
Reoved column C, because filled only empty values - compare all values if not empty values and get at least one True per column by DataFrame.any, also filter by boolean indexing with loc:
df2 = df.loc[:, (df != '').any()]
print (df2)
B D E
0 a 4.0 NaN 5
1 b 5.0 NaN 3
2 c 4.0 NaN 6
3 d 5.0 NaN 9
4 e 5.0 NaN 2
5 f NaN NaN 4
print ((df != ''))
B C D E F
0 True True False True True True
1 True True False True True True
2 True True False True True True
3 True True False True True True
4 True True False True True True
5 True True False True True False
print ((df != '').any())
True
B True
C False
D True
E True
F True
dtype: bool
Removed column D because filled only missing values with function dropna:
df3 = df.dropna(axis=1, how='all')
print (df3)
B C E F
0 a 4.0 5 a
1 b 5.0 3 a
2 c 4.0 6 a
3 d 5.0 9 b
4 e 5.0 2 b
5 f NaN 4
answered Nov 20 '18 at 11:32
jezrael
321k22263341
The first filters out columns that may not be empty but have empty field name?
– user10671234
Nov 20 '18 at 11:35
@user10671234 - added sample.
– jezrael
Nov 20 '18 at 11:38
1
upvoted and accepted thanks
– user10671234
Nov 20 '18 at 11:44
1
@user10671234 - thank you.
– jezrael
Nov 20 '18 at 11:45
1
ok understood now.
– user10671234
Nov 20 '18 at 12:49
|
show 8 more comments
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53392065%2fremove-any-empty-fields-in-a-loop%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Sample:
df = pd.DataFrame({
'':list('abcdef'),
'B':[4,5,4,5,5,np.nan],
'C':[''] * 6,
'D':[np.nan] * 6,
'E':[5,3,6,9,2,4],
'F':list('aaabb') + ['']
})
print (df)
B C D E F
0 a 4.0 NaN 5 a
1 b 5.0 NaN 3 a
2 c 4.0 NaN 6 a
3 d 5.0 NaN 9 b
4 e 5.0 NaN 2 b
5 f NaN NaN 4
Removed first column, because empty column name - it means filtering only columns with no empty values with loc and boolean indexing:
df1 = df.loc[:, df.columns != '']
print (df1)
B C D E F
0 4.0 NaN 5 a
1 5.0 NaN 3 a
2 4.0 NaN 6 a
3 5.0 NaN 9 b
4 5.0 NaN 2 b
5 NaN NaN 4
Reoved column C, because filled only empty values - compare all values if not empty values and get at least one True per column by DataFrame.any, also filter by boolean indexing with loc:
df2 = df.loc[:, (df != '').any()]
print (df2)
B D E
0 a 4.0 NaN 5
1 b 5.0 NaN 3
2 c 4.0 NaN 6
3 d 5.0 NaN 9
4 e 5.0 NaN 2
5 f NaN NaN 4
print ((df != ''))
B C D E F
0 True True False True True True
1 True True False True True True
2 True True False True True True
3 True True False True True True
4 True True False True True True
5 True True False True True False
print ((df != '').any())
True
B True
C False
D True
E True
F True
dtype: bool
Removed column D because filled only missing values with function dropna:
df3 = df.dropna(axis=1, how='all')
print (df3)
B C E F
0 a 4.0 5 a
1 b 5.0 3 a
2 c 4.0 6 a
3 d 5.0 9 b
4 e 5.0 2 b
5 f NaN 4
answered Nov 20 '18 at 11:32
jezrael
321k22263341
The first filters out columns that may not be empty but have empty field name?
– user10671234
Nov 20 '18 at 11:35
@user10671234 - added sample.
– jezrael
Nov 20 '18 at 11:38
1
upvoted and accepted thanks
– user10671234
Nov 20 '18 at 11:44
1
@user10671234 - thank you.
– jezrael
Nov 20 '18 at 11:45
1
ok understood now.
– user10671234
Nov 20 '18 at 12:49
|
show 8 more comments
Sample:
df = pd.DataFrame({
'':list('abcdef'),
'B':[4,5,4,5,5,np.nan],
'C':[''] * 6,
'D':[np.nan] * 6,
'E':[5,3,6,9,2,4],
'F':list('aaabb') + ['']
})
print (df)
B C D E F
0 a 4.0 NaN 5 a
1 b 5.0 NaN 3 a
2 c 4.0 NaN 6 a
3 d 5.0 NaN 9 b
4 e 5.0 NaN 2 b
5 f NaN NaN 4
Removed first column, because empty column name - it means filtering only columns with no empty values with loc and boolean indexing:
df1 = df.loc[:, df.columns != '']
print (df1)
B C D E F
0 4.0 NaN 5 a
1 5.0 NaN 3 a
2 4.0 NaN 6 a
3 5.0 NaN 9 b
4 5.0 NaN 2 b
5 NaN NaN 4
Reoved column C, because filled only empty values - compare all values if not empty values and get at least one True per column by DataFrame.any, also filter by boolean indexing with loc:
df2 = df.loc[:, (df != '').any()]
print (df2)
B D E
0 a 4.0 NaN 5
1 b 5.0 NaN 3
2 c 4.0 NaN 6
3 d 5.0 NaN 9
4 e 5.0 NaN 2
5 f NaN NaN 4
print ((df != ''))
B C D E F
0 True True False True True True
1 True True False True True True
2 True True False True True True
3 True True False True True True
4 True True False True True True
5 True True False True True False
print ((df != '').any())
True
B True
C False
D True
E True
F True
dtype: bool
Removed column D because filled only missing values with function dropna:
df3 = df.dropna(axis=1, how='all')
print (df3)
B C E F
0 a 4.0 5 a
1 b 5.0 3 a
2 c 4.0 6 a
3 d 5.0 9 b
4 e 5.0 2 b
5 f NaN 4
answered Nov 20 '18 at 11:32
jezrael
321k22263341
The first filters out columns that may not be empty but have empty field name?
– user10671234
Nov 20 '18 at 11:35
@user10671234 - added sample.
– jezrael
Nov 20 '18 at 11:38
1
upvoted and accepted thanks
– user10671234
Nov 20 '18 at 11:44
1
@user10671234 - thank you.
– jezrael
Nov 20 '18 at 11:45
1
ok understood now.
– user10671234
Nov 20 '18 at 12:49
|
show 8 more comments
Sample:
df = pd.DataFrame({
'':list('abcdef'),
'B':[4,5,4,5,5,np.nan],
'C':[''] * 6,
'D':[np.nan] * 6,
'E':[5,3,6,9,2,4],
'F':list('aaabb') + ['']
})
print (df)
B C D E F
0 a 4.0 NaN 5 a
1 b 5.0 NaN 3 a
2 c 4.0 NaN 6 a
3 d 5.0 NaN 9 b
4 e 5.0 NaN 2 b
5 f NaN NaN 4
Removed first column, because empty column name - it means filtering only columns with no empty values with loc and boolean indexing:
df1 = df.loc[:, df.columns != '']
print (df1)
B C D E F
0 4.0 NaN 5 a
1 5.0 NaN 3 a
2 4.0 NaN 6 a
3 5.0 NaN 9 b
4 5.0 NaN 2 b
5 NaN NaN 4
Reoved column C, because filled only empty values - compare all values if not empty values and get at least one True per column by DataFrame.any, also filter by boolean indexing with loc:
df2 = df.loc[:, (df != '').any()]
print (df2)
B D E
0 a 4.0 NaN 5
1 b 5.0 NaN 3
2 c 4.0 NaN 6
3 d 5.0 NaN 9
4 e 5.0 NaN 2
5 f NaN NaN 4
print ((df != ''))
B C D E F
0 True True False True True True
1 True True False True True True
2 True True False True True True
3 True True False True True True
4 True True False True True True
5 True True False True True False
print ((df != '').any())
True
B True
C False
D True
E True
F True
dtype: bool
Removed column D because filled only missing values with function dropna:
df3 = df.dropna(axis=1, how='all')
print (df3)
B C E F
0 a 4.0 5 a
1 b 5.0 3 a
2 c 4.0 6 a
3 d 5.0 9 b
4 e 5.0 2 b
5 f NaN 4
answered Nov 20 '18 at 11:32
jezrael
321k22263341
Sample:
df = pd.DataFrame({
'':list('abcdef'),
'B':[4,5,4,5,5,np.nan],
'C':[''] * 6,
'D':[np.nan] * 6,
'E':[5,3,6,9,2,4],
'F':list('aaabb') + ['']
})
print (df)
B C D E F
0 a 4.0 NaN 5 a
1 b 5.0 NaN 3 a
2 c 4.0 NaN 6 a
3 d 5.0 NaN 9 b
4 e 5.0 NaN 2 b
5 f NaN NaN 4
Removed first column, because empty column name - it means filtering only columns with no empty values with loc and boolean indexing:
df1 = df.loc[:, df.columns != '']
print (df1)
B C D E F
0 4.0 NaN 5 a
1 5.0 NaN 3 a
2 4.0 NaN 6 a
3 5.0 NaN 9 b
4 5.0 NaN 2 b
5 NaN NaN 4
Reoved column C, because filled only empty values - compare all values if not empty values and get at least one True per column by DataFrame.any, also filter by boolean indexing with loc:
df2 = df.loc[:, (df != '').any()]
print (df2)
B D E
0 a 4.0 NaN 5
1 b 5.0 NaN 3
2 c 4.0 NaN 6
3 d 5.0 NaN 9
4 e 5.0 NaN 2
5 f NaN NaN 4
print ((df != ''))
B C D E F
0 True True False True True True
1 True True False True True True
2 True True False True True True
3 True True False True True True
4 True True False True True True
5 True True False True True False
print ((df != '').any())
True
B True
C False
D True
E True
F True
dtype: bool
Removed column D because filled only missing values with function dropna:
df3 = df.dropna(axis=1, how='all')
print (df3)
B C E F
0 a 4.0 5 a
1 b 5.0 3 a
2 c 4.0 6 a
3 d 5.0 9 b
4 e 5.0 2 b
5 f NaN 4
answered Nov 20 '18 at 11:32
jezrael
321k22263341
edited Nov 20 '18 at 11:47
answered Nov 20 '18 at 11:32
jezrael
321k22263341
answered Nov 20 '18 at 11:32
jezrael
321k22263341
answered Nov 20 '18 at 11:32
jezrael
321k22263341
321k22263341
The first filters out columns that may not be empty but have empty field name?
– user10671234
Nov 20 '18 at 11:35
@user10671234 - added sample.
– jezrael
Nov 20 '18 at 11:38
1
upvoted and accepted thanks
– user10671234
Nov 20 '18 at 11:44
1
@user10671234 - thank you.
– jezrael
Nov 20 '18 at 11:45
1
ok understood now.
– user10671234
Nov 20 '18 at 12:49
|
show 8 more comments
The first filters out columns that may not be empty but have empty field name?
– user10671234
Nov 20 '18 at 11:35
@user10671234 - added sample.
– jezrael
Nov 20 '18 at 11:38
1
upvoted and accepted thanks
– user10671234
Nov 20 '18 at 11:44
1
@user10671234 - thank you.
– jezrael
Nov 20 '18 at 11:45
1
ok understood now.
– user10671234
Nov 20 '18 at 12:49
The first filters out columns that may not be empty but have empty field name?
– user10671234
Nov 20 '18 at 11:35
The first filters out columns that may not be empty but have empty field name?
– user10671234
Nov 20 '18 at 11:35
@user10671234 - added sample.
– jezrael
Nov 20 '18 at 11:38
@user10671234 - added sample.
– jezrael
Nov 20 '18 at 11:38
1
1
upvoted and accepted thanks
– user10671234
Nov 20 '18 at 11:44
upvoted and accepted thanks
– user10671234
Nov 20 '18 at 11:44
1
1
@user10671234 - thank you.
– jezrael
Nov 20 '18 at 11:45
@user10671234 - thank you.
– jezrael
Nov 20 '18 at 11:45
1
1
ok understood now.
– user10671234
Nov 20 '18 at 12:49
ok understood now.
– user10671234
Nov 20 '18 at 12:49
|
show 8 more comments
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53392065%2fremove-any-empty-fields-in-a-loop%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
