Check certain columns in list of paths and change their data type?
In a list of csv files (full paths) I want to check certain columns that exist only in about one dataset but I don't know which so I want to check:
If the columns 'DEC_DATE' and 'KYR_DATE' found, they should be changed to date type.
Code:
for i in li: #list with paths of csv's
df = pd.read_csv(i)
if df['DEC_DATE]: # if csv that has DEC_DATE column found, turn it to date type.
df['DEC_DATE'] = pd.to_datetime(df['DEC_DATE']) # same for the other column
python pandas
asked Nov 20 '18 at 14:30
user10680652user10680652
234
add a comment |
In a list of csv files (full paths) I want to check certain columns that exist only in about one dataset but I don't know which so I want to check:
If the columns 'DEC_DATE' and 'KYR_DATE' found, they should be changed to date type.
Code:
for i in li: #list with paths of csv's
df = pd.read_csv(i)
if df['DEC_DATE]: # if csv that has DEC_DATE column found, turn it to date type.
df['DEC_DATE'] = pd.to_datetime(df['DEC_DATE']) # same for the other column
python pandas
asked Nov 20 '18 at 14:30
user10680652user10680652
234
add a comment |
In a list of csv files (full paths) I want to check certain columns that exist only in about one dataset but I don't know which so I want to check:
If the columns 'DEC_DATE' and 'KYR_DATE' found, they should be changed to date type.
Code:
for i in li: #list with paths of csv's
df = pd.read_csv(i)
if df['DEC_DATE]: # if csv that has DEC_DATE column found, turn it to date type.
df['DEC_DATE'] = pd.to_datetime(df['DEC_DATE']) # same for the other column
python pandas
asked Nov 20 '18 at 14:30
user10680652user10680652
234
In a list of csv files (full paths) I want to check certain columns that exist only in about one dataset but I don't know which so I want to check:
If the columns 'DEC_DATE' and 'KYR_DATE' found, they should be changed to date type.
Code:
for i in li: #list with paths of csv's
df = pd.read_csv(i)
if df['DEC_DATE]: # if csv that has DEC_DATE column found, turn it to date type.
df['DEC_DATE'] = pd.to_datetime(df['DEC_DATE']) # same for the other column
python pandas
python pandas
asked Nov 20 '18 at 14:30
user10680652user10680652
234
asked Nov 20 '18 at 14:30
user10680652user10680652
234
asked Nov 20 '18 at 14:30
user10680652user10680652
234
asked Nov 20 '18 at 14:30
user10680652user10680652
234
asked Nov 20 '18 at 14:30
user10680652user10680652
234
234
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
I guess you are getting KeyError because you can't access a columns that doesn't exist. You should use in and df.columns instead:
for i in li: #list with paths of csv's
df = pd.read_csv(i)
if 'DEC_DATE' in df.columns:
df['DEC_DATE'] = pd.to_datetime(df['DEC_DATE'])
If there are only two columns, you can use two explicit ifs to obtain the full result you want:
for i in li:
df = pd.read_csv(i)
if 'DEC_DATE' in df.columns:
df['DEC_DATE'] = pd.to_datetime(df['DEC_DATE'])
if 'KYR_DATE' in df.columns:
df['KYR_DATE'] = pd.to_datetime(df['KYR_DATE'])
answered Nov 20 '18 at 14:34
Julian PellerJulian Peller
864511
Can you add also how to apply to the other column too?Thanks
– user10680652
Nov 20 '18 at 14:37
Of course, it's done!
– Julian Peller
Nov 20 '18 at 14:41
add a comment |
Check converters in read_csv
pd.read_csv('Temp.csv',converters={'DEC_DATE':pd.to_datetime})
answered Nov 20 '18 at 14:37
W-BW-B
103k73164
Will this handle the fact that the column may not exist in the dataframe?
– Julian Peller
Nov 20 '18 at 14:41
@JulianPeller yes
– W-B
Nov 20 '18 at 14:41
1
Nice, then this is the correct solution from my point of view.
– Julian Peller
Nov 20 '18 at 14:42
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53395240%2fcheck-certain-columns-in-list-of-paths-and-change-their-data-type%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
I guess you are getting KeyError because you can't access a columns that doesn't exist. You should use in and df.columns instead:
for i in li: #list with paths of csv's
df = pd.read_csv(i)
if 'DEC_DATE' in df.columns:
df['DEC_DATE'] = pd.to_datetime(df['DEC_DATE'])
If there are only two columns, you can use two explicit ifs to obtain the full result you want:
for i in li:
df = pd.read_csv(i)
if 'DEC_DATE' in df.columns:
df['DEC_DATE'] = pd.to_datetime(df['DEC_DATE'])
if 'KYR_DATE' in df.columns:
df['KYR_DATE'] = pd.to_datetime(df['KYR_DATE'])
answered Nov 20 '18 at 14:34
Julian PellerJulian Peller
864511
Can you add also how to apply to the other column too?Thanks
– user10680652
Nov 20 '18 at 14:37
Of course, it's done!
– Julian Peller
Nov 20 '18 at 14:41
add a comment |
I guess you are getting KeyError because you can't access a columns that doesn't exist. You should use in and df.columns instead:
for i in li: #list with paths of csv's
df = pd.read_csv(i)
if 'DEC_DATE' in df.columns:
df['DEC_DATE'] = pd.to_datetime(df['DEC_DATE'])
If there are only two columns, you can use two explicit ifs to obtain the full result you want:
for i in li:
df = pd.read_csv(i)
if 'DEC_DATE' in df.columns:
df['DEC_DATE'] = pd.to_datetime(df['DEC_DATE'])
if 'KYR_DATE' in df.columns:
df['KYR_DATE'] = pd.to_datetime(df['KYR_DATE'])
answered Nov 20 '18 at 14:34
Julian PellerJulian Peller
864511
Can you add also how to apply to the other column too?Thanks
– user10680652
Nov 20 '18 at 14:37
Of course, it's done!
– Julian Peller
Nov 20 '18 at 14:41
add a comment |
I guess you are getting KeyError because you can't access a columns that doesn't exist. You should use in and df.columns instead:
for i in li: #list with paths of csv's
df = pd.read_csv(i)
if 'DEC_DATE' in df.columns:
df['DEC_DATE'] = pd.to_datetime(df['DEC_DATE'])
If there are only two columns, you can use two explicit ifs to obtain the full result you want:
for i in li:
df = pd.read_csv(i)
if 'DEC_DATE' in df.columns:
df['DEC_DATE'] = pd.to_datetime(df['DEC_DATE'])
if 'KYR_DATE' in df.columns:
df['KYR_DATE'] = pd.to_datetime(df['KYR_DATE'])
answered Nov 20 '18 at 14:34
Julian PellerJulian Peller
864511
I guess you are getting KeyError because you can't access a columns that doesn't exist. You should use in and df.columns instead:
for i in li: #list with paths of csv's
df = pd.read_csv(i)
if 'DEC_DATE' in df.columns:
df['DEC_DATE'] = pd.to_datetime(df['DEC_DATE'])
If there are only two columns, you can use two explicit ifs to obtain the full result you want:
for i in li:
df = pd.read_csv(i)
if 'DEC_DATE' in df.columns:
df['DEC_DATE'] = pd.to_datetime(df['DEC_DATE'])
if 'KYR_DATE' in df.columns:
df['KYR_DATE'] = pd.to_datetime(df['KYR_DATE'])
answered Nov 20 '18 at 14:34
Julian PellerJulian Peller
864511
edited Nov 20 '18 at 14:40
answered Nov 20 '18 at 14:34
Julian PellerJulian Peller
864511
answered Nov 20 '18 at 14:34
Julian PellerJulian Peller
864511
answered Nov 20 '18 at 14:34
Julian PellerJulian Peller
864511
864511
Can you add also how to apply to the other column too?Thanks
– user10680652
Nov 20 '18 at 14:37
Of course, it's done!
– Julian Peller
Nov 20 '18 at 14:41
add a comment |
Can you add also how to apply to the other column too?Thanks
– user10680652
Nov 20 '18 at 14:37
Of course, it's done!
– Julian Peller
Nov 20 '18 at 14:41
Can you add also how to apply to the other column too?Thanks
– user10680652
Nov 20 '18 at 14:37
Can you add also how to apply to the other column too?Thanks
– user10680652
Nov 20 '18 at 14:37
Of course, it's done!
– Julian Peller
Nov 20 '18 at 14:41
Of course, it's done!
– Julian Peller
Nov 20 '18 at 14:41
add a comment |
Check converters in read_csv
pd.read_csv('Temp.csv',converters={'DEC_DATE':pd.to_datetime})
answered Nov 20 '18 at 14:37
W-BW-B
103k73164
Will this handle the fact that the column may not exist in the dataframe?
– Julian Peller
Nov 20 '18 at 14:41
@JulianPeller yes
– W-B
Nov 20 '18 at 14:41
1
Nice, then this is the correct solution from my point of view.
– Julian Peller
Nov 20 '18 at 14:42
add a comment |
Check converters in read_csv
pd.read_csv('Temp.csv',converters={'DEC_DATE':pd.to_datetime})
answered Nov 20 '18 at 14:37
W-BW-B
103k73164
Will this handle the fact that the column may not exist in the dataframe?
– Julian Peller
Nov 20 '18 at 14:41
@JulianPeller yes
– W-B
Nov 20 '18 at 14:41
1
Nice, then this is the correct solution from my point of view.
– Julian Peller
Nov 20 '18 at 14:42
add a comment |
Check converters in read_csv
pd.read_csv('Temp.csv',converters={'DEC_DATE':pd.to_datetime})
answered Nov 20 '18 at 14:37
W-BW-B
103k73164
Check converters in read_csv
pd.read_csv('Temp.csv',converters={'DEC_DATE':pd.to_datetime})
answered Nov 20 '18 at 14:37
W-BW-B
103k73164
answered Nov 20 '18 at 14:37
W-BW-B
103k73164
answered Nov 20 '18 at 14:37
W-BW-B
103k73164
answered Nov 20 '18 at 14:37
W-BW-B
103k73164
103k73164
Will this handle the fact that the column may not exist in the dataframe?
– Julian Peller
Nov 20 '18 at 14:41
@JulianPeller yes
– W-B
Nov 20 '18 at 14:41
1
Nice, then this is the correct solution from my point of view.
– Julian Peller
Nov 20 '18 at 14:42
add a comment |
Will this handle the fact that the column may not exist in the dataframe?
– Julian Peller
Nov 20 '18 at 14:41
@JulianPeller yes
– W-B
Nov 20 '18 at 14:41
1
Nice, then this is the correct solution from my point of view.
– Julian Peller
Nov 20 '18 at 14:42
Will this handle the fact that the column may not exist in the dataframe?
– Julian Peller
Nov 20 '18 at 14:41
Will this handle the fact that the column may not exist in the dataframe?
– Julian Peller
Nov 20 '18 at 14:41
@JulianPeller yes
– W-B
Nov 20 '18 at 14:41
@JulianPeller yes
– W-B
Nov 20 '18 at 14:41
1
1
Nice, then this is the correct solution from my point of view.
– Julian Peller
Nov 20 '18 at 14:42
Nice, then this is the correct solution from my point of view.
– Julian Peller
Nov 20 '18 at 14:42
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53395240%2fcheck-certain-columns-in-list-of-paths-and-change-their-data-type%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
