Pandas merging two dataframes with different number of multiindices
Welcome, I have a simple question, to which I haven't found a solution.
I have two dataframes df1 and df2:
df1contains several columns and a multiindex asyear-month-week
df2contains the multiindexyear-weekwith only one column in the df.
I would like to create an inner join of df1 and df2, joining on 'year' and 'week'.
I have tried to do the following:
df1['newcol'] = df1.index.get_level_values(2).map(lambda x: df2.newcol[x])
Which only joins on month (or year?), is there any way to expand it so that the merge is actually right?
Thanks in advance!
df1
df2
python pandas dataframe
asked Nov 22 '18 at 15:09
acronis011acronis011
83
|
show 1 more comment
Welcome, I have a simple question, to which I haven't found a solution.
I have two dataframes df1 and df2:
df1contains several columns and a multiindex asyear-month-week
df2contains the multiindexyear-weekwith only one column in the df.
I would like to create an inner join of df1 and df2, joining on 'year' and 'week'.
I have tried to do the following:
df1['newcol'] = df1.index.get_level_values(2).map(lambda x: df2.newcol[x])
Which only joins on month (or year?), is there any way to expand it so that the merge is actually right?
Thanks in advance!
df1
df2
python pandas dataframe
asked Nov 22 '18 at 15:09
acronis011acronis011
83
2
Can you provide a sample dataframe via a Minimal, Complete, and Verifiable example.
– jpp
Nov 22 '18 at 15:10
2
Show us some sample of the datasets. Refer to the guide on how to add code snippets.
– Shiv_90
Nov 22 '18 at 15:11
I added some pictures to represent the dataset, i hope it helps!
– acronis011
Nov 22 '18 at 15:22
We can't copy/paste pictures into python :D
– user3471881
Nov 22 '18 at 15:32
1
Did you trypd.merge([df1, df2], how = 'inner', on = ['year', 'week']?
– maow
Nov 22 '18 at 15:38
|
show 1 more comment
Welcome, I have a simple question, to which I haven't found a solution.
I have two dataframes df1 and df2:
df1contains several columns and a multiindex asyear-month-week
df2contains the multiindexyear-weekwith only one column in the df.
I would like to create an inner join of df1 and df2, joining on 'year' and 'week'.
I have tried to do the following:
df1['newcol'] = df1.index.get_level_values(2).map(lambda x: df2.newcol[x])
Which only joins on month (or year?), is there any way to expand it so that the merge is actually right?
Thanks in advance!
df1
df2
python pandas dataframe
asked Nov 22 '18 at 15:09
acronis011acronis011
83
Welcome, I have a simple question, to which I haven't found a solution.
I have two dataframes df1 and df2:
df1contains several columns and a multiindex asyear-month-week
df2contains the multiindexyear-weekwith only one column in the df.
I would like to create an inner join of df1 and df2, joining on 'year' and 'week'.
I have tried to do the following:
df1['newcol'] = df1.index.get_level_values(2).map(lambda x: df2.newcol[x])
Which only joins on month (or year?), is there any way to expand it so that the merge is actually right?
Thanks in advance!
df1
df2
python pandas dataframe
python pandas dataframe
asked Nov 22 '18 at 15:09
acronis011acronis011
83
asked Nov 22 '18 at 15:09
acronis011acronis011
83
edited Nov 22 '18 at 15:20
acronis011
asked Nov 22 '18 at 15:09
acronis011acronis011
83
asked Nov 22 '18 at 15:09
acronis011acronis011
83
asked Nov 22 '18 at 15:09
acronis011acronis011
83
83
2
Can you provide a sample dataframe via a Minimal, Complete, and Verifiable example.
– jpp
Nov 22 '18 at 15:10
2
Show us some sample of the datasets. Refer to the guide on how to add code snippets.
– Shiv_90
Nov 22 '18 at 15:11
I added some pictures to represent the dataset, i hope it helps!
– acronis011
Nov 22 '18 at 15:22
We can't copy/paste pictures into python :D
– user3471881
Nov 22 '18 at 15:32
1
Did you trypd.merge([df1, df2], how = 'inner', on = ['year', 'week']?
– maow
Nov 22 '18 at 15:38
|
show 1 more comment
2
Can you provide a sample dataframe via a Minimal, Complete, and Verifiable example.
– jpp
Nov 22 '18 at 15:10
2
Show us some sample of the datasets. Refer to the guide on how to add code snippets.
– Shiv_90
Nov 22 '18 at 15:11
I added some pictures to represent the dataset, i hope it helps!
– acronis011
Nov 22 '18 at 15:22
We can't copy/paste pictures into python :D
– user3471881
Nov 22 '18 at 15:32
1
Did you trypd.merge([df1, df2], how = 'inner', on = ['year', 'week']?
– maow
Nov 22 '18 at 15:38
2
2
Can you provide a sample dataframe via a Minimal, Complete, and Verifiable example.
– jpp
Nov 22 '18 at 15:10
Can you provide a sample dataframe via a Minimal, Complete, and Verifiable example.
– jpp
Nov 22 '18 at 15:10
2
2
Show us some sample of the datasets. Refer to the guide on how to add code snippets.
– Shiv_90
Nov 22 '18 at 15:11
Show us some sample of the datasets. Refer to the guide on how to add code snippets.
– Shiv_90
Nov 22 '18 at 15:11
I added some pictures to represent the dataset, i hope it helps!
– acronis011
Nov 22 '18 at 15:22
I added some pictures to represent the dataset, i hope it helps!
– acronis011
Nov 22 '18 at 15:22
We can't copy/paste pictures into python :D
– user3471881
Nov 22 '18 at 15:32
We can't copy/paste pictures into python :D
– user3471881
Nov 22 '18 at 15:32
1
1
Did you try
pd.merge([df1, df2], how = 'inner', on = ['year', 'week']?– maow
Nov 22 '18 at 15:38
Did you try
pd.merge([df1, df2], how = 'inner', on = ['year', 'week']?– maow
Nov 22 '18 at 15:38
|
show 1 more comment
1 Answer
1
active
oldest
votes
Eventually i solved with with removing the multiindex and doing a good old inner join on the two columns and then recreating the multiindex at the end.
Here are the sniplets:
df=df.reset_index()
df2=df2.reset_index()
df['year']=df['year'].apply(int)
df2['year']=df2['year'].apply(int)
df['week']=df['week'].apply(int)
df2['week']=df2['week'].apply(int)
result = pd.merge(df, df2, how='left', left_on= ['year','week'],right_on= ['year','week'])
result=result.set_index(['year', 'month','week','day'])
answered Nov 22 '18 at 15:51
acronis011acronis011
83
Since flattening index is not that intuitive for many, I'd suggest that you could add the actual code that you used, so that others could benefit from your experience ;) stackoverflow.blog/2011/07/01/…
– leoburgy
Nov 22 '18 at 16:08
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53433795%2fpandas-merging-two-dataframes-with-different-number-of-multiindices%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Eventually i solved with with removing the multiindex and doing a good old inner join on the two columns and then recreating the multiindex at the end.
Here are the sniplets:
df=df.reset_index()
df2=df2.reset_index()
df['year']=df['year'].apply(int)
df2['year']=df2['year'].apply(int)
df['week']=df['week'].apply(int)
df2['week']=df2['week'].apply(int)
result = pd.merge(df, df2, how='left', left_on= ['year','week'],right_on= ['year','week'])
result=result.set_index(['year', 'month','week','day'])
answered Nov 22 '18 at 15:51
acronis011acronis011
83
Since flattening index is not that intuitive for many, I'd suggest that you could add the actual code that you used, so that others could benefit from your experience ;) stackoverflow.blog/2011/07/01/…
– leoburgy
Nov 22 '18 at 16:08
add a comment |
Eventually i solved with with removing the multiindex and doing a good old inner join on the two columns and then recreating the multiindex at the end.
Here are the sniplets:
df=df.reset_index()
df2=df2.reset_index()
df['year']=df['year'].apply(int)
df2['year']=df2['year'].apply(int)
df['week']=df['week'].apply(int)
df2['week']=df2['week'].apply(int)
result = pd.merge(df, df2, how='left', left_on= ['year','week'],right_on= ['year','week'])
result=result.set_index(['year', 'month','week','day'])
answered Nov 22 '18 at 15:51
acronis011acronis011
83
Since flattening index is not that intuitive for many, I'd suggest that you could add the actual code that you used, so that others could benefit from your experience ;) stackoverflow.blog/2011/07/01/…
– leoburgy
Nov 22 '18 at 16:08
add a comment |
Eventually i solved with with removing the multiindex and doing a good old inner join on the two columns and then recreating the multiindex at the end.
Here are the sniplets:
df=df.reset_index()
df2=df2.reset_index()
df['year']=df['year'].apply(int)
df2['year']=df2['year'].apply(int)
df['week']=df['week'].apply(int)
df2['week']=df2['week'].apply(int)
result = pd.merge(df, df2, how='left', left_on= ['year','week'],right_on= ['year','week'])
result=result.set_index(['year', 'month','week','day'])
answered Nov 22 '18 at 15:51
acronis011acronis011
83
Eventually i solved with with removing the multiindex and doing a good old inner join on the two columns and then recreating the multiindex at the end.
Here are the sniplets:
df=df.reset_index()
df2=df2.reset_index()
df['year']=df['year'].apply(int)
df2['year']=df2['year'].apply(int)
df['week']=df['week'].apply(int)
df2['week']=df2['week'].apply(int)
result = pd.merge(df, df2, how='left', left_on= ['year','week'],right_on= ['year','week'])
result=result.set_index(['year', 'month','week','day'])
answered Nov 22 '18 at 15:51
acronis011acronis011
83
edited Nov 22 '18 at 16:20
answered Nov 22 '18 at 15:51
acronis011acronis011
83
answered Nov 22 '18 at 15:51
acronis011acronis011
83
answered Nov 22 '18 at 15:51
acronis011acronis011
83
83
Since flattening index is not that intuitive for many, I'd suggest that you could add the actual code that you used, so that others could benefit from your experience ;) stackoverflow.blog/2011/07/01/…
– leoburgy
Nov 22 '18 at 16:08
add a comment |
Since flattening index is not that intuitive for many, I'd suggest that you could add the actual code that you used, so that others could benefit from your experience ;) stackoverflow.blog/2011/07/01/…
– leoburgy
Nov 22 '18 at 16:08
Since flattening index is not that intuitive for many, I'd suggest that you could add the actual code that you used, so that others could benefit from your experience ;) stackoverflow.blog/2011/07/01/…
– leoburgy
Nov 22 '18 at 16:08
Since flattening index is not that intuitive for many, I'd suggest that you could add the actual code that you used, so that others could benefit from your experience ;) stackoverflow.blog/2011/07/01/…
– leoburgy
Nov 22 '18 at 16:08
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53433795%2fpandas-merging-two-dataframes-with-different-number-of-multiindices%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
2
Can you provide a sample dataframe via a Minimal, Complete, and Verifiable example.
– jpp
Nov 22 '18 at 15:10
2
Show us some sample of the datasets. Refer to the guide on how to add code snippets.
– Shiv_90
Nov 22 '18 at 15:11
I added some pictures to represent the dataset, i hope it helps!
– acronis011
Nov 22 '18 at 15:22
We can't copy/paste pictures into python :D
– user3471881
Nov 22 '18 at 15:32
1
Did you try
pd.merge([df1, df2], how = 'inner', on = ['year', 'week']?– maow
Nov 22 '18 at 15:38