Covariance structure with one or more random intercepts
up vote
3
down vote
favorite
I need to choose a suitable covariance structure including one or more random intercepts:
If one factor B
is nested in another factor A
, one option would of course be to consider a model with random intercepts of both A
and B
, corresponding to (1|A) + (1|B)
when using lmer
in the R package lme4
or providing the argument random = ~1|A/B
to the lme
function in the R package nlme
, but would it make any sense to consider a model with only the random intercept of B
, even if B
is nested in A
?
r regression mixed-model lme4-nlme nested-data
add a comment |
up vote
3
down vote
favorite
I need to choose a suitable covariance structure including one or more random intercepts:
If one factor B
is nested in another factor A
, one option would of course be to consider a model with random intercepts of both A
and B
, corresponding to (1|A) + (1|B)
when using lmer
in the R package lme4
or providing the argument random = ~1|A/B
to the lme
function in the R package nlme
, but would it make any sense to consider a model with only the random intercept of B
, even if B
is nested in A
?
r regression mixed-model lme4-nlme nested-data
add a comment |
up vote
3
down vote
favorite
up vote
3
down vote
favorite
I need to choose a suitable covariance structure including one or more random intercepts:
If one factor B
is nested in another factor A
, one option would of course be to consider a model with random intercepts of both A
and B
, corresponding to (1|A) + (1|B)
when using lmer
in the R package lme4
or providing the argument random = ~1|A/B
to the lme
function in the R package nlme
, but would it make any sense to consider a model with only the random intercept of B
, even if B
is nested in A
?
r regression mixed-model lme4-nlme nested-data
I need to choose a suitable covariance structure including one or more random intercepts:
If one factor B
is nested in another factor A
, one option would of course be to consider a model with random intercepts of both A
and B
, corresponding to (1|A) + (1|B)
when using lmer
in the R package lme4
or providing the argument random = ~1|A/B
to the lme
function in the R package nlme
, but would it make any sense to consider a model with only the random intercept of B
, even if B
is nested in A
?
r regression mixed-model lme4-nlme nested-data
r regression mixed-model lme4-nlme nested-data
asked Dec 15 at 19:09
user3419936
255
255
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
up vote
3
down vote
accepted
Let consider 2 levels in factor A and 2 individuals in each level of factor A, so totally 4 individuals. $i$ indicates level of factor A and $j$ for B.
When the two intercepts are used, we have
$$Y_{ij} = Xbeta + gamma_i + gamma_j + epsilon_{ij}$$
$Var(gamma_i) = sigma_A^2, Var(gamma_j) = sigma_B^2, Var(epsilon) = sigma^2$
Then $$Varleft(begin{matrix}Y_{11}\ Y_{12}\Y_{21}\Y_{22}end{matrix}right)=
left(begin{matrix}sigma^2+sigma_A^2 + sigma_B^2 & sigma_A^2 + sigma_B^2 & sigma_A^2&sigma_A^2 \ sigma_A^2 + sigma_B^2 & sigma^2+sigma_A^2 + sigma_B^2 & sigma_A^2&sigma_A^2 \ sigma_A^2 & sigma_A^2 & sigma^2+sigma_A^2 + sigma_B^2 &sigma_A^2 + sigma_B^2\ sigma_A^2 & sigma_A^2 & sigma_A^2 + sigma_B^2&sigma^2+sigma_A^2 + sigma_B^2 end{matrix}right)
$$
If intercept for factor B only, we have
$$Y_{ij} = Xbeta + gamma_j + epsilon_{ij}$$
Then $$Varleft(begin{matrix}Y_{11}\ Y_{12}\Y_{21}\Y_{22}end{matrix}right)=
left(begin{matrix}sigma^2 + sigma_B^2 & sigma_B^2 & 0&0\ sigma_B^2 & sigma^2+ sigma_B^2 & 0&0\ 0&0& sigma^2+ sigma_B^2 & sigma_B^2\ 0 & 0& sigma_B^2&sigma^2 + sigma_B^2 end{matrix}right)
$$
When two intercepts are in the model, $Y$ from the same individual has covariance $sigma_A^2+sigma_B^2$, $Y$ from different individuals but from same level of factor A has covariance $sigma_A^2$, which is smaller than covariance from the same individual. Generally this assumption is reasonable.
When just individual level intercept is kept in the model, the covariance between the difference individual is zero, even they come from the same level of factor A. Generally, this is not what we want.
add a comment |
up vote
3
down vote
In general I would recommend including both (1|A)
and (1|B)
in the model - otherwise you're not fitting the maximal model, i.e. the model that includes all terms that are statistically identifiable. If the variation in A
is reasonably large and you omit the (1|A)
term, the model for the random effects corresponding to the remaining (1|B)
term (that they are independently drawn from a zero-centred Normal distribution) might be off. Some reasons it might be reasonable to omit the (1|A)
term:
- there is something about your data that would eliminate any differences in
A
(not just experimental control: for example, maybe you've already subtracted the mean from the observations in eachA
group) - you get a singular fit, i.e. an estimate of zero variance among levels of
A
(in this case you'll get exactly the same answer if you drop this term) - you think there are too few levels of
A
to estimate the variance (in this case it would be more conservative to includeA
in the model as a fixed effect
if you are using data-driven model selection to decide what terms to include, and the model including(1|A)
is not selected (e.g. the AIC is higher). (In general you need to be very careful when doing this kind of "sacrificial pseudoreplication" (sensu Hurlbert 1984) that you don't mess up your coverage/type-I error ...)
(Note that the levels of B
need to be uniquely coded - otherwise you should specify the nested syntax (1|A/B)
(or (1|A) + (1|A:B)
) in lme4
as well in nlme
...)
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "65"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f383156%2fcovariance-structure-with-one-or-more-random-intercepts%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
3
down vote
accepted
Let consider 2 levels in factor A and 2 individuals in each level of factor A, so totally 4 individuals. $i$ indicates level of factor A and $j$ for B.
When the two intercepts are used, we have
$$Y_{ij} = Xbeta + gamma_i + gamma_j + epsilon_{ij}$$
$Var(gamma_i) = sigma_A^2, Var(gamma_j) = sigma_B^2, Var(epsilon) = sigma^2$
Then $$Varleft(begin{matrix}Y_{11}\ Y_{12}\Y_{21}\Y_{22}end{matrix}right)=
left(begin{matrix}sigma^2+sigma_A^2 + sigma_B^2 & sigma_A^2 + sigma_B^2 & sigma_A^2&sigma_A^2 \ sigma_A^2 + sigma_B^2 & sigma^2+sigma_A^2 + sigma_B^2 & sigma_A^2&sigma_A^2 \ sigma_A^2 & sigma_A^2 & sigma^2+sigma_A^2 + sigma_B^2 &sigma_A^2 + sigma_B^2\ sigma_A^2 & sigma_A^2 & sigma_A^2 + sigma_B^2&sigma^2+sigma_A^2 + sigma_B^2 end{matrix}right)
$$
If intercept for factor B only, we have
$$Y_{ij} = Xbeta + gamma_j + epsilon_{ij}$$
Then $$Varleft(begin{matrix}Y_{11}\ Y_{12}\Y_{21}\Y_{22}end{matrix}right)=
left(begin{matrix}sigma^2 + sigma_B^2 & sigma_B^2 & 0&0\ sigma_B^2 & sigma^2+ sigma_B^2 & 0&0\ 0&0& sigma^2+ sigma_B^2 & sigma_B^2\ 0 & 0& sigma_B^2&sigma^2 + sigma_B^2 end{matrix}right)
$$
When two intercepts are in the model, $Y$ from the same individual has covariance $sigma_A^2+sigma_B^2$, $Y$ from different individuals but from same level of factor A has covariance $sigma_A^2$, which is smaller than covariance from the same individual. Generally this assumption is reasonable.
When just individual level intercept is kept in the model, the covariance between the difference individual is zero, even they come from the same level of factor A. Generally, this is not what we want.
add a comment |
up vote
3
down vote
accepted
Let consider 2 levels in factor A and 2 individuals in each level of factor A, so totally 4 individuals. $i$ indicates level of factor A and $j$ for B.
When the two intercepts are used, we have
$$Y_{ij} = Xbeta + gamma_i + gamma_j + epsilon_{ij}$$
$Var(gamma_i) = sigma_A^2, Var(gamma_j) = sigma_B^2, Var(epsilon) = sigma^2$
Then $$Varleft(begin{matrix}Y_{11}\ Y_{12}\Y_{21}\Y_{22}end{matrix}right)=
left(begin{matrix}sigma^2+sigma_A^2 + sigma_B^2 & sigma_A^2 + sigma_B^2 & sigma_A^2&sigma_A^2 \ sigma_A^2 + sigma_B^2 & sigma^2+sigma_A^2 + sigma_B^2 & sigma_A^2&sigma_A^2 \ sigma_A^2 & sigma_A^2 & sigma^2+sigma_A^2 + sigma_B^2 &sigma_A^2 + sigma_B^2\ sigma_A^2 & sigma_A^2 & sigma_A^2 + sigma_B^2&sigma^2+sigma_A^2 + sigma_B^2 end{matrix}right)
$$
If intercept for factor B only, we have
$$Y_{ij} = Xbeta + gamma_j + epsilon_{ij}$$
Then $$Varleft(begin{matrix}Y_{11}\ Y_{12}\Y_{21}\Y_{22}end{matrix}right)=
left(begin{matrix}sigma^2 + sigma_B^2 & sigma_B^2 & 0&0\ sigma_B^2 & sigma^2+ sigma_B^2 & 0&0\ 0&0& sigma^2+ sigma_B^2 & sigma_B^2\ 0 & 0& sigma_B^2&sigma^2 + sigma_B^2 end{matrix}right)
$$
When two intercepts are in the model, $Y$ from the same individual has covariance $sigma_A^2+sigma_B^2$, $Y$ from different individuals but from same level of factor A has covariance $sigma_A^2$, which is smaller than covariance from the same individual. Generally this assumption is reasonable.
When just individual level intercept is kept in the model, the covariance between the difference individual is zero, even they come from the same level of factor A. Generally, this is not what we want.
add a comment |
up vote
3
down vote
accepted
up vote
3
down vote
accepted
Let consider 2 levels in factor A and 2 individuals in each level of factor A, so totally 4 individuals. $i$ indicates level of factor A and $j$ for B.
When the two intercepts are used, we have
$$Y_{ij} = Xbeta + gamma_i + gamma_j + epsilon_{ij}$$
$Var(gamma_i) = sigma_A^2, Var(gamma_j) = sigma_B^2, Var(epsilon) = sigma^2$
Then $$Varleft(begin{matrix}Y_{11}\ Y_{12}\Y_{21}\Y_{22}end{matrix}right)=
left(begin{matrix}sigma^2+sigma_A^2 + sigma_B^2 & sigma_A^2 + sigma_B^2 & sigma_A^2&sigma_A^2 \ sigma_A^2 + sigma_B^2 & sigma^2+sigma_A^2 + sigma_B^2 & sigma_A^2&sigma_A^2 \ sigma_A^2 & sigma_A^2 & sigma^2+sigma_A^2 + sigma_B^2 &sigma_A^2 + sigma_B^2\ sigma_A^2 & sigma_A^2 & sigma_A^2 + sigma_B^2&sigma^2+sigma_A^2 + sigma_B^2 end{matrix}right)
$$
If intercept for factor B only, we have
$$Y_{ij} = Xbeta + gamma_j + epsilon_{ij}$$
Then $$Varleft(begin{matrix}Y_{11}\ Y_{12}\Y_{21}\Y_{22}end{matrix}right)=
left(begin{matrix}sigma^2 + sigma_B^2 & sigma_B^2 & 0&0\ sigma_B^2 & sigma^2+ sigma_B^2 & 0&0\ 0&0& sigma^2+ sigma_B^2 & sigma_B^2\ 0 & 0& sigma_B^2&sigma^2 + sigma_B^2 end{matrix}right)
$$
When two intercepts are in the model, $Y$ from the same individual has covariance $sigma_A^2+sigma_B^2$, $Y$ from different individuals but from same level of factor A has covariance $sigma_A^2$, which is smaller than covariance from the same individual. Generally this assumption is reasonable.
When just individual level intercept is kept in the model, the covariance between the difference individual is zero, even they come from the same level of factor A. Generally, this is not what we want.
Let consider 2 levels in factor A and 2 individuals in each level of factor A, so totally 4 individuals. $i$ indicates level of factor A and $j$ for B.
When the two intercepts are used, we have
$$Y_{ij} = Xbeta + gamma_i + gamma_j + epsilon_{ij}$$
$Var(gamma_i) = sigma_A^2, Var(gamma_j) = sigma_B^2, Var(epsilon) = sigma^2$
Then $$Varleft(begin{matrix}Y_{11}\ Y_{12}\Y_{21}\Y_{22}end{matrix}right)=
left(begin{matrix}sigma^2+sigma_A^2 + sigma_B^2 & sigma_A^2 + sigma_B^2 & sigma_A^2&sigma_A^2 \ sigma_A^2 + sigma_B^2 & sigma^2+sigma_A^2 + sigma_B^2 & sigma_A^2&sigma_A^2 \ sigma_A^2 & sigma_A^2 & sigma^2+sigma_A^2 + sigma_B^2 &sigma_A^2 + sigma_B^2\ sigma_A^2 & sigma_A^2 & sigma_A^2 + sigma_B^2&sigma^2+sigma_A^2 + sigma_B^2 end{matrix}right)
$$
If intercept for factor B only, we have
$$Y_{ij} = Xbeta + gamma_j + epsilon_{ij}$$
Then $$Varleft(begin{matrix}Y_{11}\ Y_{12}\Y_{21}\Y_{22}end{matrix}right)=
left(begin{matrix}sigma^2 + sigma_B^2 & sigma_B^2 & 0&0\ sigma_B^2 & sigma^2+ sigma_B^2 & 0&0\ 0&0& sigma^2+ sigma_B^2 & sigma_B^2\ 0 & 0& sigma_B^2&sigma^2 + sigma_B^2 end{matrix}right)
$$
When two intercepts are in the model, $Y$ from the same individual has covariance $sigma_A^2+sigma_B^2$, $Y$ from different individuals but from same level of factor A has covariance $sigma_A^2$, which is smaller than covariance from the same individual. Generally this assumption is reasonable.
When just individual level intercept is kept in the model, the covariance between the difference individual is zero, even they come from the same level of factor A. Generally, this is not what we want.
edited Dec 15 at 20:31
answered Dec 15 at 19:47
user158565
5,1801318
5,1801318
add a comment |
add a comment |
up vote
3
down vote
In general I would recommend including both (1|A)
and (1|B)
in the model - otherwise you're not fitting the maximal model, i.e. the model that includes all terms that are statistically identifiable. If the variation in A
is reasonably large and you omit the (1|A)
term, the model for the random effects corresponding to the remaining (1|B)
term (that they are independently drawn from a zero-centred Normal distribution) might be off. Some reasons it might be reasonable to omit the (1|A)
term:
- there is something about your data that would eliminate any differences in
A
(not just experimental control: for example, maybe you've already subtracted the mean from the observations in eachA
group) - you get a singular fit, i.e. an estimate of zero variance among levels of
A
(in this case you'll get exactly the same answer if you drop this term) - you think there are too few levels of
A
to estimate the variance (in this case it would be more conservative to includeA
in the model as a fixed effect
if you are using data-driven model selection to decide what terms to include, and the model including(1|A)
is not selected (e.g. the AIC is higher). (In general you need to be very careful when doing this kind of "sacrificial pseudoreplication" (sensu Hurlbert 1984) that you don't mess up your coverage/type-I error ...)
(Note that the levels of B
need to be uniquely coded - otherwise you should specify the nested syntax (1|A/B)
(or (1|A) + (1|A:B)
) in lme4
as well in nlme
...)
add a comment |
up vote
3
down vote
In general I would recommend including both (1|A)
and (1|B)
in the model - otherwise you're not fitting the maximal model, i.e. the model that includes all terms that are statistically identifiable. If the variation in A
is reasonably large and you omit the (1|A)
term, the model for the random effects corresponding to the remaining (1|B)
term (that they are independently drawn from a zero-centred Normal distribution) might be off. Some reasons it might be reasonable to omit the (1|A)
term:
- there is something about your data that would eliminate any differences in
A
(not just experimental control: for example, maybe you've already subtracted the mean from the observations in eachA
group) - you get a singular fit, i.e. an estimate of zero variance among levels of
A
(in this case you'll get exactly the same answer if you drop this term) - you think there are too few levels of
A
to estimate the variance (in this case it would be more conservative to includeA
in the model as a fixed effect
if you are using data-driven model selection to decide what terms to include, and the model including(1|A)
is not selected (e.g. the AIC is higher). (In general you need to be very careful when doing this kind of "sacrificial pseudoreplication" (sensu Hurlbert 1984) that you don't mess up your coverage/type-I error ...)
(Note that the levels of B
need to be uniquely coded - otherwise you should specify the nested syntax (1|A/B)
(or (1|A) + (1|A:B)
) in lme4
as well in nlme
...)
add a comment |
up vote
3
down vote
up vote
3
down vote
In general I would recommend including both (1|A)
and (1|B)
in the model - otherwise you're not fitting the maximal model, i.e. the model that includes all terms that are statistically identifiable. If the variation in A
is reasonably large and you omit the (1|A)
term, the model for the random effects corresponding to the remaining (1|B)
term (that they are independently drawn from a zero-centred Normal distribution) might be off. Some reasons it might be reasonable to omit the (1|A)
term:
- there is something about your data that would eliminate any differences in
A
(not just experimental control: for example, maybe you've already subtracted the mean from the observations in eachA
group) - you get a singular fit, i.e. an estimate of zero variance among levels of
A
(in this case you'll get exactly the same answer if you drop this term) - you think there are too few levels of
A
to estimate the variance (in this case it would be more conservative to includeA
in the model as a fixed effect
if you are using data-driven model selection to decide what terms to include, and the model including(1|A)
is not selected (e.g. the AIC is higher). (In general you need to be very careful when doing this kind of "sacrificial pseudoreplication" (sensu Hurlbert 1984) that you don't mess up your coverage/type-I error ...)
(Note that the levels of B
need to be uniquely coded - otherwise you should specify the nested syntax (1|A/B)
(or (1|A) + (1|A:B)
) in lme4
as well in nlme
...)
In general I would recommend including both (1|A)
and (1|B)
in the model - otherwise you're not fitting the maximal model, i.e. the model that includes all terms that are statistically identifiable. If the variation in A
is reasonably large and you omit the (1|A)
term, the model for the random effects corresponding to the remaining (1|B)
term (that they are independently drawn from a zero-centred Normal distribution) might be off. Some reasons it might be reasonable to omit the (1|A)
term:
- there is something about your data that would eliminate any differences in
A
(not just experimental control: for example, maybe you've already subtracted the mean from the observations in eachA
group) - you get a singular fit, i.e. an estimate of zero variance among levels of
A
(in this case you'll get exactly the same answer if you drop this term) - you think there are too few levels of
A
to estimate the variance (in this case it would be more conservative to includeA
in the model as a fixed effect
if you are using data-driven model selection to decide what terms to include, and the model including(1|A)
is not selected (e.g. the AIC is higher). (In general you need to be very careful when doing this kind of "sacrificial pseudoreplication" (sensu Hurlbert 1984) that you don't mess up your coverage/type-I error ...)
(Note that the levels of B
need to be uniquely coded - otherwise you should specify the nested syntax (1|A/B)
(or (1|A) + (1|A:B)
) in lme4
as well in nlme
...)
answered Dec 16 at 0:46
Ben Bolker
22.1k15989
22.1k15989
add a comment |
add a comment |
Thanks for contributing an answer to Cross Validated!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f383156%2fcovariance-structure-with-one-or-more-random-intercepts%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown