Argthtjtr

I am editing my question as it was not detailed enough. I made an (unsuccessful) shortcut. Sorry, here is the entire story.

In my experiment I test subjects' reactions to some (simulated) situations. The subject read a scenario and then an expert evaluates the subject's behavior. The evaluation ranges from 1 to 5. There are 10 different simulations, and each subject takes all of them; thus, from each subject I have 10 data points. My experiment is went on 30 days. In each day, the same 10 simulations are used. In other words, in each day, subjects and simulations are fully crossed. Each day the simulations are different.

There are 3 categories of simulations (A, B and C). The categories are, from a theoretical perspective, different one from the other. Category A is tested by 3 simulations (a1, a2, a3); B by 3 (b1, b2, b3); C by 4 (c1, c2, c3, c4). a1:c4 at day 1 are different from a1:c4 at day 2 and so on.

The 3 categories are the only ones I am interested in. In that sense, I think they should be treated as fixed effects. Each category is tested/represented by some simulations. Yet, for each category there are an infinity of possible simulations and I just sampled some. In that sense simulations are random.

The only question I am interested in here, is about the effect of the subject gender on the grades. I want to control for all other parameters. My question is how to account for the simulation and category. I would like to extrapolate my results beyond participants and the simulations representing the category. Yet it is also possible that gender would interact with category or simulation.

So here is one "basic" model:

lmer(grade ~ gender + (1|subject) + (1|simulation:day), data = My_data)

Yet, this model does not account for the possibility that gender has a different effect on simulations. So here is another one trying that.

lmer(grade ~ gender + (1|subject) + (1 + gender|simulation:day), data = 

My_data)

And here I get stuck. How does category play a role? Do I need to enter it as fixed effect? If yes, what about simulations? Does the following make sense?

lmer(grade ~ gender*category + (1 + category|subject) + (1 + 

gender|simulation:day), data = My_data)

Or is it better to give up the simulations, and treat category as random? But in this case, for a given day, the same category will appear several time for each subject (e.g., A will appear 3 times). Is't that a problem? As follow:

lmer(grade ~ gender + (1 + subject) + (1 + gender|category), 

data =My_data)

A final point: I have a lot of data (several thousands of participants), so convergence should not be a problem.

Thanks a lot for the help

If you believe the different categories vary in terms of their measurement and you want to test that, you need to model the category as a random intercept with (1|category).

However, I think we need more information as to what you are actually looking to decide. For example, are you wondering if each person measures them different? Or if each stimulus is measured differently?

It seems that in the model:

lmer(measure ~ category + (1|subject) + (1|stimulus), data = My_data)

category is being used to denote the levels of stimulus. As such, this does not make sense, since category is not an actual variable.

Even though stimulus is random in the sense that it has (presumably) been randomly assigned to each subject, this does not mean that is should be included as a random effect - unless there is no interest in the treatment effect of stimulus. In that case, you would simply be partitioning variance into the subject level and the stimulus level.

It seems more likely that you are in fact interested in the associations between each level of the stimulus and the outcome - that is, you are interested in the treatment effect and therefore the model should be of the form:

measure ~ stimulus + (1|subject)

In your setting, subject and stimulus seem to be fully crossed random grouping factors - since each subject sees each stimulus and (I am assuming) you are using the subjects and stimuli included in your studies to represent all the subjects and all the stimuli you wish to generalize your study findings to.

The key word here is grouping - for your model to be a linear mixed effects model (lmer), each subject by stimulus combination should act like a container which holds together a group of values for your measure outcome. All the values of measure that belong to the same container are more similar to each other than values that belong to different containers, as they are subjected to the same subject-level and stimulus-level influences (presuming these influences are constant over time).

The group of values in a specific container could arise, for instance, if you record the value of measure at several time points for each subject by stimulus combination, or under two or more different conditions, etc.

If you only have one value of measure per subject by stimulus combination, then you're dealing with a linear model (lm). There is no grouping of observations according to each subject per stimulus combination, so there are no random grouping factors which means there aren't any effects that can vary randomly across combinations of levels of the grouping factors (i.e., random effects). If there aren't any random effects, there can't be a mixed effects model, as such a model would require both fixed and random effects to be part of it!

If you do have multiple values of measure per container (i.e., subject by stimulus combination), then your model can include subject-level predictors (e.g., subject gender, subject age) and/or stimulus-level predictors (e.g., stimulus category).