A linear mixed effects model (LMM) is a statistical technique that allows us to look at the effect of one or more factors on an outcome variable while taking into account natural groupings of the data points.
Why do we need to use an LMM in this experiment?
In this study, participants completed several trials in the memory control phase. In each trial, they attempted to control the retrieval of a specific scene-face or scene-tool pair. Similarly, in the trials in final memory test, participants recalled every pair that had been previously suppressed or substituted. We were interested in whether the success of controlling the retrieval of a given pair made it harder to recognise that specific pair.
Since trials are grouped by participants, participant-level factors such as fatigue, general memory ability, and interest can impact both success of retrieval control and final memory performance on a trial-by-trial basis. Thus, we used an LMM to examine how success of retrieval control impacts later memory for controlled information, while accounting for these participant-level factors. Further, using an LMM allowed us to examine whether success of retrieval control interacts with condition (Retrieve, Suppress and Substitute), and object category (Face, Tool) in impacting memory performance.
For this purpose, the amount of time people spent looking at the retrieved object in the memory control phase was used as a proxy for the success of retrieval control, and the speed with which they made their memory response (reaction time; RT) in the final test phase was used as a measure of how well they remembered the pair.
How did we apply an LMM to reaction time data?
The final model contained four explanatory (or independent) variables: the amount of viewing directed to the retrieved information, condition, object category, and participant. Of these, the first three were variables of interest, and were entered as fixed effects in the model. And to account for participant-level grouping of trials, participant was added as a random intercept. Finally, RT was used as the outcome (or dependent) variable. One caveat, however, is that LMMs assume that the outcome variable is normally distributed. But RT typically follows an exponential distribution, where most responses are made closer to the onset of the stimulus.
To model a non-normally distributed variable we can use a generalised linear mixed effects model (GLMM) which allows us to specify the distribution of the outcome variable. Since RT follows an exponential distribution, we can use the log link function by specifying family = Gamma(link = log) in the glmer call.
See R code
library(tidyr)
library(lme4)
# finaltest_search_data contains accuracy and RT data from the final memory
# test phase. The dataframe contains one row per trial, per participant.
# and the following columns:
# Participant: Participant identifier (sub-001, sub-002...; categorical)
# Trial: Trial identifier (1, 2, 3...; continuous)
# Condition: Trial condition (Retrieve, Suppress, Substitute; categorical)
# ObjectCategory: Category of object in scene-object pair (Face, Tool; categorical)
# Accuracy: Response accuracy for each trial (Correct, Incorrect; categorical)
# RT: Reaction time for each trial (continuous)
# Setup factors
finaltest_search_data <- finaltest_search_data %>%
mutate(Condition = factor(Condition,
levels = c("Retrieve", "Suppress", "Substitute")),
ObjectCategory = factor(ObjectCategory,
levels = c("Face", "Tool")))
# Condition is already dummy-coded. Effect-code ObjectCategory
contrasts(finaltest_search_data$ObjectCategory) <- rbind(-1, 1)
viewing_rt_glmm <- glmer(
formula = RT ~ Viewing * Condition * ObjectCategory + (1|Participant),
data = finaltest_search_data,
family = Gamma(link = log))
# Look at the results
summary(viewing_rt_glmm)
# Fixed effects:
# Estimate Std. Error t value Pr(>|z|)
# (Intercept) 6.787313 0.071580 94.821 < 2e-16 ***
# Viewing -0.539003 0.101253 -5.323 1.02e-07 ***
# conditionSubstitute -0.004900 0.046480 -0.105 0.916048
# conditionSuppress -0.078929 0.047929 -1.647 0.099603 .
# objectcategoryTool 0.116183 0.035002 3.319 0.000902 ***
# Viewing:conditionSubstitute 0.054696 0.246871 0.222 0.824660
# Viewing:conditionSuppress 0.218305 0.252242 0.865 0.386787
# Viewing:objectcategoryTool 0.123628 0.094944 1.302 0.192877
# conditionSubstitute:objectcategoryTool 0.072982 0.045658 1.598 0.109941
# conditionSuppress:objectcategoryTool 0.066695 0.046990 1.419 0.155798
# Viewing:conditionSubstitute:objectcategoryTool -0.099868 0.243637 -0.410 0.681875
# Viewing:conditionSuppress:objectcategoryTool -0.003364 0.249036 -0.014 0.989221
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
How do we interpret this output?
This table tells us that, firstly, viewing is negatively associated with RT. This is indicated by the negative estimate for Viewing (-0.54). Importantly, because Condition is dummy-coded, this row refers to the association between Viewing and RT in the reference condition (i.e., when Condition = 0). In this case, this refers to the Retrieve condition. In order to examine whether the relationship between Viewing and RT in the Suppress and Substitute conditions is different from the one in the Retrieve condition, we can look at the beta estimate for the interaction: Viewing:conditionSuppress and Viewing:conditionSubstitute. Since these beta estimates are relatively small and not significant (p > .05), we can conclude that the association between Viewing and RT is similar across all three conditions.
This is also evident when we plot this effect for the three conditions.
How can we model binary outcome variables?
A GLMM can also be used to model binary outcome variables. For instance, in our experiment, trial-by-trial response accuracy in the final memory test (i.e., whether or not participants selected the correct face or tool) was also a measure of memory performance. But accuracy is a categorical variable with two levels - correct and incorrect. In order to use it as an outcome variable in a GLMM, we can specify family = binomial(link = "logit") in the glmer call.
We will relevel accuracy (Incorrect = 0, Correct = 1) to simplify the interpretation of the results. This way, we can talk about in increase in Viewing being associated with an increase in the probability of a correct response.
See R code
library(tidyr)
library(lme4)
# We will use the same dataframe that was used for the RT analysis
# Relevel the outcome variable
finaltest_search_data <- finaltest_search_data %>%
mutate(Accuracy = factor(Accuracy,
levels = c("Incorrect", "Correct")))
viewing_acc_glmm <- glmer(
formula = Accuracy ~ Viewing * Condition * ObjectCategory + (1|Participant),
data = finaltest_search_data,
family = binomial(link = "logit"))
# Look at the results
summary(viewing_acc_glmm)
# Fixed effects:
# Estimate Std. Error z value Pr(>|z|)
# (Intercept) 1.175213 0.189618 6.198 5.73e-10 ***
# Viewing 3.124686 0.466927 6.692 2.20e-11 ***
# conditionSubstitute 0.709183 0.148835 4.765 1.89e-06 ***
# conditionSuppress 0.463247 0.151930 3.049 0.002295 **
# objectcategoryTool -0.405090 0.108650 -3.728 0.000193 ***
# Viewing:conditionSubstitute -4.668298 0.757301 -6.164 7.08e-10 ***
# Viewing:conditionSuppress -2.220972 0.897762 -2.474 0.013365 *
# Viewing:objectcategoryTool 0.540910 0.459706 1.177 0.239338
# conditionSubstitute:objectcategoryTool 0.007313 0.147398 0.050 0.960428
# conditionSuppress:objectcategoryTool -0.095530 0.150428 -0.635 0.525391
# Viewing:conditionSubstitute:objectcategoryTool -0.656521 0.751490 -0.874 0.382322
# Viewing:conditionSuppress:objectcategoryTool -0.482107 0.892988 -0.540 0.589279
# ---
# Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
The positive beta weight for Viewing here (3.12) tells us that there is a positive association between Viewing and Accuracy - as viewing of the face/tool increases, so does the probability of an accurate response. As before, this refers to the relationship in the Retrieve condition. However, here, the interaction with the Suppress and Substitute conditions is significant (p's ≤ .01). The negative beta weights for Viewing:conditionSuppress and Viewing:conditionSubstitute indicate that the relationship between Viewing and Accuracy in these conditions is significantly more negative relative to the Retrieve condition.
Importantly, this does not tell us whether or not the relationship itself is significant in the Suppress and Substitute conditions - just that it is significantly different from the relationship in the Retrieve condition. To answer this question, we need to directly test whether the slope for the association between Viewing and Accuracy is significantly different from 0. There are several ways to do this. Here I'm using the sim_slopes function from the interactions package
See R code
library(interactions)
viewing_acc_simslopes <- sim_slopes(
viewing_acc_glmm,
pred = Viewing,
modx = Condition)
# Look at the results
viewing_acc_simslopes
# SIMPLE SLOPES ANALYSIS
# Slope of Viewing when Condition = Suppress:
#
# Est. S.E. z val. p
# ------ ------ -------- ------
# 0.90 0.77 1.17 0.24
#
# Slope of Viewing when Condition = Substitute:
#
# Est. S.E. z val. p
# ------- ------ -------- ------
# -1.54 0.60 -2.59 0.01
#
# Slope of Viewing when Condition = Retrieve:
#
# Est. S.E. z val. p
# ------ ------ -------- ------
# 3.12 0.47 6.70 0.00
Based on this output, we can see that in the Suppress condition, there is a weak, positive, but non-significant association between Viewing and Accuracy (beta = 0.90). In the Substitute condition, there is a significant negative association (beta = -1.54). Note that the beta value in the Retrieve condition in the simple slopes analysis is the same as the one for the Viewing parameter in the original GLMM results (3.12). This is because, as mentioned above, result for Viewing represents the relationship of Viewing with Accuracy in the reference (i.e., Retrieve) condition.
These results are evident when we plot the model.