The metafor Package

A Meta-Analysis Package for R

User Tools

Site Tools

tips:computing_adjusted_effects

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Last revisionBoth sides next revision
tips:computing_adjusted_effects [2022/08/03 11:32] – Wolfgang Viechtbauertips:computing_adjusted_effects [2022/08/30 10:28] – Wolfgang Viechtbauer
Line 60: Line 60:
 </code> </code>
  
-To make the results easier to interpret, we can exponentiate the estimated average log risk ratio (i.e., $-0.7145$) to obtain an estimate of the average risk ratio.((Strictly speaking, since exponentiation is a non-linear transformation, the back-transformed value is an estimate of the //median// risk ratio, but we will gloss over this fact.)) This can be done with the ''predict()'' function as follows.+To make the results easier to interpret, we can exponentiate the estimated average log risk ratio (i.e., $-0.7145$) to obtain an estimate of the average risk ratio.((Strictly speaking, since exponentiation is a non-linear transformation, the back-transformed value is an estimate of the //median// risk ratio, but we will gloss over this fact for the purposes of this discussion.)) This can be done with the ''predict()'' function as follows.
  
 <code rsplus> <code rsplus>
Line 116: Line 116:
 The results indeed suggest that the size of the average log risk ratio is a function of the absolute latitude of the study locations. The results indeed suggest that the size of the average log risk ratio is a function of the absolute latitude of the study locations.
  
-In such a meta-regression model, there is no longer a single average effect (since it depends on the value of the moderator), so if we want to estimate the size of the effect, we have to specify a value for ''ablat''. One possibility is to estimate the average risk ratio based on the average absolute latitude of the 13 trials included in the meta-analysis. We can do this as follows.+In such a meta-regression model, there is no longer a single average effect (since the effect depends on the value of the moderator), so if we want to estimate the size of the effect, we have to specify a value for ''ablat''. One possibility is to estimate the average risk ratio based on the average absolute latitude of the 13 trials included in the meta-analysis. We can do this as follows.((In general, it might be safer to use ''predict(res2, newmods = mean(model.matrix(res2, asdf=TRUE)$ablat), transf=exp, digits=2)'', which takes the mean of the ''ablat'' values from the model matrix. This approach properly handles possible subsetting and/or removal of studies due to missing values.))
  
 <code rsplus> <code rsplus>
Line 180: Line 180:
 </code> </code>
  
-The intercept reflects the estimated average log risk ratio for level ''alternate'' (the reference level), while the other two coefficients estimate how much the average log risk ratios for levels ''random'' and ''systematic'' differ from the reference level. Note that the omnibus test of these two coefficients is actually not significant ($Q_M = 1.77, \mbox{df} = 2, p = .41$), which indicates that there is insufficient evidence to conclude that the average log risk ratio actually differs across the three levels. However, for illustration purposes, we'll proceed with our analysis of this moderator.+The intercept reflects the estimated average log risk ratio for level ''alternate'' (the reference level), while the other two coefficients estimate how much the average log risk ratios for levels ''random'' and ''systematic'' differ from the reference level. Note that the omnibus test of these two coefficients is actually not significant ($Q_M = 1.77, \mbox{df} = 2, p = .41$), which indicates that there is insufficient evidence to conclude that the average log risk ratio differs across the three levels. However, for illustration purposes, we'll proceed with our analysis of this moderator.
  
 First, we can compute the estimated average risk ratios for the three levels with: First, we can compute the estimated average risk ratios for the three levels with:
Line 198: Line 198:
 Note: The output above was obtained with ''predict(res3, newmods = rbind(c(0,0), c(1,0), c(0,1)), transf=exp, digits=2)'', which provides the three estimates in a single line of code. However, the code above is a bit easier to read and shows how we need to set the two dummy variables (that are created for the ''random'' and ''systematic'' levels) to 0 or 1 to obtain the estimates for the three levels. Note: The output above was obtained with ''predict(res3, newmods = rbind(c(0,0), c(1,0), c(0,1)), transf=exp, digits=2)'', which provides the three estimates in a single line of code. However, the code above is a bit easier to read and shows how we need to set the two dummy variables (that are created for the ''random'' and ''systematic'' levels) to 0 or 1 to obtain the estimates for the three levels.
  
-But what should we do if we want to compute an 'adjusted effect' again? In other words, what values should we plug into our model equation (and hence into ''newmods'') to obtain such an estimate? One common approach is to use the mean of the respective dummy variables. We can obtain these values from the 'model matrix' and taking means across columns (leaving out the intercept).+But what should we do if we want to compute an 'adjusted effect' again? In other words, what values should we plug into our model equation (and hence into ''newmods'') to obtain such an estimate? One approach is to use the mean of the respective dummy variables. We can obtain these values from the 'model matrix' and taking means across columns (leaving out the intercept).
  
 <code rsplus> <code rsplus>
Line 222: Line 222:
 What do these column means above actually represent? They are in fact proportions and indicate that 53.8% of the trials used random allocation (i.e., 7 out of the 13), 30.8% used systematic allocation (i.e., 4 out of the 13), and hence 15.4% used alternating allocation (i.e., 2 out of the 13). The predicted effect computed above is therefore an estimate for a population of studies where the relative frequencies of the different allocation methods are like those observed in the 13 trials included in the meta-analysis. What do these column means above actually represent? They are in fact proportions and indicate that 53.8% of the trials used random allocation (i.e., 7 out of the 13), 30.8% used systematic allocation (i.e., 4 out of the 13), and hence 15.4% used alternating allocation (i.e., 2 out of the 13). The predicted effect computed above is therefore an estimate for a population of studies where the relative frequencies of the different allocation methods are like those observed in the 13 trials included in the meta-analysis.
  
-But we are not restricted to setting the proportions in this way. Another common approach is to assume that the population to which we want to generalize includes studies where each allocation method is used equally often. For a three-level factor, we then need to set the values for the two dummy variables to 1/3. This yields:+But we are not restricted to setting the proportions in this way. Another approach is to assume that the population to which we want to generalize includes studies where each allocation method is used equally often. For a three-level factor, we then need to set the values for the two dummy variables to 1/3. This yields:
  
 <code rsplus> <code rsplus>
Line 280: Line 280:
  
 Some may scoff at the idea of computing such an 'adjusted effect', questioning its usefulness and interpretability. I'll leave this discussion for another day. However, the method above is nothing different than what is used to compute so-called 'marginal means' (or 'least squares means', although that term is a bit outdated). Some may scoff at the idea of computing such an 'adjusted effect', questioning its usefulness and interpretability. I'll leave this discussion for another day. However, the method above is nothing different than what is used to compute so-called 'marginal means' (or 'least squares means', although that term is a bit outdated).
 +
 +==== Addendum: Using the emmeans Package ====
 +
 +The [[https://cran.r-project.org/package=emmeans|emmeans]] package is a popular package that facilitates the computation of such 'estimated marginal means'. The [[:installation#development_version|development version]] of the metafor package provides a wrapper function called ''[[https://wviechtb.github.io/metafor/reference/emmprep.html|emmprep()]]'' that makes it possible to use the emmeans package for computing adjusted effects as shown above. As a first step, let's install and load the emmeans package:
 +
 +<code rsplus>
 +install.packages("emmeans")
 +library(emmeans)
 +</code>
 +
 +For the model that included only ''ablat'' as a predictor, we can now do the following:((With ''type="response"'', the results are automatically back-transformed as we did earlier.))
 +
 +<code rsplus>
 +sav <- emmprep(res2)
 +emmeans(sav, specs="1", type="response")
 +</code>
 +<code output>
 + 1       response     SE  df asymp.LCL asymp.UCL
 + overall    0.486 0.0523 Inf     0.393       0.6
 +
 +Confidence level used: 0.95
 +Intervals are back-transformed from the log scale
 +</code>
 +
 +Similarly, for the model that included the ''alloc'' factor, we can do:
 +
 +<code rsplus>
 +sav <- emmprep(res3)
 +emmeans(sav, specs="1", type="response", weights="proportional")
 +</code>
 +<code output>
 + 1       response    SE  df asymp.LCL asymp.UCL
 + overall    0.481 0.092 Inf     0.331       0.7
 +
 +Results are averaged over the levels of: alloc
 +Confidence level used: 0.95
 +Intervals are back-transformed from the log scale
 +</code>
 +
 +Note that the default is to use equal weighting of the levels that are averaged over, so we had to explicitly set ''weights="proportional"'' to get the proportional weighting we used initially above. Therefore, for equal weighting, we can simply use:
 +
 +<code rsplus>
 +emmeans(sav, specs="1", type="response")
 +</code>
 +<code output>
 + 1       response    SE  df asymp.LCL asymp.UCL
 + overall    0.529 0.109 Inf     0.352     0.793
 +
 +Results are averaged over the levels of: alloc
 +Confidence level used: 0.95
 +Intervals are back-transformed from the log scale
 +</code>
 +
 +Finally, for the model that included both ''ablat'' and ''alloc'' as predictors, we again use:
 +
 +<code rsplus>
 +sav <- emmprep(res4)
 +emmeans(sav, specs="1", type="response", weights="proportional")
 +</code>
 +<code output>
 + 1       response     SE  df asymp.LCL asymp.UCL
 + overall    0.474 0.0643 Inf     0.364     0.619
 +
 +Results are averaged over the levels of: alloc
 +Confidence level used: 0.95
 +Intervals are back-transformed from the log scale
 +</code>
 +
 +Note that these results are identical to the ones we obtained earlier using the ''predict()'' function. However, some of the work in having to set quantitative variables to their mean value and dummy variables to their respective proportions or fractions is automatically handled when using ''emmprep()'' in combination with ''emmeans()''.
  
 ==== References ==== ==== References ====
tips/computing_adjusted_effects.txt · Last modified: 2023/09/11 16:10 by Wolfgang Viechtbauer