Principle of marginality

Statistical principle

In statistics, the principle of marginality is the fact that the average (or main) effects of variables in an analysis are marginal to their interaction effect—that is, the main effect of one explanatory variable captures the effect of that variable averaged over all values of a second explanatory variable whose value influences the first variable's effect. The principle of marginality implies that, in general, it is wrong to test, estimate, or interpret main effects of explanatory variables where the variables interact or, similarly, to model interaction effects but delete main effects that are marginal to them.^[1] While such models are interpretable, they lack applicability, as they ignore the dependence of a variable's effect upon another variable's value.

Nelder^[2] and Venables^[3] have argued strongly for the importance of this principle in regression analysis.

Regression form

If two independent continuous variables, say x and z, both influence a dependent variable y, and if the extent of the effect of each independent variable depends on the level of the other independent variable then the regression equation can be written as:

y_{i}=a+bx_{i}+cz_{i}+d(x_{i}z_{i})+e_{i},

where i indexes observations, a is the intercept term, b, c, and d are effect size parameters to be estimated, and e is the error term.

If this is the correct model, then the omission of any of the right-side terms would be incorrect, resulting in misleading interpretation of the regression results.

With this model, the effect of x upon y is given by the partial derivative of y with respect to x; this is $b+dz_{i}$ , which depends on the specific value $z_{i}$ at which the partial derivative is being evaluated. Hence, the main effect of x – the effect averaged over all values of z – is meaningless as it depends on the design of the experiment (specifically on the relative frequencies of the various values of z) and not just on the underlying relationships. Hence:

In the case of interaction, it is wrong to try to test, estimate, or interpret a "main effect" coefficient b or c, omitting the interaction term.^[4]

In addition:

In the case of interaction, it is wrong to not include b or c, because this will give incorrect estimates of the interaction.^[5]^[6]

References

^ Fox, J. Regression Notes.
^ Nelder, J. A. (1977). "A Reformulation of Linear Models". Journal of the Royal Statistical Society. 140 (1): 48–77. doi:10.2307/2344517. JSTOR 2344517. (Section 2.1: The Neglect of Marginality)
^ Venables, W.N. (1998). "Exegeses on Linear Models". Paper presented to the S-PLUS User's Conference Washington, DC, 8–9 October 1998.
^ See Venables, p.13: "... testing main effects in the presence of an interaction is a violation of the marginality principle".
^ See Venables, p.14/15, about the S-Plus command drop1, which does not drop the main effect terms from a model with interaction: "To my delight I see that marginality constraints between factor terms are by default honoured". In R, the marginality requirement of the dropterm function (in package MASS) is stated in the Reference Manual.
^ The above regression model, with two independent continuous variables, is presented with a numerical example, in Stata, as Case 3 in What happens if you omit the main effect in a regression model with an interaction?.