Estimability of Contrasts in Two Way ANOVA |

Estimability of Contrasts in Two Way ANOVA |

January 6th 2021

The most mathematically beautiful yet practically obsolete topic of all might be the estimability in statistics. While going through a bunch of linear models practice problems, I’ve encountered the following question:

Let $Y_{ij} = \mu + \tau_i + \theta_j + \epsilon_{ij}$, where $\epsilon_{ij} \sim \mathcal{N}(0,\sigma^2)$, $i=1,\ldots,a$, $j=1,\ldots,b$. Let $\boldsymbol{\beta} = (\mu,\tau_1,\ldots,\tau_a,\theta_1,\ldots,\theta_b)^\prime$. For any $\ell=1,\ldots,a-1$, define $$\boldsymbol{c}_{\ell}^\prime \boldsymbol{\beta} = \left(\sum_{i=1}^\ell \tau_i - \ell \tau_{\ell+1} \right)/\sqrt{\ell(\ell+1)}.$$ Verify that $\boldsymbol{c}_{\ell}^\prime \boldsymbol{\beta}$ is estimable.

To begin with, note that any linear combination of $\mu + \tau_i +\theta_j$ is estimable since $\boldsymbol{X}\boldsymbol{\beta}$ is always estimable. So rewriting the “contrast” as $$\boldsymbol{c}_\ell^\prime\boldsymbol{\beta} = \left(\sum_{i=1}^\ell(\mu+\tau_i + \theta_j)-\ell(\mu+\tau_i+\theta_j) \right)/\sqrt{\ell(\ell+1)}$$ clearly reveals that it is of the form $\boldsymbol{t}^\prime \boldsymbol{X}\boldsymbol{\beta}$.

Another interesting, and more general, way of approaching this problem is to write the model in a matrix-vector form. That is, $$ \boldsymbol{X}\boldsymbol{\beta} = \begin{pmatrix}\boldsymbol{1}_b & \boldsymbol{1}_b & \cdots & \cdots & \cdots & \boldsymbol{I}_b\\ \boldsymbol{1}_b & \cdots & \boldsymbol{1}_b & \cdots & & \boldsymbol{I}_b \\ \vdots & & & \ddots & & \vdots \\ \boldsymbol{1}_b & \cdots & \cdots & \cdots & \boldsymbol{1}_b & \boldsymbol{I}_b \end{pmatrix}\begin{pmatrix}\mu \\ \tau_1 \\ \vdots \\ \tau_a \\ \theta_1 \\ \vdots \\ \theta_b \end{pmatrix}.$$ Note that $\boldsymbol{X}$ has rank $a + b - 1$ and therefore $\mathrm{dim}(\mathcal{N}(\boldsymbol{X}))=2$. Due to the estimability condition which states $\boldsymbol{c}$ must be in $\mathcal{R}(\boldsymbol{X})$, it implies that $\boldsymbol{c} \in \mathcal{N}^\perp(\boldsymbol{X})$. Thus, we can find a basis $(\boldsymbol{b}_1,\boldsymbol{b}_2)$ for $\mathcal{N}(\boldsymbol{X})$ and show that $\boldsymbol{c}^\prime \boldsymbol{b}_j=0$ for $j=1,2$.

An obvious basis for $\mathcal{N}(\boldsymbol{X})$ is $$\left\{\begin{pmatrix}1 \\ -\boldsymbol{1}_a \\ \boldsymbol{0}_b \end{pmatrix}, \begin{pmatrix}1 \\ \boldsymbol{0}_a \\ -\boldsymbol{1}_b \end{pmatrix} \right\}.$$ Thus, $\boldsymbol{c}^\prime \boldsymbol{\beta}$ is estimable if and only if $c_0 - \sum_{i=1}^a c_i = 0$ and $c_0 - \sum_{j=1}^b c_{a+j}=0$. This includes

  • grand mean - $\mu + \bar{\tau}_\cdot + \bar{\theta}_\cdot$
  • cell means - $\mu + \tau_i + \theta_j$
  • first-factor contrasts - $\sum_{i=1}^a c_i \tau_i$ where $\sum_{i=1}^a c_i = 0$
  • second-factor contrasts - $\sum_{j=1}^b d_j \theta_j$ where $\sum_{j=1}^b d_j = 0$.

The contrast we have as $\boldsymbol{c}_\ell^\prime\boldsymbol{\beta}$ falls into the third category, and thus is estimable.

Share this article:

divider graphic
arrow-up icon