5.5 Exercises

  1. Attempt exam question 1 part (b) from the 2017-18 exam paper. You will find the past exam papers on Moodle.

  2. Suppose that \(\mathbf z= (\mathbf x^\top \mathbf y^\top)^\top\) is a random vector, where both \(\mathbf x\) and \(\mathbf y\) are sub-vectors of dimension \(p\), so that \(\mathbf z\) is \((2p)\times 1\). Define \[{\mathbb{V}\operatorname{ar}}(\mathbf z)=\boldsymbol{\Sigma}_{\mathbf z\mathbf z}=\begin{pmatrix} \boldsymbol{\Sigma}_{\mathbf x\mathbf x} & \boldsymbol{\Sigma}_{\mathbf x\mathbf y}\\\boldsymbol{\Sigma}_{\mathbf y\mathbf x} & \boldsymbol{\Sigma}_{\mathbf y\mathbf y} \end{pmatrix}.\]
    1. Suppose that \(\mathbf y= \mathbf T\mathbf x\) where \(\mathbf T\) is a fixed matrix. Find \(\boldsymbol{\Sigma}_{\mathbf x\mathbf y}\) and \(\boldsymbol{\Sigma}_{\mathbf y\mathbf y}\) in terms of \(\boldsymbol{\Sigma}_{\mathbf x\mathbf x}\) and \(\mathbf T\).
    2. Assuming now that \(\mathbf T\) is an orthogonal matrix and \(\boldsymbol{\Sigma}_{\mathbf x\mathbf x}\) is of full rank, determine the singular values of the matrix \(\mathbf Q=\boldsymbol{\Sigma}_{\mathbf x\mathbf x}^{-1/2}\boldsymbol{\Sigma}_ {\mathbf x\mathbf y}\boldsymbol{\Sigma}_{\mathbf y\mathbf y}^{-1/2}\), and hence write down the canonical correlation coefficients.
    3. Suppose now that \(\mathbf T\) is non-singular but not orthogonal. Comment on whether the answer to part (b) changes.
  3. We will now prove Proposition 5.4 by induction. The case for \(k=1\) was proved in Section 5.1 in Proposition 3.9. Assume the result is true for \(k\). Consider the objective \[\mathcal{L} = \mathbf a^\top \mathbf Q\mathbf b+ \sum_{i=1}^k \gamma_i\mathbf a^\top \mathbf a_i + \sum_{i=1}^k \mu_i\mathbf b^\top \mathbf b_i + \frac{\lambda_1}{2}(1-\mathbf a^\top\mathbf a)+ \frac{\lambda_2}{2}(1-\mathbf b^\top\mathbf b)\] where \(\lambda_i, \mu_i, \gamma_i\) are Lagrangian multipliers.

    1. By differentiating with respect to \(\mathbf a\) and \(\mathbf b\) and setting the derivative to zero show that \[\begin{align} \mathbf Q\mathbf b+ \sum\gamma_i \mathbf a_i - \lambda_1 \mathbf a&= 0 \tag{5.14}\\ \mathbf Q^\top\mathbf a+ \sum\mu_i \mathbf b_i - \lambda_2 \mathbf b&= 0. \tag{5.15} \end{align}\]

    2. By left multiplying the equations above by \(\mathbf a^\top\) and \(\mathbf b^\top\) respectively show that \[\lambda_1=\lambda_2 = \mathbf a^\top \mathbf Q\mathbf b.\]

    3. By left multiplying (5.14) by \(\mathbf a_i^\top\) show that \(\gamma_i=0\) for \(i=1, \ldots, k\). Show similarly that \(\mu_i =0\) for \(i=1, \ldots, k\).

    4. Finally, by copying the proof of Proposition 3.9, prove Proposition 5.4.

  4. Show the mean of the cc variables \(\eta_k\) and \(\psi_k\) is zero. Prove Proposition 5.5 giving the variance of covariance the cc variables.