Suppose
\[\begin{eqnarray*} {\bf Y}=\begin{pmatrix} Y_1 \\ Y_2 \\ \vdots \\ Y_n \end{pmatrix} \end{eqnarray*}\]is a vector of random variables with \(E(Y_i)=\mu_i\), \(\text{Var}(Y_i)=\sigma_{ii}\), and \(\text{Cov}(Y_i,Y_j)=\sigma_{ij}\).
The expectation of the random vector \({\bf Y}\) is defined
\[\begin{eqnarray*} E({\bf Y})=\begin{pmatrix} E(Y_1) \\ E(Y_2) \\ \vdots \\ E(Y_n) \end{pmatrix} = \begin{pmatrix} \mu_1 \\ \mu_2 \\ \vdots \\ \mu_n \end{pmatrix} = \mu. \end{eqnarray*}\]Suppose \({\bf Z}\) is an \((n \times p)\) matrix of random variables. Then
\[\begin{eqnarray*} E({\bf Z})=\begin{pmatrix} E(Z_{11}) & \cdots & E(Z_{1p}) \\ \vdots & \cdots & \vdots \\ E(Z_{n1}) & \cdots & E(Z_{np}) \end{pmatrix}. \end{eqnarray*}\]Thus the expectation of a random matrix is the matrix of the expectations.
where \(\sigma_{ij}=E[(Y_i -\mu_i)(Y_j-\mu_j)]\), \(i,j=1,\ldots,n\).
Suppose \({\bf Y}_{n \times 1}\) is a random vector with mean \(\mu=E({\bf Y})\) and covariance matrix \(\Sigma=\text{Cov}({\bf Y})\). In addition, suppose \({\bf A}_{r \times n}\) is a matrix of constants and \({\bf b}_{r \times 1}\) is a vector of constants. Then
\[\begin{eqnarray*} E({\bf A Y} + {\bf b}) = {\bf A} E({\bf Y}) + {\bf b} = {\bf A} \mu + {\bf b} \end{eqnarray*}\] and \[\begin{eqnarray*} \text{Cov}({\bf A Y} + {\bf b}) = {\bf A} \text{Cov}({\bf Y}) {\bf A}' = {\bf A} \Sigma {\bf A}'. \end{eqnarray*}\]Let \({\bf W}_{r \times 1}\) be a random vector with \(E({\bf W})=\gamma\). Then
\[\begin{eqnarray*} \text{Cov}({\bf W},{\bf Y}) = E \left[ ({\bf W}-\gamma)({\bf Y}-\mu)' \right], \end{eqnarray*}\]where \(\text{Cov}({\bf W},{\bf Y})\) is an \((r \times n)\) matrix of covariances with \(ij^{th}\) element equal to \(\text{Cov}(W_i,Y_j)\).