Moment generating function?
I’m trying to upper bound the norm of the pseudoinverse of a product of a fat matrix with orthogonal rows with a tall random matrix with independent entries: 
Since 
, this is equivalent to lower bounding 
of the symmetric matrix 
. You can write this positive matrix in the form 
where 
is the 
th column of 
and the are independent of each other because the entries of 
are independent.
Since 
for a symmetric matrix 
,
If you had an almost sure upper bound on the norm of the , then you could directly use a Chernoff bound from Tropp’s paper to quickly get an upper bound on this probability, but I don’t have such an upper bound. In particular, I’d like to consider the case where is a Gaussian matrix.
However, perhaps fortunately, his general Laplace transform for random matrices still applies: if you have self-adjoint matrix functions 
which are semidefinite bounds on the cumulant generating functions of 
— that is, 
—, then
To use this result, we must find nonvacuous . Expand the moment generating function of 
:
![\displaystyle
\begin{aligned}
\mathbb{E} e^{-\theta u_i u_i^t} & = I + \mathbb{E} \sum_{k=1}^\infty \frac{(-\theta u_i u_i^t)^k}{k!} \\
& = I + \mathbb{E} \left[ (e^{-\theta \|u_i\|^2} - 1) \frac{u_iu_i^t}{\|u_i\|^2} \right] = \mathbb{E} \left[ e^{-\theta \|u_i\|^2} P_{u_i} + P_{{u_i}^\perp} \right].
\end{aligned}](/cz/latexrender/pictures/2ebba9c898c5b7fa910b867a798a7055.png "\displaystyle
\begin{aligned}
\mathbb{E} e^{-\theta u_i u_i^t} & = I + \mathbb{E} \sum_{k=1}^\infty \frac{(-\theta u_i u_i^t)^k}{k!} \\
& = I + \mathbb{E} \left[ (e^{-\theta \|u_i\|^2} - 1) \frac{u_iu_i^t}{\|u_i\|^2} \right] = \mathbb{E} \left[ e^{-\theta \|u_i\|^2} P_{u_i} + P_{{u_i}^\perp} \right].
\end{aligned}")
This is where I’m currently stuck! If there was no negative sign (bear with me), then I could bound this final quantity with
where 
, 
is the height of 
and , and 
is an orthogonal matrix which doesn’t depend on , and whose first 
columns are .
Then would be the log of this, so my problem would reduce to finding a nice expression or upper bound for the cumulant generating function of 
, which turns out to be 
, where 
is the th column of .
Unfortunately, eliminating the negative sign means I would be finding an upper bound on the norm of 
, which although interesting, isn’t the problem I’m concerned with now.
The reason for this post, other than for me to get the details down somewhere, is to ask about that moment generating function. Say if is a standard Gaussian vector, what’s a good bound on
? Does it suffice to know that 
is a rank projection, or do I need to know more about the entries of ? I don’t care about this case, since it’s trivial to bound 
when is a Gaussian matrix, but it’d be informative to see how the results from the sharp trivial bound compare with the results I can get using this Laplace method.
I fear that getting a good bound on for more general 
is hard, because iirc, dealing with double sums of even gaussians is difficult to do sharply.
Update: A colleague pointed out that when is Gaussian, then that’s just the moment generating function of a 
random variable with degrees of freedom, which apparently has a known expression.
Possibly relevant posts:
- The gist of the Laplace transform method for bounding eigenvalues of random matrices (8/27/2010)
- One direction in Khintchine’s inequality for Rademacher sums (9/26/2008)
- Sharpness of Moment bounds, exponential example (7/29/2010)