Wiener filtering
I’m a little puzzled by this, but I’m writing it up for posterity. Suppose 
is a zero-mean, (wide-sense circular) stationary random vector in 
. Stationary meaning that the entries in the autocovariance matrix depend only on the difference between the indices, i.e. ![\mathbb{E}X_nX_m = r[n-m \mod N]](/cz/latexrender/pictures/1a7fab30abac404781211ad1a49701c1.png "\mathbb{E}X_nX_m = r[n-m \mod N]")
for some vector 
, called the autocorrelation. Then the power spectrum 
is defined to be the DFT of the autocorrelation. If we consider a fixed vector 
, then the (circular) convolution product of 
and is given by ![(g\star X)[n] = \sum_{j=0}^{N-1} X[j]g[n-j \mod N]](/cz/latexrender/pictures/af90e457979dc2fb1de919d19b631a06.png "(g\star X)[n] = \sum_{j=0}^{N-1} X[j]g[n-j \mod N]")
, and we have that 
is again zero-mean and stationary, with ![P_{g\star X}[\omega] = P_X[\omega] |\hat{h}[\omega]|^2](/cz/latexrender/pictures/a458a6c897508d9e468cff0105ef5202.png "P_{g\star X}[\omega] = P_X[\omega] |\hat{h}[\omega]|^2")
.
All of which is just a setup for the main problem. Suppose we observe 
where is our random signal of interest and 
is independent noise, both of which we take WLOG to be zero-mean. We’d like to estimate linearly from 
. Specifically, we look for a vector which minimizes 
. Since independence of and carries through when we filter with , we can expand the expected MSE error into 
.
If we assume and are both stationary processes, we can use Parseval’s theorem to show that 
. In particular, by letting be the dirac vector, we find ![\mathbb{E} \|X\|^2 = \sum_{\omega=0}^{N-1} P_X[\omega]](/cz/latexrender/pictures/79443be75b75c350c743770ccac856a1.png "\mathbb{E} \|X\|^2 = \sum_{\omega=0}^{N-1} P_X[\omega]")
. Plugging in these relations, we find that the MSE error is ![\sum_{\omega=0}^{N-1} |\hat{g}[\omega]|^2(P_X[\omega] + P_W[\omega]) – 2P_X[\omega]\overline{\hat{g}[\omega]} + P_X[\omega]](/cz/latexrender/pictures/b7ec81414874ebbebb41fdfb2e05dd83.png "\sum_{\omega=0}^{N-1} |\hat{g}[\omega]|^2(P_X[\omega] + P_W[\omega]) – 2P_X[\omega]\overline{\hat{g}[\omega]} + P_X[\omega]")
.
The question now is to find which minimizes the MSE error. If we were naive about it, we could ignore the fact that the sum above is one of complex quantities, and differentiate the quadratic form w.r.t. ![g[\omega]](/cz/latexrender/pictures/0af6f55ed6f4bae8cb4908de50308e6a.png "g[\omega]")
for each 
and set the result to zero, getting ![\hat{g}[\omega] = P_X[\omega]/(P_X[\omega] + P_W[\omega])](/cz/latexrender/pictures/aefd1c697bea9a50f3aa7f06d3b8efcf.png "\hat{g}[\omega] = P_X[\omega]/(P_X[\omega] + P_W[\omega])")
. This relation for the Fourier coefficients of 
could then be used to calculate .
My problem with this, which seems to be the usual way of defining the Wiener filter, is two-fold. First, I think there should be a phase factor included in the definition of ![\hat{g}[\omega]](/cz/latexrender/pictures/04de97e6e4e317ae3974b1b4af8dc6d2.png "\hat{g}[\omega]")
to be sure that is real– to be sure, there’s nothing wrong with estimating a real quality with a complex one, but why not make 
real if you can? More seriously, I’m not sure the differentiation-to-minimize argument carries over to the complex case (
, for example, is not differentiable).
Possibly relevant posts:
- But that’s just least squares… (8/9/2006)
- Optimization problems (1/24/2006)
- I’m stumped– you give it a try (4/1/2008)
You forgot to take the real part. The actual formula is ||v+w||^2=||v||^2+||w||^2+2Re(v,w). And it’s clear that if you want to minimize, the imaginary part just contributes to L^2 norm without doing anything.
There’s a mistake when you expand the MSE.
||a-b||^2 = ||a||^2 + ||b||^2 – –
Using the corrected MSE formula, since power spectra are nonnegative, in order to minimize the MSE, g_hat has to be real. Then since the power spectra are symmetric, g_hat is symmetric, so g is real.
Thanks Torus and “TA”. I’ll go back and fix the post at some point in the future.