Markov Chains
I gave a mini-seminar today on stochastic processes, and was surprised at how we got hung up on the very concept that we’re studying stochastic processes for: Markov processes. To be more specific, Markov chains; the question was, does the Markovianity property 
imply that 
is independent of 
for 
? I don’t think so…
The next thing I am working on is showing that stationarity and Markovianity do not contradict each other. This is obvious for one-dimensional finite state Markov chains– take the initial distribution to be the stationary distribution for the chain, but I don’t really understand anything but one-dimensional Markov chains.
Possibly relevant posts:
- Markov Random Fields (6/26/2006)
- Haggstrom vs. Winkler (7/5/2006)
- Dynamic programming, the basic idea (3/19/2007)