ChapterZero

Stochastic Lyapunov functions

Filed under: Mathematics — Alex @ 2:00 am 5/6/2007

I began writing up an exposition of the stochastic version of Lyapunov functions a while ago, when we were assigned a problem relating to them on one of my stochastic controls problem sets. It seemed rather random at the time, but now– maybe a month later– we’ve started talking about stochastic stability, and I see that we were just proving the same theorems that’re in the notes, but ahead of time and out of context. Quite an ego booster, really.

The set up: let $S \subset \mathbb{R}^d$ be a compact ’state space’, and $F: S \times \mathcal{R} \rightarrow S$ be a continuous function which, in conjunction with $\{\xi_n\}$ a sequence of real-valued i.i.d r.v.s, determines the dynamics of the system: $x_{n+1} = F(x_n, \xi_{n+1})$ . Note, at least in passing, that this system is a markov chain. Done noting? Introduce the concept of an equilibrium point x^* as one satisfying $F(x, \xi) = 0$ for any value of $\xi$ . We’d like to talk about various types of stability of this point: x^* is

stable if an orbit which starts close to it remains close with high probability — for any $\epsilon > 0$ and $\alpha \in (0,1)$ there exists a $\delta < \epsilon$ such that we have $\mathbb{P}(\sup_{n \geq 0}\|x_n - x^*\| < \epsilon) > \alpha$ whenever $\|x_0 - x\| < \delta$ .
asymptotically stable if it is highly probable that an orbit which starts close to it converges to it — if it is stable and for every $\alpha \in (0,1)$ there exists a $\kappa$ such that $\mathbb{P}(x_n \rightarrow x^*) > \alpha$ whenever $\|x_0 - x^*\| < \kappa$ .
globally stable if any orbit converges to it a.s. — if it is stable and $x_n \rightarrow x^*$ a.s. for any .

Now, for the cool result:

Suppose that there is a continuous function $V : S \rightarrow [0, \infty)$ with and for $x \neq x^*$ such that $\mathbb{E}(V(F(x, \xi_n))) - V(x) = k(x) \leq 0$ for all $x \in S$ . Then is stable.